Better Analytics

Transition to ClickHouse
We have made a strategic transition from MySQL to ClickHouse, a powerful open-source columnar database management system. This move empowers us to handle large datasets with incredible efficiency, enabling faster data retrieval and analysis. Say goodbye to long wait times and welcome a seamless and responsive analytics experience.

Enhanced Cohere Model Support
In response to user feedback, we have introduced support for tokens and cost calculation across all Cohere models. This addition brings enhanced flexibility and accuracy to your language models, unlocking their full potential. Additionally, we have implemented various smaller improvements to elevate the overall user experience, ensuring a smoother and more intuitive workflow.

Predefined Metadata Keys

Portkey introduces four predefined metadata keys: _environment, _user, _organisation, and _prompt. These keys allow for easy indexing and filtering of data in Portkey's analytics and logs sections. The _user key automatically captures the user key from the OpenAI request body and can also be provided as metadata. Leveraging these predefined keys enhances data organization and filtering capabilities in Portkey.

Semantic Cache

Semantic cache is a caching mechanism in Portkey that goes beyond fixed string matching. It evaluates the contextual similarity between input prompts and cached requests using cosine similarity. When the similarity exceeds a threshold, Portkey retrieves the response from the cache, improving response retrieval efficiency. It can be enabled by setting the x-portkey-cache header to "semantic" in the requests.