You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Depends on #939 and #892, as until those are fixed the utility of this feature is minimal or none.
Anthropic and OpenAI provide prompt caching features, and their APIs return statistical data about the savings using prompt caching.
As a cost-conscious or environmentally conscious user, I would like to know that I am properly using prompt caching during my daily work, and how much efficiency is gained.
I would also like to be able to see if one of the AI assistants or tools that I am using prevents this feature.
I would like to be able to access this conveniently on the CodeGate dashboard.
Additional Context
We may have to normalize this into a common struct in order to make it usable across providers
Description
Depends on #939 and #892, as until those are fixed the utility of this feature is minimal or none.
Anthropic and OpenAI provide prompt caching features, and their APIs return statistical data about the savings using prompt caching.
As a cost-conscious or environmentally conscious user, I would like to know that I am properly using prompt caching during my daily work, and how much efficiency is gained.
I would also like to be able to see if one of the AI assistants or tools that I am using prevents this feature.
I would like to be able to access this conveniently on the CodeGate dashboard.
Additional Context
We may have to normalize this into a common struct in order to make it usable across providers
For a good starting point on prompt caching in general, you can check out this internal document: https://docs.google.com/document/d/1gEgkIyT1CRcTMAwYiU6ldeeosKKbWMX5HM8AdQx3xV0/edit?tab=t.0
The text was updated successfully, but these errors were encountered: