What a 70% vs 90% prompt cache hit rate does to an AI agent’s bill (72 models priced)
DEV Community
What a 70% vs 90% prompt cache hit rate does to an AI agent’s bill (72 models priced)
An agent sends the same long prefix on every step: system prompt, tool definitions, the files it has...
0 comments
No comments yet.