Caching LLM Responses on Your Side: An Exact-Match Cache in Front of Any OpenAI-Compatible API (Python)

I'm Aman Kumar, and I run an OpenAI-compatible gateway, so I see a lot of traffic patterns. One thing...