Why long context is expensive: the KV cache bill

Your chat history can weigh more than the model that reads it. Llama 3.1 8B is a 16 GB model. Give it...