โ‰ˆ the.bay.news

LLM agent token cost: why a kept chat grows with the square of the steps, and the measured fix (prompt caching counted)

DEV Community
LLM agent token cost: why a kept chat grows with the square of the steps, and the measured fix (prompt caching counted)
An agent that keeps its whole chat resends everything on every step, so the tokens it sends grow with...

0 comments

Sign in to join the discussion โ€” your thebay.events account works here.

No comments yet.