≈ the.bay.news

A point release halved your KV cache memory. You did not choose that trade.

DEV Community
A point release halved your KV cache memory. You did not choose that trade.
Two vLLM point releases changed serving defaults in a minor version bump - FP8 KV cache by default, and max_tokens no longer reserved from the context window. A default you did not choose is still a decision you own.

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.