Speculative Decoding Grows Up, and Prefix Caches Learn to Collapse
DEV Community
Speculative Decoding Grows Up, and Prefix Caches Learn to Collapse
Scheduling-aware spec decoding, a vLLM control plane, and a prefix-cache collapse post-mortem headline this week's inference digest.
0 comments
No comments yet.