The KV Cache Is the Bottleneck Now — 1-Bit Quantization, MoE Stragglers, and Per-Second GPU Billing

This week's LLM serving research attacks the KV cache from every angle — 1-bit quantization, MoE stragglers, and speculative search.