The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell
DEV Community
The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell
If you run vLLM with --kv-cache-dtype fp8 on a DeepSeek-family (MLA) model and your GPU is a GB10, an...
0 comments
No comments yet.