โ‰ˆ the.bay.news

The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell

DEV Community
The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell
If you run vLLM with --kv-cache-dtype fp8 on a DeepSeek-family (MLA) model and your GPU is a GB10, an...

0 comments

Sign in to join the discussion โ€” your thebay.events account works here.

No comments yet.