vLLM v0.28.0: the breaking change small GPU users must read
DEV Community
vLLM v0.28.0: the breaking change small GPU users must read
vLLM v0.28.0 moved bitsandbytes to an out-of-tree plugin and doubled a memory-hungry default. Here is what actually changes if you serve models on one GPU.
0 comments
No comments yet.