llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff
DEV Community
llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff
We examine llm-compressor's quantization-to-vLLM workflow, with a reproducible FP8 example, explicit calibration controls, and deployment gates for VRAM, accuracy, and serving cost.
0 comments
No comments yet.