Same RTX 3090, 22x cheaper per token: what concurrency does to vLLM cost
DEV Community
Same RTX 3090, 22x cheaper per token: what concurrency does to vLLM cost
I rented one RTX 3090 at $0.259/hour, served Qwen2.5-7B-Instruct with vLLM 0.27.1 and default flags,...
0 comments
No comments yet.