RTX 3090, RTX 5090, A100 and H100 on vLLM: the cheapest GPU flips with your answer time

I measured four GPUs serving the same 7B model on vLLM. Which one is cheapest per token has no single...