From Dev.to
RTX 3090, RTX 5090, A100 and H100 on vLLM: the cheapest GPU flips with your answer time
I measured four GPUs serving the same 7B model on vLLM. Which one is cheapest per token has no single answer. It flips with how fast your answers must be. Same model (Qwen2.5-7B-Instruct), same vLLM (...