vLLM Temperature 0 Isn't Deterministic: 23 of 200 Answers Changed
DEV Community
vLLM Temperature 0 Isn't Deterministic: 23 of 200 Answers Changed
Temperature 0 isn't deterministic on vLLM. Continuous batching changes kernel reduction order, near-tied logits flip, and answers diverge.
0 comments
No comments yet.