โ‰ˆ the.bay.news

vLLM Temperature 0 Isn't Deterministic: 23 of 200 Answers Changed

DEV Community
vLLM Temperature 0 Isn't Deterministic: 23 of 200 Answers Changed
Temperature 0 isn't deterministic on vLLM. Continuous batching changes kernel reduction order, near-tied logits flip, and answers diverge.

0 comments

Sign in to join the discussion โ€” your thebay.events account works here.

No comments yet.