โ‰ˆ the.bay.news

Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned

DEV Community
Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned
Strata on a rented RTX 5090: 144 tok/s for one developer, 19 each for eight, and the two flags that fixed a 5x slowdown on cloud containers.

0 comments

Sign in to join the discussion โ€” your thebay.events account works here.

No comments yet.