Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned
DEV Community
Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned
Strata on a rented RTX 5090: 144 tok/s for one developer, 19 each for eight, and the two flags that fixed a 5x slowdown on cloud containers.
0 comments
No comments yet.