Your prefill/decode split has a network bill nobody puts in the diagram
DEV Community
Your prefill/decode split has a network bill nobody puts in the diagram
A colleague's post this week made the case for splitting prefill and decode onto separate GPU pools:...
0 comments
No comments yet.