Hot experts, cold RAM: running a big MoE on a machine with more memory than VRAM
DEV Community
Hot experts, cold RAM: running a big MoE on a machine with more memory than VRAM
Two budgets, and only one of them is expensive Every local deployment spends against two...
0 comments
No comments yet.