≈ the.bay.news

Hot experts, cold RAM: a big MoE on a box with more memory than VRAM

DEV Community
Hot experts, cold RAM: a big MoE on a box with more memory than VRAM
Strata keeps resident only the experts that are hot, and streams weights through ordinary DDR instead of expensive VRAM. What that buys a small organisation, and what it asks of the box.

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.