โ‰ˆ the.bay.news

Edge0 streams MoE experts off SSD to fit 35B in 3 GB

DEV Community
Edge0 streams MoE experts off SSD to fit 35B in 3 GB
Where does a 35B model go when it only takes 2.9 GB of RAM? I went into Edge0 to find out, and I...

0 comments

Sign in to join the discussion โ€” your thebay.events account works here.

No comments yet.