≈ the.bay.news

Can a 26B MoE Run on an 8 GB GPU? Gemma 4 26B A4B on an RTX 3050

DEV Community
Can a 26B MoE Run on an 8 GB GPU? Gemma 4 26B A4B on an RTX 3050
Gemma 4 26B A4B, a 14.4 GB mixture-of-experts model, serves from an 8 GB RTX 3050 with most of its experts in system RAM: 131,072 tokens of context, 28 to 34 tok/s empty, and GSM8K and BFCL scores level with the 12B.

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.