the.bay.news

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

DEV Community
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
A field report on serving Gemma 4 E2B under vLLM on AWS G5g — the only aarch64 + SM 7.5 hardware there is. No published build covers that combination, AWS quietly solves half of it, and the thing that actually blocks you is 64 KiB of shared memory.

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.