Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
DEV Community
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
A field report on serving Gemma 4 E2B under vLLM on AWS G5g — the only aarch64 + SM 7.5 hardware there is. No published build covers that combination, AWS quietly solves half of it, and the thing that actually blocks you is 64 KiB of shared memory.
0 comments
No comments yet.