≈ the.bay.news

I Cut Dedicated GPU Memory to 4GB, and a Local LLM Still Ran at 90% Speed

DEV Community
I Cut Dedicated GPU Memory to 4GB, and a Local LLM Still Ran at 90% Speed
On a Surface Laptop Ultra (RTX Spark, unified memory), I measured local LLM speed and Windows' free memory with the GPU's dedicated memory set to 4, 64, 82 and 96GB. Speed barely moved; memory left for Windows did.

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.