โ‰ˆ the.bay.news

Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU alone

Tom
Homelabber uses second-gen Optane memory sticks to run a 2.8-trillion-parameter AI model on CPU alone
A Reddit homelab user ran Moonshot AI’s 2.8-trillion-parameter Kimi K3 CPU-only on Optane PMem 200 at 0.85 tokens/s with a 1-bit quant.

0 comments

Sign in to join the discussion โ€” your thebay.events account works here.

No comments yet.