≈ the.bay.news

56.6 GiB holds a 284B model: I audited the 64GB CPU claim

DEV Community
56.6 GiB holds a 284B model: I audited the 64GB CPU claim
The 284B model file is really 56.6 GiB and the binary really runs — but the 44.4 tok/s is a cached turn and the 14.7 is speculative decode.

0 comments

Sign in to join the discussion — your thebay.events account works here.

No comments yet.