56.6 GiB holds a 284B model: I audited the 64GB CPU claim
DEV Community
56.6 GiB holds a 284B model: I audited the 64GB CPU claim
The 284B model file is really 56.6 GiB and the binary really runs — but the 44.4 tok/s is a cached turn and the 14.7 is speculative decode.
0 comments
No comments yet.