Running MoE LLMs Larger Than Your RAM: A Practical Guide to mmap, Expert Offloading and SSD Streaming with llama.cpp
DEV Community
Running MoE LLMs Larger Than Your RAM: A Practical Guide to mmap, Expert Offloading and SSD Streaming with llama.cpp
Tuần này trên Hacker News có một project khá thú vị tên Janas: chạy LLM lớn hơn RAM bằng cách stream...
0 comments
No comments yet.