llama-server's prompt cache reuses KV computed under a different LoRA scale
DEV Community
llama-server's prompt cache reuses KV computed under a different LoRA scale
TL;DR: llama-server's RAM prompt cache (--cache-ram, on by default) saves a slot's KV when another...
0 comments
No comments yet.