Ollama: a keep_alive on the loaded model cancels the swap another request waits for
DEV Community
Ollama: a keep_alive on the loaded model cancels the swap another request waits for
TL;DR: With OLLAMA_MAX_LOADED_MODELS=1, a request for a model that isn't loaded makes Ollama schedule...
0 comments
No comments yet.