โ‰ˆ the.bay.news

Ollama: a keep_alive on the loaded model cancels the swap another request waits for

DEV Community
Ollama: a keep_alive on the loaded model cancels the swap another request waits for
TL;DR: With OLLAMA_MAX_LOADED_MODELS=1, a request for a model that isn't loaded makes Ollama schedule...

0 comments

Sign in to join the discussion โ€” your thebay.events account works here.

No comments yet.