Fifty API keys, one queue, ten GPU workers: the parts of a shared LLM that are not the model

Scoping keys, pulling from one queue, reading backlog trends and splitting an hourly GPU bill per key, plus what a mock run cannot tell you.