Serving 500 concurrent LLM chats on one 4-core box with tier-aware queueing

TL;DR — When traffic spikes on a shared LLM backend, a naive concurrency limit lets free-tier users...