A Bounded Go Worker Pool in Front of an LLM Gateway: Two Postmortems and a Benchmark
DEV Community
A Bounded Go Worker Pool in Front of an LLM Gateway: Two Postmortems and a Benchmark
Two goroutine-leak postmortems, a hardened bounded Go worker pool, and benchmark numbers for the ingest path sitting in front of an LLM gateway.
0 comments
No comments yet.