Best practices for handling concurrent async calls to multiple LLM APIs - asyncio.gather vs TaskGroup?
Discussions on Python.org
Best practices for handling concurrent async calls to multiple LLM APIs - asyncio.gather vs TaskGroup?
I’ve been building an agent-style system that needs to call 3-4 different LLM/tool APIs concurrently as part of a single task step, and I’m trying to figure out the cleanest pattern for it in modern Python (3.11+). Right now I’m using asyncio.gather() with return_exceptions=True so one failing call doesn’t kill the whole batch, then checking each result afterward for exceptions. But I’ve seen people recommend asyncio.TaskGroup (3.11+) instead for cleaner exception propagation and cancellation o...
0 comments
No comments yet.