Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin
vllm.ai
Serving LLMs on Tenstorrent Hardware: Inside the vLLM TT Plugin
Tenstorrent accelerators join vLLM as an out-of-tree platform plugin, driven by mesh-architecture choices: phase-based scheduling, single-process data paralleli
0 comments
No comments yet.