llm-d

What it is

Open-source, Kubernetes-native framework for running large language model inference across many machines at high performance.

Topics