llm-d
Open-source, Kubernetes-native framework for running large language model inference across many machines at high performance.
3 free resources on this topic. Everything here is free and hand-picked. You can also search within this topic.
Open-source, Kubernetes-native framework for running large language model inference across many machines at high performance.
InfoQ's report on AX, Google's newly open-sourced system for running fleets of autonomous AI agents. It explains how AX treats each agent as a long-lived task that can be paused and resumed to save computing resources, using control ideas borrowed from Kubernetes.
Why I recommend it: Free to read, though InfoQ asks you to register for some content. The performance claims come from Google's own announcement, so treat them as the company's figures until others test the system.
The official home of Kubernetes, the free open-source system for running and scaling containerized applications. Includes full documentation, tutorials and a browser-based interactive learning track — useful background if you are moving toward cloud, DevOps or AI infrastructure work.
Why I recommend it: The software and docs are free and open source. Running Kubernetes on a cloud provider costs money, so stick to the free local tutorials while you are learning.