5 free resources on this topic. Everything here is free and hand-picked. You can also search within this topic.
ExplainX's guide covers local AI hardware, inference engines, open-weight models and workflow automation. The publisher sells AI services; model specifications, licenses and hardware recommendations should be checked against current official documentation.
Open-source project for running large language model inference in plain C/C++ across CPUs and GPUs, maintained by the ggml organization and widely used to run open-weight models locally.
Open-source, Kubernetes-native framework for running large language model inference across many machines at high performance.
Microsoft Research blog post on running the heavy AI thinking for robots on remote computers instead of on the robot itself, and what that means for speed, cost and reliability in real-world robotics.
Why I recommend it: Free to read. Written by Microsoft researchers, so it presents their own approach favorably — Microsoft also sells the cloud computing this kind of setup relies on.
On-device AI with cloud fallback for smartphones, laptops, and edge devices, designed to cut inference costs by knowing when to hand off to frontier cloud models.
Why I recommend it: I am watching on-device AI closely because it could make powerful tools accessible at lower cost and with more privacy. Cactus is a useful example of the "know when to hand off" design pattern.
