llama.cpp

What it is

Open-source project for running large language model inference in plain C/C++ across CPUs and GPUs, maintained by the ggml organization and widely used to run open-weight models locally.

Topics