llama.cpp

llama.cpp is the foundational LLM inference engine that powers most of the local AI ecosystem — Ollama, LM Studio, Jan, and many others are built on top of it. Written in C/C++ for maximum performance, it supports both CPU and GPU inference, the GGUF model format for efficient quantization, and includes a built-in HTTP server with an OpenAI-compatible API. Features tool calling support, multi-platform builds, and is the go-to for anyone building local LLM applications.