LocalAI

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

Automatically checked · GitHub checked 2026-11-17

LocalAI

Overview

LocalAI: your own AI, on your own hardware. Agents, documents and every kind of model, for you and your team. The LocalAI llama mascot wears sunglasses.

Latest release MIT license Join the LocalAI Discord

Documentation | Quick start | Discord | FAQ | Blog

LocalAI runs open models for chat, images, speech and video on your own machines. The API speaks the OpenAI and Anthropic formats. Your team gets chats, files and documents on top, and the assistant can write and run code. It runs on a laptop, a server, in a container or on Kubernetes, and a distributed mode spreads models over several machines. No GPU needed, though a GPU makes it faster.

  • Chats, agents, Projects and a team: chats and saved agents that use tools, MCP servers, files, Knowledge collections and skills. Projects group chats, files, notes and agents around one piece of work. Users, roles, quotas and shared folders for a team (development builds).
  • Add-ons: an admin installs add-ons from a library: Office, Sandbox, Tables, chat apps, mail, calendar and more (development builds).
  • Every kind of model, one API: text, vision, speech to text, text to speech, images, video, music and sound, embeddings and reranking, in the OpenAI and Anthropic formats (with ElevenLabs-compatible text to speech and sound generation), with more than 80 backends. Each backend is its own image, pulled only when a model needs it.
  • Your hardware: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, NVIDIA Jetson or CPU only. The server, the models and your files stay on your own machines.

LocalAI is a small core with an API, chats and agents, files, collections, team sharing, a model gallery and an MCP client and server for agent tools. Backends such as llama.cpp, vLLM, MLX and whisper.cpp plug in on one side, add-ons such as Office, Sandbox, Tables and chat apps on the other. It runs on CPU, NVIDIA, AMD, Intel, Apple Silicon and Vulkan.

Created by Ettore Di Giacinto and maintained by the LocalAI team.

Quick start

Script (Linux, and macOS on Apple Silicon): it downloads the release binary for your machine and checks it against the release checksums.

curl -sSL https://localai.io/install.sh | sh
local-ai
# open http://localhost:8080 and install a model from there

The script installs the program and does not start it; local-ai starts the server. To run a model directly, name it, for example local-ai run gemma-4-e2b-it-qat-q4_0, a small Google Gemma 4 model for CPUs (4.3 GB to download: 3.35 GB for the model and 0.99 GB for the image projector).

Container (Docker, podman, ...):

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

Desktop app: it installs, updates, starts and stops the server for you and shows the web interface in its own window, on macOS and Linux. It comes with the next release. Test builds are on the dev-master pre-release. See the desktop app guide.

The install script and the latest image give 4.12.1 today. Projects, the new desktop app, the add-ons, Office, Sandbox and team features described below need a development build or a newer release.

Then open http://localhost:8080. To restart an existing container, use docker start -i local-ai. More ways to install: Install LocalAI.

GPU images (NVIDIA, AMD, Intel, Vulkan)
# NVIDIA, CUDA 13
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13

# NVIDIA, CUDA 12
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

# NVIDIA Jetson ARM64 (CUDA 12, for AGX Orin and similar)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64

# NVIDIA Jetson ARM64 (CUDA 13, for DGX Spark)
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64-cuda-13

# AMD (ROCm)
docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas

# Intel (oneAPI)
docker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel

# Vulkan
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan

LocalAI detects your GPU and downloads the matching backend. See GPU Acceleration.

Load a model:

# From the model gallery (list it with `local-ai models list` or on the Models page of the web UI)
local-ai run gemma-4-e2b-it-qat-q4_0
# From Hugging Face
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# From the Ollama OCI registry
local-ai run ollama://gemma:2b
# From a standard OCI registry (for example Docker Hub)
local-ai run oci://localai/phi-2:latest

To talk to a running server from the terminal, start the built-in agent in another shell. It asks you to approve anything that changes state. See the Terminal agent docs.

local-ai chat --model gemma-4-e2b-it-qat-q4_0

More in the Quickstart.

What you can do

AreaWhat it gives you
ChatsConversations the server keeps, with saved agents that use tools, MCP, skills and files, triggers, and chat apps
Add-onsOffice, Sandbox, Tables, chat apps, mail, calendar and more install from a library and run beside the server (development builds)
Text generationllama.cpp, transformers, vllm ... and more, with an OpenAI-compatible tools API and constrained grammars
Animated videoWith the Sandbox add-on and its media profile, the assistant can build short videos with ffmpeg from a sentence, such as a title card with a fade or a slideshow with a slow zoom, saved as MP4 in your Files. The Sandbox has no internet, and results need a look before you use them (development builds)
Music and soundMake music and sound effects from a text prompt with /v1/sound-generation. See text to audio
SpeechText to audio, audio to text, voice activity detection (Silero-VAD) and the Realtime API for speech-to-speech
Images, video and visionImage generation, video generation, the Vision API and object detection
SearchEmbeddings and a reranker
OperationsAPI key auth, user quotas and roles, distributed mode (PostgreSQL + NATS) and P2P inferencing
Models and backendsModels from Hugging Face and a Backend Gallery to install or remove backends on the fly, plus the integrated web UI

LocalAI has more than 80 backends. Hardware acceleration covers NVIDIA (CUDA 12 and 13), AMD (ROCm), Intel (oneAPI/SYCL), Apple Silicon (Metal), Vulkan and NVIDIA Jetson (L4T). See the Backend and Model Compatibility Table.

Backends we build

Most backends wrap an upstream engine such as llama.cpp or vLLM. We wrote 18 of them ourselves: native C, C++ and GGML engines with no Python at inference.

BackendWhat it does
vllm.cppFrom-scratch C++20 port of vLLM for text generation: paged KV cache, continuous batching, prefix caching, safetensors + GGUF loading, engine-enforced structured output, on CPU, CUDA, Metal and Vulkan. Also serves MiniMax-H3 joint video+audio generation
parakeet.cppC++/GGML port of NVIDIA NeMo Parakeet ASR (tdt/ctc/rnnt/hybrid), with cache-aware streaming transcription
moss-transcribe.cppC++/GGML port of OpenMOSS MOSS-Transcribe-Diarize: joint long-form transcription, speaker diarization and timestamping in a single pass
moss-tts.cppC++/GGML port of the OpenMOSS MOSS-TTS family: text-to-speech (MOSS-TTS-Local v1.5, 48 kHz stereo) with reference-audio voice cloning, through the MOSS-Audio-Tokenizer neural codec
magpie-tts.cppC++/GGML port of NVIDIA's Magpie TTS Multilingual 357M: 22.05 kHz mono text-to-speech in 5 voices and 9+ languages, with the NanoCodec neural codec and tokenizer/G2P embedded in a single GGUF
ced.cppC++/GGML port of the CED audio-tagging models: sound-event classification (527-class AudioSet) over REST and the realtime API for live recognition
voice-detect.cppSpeaker recognition and voice analysis (ECAPA-TDNN, WeSpeaker, ERes2Net, CAM++, wav2vec2 age/gender/emotion), replacing the Python speaker-recognition backend
vibevoice.cppNative port of Microsoft VibeVoice for TTS (voice cloning) and long-form ASR with speaker diarization
rf-detr.cppNative RF-DETR object detection and instance segmentation
locate-anything.cppOpen-vocabulary object detection and visual grounding (LocateAnything-3B)
depth-anything.cppDepth Anything 3 monocular metric depth + camera pose estimation
face-detect.cppFace detection, recognition, demographics and anti-spoofing (SCRFD/ArcFace, YuNet/SFace), replacing the Python insightface backend
timestral.cppMistral Timestral time-series forecasting: quantile forecasts with covariates, on the CPU
trellis2.cppC++/GGML port of Microsoft TRELLIS.2: single-image to textured 3D mesh (GLB with PBR materials)
gem-x.cppLive human motion capture with SOMA and SMPL pose streams (CPU/Vulkan)
kimodo.cppC++/GGML text-to-motion on CPU and Vulkan, exported as animated skeleton GLB
privacy-filter.cppStandalone GGML PII/NER token-classification engine powering LocalAI's PII redaction tier
LocalVQEJoint acoustic echo cancellation, noise suppression, and dereverberation

The team also ships a vector store, local-store, in the main tree, and maintains apex-quant, a per-tensor, per-layer quantization recipe for Mixture-of-Experts models that exploits their structural sparsity to produce GGUFs matching or beating Q8_0 quality. They run out of the box on stock llama.cpp.

Latest news

  • October 2026: LocalAI 4.12.1: A security fix for the policy on URLs that users give. Redirect targets are now checked on every hop. Release notes

  • October 2026: LocalAI 4.12.0: A redesigned web UI, streaming voice answers, decision routing with the /v1/decisions API, image upscaling in Studio and Qdrant stores. Release notes

  • October 2026: LocalAI 4.11.0: Audio scenes with transcription, speaker labels, and sound detection. Also the Decisions API (/v1/systemone), model failover chains, signed OCI galleries, and Kimodo text-to-animation. Release notes

  • September 2026: LocalAI 4.10.0: A fleet operations dashboard, one credentials file for private model sources, and local-ai benchmark for measuring model latency and throughput. Release notes

  • August 2026: LocalAI 4.9.0: Deny-by-default authentication, opt-in chat context compression, unified model and backend management, and MiniMax-H3 video generation with audio. Release notes

Older news and the full release notes: GitHub Releases and the blog.

Resources

Team

LocalAI is maintained by a small team of humans, together with the wider community of contributors.

A huge thank you to everyone who contributes code, reviews PRs, files issues, and helps users in Discord. See the full contributors list.

Sponsors

Support the project by becoming a backer or sponsor. A huge thank you to our sponsors, who cover CI expenses, and to the individual sponsors listed on GitHub and buymeacoffee. Special shout out to drikster80.

License

MIT. LocalAI is created by Ettore Di Giacinto and maintained by the LocalAI team. The LocalAI name, logo and mascot art are not covered by the MIT license: all rights reserved. See BRAND-USAGE.md for what you may do with them.

Citation and acknowledgements
@misc{localai,
  author = {Ettore Di Giacinto},
  title = {LocalAI: the open source AI engine},
  year = {2026},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/mudler/LocalAI}},
}

LocalAI couldn't have been built without the help of great software already available from the community: llama.cpp, stanford_alpaca, llama-go, alpaca.cpp, Stable-Diffusion-NCNN, whisper.cpp, piper and exo (MLX distributed auto-parallel sharding).

Contributors