Open source AI chat apps and local LLM tools

Four kinds of tool

• Self-hosted chat assistants are web apps that look and work like ChatGPT, and you connect them to whichever models you choose. LibreChat is the most complete for teams, with SSO, roles, agents, MCP and a code interpreter. AnythingLLM is built around chatting with your documents. Hugging Face Chat UI is the code behind HuggingChat. NextChat and Big-AGI are lighter clients that keep chats in the browser, and Khoj indexes your notes and files.
• Desktop and mobile apps run on your own device. Jan and TextGen (formerly text-generation-webui) run local models without a separate server, Chatbox is a polished client for many providers, and PocketPal AI runs small models offline on iOS and Android.
• Local model runners download open-weight models and serve them. Ollama, llama.cpp and LocalAI provide an API that the chat apps above connect to, and each also has a basic chat window of its own.
• Answer engines search the web and reply with cited sources, like Perplexity: Vane (formerly Perplexica) and Morphic.

A common setup combines two: Ollama on a GPU machine with LibreChat or Chat UI in front for the team.

The app is open source; the model may not be

An open source license covers the interface, not the model behind it. Connect LibreChat to the OpenAI, Anthropic or Google APIs and your prompts and files still go to that provider under its terms; you gain control of the history, accounts and choice of provider. Prompts stay on your hardware only when the model runs locally.

“Open” models also come under very different terms. Qwen3 and OpenAI’s gpt-oss are released under Apache-2.0. Meta’s Llama 4 license requires “Built with Llama” attribution and an acceptable use policy, and companies with more than 700 million monthly active users must ask Meta for a license. Google’s Gemma terms include a prohibited use policy that must be passed on to anyone you redistribute the model to. The Open Source Initiative’s Open Source AI Definition 1.0 also requires the training code and detailed information about the training data, which few models publish, so most are better described as open-weight. Read the model’s license as well as the app’s.

Running models locally

Most local tools build on llama.cpp and its GGUF model format: Ollama, Jan, LocalAI, TextGen, AnythingLLM’s built-in engine and PocketPal AI all use it. llama.cpp has the widest documented hardware support, including CPU-only and mixed CPU/GPU inference, CUDA, ROCm, Metal, Vulkan and SYCL. Model size is limited mainly by GPU or unified memory: OpenAI says gpt-oss-20b runs within 16 GB of memory and gpt-oss-120b on a single 80 GB GPU. Local models are generally less capable than the largest hosted ones, so test them on your own tasks.

Ollama, llama.cpp’s server and LocalAI offer OpenAI-compatible APIs, which is how nearly every chat app here connects to local models, so you can switch runners without changing the interface.

Privacy defaults differ

Telemetry defaults vary widely:

• On by default:
  • AnythingLLM (PostHog; set DISABLE_TELEMETRY)
  • Chatbox (Google Analytics, Plausible and Sentry; off in Settings > Advanced)
  • Khoj (set KHOJ_TELEMETRY_DISABLE)
  • Big-AGI’s official Docker images (build from source to avoid it)
  • NextChat’s web build, which loads Google Analytics with a measurement ID in the source unless you set your own GA_ID
• Opt-in only: Jan asks at first launch. Its built-in web search sends queries to Exa unless you change it.
• Off, or none found: Morphic documents every outbound connection and turns analytics off by default. TextGen states it runs fully offline, and we found no telemetry in LibreChat, Chat UI, llama.cpp or Vane.

Access control matters as much: anyone who can reach Vane, a default Morphic install or LocalAI without authentication can use your models and API keys, so put those behind a login or a VPN.

What to check before you choose

1. Users. One person on one machine, or a team that needs accounts, SSO (OIDC, SAML, LDAP), roles and spending limits.
2. Models. Local only, hosted APIs only, or both, and whether the app has its own engine or needs Ollama or another runner.
3. Hardware. The GPU or unified memory available for the models you want, and whether your GPU backend is supported.
4. Tools. Document chat (RAG), web search, MCP servers, code execution, image generation and voice differ widely between apps.
5. Telemetry and storage. What the app reports by default, and where chats and uploaded files are stored.
6. Editions. Whether the download you install is the open source code: AnythingLLM’s desktop app comes under separate proprietary terms, and Chatbox’s official builds are made by the company from code synced with a private repository.

Moving from ChatGPT, Claude or Gemini

ChatGPT and Claude both let you export your chat history from their settings, and LibreChat can import ChatGPT exports. Custom instructions, projects and assistants usually have to be recreated by hand. Try the apps with the models you plan to use before moving a team, since answer quality depends on the model more than on the interface.

Projects not listed here

We list only projects whose main repository is on GitHub under an OSI-approved license with no extra conditions. Several well-known projects fail that test:

• Open WebUI is under the “Open WebUI License”, which forbids removing its branding in deployments with more than 50 users in a 30-day period unless you have permission or an enterprise license.
• LobeChat uses the LobeHub Community License, which requires a commercial license to distribute derivative works.
• Cherry Studio labels its LICENSE file AGPL-3.0, but a license file it ships with the app requires organizations of more than 10 people to buy a commercial license.
• Dify adds conditions to Apache-2.0 on multi-tenant use and on its logo.
• GPT4All has had no commits since May 2025.

LM Studio is free to download but proprietary; see the LM Studio alternatives.