AnythingLLM

What AnythingLLM does

AnythingLLM, from Mintplex Labs, is built around workspaces: each one holds a set of documents and its own chats, so a question about contracts does not pull in material from a product manual. Uploaded PDFs, Word files, text and other formats are split, embedded and stored in a vector database, and answers come with citations to the source passages. Unlike most chat UIs, this works without extra services, because the default embedder runs locally and the default vector database, LanceDB, is built in.

Agents extend the chat with web search and scraping, SQL queries, file system access, chart and document generation, and connectors for Gmail, Google Calendar and Outlook. No-code flows, custom skills, MCP tools, cron-scheduled jobs, a model router, memories, image generation and voice round it out.

It comes in two forms. The desktop app for macOS, Windows and Linux is for one person. The Docker server adds multi-user mode, roles, white-labeling and an embeddable chat widget for websites.

Models and providers

AnythingLLM includes a local engine: Lemonade, AMD’s local AI server, running llama.cpp with GGUF models from Hugging Face. It is the default on the desktop app, where it uses NVIDIA, AMD and Intel GPUs or Apple Silicon. On Docker it is optional (ENABLE_BUILT_IN_LLM=true) and uses CUDA for NVIDIA or Vulkan for AMD and Intel GPUs on Linux; Docker Desktop on macOS runs models on the CPU only. It can also use Ollama, LM Studio, LocalAI, text-generation-webui and other local servers, dozens of hosted providers including OpenAI, Anthropic, Google Gemini and AWS Bedrock, and any OpenAI-compatible API.

Choosing a hosted model, embedder or vector database sends your prompts and document text to that provider. The fully local setup is what makes it private.

Privacy and data

Everything is stored in one folder: a SQLite database, the LanceDB vectors, documents and downloaded models. Mintplex Labs states it has no access to self-hosted data and that AnythingLLM can run air-gapped with local providers.

Anonymous telemetry is on by default. According to the README it records the install type, the LLM provider, model tag and vector database in use, and the fact that a chat was sent or a document added, without content or IP addresses; the desktop privacy policy names PostHog as the service. Turn it off with DISABLE_TELEMETRY=“true” or under Privacy in the app. Even with telemetry off, some model files are fetched from a Mintplex CDN mirror.

Running AnythingLLM

The Docker image (amd64 and arm64) runs as a single container with a mounted storage folder; the docs suggest at least 2 GB of RAM and 10 GB of disk on cloud servers. There are also bare-metal instructions and templates for AWS, Google Cloud, DigitalOcean, Render, Railway and others. Multi-user mode offers admin, manager and default roles and cannot be switched off once enabled.

For local models on the desktop app, the docs recommend 16 GB of RAM and an 8-core CPU, a GPU with 8 GB of VRAM or more on Windows, or any M-series Mac. The desktop app has no user accounts, and the docs warn against exposing it to the internet.

Licensing differs by edition: the repository is MIT, but the desktop installer comes under separate Desktop Terms of Use that allow personal and internal business use and prohibit modification. For open source terms throughout, use the Docker image or build from source.

Hosted version

AnythingLLM Cloud provides private, isolated instances on AWS managed by Mintplex Labs. Hosted instances have no built-in local model, and custom agents and MCP are not available on them.

Limitations

The workspace model is more structured than a plain chat window. Telemetry is opt-out rather than opt-in. Team features are Docker-only, and no OIDC, SAML or LDAP login is documented.

Who it suits

AnythingLLM suits people and teams whose main need is asking questions about their own documents, with the option to keep everything on one machine. For a general team chat with SSO, LibreChat is closer; for a pure local-model desktop app, compare Jan in the AI chat guide.