Ollama

What Ollama does

Ollama downloads open-weight language models and runs them on your own machine. One command such as ollama run gemma4 fetches a model from the Ollama library and opens a chat in the terminal. In the background, Ollama runs as a local server on port 11434, and that server is what most people end up using: many open source chat apps, editor plugins and coding agents can connect to it as their model backend.

The macOS and Windows installers also include a desktop app with a chat window. It keeps a list of conversations, accepts dragged-in text files, PDFs and code, and sends images to models that support vision. It is basic. If you want projects, document libraries, several providers or multi-user accounts, you would connect a fuller chat app to Ollama instead; Jan, for example, can use it as a custom endpoint.

Models and providers

Local models come from the Ollama library, and you can import other model files and change their parameters with a Modelfile. The README names llama.cpp as the backend Ollama builds on. The API speaks Ollama’s own format and also offers OpenAI-compatible (Chat Completions and Responses) and Anthropic-compatible (Messages) endpoints, so tools written for those APIs can be pointed at a local model.

Ollama does not connect to third-party hosted providers such as OpenAI or Google. Its only hosted option is Ollama Cloud, which runs larger models on Ollama’s servers once you sign in. On macOS, the app can also connect Claude Desktop and ChatGPT Desktop (in Codex mode) to Ollama models.

Privacy and data

When a model runs locally, the FAQ states that Ollama does not see your prompts or data. Models and settings live in ~/.ollama, and the desktop app stores its chats in a local SQLite database. Cloud models are different: Ollama processes those prompts on its servers. It says it does not store, log or train on them, but it does keep account details and usage metadata. Web search in Ollama is also a cloud feature that needs an account. You can switch off all cloud features with OLLAMA_NO_CLOUD=1 or a setting in ~/.ollama/server.json. The documentation does not say whether the app collects usage analytics.

Running Ollama

Ollama is available as installers for macOS and Windows, an install script or tarball for Linux (set up as a systemd service), and the ollama/ollama Docker image. No database or other service is needed. The documented hardware support is specific:

• macOS: Sonoma (14) or newer. Apple M-series chips use the GPU through Metal; Intel Macs run on the CPU only.
• NVIDIA: compute capability 5.0 or higher with driver 550 or newer (570 or newer for compute capability 5.0 to 6.2).
• AMD: ROCm v7 drivers on Linux and Windows, with lists of supported Radeon, Radeon PRO, Ryzen AI and Instinct parts. Vulkan covers other AMD and Intel GPUs.
• Memory: the quickstart model is about 7.2 GB and needs about 8 GB of free VRAM or Mac unified memory. With less, Ollama uses system RAM and runs more slowly.

The server listens only on localhost by default. It has no user accounts or roles, so exposing it on a network means adding your own reverse proxy and authentication.

Hosted version

Ollama Cloud is an optional service with pay-as-you-go usage and paid subscription plans. It provides hosted models and a web search API, and you can use it from the app, the CLI or the API.

Limitations

The chat window is basic, and the documentation covers it only for macOS and Windows; on Linux, Ollama is a command-line tool and server. Web search depends on Ollama’s hosted service. There are no multi-user features. Models are managed in Ollama’s own storage format, so files you download elsewhere need to be imported first.

Who it suits

Ollama suits anyone who wants local models running with as little setup as possible, and especially people who want one local model server that several chat apps, editors and agents can share. Pair it with a fuller chat interface if you need more than a simple conversation window.