Ollama Context Length Explained: Defaults, num_ctx and Memory Use
Note: Some future links on this site may be affiliate links. We never claim hands-on testing; guides are researched from the public sources listed in each article.
How we wrote this: A summary of Ollama's official documentation on context length, the Modelfile and the FAQ (linked below), checked October 28, 2026. Defaults have changed between Ollama versions, so check the docs for the version you run.
If a local model "forgets" the start of a long document, ignores earlier instructions in a chat, or does badly as a coding or agent assistant, the context length is often the reason. In Ollama it is one setting, but it has a real cost in memory.
What context length is
Ollama's context length page defines it as "the maximum number of tokens that the model has access to in memory." Tokens are chunks of text. Your prompt, any documents you paste in, the chat history and the model's reply all count toward it. Once a conversation outgrows the window, the model can no longer see the earliest part.
Ollama's defaults
The current docs say Ollama chooses a default based on your GPU memory (VRAM):
| VRAM | Default context |
|---|---|
| Less than 24 GiB | 4k tokens |
| 24 to 48 GiB | 32k tokens |
| 48 GiB or more | 256k tokens |
The FAQ still describes a flat 4,096-token default in one answer, which matches the smallest tier above. Either way, most laptops and single consumer GPUs get a fairly small window unless you change it.
The same page recommends at least 64,000 tokens for tasks that need a large context, such as web search, agents and coding tools.
The trade-off: memory
Ollama's docs are direct about the cost: "Setting a larger context length will increase the amount of memory required to run a model." The extra memory mostly goes to the KV cache, which holds the model's working state for every token in the window. Our sister site's explainer on context length and the KV cache goes into the math.
Two more FAQ details:
- Parallel requests multiply it. The FAQ gives an example: a 2K context with 4 parallel requests becomes an 8K context and more memory. It says required RAM scales with
OLLAMA_NUM_PARALLELtimesOLLAMA_CONTEXT_LENGTH. - KV cache quantization can shrink it. When Flash Attention is enabled,
OLLAMA_KV_CACHE_TYPEcan be set toq8_0(about half the memory of the defaultf16, with what the FAQ calls a very small loss in precision) orq4_0(about a quarter, with a small-to-medium loss that may be more noticeable at higher context sizes).
How to change it
In the Ollama app: the docs say to move the context length slider in Settings.
For the whole server: set an environment variable when Ollama starts:
OLLAMA_CONTEXT_LENGTH=64000 ollama serve
For one chat in the terminal: inside ollama run, the FAQ shows:
/set parameter num_ctx 4096
Saved with a custom model: the Modelfile reference uses PARAMETER num_ctx to set the window, then ollama create to build the model:
FROM llama3.2
PARAMETER num_ctx 4096
Per API request: pass num_ctx in the request's options. Our Ollama local API guide shows the request format.
Each model also has a maximum context it was built for. Check the model's page in the Ollama library before you raise the number.
Check what you actually got
Run:
ollama ps
The docs show the output's CONTEXT column (the allocated context) and its PROCESSOR column. For best performance, Ollama's guidance is to avoid offloading the model to the CPU: you want to see 100% GPU. If a bigger context pushes part of the model onto the CPU, responses slow down sharply. Lower the context, choose a smaller model or quantization (see what a GGUF file is), or use a machine with more memory.
A sensible starting point
- Short chats and quick questions: the default is usually fine.
- Summarizing long documents or chatting with your files: raise it, and check
ollama ps. See also chatting with your documents locally. - Coding tools and agents: Ollama's docs suggest 64k or more, which generally needs a GPU with plenty of VRAM.