Ollama API Usage Metrics and Errors: Measuring Tokens per Second and Handling Failures

By Web Asset House Studio · October 29, 2026

Note: Some future links on this site may be affiliate links. We never claim hands-on testing; guides are researched from the public sources listed in each article.

How we wrote this: A summary of Ollama's official API documentation pages on usage metrics, errors and streaming (linked below), checked October 29, 2026. The example numbers come from Ollama's own example response; we don't publish speed figures for any model or machine, since they depend entirely on your hardware.

When you call Ollama's local API, the response carries more than the answer: it includes timing and token counts you can use to measure speed, see whether a model had to load, and check how much of your prompt was cached. And when something goes wrong, Ollama returns errors in a predictable format. This guide covers both. If you're new to the API, start with our Ollama local API basics.

The usage fields

Ollama's usage page lists these metrics:

The detail that trips people up: all timing values are in nanoseconds. Divide by 1,000,000,000 to get seconds.

Ollama's example response

From the usage page, a non-streaming call to /api/generate:

{
  "model": "gemma4",
  "response": "Hello! How can I help you today?",
  "done": true,
  "done_reason": "stop",
  "total_duration": 174560334,
  "load_duration": 101397084,
  "prompt_eval_count": 11,
  "prompt_eval_cached_count": 8,
  "prompt_eval_duration": 13074791,
  "eval_count": 18,
  "eval_duration": 52479709
}

Turning the numbers into useful figures

This is simple arithmetic on the documented fields, using the example above:

• Generation speed (tokens per second) = eval_count ÷ (eval_duration ÷ 10⁹). Here: 18 ÷ 0.0525 s ≈ 343 tokens/s.
• Prompt processing speed = uncached prompt tokens ÷ (prompt_eval_duration ÷ 10⁹). Here, 11 prompt tokens with 8 from cache leaves 3 uncached, in about 0.013 s.
• Load time: load_duration ≈ 0.10 s, so more than half of this short response's 0.17 s total was loading the model.
• Cache hit share: prompt_eval_cached_count ÷ prompt_eval_count = 8 of 11 prompt tokens.

A few practical readings:

• A large load_duration on the first request and a near-zero one afterward usually means the model was loaded into memory on the first call. Ollama keeps models loaded for a while after a request; the API's keep_alive setting controls how long (see our API basics guide).
• To compare models or settings fairly, compare eval_count / eval_duration rather than total_duration, which also includes loading and prompt time.
• If generation seems slow, check ollama ps to see whether the model is split between CPU and GPU; our GPU support guide explains what to look for.

Here's a short Python example that reads the documented fields from a non-streaming /api/generate response (it uses the standard requests package and assumes Ollama is running locally with the model pulled):

import requests

r = requests.post("http://localhost:11434/api/generate",
                  json={"model": "gemma4", "prompt": "Say hello.", "stream": False})
d = r.json()
tps = d["eval_count"] / (d["eval_duration"] / 1e9)
print(f"{d['eval_count']} tokens at {tps:.1f} tokens/s; load {d['load_duration']/1e9:.2f}s")

Metrics in streaming responses

Ollama's docs say that for streaming endpoints, the usage fields come in the final chunk, the one where done is true. So read the last object in the stream to get them. Our streaming responses guide covers how streaming chunks are structured.

HTTP status codes

Ollama's errors page lists the common status codes:

A 404 on a local call most often means the model name is wrong or the model hasn't been pulled; ollama list shows what's installed.

The error format

Errors come back as JSON with the message in an error property:

{ "error": "the model failed to generate a response" }

Errors in the middle of a stream

This is the case to code for. Ollama says if an error happens mid-stream, it arrives as an object with an error property in the application/x-ndjson stream, and because the response already started, the HTTP status code isn't changed. In other words, a stream can start with 200 OK, deliver some tokens, and then end with:

{"error":"an error was encountered while running the model"}

So when you process a stream, check every line for an error key rather than relying on the status code alone. If errors keep happening, the server logs usually explain why; see our Ollama troubleshooting guide.

Sources

• Ollama API: Usage
• Ollama API: Errors
• Ollama API: Streaming
• Ollama: Context length