Web Asset House Guides
Local AI

Ollama Vision Models: How to Ask a Local Model About an Image

By Web Asset House Studio ·

Note: Some future links on this site may be affiliate links. We never claim hands-on testing; guides are researched from the public sources listed in each article.

How we wrote this: A summary of Ollama's official Vision and CLI documentation (linked below), checked October 29, 2026. Model names in the examples are the ones Ollama's docs use; the list of vision models changes often, so check Ollama's model library for current options.

A vision model reads an image along with your text prompt. Ollama's Vision documentation describes it simply: vision models "accept images alongside text so the model can describe, classify, and answer questions about what it sees." Because Ollama runs on your own computer, the image doesn't have to leave your machine when you use a local model.

Typical uses:

Step 1: Use a model that supports images

Not every model can see. Only models built for vision accept the images field. Ollama's model library has a vision filter that lists them. The docs' examples use gemma4:

ollama pull gemma4

If you're new to Ollama, start with our beginner walkthrough and come back here.

Step 2: Ask about an image from the command line

The quickest test, from Ollama's Vision page, puts the image path right in the ollama run command:

ollama run gemma4 ./image.png what is in this image?

The CLI reference shows the same idea inside a quoted prompt, with the image path at the end of the question. Either way, Ollama reads the file from disk and sends it to the model with your text.

Step 3: Send an image through the API

If you're building something, use the chat endpoint. The docs spell out one detail that trips people up: the REST API expects base64-encoded image data, while the official Python and JavaScript libraries accept file paths, URLs or raw bytes.

The cURL example from the docs, shortened:

IMG=$(base64 < test.jpg | tr -d '\n')
curl -X POST http://localhost:11434/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemma4",
    "messages": [{
      "role": "user",
      "content": "What is in this image?",
      "images": ["'"$IMG"'"]
    }],
    "stream": false
  }'

The images field is a list attached to a user message. Text goes in content as usual. Our guide to Ollama's local API covers the endpoints, streaming and the default port (11434).

In Python, with the official library, you can pass the path directly:

from ollama import chat

response = chat(
  model='gemma4',
  messages=[{
    'role': 'user',
    'content': 'What is in this image? Be concise.',
    'images': ['/path/to/image.jpg'],
  }],
)
print(response.message.content)

Getting answers you can use

Privacy notes

Ollama's FAQ says that when you run locally, Ollama doesn't see your prompts or data. That's the main reason to use a local vision model for receipts, ID-like documents or client files. Two cautions:

Sources

← All Local AI guides · All guides