Skip to content
TanStackAI
Adapters

Ollama

The Ollama adapter calls Qwen, Llama, gpt-oss, and other open models on your machine. It needs no API key and it works offline.

1. Install

shell
npm i @tanstack/ai-ollama

2. Start Ollama and pull a model

Ollama needs no API key. It needs a running server with a model.

  1. Install Ollama:
shell
# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows: download the installer from https://ollama.com
  1. Pull a model:
shell
ollama pull qwen3:8b
  1. Start the server:
shell
ollama serve

The server runs on http://localhost:11434 by default. If your server is somewhere else, set OLLAMA_HOST:

shell
OLLAMA_HOST=http://localhost:11434

3. Stream a chat from the server

ts
import {
  chat,
  chatParamsFromRequest,
  toServerSentEventsResponse,
} from "@tanstack/ai";
import { ollamaText } from "@tanstack/ai-ollama";

export async function POST(request: Request) {
  const { messages, threadId, runId } = await chatParamsFromRequest(request);

  const stream = chat({
    adapter: ollamaText("qwen3:8b"),
    messages,
    threadId,
    runId,
  });

  return toServerSentEventsResponse(stream);
}

ollamaText reads the host from OLLAMA_HOST. To pass the host in code, use createOllamaChat("qwen3:8b", "http://your-server:11434").

4. Render the chat

The client is the same for every provider. Point useChat at your route:

tsx
import { useState } from "react";
import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";

export function Chat() {
  const [input, setInput] = useState("");
  const { messages, sendMessage } = useChat({
    connection: fetchServerSentEvents("/api/chat"),
    threadId: "support-chat",
  });

  return (
    <form
      onSubmit={(event) => {
        event.preventDefault();
        sendMessage(input);
        setInput("");
      }}
    >
      {messages.map((message) => (
        <p key={message.id}>
          {message.parts
            .map((part) => (part.type === "text" ? part.content : ""))
            .join("")}
        </p>
      ))}
      <input value={input} onChange={(event) => setInput(event.target.value)} />
    </form>
  );
}

For Vue, Solid, Svelte, Angular, and the other frameworks, see Quick Start.

You now have a streaming chat with a local model. The speed depends on your hardware. A GPU helps for larger models.

Tools

Tools work the same as with every adapter. The tool support depends on the model. Models like llama3, mistral, and qwen2 call tools well.

Tool choice

Ollama ignores toolChoice. Ollama has no tool choice, so the model decides to call a tool or not, also with 'none' or 'required'.

To get no tool calls for a request, do not pass tools. For the other providers, see Choose when the model calls a tool.

Embeddings

ts
import { embed } from "@tanstack/ai";
import { ollamaEmbedding } from "@tanstack/ai-ollama";

const result = await embed({
  adapter: ollamaEmbedding("nomic-embed-text"),
  input: ["a red guitar", "a blue drum kit"],
});

console.log(result.embeddings[0]?.vector);
  • Pull the model first: ollama pull nomic-embed-text.
  • The known models get autocomplete: nomic-embed-text, mxbai-embed-large, all-minilm, snowflake-arctic-embed, bge-m3, and embeddinggemma. Any other model name works too.
  • Each model has a fixed output size, so the top-level dimensions option does not work.

See the Embeddings guide.

Summarization

ts
import { summarize } from "@tanstack/ai";
import { ollamaSummarize } from "@tanstack/ai-ollama";

const result = await summarize({
  adapter: ollamaSummarize("qwen3:8b"),
  text: "Your long text to summarize...",
  threadId: "thread-1",
  maxLength: 100,
  style: "concise", // "concise" | "bullet-points" | "paragraph"
});

console.log(result.summary);

No image generation

Ollama does not make images. For images, use OpenAI or Gemini.

Models and options

Model options

Ollama nests its sampling and runner options in an options object inside modelOptions. This includes temperature, top_p, and num_predict (the token limit):

ts
import { chat } from "@tanstack/ai";
import { ollamaText } from "@tanstack/ai-ollama";

const stream = chat({
  adapter: ollamaText("qwen3:8b"),
  messages: [{ role: "user", content: "Hello!" }],
  threadId: "thread-1",
  modelOptions: {
    options: {
      temperature: 0.7,
      top_p: 0.9,
      top_k: 40,
      num_predict: 1000, // Max tokens to generate
      repeat_penalty: 1.1,
      num_ctx: 4096, // Context window size
      num_gpu: -1, // GPU layers (-1 = auto)
    },
  },
});

Sampling options do not go at the root of chat(). For Ollama, they go to modelOptions.options.temperature, modelOptions.options.top_p, and modelOptions.options.num_predict. See Moving Sampling Options into modelOptions. The full option list is in All runner options.

Reasoning

Running a thinking model like Qwen 3 or gpt-oss? Set reasoning on chat():

ts
import { chat } from "@tanstack/ai";
import { ollamaText } from "@tanstack/ai-ollama";

const stream = chat({
  adapter: ollamaText("qwen3:8b"),
  messages: [{ role: "user", content: "Plan a database migration." }],
  threadId: "thread-1",
  reasoning: "off",
});

The adapter sends it as think:

ModelLevelsSent as
qwen3, deepseek-r1, deepseek-v3.1off, highthink: false or true
gpt-osslow, medium, highthink: "low" and so on
a model name that this package does not listoff, highthink: false or true

The other listed models take no reasoning.

Models

To see the models on your Ollama server, run:

shell
ollama list

Any model name works. The package knows many families and gives them autocomplete. Some of them:

  • qwen3, qwen2.5, qwq: Qwen models from Alibaba.
  • llama4, llama3.3, llama3.2, llama3.1, llama3: Llama models from Meta.
  • gpt-oss: open-weight models from OpenAI.
  • gemma3, deepseek-r1, deepseek-v3.1, phi4: models from Google, DeepSeek, and Microsoft.
  • mistral, mistral-small, mixtral, codellama: Mistral and code models.

The package exports the full list as OllamaTextModels.

Advanced

Custom host and client config

ts
import { createOllamaChat } from "@tanstack/ai-ollama";

// Custom host (URL string)
const adapter = createOllamaChat("qwen3:8b", "http://your-server:11434");

// Custom client config (for example, custom headers)
const adapter2 = createOllamaChat("qwen3:8b", {
  host: "http://your-server:11434",
  headers: { Authorization: "Bearer ..." },
});

Running on a remote server

To make Ollama listen on a network interface, start it with OLLAMA_HOST:

shell
OLLAMA_HOST=0.0.0.0:11434 ollama serve

Then point the adapter at that server:

ts
import { createOllamaChat } from "@tanstack/ai-ollama";

const adapter = createOllamaChat("qwen3:8b", "http://your-server:11434");

Behind a proxy

Send every request through a gateway, such as Cloudflare AI Gateway or a corporate proxy, with baseURL and defaultHeaders. These two option names are the same on every TanStack AI adapter, so one gateway config works for all of them.

ts
import { createOllamaChat } from "@tanstack/ai-ollama";

const adapter = createOllamaChat("qwen3:8b", {
  baseURL: "https://gateway.example.com/ollama",
  defaultHeaders: { Authorization: `Bearer ${process.env.GATEWAY_TOKEN}` },
});
  • host and headers are aliases of the same two options. If you set both forms, baseURL and defaultHeaders win.
  • Need more than a fixed header, such as a log of each model call? This adapter supports wrapFetch. An adapter with your own Ollama client ignores wrapFetch and the abort signal.

All runner options

All sampling and runner options go in modelOptions.options:

ts
modelOptions: {
  options: {
    // Sampling
    temperature: 0.7,
    top_p: 0.9,
    top_k: 40,
    min_p: 0.05,
    typical_p: 1.0,

    // Generation
    num_predict: 1000,
    repeat_penalty: 1.1,
    repeat_last_n: 64,
    penalize_newline: false,

    // Performance
    num_ctx: 4096,
    num_batch: 512,
    num_gpu: -1,
    num_thread: 0, // 0 = auto

    // Memory
    use_mmap: true,
    use_mlock: false,

    // Mirostat sampling
    mirostat: 0, // 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0
    mirostat_tau: 5.0,
    mirostat_eta: 0.1,
  },
}

API reference

FactoryCreates
ollamaText(model)Chat adapter. Reads the host from OLLAMA_HOST (default http://localhost:11434)
createOllamaChat(model, hostOrConfig?)Chat adapter with an explicit host or client config
ollamaSummarize(model) / createOllamaSummarize(model, host?)Summarization adapter. host is a URL string
ollamaEmbedding(model) / createOllamaEmbedding(model, hostOrConfig?)Embedding adapter
  • model: a model name, for example "qwen3:8b" or "mistral:7b".
  • hostOrConfig: a URL string in the OLLAMA_HOST style, or an OllamaClientConfig object (for example { baseURL, defaultHeaders }). host and headers are aliases.
Edit on GitHub
Google Vertex AI
Grok (xAI)