Models, settings and speed

Call the Ollama API by hand, stream an answer line by line, change temperature, context length and thinking, compute tokens per second, and free the memory.

By the end of this page you have called every Ollama endpoint the app uses, you know what each setting does, and you can compute tokens per second yourself. Ollama must be running, as on the previous page.

The app does nothing more than this. Everything on this page is what the Rust code sends and reads later, so the app code becomes easy to follow.

The outputs below are real, from one run with qwen3:8b on Ollama 0.40.14. Timestamps, durations, and the model's words will differ on your machine.

Find the server

Everything on this page talks to the Ollama server directly, at http://127.0.0.1:11434. See How Ollama works if that address is new to you.

Check that it runs:

curl http://127.0.0.1:11434

Ollama is running

You can also open that address in your browser. If you get an error instead, start the Ollama app again (macOS and Windows) or run sudo systemctl start ollama (Linux).

List the installed models

curl http://127.0.0.1:11434/api/tags

{"models":[{"name":"qwen3:8b","model":"qwen3:8b","size":5225388164,"details":{"format":"gguf","family":"qwen3","parameter_size":"8.2B","quantization_level":"Q4_K_M","context_length":40960},"capabilities":["completion","tools","thinking"]}]}

The output is trimmed to one model and the fields the app reads. The model picker shows name, size, and parameter_size. context_length is the largest context window the model supports, and the app never asks for more.

Ask which thinking levels a model accepts

curl -s http://127.0.0.1:11434/api/show -d '{"model":"qwen3:8b"}' | grep -o '"thinking":{[^}]*}'
# "thinking":{"values":[false,true],"default":true}

/api/show returns a lot, including the license and the prompt template. The app reads one field, thinking. It lists the values the model accepts for the think parameter, and the default. Models differ here, which is why the app asks instead of guessing.

Model	`values`	`default`
`qwen3:8b`	`[false, true]`	`true`
`gpt-oss:20b`	`["low", "medium", "high"]`	`"medium"`
`qwen3.8:27b-q4_K_M`	`[false, "low", "medium", "xhigh"]`	`"medium"`
`qwen3:30b`	`[true]`	`true`

qwen3:30b always thinks, so there is nothing to choose. gpt-oss:20b cannot switch thinking off, only turn it down.

Stream an answer

curl http://127.0.0.1:11434/api/chat -d '{"model":"qwen3:8b","messages":[{"role":"user","content":"Say hi in three words."}],"think":false}'

On Windows the request body goes in through stdin. PowerShell passes quotes inside an argument to native programs in different ways depending on its version, and stdin avoids that problem.

{"model":"qwen3:8b","created_at":"2026-11-08T15:28:08.635847Z","message":{"role":"assistant","content":"Hello"},"done":false}
{"model":"qwen3:8b","created_at":"2026-11-08T15:28:08.657791Z","message":{"role":"assistant","content":","},"done":false}
{"model":"qwen3:8b","created_at":"2026-11-08T15:28:08.678615Z","message":{"role":"assistant","content":" how"},"done":false}
{"model":"qwen3:8b","created_at":"2026-11-08T15:28:08.700191Z","message":{"role":"assistant","content":" are"},"done":false}
{"model":"qwen3:8b","created_at":"2026-11-08T15:28:08.721901Z","message":{"role":"assistant","content":" you"},"done":false}
{"model":"qwen3:8b","created_at":"2026-11-08T15:28:08.744089Z","message":{"role":"assistant","content":"?"},"done":false}
{"model":"qwen3:8b","created_at":"2026-11-08T15:28:08.767313Z","message":{"role":"assistant","content":""},"done":true,"done_reason":"stop","total_duration":3022278958,"load_duration":2797059916,"prompt_eval_count":22,"prompt_eval_cached_count":0,"prompt_eval_duration":90917000,"eval_count":7,"eval_duration":132213000}

Streaming is on by default. Each line is one complete JSON object with the next piece of the answer in message.content. That makes the parser simple. Read a line, parse it, send the piece to the window, and repeat until done is true.

The model did not quite follow the three-word instruction. Small models do that.

See the thinking

Run the same request with "think":true. The first lines now carry the reasoning in a separate field.

{"model":"qwen3:8b","created_at":"2026-11-08T15:28:08.993446Z","message":{"role":"assistant","content":"","thinking":"Okay"},"done":false}
{"model":"qwen3:8b","created_at":"2026-11-08T15:28:09.033517Z","message":{"role":"assistant","content":"","thinking":","},"done":false}

Ollama splits thinking and answer into message.thinking and message.content. The app never has to parse <think> tags out of the text.

Read the stats line

The last line, with "done":true, carries the numbers that the app shows under each answer. Every duration is in nanoseconds.

Field	Meaning	In this run
`load_duration`	loading the model into memory	2.8 s, because the model was not loaded yet
`prompt_eval_count`	tokens in the prompt	22
`prompt_eval_duration`	reading the prompt	0.09 s
`eval_count`	tokens generated	7
`eval_duration`	generating them	0.13 s

Tokens per second is the generated tokens divided by the time spent generating them.

tokens per second = eval_count / eval_duration * 1e9
                  = 7 / 132213000 * 1e9
                  = 52.9

This is the same "eval rate" that ollama run --verbose prints. Seven tokens is too short for a stable number. Longer answers give a steadier rate, which is why the speed tests ask for about 150 words.

Change the settings

Three settings go into an options object on the request. These are the ones the app uses.

curl http://127.0.0.1:11434/api/chat -d '{"model":"qwen3:8b","messages":[{"role":"user","content":"Name a color."}],"think":false,"stream":false,"options":{"temperature":0,"num_ctx":32768}}'
ollama ps

"stream":false returns the whole answer as one JSON object, which is easier to read by hand.

Setting	What it does	What the app does with it
`temperature`	How random the next token is. At `0` the model always picks the most likely token, so the same question gives the same answer.	A field in Settings. Empty means the model's own default.
`num_ctx`	The context window: how many tokens of conversation the model sees. Without it, Ollama uses its default and silently drops the start of a long chat.	Picks 8K, 16K, 32K, 64K, or 128K from the length of the chat, capped at the model's maximum.
`think`	A top-level field, not in `options`. `false`, `true`, or a level, from the list `/api/show` returned.	The thinking menu in the composer.

The context window costs memory. On one machine, ollama ps showed qwen3:8b at about 5 GB with num_ctx 4096 and at about 10 GB with 32768. The model is the same. The extra memory holds the conversation. Not every model grows this much. gpt-oss:20b uses a short sliding window in half of its layers, and going from 32K to 64K cost it only about 0.1 GB.

Every change of num_ctx makes Ollama reload the model. That is why the app jumps in large steps instead of growing by a few tokens per message.

Free the memory

A loaded model stays in memory for five minutes after the last request. A request to /api/generate with keep_alive set to 0 unloads it right away.

curl http://127.0.0.1:11434/api/generate -d '{"model":"qwen3:8b","keep_alive":0}'

{"model":"qwen3:8b","created_at":"2026-11-08T15:28:16.426867Z","response":"","done":true,"done_reason":"unload"}

The app sends this whenever you switch models, so two large models never share memory.

Why speed is about memory

Every generated token means reading the model's active weights from memory once. So the speed limit is roughly the memory bandwidth divided by the bytes of active weights.

tokens per second  ~  memory bandwidth / active weight bytes

That one line explains the three tests in the app.

A mixture of experts reads only the experts it uses, about 3B of qwen3:30b's 30B weights. It writes much faster than a dense model of the same file size.
Thinking does not change the rate. It changes how many tokens the model writes before the answer, so you wait longer.
Speculative decoding checks several guessed tokens in one read of the weights, so it gets more tokens out of each read.

Check

ollama ps

After the unload, ollama ps prints only its header line. You have now used /api/tags, /api/show, /api/chat, and /api/generate, the four endpoints the app calls.

Models, settings and speed

On this page