Connect Ollama

Copy in the Rust module that talks to Ollama, and read the four parts that matter, the request, streaming over a Channel, Stop, and the speed numbers.

By the end of this page the model button lists your Ollama models, answers stream in word by word, Stop works, and every answer shows its tokens per second.

The Rust file does exactly what you did by hand on the models, settings and speed page. It calls the same four endpoints and reads the same fields. The only new parts are how the pieces of the answer reach the window, and how Stop works.

Copy in the Ollama module

git checkout step-1-ollama -- src-tauri/src

This copies two files from the step-1-ollama tag into your clone. src-tauri/src/ollama.rs is new, and src-tauri/src/main.rs now registers its commands. If pnpm tauri dev is still running, it rebuilds and restarts the window by itself.

Full file on GitHub: ollama.rs

Register the commands

src-tauri/src/main.rs

// Hide the extra console window on Windows release builds.
#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]

mod ollama;
mod store;

fn main() {
    tauri::Builder::default()
        .plugin(tauri_plugin_opener::init())
        .plugin(tauri_plugin_dialog::init())
        .manage(ollama::ChatState::default())
        .invoke_handler(tauri::generate_handler![
            ollama::list_models,
            ollama::model_options,
            ollama::unload_model,
            ollama::chat,
            ollama::stop_chat,
            store::list_conversations,
            store::load_conversation,
            store::save_conversation,
            store::delete_conversation,
            store::load_settings,
            store::save_settings,
        ])
        .run(tauri::generate_context!())
        .expect("error while running Chat");
}

Five new commands. list_models, model_options, and unload_model call /api/tags, /api/show, and /api/generate with keep_alive: 0, the same calls you made with curl. chat and stop_chat are the interesting ones. ChatState holds the stop flag of the answer that is streaming now.

Start an answer without freezing the window

src-tauri/src/ollama.rs

/// Start an answer. Returns at once; the answer arrives on `on_event`.
#[tauri::command]
pub fn chat(request: ChatRequest, on_event: Channel<ChatEvent>, state: State<'_, ChatState>) {
    let stop = Arc::new(AtomicBool::new(false));
    // A new answer replaces the old one, so stop the old one first.
    if let Some(previous) = state.0.lock().unwrap().replace(stop.clone()) {
        previous.store(true, Ordering::SeqCst);
    }
    std::thread::spawn(move || {
        if let Err(message) = stream_answer(&request, &on_event, &stop) {
            let _ = on_event.send(ChatEvent::Error { message });
        }
    });
}

/// Stop the answer that is streaming now.
#[tauri::command]
pub fn stop_chat(state: State<'_, ChatState>) {
    if let Some(stop) = state.0.lock().unwrap().as_ref() {
        stop.store(true, Ordering::SeqCst);
    }
}

A model answers at 10 to 100 tokens per second, so a long answer takes a while. If chat waited for the whole answer, the window would freeze. So chat creates a stop flag, starts a thread, and returns at once. The thread does the waiting.

on_event is a Tauri Channel. React creates it and passes it in, and Rust calls send on it for every piece of the answer. A Channel keeps the messages in order and is built for this kind of stream.

stop_chat only sets the flag. The thread checks it before it reads the next line.

The request

src-tauri/src/ollama.rs

fn stream_answer(
    request: &ChatRequest,
    events: &Channel<ChatEvent>,
    stop: &AtomicBool,
) -> Result<(), String> {
    let mut body = json!({
        "model": request.model,
        "messages": request.messages,
        "stream": true,
        // Without num_ctx, Ollama uses its default and silently cuts the start of long chats.
        "options": { "num_ctx": context_size(&request.messages, request.context_length) },
    });
    if let Some(think) = &request.think {
        body["think"] = think.clone();
    }
    if let Some(temperature) = request.temperature {
        body["options"]["temperature"] = json!(temperature);
    }
    let response = post("/api/chat", &body)?;

This is the same body you sent with curl, built in Rust. think and temperature only go in when they are set, so a model without thinking never receives it. num_ctx is always set. context_size picks it from the length of the chat.

src-tauri/src/ollama.rs

/// The context window for this request: room for the chat so far plus a long answer.
/// It grows in large steps, because every change makes Ollama reload the model.
fn context_size(messages: &[Message], model_max: u64) -> u64 {
    let characters: usize = messages.iter().map(|m| m.content.len()).sum();
    let needed = characters as u64 / 3 + 4096; // about 3 characters per token, plus the answer
    let size = [8192, 16384, 32768, 65536, 131072]
        .into_iter()
        .find(|&size| size >= needed)
        .unwrap_or(131072);
    size.min(model_max)
}

About 3 characters per token, plus room for the answer, rounded up to the next large step. Large steps on purpose: every change of num_ctx makes Ollama reload the model.

Stream, stop, and measure

src-tauri/src/ollama.rs

    // Ollama sends one JSON object per line until `done` is true.
    let reader = BufReader::new(response.into_body().into_reader());
    for line in reader.lines() {
        if stop.load(Ordering::SeqCst) {
            // Returning drops the connection, and Ollama stops generating.
            let _ = events.send(ChatEvent::stopped());
            return Ok(());
        }
        let line = line.map_err(|e| e.to_string())?;
        if line.trim().is_empty() {
            continue;
        }
        let chunk: Value = serde_json::from_str(&line).map_err(|e| e.to_string())?;
        if let Some(error) = chunk["error"].as_str() {
            return Err(error.to_string());
        }
        let thinking = chunk["message"]["thinking"].as_str().unwrap_or("");
        if !thinking.is_empty() {
            let _ = events.send(ChatEvent::Thinking {
                delta: thinking.to_string(),
            });
        }
        let text = chunk["message"]["content"].as_str().unwrap_or("");
        if !text.is_empty() {
            let _ = events.send(ChatEvent::Text {
                delta: text.to_string(),
            });
        }
        if chunk["done"].as_bool() == Some(true) {
            // The last line carries the stats. Durations are in nanoseconds.
            let seconds = |key: &str| chunk[key].as_f64().unwrap_or(0.0) / 1e9;
            let tokens = chunk["eval_count"].as_u64().unwrap_or(0);
            let eval_seconds = seconds("eval_duration");
            let _ = events.send(ChatEvent::Done {
                stopped: false,
                tokens,
                tokens_per_second: if eval_seconds > 0.0 {
                    tokens as f64 / eval_seconds
                } else {
                    0.0
                },
                load_seconds: seconds("load_duration"),
                prompt_seconds: seconds("prompt_eval_duration"),
            });
        }
    }
    Ok(())
}

This is the loop that does the work.

Read one line, parse it as JSON.
Send message.thinking as a thinking event and message.content as a text event. Ollama keeps them in separate fields, so there are no <think> tags to parse.
On the last line, where done is true, compute the speed: eval_count divided by eval_duration in seconds. That is the formula from the API page. The load time and the prompt time go along, so the app can show them when they are slow.
When the stop flag is set, send a "stopped" event and return. Returning drops the HTTP connection, and Ollama stops generating. Stop really stops the model, not only the text on screen.

The events have a fixed shape that TypeScript reads.

src-tauri/src/ollama.rs

/// What Rust sends back over the Channel while the answer streams.
#[derive(Serialize, Clone)]
#[serde(
    tag = "event",
    content = "data",
    rename_all = "camelCase",
    rename_all_fields = "camelCase"
)]
pub enum ChatEvent {
    Thinking {
        delta: String,
    },
    Text {
        delta: String,
    },
    Done {
        stopped: bool,
        tokens: u64,
        tokens_per_second: f64,
        load_seconds: f64,
        prompt_seconds: f64,
    },
    Error {
        message: String,
    },
}

#[serde(tag = "event", content = "data")] turns each variant into JSON like {"event":"text","data":{"delta":"Hello"}}. On the React side, src/api.ts already expects exactly that.

src/api.ts

export function chat(request: ChatRequest, onEvent: (event: ChatEvent) => void) {
  const channel = new Channel<ChatEvent>();
  channel.onmessage = onEvent;
  return invoke<void>("chat", { request, onEvent: channel });
}

Check

pnpm tauri dev

The model button lists your models with their sizes. Pick qwen3:8b.
Ask "Explain in three short bullet points why local models are useful." The answer streams in. A "Thinking" line appears first, because qwen3:8b thinks by default.
Under the answer you see the tokens per second, the token count, and the model. The first answer after loading also shows the load time.
Ask for a long story and press Stop. The text stops and "Stopped" appears.
Pick another model, then run ollama ps in a terminal. Only the new model is loaded, because the app unloads the old one when you switch.
Quit and restart the app. Your conversation is still in the sidebar.

Connect Ollama

Copy in the Ollama module

Register the commands

Start an answer without freezing the window

The request

Stream, stop, and measure

Check

On this page