Local AI chat app

In this course you run AI models on your own computer and connect them to a desktop app. Nothing you type leaves the machine, and there is no API bill.

The app is called Chat. It has two modes. Chat mode is a ChatGPT-style chat on a local Ollama model, with streaming answers, Stop, markdown, a model picker, and saved conversations. Code mode is a Claude Code-style agent. The Pi coding agent works in a folder you pick, and every file it reads, every edit, and every command it runs shows up as a card.

Under every answer the app shows the tokens per second. That turns it into an instrument. You use it to see why a mixture-of-experts model is fast, what thinking costs, and what speculative decoding does.

What you learn

1. Local AI with Ollama. Install it, pick models that fit in your memory, and call its API by hand. Settings such as temperature, context length, thinking levels, and unloading. What tokens per second means and how to compute it.
2. Tauri in five minutes. What it is and why it makes small desktop apps. The app's interface comes ready-made from the repo, so you do not build any of it.
3. Connect a model. You add the Rust code that talks to Ollama, and optionally to Apple's on-device model. The pages explain the parts that matter: the request, streaming, Stop, and the speed numbers.
4. Speed tests. Dense against mixture of experts, thinking on against off, and speculative decoding, measured in the app.
5. A harness. What turns a chat model into a coding agent, and how the app runs Pi and shows its work.

How it works

Tauri runs two programs in one app. React draws everything you see. Rust owns everything that touches the machine. It makes the HTTP calls to Ollama, starts and stops Pi, and saves conversations to disk. React never calls Ollama directly. It asks Rust, and Rust streams the answer back.

What you need

• A Mac or a Windows PC with enough memory for the models. Everyday chat runs on qwen3:8b, a 5.2 GB model. The speed tests use models of about 18 GB that need about 20 GB of free memory each.
• About an hour, plus download time for the models.
• A code editor. A coding agent in it, such as Cursor, Claude Code, or Codex, helps when a command fails.

You do not need an API key or an account anywhere.

The repository

Repository: daveebbelaar/local-chat

The main branch is the starter. It holds the whole interface, the conversation storage, and the configuration, but no connection to any model. You add that connection in three steps. Each step is a tag in the repo, and one command copies its Rust files into your clone.

Only src-tauri/src changes from step to step. The pages show the parts of each file worth reading, and link to the full file. The development branch holds the same three steps as commits, and solution is the reviewed, finished app.

How to use this guide

• Read the first two pages even if you never build the app. They are about running models, and they apply to every local AI tool.
• Every page ends with a check. Do not move on until it passes.
• When a command fails, paste the error into your coding agent.

pnpm tauri dev fails with this error: <paste the error>. Fix it.