Project 01, AI in the browser

Local AI Chatbot

Chat with a real language model that runs on your own graphics card. Nothing you type leaves this page, and after the first load it works without internet.

Runs on: Your GPU (WebGPU)
Model: Qwen2.5 0.5B or Llama 3.2 1B
First download: About 0.4 to 0.9 GB
Your browser: Checking

Chat

Load a model first. The small one is fine for quick answers and code snippets.

Model not loaded

System prompt

No messages yet. Try one of these once the model is ready:

How it works

Check the GPUThe page asks the browser for a WebGPU adapter and checks for 16-bit float support to pick the right model build.
Download onceWebLLM fetches the model weights and stores them in the browser cache. Later visits skip this step.
Stream the replyYour messages go to the engine as a normal chat completion request, and tokens stream back as they are generated.

import * as webllm from "https://cdn.jsdelivr.net/npm/@mlc-ai/web-llm@0.2.85/+esm";

// 1. Load the model (cached after the first time)
const engine = await webllm.CreateMLCEngine("Qwen2.5-0.5B-Instruct-q4f16_1-MLC", {
  initProgressCallback: (r) => console.log(r.text),
});

// 2. Ask it something, same shape as the OpenAI API
const stream = await engine.chat.completions.create({
  messages: [{ role: "user", content: "Hello!" }],
  stream: true,
});

// 3. Print tokens as they arrive
for await (const chunk of stream) {
  console.log(chunk.choices[0]?.delta?.content ?? "");
}