WebGPU enables local LLM in the browser. Demo site with AI chat
25 comments
·August 2, 2025echoangle
andreinwald
It works on small Llama-3.2-1B model, specially for less powerfull GPU devices
maxmcd
Looks like this is a wrapper around: https://github.com/mlc-ai/web-llm
Which has a full web demo: https://chat.webllm.ai/
swores
Is this correct?
It doesn't seem so to me, either from the way it works or from what little of the code I've looked at...
But I don't have time to do more than the quick glance I just did at a few of the files of each and need to run, so hopefully someone cleverer than me who won't need as much time as me to answer the question could confirm while I'm afk
refulgentis
Entirely correct, c.f. LLM.ts. The bit to Show HN here, is about ~40 lines of code. Simple Typescript MVP of calling the library.
(source: maintains an LLM client that works across MLC/llama.cpp/3P providers; author of sibling comment that misunderstood initially)
refulgentis
I am to see it regardless - projects been very low activity for months. Just last night I was thinking about ripping it out before launch. No observable future.
EDIT: Ah, wrapper ~= using it as a dependency, no more
andreinwald
Browser LLM demo working on JavaScript and WebGPU. WebGPU is already supported in Chrome, Safari, Firefox, iOS (v26) and Android.
Demo, similar to ChatGPT https://andreinwald.github.io/browser-llm/
Code https://github.com/andreinwald/browser-llm
- No need to use your OPENAI_API_KEY - its local model that runs on your device
- No network requests to any API
- No need to install any program
- No need to download files on your device (model is cached in browser)
- Site will ask before downloading large files (llm model) to browser cache
- Hosted on Github Pages from this repo - secure, because you see what you are running
scottfr
There is a Prompt API in development that's available in both Chrome and Edge to give access to a local LLM. Chrome extensions have access to it and I believe websites can request access as part of an origin trial.
The model is fully managed by the browser. It's currently the Gemini Nano model on Chrome, and they are testing a version of the Gemma 3n model in beta channels. Edge uses phi-4-mini.
More information is available here: https://github.com/webmachinelearning/prompt-api
om8
To have a gpu inference, you need a gpu. I have a demo that runs 8B llama on any computer with 4 gigs of ram
adastra22
Any computer with a display has a GPU.
petermcneeley
This demo only works if you have the webgpu feature "f16". You can find out if you have this by checking for the feature list in https://webgpureport.org/ . The page itself can of course check for this but since f16 support is common they probably just didnt bother.
andreinwald
Also here https://caniuse.com/webgpu
andreinwald
Model used: Llama 3.2 1B (small). Quality hould be similar with running Ollama app with same small model.
asim
What's the performance of a model like vs an OpenAI API? What's the comparable here? Edit: I see it's same models locally that you'd run using Ollama or something else. So basically just constrained by the size of the model, GPU and perf of the machine.
andreinwald
Yes, its very similar to Ollama app, and Llama-3.2-1B model used
pjmlp
Beware of opening this on mobile Internet.
andreinwald
Demo site is asking before download
lukan
Well, I am on a mobile right now, can someone maybe share anything about the performance?
andreinwald
On my Android device works pretty fast.
But keep in mind that it's small Llama-3.2-1B model, specially for less powerfull GPU.
pjmlp
Not everyone enjoys unlimited data plans, and 500 MB is schon a lot.
andsoitis
very cool. improvement would be if the input text box is always on screen, rather than having to manually scroll down as the screen fills.
Martin_Silenus
[dead]
Fun demo but the model that's used seems to be pretty stupid:
> What's the best way to get to space?
>> Unfortunately, it's not currently possible for humans to travel to space in the same way that astronauts do. While there have been several manned missions to space, such as those to the International Space Station, the technology and resources required to make interstellar travel feasible are still in the early stages of development.
>> Currently, the fastest spacecraft is the Apollo missions, which took around 77 hours to reach the Moon. From there, they traveled to other planets, but the distances between them are so great that even at high speeds, it would take many years to reach them.
>> As for traveling to space itself, there are no commercial or private companies that offer space travel services, and the cost of such a journey is estimated to be in the tens of millions of dollars per person.
>> However, there are some concepts and technologies being explored that might one day make space travel more accessible and affordable. For example, private companies like SpaceX and Blue Origin are working on developing reusable rockets that could potentially take people to space more quickly and reliably.
>> But for now, let's just say that space travel is still in the future, and we'll have to wait a bit longer for humans to venture into the cosmos!