Browser-Provided Language Models

This is a note from my public notes. View the canonical version: Browser-Provided Language Models.

An emerging pattern where web browsers ship with or download trained language models, exposing them to web applications through standardized APIs. Enables developers to access LLM capabilities without managing model weights, infrastructure, or API keys.

This is the substrate underlying Prompt API and Writing Assistance APIs.

How It Works

1. Browser vendors bundle a small LLM or download one on first use (with user consent)
2. The model is cached locally and updated via browser update channels
3. Web apps access it through standard APIs (e.g., Prompt API)
4. Inference runs on-device using WebNN API or OS-level ML frameworks

Examples (2026)

Developer Implications

• Zero setup: no API keys, no server, no download management
• Availability uncertainty: model may not be available on all devices; check ai.languageModel.capabilities()
• Model opacity: developer doesn't control model version or weights
• Context limits: smaller models = smaller context windows than cloud APIs
• Download time: first-run experience may require model download

Privacy Model

Because models run locally, prompts and outputs never leave the device unless the app explicitly sends them elsewhere. This is a stronger privacy guarantee than cloud LLM APIs.

See AI Privacy and On-Device Machine Learning.

References

• https://github.com/webmachinelearning/prompt-api
• https://github.com/webmachinelearning

Related

• Prompt API
• WebNN API
• WebMachineLearning
• Writing Assistance APIs
• On-Device Machine Learning
• Large Language Models (LLMs)
• AI Privacy
• AI Inference
• Gemini Nano
• LLM Tool Calling
• LLM Structured Outputs
• LLM Streaming
• Edge AI
• Neural Processing Unit (NPU)
• Apple Intelligence
• Windows Copilot Runtime
• Apple Core ML

About Sébastien

Ready to get to the next level?