DiffusionGemma

Canonical version: DiffusionGemma.

DiffusionGemma is an experimental open model from Google DeepMind in the Gemma family that generates text by diffusion instead of autoregression. Rather than predicting one token at a time, it denoises whole spans in parallel, built on Gemma 4 and Gemini Diffusion research. The payoff is speed: up to ~4× faster output, exceeding 1,000 tokens/second on a single NVIDIA H100.

How diffusion text generation differs

Standard Large Language Models (LLMs) are autoregressive: strictly sequential, token-by-token, bottlenecked on memory bandwidth. DiffusionGemma uses discrete text diffusion with bi-directional attention to generate many tokens per forward pass (a "canvas" of 256), then iteratively refines them, enabling self-correction and better global consistency, and shifting the bottleneck from memory bandwidth to raw compute.

Architecture

• ~26B total parameters (25.2B), 3.8B active: AI Mixture of Experts (MoE) (8 of 128 experts active + 1 shared)
• Encoder-decoder: an autoregressive encoder caches the prompt context, paired with a diffusion decoder
• Up to 256K token Context Window; 262K vocabulary; sliding window 1024
• 15–20 tokens generated per forward pass (>1100 tok/s on H100 in FP8)
• Multimodal input (text, image, video → text); ~550M vision params; built-in thinking mode
• Supports NVIDIA NVFP4 (4-bit float) on Blackwell GPUs

Performance (instruction-tuned)

• MMLU Pro: 77.6% · GPQA Diamond: 73.2% · LiveCodeBench v6: 69.1% · MATH-Vision: 70.5%

Availability

• Apache 2.0 License; an open-weight release
• On HuggingFace (google/diffusiongemma-26B-A4B-it), Kaggle, and Google Vertex AI Model Garden

References

• https://deepmind.google/models/gemma/diffusiongemma/
• https://blog.google/innovation-and-ai/technology/developers-tools/diffusion-gemma-faster-text-generation/
• https://huggingface.co/google/diffusiongemma-26B-A4B-it
• https://ai.google.dev/gemma/docs/diffusiongemma
• https://developers.googleblog.com/diffusiongemma-the-developer-guide/
• https://www.xda-developers.com/tried-google-diffusiongemma-generate-text-like-image-local-llm/
• https://vllm.ai/blog/2026-06-10-diffusion-gemma

Related

• Gemma
• Google DeepMind
• Gemini
• Diffusion Models
• Large Language Models (LLMs)
• AI Mixture of Experts (MoE)
• AI Open Weight Models
• Context Window
• Apache 2.0 License

About Sébastien

Ready to get to the next level?