DiffusionGemma Challenges Traditional LLMs: Parallel Generation Up to 4 Times Faster on GPU
Google DeepMind
Google DeepMind has announced DiffusionGemma, a new experimental open-source model that explores an alternative approach to text generation compared to current autoregressive Large Language Models (LLMs). Distributed under the Apache 2.0 license, the project aims to drastically reduce latency in local inference by leveraging a technique derived from diffusion models that are already widely used in image generation.
The defining feature of DiffusionGemma is the abandonment of traditional sequential token-by-token generation. Instead of producing one word at a time in a strictly linear order, the model generates and refines entire blocks of text in parallel, processing up to 256 tokens per step. According to Google DeepMind, this approach allows for performance that is up to four times superior compared to equivalent autoregressive models when run in low-concurrency scenarios, such as typical local use by a single user.
DiffusionGemma is built on the Gemma 4 architecture and utilizes a Mixture of Experts (MoE) configuration with a total of 26 billion parameters. However, during inference, only 3.8 billion parameters are activated, significantly reducing the computational load. The model also integrates a new “diffusion head” specifically designed to maximize generation speed.
The idea behind the project is similar to that employed by image generators based on diffusion: the system starts from a sequence made up of random tokens and progressively refines it through successive steps until it converges on coherent text. This process allows the model to consider the entire block of text simultaneously during generation, introducing capabilities that are difficult to achieve with traditional autoregressive models.
Among the advantages highlighted by Google DeepMind is the adoption of bidirectional attention. Because all tokens in the block can be analyzed simultaneously, each element has the complete context of the sentence or paragraph being constructed. This can be particularly useful in tasks such as inline editing, code completion, infilling, generating biological sequences, and constructing complex mathematical structures. The model is also capable of iteratively correcting its output during generation, evaluating the text in its entirety rather than being limited to the previous context.
The claimed performance shows particularly interesting results on modern GPUs. Google indicates speeds exceeding 1,000 tokens per second on a single NVIDIA H100 and over 700 tokens per second on a GeForce RTX 5090. NVIDIA, which collaborated in optimizing the software, provides additional data related to its platforms: about 150 tokens per second on DGX Spark and up to 2,000 tokens per second on DGX Station. According to the company, the reason lies in the very nature of the workload. While traditional LLMs are often limited by memory bandwidth, the diffusion paradigm transforms much of the processing into a problem more oriented towards parallel computing, an area where GPUs and Tensor Cores are particularly effective.
On the hardware front, DiffusionGemma has been designed to run on high-end consumer systems as well. A quantized version of the model can be executed within the limit of about 18 GB of VRAM, making it compatible with graphics cards such as GeForce RTX 4090 and RTX 5090. Optimizations for Hopper and Blackwell architectures are also planned, including support for the NVFP4 4-bit format, which aims to increase throughput while maintaining accuracy close to that of traditional configurations.
Google DeepMind emphasizes that DiffusionGemma remains an experimental project. The priority assigned to speed and parallel generation comes with trade-offs in output quality compared to standard Gemma 4 models, which remain the recommended solution for productive applications where text quality is the primary requirement.
The company, however, sees significant potential for developers and researchers interested in low-latency interactive workflows, local chatbots, AI assistants running directly on the device, and agentic applications that require quick responses. Supporting this vision is the immediate integration with tools prevalent in the open-source ecosystem, including Hugging Face Transformers, vLLM, Unsloth, and NVIDIA NeMo, while official support for llama.cpp is currently under development.