One of the most interesting new #AI models this year is DiffusionGemma, a Mixture-of-Experts (MoE) model from Google DeepMind. While its architecture isn't well-supported by all engines yet, variants currently work on Ollama and LM Studio.

While diffusiongemma-26B-A4B-it lacks some of the features of other Gemma 4 models, it is blazingly fast. It activates only 4 billion of its 26 billion parameters at a time, which is a now-common technique for speeding up processing on local systems, and its mixture-of-experts architecture easily supports 8 #MoE experts on higher-end consumer grade hardware. However, what really sets DiffusionGemma apart is that instead of using standard autoregression for inference, which works token-by-token, it borrows a technique from image-generating models to iteratively refine blocks of tokens in parallel for increased performance.

Autoregression is a type of sequential processing where the model looks backwards at recently generated tokens to create the next set. In contrast, discrete diffusion models iteratively add and remove tokens from each block in a non-linear way. The result is not only a noticeable increase in processing speed, but also in more dynamic and creative output even at more rigorous instruction-following temperatures.

In contrast to models that can generate tens or hundreds of tokens per second on consumer-grade hardware, DiffusionGemma can churn out over 1,000 tokens-per-second on the same hardware. Nothing in life is free, though; DiffusionGemma trades a slight reduction in accuracy (roughly 5-15% on most benchmarks compared to other Gemma 4 models) for much lower latency and a massive increase in tokens-per-second. For tasks where speed and creative output matter most, this is a viable trade-off.

As an open-weight model released under the Apache 2.0 open-source license, DiffusionGemma is a great candidate for further research and post-training. As an excellent example of the rapid advancements in AI #edge_computing and the ability of smaller language models to compete with frontier models for various use cases, I highly recommend giving DiffusionGemma a try.