Hey Reader,

For years, autoregressive models (ARMs) have dominated the landscape of large language models (LLMs).

They generate text one token at a time, a process that has proven effective but comes with inherent limitations.

What if there was another way—one that could model entire sequences holistically rather than predicting words step by step?

Enter Large Language Diffusion Models (LLaDA), an approach inspired by the success of diffusion models in image generation.

Instead of relying on autoregressive decoding, LLaDA gradually refines predictions through a reverse denoising process, leading to strong performance in in-context learning, instruction-following, and even complex reasoning tasks.

​The latest research introduces LLaDA, a large-scale diffusion-based language model that challenges the status quo.

It not only scales effectively but also demonstrates competitive performance with state-of-the-art ARMs.

Figure 1 illustrates that LLaDA 8B achieves performance on par with strong LLMs like LLaMA3 8B across various benchmarks, including MMLU, TruthfulQA, ARC-C, GSM8K, Math, HumanEval, MBPP, CMMLU, and C-Eval.

​

Table 3 demonstrates LLaDA's ability to address the "reversal curse," surpassing GPT-4o in a reversal poem completion task, highlighting its proficiency in tasks requiring bidirectional context.

​

These results suggest that diffusion models are no longer limited to images; they may redefine how we build and scale LLMs.

If you’re interested in discussing this further, I’d love to connect.

Let me know what you think!

Best,
Diogo