Diffusion models can't replace LLMs, says Denis O.

Gary Marcus Bogdan Grigorescu Purnima Gauthron Victoria Coté Dr Jason Price Anindita Biswas Dr. Jeffrey Funk Onyinye Ozor Prof J. Mark Bishop Mick Fews Shobhit Tankha Ivo Koutsaroff (掘露 伊保龍, KUTSURO Iboryu) Koutsaroff (掘露 伊保龍, KUTSURO Iboryu) Dinakar R. Denis O. Terry Bollinger Dr Sara Saravi Russ Guillette @Ellzabeth abete Hernandez Axel C.Mykel G. Larson ♞ Neil Gentleman-Hobbs Peter Bednar Susan Furnell Beth Carey John Ball Stephen Klein Tara TAUBMAN-BASSIRIAN LL.M Maria Santacaterina Dr Eliza Mik Dagmar Monett Ulla Coester Donn Gurule Evelyn Van Orden Debbie Reynolds Garrick Villaume Jason Gulya Tim El-Sheikh Tim Grice Hristo Georgiev Moira S.

Bear in mind that AI celebrities like Andrej Karpathy and Andrew Ng are also deep-pocketed investors in AI. They have to keep the drumbeat going in the "AI base". dLLMs, same inaccuracies, but faster and cheaper will diffuse away..:)

Richard Self saying “parallel text generation is semantically wrong” feels a bit too strong. Language definitely works best when it’s built step by step. We process ideas in order, and that sequence matters. But that doesn’t mean everything a model does has to happen one word at a time. In fact, during training, models often use parallel structures like masked language modeling or bidirectional attention to learn how words relate across a sentence, even if they’re far apart. Some of the newer diffusion based models, like Inception or CoDi, actually try to mix in that step by step logic too. They don’t just throw sequence out the window they’re trying to keep the benefits of autoregressive models while exploring new ways to generate text. So no, parallel generation isn’t “wrong.” It’s just a different approach, and in some cases, it might actually help with things like long range coherence or editing larger chunks of text. It’s not magic, but it’s also not something to dismiss out of hand.

"We also know that diffusion models eat TFLOPs" - TSLOP that's what the article looks and feels like. It's written at least partly with a generative LLM (looks more like GPT than diffusion). Checkout the key points, key aspects and key potential advantages paragraphs (former two at the beginning of article, latter one at the end of the piece).

At this point such pronouncements seem like a social experiment. Keep repeating unsubstantiated claims until people believe them to be the truth.

You're absolutely right—language unfolds over time, and its meaning is shaped by how ideas are introduced, developed, and linked together. Each statement depends on what came before. You can't understand a conclusion without grasping its premises. Arguments typically follow a flow—claims, evidence, counterarguments, then conclusions. Readers or listeners refine their understanding step by step, adjusting their perspective with each new piece of information. Whether it's a persuasive essay or a philosophical debate, the arrangement of ideas steers how they're understood and judged. Parallel processing, like analyzing multiple perspectives at once, can supplement this, but at its core, language guides us linearly.

It can be assumed that in the process of human thinking, there are constant 'recall' steps that lead to the generation of a set of relevant concepts and meanings in the mind. This is most likely a parallel process. Diffusion models could be quite suitable for implementing such a mechanism.

The way to read this: Richard Self Denis O. The honest facts about LLMs and their limitations are well-exposed. Another prefix to LLMs, dLLMs to spin a new story. This fits into both categories: predictably irrational at scale and lipstick on Yaatt

Good points!

This is great. The more people are exploring new techniques the better. I think we can now get past the "AGI" / sentience is just round the corner and just accept them as useful tools for the vast majority of people using them with care.