Skip to content(if available)orjump to list(if available)

TiDAR: Think in Diffusion, Talk in Autoregression

Workaccount2

An update to Gemini diffusion is one of my most eagerly anticipated AI releases. It released to mild fanfare (mostly because you needed to request access to use it), and there has been silence ever since.

Hopefully it's not more Google abandonware, because it was wicked fast and a delight to use

ACCount37

It's not a very promising direction because autoregressive LLMs still deliver better output quality per model weight, as a rule.

Now, is it possible that a model can combine advantages of both? Combine fast generation and multidirectional causality of diffusion with precision, capabilities and generalization of autoregression?

Maybe. This paper is research in that direction. So far, it's not a clear upgrade over autoregressive LLMs.

Alifatisk

I've tried dLLMs like Mercury and they look promising.