LLM Visualization

I just want to say that this is fantastic and I'm planning to show it to my 5yo son's computer club.

Wow, this is tremendously intricate and very impressive! What an awesome way to visualize the process.

Related. Others?

LLM Visualization - https://news.ycombinator.com/item?id=38505211 - Dec 2023 (131 comments)

Here is another take on visualizing transformers from Georgia Tech researchers: https://poloclub.github.io/transformer-explainer/

The Illustrated Transformer: https://jalammar.github.io/illustrated-transformer/

Sebastian Raschka, PhD has a post on the architectures: https://magazine.sebastianraschka.com/p/from-gpt-2-to-gpt-os...

This HN comment has numerous resources: https://news.ycombinator.com/item?id=35712334

Fascinating visualization. To think, we can visualize the entire* process but cannot understand the inner workings of a model with regards to decision making. This was true last I looked into it a year or so ago, not aware on any advancements in that aspect.

This is awesome! Would be cool if these LLM visualizations were turned into teaching tools, like showing how attention moves during generation or how prompts shift the model’s output. Feels like that kind of interactive view could really help people get what’s going on under the hood.

I always liked this visualization from a while ago https://alphacode.deepmind.com/ (Press play, zoom all the way out and scroll down if on mobile)

I wish n-gate was still around. He would note the high vote to comment ratio. When HN has little to say, it's always a sign of a high quality very technical article.

On a more serious note, this highlights a deeper issue with HN, similar sites and the attention economy. When an article takes a lot of time to read:

- The only people commenting at first have no read it.

- By the time you are done reading it, it's no longer visible on the front page so new people are not coming in anymore and the discussion appears dead. This discourages people who read it from making thoughtful comments because few people will read it.

- There are people who wait for the discussion to die down so they can read it without missing the later thoughtful comments but they are discouraged from participating earlier while the discussion is alive because then they'd have to wade through the constantly changing discussion and separate what they have already seen from what they haven't.

---

Back on topic, I'd love to see this with weights from an actual working model and a customizable input text so we could see how both the seed and input affects the output. And also a way to explore vectors representing "meanings" the way 3blue1brown did in his LLM videos.

[dead]

[dead]