Diagrams AI can, and cannot, generate
19 comments
·March 18, 2025LASR
We use mermaidjs as a supercharged version of chain-of-thought for generating some sophisticated decompositions of the intent.
Then we injected the generated mermaid diagrams back into subsequent requests. Reasoning performance improves for a whole variety of applications.
Garlef
Neat idea!
Could you go into a bit more detail on how you encode the intent?
BOOSTERHIDROGEN
Any simple examples?
WesleyLivesay
Given the pace of development in this space, it is probably worth noting in the title that this is from November 2024 so the results might be a bit dated.
trash_cat
Sonnet 3.7 is perticularly good to generated xml diagrams that can be imported into draw.io. If you are using Cline, Windusurf or Cursor, you can ask it to create the xml file and immediately open it up. Combine it together with CONTEXT.md or ARCHITECTURE.md and you can get a very good overview of the codebase and have discussions around it.
stared
GPT 4o is not particularly good at this kind of logic, at least compared to other current models. Trying something that is at least in the top 10 from this WebDev Areans leaderboard: https://web.lmarena.ai/leaderboard would help.
Make sure it is allowed to think before doing (not necessarily in a dedicated thinking mode, it can be a regular prompt to design a graph before implementing it; make sure to add in a prompt who the graph is for (e.g. "a clean graph, suitable for a blog post for technical audience").
vunderba
Related - a nice time saver that I've been using since they added image recognition support to ChatGPT has been taking a quick snap of my crudely drawn hand sketched diagrams (on graph paper) with my phone and asking ChatGPT to convert them to mermaid UML syntax.
victorbjorklund
I have had good success with D2 diagrams with Claude: https://victorbjorklund.com/build-diagrams-as-code-with-d2-d...
They have icons for common things like cloud things.
submeta
Try asking llm to generate plantuml markup (use case, statechart, etc) which has some other diagram types in addition to mermaid markup. Then paste it into the free plantuml renderer. Works pretty well.
I also experimented with bpmn markup (xml). Realized there are already repos on GitHub creating bpmn diagrams from prompt.
You can also ask llms to create svg.
cptnmrvl
plantuml also supports AWS icons
james-bcn
Why just stick to Mermaid? I expect that there is a lot more material with regards to SVG that large models have been trained on. And it's a fairly simple format. Asking it to create diagrams in SVG format gives it much more flexibility. Of course there may be a bit less consistency, but there are ways around that (e.g. giving an example/template to follow).
Simon Willison has shown that current models aren't very good at creating an SVG of a pelican on a bicycle, but drawing a box diagram in SVG is a much simpler task.
TeMPOraL
Because few ever write diagrams in SVG. SVG is an output format, not an input format, and asking for a diagram in SVG is asking for the model to translate whatever you're asking about (e.g. names of systems you expect the model to know, like AWS, or regular code that needs to be turned into a diagram) into some hidden diagram-ish form, and then generate an SVG out of it.
Can't see it working without letting the model output an intermediary form in PlantUML or Mermaid or Dot - going straight to SVG is cramming too much work into too few tokens.
For the same reason, textual diagram formats are better for iterative work. SVG is too open-ended, and carries little to no semantics. Diagramming language are all about semantics, have fewer degrees of freedom, and much less spurious token noise.
cadamsdotcom
Thanks for writing this up. Some questions for the author:
Interesting perspective but it’s a bit incomplete without a comparison of various models and how they perform.
Kind of like Simon Willison’s now-famous “pelican on a bicycle” test, these diagrams might be done better by some models than others.
Second, this presents a static picture of things, but AI moves really fast! It’d also be great to understand how this capability is improving over time.
notTooFarGone
I used AI to generate some UML diagrams on a loosely coupled system - just fed it the actual classes where only names identify the actual links. It did quite a good job there.
It was a well defined domain so I guess the training data argument doesn't fit for stuff that is within a "natural" domain like graphs. LLMs can infer the behavior based on naming quite well.
peter_retief
I ask AI to generate diagrams in LaTeX, works well for me.
Random thoughts:
Sketching backed by automated cleanup can be good for entering small diagrams. There used to be an iOS app based on graphviz: http://instaviz.com
Constraint-based interactive layout may be underinvested, as a consequence of too many disappointments and false starts in the 1980s.
LLMs seem ill-suited to solving the optimization of combinatorial and geometric constraints and objectives required for good diagram layout. Overall, one has to admire the directness and simplicity of mermaid. Also, it would be great to someday see a practical tool with the quality and generality of the ultra-compact grid layout prototype from the Monash group, https://ialab.it.monash.edu/~dwyer/papers/gridlayout2015.pdf (2015!!)