Teaching LLMs how to solid model

Most likely you won’t be asking for specific things like “3mm hole 3in from the side”, you’ll say things like “Create a plastic enclosure sized to go under a desk, ok add a usb receptacle opening, ok add flanges with standard screw holes”

In the text to CAD ecosystem we talk about matching our language/framework to “design intent” a lot. The ideal interface is usually higher level than people expect it to be.

I think this is along the lines of the AI horseless carriage[1] topic that is also on the front page right now. You seem to be describing the current method as operated through an AI intermediary. I think the power in AI for CAD will be at a higher level than lines, faces and holes. It will be more along the lines of "make a bracket between these two parts". "Make this part bolt to that other part". "Attach this pump to this gear train" (where the AI determines the pump uses a SAE 4 bolt flange of a particular size and a splined connection, then adds the required features to the housing and shafts). I think it will operate on higher structures than current CAD typically works with, and I don't think it will be history tree and sketch based like Solidworks or Inventor. I suspect it will be more of a direct modelling approach. I also think integrating FEA to allow the AI to check its work will be part of it. When you tell it to make a bracket between two parts, it can check the weight of the two parts, and some environmental specification from a project definition, then auto-configure FEA to check the correct number of bolts, material thickness, etc. If it made the bracket from folded sheet steel, you could then tell it you want a cast aluminum bracket, and it could redo the work.

[1]https://news.ycombinator.com/item?id=43773813

I have come across a significant number of non engineers wanting to do, what ultimately involves some basic CAD modelling. Some can stall on such tasks for years (home renovation) or just don't do it at all. After some brief research, the main cause is not wanting to sink over 30 hours into learning basics of a cad package of choice.

For some reason they imagine it as a daunting, complicated, impenetrable task with many pitfalls, which aren't surmountable. Be it interface, general idea how it operates, fear of unknown details (tolerances, clearances).

It's easy to underestimate the knowledge required to use a cad productively.

One such anecdata near me are highschools that buy 3d printers and think pupils will naturally want to print models. After initial days of fascination they stopped being used at all. I've heard from a person close to the education that it's a country wide phenomena.

Back to the point though - maybe there's a group of users that want to create, but just can't do CAD at all and such text description seem perfect for them.

> and I want a 3mm hole in one side of the plate. No the other side. No, not like that, at the bottom. Now make it 10mm from the other hole. No the other hole. No, up not sideways.

One thing that is interesting here is you can read faster than TTS to absorb info. But you can speak much faster than you can type. So is it all that typing that's the problem or could be just an interface problem? and in your example, you could also just draw with your hand(wrist sensor) + talk.

As I've been using agents to code this way. Its way faster.

> I'm having trouble understanding why you would want to do this.

You would be amazed at how much time CAD users spend using Propriety CAD Package A to redraw things from PDFs generated by Propriety CAD Package B

If you would use LLM-assisted CAD for real industrial design, you would have to end up by specifying exactly where everything has to go and what size it has to be. But if you are doing that then you may as well make an automated program to convert those specific requirements into a 3D model.

Oh wait, that's CAD.

Cynical take aside, I think this could be quite useful for normal people making simple stuff, and could really help consumer 3D printing have a much larger impact.

Here's how it might work, by analogy to the workflow for image generation:

"An aerodynamically curved plastic enclosure for a form-over-function guitar amp."

Then you get something with the basic shapes and bevels in place, and adjust it in CAD to fit your actual design goals. Then,

"Given this shape, make it easy to injection mold."

Then it would smooth out some things a little too much, and you'd fix it in CAD. Then, finally,

"Making only very small changes and no changes at all to the surfaces I've marked as mounting-related in CAD, unify my additions visually with the overall design of the curved shell."

Then you'd have to fix a couple other things, and you'd be finished.

So maybe the future is to draw a picture, and go from there?

For instance: My modelling abilities are limited. I can draw what I want, with measurements, but I am not a draftsman. I can also explain the concept, in conversational English, to a person who uses CAD regularly and they can hammer out a model in no time. This is a thing that I've done successfully in the past.

Could I just do it myself? Sure, eventually! But my modelling needs are very few and far between. It isn't something I need to do every day, or even every year. It would take me longer to learn the workflow and toolsets of [insert CAD system here] than to just earn some money doing something that I'm already good at and pay someone else to do the CAD work.

Except maybe in the future, perhaps I will be able use the bot to help bridge the gap between a napkin sketch of a widget and a digital model of that same widget. (Maybe like Scotty tried to do with the mouse in Star Trek IV.)

(And before anyone says it: I'm not really particularly interested in becoming proficient at CAD. I know I can learn it, but I just don't want to. It has never been my goal to become proficient at every trade under the sun and there are other skills that I'd rather focus on learning and maintaining instead. And that's OK -- there's lots of other things in life that I will probably also never seek to be proficient at, too.)

As a MCAD user this makes me feel more confident that my skills are safe for a bit longer. The geometry you were trying to generate (minus bayonet lock, which is actually a tricky thing to make because it relies on elastic properties of the material) takes maybe a few minutes to build in Solidworks or any modern CAD package.

This matches my experience having Claude 3.5 and Gemini 2.0-flash generate openSCAD, but I would call it interesting instead of a game changer.

It gets pretty confused about the rotation of some things and generally needs manual fixing. But it kind of gets the big picture sort of right. It mmmmayybe saved me time the last time I used it but I'm not sure. Fun experiment though.

Sounds like he could have been using an implementation of stable-diffusion+control-net. I’ve used Automatic1111, but I understand comfyUI and somethingsomethingforge are more modern versions.

Happy to be corrected but this sounds like the kind of bullshit that crops up from time to time confusing "old" AI with generative AI.

Not that I don't believe it's possible. I just think the alternative (that it's bullshit) is more likely.

I’m not sure that CAD interfaces are terrible, it’s just hard work

OpenSCAD has some fundamental issues with which folks are well aware. Build123d is a Python alternative that shows promise and seems more capable, and there's others around.

Hard to beat the mindshare of OpenSCAD at the moment though.

I definitely agree with your point about the long prompts.

The long prompts are primarily an artifact of trying to make an eval where there is a "correct" STL.

I think your broader point, text input is bad for CAD, is also correct. Some combo of voice/text input + using a cursor to click on geometry makes sense. For example, clicking on the surface in question and then asking for "m6 threaded holes at the corners". I think a drawing input also make sense as its quite quick to do.

There are diffusion models for 3D generation. They make pretty good decorative or ornamental models, like figurines. They are less good for CAD.

Zoo co-founder here. Our product is still pre-v1. But getting to v1 very soon. We actually built a whole new CAD kernel from the ground up. I say this because we can't actually train on models the CAD engine does not yet support. Just 2 weeks ago we shipped csg boolean operations to the CAD engine. This unlocked new data to train our model on that use those operations. So its actually fair to say at the time he used our model we using about 2% of the data we actually have. Once we can use more and more the ability of the model will only get better.