Show HN: Why write code if the LLM can just do the thing? (web app experiment)

Amazing! Very similar approach, would love to heae what you think: https://github.com/gerkensm/vaporvibe

If you added a few more tools that let the LLM modify code files that would directly serve requests, that would significantly speed up future responses and also ensure consistency. Code would act like memory. A direct HTTP request to the LLM is like a cache miss. You could still have the feedback mechanism allowing a bypass that causes an update to the code. Perhaps code just becomes a store of consistency for LLMs over time.

It's orders of magnitude cheaper to serve requests with conventional methods than directly with LLM. My back-of-envelope calculation says, optimistically, it takes more than 100 GFLOPs to generate 10 tokens using a 7 billion parameter LLM. There are better ways to use electricity.

The question posed sounds like "why should we have deterministic behavior if we can have non-deterministic behavior instead?"

Am I wrong to think that the answer is obvious? I mean, who wants web apps to behave differently every time you interact with them?

This is amazing. It very creatively emphasizes how our definition of "boilerplate code" will shift over time. Another layer of abstraction would be running N of these, sandboxed, responding to each request, and then serving whichever instance is internally evaluated to have done the best. Then you're kind of performing meta reinforcement learning with each whole system as a head.

The hard part (coming from this direction) is enshrining the translation of specific user intentions into deterministic outputs, as others here have already mentioned. The hard part when coming from the other direction (traditional web apps) is responding fluidly/flexibly, or resolving the variance in each user's ability to express their intent.

Stability/consistency could be introduced through traditional mechanisms: Encoded instructions systematically evaluated, or, via the LLMs language interface, intent-focusing mechanisms: through increasing the prompt length / hydrating the user request with additional context/intent: "use this UI, don't drop the db."

From where I'm sitting, LLMs provide a now modality for evaluating intent. How we act on that intent can be totally fluid, totally rigid, or, perhaps obviously, somewhere in-between.

Very provocative to see this near-maximum example of non-deterministic fluid intent interpretation>execution. Thanks, I hate how much I love it!

Wrote a similar PoC here: https://github.com/s1liconcow/autoapp

Some ideas - use a slower 'design' model at startup to generate the initial app theme and DB schema and a 'fast' model for responses. I tried a version using PostREST so the logic was in entirely in the DB and but then it gets too complicated and either the design model failed to one-shot a valid schema or the fast model kept on generating invalid queries.

I also use some well known CSS libraries and remember previous pages to maintain some UI consistency.

It could be an interesting benchmark or "App Bench". How well can an LLM one-shot create a working application.

Just in time UI is incredibly promising direction. I don't expect (in the near term) that entire apps would do this but many small parts of them would really benefit. For instance, website/app tours could be just generated atop the existing ui.

Yes, why not burn a forest to make a up of tea, if we can fully externalise the cost.

Even if LLMs do get 10x as fast, that's not even remotely enough. They are 1e9 times as compute intensive.

I tried this as well at https://github.com/jasonthorsness/ginprov (hosted at https://ginprov.com). After a while it sort of starts to all look the same though.

Currently today, I would say these models can be used by someone with minimal knowledge to churn out SPAs with React. They can probably get pretty far into making logins, message systems, and so on because there is lots of training data for those things. They can struggle through building desktop apps as well with relative ease compared to how I had to learn in years long past.

What these LLMs continue to prove those is they are no substitute for real domain knowledge. To date, I've yet to have a model implement RAFT consensus correctly in testing to see if they can build a database.

The way I interact with these models is almost adversarial in nature. I prompt them with the bare minimum that a developer might get in a feature request. I may even have a planning session to populate the context before I set it off on a task.

The bias in these LLMs really shines through an proves their autocomplete properties when they have a strong bias towards changing the one snippet of code I wrote because it doesn't fit in how it's training data would suggest the shape of it's code should be. Most models will course correct with instructions that they are wrong and I am right though.

One thing I've noted is that if you let it generate choices for you from the start of a project, it will make poor choices in nearly every language. You can be using uv to manage a python project and it will continue to try using pip or python commands. You can start an electron app and it will continuously botch if it's using commonjs or some other standard. It persistently wants to download go modules before coding instead of just writing the code and doing `go mod tidy` after (it literally doesn't need the module in advance, it doesn't even have tools to probe the module before writing the code anyway).

RAFT consensus is my go-to test because there is no 1 size fits all way for you to implement it. It might get an in-memory key store system right, but what if you want it to organize etcd/raft/v3 in a way that you can do multi-group RAFT? What if you need RAFT to coordinate some other form of data replication? None of these LLMs can really do it without a lot of prep work.

This is across all the models available from OpenAI, Claude, and Google.

Kind of similar to the Minecraft game which computed frames on the fly without any code behind the visuals?

I don't see a point in using probabilistic methods to perform a deterministic logic. Even if it's output is correct, it's wasteful.