Spec-Driven Development: The Waterfall Strikes Back

This is a weird article. How many times in your career have you been handed a grossly under-specified feature and had to muddle your way through, asking relevant people along the way and still being told at the end that it’s wrong?

This is exactly the same thing but for AIs. The user might think that the AI got it wrong, except the spec was under-specified and it had to make choices to fill in the gaps, just like a human would.

It’s all well and good if you don’t actually know what you want and you’re using the AI to explore possibilities, but if you already have a firm idea of what you want, just tell it in detail.

Maybe the article is actually about bad specs? It does seem to venture into that territory, but that isn’t the main thrust.

Overall I think this is just a part of the cottage industry that’s sprung up around agile, and an argument for that industry to stay relevant in the age of AI coding, without being well supported by anything.

This article if for those who already made up their mind that "spec-based-development" isn't for them.

I believe (and practice) that spec-based development is one of the future methodologis for developing projects with LLMs. At least it will be one of the niches.

Author thinks about specs as waterfalls. I think about them as a context entrypoint for LLMs. Giving enough info about the project (including user stories, tech design requirements, filesystem structure and meaning, core interfaces/models, functions, etc) LLM will be able to build sufficient initial context for the solution to expand it by reading files and grepping text. And the most interesting is that you can make LLM to keep the context/spec/projetc file updated each time LLM updates the project. Viola: now you are in agile again: just keep iterating on the context/spec/project

I just tried an experiment using Spec-Kit from GitHub to build a CLI tool. Perhaps the scope of the tool doesn't align itself with Spec-Driven Development, but I found the many many hours—tweaking, asking, correcting, analyzing, adapting, refining, reshaping, etc—before getting to see any code, challenging. As would be the case with Waterfall today, the lack of iterative end-to-end feedback is foreign and frustrating to me.

After Claude finally produced a significant amount of code, and after realizing it hadn't built the right thing, I was back to the drawing board to find out what language in the spec had led it astray. Never mind digging through the code at this point; it would be just as good to start again than to try to onboard myself to the 1000s of lines of code it had built... and I suppose the point is to ignore the code as "implementation detail" anyway.

Just to make clear: I love writing code with an LLM, be it for brainstorming, research, or implementation. I often write—and have it output—small markdown notes and plans for it to ground itself. I think I just found this experience with SDD quite heavy-handed and the workflow unwieldy.

And this is absolutely fine, because the problem with waterfall wasn't the detailed spec, it was

a) the mulyi-year lead time from starting the spec to getting a finished product

b) no (cheap) way to iterate or deliver outside the spec

Neither of these are a problem with SDD.

I think I've seen enough of a trend: all these LLM ideas eventually get absorbed by the LLM provider and integrated. The OSS projects or companies with products eventually become irrelevant.

So they're more like 3rd party innovations to lobby LLM providers to integrate functionalities.

X prompting method/coding behaviors? Integrated. Media? Integrated. RAG? Integrated. Coding environment? Integrated. Agents? Integrated. Spec-driven development? It's definitely present, perhaps not as formal yet.

A point I like to make in discussions like this is that software and hardware specifications are very different. We think of software as the thing we're building. But it's really just a spec that gets turned into the thing we actually run. It's just that the building process is fully automated. What we do when we create software is creating a specification in source code form.

Compared to what an architect does when they create a blueprint for a building, creating blueprints for software source code is not a thing.

What in waterfall is considered the design phase is the equivalent of an architect doing sketches, prototypes, and other stuff very early in the project. It's not creating the actual blue print. The building blue print is the equivalent of source code here. It's a complete plan for actually constructing the building down to every nut and bolt.

The big difference here is that building construction is not automated, costly, and risky. So architects try to get their blueprint to a level where they can minimize all of that cost and risk. And you only build the bridge once. So iterating is not really a thing either.

Software is very different; compiling and deploying is relatively cheap and risk free. And typically fully automated. All the effort and risk is contained in the specification process itself. Which is why iteration works.

Architects abandon their sketches and drafts after they've served their purpose. The same is true in waterfall development. The early designs (whiteboard, napking, UML, brainfart on a wiki, etc.) don't matter once the development kicks off. As iterations happen, they fall behind and they just don't matter. Many projects don't have a design phase at all.

The fallacy that software is imperfect as an engineering discipline because we are sloppy with our designs doesn't hold up once you realize that essentially all the effort goes into creating hyper detailed specifications, i.e. the source code.

Having design specifications for your specifications just isn't a thing. Not for buildings, not for software.

It's a nice observation that Spec-Driven Development essentially implements the waterfall model.

Personally, I tried SDD, consciously trying to like it, but gave up. I find writing specs much harder than writing code, especially when trying to express the finer points of a project. And of course, there is also that personal preference: I like writing code, much more than text. Yes, there are times where I shout "Do What I Mean, not what I say!", but these are mostly learning opportunities.

Bit of a tangent, but this reminds me of a video[1] I watched a bit ago where there was someone who interviewed 20 or so people who were both engineers and programmers, and asked them what the two fields could learn from each other. One of the things it mentioned from the perspective of a physically-based engineer is that a little more up-front planning can make a big difference, and that's stuck with me ever since.

[1] (pretty sure this is the right one): https://youtu.be/CmIGPGPdxTI

But specs are per feature, it’s just an up front discussion first like you’d have on may things rather than questions-> immediate code writing from a model.

That's Event-B (minus the formal side) with LLM

I, for one, welcome the fact that agile/scrum/daily standup/etc. rituals will be outdated. While they might be somehow useful in some software development projects, in the past 10 years it turned out to be a cult of lunatics who want to apply it to any engineering work, not just software, and think any other approach than that will result in bad outcomes and less productivity. Can't wait for the "open office" BS to die next too, literally a boomer mindset that came from government offices back in the day, and they think it's more productive that way.

For myself, I found that having a work methodology similar to spec-driven development is much better than vibe coding. The agent makes less mistakes, it stays on the path and I have less issues to fix.

And while at it, I found out that using TDD also helps.