Vibe Debugging: Enterprises' Up and Coming Nightmare

It’s based on the same category error that says I can buy a company but lose the employees and still have the value.

You don’t. You either don’t get that, or you do but would rather people not know that you really just wanted to destroy a competitor and snake some of their customers.

queue the South Park "They took meh jeb!"

I always love to compare Peter Norvig’s sudoku solver implementation [1] versus Ron Jeffries TDD approach [2].

Norvig starts with the theory building and creates a constraint solver in about 50 lines of code. Jeffries starts with TDD, assumes an implementation, has to change that implementation, therefore has to change the tests, and after a series of five blog posts kind of fizzles out on it.

To me it just highlights that defining the problem is with so much more than defining the tests, as you can’t write a test for a problem you haven’t defined yet. In this way the tests are an imposition. In short to me it shows that TDD only really works if you already knew how to buold the project to begin with.

[1] https://norvig.com/sudoku.html [2] https://ronjeffries.com/xprog/articles/oksudoku/

The proof for TDD is usually looking at bug detection rates. Similar for code review. OTOH, the "design damage" of TDD is something overlooked by those metrics.

What it boils down to: - TDD in the hands of a junior is very good. Drastically reduces bugs, and teaches the junior how to write code that can be tested and is not just a big long single method of spaghetti with every data structure represented as another dimension on some array.

- TDD in the hands of a midlevel can be a mixed bag. They've learned how to do TDD well, but have not learned when and why TDD can go bad. This creates design damage, where everything is shoe-horned into TDD and the goal of 90% line coverage is a real consideration. This is maximum correctness but also potentially maximum design damage.

- TDD in the hands of a senior is a power tool. The "right" tests are written for the right reasons with the right level of coupling and the tests overall are useful. Every really complicated algorithm I've had to write, TDD was a life saver for getting it landed.

Feels a lot like asking someone if they prefer X or Y and they say "X" is the industry best practice. My response universally is now an eye brow raise "oh, is it? For which segments of the industry? Why? How do we know it's actually a best practice? Okay, given our context, why would it be a best practice for US". Juniors don't know the best practices, mid-levels apply them everywhere, seniors evaluate and consider when best practices are not best practices.

TDD slows development when tests are written in a blind way with an eye on code coverage and not correctness and design. TDD speeds up development in being a good way to catch errors and is one of the best ways to ensure correctness.

TDD works if know what 100% of the inputs/outputs look like, AND you know the implementation will be complex, terse, whatever. Write tests, then twiddle a bunch of regex until the tests pass. This situation rarely comes up in real software jobs, but when it does, yeah use it!

I think it works only for people with a certain thought pattern, the ones that like to think everything up-front before doing anything.

Most people prefer to play around and make several crappy attempts and combine them until the whole is somewhat solved, then go over and polish it a little, and maybe then add tests and fix the behavior in place.

For this last group, TDD it's jarring, unnatural and requires a lot of willpower to follow.

It's not bad in itself, it's just not for everyone.

TDD doesn't work for the same reasons why Todo lists don't work. I don't have a good explanation why that is so, but I know from countless examples (mine and other people's) that they don't work.

In general, doing things work, planning to do things don't.

So, there you have it. TDD is good if applied correctly, and only if you apply it 100% correct. And so it seems for LLM usage. If it doesn't work for you, then you are obviously doing it wrong according to many folks here. TDD is nice to catch refactoring mistakes, LLMs are nice to maybe do some initial refactoring on a small enough code base. And it doesn't mean that one precludes the other. But I haven't seen TDD put engineers out of work and neither should LLMs. Trust either model fully and you are in for a world of hurt.

I see TDD exactly as the best practice for vibe coding! Context is stored as test coverage, to make sure they don't break things when they hallucinate.

YMMV though.

Who did you hear the it slowed development from?

My personal experience (and I think the experience of many who do it full time) is that it makes things faster.

I think employees (myself included) often think, "I'm more productive" when in reality what they are actually experiencing is, "My job is now easier."

Easy does not necessarily mean more productive when you're trading ease for something else. In the case of coding, you're trading ease for things like understanding and maintainability.

The recent RCT on open source ai bug fixing had most participants feeling like they were more productive but actually being less productive. It may just be perception error, and then the issue you identify makes it even worse.

Right, a machine that automates the planting of mines may make the user more productive at their job, but that doesn't really account for the time spent cleaning them up on the backend and how many limbs will be lost in the process. AI is an automated landmine planting machine.

Sure AI increases developer output, which is sometimes correlated with productivity -- but often times not. Insofar as AI is accelerating positive outcomes (we were able to write some tricky code that was blocking us), it's also accelerating the negative outcomes (we used the LLM to write 40k lines of code in an hour and no one know what any of it does). One of these things is "productive" the other is just performative work.

If "being more productive" is using an ai to write an email which is then summarized by AI on the receiving end, or students using AI to write papers which are graded by AI, or developers using AI to write code which is then reviewed by AI, then AI isn't actually making anything better.

AI search makes me personally way more productive, e.g.: write snippet how to do X using library Y I never touched before.

Hah! Especially seeing the same articles, and pro/con discussions feels like a form of the preceding lines: _"There's only one subject on HN for Christ's sake. _Tips For Agentic AI Use_,_Gemini CLI_, _Building X with Claude_, they're the same thing!"_

Sure. But, and I will die on this hill, it's not vibe coding if you're looking at and understanding the code.

Let's see how many forget how to write and think through their own code while primarily vibe coding.

That was the theory, but it ran straight into the reality that most of the time “throwaway prototyping code” gets shipped straight to production and never updated, because it’s addressing real needs right now and there’s no time to fix it because a dozen other feature requests and support issues have arrived in the meantime.

The quip that “there’s nothing more permanent than a temporary solution” has been a truism of software engineering since long before AI arrived on the scene, vibe coding is just making the problem much worse.

There’s nothing more permanent than a throwaway prototype.

Nice in theory, but I once made a throwaway prototype which the client used as the foundation for their mobile app. It was translated into 26 languages and deployed across the globe. I had to work with their mobile engineers who asked why some of the views were so knotty and complex. They were complex because I was quickly patching and tweaking things between user interviews until the end of the project! The code was a mess.

Once the business sees that the prototype more or less works, it's incredibly difficult to get them to spend money on a "sane" clean-sheet rewrite.