The copilot delusion
153 comments
·May 23, 2025padolsey
> if you want to sculpt the kind of software that gets embedded in pacemakers and missile guidance systems and M1 tanks—you better throw that bot out the airlock and learn.
But the bulk of us aren't doing that... We're making CRUD apps for endless incoming streams of near identical user needs, just with slightly different integrations, schemas, and lipstick.
Let's be honest. For most software there is nothing new under the sun. It's been seen before thousands of times, and so why not recall and use those old nuggets? For me coding agents are just code-reuse on steroids.
Ps. Ironically, the article feels AI generated.
queenkjuul
Fundamentally, mission critical low level code isn't the kind of software i want to write anyway. I don't find AI tools super useful for most of the same reasons as the author, but i do kind of get tired of the idea that if you're not writing systems in C you're not really programming.
I like writing front end code. I'm probably never going to have a job where i need or would even want to write a low level graphics library from scratch. Fine, I'm not red-eyed 3am hacker brained, but I'm passionate and good at what i do. I don't think a world where every person working in software has the author's mentality is realistic or desirable.
throwaway314155
I don't mind your rebuttal of the article, but to suggest that this particular article is AI generated is foolish. The style the author presents is vivid, uses powerful imagery and metaphor and finally, at times, is genuinely funny. More qualitatively, the author incorporates a unique identity that persists throughout the entirety of a long form essay.
All of that is still difficult to get an LLM to do. This isn't AI generated. It's just good writing. Whether you buy the premise or not.
chrismorgan
Yeah, it feels a very different style of unhinged to LLMs. I can’t yet imagine an LLM producing such a beautiful and contextually-appropriate sentence as “They’ll be out there trying to duct-tape horses to an engine block, wondering why it doesn’t fly.”
nick3443
This brought tears to my eyes:
But you—at your most frazzled, sleep-deprived, raccoon-eyed best—you can try. You can squint at the layers of abstraction and see through them. Peel back the nice ergonomic type-safe, pure, lazy, immutable syntactic sugar and imagine the mess of assembly the compiler pukes up.
Amazing
queenkjuul
I got slightly LLM vibes for the first few paragraphs ngl. It became very clear it wasn't very fast, though.
boxed
Maybe he meant it was long. Some people seem to think that long walls of text is how you spot AI slop.
nozzlegear
And god forbid you use an emdash these days.
anon7000
Yeah, and the article talks about those ways in which AI is useful. Overall, the author doesn’t have a problem with experts using AI to help them. The main argument is that we’re calling AI a copilot, and many newbies may be trusting it or leaning on it too much, when in reality, it’s still a shitty coworker half the time. Real copilots are actually your peers and experts at what they do.
> Now? We’re building a world where that curiosity gets lobotomized at the door. Some poor bastard—born to be great—is going to get told to "review this AI-generated patchset" for eight hours a day, until all that wonder calcifies into apathy. The terminal will become a spreadsheet. The debugger a coffin.
On the other hand, one could argue that AI is just another abstraction. After all, some folks may complain that over-reliance on garbage collectors means that newbies never learn how to properly manage memory. While memory management is useful knowledge for most programmers, it rarely practically comes up for many modern professional tasks. That said, at least knowing about it means you have a deeper level of understanding and mastery of programming. Over time, all those small, rare details add up, and you may become an expert.
I think AI is in a different class because it’s an extremely leaky abstraction.
We use many abstractions every day. A web developer really doesn’t need to know how deeper levels of the stack work — the abstractions are very strong. Sure, you’ll want to know about networking and how browsers work to operate at a very high level, but you can absolutely write very nice, scalable websites and products with more limited knowledge. The key thing is that you know what you’re building on, and you know where to go learn about things if you need to. (Kind of like how a web developer should know the fundamental basics of HTML/CSS/JS before really using a web framework. And that doesn’t take much effort.)
AI is different — you can potentially get away with not knowing the fundamental basics of programming… to a point. You can get away with not knowing where to look for answers and how to learn. After all, AIs would be fucking great at completing basic programming assignments at the college level.
But at some point, the abstraction gets very leaky. Your code will break in unexpected ways. And the core worry for many is that fewer and fewer new developers will be learning the debugging, thinking, and self-learning skills which are honestly CRITICAL to becoming an expert in this field.
You get skills like that by doing things yourself and banging your head against the wall and trying again until it works, and by being exposed to a wide variety of projects and challenges. Honestly, that’s just how learning works — repetition and practice!
But if we’re abstracting away the very act of learning, it is fair to wonder how much that will hurt the long-term skills of many developers.
Of course, I’m not saying AI causes everyone to become clueless. There are still smart, driven people who will pick up core skills along the way. But it seems pretty plausible that the % of people who do that will decrease. You don’t get those skills unless you’re challenged, and with AI, those beginner level “learn how to program” challenges become trivial. Which means people will have to challenge themselves.
And ultimately, the abstraction is just leaky. AI might look like it solves your problems for you to a novice, but once you see through the mirage, you realize that you cannot abstract away your core programming & debugging skills. You actually have to rely on those skills to fix the issues AI creates for you — so you better be learning them along the way!!
Btw, I say this as someone who does use AI coding assistants. I don’t think it’s all bad or all good. But we can’t just wave away the downsides just because it’s useful
jgraettinger1
> On the other hand, one could argue that AI is just another abstraction
I, as a user of a library abstraction, get a well defined boundary and interface contract — plus assurance it’s been put through paces by others. I can be pretty confident it will honor that contract, freeing me up to not have to know the details myself or second guess the author.
seanmcdirmid
> Btw, I say this as someone who does use AI coding assistants. I don’t think it’s all bad or all good. But we can’t just wave away the downsides just because it’s useful
Isn't this just the rehashed argument against interactive terminals in the 60s/70s (no longer need to think very carefully about what you enter into your punch cards!), debuggers (no longer spending time looking carefully at code to find bugs), Intellisense/code completion (no need to remember APIs!) from the late 90s, or stackoverflow (no need to sift to answer questions that others have had before!) from the 00s? I feel like we've been here before and moved on from it (hardly anyone complains about these anymore, no one is suggesting we go back to programming by rewiring the computer), I wonder if this time it will be any different? Kids will just learn new ways of doing things on top of the new abstractions just like they've done for the last 70 years of programming history.
sgarland
Interactive terminals didn’t write code for you, and also unlocked entirely new paradigms of programs. Debuggers, if anything, enabled deeper understanding. Intellisense is in fact a plague and should not exist. Stack Overflow, when abused, is nearly as bad as AI.
ivape
It’s funny because I made a few funny clips (to my taste) on Google Whisk and figured, hey why not, let’s make a TikTok. Did you know that all of TikTok is full of millions of ai generated stuff or other people just copying each others stuff? I really thought there was something to this “original creation” stuff.
We are all so simply reproducible. No one’s making anything special, anywhere, for the most part. If we all uploaded a TikTok video of daily coding, it would be the same fucking app over and over, just like the rest of TikTok.
Elon may have be right all along, there’s literally nothing left to do but goto Mars. Some of us were telling many of you that the LLMs don’t hallucinate as much as you think just two years ago, and I think the late to the party crowd need to hear us again - we humans are not really necessary anymore.
!RemindMe in 2 years
whattheheckheck
There's literally genocide and war going on, solve that
queenkjuul
Don't encourage them. They'll just build an AI to do the genocide faster.
felubra
> The machine is real. The silicon is real. The DRAM, the L1, the false sharing, the branch predictor flipping a coin—it’s all real. And if you care, you can work with it.
This is one of the most beautiful pieces of writing I’ve come across in a while.
bwfan123
Same, The author writes like Dave Barry. I burst out laughing more than once. He was able to articulate with a lot of humor exactly what I think of co-pilot.
dlnovell
Beautiful and witty prose to say "vibe coding sucks". He's not at all wrong about the state of AI coding in May of 2025. The 3 hours I just burned trying to get it to correct output bugs in a marimo notebook (which I started learning this week) is demonstrable evidence.
But it completely ignores the fact that AI generated code is getting better on a ~weekly basis. The author acknowledges that it is useful in some contexts for some uses, but doesn't acknowledge that the utility is constantly growing. We certainly could plateau sometime soon leaving us in the reckless intern zone, but I wouldn't bet on it.
lunarcave
The things that's most often missed in these discussions that "writing code" is the end artefact. It doesn't take into account the endless tradeoffs made in producing the said artefact - the journey to get there.
Just try implementing a feature with a junior, in a mildly complex codebase and you'd catch all the unconscious tradeoffs that you're making as an experienced developer. AI has some concept of what these tradeoffs are, but that's mostly by observation.
AI _does_ help with writing code. Keyword there being - "help".
But thinking is the human's job. LLMs can't/don't "think". Thinking how to get the AI to produce the output you want is also your job. You'd think less and less if models get better.
CGamesPlay
This resonates with me, for sure; both the benefits and the drawbacks of copilot. But while I think kids and hackers were artisans, engineers were always just engineers. The amazing technical challenges they had to solve to create some of the foundational technologies we have today, exist because they had to solve those challenges. Looking at only these and saying "that's how things used to be" is survivorship bias.
gleenn
I feel privileged to be able to say that as a software engineer who's been doing it the hard way for 20+ years, I relish the hard problems. The CRUD app updates are unbearable without the random in-between challenges that bend my mind. The rare recursive algorithm, the application of some esoteric knowledge I actually learned in college, actually having to do big-o estimates. These are the gems of my career that keep me sane. I hope the next flock of AI-driven SWEs appreciates these things even more given the AI can spout off answers which sometimes are right and sometimes are horribly wrong. Challenges like these will always have to have someone who actually knows what to do when the AIs start hallucinating or the context of the situation is beyond the context window.
Jcampuzano2
As a preface, I think lots of people will not like this take.
A lot of people are going to have to come to the realization that has already been mentioned before but many find it hard to grasp.
Your boss, stakeholders, and especially non-technical people literally give 0 fucks about "quality code" as long as it does what they want it to do. They do not care about tests insofar as if it works it works. Many have no clue about nor do they care about whether something just refetches the world in certain scenarios. And AI whether we like it or not, whether it repeats the same shit and isnt DRY, doesn't follow patterns, reinvents the wheel, etc - is already fairly good at that.
This is exactly why all your stakeholders and executives are pushing you to use it. they've been fed that it just gets shit done and pumps out code like nothing else.
I really think a lot of the reason some people say it doesn't give them as much productivity as they would like is due largely to a desire to write "clean" code based on years and years of our own training, and due to having to be able to pass code review done by your peers. If these obstacles were entirely removed and we went full bandaid off I do think AI even in its current state is fairly capable of replacing plenty of roles. But it does require a competent person to steer to not end up in a complete mess.
If you throw away the guardrails a little bit and not obsess about how nice code looks anymore, it absolutely will move things along faster than you could before.
allthenopes25
A sane person writes clean code because they are going to have to maintain it themselves one day a few years into the future when the baby kept them up all night for three nights in a row and coffee isn't working for them anymore and they can't remember anything about it and it's falling over so it's really urgent and f**** those guys they swore they'd get someone to take this on with a proper handover and surely there was a goddamn sent email somewhere about it but nothing is coming up when you search and goddamnit it used to compile did people ignore your comment about how it won't build with the new version yet so don't update the build tools and and and
You write good code because you own it.
If you get ChatGPT or Copilot or Claude or whateverthe****else to write it, you're going to have a whole lot less fun when it's on fire.
The level of irresponsibility that "vibe coding" is introducing to the world is actually worse than the one that had people pouring their savings into a shitcoin. But it's the same arseholes talking it up.
bravetraveler
I suspect experienced folks are well aware of the subjectivity of quality. How? Change control.
Quality control exists until The Business deems otherwise. The reasons vary: vulnerability, promotion, whatever. Usually not my place to say.
Personally, my 'product' isn't code. Even the 'code' isn't code. For every 8 hours of meetings I do, I crank out maybe 20 lines of YAML (Ansible). Then, another 4 hours of meetings handing that out/explaining the same basics for Political Points.
The problem(s) relating to speed or job security have remarkably little to do with code; generated or not. The people I work with are functionally co-dependent because they don't use LLMs or... manuals.
All this talk about "left behind"... to survive a bear, one doesn't have to be the fastest. Just not the slowest.
lukan
I lost all my snobism about that years ago and I do just follow the paradigm, "does it work". But for some reasons, even with AI I ain't on another level.
And those reasons are, it all collapses very quickly once the complexity reaches an medium amount.
And if I want to rely on things and debug them - I cannot just have a pile of generated garbage, that works as long as the sun is shining. For isolated tasks it works for me. For anything complex, I am faster on my own.
Jcampuzano2
Most replies here make the claim that all AI generated code is "garbage". And I can't help but think most of the people who say that do not actually use it in their day to day with the most recent models and actually give it good instructions/requirements.
No, it is not always perfect. Yes you will have to manually edit some of the code it generates. But yes it can and will generate good code if you know how to use it and use sophisticated tools with good guidance. And there are times where it will even write better more performant code than you could given the time requirements.
mplanchard
It writes code well enough in simple contexts, sometimes. But that code is also easy to write, indeed often easier to write than to review. It struggles in more complex contexts and with more complex constraints. Unfortunately, it’s the latter case where I most often have any desire to reach for an aid, and it has failed so consistently and so often there that I have largely stopped trying.
It’s nice when you need to do something simple in an unfamiliar but simple context, though.
It seems though that a lot of the narrative here from its proponents is that we’re just not trying hard enough to get it to solve our problems. It’s like vimmers who won’t shut up about how it’s worth the weeks of cratered productivity in order to reach editing nirvana (I say this as one of them).
Like with any tool, the learning curve has to be justified by the results, but the calculation is further complicated by the fact that the AI tooling landscape changes completely every 3-6 months. Do I want to spend all that time getting good at it now? No. I’ll probably spend more time learning to use it when it’s either easier to get results that actually feel useful or when it stops changing so often.
Until then I’ll keep firing it up every once in a while to have it write some bash or try to get it to write a unit test.
lyu07282
I think the root of the argument is that the AI critics are worried because they have the assumption that 1) you aren't experienced enough to know what "good code" looks like, and/or 2) you only care if it "works" and don't understand all the implications bad code will have downstream, again because of inexperience.
edoceo
This is true. But the cost after is high. Creating code, by hand or AI is easier than maintaining or modifying the system.
And this is where a problem (still) appears - except now the AI-assiated authors have even less comprehension of the system.
lamename
They will demand use of AI tools for "productivity" and then complain when there are bugs in prod without realizing the root cause.
Jcampuzano2
They won't blame the AI as the root cause. They will blame you.
This is why I mention you need to be competent enough to understand what is being generated or they will find someone else who does. There's no 2 ways around it. AI is here to stay.
lamename
Yes I agree. They will and should blame the human. That's a problem when the human isnt given enough time to complete projects because "AI is SO productive"
yencabulator
The idea that you will actually really review all that vibecoded slop code comes across as very naive.
The real question is how many companies have to accidentally expose their databases, suffer business-ruining data losses, and have downtime they are utterly unable to recover from quickly before CxOs start adjusting their opinions?
Last time I saw a "Show HN" of someone showing off their vibecoded project, it leaked their OpenAI API key to all users. If that's how you want to run your business, go right ahead.
drekipus
> you need to be competent enough to understand what is being generated
We're all competent enough to understand what is generated. That's why everyone is doomer about it.
What insights do you have above us when the LLM generates
true="false"
while i < 10 {
i++
}
Frankly. Businesses that use this, drop all their actual engineers, and then fall over when the slightest breeze comes.
I am actually in favour, in a accelerationist sense.
null
joks
And then you have 0 domain experts because nobody has built the mental model of the code and what it's doing that you inherently build when you're actually doing the problem solving and code-writing yourself.
charles_f
As long as I am going to be the one being called at night when there's a crash, I'll be the one to dictate what's "good enough". Throw away guardrails if you want to, I like sleeping.
jboggan
This is the crux of the piece to me:
"We'll enshrine this current bloated, sluggish, over-abstracted hellscape as the pinnacle of software—and the idea of squeezing every last drop of performance out of a system, or building something lean and wild and precise, will sound like folklore."
This somewhat lines up with my concerns about libraries and patterns before 2023 getting frozen in stone once we pass over the event horizon where most new code to train on is generated by LLMs. We aren't innovating, we are going to forever reinforce the screwed up dependency stack and terrible kludges of the last 30 years of development. Javascript is going to live forever.
malfist
I feel this in my bones. Every day I'm getting challenged by leadership that we're not using AI enough, told that I should halve my estimates because "we'll use AI", and being told that there's a new AI tool that I have to adopt because someone is tracking KPIs related to adoption and if our team doesn't adopt enough AI tools we're going to be fired to give more headcount to those that do.
It's like the world has lost it's goddamn mind.
AI is always being touted as the tool to replace the other guy's job. But in reality it only appears to do a good job because you don't understand the other guy's job.
Management has an AI shaped hammer and they're hitting everything to see if it's a nail.
sanderjd
Yeah, I mean, this is just the current phase of the hype cycle. It'll settle down. Some of the tools and techniques will have staying power, most won't. If you can figure out which is which and influence others, you'll be in good shape.
bluefirebrand
> Management has an AI shaped hammer and they're hitting everything to see if it's a nail.
I really think we need to figure out how to cut back on management so we can get back to the business of actually doing work
ghaff
Coordinating teams, talking to stakeholders/customers (including spending a lot of time with them), having someone manage individual contributors at some level, etc. is work that can't just be ignored at a company of any size. The only way to avoid (a lot of) it is to be very small and that has its own set of issues.
lyu07282
I mean don't bite the boot that you can lick amirite
cjbgkagh
Well how hard would it be to replace management with AI? Perhaps a developer could use AI to recreate the other tasks of the company without all of the overhead of actual people.
queenkjuul
Yeah i can't wait to discuss product with a sycophantic chatbot instead of the people who actually have a stake in the product.
Management can and usually does suck but i can reason with a person, for now. And sadly only the product people actually know what they want, usually right when you've built it the way they used to want it lol.
toomuchtodo
Unions. What other way is there?
https://www.epi.org/blog/americans-favor-labor-unions-over-b...
yencabulator
Clearly the answer is to replace them with AI.
nyarlathotep_
> I feel this in my bones. Every day I'm getting challenged by leadership that we're not using AI enough, told that I should halve my estimates because "we'll use AI", and being told that there's a new AI tool that I have to adopt because someone is tracking KPIs related to adoption and if our team doesn't adopt enough AI tools we're going to be fired to give more headcount to those that do.
This--all of this--seems exactly antithetical to computing/development/design/"engineering"/architecture/whatever-the-hell people call this profession as I understood it.
Typically, I labored under the delusion that competent technical decision makers would integrate tooling or choose to use a language, "service", platform, whatever, if they saw benefits and if they could make a "case" for why something was the correct approach, i.e how it met some product's needs, addressed some shortcomings, made things more efficient.
Like "here's my design doc, I chose $THING for caching for $REASON and $DATASTORE as it offers blah blah"
"Please provide feedback and questions"
This is totally alien to that approach.
Ideally, "hey we're going to use CoPilot/other LLM thingy, let us know if it aids your workflow, give us some report in a month and we'll go from there to determine if we want to keep paying for it"
lamename
> AI is always being touted as the tool to replace the other guy's job. But in reality it only appears to do a good job because you don't understand the other guy's job.
This is a well considered point that not enough of us admit. Yes many jobs are rote or repetitive, but many more jobs, of all flavors, done well have subtleties that will be lost when things are automated. And no I do not think that some "80% done by AI is good enough" because errors propagate through a system (even if that system is a company or society), AND the people evaluating that "good enough" are not necessarily going to be those experienced in that same domain.
ivape
But, management is the one to go soon. The other shoe is going to drop dear brother, this I promise you. Stay strong.
abletonlive
Well when you have a hammer big enough everything is indeed a nail.
Have you considered that instead of resisting you should do more to figure out why you're not getting the results that a lot of us are talking about? If nothing has changed for you in the past 2 years in your productivity the problem is most likely you. Don't you think it's your responsibility as an engineer to figure out what you're doing wrong when there are a lot of people telling you that it's a life changing tool? Or did you just assume that everybody was lying and you were doing everything correctly?
Sorry to say it. It's an unpopular opinion but I think it's pretty much a harsh truth.
dpistole
> why you're not getting the results that a lot of us are talking about?
IMO the problem occurs when "the results" are hyped up linkedIn posts not based in reality, AI is a boon but it's not lived up to the "IDEs are a thing of the past, youre all prompt engineers now" expectations that we hear from executives
MeetingsBrowser
Kind of brutal, but if LLMs drastically improved your productivity I think it speaks more to your baseline productivity than the power of LLMs.
abletonlive
What's more likely
A) all of this money being funneled into tech to build out trillions of dollars worth of infrastructure, a month over month increasing user base buying subscriptions for these llm services, every company buying seats for LLM because of the value that it provides - these people are wrong
B) yappers on hackernews that claim they derive no productivity boost out of llms while showing absolutely nothing about their workflow or method when the interface is basically a chat box with no guardrails - these people are wrong
Sorry I'm going to be it's B and you just suck at it
sanderjd
People keep saying this kind of thing, but sorry, it's nonsense.
Many of my colleagues that I most admire are benefiting greatly and increasingly from LLM tooling.
oh_my_goodness
"Well when you have a hammer big enough everything is indeed a nail."
I think this pretty much speaks for itself.
malfist
Where did you see that I didn't use AI and that _nothing_ has changed for me?
heddycrow
I couldn't help but read parts of this in Bertram Gilfoyle's voice.
Someone tell me I'm not alone.
swyx
i think the key is always having the ability to telescope - think coding agents enable you to stay high level, but you always need the ability to go down and fix/understand code when needed.
webprofusion
It's right, AI does require giving up control and letting things be done differently, depends how much you really use it.
It's the same when you get a junior dev to work on things, it's just not how you would do it yourself and frequently wrong or naive. Sometime is brilliant and better than you would have done yourself.
That doesn't mean don't have junior devs, but having one doesn't mean you don't have to do corrective stuff and refinements to their work.
Most of us aren't changing the world with our code, we're contributing an incredibly small niche part of how it works. People (normal people, lol) only care what your system does for them, not how it works or how great the code is.
The author is clearly a C++ programmer. I've been noticing that these AI tools are worse at C++ than other languages, especially scripting languages. Whenever I try to learn from people that are using these tools successfully, they always seem to be using a scripting language and working on some CRUD app.