Why agents are bad pair programmers

Just like with human engineers, you need to start with a planning session. This involves a back and forth discussion to hammer out the details before writing any code. I start off as vague as possible to see if the LLM recommends anything I hadn't thought of, then get more detailed as I go. When I'm satisfied, I have it create 2 documents, initialprompt.txt and TODO.md. The initial prompt file includes a summary of the project along with instructions to read the to do file and mark each step as complete after finishing it.

This ensures the LLM has a complete understanding of the overall goals, along with a step by step list of tasks to get there. It also allows me to quickly get the LLM back up to speed when I need to start a new conversation due to context limits.

This absolutely captures my experience.

My successful AI written projects are those where I care solely on the output and have little to no knowledge about the subject matter.

When I try to walk an agent through creating anything about which I have a deeply held opinion of what good looks like, I end up frustrated and abandoning the project.

I've enjoyed using roo code's architect function to document an agreed upon approach, then been delighted and frustrated in equal measure by the implementation of code mode.

On revelation is to always start new tasks and avoid continuing large conversations, because I would typically tackle any problem myself in smaller steps with verifiable outputs, whereas I tend to pose the entire problem space to the agent which it invariably fails at.

I've settled on spending time finding what works for me. Earlier today I took 30 minutes to add functionality to an app that would've taken me days to write. And what's more I only put 30 minutes into the diary for it, because I knew what I wanted and didn't care how it got there.

This leads me to conclude that using AI to write code that a(nother) human is one day to interact with is a no-go, for all the reasons listed.

Anthropic's guide to using Claude Code [1] is worth reading.

Specifically, their recommended workflow is "first ask it to read the code, then ask it to make a plan to implement your change, then tell it to execute". That sounds like the workflow you're asking for - you can read its plan and make adjustments before it writes a single line of code.

One of the weird things about using agents is that if they're doing things in a way you don't like, including things like writing code without first running the design by you, you can simply ask them to do things a different way.

[1] https://www.anthropic.com/engineering/claude-code-best-pract...

Yeah, and then you just wind up feeling exhausted AND unsatisfied with where you wind up. You are exactly who I posted this for.

100% of my positive experiences with agent coding are when I don't have reason to care about the intrinsic qualities of the code (one-off scripts or genuine leaf node functions that can't impact anything else.

Your ai is generating 2000 line code chunks? Are you prompting it to create the entire Skyrim game for SNES? Then after taking long lunch, getting mad when you press run and you find out it made fallout with only melee weapons in a ps1 style?

> It's like working with someone on their first day.

This matches my experience exactly, but worse than working with a human on their first day, day 100 for an AI is still like working with them on their first day. Humans have effectively infinite context windows over a long enough period of time, AIs context windows are so constrained that it's not worthwhile to invest the effort to 'teach' it like you would a junior engineer.

At the start of prompt before project requirements I copy paste paragraph about the code I want.

No emojis, no comments, no console log statements, no read me file, no error handling. Act as a senior developer working with other experienced developers.

Otherwise it happily generates bunch of trash that is unreadable. Error handling generated will most of the times just hide errors instead of actually dealing with them.

The trick is to have rules specific to the project and your programming style & preferences.

Think of the AI like an outsourced consultant who will never say no. It'll always do everything it can to solve the problem you've given it. If it doesn't know how, it'll write a thousand lines when a single additional dependency and 2 lines of code would've done it.

Hilarious but realistic take. I've noticed a similar trend with other posts. I'm a fan of the discourse either way tbh.

I liked your post, a bit of a “how to enjoy pair programming with an AI”. Useful, so thank you!

Have you tried recently? This hasn't been my experience. I modify the code it's written, then ask it to reread the file. It generally responds "I see you changed file and [something.]" Or when it makes a change, I tell it I need to run some tests. I provide feedback, explain the problem, and it iterates. This is with Zed and Claude Sonnet.

My approach has generally been to accept, refactor and reprompt if I need to tweak things.

Of course this does artificially inflate the "accept rate" which the AI companies use to claim that it's writing good code, rather than being a "sigh, I'll fix this myself" moment.

I usually add “discuss first. Don’t modify code yet”. Then we do some back and forth. And finally, “apply”.

In all honesty - have you tried doing what you would do with a paired programmer - that is, talk to them about it? Communicate? I’ve never had trouble getting cursor or copilot to chat with me about solutions first before making changes, and usually they’ll notice if I make my own changes and say “oh, I see you already added XYZ, I’ll go ahead and move on to the next part.”

You can totally do that. Just tell it to.

If you want an LLM to do something, you have to explain it. Keep a few prompt docs around to load every conversation.

I do this all the time with Claude Code. I’ll accept its changes, make adjustments, then tell it what I did and point to the files or tell it to look at the diff.

Pair programming requires communicating both ways. A human would also lose context if you silently changed their stuff.

Hmm you can tweak fine these days without messing up context. But, I run in “ask mode” only, with opus in claude code and o3 max in cursor. I specifically avoid agent mode because, like in the post, I feel like I gain less over time.

I infrequently tab complete. I type out 80-90% of what is suggested, with some modifications. It does help I can maintain 170 wpm indefinitely on the low-medium end.

Keeping up with the output isn’t much an issue at the moment given the limited typing speed of opus and o3 max. Having gained more familiarity with the workflow, the reading feels easier. Felt too fast at first for sure.

My hot take is that if GitHub copilot is your window into llms, you’re getting the motel experience.

I've asked for hints/snippets to give ideas and then implemented what I wanted myself (not commercially). Worked OK for me.

Second this. My solution is to have a 'non-AI' IDE and then a Cursor/VS Code to switch between. Deep work cannot be achieved by chatting with the coding bots, sorry.

I recently got a new laptop and had to setup my IDE again.

After a couple hours of coding something felt "weird" - turns out I forgot to login to GitHub Copilot and I was working without it the entire time. I felt a lot more proactive and confident as I wasn't waiting on the autocomplete.

Also, Cursor was exceptional at interrupting any kind of "flow" - who even wants their next cursor position predicted?

I'll probably keep Copilot disabled for now and stick to the agent-style tools like aider for boilerplate or redundant tasks.

AI "auto-complete" or "code suggestions" is the worst, especially if you are in a strongly-type language because it's 80% correct and competing with an IDE that can be 100% correct.

AI agents are much better for me because 1) they don't constantly interrupt your train of thought and 2) they can run compile, run tests, etc. to discover they are incorrect and fix it before handing the code back to you.

I love the autocomplete, honestly use it more than any other AI feature.

But I'm forced to write in Go which has a lot of boilerplate (and no, some kind of code library or whatever would not help... it's just easier to type at that point).

It's great because it helps with stuff that's too much of a hassle to talk to the AI for (just quicker to type).

I also read very fast so one line suggestions are just instant anyway (like non AI autocomplete), and longer ones I can see if it's close enough to what I was going to type anyway. And eventually it gets to the point where you just kinda know what it's going to do.

Not an amazing boost, but it does let me be lazy writing log messages and for loops and such. I think you do need to read it much faster than you can write it to be helpful though.

> Pair programming is also not suitable for all cases

I think this is true but pair programming can work for most circumstances.

The times where it doesn't work is usually because one or both parties are not all-in with the process. Either someone is skeptical about pair programming and thinks it never works or they're trying to enforce a strict interpretation of pair programming.

I’m a Vim user and couldn’t agree more.

Didn’t like any of the AI-IDEs, but loved using LLMs for spinning up one off solutions (copy/paste).

Not to be a fan boy, but Claude Code is my new LLM workflow. It’s tough trying to get it to do everything, but works really well with a targeted task on an existing code base.

Perfect harmony of a traditional code editor (Vim) with an LLM-enhanced workflow in my experience.

I’ve always seen it as primarily an _education_ tool; the purpose of pair programming isn’t that two people pair programming are more productive than two people working individually, they’re generally not. So pair programming with a magic robot seems rather futile; it’s not going to learn anything.

Zed has a "subtle" mode, hopefully that feature can become table stakes in all AI editor integrations

I expect a lot of the things people don't like ("output too long, too many comments in code") are side effects of making the LLM good in other areas.

Long output correlates with less laziness when writing code, and higher performance on benchmarks due to the monotone relationship between number of output tokens and scores. Comment spam correlates with better performance because it's locally-specific reasoning it can attend on when writing the next line of code, leading to reduced errors.

I was trying out sonnet 4 yesterday and it spent 15 minutes changing testing changing etc just to get one config item changed. It ended up changing 40 files for no reason. Also kept trying to open a debugger that didn’t exist and load a webpage that requires auth.

They’re far from perfect that’s for sure.

Yeah, I could completely see this. Reminds me of the "Slow Internet vs No Internet" oatmeal comic

> that I tune out

You need a 30L fishtank for your desk. Great for tuning out.

I basically only use it in the workplace, and largely because of one of those AI mandates.

I don't think it actually saves me enough time (or for many tasks, any time) so I wouldn't pay for it for my own projects, and also for my own projects, the enjoyability is a big factor, and I enjoy doing more than prompting.

No, I don't. This goes for internal projects as well, we're not going to share code unless payed to do so.

We commonly work with personal information so it would also introduce rather harsh legal risks if usian corporations could reach it.

I have a client that actively asks me to use AI more and more. They expect to get better quality code faster, ie. to reduce costs. (That's not my experience but that's beside the point).

I don't share anything with openai/anthropic that I wouldn't feel comfortable pasting into a web search prompt.

Most stacks now support some form of "plan" workflows. You'd want to first do this, and see if it improves your experience.

One workflow that works well for me, even with small local models, is to start a plan session with something like: "based on @file, and @docs and @examples, I'd like to _ in @path with the following requirements @module_requirements.md. Let's talk through this and make sure we have all the info before starting to code it."

Then go back and forth, make sure everything is mentioned, and when satisfied either put it into a .md file (so you can retry the coding flow later) or just say "ok do it", and go grab a cup of coffee or something.

You can also make this into a workflow with .rules files or .md files, have a snippets thing from your IDE drop this whenever you start a new task, and so on. The idea with all the advancements in LLMs is that they need lots of context if you want them to be anything other than what they were trained on. And you need to try different flows and see what works on your specific codebase. Something that works for projectA might not work for projectB ootb.

Also giving them more details seems to confuse them. There is probably a way around this, though. They are pretty good in finding a tiny silver of information out of the ocean. What I hate is that the industry is all geared toward the same model (chat bot). Imagine if we never invented the keyboard, mouse, GUI, touch screen, etc...

[dead]

> I use an LLM as a reference (on-demand), and don't use agents (yet)

This was me until about three weeks ago. Then, during a week of holiday, I decided I didn't want to get left behind and tried a few side-projects using agents -- specifically I've been using Roo. Now I use agents when appropriate, which I'd guess is about 50% of the work I'm doing.

I think coding will eventually go away in favor of models with metadata built around them.

How many times did you have a mutation operation where you had to hand code the insert of 3 or 4 entities and make sure they all come back successful, or you back out properly (and perhaps this is without a transaction, perhaps over multiple databases).

Make sure the required fields are present Grab the created inserted ID Rinse, repeat

Or if you're mutating a list, writing code that inserts a new element, but you don't know which one is new. And you end up, again, hand coding loops and checking what you remember to check.

What about when you need to do an auth check.

And the hand coder may fail to remember one little thing somewhere.

With LLM code, you can just describe that function and it will remember to do all the things.

An LLM with a model + metadata - we won't really need to think of it as editing User.java or User.py anymore. Instead User.yaml - and the LLM will just consume that, and build out ALL of your required biz-logic, and be done with it. It could create a fully authenticating/authorizing REST API + GraphQL API with sane defaults - and consistent notions throughout.

And moving into UIs- we can have the same thing. The UI can be described in an organized way. What fields are required for user registration. What fields are optional according to the backend. It's hard to visualize this future, but I think it's a no-code future. It's models of requirements instead.