Peer Programming with LLMs, for Senior+ Engineers
25 comments
·May 24, 2025mattnewton
brumar
Very important comment. My workflow changed dramatically with the increased capabilities of these tools.
notepad0x90
is it really more efficient to have an LLM generate code, then review that code, fix errors and spend some time to fully understand it? I wish there were tangible stats and metrics around this. Is it really more efficient than just writing the code yourself, but using LLMs to look up things or demo solutions?
viraptor
Depends on the code, but often yes. The less you care about that specific result, the more efficient it is. One-off tools under 2k lines where you can easily verify the result? Why would I not generate that and save time for more interesting stuff?
tptacek
Yes, but the bar for skepticism is higher than that, because LLMs also compile code and catch errors, and generate and run tests; compile errors and assertion failures are just more prompts to an LLM agent.
what
Citation needed. Here is copilot “fixing” a failing test: https://github.com/dotnet/runtime/commit/fe173fc8f44dbd0a9ed...
It rewrote some comments, changed the test name and added extra assertions to the test. Baby sitting something like that seems like an absolute waste of time.
hoppp
I use the llm as a glorified search engine. Instead of googling I ask it stuff. Its fine for that but its a hit or miss. Often the output is garbage and its better to just use google.
I dont use it much to generate code, I ask it higher level questions more often. Like when I need a math formula.
SoftTalker
Though I haven't tried it, I would probably enjoy peer programming with an LLM more than I do with a real person (which I have tried and hated).
I could assign the LLM the simple drudgery that I don't really want to do, such as writing tests, without feeling bad about it.
I could tell the LLM "that's the stupidest fucking thing I've ever seen" whereas I would not say that to a real person.
pmbanugo
That’s what the recent Copilot feature on GitHub can do. You assign it tasks and it comes back with a PR. You could also assign it to review a PR.
gadflyinyoureye
It seems like we need to use forceful language with these things now. I've had copilot censor everything I asked it. Finally I had to to say, "listen you cracked up piece of shit, help me generate a uuid matcher. "
diggan
> Peer Programming with LLMs, For Senior+ Engineers
> [...] a collection of blog posts written by other senior or staff+ engineers exploring the use of LLM in their work
It seems to be by senior engineers if anything, I don't see anything in the linked articles indicating they're for senior engineers, seems programmers of all seniority could find them useful, if they find LLMs useful.
OutOfHere
Yes, although those who are not senior engineers will not preemptively see the value in the documented approaches. One has to be a senior to preemptively appreciate the value in them.
pmbanugo
I've been experimenting with LLMs for coding for the past year - some wins, plenty of frustrations. Instead of writing another "AI will change everything" post, I collected practical insights from other senior engineers who've figured out what actually works. No hype, just real experiences from people in the trenches.
jbellis
I would have said that Harper Reed's workflow (brainstorm spec, then co-plan a plan, then execute using LLM codegen) is basically best practice today and I'm surprised that the author adds that "I’ve not been successful using this technique to build a complete feature or prototype."
Here's an example of using this pattern with Brokk to solve a real world bug: https://www.youtube.com/watch?v=t_7MqowT638
cheshire_cat
This is showing the workflow of your tool quite well, but would be way more convincing & impressive if you had actually fixed the bug and linked to the merged PR.
dietr1ch
(Site is unreadable for me on Firefox 138, but the text is still there if you select all. Qutebrowser based on Chromium 130 doesn't render it either.)
vvillena
No problems here, both the normal view and reader mode seem to work well.
westurner
What are some of the differences between Peer Programming with LLMs and Vibe Coding?
diggan
> What are some of the differences between Peer Programming with LLMs and Vibe Coding?
"Vibe Coding" is specifically using the LLM instead of programming anything, barely caring about the output. If something is wrong, don't even open the file, just ask the LLM. Basically "prompting while blindfolded" I guess you could say.
Peer programming with an LLM would be to use it as another tool in the toolbox. You still own your program and your code. Edit away, let the LLM do some parts that are either too tricky, or too trite to implement, or anything in-between. Prompts usually are more specific, like "Seems X is broken, look into Y and figure out if Z could be the reason".
lowbloodsugar
I would say that the difference is taking an engineering approach to the process itself. Iterating on the context, putting the system into various states, etc. Treating the AI like a very knowledgeable intern who also has a very fixed short term memory and can’t form new long term memories but can be taught to write things down like in Memento. The thing is, though, it has a much much larger short term memory than me.
nickpsecurity
I want to note that the headlines gave me an idea for a nonprofit: "Peer Programming with LLM's for Seniors."
Somebody jump on that. It's yours. :)
pmbanugo
re-reading the title makes me feel like I used a wrong title.
Could be a good idea for a non-profit like you said. I know someone who’s exploring something similar but for disabled folks who aren’t tech-savvy (for-profit)
nickpsecurity
That's kind of them. I'll pray their effort succeeds.
The space is moving so fast that, if I wrote down my workflows and workarounds just two months ago, so much of it would be stale today. I think all these recommendations need to list the models and harnesses being described front and center.