Coding with LLMs in the summer of 2025 – an update
169 comments
·July 20, 2025dakiol
> Gemini 2.5 PRO | Claude Opus 4
Whether it's vibe coding, agentic coding, or copy pasting from the web interface to your editor, it's still sad to see the normalization of private (i.e., paid) LLM models. I like the progress that LLMs introduce and I see them as a powerful tool, but I cannot understand how programmers (whether complete nobodies or popular figures) dont mind adding a strong dependency on a third party in order to keep programming. Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools. I am afraid that in a few years, that will no longer be possible (as in most programmers will be so tied to a paid LLM, that not using them would be like not using an IDE or vim nowadays), since everyone is using private LLMs. The excuse "but you earn six figures, what' $200/month to you?" doesn't really capture the issue here.
simonw
The models I can run locally aren't as good yet, and are way more expensive to operate.
Once it becomes economical to run a Claude 4 class model locally you'll see a lot more people doing that.
The closest you can get right not light be Kimi K2 on a pair of 512GB Mac Studios, at a cost of about $20,000.
Fervicus
> I cannot understand how programmers don't mind adding a strong dependency on a third party in order to keep programming
And how they don't mind freely opening up their codebase to these bigtech companies.
ozgung
> The excuse "but you earn six figures, what' $200/month to you?" doesn't really capture the issue here.
Just like every other subscription model, including the one in the Black Mirror episode, Common People. The value is too good to be true for the price at the beginning. But you become their prisoner in the long run, with increasing prices and degrading quality.
lencastre
Can you expand on your argument?
nico
Currently in the front page of HN: https://news.ycombinator.com/item?id=44622953
It isn’t specific to software/subscriptions but there are plenty of examples of quality degradation in the comments
x______________
Not op but a something from a few days ago that might be interesting for you:
259. Anthropic tightens usage limits for Claude Code without telling users (techcrunch.com) 395 points by mfiguiere 2 days ago | hide | 249 comments
signa11
enshittification/vendor-lockin/stickiness/… take your pick
jordanbeiber
The argument is perhaps ”enshittification”, and that becoming reliant on a specific provider or even set of providers for ”important thing” will become problematic over time.
positron26
If the models are getting cheaper, better, and freer even when we use paid ones, then right now is the time to develop techniques, user interfaces, and workflows that become the inspirations and foundations of a future world of small, local, and phenomenally powerful models that have online learning, that can formalize their reason, that can bake deduction into their own weights and code.
muglug
> Programming used to be (and still is, to a large extent) an activity that can be done with open and free tools.
Yet JetBrains has been a business longer than some of my colleagues have been alive, and Microsoft’s Visual Basic/C++/Studio made writing software for Windows much easier, and did not come cheap.
dakiol
I see a big difference: I do use Jetbrains IDEs (they are nice), but I can switch to vim (or vscode) any time if I need to (e.g., let's say Jetbrains increase their price to a point that doesn't make sense, or perhaps they introduce a pervasive feature that cannot be disabled). The problem with paid LLMs is that one cannot easily switch to open-source ones (because they are not as good as the paid ones). So, it's a dependency that cannot be avoided, and that's imho something that shouldn't be overlooked.
layer8
> because they are not as good as the paid ones
The alternative is to restrict yourself to “not as good” ones already now.
rolisz
I was a hardcore vim user 10 years ago, but now I just use PyCharm to work. I'm paid to solve problems, not to futz around with vim configs.
Can you make vim work roughly the same way? Probably you can get pretty close. But how many hours do I have to sink into the config? A lot. And suddenly the PyCharm license is cheap.
And it's exactly the same thing with LLMs. You want hand crafted beautiful code, untainted by AI? You can still do that. But I'm paid to solve problems. I can solve them faster/solve more of them? I get more money.
throwaway8879p
People who understand the importance of this choice but still opt for closed source software are the worst of the worst.
You won’t be able to switch to a meaningful vim if you channel your support to closed source software, not for long.
Best to put money where the mouth is.
eevmanu
Open-weight and open-source LLMs are improving as well. While there will likely always be a gap between closed, proprietary models and open models, at the current pace the capabilities of open models could match today’s closed models within months.
wepple
Anyone can switch from Claude to llama?
ta12653421
Ah, there are community editions of the most imiportant tools (since 10+ years), and i doubt e.g. MS will close VS.NET Community Version in the future.
antirez
It’s not that bad: K2 and DeepSeek R1 are at the level of frontier models of one year ago (K2 may be even better: I have enough experience only with V3/R1). We will see more coming since LLMs are incredibly costly to train but very simple in their essence (it’s like if their fundamental mechanic is built in the physical nature of the computation itself) so the barrier to entry is large but not insurmountable.
bgwalter
Yes, and what is worse is that the same mega-corporations who have been ostensibly promoting equity until 2025 are now pushing for a gated development environment that costs the same as a monthly rent in some countries or more than a monthly salary in others.
That problem does not even include lock-in, surveillance, IP theft and all other things that come with SaaS.
azan_
Paid models are just much, much better.
dakiol
Of course they are. I wouldn't expect otherwise :)
But the price we're paying (and I don't mean money) is very high, imho. We all talk about how good engineers write code that depends on high-level abstractions instead of low-level details, allowing us to replace third party dependencies easily and test our apps more effectively, keeping the core of our domain "pure". Well, isn't it time we started doing the same with LLMs? I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. That would at least allow us to switch to a free and opensource version if the companies behind the private LLMs go rogue. I'm afraid tho that wouldn't be enough, but it's a starting point.
To put an example: what would you think if you need to pay for every single Linux process in your machine? Or for every Git commit you make? Or for every debugging session you perform?
azan_
> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones. That would at least allow us to switch to a free and opensource version if the companies behind the private LLMs go rogue. I'm afraid tho that wouldn't be enough, but it's a starting point.
There are open source tools that do exactly that already.
simonw
> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones.
I have been building that for a couple of years now: https://llm.datasette.io
ghm2180
> I'm not talking about MCP, but rather an open source tool that can plug into either free and open source LLMs or private ones.
Has someone computed/estimated what is at cost $$$ value of utilizing these models at full tilt: several messages per minute and at least 500,000 token context windows? What we need is a wikipedia like effort to support something truly open and continually improving in its quality.
airstrike
None of that applies here since we could all easily switch to open models at a moment's notice with limited costs. In fact, we switch between proprietary models every few months.
It just so happens that closed models are better today.
1024core
I have found that if I ask the LLM to first _describe_ to me what it wants to do without writing any code, then the subsequent code generated has much higher quality. I will ask for a detailed description of the things it wants to do, give it some feedback and after a couple of iterations, tell it to go ahead and implement it.
quantumHazer
I'm going a little offtopic here, but I disagree with the OPs use of the term "PhD-level knowledge", although I have a huge amount of respect for antirez (beside that we are born in the same island).
This phrasing can be misleading and points to a broader misunderstanding about the nature of doctoral studies, which it has been influenced by the marketing and hype discourse surrounding AI labs.
The assertion that there is a defined "PhD-level knowledge" is pretty useless. The primary purpose of a PhD is not simply to acquire a vast amount of pre-existing knowledge, but rather to learn how to conduct research.
ghm2180
> but rather to learn how to conduct research
Further, I always assumed PhD level of knowledge meant coming up with the right questions. I would say it is at best a "Lazy Knowledge Rich worker", it won't explore hypothesis if you don't *ask it* to. A PHD would ask those questions to *themselves*. Let me give you a simple example:
The other day Claude Code(Max Pro Subscription) commented out a bunch of test assertions as a part of a related but separate test suite it was coding. It did not care to explore — what was a serious bug — why it was commenting it out because of a faulty assumption in the original plan. I had to ask it to change the plan by doing the ultra-think, think-hard trick to explore why it was failing, amend the plan and fix it.
The bug was the ORM object had null values because it was not refreshed after the commit and was fetched before by another DB session that had since been closed.*
antirez
Agree with that. Read it as expert-level knowledge without all the other stuff LLMs can’t do as well as humans. LLMs way to express knowledge is kinda of alien as it is different, so indeed those are all poor simplifications. For instance an LLM can’t code as well as a top human coder but can write a non trivial program from the first to the last character without iterating.
spyckie2
Hey antirez,
What sticks out to me is Gemini catching bugs before production release, was hoping you’d give a little more insight into that.
Reason being is that we expect ai to create bugs and we catch them, but if Gemini is spotting bugs by some way of it being a QA (not just by writing and passing tests) then that perks my interest.
chis
If you understand that a PhD is about much more than just knowledge, it's still the case that having easy access to that knowledge is super valuable. My last job we often had questions that would just traditionally require a PhD-level person to answer, even if it wasn't at the limit of their research abilities. "What will happen to the interface of two materials if voltage is applied in one direction" type stuff, turns out to be really hard to answer but LLMs do a decent job.
quantumHazer
Have you checked experimentally the response of the LLM?
Anyway I don't think this is ""PhD-knowledge"" questions, but job related electrical engineering questions.
kgwgk
> The primary purpose of a PhD is not simply to acquire a vast amount of pre-existing knowledge, but rather to learn how to conduct research.
It’s not like once you have a PhD anyone cares about the subject, right? The only thing that matters is that you learnt to conduct research.
quantumHazer
I can't understand why once you have a PhD anyone should care more about the subject.
airstrike
I think all conversations about coding with LLMs, vibe coding, etc. need to note the domain and choice of programming language.
IMHO those two variables are 10x (maybe 100x) more explanatory than any vibe coding setup one can concoct.
Anyone who is befuddled by how the other person {loves, hates} using LLMs to code should ask what kind of problem they are working on and then try to tackle the same problem with AI to get a better sense for their perspective.
Until then, every one of these threads will have dozens of messages saying variations of "you're just not using it right" and "I tried and it sucks", which at this point are just noise, not signal.
cratermoon
They should also share their prompts and discuss exactly how much effort went into checking the output and re-prompting to get the desired result. The post hints at how much work it takes for the human, "If you are able to describe problems in a clear way and, if you are able to accept the back and forth needed in order to work with LLMs ... you need to provide extensive information to the LLM: papers, big parts of the target code base ... And a brain dump of all your understanding of what should be done. Such braindump must contain especially the following:" and more.
After all the effort getting to the point where the generated code is acceptable, one has to wonder, why not just write it yourself? The time spent typing is trivial to all the cognitive effort involved in describing the problem, and describing the problem in a rigorous way is the essence of programming.
null
tines
I would assume the argument is that you only need to provide the braindump and extensive information one time (or at least, collect it once, if not upload once) and then you can take your bed of ease as the LLM uses that for many tasks.
UncleEntity
> After all the effort getting to the point where the generated code is acceptable, one has to wonder, why not just write it yourself?
You know, I would often ask myself that very question...
Then I discovered the stupid robots are good at designing a project, you ask them to produce a design document, argue over it with them for a while, make revision and changes, explore new ideas, then, finally, ask them to produce the code. It's like being able to interact with the yaks you're trying to shave, what's not to love about that?
null
skippyboxedhero
Have used Claude's GitHub action quite a bit now (10-20 issue implementations, a bit more PR reviews), and it is hit and miss so agree with the enhanced coding rather than just letting it run loose.
When the change is very small, self-contained feature/refactor it can mostly work alone, if you have tests that cover the feature then it is relatively safe (and you can do other stuff because it is running in an action, which is a big plus...write the issue and you are done, sometimes I have had Claude write the issue too).
When it gets to a more medium size, it will often produce something that will appear to work but actually doesn't. Maybe I don't have test coverage and it is my fault but it will do this the majority of the time. I have tried writing the issue myself, adding more info to claude.md, letting claude write the issue so it is a language it understands but nothing works, and it is quite frustrating because you spend time on the review and then see something wrong.
And anything bigger, unsurprisingly, it doesn't do well.
PR reviews are good for small/medium tasks too. Bar is lower here though, much is useless but it does catch things I have missed.
So, imo, still quite a way from being able to do things independently. For small tasks, I just get Claude to write the issue, and wait for the PR...that is great. For medium (which is most tasks), I don't need to do much actual coding, just directing Claude...but that means my productivity is still way up.
I did try Gemini but I found that when you let it off the leash and accept all edits, it would go wild. We have Copilot at work reviewing PRs, and it isn't so great. Maybe Gemini better on large codebases where, I assume, Claude will struggle.
Keyframe
Unlike OP, from my still limited but intense month or so diving into this topic so far, I had better luck with Gemini 2.5 PRO and Opus 4 on more abstract level like architecture etc. and then dealing input to Sonnet for coding. I found 2.5 PRO, and to a lesser degree Opus, were hit or miss; A lot of instances of them circling around the issue and correcting itself when coding (Gemini especially so), whereas Sonnet would cut to the chase, but needed explicit take on it to be efficient.
antirez
Totally possible. In general I believe that while more powerful in their best outputs, Sonnet/Opus 4 are in other ways (alignment / consistency) a regression on Sonnet 3.5v2 (often called Sonnet 3.6), as Sonnet 3.7 was. Also models are complex objects, and sometimes in a given domain a given model that on paper is weaker will work better. And, on top of that: interactive use vs agent requires different reinforcement learning training that sometimes may not be towards an aligned target... So also using the model in one way or the other may change how good it is.
khaledh
This is my experience too. I usually use Gemini 2.5 Pro through AI Studio for big design ideas that need to be validated and refined. Then take the refined requirements to Claude Code which does an excellent job most of the time in coding them properly. Recently I tried Gemini CLI, and it's not even close to Claude Code's sharp coding skills. It often makes syntax mistakes, and get stuck trying to get itself out of a rut; its output is so verbose (and fast) that it's hard to follow what it's trying to do. Claude Code has a much better debugging capability.
Another contender in the "big idea" reasoning camp: DeepSeek R1. It's much slower, but most of the time it can analyze problems and get to the correct solution in one shot.
bgwalter
Translation: His company will launch "AI" products in order to get funding or better compete with Valkey.
I find it very sad that people who have been really productive without "AI" now go out of their way to find small anecdotal evidence for "AI".
brokencode
I find it even more sad when people come out of the woodwork on every LLM post to tell us that our positive experiences using LLMs are imagined and we just haven’t realized how bad they are yet.
skippyboxedhero
Some people got into coding to code, rather than build things.
If the AI is doing the coding then that is a threat to some people. I am not sure why, LLMs can be good and you can enjoy coding...those things are unrelated. The logic seems to be that if LLMs are good then coding is less fun, lol.
Cheer2171
Software jobs pay more than artist jobs because coding builds things. You can still be a code artist on your own time. Nobody is stopping you from writing in assembler.
halfmatthalfcat
Could it not be that those positive experiences are just shining a light that the practices before using an LLM were inefficient? It’s more a reflection on the pontificator than anything.
johnfn
Sure, but even then the perspective makes no sense. The common argument against AI at this point (e.g. OP) is that the only reason people use it is because they are intentionally trying to prop up high valuations - they seem unable to understand that other people have a different experience than they do. You’d think that just because there are some cases where it doesn’t work doesn’t necessarily mean that 100% of it is a sham. At worst it’s just up to individual taste, but that doesn’t mean everyone who doesn’t share your taste is wrong.
Consider cilantro. I’m happy to admit there are people out there who don’t like cilantro. But it’s like the people who don’t like cilantro are inventing increasingly absurd conspiracy theories (“Redis is going to add AI features to get a higher valuation”) to support their viewpoint, rather than the much simpler “some people like a thing I don’t like”.
jstanley
Tautologically so! That doesn't show that LLMs are useless, it perfectly shows how they are useful.
cratermoon
We don't just tell you they were imagined, we can provide receipts.
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
nojito
Cursor is an old way of using LLMs.
Not to mention in the study less than 1/2 have ever used it before the study.
on_the_train
If LLMs were actually useful, there would be no need to scream it everywhere. On the contrary: it would be a guarded secret.
logsr
posting a plain text description of your experience on a personal blog isn't exactly screaming. in the noise of the modern internet this would be read by nobody if it wasn't coming from one of the most well known open source software creators of all time.
people who believe in open source don't believe that knowledge should be secret. i have released a lot of open source myself, but i wouldn't consider myself a "true believer." even so, i strongly believe that all information about AI must be as open as possible, and i devote a fair amount of time to reverse engineering various proprietary AI implementations so that i can publish the details of how they work.
why? a couple of reasons:
1) software development is my profession, and i am not going to let anybody steal it from me, so preventing any entity from establishing a monopoly on IP in the space is important to me personally.
2) AI has some very serious geopolitical implications. this technology is more dangerous than the atomic bomb. allowing any one country to gain a monopoly on this technology would be extremely destabilizing to the existing global order, and must be prevented at all costs.
LLMs are very powerful, they will get more powerful, and we have not even scratched the surface yet in terms of fully utilizing them in applications. staying at the cutting edge of this technology, and making sure that the knowledge remains free, and is shared as widely as possible, is a natural evolution for people who share the open source ethos.
neuronexmachina
In my experience, devs generally aren't secretive about tools they find useful.
victorbjorklund
If Internet was actually useful there would be no need to scream it everywhere. Guess that means the internet is totally useless?
brokencode
So ironic that you post this on Hacker News, where there are regularly articles and blog posts about lessons from the industry, both good and bad, that would be helpful to competitors. This industry isn’t exactly Coke guarding its secret recipe.
alwillis
If LLMs were actually useful, there would be no need to scream it everywhere. On the contrary: it would be a guarded secret.
LLMs are useful—but there’s no way such an innovation should be a “guarded secret” even at this early stage.
It’s like saying spreadsheets should have remained a secret when they amplified what people could do when they became mainstream.
hobs
I think many devs are guarding their secrets, but the last few decades have shown us that an open foundation can net huge benefits for everyone (and then you can put your secret sauce in the last mile.)
null
antirez
Did you read my post? I hope you didn’t.
This post has nothing to do with Redis and is even a follow up to a post I wrote before rejoining the company.
babuloseo
OP as a free user of Gemini 2.5 Pro via Ai studio my friend has been hit by the equivalent of a car breaking approximately 3 weeks, I hope they can recover soon, it is not easy for them.
syntheticnature
This is HN. We don't read posts here.
kgwgk
Amen. I have to confess that I made an exception here though. This may be the first submission I read before going into the comments in years.
dcre
“Always be part of the loop by moving code by hand from your terminal to the LLM web interface: this guarantees that you follow every process. You are still the coder, but augmented.”
I agree with this, but this is why I use a CLI. You can pipe files instead of copying and pasting.
lmeyerov
Yeah it is also a bit of a shibboleth: vibes coding, when I'm productive for the 80% case with Claude code, is about the LLM cranking for 10-20min. I'm instructing & automating the LLM on how to do its own context management, vs artisanally making every little decision.
Ex: Implementing a spec, responding to my review comments, adding wider unit tests, running a role play for usability testing, etc. The main time we do what he describes of manually copying into a web ide is occasionally for a better short use of a model, like only at the beginning of some plan generation, or debug from a bunch of context we have done manually. Like we recently solved some nasty GPU code race this way, using a careful mix of logs and distributed code. Most of our job is using Boring Tools to write Boring Code, even if the topic/area is neato: you do not want your codebase to work like an adventure for everything, so we invest in making it look boring.
I agree the other commenter said: I manage context as part of the skill, but by making the AI do it. Doing that by hand is like slowly handcoding assembly. Instead, I'm telling Claude Code to do it. Ex: Download and crawl some new dependency I'm using for some tricky topic, or read in my prompt template markdown for some task, or generate and self-maintain some plan.md with high-level rules on context I defined. This is the 80% case.
Maybe one of the disconnects is task latency vs throughput as trade-offs in human attention. If I need the LLM to get to the right answer faster, so the task is done faster, I have to lean in more. But my time is valuable and I have a lot to do. If rather spend 50% less of my time per task, even if the task takes 4x longer, by the LLM spinning longer. In that saved human time, I can be working on another task: I typically have 2-3 terminals running Claude, so I only check in every 5-15min.
airstrike
Your strategy only works for some domains.
lmeyerov
Totally
We do this ~daily for:
* Multitier webapps
* DevOps infrastructure: docker, aws, ci systems, shell scripts, ...
* Analytics & data processing
* AI investigations (logs, SIEMs, ..) <--- what we sell!
* GPU kernels
* Compilers
* Docs
* Test amplification
* Spec writing
I think ~half the code happening by professional software engineers fits into these, or other vibes friendly domains. The stuff antirez does with databases seems close to what we do with compilers, GPU kernels, and infra.
We are still not happy with production-grade frontend side of coding, though by being strong on API-first design and keeping logic vs UI seperated, most of our UI code is friendly to headless.
fumeux_fume
I currently use LLMs as a glorified Stack Overflow. If I want to start integrating an LLM like Gemini 2.5 PRO into my IDE (I use Visual Studio Code), whats the best way to do this? I don't want to use a platform like Cursor or Claude Code which takes me away from my IDE.
physicles
Cursor is an IDE. You can use its powerful (but occasionally wrong) autocomplete, and start asking it to do small coding tasks using the Ctrl+L side window.
fumeux_fume
I don't want to leave my IDE
Philpax
Worth noting that Cursor is a VS Code fork and you can copy all of your settings over to it. Not saying that you have to, of course, but that it's perhaps not as different as you might be imagining.
hedgehog
GitHub Copilot is pretty easy to try within VS Code
fumeux_fume
I want to use Gemini 2.5 PRO. I was an early tester of Copilot and it was awful.
haiku2077
Copilot has 2.5 Pro in the settings in github.com, along with claude 4
speedgoose
Thanks for writing this article.
I used a similar setup until a few weeks ago, but coding agents became good enough recently.
I don’t find context management and copy pasting fun, I will let GitHub Copilot Insiders or Claude Code do it. I’m still very much in the loop while doing vibe coding.
Of course it depends on the code base, and Redis may not benefit much from coding agents.
But I don’t think one should reject vibe coding at this stage, it can be useful when you know what the LLMs are doing.
theodorewiles
My question on all of the “can’t work with big codebases” is how would a codebase that was designed for an LLM look like? Composed of many many small functions that can be composed together?
antirez
I believe it’s the same as for humans: different files implementing different parts of the system with good interfaces and sensible boundaries.
afro88
Well documented helps a lot too.
You can use an LLM to help document a codebase, but it's still an arduous task because you do need to review and fix up the generated docs. It will make, sometimes glaring sometimes subtle, mistakes. And you want your documentation to provide accuracy rather than double down on or even introduce misunderstanding.
exitb
And on top of that - can you steer an LLM to create this kind of code? In my experience the models don’t really have a „taste” for detecting complexity creep and reengineering for simplicity, in the same way an experienced human does.
lubujackson
I am vibe coding a complex app. You can certainly keep things clean but the trick is to enforce a rigid structure. This does add a veneer of complexity but simplifies " implement this new module" or "add this feature across all relevant files".
victorbjorklund
I found that it is beneficial to create more libraries. If I for example build a large integration to an API (basically a whole api client) I would in the past have it in the same repo but now I make it a standalone library.
Hasnep
And my question to that is how would that be different from a codebase designed for humans?
__MatrixMan__
I think it means finer toplevel granularity re: what's runnable/testable at a given moment. I've been exploring this for my own projects and although it's not a silver bullet, I think there's something to it.
----
Several codebases I've known have provided a three-stage pipeline: unit tests, integration tests, and e2e tests. Each of these batches of tests depend on the creation of one of three environments, and the code being tested is what ends up in those environments. If you're interested in a particular failing test, you can use the associated environment and just iterate on the failing test.
For humans with a bit of tribal knowledge about the project, humans who have already solved the get-my-dev-environment-set-up problem in more or less uniform way, this works ok. Humans are better at retaining context over weeks and months, whereas you have to spin up a new session with an LLM every few hours or so. So we've created environments for ourselves that we ignore most of the time, but that are too complex to be bite sized for an agent that comes on the scene as a blank slate every few hours. There are too few steps from blank-slate to production, and each of them is too large.
But if successively more complex environments can be built on each other in arbitrarily many steps, then we could achieve finer granularity. As a nix user, my mental model for this is function composition where the inputs and outputs are environments, but an analogous model would be layers in a docker files where you test each layer before building the one on top of it.
Instead of maybe three steps, there are eight or ten. The goal would be to have both whatever code builds the environment, and whatever code tests it, paired up into bite-sized chunks so that a failure in the pipeline points you a specific stage which is more specific that "the unit tests are failing". Ideally test coverage and implementation complexity get distributed uniformly across those stages.
Keeping the scope of the stages small maximizes the amount of your codebase that the LLM can ignore while it works. I have a flake output and nix devshell corresponding to each stage in the pipeline and I'm using pytest to mark tests based on which stage they should run in. So I run the agent from the devshell that corresponds with whichever stage is relevant at the moment, and I introduce it to onlythe tests and code that are relevant to that stage (the assumption being that all previous stages are known to be in good shape). Most of the time, it doesn't need to know that it's working stage 5 of 9, so it "feels" like a smaller codebase than it actually is.
If evidence emerges that I've engaged the LLM at the wrong stage, I abandon the session and start over at the right level (now 6 of 9 or somesuch).
Keyframe
like a microservice architecture? overall architecture to get the context and then dive into a micro one?
IMO Claude code was a huge step up. We have a large and well structured python code base revolving mostly around large and complicated adapter pattern Claude is almost fully capable to implement a new adapter if given the right prompt/resources.