AI tooling must be disclosed for contributions

One of my mental models is that the notion of "effective engineer" used to mean "effective software developer" whether or not they were good at system design.

Now, an "effective engineer" can be a less battle-tested software developer, but they must be good at system design.

(And by system design, I don't just mean architecture diagrams: it's a personal culture of constantly questioning and innovating around "let's think critically to see what might go wrong when all these assumptions collide, and if one of them ends up being incorrect." Because AI will only suggest those things for cut-and-dry situations where a bug is apparent from a few files' context, and no ambitious idea is fully that cut-and-dry.)

The set of effective engineers is thus shifting - and it's not at all a valid assumption that every formerly good developer will see their productivity skyrocket.

I've been starting to think of it like this:

Great Engineer + AI = Great Engineer++ (Where a great engineer isn't just someone who is a great coder, they also are a great communicator & collaborator, and love to learn)

Good Engineer + AI = Good Engineer

OK Engineer + AI = Mediocre Engineer

This fits my observations as well. With the exception that it’s sometimes the really sharp engineers who can do wonders themselves who aren’t really great at communication. AI really needs you to be verbose, and a lot of people just can’t.

It’s like the difference between someone who can search the internet or codebase well bs someone who can’t

Using search engines is a skill

I've been struggling to apply AI on any large scale at work. I was beginning to wonder if it was me.

But then my wife sort of handed me a project that previously I would have just said no to, a particular Android app for the family. I have instances of all the various Android technologies under my belt, that is, I've used GUI toolkits, I've used general purpose programming languages, I've used databases, etc, but with the possible exception of SQLite (which even that is accessed through an ORM), I don't know any of the specific technologies involved with Android now. I have never used Kotlin; I've got enough experience that I can pretty much piece it together when I'm reading it but I can't write it. Never used the Android UI toolkit, services, permissions, media APIs, ORMs, build system, etc.

I know from many previous experiences that A: I could definitely learn how to do this but B: it would be a many-week project and in the end I wouldn't really be able to leverage any of the Android knowledge I would get for much else.

So I figured this was a good chance to take this stuff for a spin in a really hard way.

I'm about eight hours in and nearly done enough for the family; I need about another 2 hours to hit that mark, maybe 4 to really polish it. Probably another 8-12 hours and I'd have it brushed up to a rough commercial product level for a simple, single-purpose app. It's really impressive.

And I'm now convinced it's not just that I'm too old a fogey to pick it up, which is, you know, a bit of a relief.

It's just that it works really well in some domains, and not so much in others. My current work project is working through decades of organically-grown cruft owned by 5 different teams, most of which don't even have a person on them that understands the cruft in question, and trying to pull it all together into one system where it belongs. I've been able to use AI here and there for some stuff that is still pretty impressive, like translating some stuff into psuedocode for my reference, and AI-powered autocomplete is definitely impressive when it correctly guesses the next 10 lines I was going to type effectively letter-for-letter. But I haven't gotten that large-scale win where I just type a tiny prompt in and see the outsized results from it.

I think that's because I'm working in a domain where the code I'm writing is already roughly the size of the prompt I'd have to give, at least in terms of the "payload" of the work I'm trying to do, because of the level of detail and maturity of the code base. There's no single sentence I can type that an AI can essentially decompress into 250 lines of code, pulling in the correct 4 new libraries, and adding it all to the build system the way that Gemini in Android Studio could decompress "I would like to store user settings with a UI to set the user's name, and then display it on the home page".

I think I recommend this approach to anyone who wants to give this approach a fair shake - try it in a language and environment you know nothing about and so aren't tempted to keep taking the wheel. The AI is almost the only tool I have in that environment, certainly the only one for writing code, so I'm forced to really exercise the AI.

This is it exactly.

Slop generators being available to everyone makes everyone less trustworthy, from a maintainer's POV. Thus, the circle of trust, for any given maintainer, shrinks starkly.

People do not become maintainers because they want to battle malicious, or even criminally negligent crap. They expect benign and knowledgeable contributors, or at least benign and willing to do their homework ones.

Being a maintainer is already hugely thankless. It's hard work (harder than writing code), and it comes with a lot less recognition. Not to mention all the newcomers that (a) maintainers usually eagerly educate, but then (b) disappear.

Screw up the social contract for maintainers even more, and they'll go extinct. (Edit: if a maintainer gets a whiff of some contributor working against them, rather than with them, they'll either ban the contributor forever, or just quit the project.)

Any sane project should categorically ban AI-assisted contributions, and extend their Signed-off-by definition, after a cut-off-date, to carry an explicit statement by the contributor that the code is free of AI-output. If this rules out "agentic IDE"s, that's a win.

> You must ask "Do I trust where this came from?" And to answer that, you need to know where it come from.

No you don’t. You can’t outsource trust determinations. Especially to the people you claim not to trust!

You make the judgement call by looking at the code and your known history of the contributor.

Nobody cares if contributors use an LLM or a magnetic needle to generate code. They care if bad code gets introduced or bad patches waste reviewers’ time.

> I can generate 1,000 PRs today against an open source project using AI.

Then perhaps the way you contribute, review, and accept code is fundamentally wrong and needs to change with the times.

It may be that technologies like Github PRs and other VCS patterns are literally obsolete. We've done this before throughout many cycles of technology, and these are the questions we need to ask ourselves as engineers, not stick our heads in the sand and pretend it's 2019.

In that case a more correct rule (and probably one that can be automatically enforced) for that issue is a max number of PRs or opened issues per account.

Both sides will use AI and it will ultimately increase economic productivity.

Imagine living before the invention of the printing press, and then lamenting that we should ban them because it makes it "too easy" to distribute information and will enable "low quality" publications to have more reach. Actually, this exact thing happened, but the end result was it massively disrupted the world and economy in extremely positive ways.

You are absolutely right. AI is just a tool to DDoS maintainers.

Any contributor who was shown to post provably untested patches used to lose credibility. And now we're talking about accommodating people who don't even understand how the patch is supposed to work?

Unenforceable rules are bad, but if you tweak the rule to always require some sort of authorship statement (e.g. "I wrote this by hand" or "I wrote this with Claude"), then the honor system will mostly achieve the desired goal of calibrating code review effort.

Except you can enforce this rule some of the time. People discover that AI was used or suspect it all the time, and people admit to it after some pressure all the time.

Not every time, but sometimes. The threat of being caught isn't meaningless. You can decide not to play in someone else's walled garden if you want but the least you can do is respect their rules, bare minimum of human decency.

> As a project maintainer, you shouldn't make rules unenforceable rules

Total bullshit. It's totally fine to declare intent.

You are already incapable of verifying / enforcing that a contributor is legally permitted to submit a piece of code as their own creation (Signed-off-by), and do so under the project's license. You won't embark on looking for prior art, for the "actual origin" of the code, whatever. You just make them promise, and then take their word for it.

> Courts (at least in the US) have already ruled that use of ingested data for training is transformative

This is far from settled law. Let's not mischaracterize it.

Even so, an AI regurgitating proprietary code that's licensed in some other way is a very real risk.

> I don't think anyone who's not monetarily incentivize to pretend there are IP/Copyright issues actually thinks there are.

Why do you think that about people who disagree with you? You're responding directly to someone who's said they think there's issues, and not pretending. Do you think they're lying? Did you not read what they said?

And AFAICT a lot of other people think similarly to me.

The perverse incentives to rationalize are on the side of the people looking to exploit the confusion, not the people who are saying "wait a minute, what you're actually doing is..."

So a gold rush person claiming opponents must be pretending because of incentives... seems like the category of "every accusation is a confession".

Huge difference, and companies recognized the difference, right up until "AI" hype.

"see" and "copy" are two different things. It's fine to look at StackOverflow to understand the solution to a problem. It's not fine to copy and paste from StackOverflow and ignore its license or attribution.

Content on StackOverflow is under CC-by-sa, version depends on the date it was submitted: https://stackoverflow.com/help/licensing . (It's really unfortunate that they didn't pick license compatible with code; at one point they started to move to the MIT license for code, but then didn't follow through on it.)

How is that same thing?

I doubt a PR is going to be buried if it's useful, well designed, good code, etc, just because of this disclosure. Articulate how you used AI and I think you've met the author's intent.

If the PR has issues and requires more than superficial re-work to be acceptable, the authors don't want to spend time debugging code spit out by an AI tool. They're more willing to spend a cycle or two if the benefit is you learning (either generally as a dev or becoming more familiar with the project). If you can make clear that you created or understand the code end to end, then they're more likely to be willing to take these extra steps.

Seems pretty straightforward to me and thoughtful by the maintainers here.

What, building systems where we’re specifically incentivised not to disclose ai use?

It might encourage people to be dishonest, or to not contribute at all. Maybe that's fine for now, but what if the next generation come to rely on these tools?