Skip to content(if available)orjump to list(if available)

The way we're thinking about breaking changes

ivan_gammel

This post has clickbait vibe unfortunately and is really shallow regarding the analysis of the problem. No, it is not silly how we think about breaking changes. The idea of using some change log isn’t new and there are reasons why it’s not implemented by most build systems.

The elephant in the room is changing business requirements. If the change requires the code to consume new parameter without reasonable defaults, no automatic migration would solve it reliably. Let’s say you deal with the money and you introduce currency field. The compiler of the client code cannot just assume the currency, neither it can take it from anywhere by just applying the migration. It is a manual operation to fix the broken code.

TimTheTinker

I think the idea is the kernel of something very interesting. TA is right - in typed languages especially, why not add some extra metadata what would render many changes non-breaking?

I think part of our hang-up is that we tend to think of code and data in very separate categories (except for literals, of course)--and only allow treating code as data after a lexer starts processing it (or via reflection or first-class functions at run time).

Correcting this category error and adding metadata (not just types) could yield all sorts of interesting ideas.

Like - as TA suggests, why not encode the notion of how a function signature has changed over time to enable automatic migrations?

ivan_gammel

It’s not going to work. You cannot capture the meaning of the change with metadata.

Let’s say you have this change:

   BigDecimal doSomething();
to

   Money doSomething();
In theory metadata can capture the migration from a plain number to Money#amount() and compiler will generate a facade for the client code. But simply discarding the currency will have disastrous effects at some later point. Even relatively simple migration String->Number may go wrong without semantics. Language feature allowing such metadata and migrations will be a minefield.

TimTheTinker

Sure, there are lots of examples of lossy migrations, even in databases.

That doesn't mean migrations in general aren't a good idea.

And it doesn't invalidate the notion of metadata for code in general and all the possibilities that opens up.

AtlasBarfed

A lot of that kernel is that dependency information is grossly coarse. For the vast majority of langs I ve used it is simply the library name: not even a library version (build files like gradle will).

The node approach is very fine grained libraries, but that makes its own headaches

Really every function or interface invocation has an implicit version number. It should be explicit in the code, and that assumes major minor breaking change versioning.

steego

I think this proposal ultimately introduces more complexity than it solves. By automatically inserting migrations (whether at compile time or via “migration files”), you end up with a form of hidden control flow that’s arguably worse than traditional overloading or type coercion. In normal type coercion, the transformation is explicit and visible in the language syntax, whereas these “migrations” happen magically behind the scenes.

Second, database migrations are notoriously tricky for developers to manage correctly. They often require significant domain knowledge to avoid breaking assumptions further down the line. Applying that paradigm directly to compiler-managed code changes feels like it would amplify the same problems—especially in languages that rely on strong type inference. The slightest mismatch in inferred types could ripple through a large codebase in ways that are far from obvious.

While it’s an interesting idea in theory, I think the “fix your old code with a macro-like script” approach just shifts maintenance costs elsewhere. We’d still be chasing edge cases, except now they’re tucked away behind code-generation layers and elaborate type transformations. It may reduce immediate breakage, but at the expense of clarity and predictable behavior in the long run.

lolinder

> In normal type coercion, the transformation is explicit and visible in the language syntax, whereas these “migrations” happen magically behind the scenes.

I was assuming that the migration would actually alter the call site in a way that is reviewable and committable, not implicitly do it every time it's needed at compile time. If that's the idea, it doesn't alter anything behind the scenes, it does so explicitly and visibly one time to change everything to meet the new requirements.

The larger problem I see is that the applications for this would be extremely limited. There's a reason why they used a simple s/a/Some a/ as an illustration—once you get beyond that the migrations will become a pain to write, a pain to validate, and likely to break tons of code in subtle ways. And since most library code changes aren't this simple kind, it's likely that it's not worth the effort of building and maintaining this migration syntax for something with so narrow an application.

kmeisthax

The way you handle breaking changes in SQL is views, not migrations; the latter is only appropriate single applications abusing SQL as a data store[0].

In code, if you want to handle a breaking change transparently, you don't need the moral equivalent of a linker fix-up table. You need to keep the old definition around and redirect it to the new function. This can be as simple as having, say, the old version of your code wrap something in a list or closure and then call the new version. In languages with overloading, this can be done transparently; in others you'd have to give the new function a new name. Maybe API versioned symbols and a syntax for them is what you would want?

[0] The original idea with SQL is that multiple applications would store data in the same place in a common format you could meaningfully interchange.

morsecodist

I am a bit confused by this. Typically when you make a breaking change it is because the call site has to make a new sort of decision so you may not want all calls to be refactored the same way and there is no way of determining which you want at runtime.

The example the author uses is modifying a function to return an int or null instead of an int. Let's say you implemented the function a bit naively and in the null case your function would crash the program. Now you are going to refactor your codebase so the caller gets to decide what happens in the null case. Some callers may be unable to handle the situation and will need to implement the crash/exception some may use some sort of fallback behavior.

I think haskell's type system probably would probably prevent someone from having the issue I described above but the problem still stands. Let's say you had a function that returns a union type and you have areas in the code that are supposed to handle every case of that type. If you had a case you need to handle it everywhere and the compiler can't know how you want to handle it. A lot of type systems will catch the missing case which is awesome but you still need to handle each one.

immibis

If the null case crashed the program before, and still crashes the program after but at the call site instead, then compatibility has not been sacrificed. It still crashes in the null case, but now it runs at all in the non-null case, whereas if you don't have this migration feature, your "fix" made it crash in both cases.

lolinder

It's not perfect, but Kotlin already has a limited form of this with its ReplaceWith annotation [0]. It allows you to mark something as @Deprecated and to specify what should be used instead. IDEs and other tooling can pick that up and apply it automatically.

This assumes that you're approaching the breaking change problem from the perspective of immutable names and deprecations: don't make breaking changes to existing names, create a new name for the new behavior and communicate that the old behavior is no longer going to be supported.

[0] https://kotlinlang.org/api/core/kotlin-stdlib/kotlin/-replac...

samus

Databases have empathically not solved this problem. I have never heard of an DBS with first-class support of migrations; they are usually layered on top using something like Flyway or Liquibase. A query still working after a migration is still either an accident or due to a deliberate backwards compatibility effort.

The proposal essentially expects the dependency to provide a migration path. However, to the extent described this is only necessary in languages with no overloading. And it doesn't help for cases when the change is not as simple as providing a few rewrite rules or things like that.

Also, the jab about static type systems is misplaced. Their purpose is specifically to reject certain invalid programs, at the cost of rejecting a few valid. You can't have one without the other. With dynamic typing, backwards compatibility breakages fly under the radar until they gleefully blow up in production.

wavemode

The code equivalent of a "migration" is that you just write a new function that takes your new desired argument types, and then have the old function perform whatever type conversion is necessary and call the new.

If writing such a function is not possible (because there is no automated way to convert from the old function call style to the new) then a database-style automated migration isn't going to be possible either.

oftenwrong

Alternatively, we could use a model in which functions are immutable, and therefore make such breaking changes impossible. This is the approach taken by Unison:

https://www.unison-lang.org/docs/the-big-idea/

theamk

I've read about Unison a lot, and while "remote execution" idea sounds extremely cool, all of other properties seem very dubious. For example re "no breaking changes":

- Imagine that in my project, I have a "FOO" function, which is called by many others

- I've decide to change type of one parameter of FOO function. This would be a breaking change in regular language, but in Unison, nothing breaks - I push the new definition, but every caller is still using old version.

- New callers come up, and they use new version. So far so good.

- Some times later, I've discovered a critical business-logic bug in FOO function! So I fix it, and I have to update all the callers to use the latest version... except for half of them I can not, because the parameter types do not match. Seems like I cannot ship the fix to the customer until I spend a bunch of time rewriting the existing code to accommodate argument type change.

As long as there are functions, there are always some kinds changes to them that require one to fix up the callers. How this is enforced can be different - in strictly typed languages, code may fail to compile; in dynamic languages, you may see runtime failures; and in Unison, things will work until you try to edit the caller, at which case it'll fail to compile (unison docs call that operation, converting from text to internal language's representation, "typecheck").

I am not convinced that the "postpone failure until you edit the caller" is the best approach here. When I refactor, I normally want to see any problems surface right away, while I still have the context for the change.

naasking

This also has issues with security fixes, unfortunately.

lolinder

How so? You can always deprecate the old functions and encourage people to use the new ones, it just doesn't automatically force everyone to do so immediately by breaking their compilation.

jcelerier

but that's the crux of the issue: some people think that if software is insecure, it is better for it to be actually unuseably broken. see for instance how many valid usecases of software that "spies" on global key input are broken by X11 -> wayland for the sake of better security: stuff like autohotkey, global recording in apps like OBS Studio, global key displays, yaquake-style terminals..

bluGill

That only works if your tools have planned ahead and so when an 'old call' it somehow goes to a translation system now. It also assumes you have good translation for the changee function and the performance and memory costs of it are acceptable.

none of the above as always true.

tags2k

I'm entirely unsure how database migrations aren't breaking changes - you migrate to a new version of your schema, queries that use an older schema aren't going to work. Database server functions can be changed through migrations too.

blixt

I think interactions between many types and functions would be harder to migrate, especially if you go so granular as to target individual types independently. Maybe if it was a migration from some checkpoint (timestamp / version / commit / whatever you like that describes an entire state of the code) to another, then you could migrate indirect code patterns that need to change as well.

I think most of all one must have discipline to thoroughly think of old code and how it should behave now. I found that one can already do this by versioning code and types, similar to how an API endpoint might go from /v2/… to /v3/…. But again, it requires discipline and it’s not something I’d do for anything but very critical code.

chromanoid

OpenRewrite is quite nice and has a great collection of recipes for migrating, e.g. from javax to jakarta: https://docs.openrewrite.org/recipes/java/migrate

There are some quirks you have to work around for big projects since the free tooling has some limitations.