There is no memory safety without thread safety

I agree that there are two groups here talking past each other. I think it would help a lot to clarify this:

> the issue here is that the "Rust and Java sense" of memory safety is not the actual meaning of the term

So what is the actual meaning? Is it simply "there are no cases of actual exploited bugs in the wild"?

Because in another comment you wrote:

> a term of art was created to describe something complicated; in this case, "memory safety", to describe the property of programming languages that don't admit to memory corruption vulnerabilities, such as stack and heap overflows, use-after-frees, and type confusions. Later, people uninvolved with the popularization of the term took the term and tried to define it from first principles, arriving at a place different than the term of art.

But type confusion is exactly what has been demonstrated in the post's example. So what kind of memory safety does Go actually provide, in the term of art sense?

> But: everybody understands that.

Everybody does not understand that otherwise there would be zero of these issues in shipping code.

This is the problem with the C++ crowd hoping to save their language. Maybe they'll finally figure out some --disallow-all-ub-and-be-memory-safe-and-thread-safe flag but at the moment it's still insanely trivial to make a mistake and return a reference to some value on the stack or any number of other issues.

The answer can not be "just write flawless code and you'll never have these issues" but at the moment that's all C++, and Go, from this article has.

> But: everybody understands that.

I had to convince Go people that you can segfault with Go. Or you mean the language designers with using everybody?

> People talk as if "memory safety" was a PLT axiom. It's not; it's a software security term of art.

It's been in usage for PLT for at least twenty years[1]. You are at least two decades late to the party.

    Software is memory-safe if (a) it never references a memory location outside the address space allocated by or that entity, and (b) it never executes intstruction outside code area created by the compiler and linker within that address space.

The problem Rust has is that it’s not enough to be memory safe, because lots of languages are memory safe and have been for decades.

Hence the focus on fearless concurrency or other small-scale idioms like match in an attempt to present Rust as an overall better language compared to other safe languages like Go, which is proving to be a solid competitor and is much easier to learn and understand.

One of the biggest hurdles is just getting all the iOS/macOS/etc APIs up to speed with the thread safety improvements. It won’t make refactoring all that application code any easier, but as things stand even if you’ve done that, you’re going to run into problems anywhere your code makes contact with UI code because there’s a lot of AppKit and UIKit that have yet to make the transition.

Swift 6 is only painful if you wrote a ton of terrible Swift 5, and even then Swift 5 has had modes where you could gracefully adopt the Swift 6 safety mechanisms for a long time (years?)

~130k LoC Swift app was converted from 5 -> 6 for us in about 3 days.

It is still incomplete and a mess. I don't think they thought through the actual main cases Swift is used for (ios apps), and built a hypothetical generic way which is failing on most clients. Hence lots of workarounds, and ways to get around it (The actor system). The isolated/nonisolated types are a bit contrived and causing real productivity loss, when the old way was really just 'everything ui in main thread, everything that takes time, use a dispatch queue, and call main when done'.

Swift is strating to look more like old java beans. (if you are old enough to remember this, most swift developers are too young). Doing some of the same mistakes.

Anways https://forums.swift.org/t/has-swifts-concurrency-model-gone... Common problems all devs face: https://www.massicotte.org/problematic-patterns

Anyways, they are trying to reinvent 'safe concurrency' while almost throwing the baby with the bathwater, and making swift even more complex and harder to get into.

There is ways to go. For simple apps, the new concurrency is easy to adopt. But for anything that is less than trivial, it becomes a lot of work, to the point that it might not make it worth it.

I think the surprise here is that failing to synchronize writes leads to a SEGFAULT, not a panic or an error. This is the point GP was making, that Go is not fully memory safe in the presence of unsynchronized concurrent writes. By contrast, in Java or C#, unsynchronized writes will either throw an exception (if you're lucky and they get detected) or let the program continue with some unexpected values (possibly ones that violate some invariants). Getting a SEGFAULT can only happen if you're explicitly using native code, raw memory access APIs, or found a bug in the runtime.

I thought the same thing. Maybe the point of the story isn’t “we were surprised to learn you had to synchronize access” but instead “we all thought we were careful, but each of us made this mistake no matter how careful we tried to be.”

In Java, there are separate synchronized collections, because acquiring a lock takes time. Normally one uses thread-unsafe collections. Java also gives a very ergonomic way to run any fragment under a lock (the `synchronized` operator).

Rust avoids all this entirely, by using its type system.

You really have to go hunting for a segfault in Go. The critical sentence in OP article is: in practice, of course, safety is not binary, it is a spectrum, and on that spectrum Go is much closer to a typical safe language than to C. OP just has a vested interest in proving safety of languages and is making a big deal where in practice there is none. People are not making loads of unsafe programs in Go nor deploying as such because it would be pretty quickly detected. This is much different to C and C++.

well just dont do it then

It is very unfortunate that we use fixed width numbers by default in most programming languages and that common ops will silently overflow. Smarter compilers can work with richer numeric primitives and either automatically promote machine words to big numbers or throw an error on overflow.

People talk a lot about the productivity gains of ai, but fixing problems like this at the language level could have an even bigger impact on productivity, but are far less sensational. Think about how much productivity is lost due to obscure but detectable bugs like this one. I don't think rust is a good answer (it doesn't check overflow by default), but at least it points a little bit in the vaguely correct direction.

I think the true answer is that the moment you have to do tricky concurrency in Go, it becomes less desirable. I think that Go is still better at tricky concurrency than C, though there are some downsides too (I think it's a bit easier to sneak in a torn read issue in Go due to the presence of fat pointers and slice headers everywhere.)

Go is really good at easy concurrency tasks, like things that have almost no shared memory at all, "shared-nothing" architectures, like a typical web server. Share some resources like database handles with a sync.Pool and call it a day. Go lets you write "async" code as if it were sync with no function coloring, making it decidedly nicer than basically anything in its performance class for this use case.

Rust, on the other hand, has to contend with function coloring and a myriad of seriously hard engineering tasks to deal with async issues. Async Rust gets better every year, but personally I still (as of last month at least) think it's quite a mess. Rust is absolutely excellent for traditional concurrency, though. Anything where you would've used a mutex lock, Rust is just way better than everything else. It's beautiful.

But I struggle to be as productive in Rust as I am in Go, because Rust, the standard library, and its ecosystem gives the programmer so much to worry about. It sometimes reminds me of C++ in that regard, though it's nowhere near as extremely bad (because at least there's a coherent build system and package manager.) And frankly, a lot of software I write is just boring, and Go does fine for a lot of that. I try Rust periodically for things, and romantically it feels like it's the closest language to "the future", but I think the future might still have a place for languages like Go.

> It ultimately resulted in a loop counter overflowing, which recomputed the same thing a billion of time (but always the same!). So the visible effect was a request would randomly take 3 min instead of 100ms.

This means that multiple goroutines were writing to the same local variable. I've never worked on a Go team where code that is structured in such a way would be considered normal or pass code review without good justification.

Rust does have loop counter overflow.

If the Java version creates 4 concurrent tasks (could be threads, fibers, futures, etc.) but the Go version creates 32 goroutines, that's 8x the concurrency.

The "go" keyword is that demarcation

Everyone knows that there's something very magical about the choice of 42.

If that were the case, you'd be able to support the argument with evidence.

You need a non-contrived example of a memory-corrupting data race that gives attackers the ability to control memory, through type confusion or a memory lifecycle bug or something like it. You don't have to write the exploit but you have to be able to tell the story of how the exploit would actually work --- "I ran this code and it segfaulted" is not enough. It isn't even enough for C code!

There's no "revelation" here, it's always been well known among experts that Go is not fully memory safe for concurrent code, same for previous versions of Swift. OP has simply spelled out the argument clearly and made it easier to understand for average developers.

> but a bug not being exploitable is still a bug that needs to be fixed.

Do you? Not every bug needs to be fixed. I've never see a data race bug in documented behaviour make it past initial development.

I have seen data races in undocumented behaviour in production, but as it isn't documented, your program doesn't have to do that! It doesn't matter if it fails. It wasn't a concern of your program in the first place.

That is still a problem if an attacker uses undocumented behaviour to find an exploit, but when it is benign... Oh well. Who cares?

Was it open source? Would be interested to know more.

A thread-safety issue does not always lead to a segfault. Here it did because the address written was 42, but if you somehow manage to obtain the address of some valid value then you could read from that instead, and not cause an immediate segfault.

I agree with the sentiment that data races are generally harder to exploit, but it _is possible_ to do.

All bets aren’t off, we empirically measure the safety of software based on exploits. C memory handling is most of its exploits.

Show me the exploits based on Go parallelism. This issue has been discussed publicly for 10 years yet the exploits have not appeared. That’s why it's a nice theoretical argument but does not hold up in practice.

But it's not why I stopped writing C programs. It's just a bug and I create and fix a dozen bugs every day. Security is the only argument for memory safety that moves mountains.

Originally Rust is something altogether different. Graydon has written about that extensively. Graydon wanted tail calls, reflection, more "natural" arithmetic with Python style automatic big numbers, decimal for financial work and so on.

The Rust we have from 1.0 onwards is not what Graydon wanted at all. Would Graydon's language have been broadly popular? Probably not, we'll never know.

100%.

Some more modern languages - eg. Swift – have "sendable" value types that are inherently thread safe. In my experience some developers tend to equate "sendable" / thread safe data structures with a silver bullet. But you still have to think about what you do in a broader sense… You still have to assemble your thread safe data structures in a way that makes sense, you have to identify what "transactions" you have in your mental model and you still have to think about data consistency.

> In go, it's confusing. If you use `bytes.Buffer.ReadBytes("\n")` you get a copy back, so you can send it. Same for `bytes.Buffer.String()`.

>

> But if you use `bytes.Buffer.Bytes()`

If you're experienced, it's pretty obvious that a `bytes.Buffer` will simply return its underlying storage if you call `.Bytes()` on it, but will have to allocate and return a new object if you call say `.String()` on it.

> unless you also never use that bytes.Buffer again.

I'm afraid that's concurrency 101. It's exactly the same in Go as in any language before it, you must make sure to define object lifetimes once you start passing them around in concurrent fashion.

Channels are nice in that they model certain common concurrency patterns really well - pipelines of processing. You don't have to annotate everything with mutexes and you get backpressure for free. But they are not supposed to be the final solution to all things concurrency and they certainly aren't supposed to make data races impossible.

> Even if you use channels to send things between goroutines, go makes it very hard to do so safely

Really? Because it seems really easy to me. The consumer of the channel needs some data to operate on? Ok, is it only for reading? Then send a copy. For writing too? No problem, send a reference and never touch that reference on our side of the fence again until the consumer is done executing.

Seems about as hard to understand to me as the reason why my friend is upset when I ate the cake I gave to him as a gift. I gave it to him and subsequently treated it as my own!

Such issues only arise if you try to apply concurrency to a problem willy-nilly, without rethinking your data model to fit into a concurrent context.

Now, would the Rust approach be better here? Sure, but not if that means using Rust ;) Rust's fancy concurrency guarantees come with the whole package that is Rust, which as a language is usually wildly inappropriate for the problem at hand. But if I could opt into Rust-like protections for specific Go data structures, that'd be great.

That code would never pass a human pull request review. It doesn't even pass AI code review with a simple "review this code" prompt: https://chatgpt.com/share/68829f14-c004-8001-ac20-4dc1796c76...

"2. Shared buffer causes race/data reuse You're writing to buf, getting buf.Bytes(), and sending it to the channel. But buf.Bytes() returns a slice backed by the same memory, which you then Reset(). This causes line in processData to read the reset or reused buffer."

I mean, you're basically passing a pointer to another thread to processData() and then promptly trying to do stuff with the same pointer.

I am not sure sync.Mutex fixes either of these problems. Press C-\ on a random Go server that's been up for a while and you'll probably find 3000 goroutines stuck on a Lock() call that's never going to return. At least you can time out channel operations:

   select {
   case <-ctx.Done():
          return context.Cause(ctx)
   case msg := <-ch:
          ...
   }

> it met one common definition of "memory safety", which was essentially "have a garbage collector"

This is the first time I hear that being suggested as ever having been the definition of memory safety. Do you have a source for this?

Given that except for Go every single language gets this right (to my knowledge), I am kind of doubtful that this is a consequence of the term changing its meaning.

That seems contrasted by Rob Pike's statement in 2012 in the linked presentation being one of the places where it's called "not purely memory safe". That would have been early, and Go is not called memory safe then. It seems like calling Go memory safe is a more recent thing rather than a historical thing.