Skip to content(if available)orjump to list(if available)

Zlib-rs is faster than C

Zlib-rs is faster than C

160 comments

·March 16, 2025

YZF

I found out I already know Rust:

        unsafe {
            let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10);
            xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01);
            xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0);
            xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0);
Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mattering if this is C or Rust?

Presumably with inline assembly both languages can emit what is effectively the same machine code. Is the Rust compiler a better optimizing compiler than C compilers?

Aurornis

Using unsafe blocks in Rust is confusing when you first see it. The idea is that you have to opt-out of compiler safety guarantees for specific sections of code, but they’re clearly marked by the unsafe block.

In good practice it’s used judiciously in a codebase where it makes sense. Those sections receive extra attention and analysis by the developers.

Of course you can find sloppy codebases where people reach for unsafe as a way to get around Rust instead of writing code the Rust way, but that’s not the intent.

You can also find die-hard Rust users who think unsafe should never be used and make a point to avoid libraries that use it, but that’s excessive.

chongli

Isn't it the case that once you use unsafe even a single time, you lose all of Rust's nice guarantees? As far as I'm aware, inside the unsafe block you can do whatever you want which means all of the nice memory-safety properties of the language go away.

It's like letting a wet dog (who'd just been swimming in a nearby swamp) run loose inside your hermetically sealed cleanroom.

timschmidt

It seems like you've got it backwards. Even unsafe rust is still more strict than C. Here's what the book has to say (https://doc.rust-lang.org/book/ch20-01-unsafe-rust.html)

"You can take five actions in unsafe Rust that you can’t in safe Rust, which we call unsafe superpowers. Those superpowers include the ability to:

    Dereference a raw pointer
    Call an unsafe function or method
    Access or modify a mutable static variable
    Implement an unsafe trait
    Access fields of a union
It’s important to understand that unsafe doesn’t turn off the borrow checker or disable any other of Rust’s safety checks: if you use a reference in unsafe code, it will still be checked. The unsafe keyword only gives you access to these five features that are then not checked by the compiler for memory safety. You’ll still get some degree of safety inside of an unsafe block.

In addition, unsafe does not mean the code inside the block is necessarily dangerous or that it will definitely have memory safety problems: the intent is that as the programmer, you’ll ensure the code inside an unsafe block will access memory in a valid way.

People are fallible, and mistakes will happen, but by requiring these five unsafe operations to be inside blocks annotated with unsafe you’ll know that any errors related to memory safety must be within an unsafe block. Keep unsafe blocks small; you’ll be thankful later when you investigate memory bugs."

janice1999

Claiming unsafe invalidates "all of the nice memory-safety properties" is like saying having windows in your house does away with all the structural integrity of your walls.

There's even unsafe usage in the standard library and it's used a lot in embedded libraries.

sunshowers

What language is the JVM written in?

All safe code in existence running on von Neumann architectures is built on a foundation of unsafe code. The goal of all memory-safe languages is to provide safe abstractions on top of an unsafe core.

vlovich123

You only lose those guarantees if and only if the code within the unsafe block violates the rules of the Rust language.

Normally in safe code you can’t violate the language rules because the compiler enforces various rules. In unsafe mode, you can do several things the compiler would normally prevent you from doing (e.g. dereferencing a naked pointer). If you uphold all the preconditions of the language, safety is preserved.

What’s unfortunate is that the rules you are required to uphold can be more complex than you might anticipate if you’re trying to use unsafe to write C-like code. What’s fortunate is that you rarely need to do this in normal code and in SIMD which is what the snippet is representing there’s not much danger of violating the rules.

wongarsu

If your unsafe code violates invariants it was supposed to uphold, that can wreck safety properties the compiler was trying to uphold elsewhere. If you can achieve something without unsafe you definitely should (safe, portable simd is available in rust nightly, but it isn't stable yet).

At the same time, unsafe doesn't just turn off all compiler checks, it just gives you tools to go around them, as well as tools that happen to go around them because of the way they work. Rust unsafe is this weird mix of being safer than pure C, but harder to grasp; with lots of nuanced invariants you have to uphold. If you want to ensure your code still has all the nice properties the compiler guarantees (which go way beyond memory safety) you would have to carefully examine every unsafe block. Which few people do, but you generally still end up with a better status quo than C/C++ where any code can in principle break properties other code was trying to uphold.

SkiFire13

You lose the nice guarantees inside the `unsafe` block, but the point is to write a sound and safe interface over it, that is an API that cannot lead to UB no matter how other safe code calls it. This is basically the encapsulation concept, but for safety.

To continue the analogy of the dog, you let the dog get wet (=you use unsafe), but you put a cleaning room (=the sound and safe API) before your sealed room (=the safe code world)

pdimitar

Where did you even get that weird extreme take from?

O_o

timeon

> unsafe even a single time, you lose all of Rust's nice guarantees

Not sure why would one resulted in all. One of Rust's advantages is the clear boundary between safe/unsafe.

timschmidt

Unsafe is a very distinct code smell. Like the hydrogen sulfide added to natural gas to allow folks to smell a gas leak.

If you smell it when you're not working on the gas lines, that's a signal.

mrob

There's no standard recipe for natural gas odorant, but it's typically a mixture of various organosulfur compounds, not hydrogen sulfide. See:

https://en.wikipedia.org/wiki/Odorizer#Natural_gas_odorizers

cmrdporcupine

Look, no. Just go read the unsafe block in question. It's just SIMD intrinsics. No memory access. No pointers. It's unsafe in name only.

No need to get all moral about it.

colonwqbang

Can’t rust do safe simd? This is just vectorised multiplication and xor, but it gets labelled as unsafe. I imagine most code that wants to be fast would use simd to some extent.

steveklabnik

It's still nightly-only.

api

The idea is that you can trivially search the code base for "unsafe" and closely examine all unsafe code, and unless you are doing really low-level stuff there should not be much of it. Higher level code bases should ideally have none.

It tends to be found in drivers, kernels, vector code, and low-level implementations of data structures and allocators and similar things. Not typical application code.

As a general rule it should be avoided unless there's a good reason to do it. But it's there for a reason. It's almost impossible to create a systems language that imposes any kind of rules (like ownership etc.) that covers all possible cases and all possible optimization patterns on all hardware.

timschmidt

To the extent that it's even possible to write bare metal microcontroller firmware in Rust without unsafe, as the embedded hal ecosystem wraps unsafe hardware interfaces in a modular fairly universal safe API.

formerly_proven

My understanding from Aria Beingessner's and some other writings is that unsafe{} rust is significantly harder to get right in "non-trivial cases" than C, because the semantics are more complex and less specified.

gf000

Rust's borrow checker still checks within unsafe blocks, so unless you are only operating with raw pointers (and not accessing certain references as raw pointers in some small, well-defined blocks) across the whole program it will be significantly more safe than C. Especially given all the other language benefits, like a proper type system that can encode a bunch of invariants, no footguns at every line/initialization/cast, etc.

acdha

Yes. I think it’s easy to underestimate how much the richer language and library ecosystem chip away at the attack surface area. So many past vulnerabilities have been in code which isn’t dealing with low-level interfaces or weird performance optimizations and wouldn’t need to use unsafe. There’ve been so many vulnerabilities in crypto code which weren’t the encryption or hashing algorithms but things like x509/ASN parsing, logging, or the kind of option/error handling logic a Rust programmer would use the type system to validate.

pcwalton

> Presumably with inline assembly both languages can emit what is effectively the same machine code. Is the Rust compiler a better optimizing compiler than C compilers?

rustc uses LLVM just as clang does, so to a first approximation they're the same. For any given LLVM IR you can mostly write equivalent Rust and C++ that causes the respective compiler to emit it (the switch fallthrough thing mentioned in the article is interesting though!) So if you're talking about what's possible (as opposed to what's idiomatic), the question of "which language is faster" isn't very interesting.

akx

To quote the Rust book (https://doc.rust-lang.org/book/ch20-01-unsafe-rust.html):

  In addition, unsafe does not mean the code inside the
  block is necessarily dangerous or that it will definitely
  have memory safety problems: the intent is that as the
  programmer, you’ll ensure the code inside an unsafe block
  will access memory in a valid way.
Since you say you already know that much Rust, you can be that programmer!

silisili

I feel like C programmers had the same idea, and well, we see how that works out in practice.

sunshowers

No, C lacks encapsulation of unsafe code. This is very important. Encapsulation is the only way to scale local reasoning into global correctness.

dijit

the problem in those cases is that C can’t help but be unsafe always.

People can write memory safe code, just not 100% of the time.

dietr1ch

> I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library.

Which is exactly the point, other languages have unsafe implicitly sprinkled in every single line.

Rust tries to bound and explicitly delimit where unsafe code is to makes review and verification efforts precise.

koito17

The purpose of `unsafe` is for the compiler to assume a block of code is correct. SIMD intrinsics are marked as unsafe because they take raw pointers as arguments.

In safe Rust (the default), memory access is validated by the borrow checker and type system. Rust’s goal of soundness means safe Rust should never cause out-of-bounds access, use-after-free, etc; if it does, then there's a bug in the Rust compiler.

no_wizard

How do we know if Rust is safe unless Rust is written purely in safe Rust?

Is that not true? Even validators have bugs or miss things no?

steveklabnik

> Even validators have bugs

Yep! For example, https://github.com/Speykious/cve-rs is an example of a bug in the Rust compiler, which allows something that it shouldn't. It's on its way to being fixed.

> or miss things no?

This is the trickier part! Yes, even proofs have axioms, that is, things that are accepted without proof, that the rest of the proof is built on top of. If an axiom is incorrect, so is the proof, even though we've proven it.

int_19h

Out of curiosity, why do they take raw pointers as arguments, rather than references?

steveklabnik

From the RFC: https://rust-lang.github.io/rfcs/2325-stable-simd.html

> The standard library will not deviate in naming or type signature of any intrinsic defined by an architecture.

I think this makes sense, just like any other intrinsic: unsafe to use directly, but with safe wrappers.

I believe that there are also some SIMD things that would have to inherently take raw pointers, as they work on pointers that aren't aligned, and/or otherwise not valid for references. In theory you could make only those take raw pointers, but I think the blanket policy of "follow upstream" is more important.

sesm

Rust code emitter is Clang, the same one that Apple uses for C on their platforms. I wouldn't expect any miracles there, as Rust authors have zero influence over it. If any compiler is using any secret Clang magic, that would be Swift or Objective-C, since they are developed by Apple.

nindalf

You’re conflating clang and LLVM.

sesm

Yes, you are right, should be 'code emitter is LLVM, the same that Clang uses for C'

datadeft

I thought that the point of Rust is to have safe {} blocks (implicit) as a default and unsafe {} when you need the absolute maximum performance available. You can audit those few lines of unsafe code very easily. With C everything is unsafe and you can just forget to call free() or call it twice and you are done.

steveklabnik

> unsafe {} when you need the absolute maximum performance available.

Unsafe code is not inherently faster than safe code, though sometimes, it is. Unsafe is for when you want to do something that is legal, but the compiler cannot understand that it is legal.

WD-42

It’s not about performance, it’s about undefined behavior.

cb321

I think this may not be a very high bar. zippy in Nim claims to be about 1.5x to 2.0x faster than zlib: https://github.com/guzba/zippy I think there are also faster zlib's around in C than the standard install one, such as https://github.com/ebiggers/libdeflate (EDIT: also mentioned elsethread https://news.ycombinator.com/item?id=43381768 by mananaysiempre)

zlib itself seems pretty antiquated/outdated these days, but it does remain popular, even as a basis for newer parallel-friendly formats such as https://www.htslib.org/doc/bgzip.html

JoshTriplett

The bar here is not zlib, it's zlib-ng, which aims primarily for performance.

libdeflate is an impressive library, but it doesn't help if you need to stream data rather than having it all in memory at once.

mastax

The benchmarks in the parent post are comparing to zlib-ng, which is substantially faster than zlib. The zippy claims are against "zlib found on a fresh Linux install" which at least for Debian is classic zlib.

lern_too_spel

They're comparing against zlib-ng, not zlib. zlib-ng is more than twice as fast as zlib for decompression. https://github.com/zlib-ng/zlib-ng/discussions/871

libdeflate is not zlib compatible. It doesn't support streaming decompression.

hinkley

Zlib is unapologetically written to be portable rather than fast. It is absolutely no wonder that a Rust implementation would be faster. It runs on a pathetically small number of systems by contrast. This is not a dig at Rust, it’s an acknowledgement of how many systems exist out there, once you include embedded, automotive, aerospace, telecom, industrial control systems, and mainframes.

Richard Hipp denounces claims that SQLite is the widest-used piece of code in the world and offers zlib as a candidate for that title, which I believe he is entirely correct about. I’ve been consciously using it for almost thirty years, and for a few years before that without knowing I was.

johnisgood

"faster than C" almost always boils down to different designs, implementations, algorithms, etc.

Perhaps it is faster than already-existing implementations, sure, but not "faster than C", and it is odd to make such claims.

xxs

zlib-ng is pretty much assembly - with a bit of C. There is this quote: but was not entirely fair because our rust implementation could assume that certain SIMD capabilities would be available, while zlib-ng had to check for them at runtime

zlib-ng can be compiled to whatever target arch is necessary, and the original post doesn't mention how it was compiled and what architecture and so on.

It's another case not to trust micro benchmarks

tdiff

Nevertheless Russinovich actually says something in the lines of "simple rewriting in rust made some our code 5-15% faster (without deliberate optimizations)": https://www.youtube.com/watch?v=1VgptLwP588&t=351s

pinkmuffinere

I’m sure I’m missing context, and presumably there are other benefits, but 5-15% improvement is such a small step to justify rewriting codebases.

I also wonder how much of an improvement you’d get by just asking for a “simple rewrite” in the existing language. I suspect there are often performance improvements to be had with simple changes in the existing language

sedatk

> 5-15% improvement is such a small step to justify rewriting codebases

They hadn't expected any perf improvements at all. Quite the opposite, in fact. They were surprised that they saw perf improvements right away.

turtletontine

Far better justification for a rewrite like this is if it eases maintenance, or simplifies building/testing/distribution. Taking an experienced and committed team of C developers with a mature code base, and retraining them to rewrite their project in Rust for its own sake is pretty absurd. But if you have a team that’s more comfortable in Rust, then doing so could make a lot of sense - and, yes, make it easier to ensure the product is secure and memory-safe.

tdiff

I agree that simple rewriting could have given some if not all perf benefits, but can it be the case that rust forces us to structure code in a way that is for some reason more performant in some cases?

5-15% is a big deal for a low-level foundational code, especially if you get it along with some other guarantees, which may be of greater importance.

qweqwe14

The fact that it's faster than the C implementation that surely had more time and effort put into it doesn't look good for C here.

johnisgood

It says absolutely nothing about the programming language though.

acdha

Doesn’t it say something if Rust programmers routinely feel more comfortable making aggressive optimizations and have more time to do so? We maintain code for longer than the time taken to write the first version and not having to pay as much ongoing overhead cost is worth something.

vkou

I think you'll find that if you re-write an application, feature-for-feature, without changing its language, the re-written version will be faster.

renewiltord

This is known as the Second System Effect: where Great Rewrites always succeed in making a more performant thing.

layer8

If anything, this should be “zlib-rs is faster than zlib-ng”, but not “$library is faster than $programming_language”.

chjj

It should be, but you'll never convince the rust people of that. It's always a competition with them.

kgeist

I heard that aliasing in C prevents the compiler from optimizing aggressively. I can believe Rust's compiler can optimize more aggressively if there's no aliasing problem.

layer8

C has the restrict type qualifier to express non-aliasing, hence it shouldn’t be a fundamental impediment.

gf000

Which is so underused that the whole compiler feature was buggy as hell, and was only recently fixed because compiling Rust where it is the norm exposed it.

null

[deleted]

oneshtein

... because by "C" we mean handwritten inline assembler.

Typical realworld C code uses \0 terminated strings and strlen() with O(len^2) complexity.

jrockway

Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this by picking a better algorithm. Zstandard is faster and compresses better. LZ4 is much faster, but not quite as small.

Some reading: https://jolynch.github.io/posts/use_fast_data_algorithms/

(As an aside, at my last job container pushes / pulls were in the development critical path for a lot of workflows. It turns out that sha256 and gzip are responsible for a lot of the time spent during container startup. Fortunately, Zstandard is allowed, and blake3 digests will be allowed soon.)

jeffbee

Yeah I just discovered this a few days ago. All the docker-era tools default to gzip but if using, say, bazel rules_oci instead of rules_docker you can turn on zstd for large speedups in push/pull time.

IshKebab

It's barely faster. I would say it's more accurate to say it's as fast as C, which is still a great achievement.

throwaway48476

But it is faster. The closer to theoretical maximum the smaller the gains become.

mananaysiempre

Zlib-ng is between a couple and multiple times away from the state of the art[1], it’s just that nobody has yet done the (hard) work of adjusting libdeflate[2] to a richer API than “complete buffer in, complete buffer out”.

[1] https://github.com/zlib-ng/zlib-ng/issues/1486

[2] https://github.com/ebiggers/libdeflate

ajross

It's... basically written in C. I'm no expert on zlib/deflate or related algorithms, but digging around https://github.com/trifectatechfoundation/zlib-rs/ almost every block with meaningful logic is marked unsafe. There's raw allocation management, raw slicing of arrays, etc... This code looks and smells like C, and very much not like rust. I don't know that this is a direct transcription of the C code, but if you were to try something like that this is sort of what it would look like.

I think there's lots of value in wrapping a raw/unsafe implementation with a rust API, but that's not quite what most people think of when writing code "in rust".

gf000

C is not assembly, nor is it portable assembly at all in this century, so your phrasing is very off.

C code will go through a huge amounts of transformations by the compiler, and unless you are a compiler expert you will have no idea how the resulting code looks. It's not targeting the PDP-11 anymore.

hermanradtke

> basically written in C

Unsafe Rust still has to conform to many of Rust’s rules. It is meaningfully different than C.

est31

It has also way less tooling available than C to analyze its safety.

ajross

Are there examples you're thinking about? The only good ones I can think of are bits about undefined behavior semantics, which frankly are very well covered in modern C code via tools like ubsan, etc...

xxs

I mentioned in under another comment - and while I consider myself versed enough in deflate - comparing the library to zlib-ng is quite weird as the latter is generally hand written assembly. In order to beat it'd take some oddity in the test itself

johnisgood

It does actually seem like what a C -> Rust transpiler would spit out.

oneshtein

Cannot understand your complain. It written in Rust, but for you it looks like C. So what?

Alifatisk

So, it is basically like it was written in C.

ajross

It doesn't exploit (and in fact deliberately evades) Rust's signature memory safety features. The impression from the headline is "Rust is as fast as C now!", but in fact the subset of the language that has been shown to be as fast as C is the subset that is basically isomorphic to C.

The impression a naive reader might take is that idiomatic/safe/best-practices Rust has now closed the performance gap. But clearly that's not happening here.

qweqwe14

"Barely" or not is completely irrelevant. The fact is that it's measurably faster than the C implementation with the more common parameters. So the point that you're trying to make isn't clear tbh.

Also I'm pretty sure that the C implementation had more man hours put into it than the Rust one.

bee_rider

I think that would be really hard to measure. In particular, for this sort of very optimized code, we’d want to separate out the time spent designing the algorithms (which the Rust version benefits from as well). Actually I don’t think that is possible at all (how will we separate out time spent coding experiments in C, then learning from them).

Fortunately these “which language is best” SLOC measuring contests are just frivolous little things that only silly people take seriously.

akagusu

Bravo. Now Rust has its existence justified.

kahlonel

You mean the implementation is faster than the one in C. Because nothing is “faster than C”.

gf000

Wtf, since when?

Besides the famous "C is not a low-level language" blog post.. I don't even get what you are thinking. C is not even the performance queen for large programs (the de facto standard today is C++ for good reasons), let alone for tiny ultra hot loops like codecs and stuff, which are all hand-written assembly.

It's not even hard to beat C with something like Rust or C++, because you can properly do high level optimizations as the language is expressive enough for that.

nindalf

Why can’t something be faster than C? If a language is able to convey more information to a backend like LLVM, the backend could use that to produce more optimised code than what it could do for C.

For example, if the language is able to say, for any two pointers, the two pointers will not overlap - that would enable the backend to optimise further. In C this requires an explicit restrict keyword. In Rust, it’s the default.

By the way this isn’t theoretical. Image decoders written in Rust are faster than ones written in C, probably because the backend is able to autovectorise better. (https://www.reddit.com/r/rust/comments/1ha7uyi/memorysafe_pn...).

grep (C) is about 5-10x slower than ripgrep (Rust). That’s why ripgrep is used to execute all searches in VS Code and not grep.

Or a different tack. If you wrote a program that needed to sort data, the Rust version would probably be faster thanks to the standard library sort being the fastest, across languages (https://github.com/rust-lang/rust/pull/124032). Again, faster than C.

Happy to give more examples if you’re interested.

There’s nothing special about C that entitles it to the crown of “nothing faster”. This would have made sense in 2005, not 2025.

burntsushi

Narrow correction on two points:

First, I would say that "ripgrep is generally faster than GNU grep" is a true statement. But sometimes GNU grep is faster than ripgrep and in many cases, performance is comparable or only a "little" slower than ripgrep.

Secondly, VS Code using ripgrep because of its speed is only one piece of the picture. Licensing was also a major consideration. There is an issue about this where they originally considered ripgrep (and ag if I recall correctly), but I'm on mobile so I don't have the link handy.

dijit

The kind of code you can write in rust can indeed be faster than C, but someone will wax poetic about how anything is possible in C and they would be valid.

The major reason that rust can be faster than C though, is because due to the way the compiler is constructed, you can lean on threading idiomatically. The same can be true for Go, coroutines vs no coroutines in some cases is going to be faster for the use case.

You can write these things to be the same speed or even faster in C, but you won’t, because it’s hard and you will introduce more bugs per KLOC in C with concurrency vs Go or Rust.

Jaxan

Of course many things can be faster than C, because C is very far from modern hardware. If you compile with optimisation flags, the generated machine code looks nothing like what you programmed in C.

pornel

If you don't count manual SIMD intrinsics or inline assembly as C, then Rust and FORTRAN can be faster than C. This is mainly thanks to having pointer aliasing guarantees that C doesn't have. They can get autovectorization optimizations where C's semantics get in the way.

kllrnohj

It is quite easy for C++ and Rust to both be faster than C in things larger than toy projects. C is hardly a panacea of efficiency, and the language makes useful things very hard to do efficiently.

You can contort C to trick it into being fast[1], but it quickly becomes an unmaintainable nightmare so almost nobody does.

1: eg, correct use of restrict, manually creating move semantics, manually creating small string optimizations, etc...

arlort

Tachyons?

einpoklum

Maybe if you reverse the beam polarity and route them through the main deflector array.

layer8

But that requires rerouting auxiliary power from life support to the shield generators. In Rust you would need to use unsafe for that.

mkoubaa

C after an optimizing compiler has chewed through it is faster than C

water9

[flagged]

berkes

This sounds embittered. But if it isn't that, what problem with rust do you see?

qweqwe14

> Add a borrow checker to C++ and put rust to bed once and for all

Ah yes, C++ is just one safety feature away from replacing Rust, surely, any moment now. The bizzare world C++ fanboys live in.

Every single person that had been writing C++ for a while and isn't a victim of Stockholm syndrome would be happy when C++ is put to bed once and for all. It's a horrible language only genuinely enjoyed by bad programmers.

amorio2341

Not surprised at all, Rust is the future.

null

[deleted]

kccqzy

While AI can certainly produce code that's faster or otherwise better than human-written code, I am utterly skeptical of LLMs doing that. My own experience with LLM is that humans can do everything they do, but they are faster than humans. I believe we should look at non-LLM AI technologies for going beyond what a skilled human programmer can expect to do. The most famous example of AI doing that is https://www.nature.com/articles/s41586-023-06004-9 where no LLM is involved.