Build It Yourself

How is Debian / Ubuntu secure?

It's signed by a maintainer. And maintainers are vetted. You trust Debian/Ubuntu to only allow trustworthy people to sign packages.

How are Docker / Python / Rust secure? I don't know any of the people who created my docker images, PyPi packages, or Rust crates.

Yes.

We're basically back to sending around EXE and DLL files in a ZIP. It's just that now we call it a container and proudly start it as root.

BTW, I agree with the author of the article: Sometimes you're best off just merging dependency source code. It used to be called "vendoring" and was a big thing in Rails / Ruby. The big advantage is that you're not affected by future malicious package takeovers. But you can still merge security patches from upstream, if you choose to do that.

I genuinely don't know what people with this opinion work on that they can so easily choose to completely re-invent the wheel in every project, just because they're afraid of hypothetical dangers. Most of which are still present for private re-implementations (like bugs).

I cannot think of a single project where NIH syndrome would've been a net positive. Even dependencies that aren't "essential" are super helpful in saving time.

When you recreate parts of an "optional" dependency for every single project, how do you find the time to fix all the extra bugs, the edge cases, the support for different versions of lower level dependencies, of different platforms?

That's what I say when I converse about my colleagues: having a package manager from the start that greatly lowers the friction of adding or publishing yet another package deprives a language's ecosystem from something very useful: a good measure of natural selection.

Add to that a free-for-all/no curation repository situation like pypi, npm or cargo together with a too small standard library and prepare to suffer.

So, the feature of "Here's a set of easily pulled libraries" is an anti-feature because it makes it easy to pull supporting libraries? I suspect this is actually about developers and not Rust or JS. Developers choose what dependencies to pull. If nobody pulled a dependency it would wither and die. There are a lot of dependencies for most libraries because most developers prefer to use dependencies to build something.

But I digress. If we're talking build system, nobody is forcing you to use Crates.io with cargo, they just make it easy to. You can use path-based dependencies just like CMake/VCPkg,Conan, or you can DIY the library.

Even with crates.io, nobody is forcing you to not use version pinning if you want to avoid churn, but they just make it easy to get latest.

It's easy to build software on existing software in Rust. If you don't like the existing software or the rate it changes don't blame Cargo. Just do it the way you like it.

> People will complain that "it's hard to add a dependency to a C++ project"

The way I see it the issue is that it's hard to add a dependency _in such a way that no people will have issues building your project with it_. This is problematic because even if you manage to make it work on your machine it may not work on some potential user or contributor's.

> But that just destroys the OS package distribution model: when I choose a Linux distribution, I choose to trust those who build it.

Distros still build Rust packages from sources and vendor crate dependencies in their repos. It's more painful because there are usually more dependencies with more updates, but this has nothing to do with shared libraries.

There's a corollary here to "build it yourself", which is "vet it yourself". Cargo, npm, and pip all default to a central registry which you're encouraged to trust implicitly, but we've seen time and time again that central registries do not adequately protect against broken or malicious code that causes major security flaws downstream. Each of these ecosystems trains its developers to use hundreds of dependencies—far more than they can personally vet—with the understanding that someone else must surely have done so, even though we've seen over and over again that the staff of these registries can't actually keep up and that even long-running and popular projects can suddenly become insecure.

I'd like to see an ecosystem develop that provides a small amount of convenience on top of plain old vendoring. No central repository to create a false sense of security. No overly clever version resolution scheme to hide the consequences of libraries depending on dozens of transitive dependencies. Just a minimal version resolution algorithm and a decentralized registry system—give each developer an index of their own libraries which they maintain and make downstream developers pick and choose which developers they actually trust.

Maybe a bit like Maven if Maven Central didn't exist?

I agree. It's really nice how few dependencies get pulled into a typical c++ project. When I started playing with rust, I was shocked at at how many dependencies got pulled in to just build hello world. I'm just not interested in adopting the npm left-pad model.

> In C++, I control my dependencies. In Rust I get above 100 so quickly I just

Just don't add dependencies, it's that simple. If you have enough time to control C++ dependencies, you can control them in Rust as well.

This was vaguely my take away from the article: It's not that his replacements are simpler because they're better or made by him. They're simpler because they're only handling his use-cases.

Sometimes - that's fine.

Sometimes - that's making his software worse for folks who have different use-cases, or are running on systems he doesn't understand or use himself.

The real value of a library, even with all those dependencies (and to be clear, I disagree that 3 or 4 dependencies for a library that runs across windows/linux is "all that many", esp when his platform specific implementation still uses at least 1), is that it turns out even relatively simple problems have a wealth of complexity to them. The person who's job it is to write that library is going to be more experienced in the subject domain than you (at least in the good cases) and they can deal with it. Most importantly - they can deal with your unknown, unknowns. The places you don't even have the experience to know you're missing information.

It's not standardized but those calls do not change. The windows calls in particular are guaranteed ABI stable since they are compiled into a lot of binaries. There are definitely issues with ioctl but the changes landing in terminal-size or any of the dependencies that caused all these releases, are entirely unrelated to ioctl/TIOCGWINSZ constants/winsize struct. That code hasn't changed.

In this case the terminal-size crate just calls Rustix tcgetwinsize, which in turn just calls the libc tcgetwinsize. So I suppose you could save yourself a whole bunch of dependencies by just doing the same yourself. The only cost is Windows support.

If this particular API has been stable, or at least reasonably defined for 50 or 25 years is a detail, because the dependency doesn't even pretend to deal with that and the function is unlikely to change or be removed in the near future.

Terminals are a good example of something that seems really simple but is a major PITA because of too many different vendors in the early days, and no industry standard emerged. What is the closest thing? VT100? VT102? I mostly write raw to those, but stuff like terminal size and various other features like raw (non-cooked) mode are crappy and require ioctl's and such. Frankly, it sucks.

...but the libraries suck even more! If you don't want to link against ncurses then may God have mercy on your soul.

Go and the C linux world have sold me on the fat library philosophy. You brought a library to solve a problem in a domain, then add your specific bits. You don't go and bring a dependency for each item in your check list. Yes there may be duplicate effort, but the upgrade path is way easier.

Most of that new CI/CD tech is standard now precisely because of all the complexity added by maintaining third-party dependencies and constant changes in the runtime environment. This isn't a problem for the most part for an old LAMP application deployed by scp.

While I might agree with you on some points, I don't want to spend time reinventing the wheel and introduce bugs into it.

Take for example standard communication message formats like FHIR or HL7. You definitely don't want to implement the whole definitions for the standard, which is already complicated.

Writing Cryptographic functions by yourself is also typically a shot in your foot, has proved in all these years of found critical security issues.

We live in a time where you want to actual solve a business problem, by focusing on the problem and not on how the solution is built properly. With the advent of AI this is even more critical, since all the code feels like stitched together blindly.

Spending time on developing all by yourself might give you a good shot in the long run, but first you need to survive the competition, who maybe has already caught the market, by using fast and throw-away code at the beginning.

> my day job, wherein we run scores of Spring Boot apps written in Java 8 that have pages of vulnerabilities from tens of dozens of dependencies, which are not easy to update because updating one library necessitates updating many other libraries, and oh my goodness, the transitive dependencies.

At my job we have a fairly strict static analysis policy and starting in April it is going to get even more strict.

Have you looked at https://docs.openrewrite.org/ to automatically upgrade your dependencies?

I just migrated from Java 8, Spring Boot2 and Swagger to Java 17, Spring Boot 3.3 and OpenApi 3. It was pretty painless.

Now, I still have update some dependencies and transient dependencies but the biggest hurdles were taken care of by the migrations.

An important point here is that the transitive dependency issue completely does not exist in Rust. If you upgrade a crate to a version which upgrades its public dependency, i.e. it uses it in its APIs and you need to interact with it to interact with those APIs, then you obviously need to upgrade your copy of the subdependency at the same time. But private transitive dependencies are totally irrelevant unless they link to C libraries. You can have as many semver-incompatible versions of a crate in the same dependency tree as you want, and even depend on multiple versions directly if you need to. No Java-style sweeping upgrades are ever needed, just upgrade the one thing with the vulnerability. (I believe C# has the same feature, though it's a little more baroque about it.)

I’m curious if the OpenRewrite project has any value to you in keeping your Java stuff up to date?

(I’m not affiliated with it; just curious about strategies for upgrading and maintaining apps that use big frameworks.)

This is a problem for Node, but it isn't a problem for Rust. You don't need any dependencies to 'like' other dependencies. You can have all the versions at the same time and nothing breaks.

Certainly nodejs and npm getting out of hand was a wakeup call for me, about 15 years ago.

I started to view it has a personal failure when I brought in that first dependency, which meant needing all of the packaging and organization just to use that first package.json, rather than just having plain js loaded from a script tag.

One big reason is because Go has a very nice complete standard library, and Rust really does not.

Things you can find in go’s built in to the language or in standard libraries that need a dependency in Rust:

- green threads

- channels

- regular expressions

- http client

- http server

- time

- command line flags

- a logger

- read and write animated GIFs

I don’t love the Go language, but it’s the leader for tooling and standard library, definitely the best I’ve used.

For Rust and Go in particular, the difference is in the standard library. The Rust stdlib is (intentionally) small.

Go's vast standard library helps a lot with keeping dependency numbers down

I’ve definitely become much more likely to start with small internal utility functions implemented by AI before adding a library than I would have been in the past—it’s quite effective for contained problems, I can have the AI write much more complete/robust implementations than I would myself, and if I do eventually decide to add a library I have a natural encapsulation: I can change the implementation of my own functions to use the library without necessarily having to touch everywhere it’s being used. Makes it easy to test a couple of different libraries as well, when the time comes.

In Rust, a library can define "features" that can be conditionally enabled or disabled when depending on it, which give a a built-in way to customize how much of the library is actually included. Tokio is a great example of this; people might be surprised to learn that the total number of direct dependencies that are required by tokio is only two[1]; everything else is optional!

Unfortunately, it doesn't seem like people are super diligent about looking into the default feature set that's used by their dependencies and proactively trimming that down. It doesn't help that the syntax for pulling in extra features is less verbose than removing optional but default ones (which requires both specifying "no-default-features" and then manually adding every one of the default features that you do still want back to the list of ones you pull in), and it _really_ doesn't help that the only way for libraries to expose the ability to prune unneeded features from their own dependencies to the users who inherit them is by manually making their own feature that maps to the features of every single one of their own dependencies. For example, if you're writing a library with five dependencies, and every one of them has one required dependency and four optional ones, giving the users of your library full control over what transitive features they pull in would mean making 20 features in your own library mapping to each of those transitive features, and that's not even counting the ones that you'd want to make for your own code in order to be a good citizen and not force downstream users to include all of your own code.

More and more I'm coming to the opinion that the ergonomics around features being so much worse for trying to cut down on the bloat is actually the catalyst for a lot of the issues around compile times in Rust. It doesn't seem to be super widely discussed when things like this come up though, so maybe it's time that I try to write a blog post or something with my strong feelings on this so at least I'll have something to point to assuming the status quo continues indefinitely.

[1]: https://github.com/tokio-rs/tokio/blob/ee19b0ed7371b069112b9...

That's how we get things like leftpad in the JS ecosystem.

On one side, I think if we had a good system of trust, that's not a problem.

And part of me likes the idea of something like Shadcn - you like a component? Copy it into your library. However, if there ends up being a vulnerability, you have no idea if you are affected.

For some code, that's not a problem. For other code, we truly depend on having as many eyes as possible on it.

Wow. For those who don’t know, here’s a pseudocode implementation:

    Median(list) {
      let len = length(list)
      if len % 2 == 0 {
        let x = floor(len/2)
        return (list[x] + list[x+1]) / 2
      }
      return list[len/2]
    }

Managed to resist calling is_odd there!

I've often thought the same. I would love a depedency manager that not only surfaced this information, but required you to declare upfront what level your library is.

I think it would reign in the bloat.

Software that hasn’t been touched in ten years and still does the job is about as ideal as a dependency can be.

Or they bait-and-switch freedom.

So they start off with an open source Free solution, and then later switch to a paid for model, and abandon the Free version. This is particularly painful when it's a part of your system that's key to security.

You're left between wondering if you should just pay the ransom or switching to a different solution entirely, or gamble and leaving it on an old unpatched version.

( Looking at you, IdentityServer )

Either way you regret ever going with them.

> I'm just annoyed by the lack of support

You know you can hire someone to support them right?