Bloat is still software's biggest vulnerability (2024)

I feel like "shouldn't be implementing your own authentication" is overblown. Don't write the crypto algorithms. But how hard is it to write your own auth? If you are pulling in a third party dependency for that you still would need to audit it, and if you can audit authentication software why can't you implement it?

Just follow OWASP recommendations. A while back this was posted to HN and it also provides great recommendations: https://thecopenhagenbook.com/ .

There have been major F-ups in recent history with Okta, CrowdStrike, and so on. Keycloak had some major long-standing vulnerabilities. I've had PRs accepted in popular open-source IAM libraries a bit too easily.

Yeah, we shouldn't roll our own cryptography, but security isn't as clean cut as this comment implies. It also frequently bleeds into your business logic.

Don't confuse externalizing security with externalizing liability.

As far as I know tacking on security after the fact usually leads to issues. It should be a primary concern from the beginning. Even if you don't do it 100% right, you'd be surprised how many issues you can avoid by thinking about this during (and not after) development.

Dropping your rights to open files as soon as possible, for example, or thinking about what information would be available to an attacker should they get RCE on the process. Shoehorning in solutions to these things after the fact tends to be so difficult that it's a rare sight.

I have been recommended to think of security as a process rather than an achievable state and have become quite fond of that perspective.

You are describing domain-driven design. Outsource generic subdomains, focus your expertise on the core subdomains.

https://blog.jonathanoliver.com/ddd-strategic-design-core-su...

I think this helps, but I also think the default for any dev (particularly library authors) should be to minimize dependencies as much as possible. Dependencies have both a maintenance and a security cost. Bad libraries have deep and sprawling trees.

I've seen devs pull in frameworks just to get access to single simple to write functions.

Even if you make the obviously wrong assumption that every library is more secure than the one you would write (that will do less the vast majority of the time) We still end up in a eggs in one basket situation.

You haven't thought through any cyber security games or you are funded to post this bad argument over and over again by state agencies with large 0-day stockpiles.

I would like to dig into point 2 a bit. Do you think this is a matter of degree, or of kind? Does security, in this, imply a network connection, or some other way that exposes your application to vulnerabilities, or is it something else? Are there any other categories that you would treat in a similar way as security, but to a lesser degree, or that almost meet that threshold for a special category, but don't?

>> Look for example at Qt, it is supposed to be a UI framework but it ends up implementing vectors, strings, json parser and who knows how much more stuff. But it's just 1 dependency so it's fine, right?

Qt is an application development framework, not a GUI toolkit. This is one reason I prefer GTK (there are things I dislike about it too).

There is still an advantage to using Qt over dozens of libraries that offer the same functionality.

Qt is backed by a single company, so all you have to watch out for is that company. Also, Qt is generally high quality, I have worked with it, read the source code, etc... and I generally liked what I saw. So I can reasonably assume that quality is consistent overall. When you have many libraries from many independent developers, it doesn't work. The JSON parser may be good, but it doesn't tell me anything about the library that deal with internationalization for instance, and if I wanted to keep track of everything, that's several time the work compared to a single vendor.

I agree that Qt is bloated though, but multiplatform UI frameworks are hard to keep light. There is a lot going on in a desktop UIs that people only notice when it isn't there. I tend to treat them like I treat the standard libraries, the OS, and for web apps, the browser. Big components, but you reasonably can't do without.

How many vulnerabilities were due to badly reinventing the wheel in C/C++ though?

I don't know. Suppose you tell us.

It definitely does solve one problem. Like it or not, you can't be hit by supply chain attacks if you don't have a supply chain.

> every developer, company, team, sub-team, etc has their own "library" of random functions, utilities, classes, etc

You are right. But my conclusion is different.

If it is a stable and people have been there for a while then developers know that code as well as the rest. So, when something fails they know how to fix it.

Bringing generic libraries may create long callstacks of very generic code (usually templates) that is very difficult to debug while adding a lot of functionality that is never used.

Bringing a new library into the code base need to be a though decision.

> In my experience every developer, company, team, sub-team, etc has their own "library" of random functions, utilities, classes, etc that just end up being included into new projects sooner or later

Same here. And a lot of those homegrown functions, utilities and classes are actually already available, and better implemented, in the C++ Standard Library. Every C++ place I've worked had its own homegrown String class, and it was always, ALWAYS worse in all ways than std::string. Maddening. And you could never make a good business case to switch over to sanity. The homegrown functions had tendrils everywhere and many homegrown classes relied on each other, so your refactor would end up touching every file in the source tree. Nobody is going to approve that risky project. Once you start down the path of rolling your own standard library stuff, the cancer spreads through your whole codebase and becomes permanent.

When doing SBOM/SCA we see apps with 1000+ deps. It's insane. It's so often we see large packages pulled in because a single function/behavior is needed and ends up massively increasing the risk profile.

Cathedrals vs Bazaars.

Cathedrals are conservative. Reactionary, even. You can measure the rate of change by generations.

Bazaars are accessible and universal. The whole system is chaotic. Changes happen every day. No single agent is in control.

We need both to make meaningful progress, and it's the job of engineers to take any given problem and see where to look for the solution.

> While the idea of being able to trivially share code worldwide is appealing, so far it seems to encourage shittier software more than anything else, and the benefit of sharing trivially seems to be defeated by the downsides that bloat and bad software bring with it.

A lot of projects would simply not exist without it. Linux, comes to mind. I guess one might take the position that "Windows is fine" but would there ever have been even competition for Windows?

Another example, everyone would be rolling their own crypto without openssl, and that would mean software that's yet a lot more insecure than what we have. Writing software with any cryptography functionality in mind would be the privilege of giant companies only (and still suck a lot more than what we have).

There's a lot more things. The internet and software in general would be set back ~20years. Even with all the nostalgia I can muster, that seems like a much worse situation than today.

This is all heuristic (read "guessing") and not a real solution to the problem.

The ground truth is that software bloat isn't bad enough of a problem for software developers to try and fight it. We already know how to prevent this, if really want to. And if the problem was really hurting so much, we'd have automated ways of slimming down the executables / libraries.

In my role in creating CI for Python libraries, I did more hands-on dependency management. My approach was to first install libraries with pip, see what was installed, research why particular dependencies have been pulled in, then, if necessary, modify the packages in such a way that unnecessary dependencies would've been removed, and "vendor" the third party code (i.e. store it in my repository, at the version I need). This, obviously, works better for programs, where you typically end up distributing the program with its dependencies anyways. Less so for libraries, but in the context of CI this saved some long minutes of reinstalling dependencies afresh for every CI run.

In the end, it was a much better experience than what you usually get with CI targeting Pyhon. But, in the end, nobody really cared. If CI took less than a minute to complete instead of twenty minutes, very little was actually gained. The project didn't have enough CI traffic for this to have any actual effect. So, it was a nice proof of concept, but ended up being not all that useful.

The phrase "cheap and nasty" comes to mind. Over time, some markets tend towards the cheap and nasty.

That's not how it works.

Here's a scenario. You pull in some library - maybe it resizes images or something. It in turn pulls in image decoders and encoders that you may or may not need. They in turn pull in metadata readers, and those pull in XML libraries to parse metadata, and before you know it a fairly simple resize is costing you 10s of MB.

Worse, you pull in different libraries and they all pull in different versions of their own dependencies, with lots of duplication of similar but slightly different code. Node_modules usually ends up like this.

The point is not writing the resize code yourself. It's the cultural effect of friction. If pulling in the resize library means you need to chase down the dependencies yourself, first, you're more aware of the cost, and second, the library author will probably give you knobs to eliminate dependencies. Perhaps you only pull in a JPEG decoder because that's all you need, and you exclude the metadata functionality.

It's an example, but can you see how adding friction to pulling in every extra transitive dependency would have the effect of librabry authors giving engineers options to prune the dependency tree? The easier a library is to use, the more popular it will be, and a library that has you chasing dependencies won't be easy to use.

There's a difference between "I need to connect to the database and I need to parse json, so I need two commonly used libs for those two things" and whatever npm is doing, and to some extent cargo or popular java frameworks are doing.

Building everything from scratch is insane, but so's uncritically growing a dependency jungle

I feel you are arguing a bit of a strawman. The take is much more nuanced than write everything from scratch.

That wheel is only as good as your skill in making it. For many people (the majority i'd guess) someone else making that wheel will have a better end result.

>many Apple UX choices prioritize user perceived performance instead of real performance.

Then why does Apple still ship 60Hz displays in 2025? The perceived performance on scrolling a web page on 60Hz is jarring no matter how performant your SoC is.

Alpine is not as good as it seems. It's mostly broken it just works when you ask it to run a handful of common tools. Everything out of view is completely broken.

I disagree. Most people here myself included aren't using Java or .NET. You are in a microcosm in this audience.

> I think we lost something with static linking when going from C to Dotnet. (And I guess Java.) Many C (and C++, especially "header only") libraries when statically linked are pretty good at filtering out unused code.

This is an interesting statement because, for example, in C version of Mimalloc you end up paying for opt-in assertions because they still exist in the code unless you compile a different version that strips them away. In C# port, you can set the same assertions/checks early with AppContext switch, and then the values will be cached in static readonly fields. Then, when JIT recompiles the code to a more optimized version, these values will become JIT constants leading to all the unreachable code to be optimized away completely (and to much better inlining of now streamlined methods).

> Even when generating a self contained EXE, the standard toolchain performs no end-linking of the program, it just bundles everything up in one file.

  /p:PublishTrimmed=true

  /p:PublishAot=true # please note it's better to set it as a project property, but either way it requires non-optional linking

There is more to size than storage space. Larger applications take more memory, more cpu caches; things spill over to normal memory, latencies grow and everything runs much slower

We do use docker compose (thank god) but I also need to run a server from source for most of the microservices in order to modify and debug the code. There are around 20 something containers in practice, 6 pods/services. All interdependent and necessary to run the product (it's a legacy codebase 10+ years old, I joined less than 1 year ago and had nothing to do with architecture decisions). Most features touch on at least 3 to 4 repos/microservices all impossible to decouple. The problem is really opening and launching code across 6 bash consoles some of which require an additional manual authentication step with various cloud providers. I need the ability to restart some independently after making code changes. It's just a very complicated system.

I'm sure the launch can be fully automated but it's kind of at the edge of not worth automating because of how relatively infrequently I need to restart everything... Also the CEO doesn't like to make time for work which doesn't yield visible features for end users.

I actually handed my resignation a month ago, without another job lined up. It became too much haha. Good practice though. Very stressful/annoying.

Completely worth it IMO. My philosophy nowadays (on my side projects) is to make every software feel like a complete product that you can run out of the box, batteries included... I also try to support older engine versions to avoid setup issues.

If you take care of the developer, the project looks after itself.

Requirement. Had to use Azure too. I use Linux at home.

Look at their code, from projects or any open source contributions. Ask how they intend to write secure code, rather than asking a bunch of useless algorithmic problems

Back in 2004 I was regularly setting up Apache and IIS servers with 80 or more applications running on them simply by being systematic about how they were configured. In 2014 somebody wants to sell it back to me with 10x the disk I/O and a lot more that can go wrong, no thanks!

There are some places where people really want to run 8 versions of Java and 3 versions of PHP and think it's going to make them productive that they can write 15 microservices in 15 different languages... It's a delusion. If you get purposeless variation of variances in your system in control you are in control and have a huge competitive advantage over 10x larger teams who use tools that let them barrel on without being in control.

Exactly

And because it is based on nothing, you can just lie about it