Skip to content(if available)orjump to list(if available)

How good engineers write bad code at big companies

area51org

Maybe it's things like 4-year tenure, or shorter tenure, or something else.

But I think it's a matter of motivation, Bob.

> The thing is, Bob, it's not that I'm lazy, it's that I just don't care. It's a problem of motivation, all right? Now if I work my ass off and Initech ships a few extra units, I don't see another dime, so where's the motivation? ... my only real motivation is not to be hassled. That, and the fear of losing my job. But you know, Bob, that will only make someone work just hard enough not to get fired.

mikert89

what I see alot is that the syntax and overall code architecture is text book, but its the completely wrong approach that creates extremely complicated tech debt. All the code reviews will be on the syntax, and none on the big picture of the business problem, or whether the implementation is overcomplicated.

in the short run (1-2 years) there is no repercussion for this, but eventually making changes will be extremely risky and complicated. The individuals that built the software will lord over everyone else with their arcane knowledge of this big pile of junk

bad_haircut72

100% this. Stuff like database schemas gets comitted in the first sprint and never gets refactored, which completely locks you in to long term design decisions, then every subsequent PR will get held up for days in arguments around meaningless "code quality" arguments which ultimately affect nothing

mikert89

ive never actually seen someone get fired for making some deep architectural software mistake. its alway for moving too slow, or "low code quality". i think people that were promoted for building systems that turned out bad, should be demoted

jjmarr

Because businesses, as a rule, value moving fast. Being first to market makes money and generally results in winning.

Oftentimes the circumstances are "we don't know the requirements", not because of shitty management, but because the problem is inherently hard to define.

The business conditions that do heavily penalize bad architectural decisions, like physical structural engineering, can suck to work in compared to SWE.

It takes a decade or more before you're trustworthy enough to architect a building and there's a million layers of approvals. Then it takes years before groundbreaking, and years more as the building increases in size.

Your whole life might be dominated by a single large project like Hudson Yards, which has been floating around as an idea since 1956. The most recent proposal started in 2006, broke ground in 2012, and another 6+ years to finish. Then when companies were about to move their offices there, COVID-19 happened and the leases fell through.

Personally I'd rather ship new stuff every few years.

dave_sid

You can’t often demote them because usually the people responsible for bad initial design decisions left the company years ago with a desperate need to go and start a new mess somewhere else.

skhameneh

> i think people that were promoted for building systems that turned out bad, should be demoted

Nope, in the same vein of "lording" over others, they become the expert of knowledge of bullshit. The environments that allow such behavior have already engrained reward of such behavior.

rented_mule

I've seen too much of the same. It strikes me that the pattern you describe also matches a lot of AI generated code I see, especially when it's big chunks of generated code. Are we automating this problem and going all-in on the long term costs?

jeltz

> Are we automating this problem and going all-in on the long term costs?

I feel that is a very likely scenario.

Herring

> Big companies know that treating engineers as fungible and moving them around destroys their ability to develop long-term expertise in a single codebase. That’s a deliberate tradeoff. They’re giving up some amount of expertise and software quality in order to gain the ability to rapidly deploy skilled engineers onto whatever the problem-of-the-month is.

And also to "keep the balance of power tilted away from engineers and towards tech company leadership." The author touched on that and forgot about it. You don't want key projects depending on a group of engineers that might get hit by a bus or unionize or demonstrate against Israel or something. Network effects and moats and the occasional lobbying/collusion mean the quality of your product is less important.

zanellato19

Yeah, this is a deliberate choice to make labor less powerful. Capital is willing to be less efficient for that. He does touch upon this by saying that Capital wants every worker to be replaceable.

bdangubic

if I learned anything in my (too) long career is that one should do everything possible to ensure that whoever pays you needs you more than you need the money they are paying you. it is not easy to get there right away but if you make this core thing in your career it is achievable and your career will be happy and prosperous

jeltz

I don't really buy that this is the main reason. A good senior engineer is for the most part able to not write bad code from day one, just at a very low speed and with the need to ask other people frequenyly. Even if you do not know the code base or domain yet there are a lot of things you can do to avoid writing bad code. Yes, as someone new you will make mistakes and misunderstand things but a lot of the bad code I have personally seen has not been caused by that. Most bad code I have seen has been caused by people rushing and not having their fundamentals in order. Like not actually doing reviews, not spending a few extra hours to think about architecture, etc. Also a big issue is that people just let the complexity of systems explode for the gain of short term projects.

I think the issue is more that engineers face unreasonable pressure to deliver short term value and that there is no respect for the craft/engineering from many managers or even engineers.

nosianu

Then how do you work with this: https://news.ycombinator.com/item?id=18442941

I did that job, just after university, but that is not my comment. I bookmarked it though because that person said it so well.

You will write bad code, because what you already find there - and that one company is not alone! - is already so bad, there is no way to do a good job on top of literally millions of escalating hacks.

And don't think that you could clean this up - not even with ten years of time is that possible. You can only rewrite from scratch. Trying to rewrite even a tiny part is like picking up one spaghetti and always ending up with the whole bowl on your fork.

jeltz

It is possible that some projects like Oracle are beyond hope but in general cleaning up a messy code base is done piece by piece and by refusing to merge most patches which make things worse. Better than you found it is the main principle.

veverkap

I think it's probably a bit of both. A good senior engineer may pick up a task and look at the system, seeing hacks duct taped together with kite string, and have the choice between "doing it right" (aka rewrite/refactor) and getting shit done.

dave_sid

They wouldn’t have to fix the entire codebase to be doing the right thing. That approach would be more likely taken by a junior who would soon find they are fighting a losing battle. The senior engineer would accept the state of the world and consider how to make small, deliverable, manageable improvements one PR at a time, to improve the code over a longer period of time without breaking the system.

godelski

  > A good senior engineer is for the most part able to not write bad code from day one
This seems unlikely. Self contained, I'd go further and say you're not a senior if your code isn't good you shouldn't be a senior.

But what is good code is in context of the codebase. It takes time to get the context of a reasonably sized codebase and no one is doing that in a single day or single week, even the wizards.

I don't agree with everything the OP writes but I think they're correct in that many companies don't value institutional knowledge. To me that is lunacy. I'm not sure how any programmer could think a reasonably complex codebase could be learned quickly. The larger and more complex your codebase the more valuable institutional knowledge is. Adding new actors just accelerates complexity and redundancy. Pushing people to be quick only worsens this. The best thing to do with new employees is get them deep into the code. Have them write/update docs, do cleanup, or other tasks that force them to understand the code and how everything interacts

tim1994

Another reason I can think of is the requirement not to introduce a breaking change. It is very frustrating if the codebase has a lot of hacky/bad code in it but a lot of it can't be changed...

morkalork

The worst code I've ever written is because of shifting or unforeseen requirements. It doesn't matter how good the architect is if the foundation is built on sand.

jeltz

100% agreed. But to me that sounds like a typical case of rushing instead of working like responsible engineers. If the foundation is built on sand then that needs to be fixed. Engineers being expected to magically paper over a lack of clear requirements is what leads to bad code. I am fine with helping gather the requirements but if I get a list of unclear and shifting requirements and just is expected to fix it I obviously will fail.

toast0

I've worked on projects where if you wait for the requirements to be firmed up, you'll never be able to do anything. Depends on what you're trying to build if that means you need to stop and figure out the requirements or if you need to just deal with the shifting sands. Aircraft built for moving requirements don't work so well; but lots of things are fine with moving requirements. It'd sure be nice to know how users are going to use your product before you build it, but sometimes you build what you think is wanted, and people only use part of it or use it for different things than what was intended, and it's better to adjust and refocus than to start a whole new development process with the found requirements.

veverkap

> If the foundation is built on sand then that needs to be fixed.

Except this is the system working as designed. Leadership 1000% wants to do things as fast and as cheap as possible.

pxc

I think, sadly, that's often "the job". My career has been good so far, all things considered, but I think it would probably be better if embracing that idea came more naturally to me.

One of my first strange and unpleasant realizations in transitioning from studying computer science to "working in the real world" came in a 1:1 meeting with my manager at my first job out of school. I was complaining about code quality both in the context of some of our existing codebases and some new code one of my peers (also a junior developer) had recently written. When the light bulb finally lit up in my naive little head, the question I asked my manager with a sense of horror and outrage was "... so you're saying they wrote bad code on purpose?". The painful thought was that I, too, would (or had already) found myself tasked with pushing code that I knew sucked, for reasons entirely unrelated to architecture or design or other purely "technical" constraints.

I used to fantasize about moving into a different software niche, maybe in safety critical systems, where correctness is more highly valued. But recently I'm coming to realize that the thing I crave (and miss from my school days) is the joy of the craft— something involving elegance and taste in a way that even the strictest standards of correctness doesn't necessitate.

I think for the most part, truly excellent code isn't something many businesses perceive themselves as needing (even if many practical benefits can flow from its virtues). And, probably, for many businesses, such indifference is right. So excellent code, where it exists, is probably more often "gotten away with", half-snuck in by stubborn engineers who are productive enough to burn time injecting some extra consideration and effort into their code, than it is commissioned by a business which understands that it wants good code.

lambersley

Remember the Stanford Prison Experiment; "bad company corrupts good people."

austin-cheney

The short tenure is a symptom of a larger problem. The deeper problem is that very little is expected of big company software employees. Conversely those same employees tend to expect a lot in return. You can call that entitlement, poor expectation management, first world problems, and all kinds of other names.

I have not worked for a FAANG, so maybe things are different there, but I don't suspect so. People are people no matter where you put them.

Increasing compensation is not the solution. It can be a factor in a larger solution, but just increasing compensation increases employee entitlement which makes this problem worse, not better.

The best solution I have seen is risk/reward. Put people in charge of their assigned effort with real adult danger of liabilities. Likewise, award them for their successes. This is called ownership, and it works because it modifies people's behavior. The rewards and liabilities do not have to be tied to compensation. Actually, associating these rewards/liabilities to social credibility within the team/organization appears more effective because it reinforces the targeted behaviors.

I have seen this missing in all of my software career until my current employment. Conversely people in the military are pushed into this liability/reward scenario from the very beginning and its very effective. It has always been striking to see the difference in my dual career progression.

null

[deleted]

matt3210

It’s always a trade off between raising the bar and making a deadline. The deadline always wins since the boss doesn’t know how to read code

jeltz

Sadly a lot of engineers have been indoctrinated into this mindset and I have had to fight quite many battles to conceive my fellow engineers that missing a deadline is not the worst thing in the world.

lukan

"I love deadlines. I love the whooshing noise they make as they go by." ― Douglas Adams

veverkap

Then you've been fortunate to work at places that respect engineering.

jeltz

Yes, I have. I have also worked at placed which do not. And the difference is night and day. The places which respect engineering are more fun to work at, deliver better features for less cost and the code is better. Only places which can deliver faster are crazy startups which constantly crunch time (I have worked at those too) but those are hell and the code is a mess.

The main cost I have seen at places which respect engineering is lower predictability. It is harder to budget and plan even if the end result in average is usually cheaper and always better.

nathancspencer

The claim this article makes about very short tenures at big tech is misleading. Because of headcount growth, the median tenure is naturally going to be short. Google grew headcount by 60% the year before 2013, so no wonder the median tenure was 1.1 years. A better statistic to use would be median tenure conditional on that the employee has already left.

tyleo

I don’t think the underlying point is true: big companies don’t necessarily write bad code.

A big company is like a collection of small companies. Code quality varies depending on where you are in it.

Similarly, nothing leads me to believe small companies are any better. Some are excellent. Some are nightmare spaghetti.

cm2012

Big companies are insanely slower due to beauracracy and rules, even if talent is identical. I have seen it happen so many times as companies grow and go IPO.

veverkap

Big companies oftentimes have cultures and leadership that lead to bad code.

can3p

The other reason is the volume of the code being produced combined with the constant product changes. An innocent change like mixing two close but still different concepts can easily poison the whole codebase and take years to undo and may even be nearly impossible to fix if it propagates to external systems outside of direct control

hkleppe

You have to realise there is a almost full complete disconnect between engineering and business value

wubrr

The disconnect is more between long term business value, and short term benefit for the most parasitic and manipulative actors within the business.

Engineering and business value go hand-in-hand in a healthy tech/engineering business.

A business that was built on great/innovative engineering, became successful, and then got taken over by various impostors and social manipulators, who's primary goal is gaming various internal metrics for their own gain, is not a healthy business.

Herring

I think that’s a bit unfair. I’d say shipping a product that solves a problem is the baseline entry fee into the market, just table stakes. Profitability is determined by the machine built around the product: the efficiency of capital deployment, the speed of distribution, the defensibility of the business model against competition, etc. The product is just one variable in a much bigger equation.

pyrale

That is, until planes fall from the sky.

tjr

It's popular to mock aerospace engineering, but it's usually quite robust. Even if people still sometimes make bad decisions.

brazukadev

And move slowly. So when things turn bad, they will be bad for a decade - or more. See Boeing.

DarkNova6

Only if there are is organizational disconnect… which is the norm.

alephnerd

Ime, a lot of the onus falls on Engineering and Product Management failing to make a case for why certain engineering decisions (eg. Investing in continual tech debt grooming) have business value.

The point of a business is to generate revenue. The point of employees is to do work that helps generate revenue. As such, any decision needs to ensure it has a business case aligned with revenue generation.

Good engineering hygine has significant business value such as in speeding up delivery of new features as well as keeping certain customers happy, but in a lot of cases there is an inability to communicate from either direction (eg. PMs not giving Eng full visibility into business decisions, and Eng not being able to explain why certain engineering choices have business value). If you cannot communicate why this matters, you aren't going to get it prioritized.

Unsurprisingly, at big organizations, communication can take the backseat because communication is hard and at a large company, there is some amount of complacency because the product is good enough.

Edit: Unsurprisingly got downvoted.

The only reason you are employed is to make value (which generally is measured in revenue generated). You are not paid $200k-$400k TCs to write pretty or a e s t h e t i c code. You can make a case for why that matters, but if you choose to bury your head in the sand and not make that case, I have no sympathy for you.

wubrr

Communication is not hard, it's very easy, but there are actors who's goal is to obfuscate communication and prevent others from participating.

At the end of the day it comes down to who the decision makers are and how they are incentivized to act. As a simple example - company X has product C, and they set a goal of increasing usage of feature F (of product C). Currently this feature F completely sucks and users don't want to use it - so the idea is to improve it and thus increase usage.

There are 2 ways of increasing usage:

1) Make the feature F more useful/better.

2) Force/push your users to use feature F, by aggressively marketing it, and pushing it within the product surfaces, making it non-optional, etc. and other dark patterns.

Option (1) is hard to do - it requires deep understanding of the product, user needs, the related tech, etc. It requires close tactical collaboration between product and engineering.

Option (2) is easy to do - it requires ~zero innovative thinking, very surface-level understanding of the problem, and relies purely on dark patterns and sketchy marketing tricks. You can almost completely ignore your engineers and any technical debt when following this approach.

If your decision makers are imposter PMs and marketing/sales people - they will almost always choose option 2. They will increase the 'apparent usage' of this feature in the short term, while reducing overall customer satisfaction increasing annoyance, and reducing the company's overall reputation. This is exactly how many 'growth' teams operate. Short term benefit/gaming of metrics for long term loss/reputational damage. Their success metrics are always short-term and linked directly to bonuses - long term effects of these kinds of strategies are ~always completely ignored.

ethanwillis

The point of a business is to generate profit.

zwnow

I work for some event ticketing business and I'd sign this. My bosses want features quickly. Does not matter to them if I need extra time to make stuff secure, doesn't matter to them if it wont scale. Its about short term revenue. Can always rebuild the software to fit the next short term goal...

alephnerd

If you understand what are the metrics being tracked, and what are the primary goals that an initiative or product has, you can make a case.

We are an engineering discipline and engineering decisions can have revenue making implications. But it is hubris to assume why you should care about the nitty gritties of a codebase. It's the same way no one in leadership cares about the nitty-gritties of HR policies or accounting practices - people are hired to deal with the intricacies.

When I was a PM, I didn't have a difficult time making a business case for "keep the lights on" or tech debt work so long as I was able to attach tangible revenue implications (eg. X customer might churn because of subpar experience and we have both customer testimony and user stats showing that) or roadmap implications (eg. If we spend 6 months refactoring our monorepo, we can add new revenue generating modules every quarter instead of annually).

ihsw

* Product folks over-promise

* Engineers are consulted for estimates to facilitate long-term planning

* Middle management slashes those estimates in half due to pathological myopia

* Executives enforce musical chairs to assert their authority

* Consultants muck everything up while collecting enormous payouts

And yet the business cycle keeps cycling.