Take this on-call rotation and shove it

There's a big difference, though.

In your case, you're the only employee of your business. And if you're not there the business will literally go under. And you also get directly rewarded for being there. I would guess that being 'on call' in this manner is possibly less draining on a person's soul (depending on how well they tolerate the risk of owning a business).

Contrast that with being 'on call' for your megacorporation, who isn't giving you anything extra for your on call time because they 'already pay you enough'. And where the only negative consequence for the company if you fail to immediately respond within 15 minutes is that some executive in the company is kept waiting longer than 15 minutes, or some ads aren't being shown for 15 minutes.

But if you aren't there, your boss is going to get a phone call and that's definitely not going to look good on you. And there's no bonus for fixing the problem, that was already your job in the first place. Sucks that you had to do it outside of scheduled hours, oh well.

I'm with the author of this article. Take your on-call rotation and shove it (if you're a large corporation). I'm fortunate enough to be able to take a firm stance on this point, and do so happily.

My friend worked for Amazon in India (software). He was often on 24x7 "on calls", which is touted as "good" here (because how else will you "learn"), during his 3rd or 4th week. By third night he was vomiting and had to visit the doctor. His manager called and had asked whether he had brought his laptop with him. His mother forced him to resign next day (he is from an extremely rich kind of family though). It is common here in most companies and among famous MNCs it is especially known in Amazon and Uber in India.

What shocks more is these are the companies that can "follow the sun" w/o breaking a drop of fucking sweat!

I have lost too many interviews just because I clearly asked for this, I always do, and I am doing it even now while I have been without a job after taking a year gap (which makes getting calls already difficult esp. with this AI and vibe onslaught). I am not giving up on this. I personally have never agreed to this which has caused lots of confrontations and stress(!!); a major source of my burnout WITHOUT ever doing the 24x7 on-call - so by just fighting it and keeping it away from me alone I was burnt out to the bits. It took me finally seeking medical advice to realise I was burnt out.

I hope this is not sounding like dramatic but even now when I have been resting, travelling for a year the mere mention of words like Splunk, VictorOps (same as Splunk iirc), PagerDuty give me minor trigger attack kinda sensation - make me very agitated.

But this is so common here. So common that it is considered one of the realities, truths. Yet, I have never understood, how, how can one agree to this? How? Is it some kind of social (if not racial) slave mentality? Is it some kind of grand coercion that they have no escape from? Or maybe it's just generation after generation subjecting the next generation to what they were subjected to while the stakeholders in the richer countries (because that is the structure) demand of this implicitly as they are stopped by health and safety laws from subjecting their underlings in their own developed home nations maybe.

Working for yourself is totally different.

It's like demands from tech executives for long hours: "I worked long hours to make myself rich; why won't all of you work long hours to make me richer?"

Would it make sense to hire someone? When you run a solo person business getting over the mental hump about hiring someone is difficult. If rewards are great and things are stable (16 years is stable) what is preventing you from hiring someone to help with at least some aspect?

One of the sidebars mentions that: "The production system in question is almost certainly a schizophrenic box of compromises brought about through poor decision-making, unaddressed technical debt, design-by-committee, and impossible timelines and budgets. This is not a system that any single rational human being on the team would’ve chosen to build if permitted to do so alone. Trying to assert ownership over an environment like that is just begging to get your shit rocked."

> Also some systems have no clear ownership, so nobody has an incentive to fix them.

It’s even worse when the system isn’t business-critical: a reporting service, a manual intervention tool, something that quietly supports a process. When it fails, everyone is affected, but no one is accountable.

Ironically, these are often legacy systems that have been rock-solid for years — so reliable they’re forgotten… until they break.

I've been on call for almost 20 years. If a system is crashing often and disturbing your sleep but is not being prioritized to get fixed, then stop answering the calls. If it was important last night then it's still important this morning.

Be vocal and say "I will no longer respond when system XYZ goes down unless serious efforts are made to fix it."

If you get push back explain that you will also call the person telling you it's not important enough to fix each time it pages, and be willing to do so. What can they say?

I feel for you, I’ve also suffered through this a lot over the years, and am finally at the stage of career and wisdom to start pushing back on the quality that I can’t control and ensuring that others are equally as accountable for their mess.

For one particular occasion , once we took blame out of the equation (at least within the engineering team) and started doing Post Incident Reports, the incentives finally became clear for the business as we were able to compile a list of recurrent issues during every issue, calculate a financial loss and present it for inspection each and every time they either began a witch hunt for downtime or refused to allocate time to backlog. Small wins.

there's nothing wrong with shift work, and there's nothing wrong with the graveyard shift (for people who want that job), but there is a LOT wrong with alternating day and night shifts in the same week every week, and nobody should agree to do that. (perhaps if you are in your early twenties you feel like you can handle it, but I'm not sure that's a good idea either)

Shift work sucks more than on call for me. I like having a flexible schedule that I can work whenever I want, and I'll happily take the remote risk that I'll get paged during my one week per quarter on call rotation in order to guarantee that no one expects me to be on during set hours the rest of the year.

I think the problem isn't on call itself, it's that a lot of companies suck at on call. If your rotation is every 3 weeks and you get woken up in the night at least once per rotation, then yeah, that's awful. But the problem you have in that case is that stuff is always on fire and you don't have enough people on the rotation, not with on-call as a concept.

Wouldn't that incentivise staff to take longer to fix issues? Once you've been interrupted, you might as well turn 30 minutes into 60 minutes, etc.

Good suggestion and I can see the benefit for honest people, but unfortunately it’s as equally a system for financial abuse for others - sometimes enough to prevent people fixing things during their regular hours just to benefit at other times.

A good counter balance to this might be to offer even more compensation for no incidents, or otherwise well handled incidents that go on to squash types of that incident now and into the future.

Some long haul truckers have a relatively workable pay scale. If it wasn’t for how far they expect you to drive in 24 hours, it might be considered good.

The general shape of it at least makes sense. You get paid when you’re on the road. More if you’re driving than if you’re parked, and more if you drive for more than your 40 hours a week.

This makes it in the employee's interest to obfuscate and extend any remediation, to get paid more.

> Every minute you spend doing on-call work is time that you can't spend on the things you've actually been assigned to do.

In my experience at least if you're oncall during a sprint you would have less work assigned to you than otherwise (2 week sprint and 1 week you are oncall? 50% allocation) as the expectation is that week you will spend responding to alerts, or investigating issues, or even improving alerting and dashboard and fixing bugs. If this does not happen, devs don't push for it and management is completely blind to it you have an organization issue. If leadership does not care about the problem it's time to jump ship ASAP.

But I've seen people stubbornly defending an alert on >60% CPU usage of their 1 CPU allocated kubernetes pods where there was no impact in p99.9 latency (which was measured and was the actual metric that mattered as agreed with the rest of the business and internal customers of the service). Or alerting on each single pod restart. That is self inflicted pain.

It's just more hazing so the underclass of engineers don't realize they actually build everything big tech is selling and more.

I refused a contract to hire that was talking pager duty.

I saw how they freaked out about things outside the team’s control during business hours. The first time someone called me after 11 pm I was going to get myself fired talking to them.

To begin with, airplanes do fall out of they sky, sometimes right on your backup generator. (speaking from military experience where yes, there is an entire backup studio waiting to take over, just in case. Or rather, the 'studio' is geographically dispersed, with 100% redundancy, which is another way of saying the same thing.)

But more importantly, *this* is what you noticed from that article?

That just covers electricity. They seem to be implying coverage of a multi-failure scenario.

Also, I don't think broadcast TV news is quite as reliable as the post makes it out to be.

Like this exchange happens all the time:

"and now we're going to our on site reporter, Onda Premises"

<45s silence>

"Oh, it appears we've lost them for now, we'll cycle back later. In our next story...."

In Finland you usually get a static pay just for being on call, meaning you're X minutes away from a company-approved device you can use to fix things if they break.

If there's an actual alert you need to respond to, your pay goes to 2x hourly rate, or 3-4x if it's the weekend and there usually is a minimum amount of billing you do if you have to do any work, usually 30-60 minutes. So if you get an alert and you fix it by pressing the "fix it" button on the dashboard on a Sunday, you just got paid hundreds of euros.

On the other hand you most likely saved the company from losing multiple times that in revenue, so it's worth it to the company.

I have a bunch of friends who were single and/or child free in their 20s and have fully paid apartments/houses because they could be on call at any time because they didn't have any commitments.

I also know a good amount of incidents that were fixed in a pub corner table after a few drinks. I may or may not have contributed to that number. =)

Yes, basically every eng position paying what HN people expect will have the exact sort of described oncall.

Curious how this is the case. Do European companies not provide 24/7 services? Or staff a "follow the sun" model so no one has to answer pages outside of working hours? Do Europeans write better code so they don't need on-call?

I've worked at a few FAANGs in the US, and every single one after 2010 had an on-call rotation.

I've been on-call (sometimes on a rotation, sometimes always) for the past 10+ years. So, yes. It's common.

on-call is ubiquitous in the US tech industry. I've never had a job without it.