Serverless Horrors

I mean, would you rather have a $10k build or have your server forcefully shut down after you hit $1k in three days?

One of those things is more important to different types of business. In some situations, any downtime at all is worth thousands per hour. In others, the service staying online is only worth hundreds of dollars a week.

So yes, the solution is as simple as giving the user hard spend caps that they can configure. I'd also set the default limits low for new accounts with a giant, obnoxious, flashing red popover that you cannot dismiss until you configure your limits.

However, this would generate less profit for Amazon et al. They have certainly run this calculation and decided they'd earn more money from careless businesses than they'd gain in goodwill. And we all know that goodwill has zero value to companies at FAANG scale. There's absolutely no chance that they haven't considered this. It's partially implemented and an incredibly obvious solution that everyone has been begging for since cloud computing became a thing. The only reason they haven't implemented this is purely greed and malice.

Or simply returns 503? Why would you go directly to destroying things??

Yes, that’s exactly the expected behavior. It can alert if it’s closed to threshold. Very straightforward from my point of view.

Stop accepting requests like has been the case since the beginning of time?

Surely that's the fault of the purchaser setting the cap too low.

Maybe rather than completely stopping the service, it'd be better to rate limit the service when approaching/reaching the cap.

I agree, but I could also see how someone would complain about that: “Our e-commerce site was taken down by Amazon right on our biggest day of the year. They should have just moved us up to the next tier.”

Good news! This is exactly how the free tier works now.

stop putting stuff on the internet you don't understand.

putting stuff on the internet is dangerous. if you're not prepared to secure public endpoints stop creating them.

You're right, but even if I cut the water pipe right after the meter and run it for a month I might get a few thousand dollar charge.

You can ring up tens of thousands+ overnight with AWS. The scale of potential damages is nowhere even close.

I don't know in US, but having limits on how much electricity a house is able to take from the gride is absolutely something in some countries out there.

The first instance is difficult to fix as crime can often involve substantial losses to people and often there's no route to getting a refund.

The broken water pipe should be covered by buildings insurance, but I can imagine it not being covered by some policies. Luckily a broken water pipe is likely not as expensive as not having e.g. third party liability protection if part of your roof falls off and hits someone.

Why don't you run locally?

These aren’t limits though, they are just budget notifications.

What would be helpful, would be if when you set up your account there was a default limit – as in an actual limit, where all projects stop working once you go over it - of some sane amount like $5 or $50 or even $500.

I have a handful of toy projects on AWS and Google cloud. On both I have budgets set up at $1 and $10, with notifications at 10% 50% and 90%. It’s great, but it’s not a limit. I can still get screwed if somehow, my projects become targets, and I don’t see the emails immediately or aren’t able to act on them immediately.

It blows my mind there’s no way I can just say, “there’s no conceivable outcome where I would want to spend more than $10 or more than $100 or whatever so please just cut me off as soon as I get anywhere close to that.”

The only conclusion I can come to is that these services are simply not made for small experimental projects, yet I also don’t know any other way to learn the services except by setting up toy projects, and thus exposing yourself to ruinous liability.

Those are not in fact limits:

> There can be a delay between when you incur a charge and when you receive a notification from AWS Budgets for the charge. This is due to a delay between when an AWS resource is used and when that resource usage is billed. You might incur additional costs or usage that exceed your budget notification threshold before AWS Budgets can notify you, and your actual costs or usage may continue to increase or decrease after you receive the notification.

As far as I know, neither Google, Amazon or Azure have a budget limit, only alerts.

This is a reason why I am not only clueless of anything related to cloud infrastructure unless it's stuff I am doing on the job, nor I am willing to build anything on these stacks.

And while I guess I have less than 10 products build with these techs, I am appeal by the overall reliability of the services.

Oh lastly, for Azure, in different European regions you can't instance resources, you need to go through your account representative who asks authorization from the US. So much for now having to deal with infrastructure pain. It's just a joke.

As others have said these are not limits, just notifications. You can’t actually create a limit unless you self create one using another AWS service (surprise) like lambda to read in the reports and shut things down.

And as others have also mentioned, the reports have a delay. In many cases it’s several hours. But worst case, your CURs (Cost usage reports) don’t really reflect reality for up to 24 hours after the fact.

I work in this space regularly. There can be a delay of 2-3 days from the event to charge. Seems some services report faster than others. But this means by the time you get a billing alert it has been ongoing for hours if not days.

To all of those who say "this is not limit, only notifications": yes, notifications that can trigger whatever you want, including a shutdown of whatever you have

Is this a perfect solution: no Is this still a solution: yes

"Limits" like this are how I woke up one Sunday morning in my college dorm with a $7k bill from dreamhost.

To paraphrase Rainer Wolfcastle - the budgets do nothing!

You get a warning. There's no service cutoffs or hard limits on spending.

Yes, a sign of resilient architecture is to shut down when it encounters some stress.

> Seems an interesting oversight. I can just imagine the roundtable, uhh guys who do we charge for 403? Who can we charge? But what if people hit random buckets as an attack? Great!

It is amazing, isn't it? Something starts as an oversight but by the time it reaches down to customer support, it becomes an edict from above as it is "expected behavior".

> AWS was kind enough to cancel my S3 bill. However, they emphasized that this was done as an exception.

The stench of this bovine excrement is so strong that it transcends space time somehow.

I recently had customer who had smart idea to protect Container Registry with firewall... Breaking pretty much everything in process. Now it kinda works after days of punching enough holes in... But I still have no idea where does something like Container registry pull stuff from, or App Service...

And does some of their suggested solutions actually work or not...

There are many paths. Worst case, I've witnessed developers editing Lambda code in the AWS console because they had no way to recreate the environment locally.

If you can't run locally, productivity drops like a rock. Each "cloud deploy" wastes tons of time.

Mocks usually don’t line up with how things run in prod. Most teams just make small branch or dev environments, or test in staging. Once you hit odd bugs, serverless stops feeling simple and just turns into a headache.

That's the equivalent of saying "just audit your cloud usage and remove stuff that's no longer used".

Don't they charge for every TB exceeding the included limit? (website says "For each additional TB, we charge € 1.19 in the EU and US, and € 8.81 in Singapore.")

Buckets are private by default.

And it's getting harder and harder to make them public because of people misconfiguring them and then going public against AWS when they discover the bill.

Because just using a cdn without proper caching headers is just another service you're paying for without any savings.

The real question is if they considered caching and thus configured it appropriately. If you don't, you're telling everyone you want every request to go to origin

Because not all uses for buckets fit that.

Buckets are used for backups, user uploads, and lots of things other than distributing files publicly.

That might be the more general solution but in this context it is absolutely also an access control issue.