In S3 simplicity is table stakes

I mainly use GCP but keep hearing how great AWS is in comparison.

Imagine my surprise when porting some GCS code to S3 last year and realizing there is no way to get consistency guarantees without external lock service.

> Last thing I'll say is, you know your API is good when "S3 compatable API" is a selling point of your competitors.

Counter-point: You know that you're the dominant player. See: .psd, .pdf, .xslx. Not particularly good file types, yet widely supported by competitor products.

It’s designed for that level of durability, but it’s only as good as a single change or correlated set of hardware failures that can quickly change the theoretical durability model. Or even corrupting data is possible too.

I've used it for server backups too, just a simple webserver. Built a script that takes the webserver files, config files and makes a database dump, packages it all into a .tar.gz file on monday mornings, and uploads it to S3 using a "write only into this bucket" access key. In S3 I had it set up so it sends me an email whenever a new file was added, and that anything older than 3 weeks is put into cold storage.

Of course, I lost that script when the server crashed, the one thing I didn't back up properly.

I did a GCP training a while back, and the anecdote from one of the trainers was that the Cloud Storage team (GCP’s S3-compatible product) hadn’t lost a single byte of data since GCS had existed as a product. Crazy at that scale.

> their durability number means that one can expect to loose data about once every 10,000 years

What does that mean? If I have 1 million objects, I loose 100 per year?

I’ve never worked with AWS, but have a certification from GCP and currently use Azure.

What do you see as special for S3? Isn’t it just another bucket?

> but as I learned more I realized that it had some wild features that they all managed to hide properly so when you start it looks like a simple system but you can gradually get deeper and deeper until your S3 bucket is doing some pretty sophisticated stuff.

Over the years working on countless projects I’ve come to realize that the more “easy” something looks to an end user, the more work it took to make it that way. It takes a lot of work to create and polish something to the state where you’d call it graceful, elegant and beautiful.

There are exceptions for sure, but often times hidden under every delightful interface is an iceberg of complexity. When something “just works” you know there was a hell of a lot of effort that went into making it so.

The durability is not so good when you have a lot of objects

Maybe this is a faster way of getting AWS feature requests heard.

I'm going to write a script that keeps trying to call ecs:MakeFargateCacheImages.

I instinctively distrust any software or protocol that implies it is "simple" in its name: SNMP, SMTP, TFTP, SQS, etc. They're usually the cause of an equal or more amount of headaches than alternatives.

Maybe such solutions are a reaction to previous more "complex" solutions, and they do indeed start simple, but inevitably get swallowed by the complexity monster with age.

SQS went into beta first, S3 went "GA" first.

AWS typically considers the "GA" milestone as the "public launch" date, which is silly because the criteria for what is good enough for GA has changed over the years.

The API, absolutely.

It's only sad that the SDKs are often on the heavy side, I remember that the npm package used to be multiple megabytes as it bundled large parts of the core AWS SDK. Nowadays, I believe it's still around half a megabyte.

Now add versioning, replication, logging, encryption, ACLs, CDN integration, event triggering (Lambda). I could go on. These are just some other features I can name off the top of my head. And it all has to basically run with zero downtime, 24x7...

That's the public interface. The underlying architecture is where the power is.

Anyone can create a CRUD API. It takes a _lot_ of work to make a CRUD API that scales with high availability and a reasonable consistency model. The vast majority of engineers would take months or years to write demo.

If you don't believe me, you might want to reconsider how skilled the average developer _really_ is.

These comments are so uniquely "HN" cringe-worthy.

I used to have the same opinion until I built my own CDN. Scaling something like that is no joke, let alone ensuring you handle consistency and caching properly.

A basic implementation is simple, but at S3 scale, that's a whole different ball game.

Isn't that the most impressive part? That the abstraction makes it seem so simple

That’s when you really know you hid all the complexity well. When people call your globally replicated data store with granular permissions, sophisticated data retention policies, versioning, and manage to have, what, seven (ten?) nines or something, “simple”.

No problem. I’m sure ChatGPT could cook up a replacement in a weekend. Like Dropbox it’s just rsync with some scripts that glue it together. How hard could it possibly be?

I mean people serve entire websites right out of s3 buckets. Using it as a crude CDN of sorts.

It’s a modern marvel.

A file system is simple. Open, read, close. Most tech problems are not that simple. How hard could a filesystem be?

That's going to depend a lot on what your needs are, particularly in terms of redundancy and durability. S3 takes care of a lot of that for you.

One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. The remaining drives in the array will have to do more work when a failed drive is replaced and data is copied to the new array member. This sometimes leads to additional failures before replacement completes, because all the drives in the array are probably all the same model bought at the same time and thus have similar manufacturing quality and materials. This is part of why it's generally said that RAID != backup.

You can make a local backup to something like another server with its own storage, external drives, or tape storage. Capacity, recovery time, and cost varies a lot across the available options here. Now you're protected against the original server failing, but you're not protected against location-based impacts - power/network outages, weather damage/flooding, fire, etc.

You can make a remote backup. That can be in a location you own / control, or you can pay someone else to use their storage.

Each layer of redundancy adds cost and complexity.

AWS says they can guarantee 99.999999999% durability and 99.99% availability. You can absolutely design your own system that meets those thresholds, but that is far beyond what one server with a RAID array can do.

One interesting thing about S3 is the vast scale of it. E.g. if you need to store 3 PB of data you might need 150 HDDs + redundancy, but if you store it on S3 it's chopped up and put on tens of thousands of HDDs, which helps with IOPS and throughput. Of course that's shared with others, which is why smart placement is key, so that hot objects are spread out.

Some details in https://www.allthingsdistributed.com/2023/07/building-and-op... / https://www.youtube.com/watch?v=sc3J4McebHE

There is a diminishing return of what percentage you save, sure. But amazon will always be at that edge. They already have amortized the equipment, labour, administration, electricity, storage, cooling, etc.

They also already have support for storage tiering, replication, encryption, ACLs, integration with other services (from web access to sending notifications of storage events to lambda, sqs, etc). Uou get all of this whether you're saving 1 eight bit file or trillions of gigabyte sized ones.

There are reasons why you may need to roll your own storage setup (regulatory, geographic, some other unique reason), but you'll never be more economical than S3, especially if the storage is mostly sitting idle.

There are a few stories from companies that moved away from S3, like Dropbox, and who shared their investments and expenses.

The long and short of it is that getting anywhere near the redundancy, reliability, performance etc of S3, you're spending A Lot of money.

Probably never. The complexity is borne by Amazon. Even before any of the development begins if you want a RAID setup with some sort of decent availability you've already multiplied your server costs by the number of replicas you'd need. It's a Sisyphean task that also has little value for most people.

Much like twitter it's conceptually simple but it's a hard problem to solve at any scale beyond a toy.

> At what point does it become more costly and complicated to store your data on S3 versus just maintaining a server with RAID disks somewhere?

It's more costly immediately. S3 storage prices are above what you would pay even for triply redundant media and you have to pay for data transfer at a very high rate to both send and receive data to the public internet.

It's far less complicated though. You just create a bucket and you're off to the races. Since the S3 API endpoints are all public there's not even a delay for spinning up the infrastructure.

Where S3 shines for me is two things. Automatic lifecycle management. Objects can be moved in between storage classes based on the age of the object and even automatically deleted after expiration. The second is S3 events which are also _durable_ and make S3 into an actual appliance instead of just a convenient key/value store.

Since everything you need to run "a full SPA" is to serve some static files over an internet connection I'm not sure how that tells you anything interesting about the platform. It's basically the simplest thing a web server can do.

I use S3+Cloudfront for static sites and Cloudflare workers if it needed.

It's always crazy to me that people will run a could be static site on Netlify/Vercel/etc.

https://www.vantage.sh/blog/amazon-s3-tables#s3-tables-cost

I can't make head or tails of the beginning of this sentence:-

> Pricing for S3 Tables is all and all not bad.

Otherwise lovely article!