Theory crafting a system for 1000 simultaneous micro SD card ingests

I hope they have a robot for physically handeling the SD card insertions and retrieval. Even then, lets say you can load or swap 1 card per second, by the time you are dealing with the 200th card, would the first one not already be ingested?

Seems like more of a scale out than a scale up problem, with a view on the whole operation from recording footage down to final archive to tackle the real bottleneck.

I know it sounds boring, but did anyone suggest a second computer if this is an actual problem and not just a for fun challenge?

The physical interface itself is going to be a big deal, people are going to be just swapping cards constantly and you'll need multiple people just doing this. Status of read is a big part of this as well so some amount of custom LEDs and such for status of ingress will be required. What a horrible job being in a room with 100s of thousands of SD cards just feeding them into slots and then into an empty bin!

Its possible from a PCI-E bandwidth point of view but its going to require some seriously specialist USB interfaces. I am tempted by the same solution others suggest, smaller amounts per machine less expensive and extreme solutions into normal switches and then out across fibre.

What a crazy thing to be doing, the mad situations companies get themselves into when they should just have networked cameras and VPNs or at the very least distributed ingress machines!

Such a nerd bait request. I’m like “wait, how long between downloads of cards do you need? How many people will be placing these cards in? How will you know when one is downloaded??”

Only reading the comments did I think “Wait, who needs this and why?” I like the Mr. Beast angle - it’s fun to think about if nothing else.

One use case is to mount 360 cams on a bunch of motorcycles (fleet) and do a google streetview equivalent. You build a city level streetview pretty quickly.

I think at that point you'd just need to do 1 pc every 200 readers and pull the videos to a local server for processing. There's no magic here, just work.

They should simply outsource it to lowest bidder, preferably on a rando internet forum.

I'd go in a completely different direction from this.

I'm assuming the SD cards come from a fleet of dashcams or similar, and that the driver is responsible for turning in their card at the end of a run or whatever.

I'd deploy a fleet of tiny WiFi card readers. Just an ESP32 and a card slot. Each individual unit is not fast, but if you have a few dozen units running at once you could easily saturate the dedicated ingest WiFi network at each hub location. The readers are dirt cheap and the only supporting infrastructure required is WiFi and a bunch of 5v power adapters.

You'd have a rack of these guys next to the key locker or timeclock. Pick up your keys and a blank card, drop off your keys and card at the end of the shift. Could even get extra fancy and use LEDs to indicate which card is yours for the day, or flip on an error light when the card is worn out.

You probably don't need each video to transfer as fast as possible, more likely you just want maximum overall throughput. So just build a bunch of very cheap and slow nodes. On aggregate you should get decent performance.

I was not expecting that thread to go in that direction. At first my nerd brain kicked in and I started doing the PCIe napkin math - but then I got to the post asking "why?" and found myself asking the same question.

I am also a bit perplexed on why they'd need to ingest 1,000 micro-SD cards at once. If they are such a large org, why not investigate alternative solutions?

At that scale, wouldn't it make sense to just have raw block i/o copies being done to a disk image, and then mount the image onboard to do the actual data transfers?

Seems to me a majority of the overhead with 1000 SD cards is in the filesystem translation layer, multiplied by the USB contention dealing with so many devices.

So if it were me, I'd have a big fat, fast mounted filesystem whose purpose is only to serve as a dd destination, get the dd done as quickly as possible with system bus buffer sizes, and then do the data transfer 'offline' once the SD has been mirrored...

So, the 1000 card problem is pretty easy. What matters is how much is on each card, and how long the DL window is.

For instance, if you can spend 2 hours downloading each card, (SPI to an esp32 can do about 32GB an hour) so you could Download 64GB in 2 hours via ESP32 boards with SPI uSD slots, pushed onto your server over wifi6.

If that’s enough, that’s a simple solution. If you need more than 32GB an hour per card, I’ve seen some adapters that hold 10? uSD cards and connect over SATA, so that might get you a better way into the bus than typical USB?

There is only one scenario that remotely makes sense for why this would be even necessary and that's fleet vehicles with dash cams. Even in that case, there are way better approaches because there are dashcams designed for fleet usage that upload to central servers over WiFi.

Basically this scenario only makes sense if there is something horribly wrong elsewhere to the point that the only reasonable thing to do is fix things to involve a network, not to try to figure out how to scale a sneakernet operation.

"i can provide more details if needed but some things i am under an NDA so i may not be able to answer specifics"

LOL why are these people helping this guy for free?