"Streaming vs. Batch" Is a Wrong Dichotomy, and I Think It's Confusing
5 comments
·May 14, 2025brudgers
Streams have unknown size and may be infinite.
Batches have a known size and it are not infinite.
oatmeal_croc
Streams -> optimized for latency
Batches -> optimized for efficiency
pestatije
streaming should be used exclusively for live data...if you keep things that way everything falls smoothly in place
aeonik
This gets messy though because the deeper you dig the more the word "live" loses meaning. Is "live" data something emitted within milliseconds? Seconds? Is it "live" if it’s replayed from a buffer with minimal delay? Real-time systems aren’t always real-time. Some "live" streams are just batched updates in disguise. You blink and suddenly you’re in temporal quantum soup.
And once you push the boundaries—high-frequency trading, deep space comms, even global-scale latency—you run into the brick wall of physics. At certain speeds and distances, simultaneity stops being objective. Two observers won’t agree on what "just happened." Causality gets slippery. Streams bifurcate.
At that point, "live" isn’t just fuzzy—it’s frame-dependent. You’re not streaming reality anymore. You’re curating a perspective.
If streaming is 5x as expensive as batch, that might be a factor worth considering.