Skip to content(if available)orjump to list(if available)

Netflix Revamps Tudum's CQRS Architecture with Raw Hollow In-Memory Object Store

thinkindie

Am I naive thinking this infra is overblown for a read-only content website?

As much as this website could be very trafficked I have the feeling they are overcomplicating their infra, for little gains. Or at least, I wouldn't expect to end having an article about.

__alexs

Doing weird pointlessly complicated stuff on a niche area of your website is a not entirely ridiculous way to try out new things and build new skills I guess.

pram

I’m gonna take a wild guess: the actual problem they’re engineering around is the “cloud” part of the diagram (that the “Page Construction Service” talks to)

There is probably some hilariously convoluted requirement to get traffic routed/internal API access. So this thing has to run in K8s or whatever, and they needed a shim to distribute the WordPress page content to it.

piva00

Having to run in k8s doesn't change that much, the description of a whole Cassandra + Kafka stack to deliver the ingestion of articles already says there's a lot more architecture astronaut-ing going on than simply deployment.

I cannot imagine why you'd need a reactive pipeline built on top of Kafka and Cassandra to deliver some fanservice articles through a CMS, perhaps some requirement about international teams needing to deliver tailored content to each market but even with that it seems quite overblown.

In the end it will be a technical solution to an organisational issue, some parts of their infrastructure might be quite rigid and there are teams working around that instead of with that...

pram

Probably because they were available, and now they’ve done the equivalent of throwing redis on their containers. My main point is, the blog content is probably the “easy part” and whatever is consuming it is the “hard part”

rokkamokka

From an outsiders perspective Tudum does seem to be an extremely simple site... But maybe they have complicated use cases for it? I'm also not convinced it merits this level of complexity

boxed

I mean, this is the company that invented microservices so....

thinkindie

very fair point - but there are valid use cases for microservices.

hnthrow20938572

>However, due to the caching refresh cycle, CMS updates were taking many seconds to show up on the website. The issue made it problematic for content editors to preview their modifications and got progressively worse as the amount of content grew, resulting in delays lasting tens of seconds.

This seems superficial, why not have a local-only CMS site to preview changes for the fast feedback loop and then you only have to dump the text to prod?

>got progressively worse as the amount of content grew, resulting in delays lasting tens of seconds.

This is like the only legit concern to justify redoing this, but even then, it was still only taking seconds to a minute.

yunohn

I don’t normally comment on technical complexity at scale, but having used Tudum, it is truly mind boggling why they need this level of complexity and architecture for what is essentially a WP blog equivalent.

anonzzzies

refset

> Hollow employs compression techniques

Given how prominently 'compression' gets mentioned I was excited to learn more, but it looks like it simply amounts to using GZIPInputStream/GZIPOutputStream within the blob APIs, or am I missing something...?

conceptme

If it fits in memory why choose for cqrs with kafka in the first place?

immibis

https://netflixtechblog.com/netflix-tudum-architecture-from-...

Concerning that their total uncompressed data size including full history is only 520MB and they built this complex distributed system rather than, say, rsyncing an sqlite database. It sounds like only Netflix staff write to the database.

thecupisblue

Holy shit the amount of overcomplications to serve simple HTML and CSS. Someone really has to justify their job security to be pulling shit like this, or they really gotta be bored.

If anyone can _legitimately_ justify this, please do, I'd love to hear it.

And don't go "booohooo at scale" because I work at scale and am 100% not sure what is the problem this is solving that can't just be solved with a simpler solution.

Also this isn't "Netflix scale", Tudum is way less popular.