Show HN: Real-time system that tracks how news spreads across 200k websites
27 comments
·November 26, 2025kburman
This is absolutely brilliant. If you integrate Reddit and Twitter/X, you’d get a much more complete picture of how stories spread across the internet.
ewuhic
Without evaluating it thoroughly and judging just from description - I really hope this ends up open-sourced - will help drastically to many good-intent parties.
null
YmiYugy
The idea is pretty cool, but it doesn't work super well. 1. I imagine most major news outlets don't have RSS feeds these days. 2. A lot of stuff originates from news agencies, so they don't spread from website to website, but radiate out from the agency. 3. Most of the included sources are pretty small. To draw meaningful conclusions we would need infos like popularity, political leaning, nation of origin, etc. 4. The similarity check doesn't appear to do translation. So when news spreads from one country to another we loose the thread.
badestrand
The devil really is always in the details.
gioele
Kudos on releasing Yandori!
We have been (low-keep) working on something similar (more from an academic point of view) for the past few years:
This is the introductory article (open access): "Comparison of news commonality and churn in international news outlets with TARO" https://dl.acm.org/doi/abs/10.1145/3603163.3609062
(Allow me a moment of pride for the student leading this project: the paper won the Ted Nelson Award at ACM Hypertext 2023.)
masterphai
Interesting project - it’s rare to see news-flow tracking done in real time at this scale. One thing you may want to stress-test is how stable the clustering remains when stories evolve semantically over a few hours. Embeddings tend to drift as outlets rewrite or localize a piece, and HNSW can sometimes over-merge when the centroid shifts.
A trick that helped in a similar system I built was doing a second-pass “temporal coherence” check: if two articles are close in embedding space but far apart in publish time or share no common entities, keep them in adjacent clusters rather than forcing a merge. It reduced false positives significantly.
Also curious how you handle deduping syndicated content - AP/Reuters can dominate the embedding space unless you weight publisher identity or canonical URLs.
Overall, really nice work. The propagation timeline is especially useful.
supriyo-biswas
Thanks for your comment, unfortunately it seems that your comments are primarily LLM-generated (for people looking for evidence, the first comments of this user should provide enough evidence, although they’re getting better by fine tuning the prompt). As HN is primarily a place for humans, please do not do this here. Thanks.
KomoD
I think the idea is interesting but it includes a lot of spam and non-news (e.g. archive.fo, .vn, .today, etc.)
hk1337
This seems like it could have an additional use case of labeling each news source left, right, center, neutral/factual and tracking how or if each one releases an article.
analogears
Tried this on iPhone - the category tabs (Sports, World News, Business) get cut off on the right and there's no horizontal scroll indicator, so I didn't realise there were more options at first. The story cards also aren't using the full screen width, leaving wasted space on both sides.
Cool concept though - the source count and "+N" spread metrics give a quick sense of which stories have legs.
dmix
How do you handle time zone issues with the dates?
I’ve been curious how much news starts from social media. So many news stories today are “someone said x on twitter”.
Havoc
That's really cool!
Curious how you sourced the feeds? It seems to have a bias towards Indian/Srilanka/Iran/Indonesia/Turkey etc - i.e. not the traditional western centric reporting. Always interested in trying to get a more balanced news diet so anything you could share around that would be interesting. Most out of the box news tools seem to automatically lean west
FYI layout sometimes breaks like so:
supermatt
“Traditional western reporting” is traditionally a western thing. That’s only 15% of the global population - so if anything it seems bias towards that.
codethief
Cool idea! On mobile (Chromium on Android) I was confused at first because nothing happened when I tapped any of the stories – until I realized I can zoom out and the info about how the story propagated is at the end of the page.
hmokiguess
Cool idea! What I liked the most was the breakdown into categories like “breaking” and “trending” plus the number of sources.
The view showing the flow with a play animation was a nice concept but I couldn’t see much value in it, wondering if you could try to get a more aggregate stats that shows a connection between these different flows, maybe they follow a pattern like ad-based campaigns or publishers who own these domains, which would explain things. Expanding on this idea, could even try and setup different scores and metrics based on major groups and sponsored content versus organic spread.
I built a system that monitors ~200,000 news RSS feeds in near real-time and clusters related articles to show how stories spread across the web.
It uses Snowflake’s Arctic model for embeddings and HNSW for fast similarity search. Each “story cluster” shows who published first, how fast it propagated, and how the narrative evolved as more outlets picked it up.
Would love feedback on the architecture, scaling approach, and any ways to make the clusters more accurate or useful.
Live demo: https://yandori.io/news-flow/