The Future of Comments Is Lies, I Guess
114 comments
·May 29, 2025intended
photonthug
> increasing the cognitive effort required for humans to interact or be heard. I dont see this playing out in a healthy manner
We are at the stage where it’s still mostly online but the first ways this will leak into the real world in big ways are easy to guess. Job applications, college applications, loan applications, litigation. The small percentage of people who are savvy and naturally inclined towards being spammy and can afford any relevant fees will soon be responsible for over 80 percent of all traffic, not only drowning out others but also overwhelming services completely.
Fees will increase, then the institutions involved will use more AI to combat AI submissions, etc. Schools/banks/employers will also attempt to respond by networking, so that no one looks at applicants directly any more, they just reject if some other rejected. Other publishing from calls for scientific papers to poetry submissions kind of progresses the same way under the same pressures, but the problem of “flooded with junk” isn’t such a new problem there and the stakes are also a bit lower.
jauntywundrkind
In Peter Watts' Maelstrom (2002) it's ultimately self replicating code that pushes the internet from a brutal and rough and competitive infoscape into something worse & even more rawly aggressive. But the book and it's tattered wasteland of the internet still has such tone setting power for me, set such an image up of an internet after humans: where the competing forces of exploitation have degraded and degraded and degraded the situation, pushing humans out.
Recently revisited on Peter's blog: https://www.rifters.com/crawl/?p=11220
ivan_gammel
The only way to solve it for decentralized messaging systems is a decentralized system for verification of identities based on chain of trust and use of digital signatures by default. It must be a legal framework supported by technical means. For example, id providers may be given a responsibility to confirm certain assumptions about their clients (is a real human, is adult etc) while keeping their identity confidential. The government and the corporations will know only what this person allows the id provider to disclose (unless there’s a legal basis for more, like a decision of the court to accept a lawsuit or a court order to identify suspect or witness). Id provider can issue an ID card that can be used as authentication factor. As long as a real person can be confirmed behind the nickname or email address, the cost of abuse will be permanent ban on a platform or on a network. Not many people will risk it. Natural candidates for id providers can be notaries.
vannevar
Yes, I think we'll see the rise of id-verified online communities. As long as all the other members of the community are also id-verified, the risk of abuse (bullying, doxing, etc) is minimized. This wouldn't stop someone from posting AI-generated content, but it would tend to suppress misinformation and spam, which arguably is the real issue. Would people complain about AI-generated content that is genuinely informative or thought-provoking?
intended
Verification does not stop harassment or bullying.
It will not stop misinformation either.
Verification is expensive and hard, and currently completely spoof-able. How will a Reddit community verify an ID? In person?
If Reddit itself verifies IDs, then nations across the world will start asking for those IDs and Reddit will have to furnish them.
ivan_gammel
The key is „decentralized“ and „chain of trust“. An ID provider does actual identification in person first, maybe collects some biometrics. An online community trusts the ID provider and just asks the necessary questions. A foreign government may force this online community to provide only the data it owns, i.e. the flag „true“ in „verification_completed“ column of the database, maybe an uuid of the person at the ID provider. How does it protect from harassment and bullying? It provides means to address them legally, because court will be able to get real identity of the criminal and the platform can just ban the real person for life, no new registrations and nicknames. Initially this may result in a surge of moderation requests, but eventually it will become less and less as people learn the new rules.
As for misinformation, as long as all actors are known and are real people, they should be allowed to speak. It’s not good to be a flood of fakes.
vannevar
>Verification does not stop harassment or bullying.
>It will not stop misinformation either.
I'm open to any evidence that either statement is true. The rational argument that verification will reduce harrassment, bullying, and misinformation is that the verified perpetrator can be permanently banished from the community for anti-social behavior, whereas an anonymous perpetrator can simply create a new account.
Do you have a rational counter-argument?
>If Reddit itself verifies IDs, then nations across the world will start asking for those IDs and Reddit will have to furnish them.
Every community will have to decide whether the benefits of anonymity outweigh the risks. On the whole, I think anonymity has been a net negative for online community, but I understand that others may disagree. They'll still be free to join anonymous communities. But I suspect that large-scale, verified communities will ultimately be the norm, because for everyday use people will prefer them. Obviously, they work better in countries with healthy, functional liberal democracies.
ChrisMarshallNY
Having links in comments has always been problematic.
For myself, I usually link to my own stuff; not because I am interested in promoting it, but as relevant backup/enhancement of what I am writing about. I think that a link to an article that goes into depth, or to a GitHub repo, is better than a rough (and lengthy) summary in a comment. It also gives others the opportunity to verify what I say. I like to stand behind my words.
I suspect that more than a few HN members have written karmabots, and also attackbots.
AJ007
I recall blogs from over 20 years ago, with blatant comment spam, where the blog author would respond to the comment spam individually as if it was real readers. Most didn't fall for that, but a few clearly didn't understand it.
I'm not sure LLMs deviate from a long term trend of increasing volume of information production. It certainly does break the little bubble we had from the early 1990s until 2022/3 where you could figure out you were talking to a real human based on the sophistication of the conversation. That was nice, as was usenet before spammers.
There is a bigger question of identify here. I believe the mistake is to go the path of: photo ID, voice verification, video verification (all trivially by-passable now.) Take another step further with Altman's eyeball thing, another mistake since a human can always be commandeered by a third party. In the long term do we really care that the person we are talking to is real or an AI model? Most of the conversations generated in the future will be AI. They may not care.
I think what actually matters more is some sort of larger history of identify and ownership, matching to what one wishes to (I see no problem with multiple IDs, nicks, avatars.) What does this identify represent? In a way, proof of work.
Now, when someone makes a comment somewhere, if it is just a throw away spam account there is no value. Sure, the spammers can and will do all of the extra stuff to build a fake identity just to promote some bullshit produce, but that already happens with real humans.
ChrisMarshallNY
> That was nice, as was usenet before spammers.
Not so sure I'd call it "nice."
I am ashamed to say that I was one of the reasons that it wasn't so "nice."
aspenmayer
Next, I'm sure, you'll be telling me you're not a bot, Mr Marshall?
https://www.youtube.com/watch?v=4VrLQXR7mKU
Previously (6 months ago but didn't trend, perhaps due for a repost?):
ChrisMarshallNY
Love it!
Thanks!
aspenmayer
Glad you liked it, though I will mention for others that it does involve self-harm, as that may be relevant information, but it is necessary to the story, and it did win an Academy Award for what it's worth, which I think was probably deserved, though I didn't see any of the other also-rans.
Thanks for having the courage to post under your real name, also, as you mentioned in another thread of yours I was reading. It's been a growth experience for me as well.
cafard
The one comment is either a splendid illustration or a great piece of sarcasm.
amelius
True. However I'm sure an LLM would be able to filter that one out without problems. /s
rsfern
In the case that it’s a human posting sarcastically, wouldn’t that be a false positive?
Smaug123
Depends what you're trying to identify. If you're trying to identify "machine-generated", yes. If you're trying to identify "spam", probably not? Spam posted sarcastically is no more something I'd want in my comments section than spam posted by a bot.
codr7
No worries, this won't last long.
Once the algorithms predominantly feed on their own shit the bazillion dollar clown party is over.
vanschelven
This has been debunked (to me) here: https://simonwillison.net/2024/Dec/31/llms-in-2024/#syntheti...
roxolotl
That seems to only say that synthetic data is a larger part of models today than in the past. The newer OpenAI models knowingly hallucinate more. Claude 4 seems great but not a multiplier better. Makes me think the effect of synthetic data is at best a net 0. Still has yet to really be seen though.
lucianbr
Disagreeing with something is not debunking.
sorokod
Debunked is a bit too strong. He qoutes from phi-4 repor that it is easier for the LLM to digest synthetic data. A bit like feeding broiler chickens other dead chickens.
Maybe one day we will have organic LLMs guaranteed to be fed only human generated content.
blahaj
[dead]
akoboldfrying
Even supposing the purported "model collapse" does occur, it doesn't destroy the LLMs we already have -- which are clearly already capable of fooling humans. I don't see the clown party being over, just reaching a stable equilibrium.
energy123
Exactly. It logically can't occur, even by the own flawed assumptions of the people that say this. Just freeze all training data at 2024 or keep existing models, the worse case scenario is the models will plateau.
monkeyelite
Google discovered the only way to ultimately resolve spam is to raise the cost to create it.
For web spam this was HTTPS. For account spam this is phone # 2fa. I think requiring a form of id or payment card is the next step.
wiether
So they are going to allow only YT premium subs to post comments?
Because if there's one place where Google didn't solve spam, it's on YT's comments
aleph_minus_one
> Because if there's one place where Google didn't solve spam, it's on YT's comments
I do believe that this problem is very self-inflicted (and perhaps even desired) by YouTube:
- The way the comments on YouTube are structured and ordered makes it very hard to make deep discussions on YouTube
- I think there is also a limit on the comment length on YouTube, which again makes it hard to write longer, sophisticated arguments.
- Videos for which a lot of comments a generated tend to become promoted by YouTube's algorithm. Thus YouTubers encourage viewers to write lots of comments (thus also a lot of low-quality comments), i.e. YouTube incentivizes that videos are "spammed" with comments. The correct solution would be to incentivize few, but high-quality comments (i.e. de-incentivize comments that contribute nothing valuable (i.e. worth your time to read)). This makes it much easier to detect and remove the (real) spam among them.
monkeyelite
Maybe, but I mean in general for internet participation.
safety1st
Indeed it is https://youtu.be/-gGLvg0n-uY?feature=shared
wslh
Twitter, LinkedIn, and others are following the credit card and id (KYC) way but the issue remains when people start automating interactions, not spam per se but it creates a waste of time since users cannot cope with the triggering of zillions of interactions that cannot be followed by human-time.
eastbound
If you make people pay to comment, content farms will gladly pay.
JimDabell
This doesn’t work in perpetuity. One of the reason why spam is so persistent is that when you ban a spammer, they can just create a new identity and go again. If payment is required then not only do they have to repeatedly pay every time they get banned, they need a new payment card too because you aren’t limited to banning their account – you can ban the payment mechanism they used.
intended
This is only to the point that it’s not profitable to spam further.
At some point your cost to dissuade spammers becomes a huge risk for humans to make mistakes of any sort.
At this point users mutiny.
monkeyelite
Yes… but there will be less spam and it will be more intelligent because the creator must break even.
d6e
What if we charged a small toll for comments. We create a web standard where you can precharge an amount to your browser account, then you get charged $0.02 for making a comment. The price could be progressively raised until the spammers stop. The profit could pay for website hosting. This would be affordable for users but prohibitively expensive for spammers.
ThrowawayR2
The problem originates from LLM service so the toll needs to be on LLM usage in a way that doesn't harm legitimate users but makes it unprofitable to abuse in bulk.
philjackson
I seem to remember MS having this idea for email many years ago.
Philpax
Amused that the third comment is the Tirreno guy continuing to spam his project [0]. Good ol' human spam will never go out of style!
[0]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
reconnecting
^~~ tirreno guy is here
Spam is one of the use cases for tirreno. I'm not sure why you'd call this spamming, as the tool is relevant to the problem.
hibikir
Since detecting LLMs is a silly end goal, the future of moderation probably needs LLMs too, but to evaluate text and see if it fits into blatant commercial speech. It will ruin places where some kinds of commercial speech is wanted (say, asking for a recommendation on reddit). Still, the mindless recommendation of crypto rugpulls and other similar scams will go away.
I am more concerned about voice alignment efforts, like someone creating over time 10k real-ish accounts attempt to contribute, but are doing so to just abuse upvote features to change perception. Ultimately figuring out what is a real measure of popularity , and what is just a campaign to, say, send people to your play is going to get even harder than it is now
aleph_minus_one
> It will ruin places where some kinds of commercial speech is wanted (say, asking for a recommendation on reddit).
There is also a dependence on the culture. For example, what in the USA would be considered a "recommendation" (such as on Reddit) would often be considered "insanely pushy advertising" in Germany.
With this in mind, wouldn't a pertial solution also be to become less tolerant of such pushy advertisement in such places (say on Reddit), even if they are done by honest users?
aspenmayer
When it's obvious that entire posts and users are fake, and knowing that product pages on Amazon (which are also sometimes fake) can change what product they list for sale, and since it is known that upvotes/likes/shares are openly for sale, is it really such a stretch to assume that all "recommendations" are as fake as the original question also likely is, until we have evidence to the contrary?
washmyelbows
There's a lot of people downplaying the importance of genuine online comments but the reality is that (outside of the bubbles lives in by many HN users) millions upon millions of people are meaningfully participating and forming their viewpoints based on them.
I suspect even the 'well I never trust online comments!' crowd here is not as immune to propaganda as they'd like to believe themselves to be
This keeps me up at night too. I’d like to stake the position that LLMs are antagonistic to the (beleaguered) idea of an internet.
LLMs increase the burden of effort on users to successfully share information with other humans.
LLMs are already close to indistinguishable from humans in chat; Bots are already better at persuading humans[1]. Suggesting that users who feel ineffective at conveying their ideas online, are better served by having a bot do it for them.
All of this, is effectively putting a fitness function on online interactions, increasing the cognitive effort required for humans to interact or be heard. I dont see this playing out in a healthy manner. The only steady state I can envision is where we assume that we ONLY talk to bots online.
Free speech and the market place of ideas, sees us bouncing ideas off of each other. Our way of refining our thoughts and forcing ourselves to test our ideas. This is the conversation that is meant to be the bedrock of democratic societies.
It does not envisage an environment where the exchange of ideas is into a bot.
Yes yes, this is a sky is falling view - not everyone is going to fall off the deep end, and not everyone is going to use a bot.
In a funny way, LLMs will outcompete average forum critters and trolls for their ecological niches.
[1] (https://arxiv.org/pdf/2505.09662)