Fans Are Better Than Tech at Organizing Information Online (2019)
56 comments
·February 22, 2025cjs_ac
cameronh90
That "refusal" is by design.
Using standardisation to improve productivity has been the backbone of the industrial revolution ever since we first standardised screw threads in the 19th century.
It's undeniable that we've lost some cultural richness in the process, but if that means I don't have to work a field for 12 hours a day just to get enough food to survive like my ancestors did, then I'll accept the tradeoff.
hn_throwaway_99
> I don't have to work a field for 12 hours a day just to get enough food to survive like my ancestors did
Not disagreeing with your overall point but this isn't correct. Medieval peasants worked about 150 days a year, and while they were obviously very busy during planting and harvesting times, they had lighter workloads at other points in the year.
derektank
These studies generally don't include the large amounts of domestic labor that had to be done in addition to farming. Lots of food preservation, home repairs, tending to fires, etc that we simply don't have to worry about today in nearly the same way. There was a lot less work in winter, but there was also substantially less food and with less energy for recreational activity, a lot of people's leisure time was occupied with sleeping.
afthonos
Medieval peasants were required to work 150 days a year on their lord’s estate. That was how they paid for the land they had to farm the rest of the time to survive. They also had to spin, weave, and make clothes.
Reference: https://www.adamsmith.org/blog/regulation-industry/medieval-...
nthingtohide
I think there was a article here which stated that 1 person had to work for 1 month to get 1 pound of sugar or something. Today it is equivalent to 1 hours of work, something like that. You are narrowing your focus too much. The abundance is really the key here.
tmpz22
This is poignant in the age of LLMs. What cultural richness are we going to lose as a result? What societal cohesion are we going to lose? What wealth inequality are we going to create?
Secretmapper
There's actually something interesting here in my opinion, which is that LLMs do not necessarily need to hinder standardization.
For example, we have standardized schooling/exams, because that's the proven scalable way we can have for children - essentially a factory spitting out different grade levels/seniority.
But LLMs can break this standardization by being able to tailor student needs in a scalable manner.
However this takes a huge amount of action, and that's going to be the pain point in the near future as we humans tend towards the easy/greedy paths.
Cultural richness - LLMs are also very bad in this regard due to the bias towards majority viewpoints. This reminds me of a recent HN thread [0] discussing how AI is hindering the adoption of new technologies. In some ways, this happens because AI tends to favor mainstream perspectives, making it biased against "new" or "fringe" viewpoints.
bloomingkales
We will only lose our culture if powerful people exert dominance. That's how cultures were withered throughout history, a dominant violent culture comes and pushes its wicked pursuit. It takes decades or centuries to recover your culture after it happens. Shout out to Native Americans, keep trucking, you'll get it all back (truly sorry about all this).
People talk about the glory of Rome, but it was utterly vile. They either enslaved or exterminated whoever they conquered. If they had an LLM, they might keep you alive and brainwash you with it. Maybe.
Per gpt:
"In the military of ancient Rome, decimation (from Latin decimatio 'removal of a tenth') was a form of military discipline in which every tenth man in a group was executed by members of his cohort."
This was done to thwart desertion.
This is similar to:
- PIP.
- Email me what you did last week or your fired.
- Blanket 10% layoffs in high profit environments
- Blanket 10% cuts in general
I'm less scared about AI than the culture of dominance utilizing it. We are not the only people that ever lived on planet fucking earth, everything has been done already in some other form of matter. The wicked animals you see now days in power have lived many times over in past histories, we've seen these exact people many times. We simply get hoodwinked when they seduce us with casual talk about space, science, and the world. Decent people begin to think they are just like us. Nope, they are not like us.
Sometimes you just have to connect the dots and accept it. History precedes us and owns us.
computerthings
[dead]
monknomo
I saw someone saying that it is as though tech read Seeing Like a State and took the wrong lesson
I think tech does have the drive to make things legible, and is falling into the same trap as described in the book where efficiencies or processes that cannot be described in the format desired at the top leads to them being discarded. And the legibility issues mean that the impact of discarding these types of things is not properly understood
_Algernon_
And this has been true since states wrought the cadastral map, last names, the metric system and taxes on us.
The main issue today is that the techologies shaping reality are in the control of private, non-democratic institutions, selling this power to the highest bidder (including hostile foreign powers).
drivers99
> This is just that 'reality has a surprising amount of detail' phenomenon all over again
Ever since that was posted[0] a week ago (reposted, but it was the first time I saw it), I've been filtering almost everything I see or do through that lens. It's really eye opening.
waveBidder
if no one has ever pointed you at Seeing like a State, it's about the political and economic implications of this fact. https://en.wikipedia.org/wiki/Seeing_Like_a_State
There's also Bit's About Money describing this effect in banks https://www.bitsaboutmoney.com/archive/seeing-like-a-bank/
mschuster91
> That said, this refusal is not new; my parents worked for a bank that was trying to adopt some IBM technologies in the 1980s, and they said that IBM couldn't accommodate the bank's requirements.
That misunderstanding is something that causes a lot of grief in SAP introductions.
When working with large enterprise software, customization is your enemy - and every time you have to customize something there, you should ask yourself if you shouldn't re-think your business processes instead. Often enough IBM, SAP or whatever have considerably more experience than you.
6DM
Ironically at my company, our custom software made us too flexible. There was too many crazy left field demands that weren't really that useful.
So when it came time to think about next steps. There was real appeal in being able to say, "No it's not supported in xyz software we just adopted". This prevents us from looking like the bad guy who's just getting in the way and should be laid off because we didn't want to spend 2 months implementing a hair brained idea that would only give us a net return of like four or five thousand dollars.
BJones12
The #1 purpose of ERP software is to take the blame.
hn_throwaway_99
And that customization kills you over time. Heck, a big reason a lot of banks and other big enterprises eventually move over to one of the big vendors is because they've built up a hodgepodge of now unmaintainable crap that nobody can touch.
This is also why Salesforce was so successful and why it killed on-prem enterprise deployments in a lot of places. No longer a ton of different essentially unique versions to manage that make it nearly impossible to upgrade. Salesforce obviously supports customization but in a much more controlled fashion than was common at the time, and it's why it won out over Seibel Systems in the early 00s.
robocat
Customisation has other major costs.
1: It locks you into a vendor. The vendor strongly encourages customisation for this reason (and for other reasons that are mostly against a business' interests)
2: onboarding staff is expensive because they need to learn your unique systems. If you're using a package/service off the shelf you can often advertise for people with knowledge of the package/service e.g. AWS versus inhouse e.g. known payroll system versus custom.
3: internal costs to upgrade. With stock standard systems your upgrades are cheap because the costs are amortised across many companies e.g. mobile app development in 2010's. With custom systems you pay for everything. The costs are often invisible e.g. horrific Mobile App support.
4: shitty custom internal systems. Sometimes the customisation provides business benefits but we have all worked with crappy internal systems where the benefits versus costs were slim.
Two examples of bad customisation requests:
A) HR systems in small companies. For some reason HR managers demand fractally complex systems and they all want different things so developing common features is really hard.
B) larger customers with demanding internal rules/regulations e.g. a client asking us to conform our UI to their UI standard. No way it was worth it for us or for them. Plus dealing with their UI despots would have been hell. I think their internal software teams decided to build inhouse instead (I bet the outcome was shit).
magicalhippo
We got one of our largest and definitely most complex customers not long ago. Being rather small, we pushed hard against any customization that we didn't feel was necessary due to business demands, and asked them to follow the way we'd successfully used with our many other customers.
After the went live and the dust had settled, they thanked us and said they were glad we had pushed back. It had forced them to rethink how they worked but the result was much better and more optimized processes.
BJones12
> every time you have to customize something there, you should ask yourself if you shouldn't re-think your business processes instead
I've heard this as "the best flavor is vanilla". It referred to a hospital aligning healthcare business processes with industry-standard software workflows.
ziddoap
>users can put in whatever tags they want. (Autocomplete is there to help, but they don't have to use it.) Then behind the scenes, human volunteers look up any new tags that no one else has used before and match them with any applicable existing tags, a process known as tag wrangling.
This is what booru sites have done for ages. Danbooru is the only booru mentioned in the article (and just as a passing mention despite being a few years older than Archive of our Own). I can only guess that AO3 is a bit more palatable than boorus for a general audience.
But this style of tagging really is the best of both worlds. The only downside is, as mentioned, the requirement for manual labor. I implement a similar system myself, obviously at a way smaller scale, for my own pictures. Basically I tag things as I feel like it at the time, and then every few months I "wrangle" the tags, as they do here.
jchw
It's mildly amusing that AO3 is considered more palatable than booru websites. Both AO3 and many of the boorus have content that would be hard for most folks to stomach. I guess AO3 has the advantage of being mostly text, (which seems to be generally less visceral to people,) but it is also nearly completely uncensored, since that's kind of the point of it.
ziddoap
Yes, overall I wouldn't describe either of them as particularly palatable to a general audience, haha. Your hunch is the same as mine -- text has much less potential for instant shock. On AO3 you have to, sort of, go out of your way (by reading, or searching for specific tags, etc.) to hit the unsavory stuff.
Cthulhu_
Exactly that, it's labor intensive, while tech tries to reduce labor intensivity by using magic or rules or whatnot. Related and larger scale projects are sites like Wikipedia or TVTropes, although Wikipedia makes use of a lot of automation as well.
canadiantim
Can't you just use AI to wrangle now instead?
JansjoFromIkea
The huge number of communities that have elected to move to Discord, an information black hole, seems like a strong counterargument to the generality of this claim.
pjc50
This seems to be a generational issue. It makes me sad, but I suspect it's because doing things "in public" on the Internet has become increasingly exhausting and people have retreated to gated communities. It's not a black hole, it's a "dark forest": you can't find their information because they're scared of you.
bonoboTP
It's not super new though, people had private forums in the phpbb era as well, which you could only read after login. Or think about IRC, private torrent trackers etc.
newsclues
OSINT communities use discord to communicate and discuss information from mostly twitter and telegram and organize and share the packaged information.
Information organization is hard and lots of work. Not everyone (or every community) has the same need to organize information, and not everyone has the ability or inclination to do so.
Information getting lost on discord is a problem, but that just means the useful stuff needs to be archived elsewhere, and frequently there is another distribution network involved.
InDubioProRubio
Accurate Problem description, which can easily be countered with .. [extra effort expended in idealized behaviours] .. [I could paste Lore Ipsum here everybody stopped reading at the idealization]
null
mrweasel
The upsetting part isn't that fans are better at organising information, it's that companies and governments are so incredibly bad at it.
Limiting this to "tech" isn't really fair, because most other organisations isn't doing much better. Right now the entire world is trying to avoid collecting, creating and organising information by feeding it through AIs, which pretty much depend on organisation having done exactly that in advance.
There's a huge potential of business and organisations that will do the dirty work and focus on information creation, collection and organisation internally. Just think of customer service, when was the last time an FAQ or self service guide provided any value? It happens so rarely because business don't care to keep things updated or even spend money on good writing. Nope, better to invest in an AI chat bot than updating your website.
carlosjobim
Businesses count on middle men to do all that, so that they can focus on the things they know how to do.
ks2048
Reminds me of Google's old motto, "Organizing the World's information" (is it still?). Even at their best, Google was never really "organizing" as much as "making it searchable". e.g. YouTube - for a given band, why can't I browse a list of their past concerts (with dates and locations) and see all videos from each?
Apocryphon
Funnily enough, that was the domain of Yahoo! with its directories.
datadrivenangel
People who care deeply tend to do things better than people who don't care. Large organizations only organize information to the point of short term marginal profitability. If being better organized beyond a point doesn't get you more money in the next few months/years, it isn't worth it to most businesses.
Likewise, being extremely organized personally is probably unnecessary, but to each their own.
commandlinefan
> better than people who don't care
IME it goes deeper than that: caring is ruthlessly stamped out by crushing bureaucratic processes. Results don't matter, what matters is never being responsible for a mistake. Fans can fix minor mistakes - in corporate America, even the smallest misjudgment is fatal. The only way to win is to never actually do anything, but keep up the appearance of being busy.
pjc50
Process was invented as a means to get useful output from people who don't care. Of which there's an unlimited supply.
sumtechguy
I think it is more simple.
They really do not care much about it. Lets say you sell a product. You have had 3 revisions of that product. Support on that item is very minimal. The current version is what you sell and that is it. The previous versions are just historical interesting things. Properly archiving it and cataloging it takes time and money. Does talking about your older items sell you more items? Maybe maybe not. It is just a thing you sell. Not any sort of historical artifact to be preserved. It is just how you make a living scraping the margin.
But to a collector or 'fan'. All 3 hold importance to your collection. All of the details/metadata are mapped out so you know why v1 is worse/better/interesting from v2/v3.
What is worse in many of the 'fan' cases is you find it is usually 1-2 people mapping that stuff out. They map out what they find interesting. Many also get heaps of verbal and legal abuse from both the companies and other people on the net. So they bail out and whatever 'fan' site came out of it, rots. Like one project I found a few months ago. Tons of stuff mapped out but some errors here and there, no big deal. I have a set of patches ready to go to fix it. But the orig author has ghosted. They got tired of tons of abuse from other 'fans'. Frankly what I see in the previous requests I want nothing to do with it. So I keep my stuff private.
commandlinefan
> They really do not care much about it
Or rather, the people who do care aren't the ones making the decisions.
kuharich
Past comments: https://news.ycombinator.com/item?id=20156791
Kinrany
Archive Of Our Own is not just any group of fans of course, the very name of the website refers to them deciding to write a better version of fanfiction.net. The award reflects the main goal of the website, not something that happened by accident.
Daniel_Van_Zant
Is there any effort to organize scientific literature like this? I know journals often generate tags for papers but those can often be quite poor and restricted to the field The journal is in. I would happily join a volunteer effort to create tags and do some tag-wrangling for scientific literature in my research area.
xdavidliu
> At a time when we're trying to figure out how to make the internet livable for humans, without exploiting other humans in the process, AO3 (AO3, to its friends) offers something the rest of tech could learn from.
> AO3 (AO3, to its friends)
What was that supposed to be? Is that a typo?
pjc50
Somebody probably wrote it expanded (Archive of Our Own), but it got automatically shortened again by search and replace.
esafak
Sure, it works for your anime fan site, but what about when money is involved, like in a search engine? That attracts bad actors, who can use their power to abuse your site.
jjk166
Solvable with a combination of robustness and stochasticity. If you say need two randomly selected people to approve an edit, and you flag users who make too many attempts at making rejected edits (either one user, lots of bad edits or lots of users trying to make the same bad edit), the only way for a bad actor to reliably make undesirable edits would be to gain control of a very large number of potential approvers. More generally, if the cost to effectively manipulate the system is greater than the perceived reward from manipulating the system, bad actors aren't an issue.
AntiEgo
I noticed the article didn't speculate on why, but I think you nailed it. This system is probably incompatible with a commercial site. It requires too many volunteers.
This is just that 'reality has a surprising amount of detail' phenomenon all over again. Big tech simply isn't willing to engage with the detail, and increasingly expects the world to conform to its expectations. That said, this refusal is not new; my parents worked for a bank that was trying to adopt some IBM technologies in the 1980s, and they said that IBM couldn't accommodate the bank's requirements.