Preparing for when the machine stops
46 comments
·May 6, 2025robin_reala
bayindirh
I want to add that "Pump Six and Other Stories" by Paolo Bacigalupi is also a great read. "Pump Six" also paints a similar picture to Forster's "The Machine Stops", but with a different perspective.
Both are great and horrifying at the same time.
kreelman
I'm reminded of "The Feeling of Power" by Isaac Asimov.
In this story, everyone relies on automation so much that they forget how to do mathematics. A lone, nerdy operator re-learns the skill of 2 digit multiplication in the process of working through automated weapons targeting issues, astounding his superiors.
The concept of forgotten arts seems like a common thread here.
This is a PDF of the text. https://ia600806.us.archive.org/20/items/TheFeelingOfPower/T...
This is the Wikipedia overview. https://en.wikipedia.org/wiki/The_Feeling_of_Power
First heard this one at school.
taneq
Or in the opposite direction, “Who can replace a man?”
strict9
>No one remembers how to fix it.
This thought exercise also gets interesting with software development. There are few prospects available for new grads and junior devs now. Relentless offshoring, headcount reductions, and AI promises from CEOs have hollowed out the tech landscape. There are very few opportunities for young people to professionally develop the knowledge to keep the systems running.
And now most startups are focused on eliminating staff via AI. The people who would keep the systems running. I'm not sure where all of this leads to in a few years.
kmoser
"No one remembers how to fix it" sounds like the COBOL situation today. The reality is that the bar to learning COBOL is quite low: compilers and documentation are readily available for most platforms.
What's lacking is incentive and drive: there are just too many other shiny tools to distract us, including jobs that sound more glamorous and/or pay just as well.
A young person with the drive to learn just about anything has the means to do so, if they really want.
null
hnthrow90348765
Just go back to books or other sources of knowledge and restart from scratch. It shouldn't take long to get your thinking back.
W.R.T. any large industry crash, no one's going to care if you can do your Angular (or specific tech skill) stuff in 1 days vs. 5 days, so why the emphasis on speed in this scenario? Both System 1 or 2 thinking is fine here.
We've enjoyed some pretty great technical advances for the past 40 years, even with a 10 year "dark age" we're still net ahead. We can rebuild and relearn a lot of stuff in 10 years.
gnatolf
A lot of the comments on HN lately are rightfully focused on this formative brain exercise that leads to intuition and conceptual understanding that is chiselled away by the shortcuts that GenAI provides. I wonder where the gain of productivity from GenAI and the drop off in 'our brain'-quality intersects.
AJ007
It's actually not that different than talking with employees, however, the LLMs still have very significant shortfalls (which you know about after using them a lot.)
If a manager doesn't know anything about what their employees are working on, they are basically fucked. That much holds up with LLMs. The simple stuff mostly works, but the complex stuff isn't going to pan out for you, and it will take a while to figure out that's the direction you went in.
turtleyacht
One comparison is with Stack Overflow (SO). Given a task, there are usually multiple answers. The question may not even be relevant; often, multiple question pages must be compared.
The best answer is the one that fits the aesthetics of my approach--one that didn't exist before (there was only the problem before), but the answer is simple, straightforward, or adaptable.
Having multiple answers is good because different minds evaluated the question. It is a buffet of alternatives, starting from others' first principles, mistakes, and experience. Some are rejected outright from some tacit taste organ. Others become long-lived browser tabs, a promise to read carefully someday (never).
All this is void if it turns out using SO is similarly degenerative in the same way, though.
emorning3
We should probably require AI to always be able to explain it's conclusions.
That way we can quickly assimilate knowledge from the AI and theoretically always have at least as much knowledge as the AI.
I suppose it also means that we can verify that the AI is not lying to us.
null
ptx
Unfortunately we don't have that kind of AI. We only have the useless kind.
lurk2
> In E.M. Forster’s short story The Machine Stops, he paints a future where a vast machine handles every aspect of human life. People live isolated lives, fully dependent on the machine. They don’t know how it works. They only know how to ask it for things. When the machine breaks down, society collapses. No one remembers how to fix it.
I used to think about this in math class. I could figure out what to do with my calculator most of the time, but I didn’t have any intuitive sense of how things worked. The sine, cosine, and tangent functions are still just black boxes to me, I have no idea what they actually do or how I would calculate their values. I often daydreamed about finding myself on a desert island, needing to make use of trigonometry to rebuild civilization, but not being able to find the angles that I needed.
Lots of other skills are lost this way. I don’t know how to join wood or sew a stitch, but I do know how to operate a nail gun and work a sewing machine. I couldn’t fix either device, but if I couldn’t find anyone to fix them and couldn’t obtain new ones, I would likely have bigger problems to worry about. Most people will only ever need to view these devices as black boxes; the benefits of specialization generally offset the costs introduced by abstraction, absent major market disruptions (e.g. supply chain breakdowns, changing regulatory frameworks, etc.). Most people in human history have spent their lives as generalists on a farm. This hedges the individual against a lot of risk (the generalist can likely always find some work to do), but the real strides in risk management are made by specialists living in urban settlements.
gnfargbl
>The sine, cosine, and tangent functions are still just black boxes to me, I have no idea what they actually do
They are functions which tell you how to relate different parameters of a triangle. Concepts in mathematics are often interconnected, and trig functions appear in a lot of other (interesting) contexts as well, but fundamentally I don't see them as anything "more" than the SOHCAHTOA you were taught at school.
> or how I would calculate their values.
Without a lookup table? You would need some kind of a way to express the functions in terms of other mathematical functions which you know how to do, like multiplications and additions. Sometimes you can do this with a series expansion. Computers sometimes use a variant of the CORDIC algorithm. Both of those things are clever ideas in their own right, but you don't need to understand them to know what trig functions do.
> I often daydreamed about finding myself on a desert island, needing to make use of trigonometry to rebuild civilization, but not being able to find the angles that I needed.
If you had a circle, a ruler, a pen and some paper then you'd get the idea to make yourself a lookup table pretty quickly!
I think the point I'm making is that we get used to assuming that knowledge is so deep and complex a thing that we can never really know anything. But often, knowledge isn't as intricate as that. If everyone forgot about trig functions today, they'd be rediscovered tomorrow.
XorNot
The thing is YAGNI applies here. "What if society collapses?" is really just a LARP justification for doing whatever you already wanted to do.
It's why "preppers" buy an armory of weapons, but don't make friends with their neighbors, become contributors to their local town or advocate for infrastructure improvements or sustainable farming policies.
tmnvix
> It's why "preppers" buy an armory of weapons, but don't make friends with their neighbors
Interestingly, when I've seen interviews with deeply committed preppers, they almost always seem to come around to the conclusion that community is the most significant factor in their planning.
I find it fascinating how that juxtaposes with the (possibly well earned) cliche of preppers as intensely individualist libertarians who reject society. I suspect there's some relationship here to the idea that if you go far enough to the left or right, you find that the spectrum is circular and not linear.
cadamsdotcom
> When the machine breaks down, society collapses. No one remembers how to fix it.
The example is unrealistic because that's a very easily anticipated single point of failure.
We've been building systems without single points of failure for millennia.
There will never be only one machine. Already in AI we have dozens of models. They come from multiple diverse - unaffiliated! - cultures, countries, and ideologies.
metalrain
If there would be massive EMP that fried all chips around me, I would be for short while somewhat useless.
Most of what I've done is in software, I could not build computer from electronic parts, not even full adder from memory. Maybe I could read some schematics, but most measurement devices use chips as well, it would very difficult.
I think preparation should be skills, useful in any environment.
kmoser
If all chips were fried, you'd have more pressing things to worry about than rebuilding a crude computer. I think the planning, documenting, and problem solving skills you learned as a software developer would still help you with whatever you were called upon to do.
Zigurd
Having AI write the kind of software humans write is not even scratching the surface of what will probably happen. Just as generative CAD tools produce mechanical designs that would be prohibitively complex and time consuming for humans to design and verify, we're eventually going to see, and by eventually, I mean pretty soon, software no human could have written.
holoduke
Its already the case for a long time with assembly generated code. No human could recreate it. Ai code is just another abstraction on top of a layer that soon won't be touched anymore by common devs.
huphreem
It’s a fine enough journal entry, and I agree with the underlying sentiment of the piece, but can I crowdsource an explanation of the conclusive pith?
> Learning to learn is a noble idea. But more important is learning to unlearn, and knowing when to resist the comfort of automation.
I feel strongly about “teaching people to teach themselves” which seems a direct analog to “learning to learn”, but I am at a complete loss for what “learning to unlearn” means, especially as it relates to resisting automation.
Is the idea that you need to “unlearn” wanting to “learn” automation so you can keep “learning” more deeply about things you have already “learned”?
jjmarr
If we had to colonize another planet I think we'd need 10,000 of the absolute best in just semiconductor fabrication/design if you wanted to create computers.
craftkiller
That may be if we wanted to create computers similar to what we use today, but wiring diagrams of an extremely basic CPU was part of everyone's CS undergrad at my school (and I assume most other schools too?) so if we just needed to make a turing machine with discrete components I'm sure a couple of us could dig up those memories and figure it out again in a pinch.
jjmarr
> wiring diagrams of an extremely basic CPU was part of everyone's CS undergrad at my school (and I assume most other schools too?)
I went to Toronto Metropolitan University (ranked 800-1000 globally) and actually designing that basic CPU was too hard for most students, so they just gave us VHDL code to copy & paste into Quartus II.
Their VHDL code had bugs in it which I discovered after doing the project myself from scratch and comparing my answer with the "correct" one.
This has been the policy for many years now. Shockingly our co-op placement rates have dropped from 70% to 30%.
cheschire
If farmers had blogs during industrialization, I suspect there would’ve been a lot of this.
Most people don’t know how to grow a potato. So?
korse
Separation from supply leads people to take supply for granted. In general, taking things for granted is risky both mentally and physically. This applies to more than potatoes.
jonnypotty
The problem comes when no one knows how to grow a potato any more, not when most people don't.
theamk
I don't think "no one" will even happen - nor with potato, nor with mechanical skills. There are books which tell how to do all sort of things, starting from the refining metal from ores, and there are people reading them. I have no doubt that minute details will be forgotten, but they were discovered once and could be discovered again if needed.
Yes, horses were replaced, but go outside of city and there are many horse farms, and some of them offer horse riding classes to general public. Should something happen to the cars, the knowledge will come back. Same way, even if an average JS programmer has no idea what "differential pair" stands for, there are plenty of people who _know_ what this is, and can at least read the eye diagram.
cheschire
And yet people still do. So the problem being discussed is demonstrably unlikely compared to the more likely benefit of significant scaling potential for humanity
NathanKP
Back to AI: Would you agree that the problem comes when no knows how to think any more, not when most people don't?
Personally, I'm pretty concerned about both. The fact that many people don't have basic survival skills like sourcing their own food, safe drinking water, and heat. And the fact that that many people lack basic thinking skills: ability to detect misinformation, or deal with the challenges and inaccuracies of flaky AI.
In an ideal world society everyone who is capable has a higher level of training in both. But under modern oligarchic capitalism there are advantages to ensuring that people have neither skill: survival or thinking.
ChrisMarshallNY
> it’s easy to get by without deeply understanding the code you’re deploying.
I feel that's been an issue for a long time, with the heavy reliance on dependencies. AI tools are really just a furtherance of the model already in use.
I have the luxury of developing software as a craft; not as a vocation. I deliberately do stuff "the hard way," because I feel more fulfilled, doing so.
You can read the Forster short story referenced at https://standardebooks.org/ebooks/e-m-forster/short-fiction/... . It’s impressive speculative fiction by any measure, let alone for 1909.