US and UK refuse to sign AI safety declaration at summit

It's not the fault of the printing press that the Church built its empire upon the restriction of information and was willing to commit bloodshed to hold onto its power.

All you've done is explain why the printing press was so important and necessary in order to break down previous unwarranted power structures. I have a similar hope for AGI. The alternative is that the incompant power structure instead benefits from AGI and uses it for oppression, which would mean it's not comparable to the printing press as such.

Lots of good content here, but the main group that “suffered” from the invention and spread of the printing press was the aristocracy, so i am not shedding tears.

As for “breaking” Christianity: Christianity has been one schism after another for 2000 years: a schism from a schism from a schism. Power plays all the way down to Magog.

Socrates complained about how writing and the big boom in using the new Greek alphabet was ruining civilization and true learning.

And on and on it goes.

I think that is overstating the relevance of the printing press vs existing power struggles, rivalries, discontent, etc. - it wasn't some sort of vacuum that the reformation happened in, for example.

Religious schisms happened before the printing press, too. There was the Great Schism in 1054 in Christianity, for example.

I dont understand, why any highly sophisticated AI should invest that much resources to kill us instead of investing it to relocating and protecting itself.

Yes, ants could technically conspire to sneak up to you while you sleep and bite you all at once to kill you, so do you go out to eradicate all ants?

I'm very glad that it broke the power of the Catholic church (and I was raised in a Catholic family). It allowed the enlightenment to happen and freedom from dogma. I don't think it it broke Christianity at all. It brought actual Christianity to the masses because the bible was printed in their own languages rather than Latin. The catholic church burnt people at the stake for creating non Latin bibles (William Tyndale for example).

That's a very thought provoking insight regarding to the often repeated "printing press doomsayer" talking point. Thank you!

> And since their share of the fruits of that invention was mostly bloodshed, job loss, and shattering of the world order they knew, I wouldn't blame them from being pissed off about getting the short end of the stick, and perhaps looking for ways to undo it.

“A society grows great when old men plant trees in whose shade they know they shall never sit”

So much this

But if the state approaches a technology with intent it is usually for the purposes of a military offence. I don't think that is a good idea in the context of AI! Although I also don't think there is any stopping it. The US has things like DARPA for example and a lot of Chinese investment seems to be done with the intent of providing capabilities to their army.

The list of things states have attempted to deploy offensively is nearly endless. Modern operations research arguably came out of the British empire attempting (succeeding) to weaponise mathematics. If you give a state fertiliser it makes bombs, if you give it nuclear power it makes bombs, if you give it drones it makes bombs, if you give it advanced science or engineering of any form it makes bombs. States are the most ingenious system for turning things into bombs that we've ever invented; in the grand old days of siege warfare they even managed to weaponise corpses, refuse and junk because it turned out lobbing that stuff at the enemy was effective. The entire spectrum of technology from nothing to nanotech, hurled at enemies to kill them.

We'd all love if states commit to not doing evil but the state is the entity most active at figuring out how to use new tech X for evil.

> I think it's important for a state to declare it's intent with a technology. The alternative is arguing the idea that technology advances autonomously, independent of human interactions, values, or ideas

The sleight of hand here is the implication that human interactions, values, and ideas are only expressed through the state.

The idea is that by its very nature as an agent that attempts to make the best action to achieve a goal, assuming it can get good enough, the best action will be to improve itself so it can better achieve its goal. In fact we humans are doing the same thing, we can't really improve our intelligence directly but we are trying to create AI to achieve our goals, and there's no reason that the AI itself wouldn't do so assuming it's capable and we don't attempt to stop it, and currently we don't really know how to reliably control it.

We have absolutely no idea how to specify human values in a robust way which is what we would need to figure out to build this safely

I usually don't engage on A[GS]I on here, but I feel like this is a decent time for an exception -- you're certainly well spoken and clear, which helps! Three things:

  (I) All of the focus on AGI is a distraction.

  (II) "AI" (the goalpost shifting for marketing purposes to make that name apply to chatbots is a stretch honestly)

I'd love to get into the weeds on the different kinds of intelligence and why being to absolutist about the term can get real Faustian real quick, but these quotes bring up a more convincing, fundamental point: these chatbots are damn impressive. They do something--intuitive inference+fluent language use--that was impossible yesterday, and many experts would've guessed was decades away at least, if not centuries. Truly intelligent or not on their own, that's a more important development than you imply here.

Finally, that brings me to the crux:

  (III) AI... is a technology like any other

  I’ve always thought of A.I. as the most profound technology humanity is working on-more profound than fire or electricity or anything that we’ve done in the past. It gets to the essence of what intelligence is, what humanity is. We are developing technology which, for sure, one day will be far more capable than anything we’ve ever seen before. [3]

1) This kind of talk is so grandiose that it seems like a questionable move if that's the goal,

2) it's a sentiment echoed by many scientists (as I mentioned at the start of this rant) and

3) the unprecedented investments made across the world into the DL boom speak for themselves, sincerity-wise.

Yes, this is because AI will create uber-efficient factories, upset labor relations, produce terrifying autonomous weapons, and all that stuff we're used to hearing about from the likes of Bostrom[4], Yudkowsky[5], and my personal fave, Huw Price[6]. But Pichai's raising something even more fundamental: the prospect of artificial people. Even if we ignore the I-Robot-style concerns about their potential moral standing, that is just a fundamentally spooky prospect, bringing very fundamental questions of A) individual worth and B) the nature of human cognition to the fore. And, to circle back: distinct from anything we've seen before.

To close this long anxiety-driven manuscript, I'll end with a quote from an underappreciated philosopher of technology named Lewis Mumford on what he called "neotechnics":

  The scientific method, whose chief advances had been in mathematics and the physical sciences, took possession of other domains of experience: the living organism and human society also became the objects of systematic investigation... instead of mechanism forming a pattern for life, living organisms began to form a pattern for mechanism.
  In short, the concepts of science--hitherto associated largely with the cosmic, the inorganic, the "mechanical"--were now applied to every phase of human experience and every manifestation of life... men sought for an underlying order and logic of events which would embrace more complex manifestations.[7]

[1] OpenAI's Charter https://web.archive.org/web/20230714043611/https://openai.co...

[2] Investigation of a famous AI quote https://quoteinvestigator.com/2024/06/20/not-ai/

[3] Pichai, 2023: "AI is more profound than fire or electricity" https://fortune.com/2023/04/17/sundar-pichai-a-i-more-profou...

[4] Bostrom, 2014: Superintelligence https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...

[5] Yudkowsky, 2013: Intelligence Explosion Microeconomics https://intelligence.org/files/IEM.pdf

[6] Huw Price's bio @ The Center for Existential Risk https://www.cser.ac.uk/team/huw-price/

[7] Mumford, 1934: Technics and Civilization https://archive.org/details/in.ernet.dli.2015.49974

And imagine if private companies had had the resources to develop nuclear weapons and the US government had decided it didn’t need to even regulate them.

If it weren't for one guy -- literally one person, one vote -- out of three who were on a submarine, the Cuban Missile Crisis would have escalated to a nuclear strike on the US Navy. Whether we would have followed with nuclear strikes on Russia, who knows. But you trying to pretend that we didn't come incredibly close to disaster is just totally unfounded in history.

Especially when you consider -- we came that close despite incredible international efforts at constraining nuclear escalation. What you are arguing for now is like arguing to go back and stop all of that because it clearly wasn't necessary.

i see your point but the analogy doesn't get very far. For example, nuclear weapons were never mass marketed to the public. Nor is it possible to push the bounds of nuclear weapon yield by a private business, university, r/d lab, group of friends, etc.

Note that we only got to observe outcomes in which we didn't die from nuclear annihilation. https://en.wikipedia.org/wiki/Anthropic_principle

>Just because it has not come to pass yet does not mean they were wrong.

This assertion is meaningless because it can be applied to anything.

"I think vaccines cause autism and will cause human annihilation" - just because it has not yet come to pass does not mean it is wrong.

> What advice can we can take from it?

That we often won't have control over big changes affecting our lives, so be prepared. If possible, get out in front and ride the wave. If not, duck under and don't let it churn you up too much.

They had some sort of revolution the previous few centuries too.

Pretending that Europe wasn't in a perpetual blood bath since the end of the Pax Romana until 1815 shows a gross ignorance of basic facts.

The printing press was a net positive in every time scale.

Given countries at the time were all monarchies with limited rights, I'm not sure if it's too comparable.

I don't disagree that money (and therefore capitalism or frankly any financial system) is unable to create happiness.

I disagree with your example, however, as the most basic tenet of capitalism is that when there is a demand, someone will come along to fill it.

Addressing your example specifically, there's a fixed supply of housing in capitalist countries not because people don't want to build houses, but because government or bureacracy artificially limits the supply or creates other disincentives that amount to the same thing.

Let's not ascribe the possession of higher level concepts like a 'promise' to abstract entities. Reserve that for individuals. As with some economic theories, you appear to have a zero sum game outlook which is, I submit, readily demolished.

There are some thoughts on this here: https://www.playforthoughts.com/blog/concepts-from-game-theo...

> There's a lot of pearl-clutching about social media, algorithms, and "data", but you'll find far more people in tech (including FAANG) who are actively working on privacy technology, sustainable development and so on then you will find people caring about the environment by going into oil & gas, for example.

Sure, we don't need to talk about how certain Big Oil companies knew about the climate catastrophe before any scientists publicly talked about it, or how tobacco companies knew their product was an addictive drug while blatantly lying about it even in public hearings.

But it's ironic to mention FAANG given what the F is for if you recall that when the algorithmic timeline was first introduced by Facebook, the response from Facebook to criticism was literally that satisfaction went down but engagement went up. People directly felt that the algorithm made them more unhappy, more isolated and overall less satisfied but because it was more addictive, because it created more "engagement", Facebook doubled down on it.

Also "sustainable" stopped being a talking point when the tech industry became obsessed with LLMs. Microsoft made a big show of wanting to become "carbon neutral" (of course mostly using bogus carbon offset programs that don't actually do anything and carbon capture technologies that are net emission positive and will be for decades if not forever but still, at least they pretended) and then silently threw all of that away when it became more strategically important to pursue AI at any cost. Companies that previously desperately tried to sell messages of green washing and carbon neutrality now talk about building their own non-renewable power plants because of all the computational power they need to run their LLMs (not to mention how much more hardware needs to be produced and replaced for this - the same way the crypto bubble ate through graphics cards).

I think the pearl-clutching is justified considering that ethics and climate protection have now been folded into "woke" and there's a tidal wave in Western politics to dismantle civil rights and capture democratic systems for corporate interests that is using the "anti-woke" culture war to further its goals - the Trump government being the most obvious example. It's no longer in FAANG's financial interests to appear "green" or "privacy conscious", it's now in their interest to be "anti-woke" and that now means no longer having to care about these things and having freedom to crack down on any dissident voices within without fearing public backlash or "cancel culture".

> Tell me what is happening now is not exactly how Greece and Rome fell.

I'm not sure what you mean by that. Ancient Greece was a loose coalition of city states, not an empire. You could say they were short-sighted by being more concerned about their rivalry than external threats but the closest they came to being united was under Alexander the Great, whose death left a power vacuum.

There was no direct cause of "the fall" of Ancient Greece. The city states were suffering greatly from social inequality, which created tensions and instability. They were militarily weakened from the war with the Persians. Alexander's death left them without a unifying force. Then the Roman Empire knocked on its door and that was the end of it.

Rome likewise didn't fall in one single way. "Rome" isn't even what people think it is. Roman history spans several different entities and even if you talk about the "empire in decline" that's covering literally hundreds of years, ending with the Holy Roman Empire, which has been retroactively reimagined as a kind of proto-Germany. But even then that's only the Western Roman Empire - the Eastern Roman Empire continued to exist as the Byzantine Empire until the Ottoman Empire conquered Constaninople. And this distinction between the two empires is likewise retroactive and did not exist in the minds of Romans at the time (although they were de facto independent of each other).

If you only focus on the century or so that is generally considered to represent the fall of Western Rome, the ultimate root cause actually seems to be natural climate change. The Huns fled climate change, chasing away other groups that then fled into the Empire. Late Western Rome also again suffered from massive wealth inequality, which the ruling class attempted to maintain with increasingly cruel punishments.

So, if you want to look for a common thread, it seems to be the hubris of the financial elite, not "society" as a whole.

When tech does it, it's on record because the Internet never forgets. And it's a very, very long record, and it saddens me a lot.

I'd argue you've got things mixed up, actually.

Humans are social animals. We are individually physically weak and defenseless. Unlike other animals, we are born into this world immobile, naked, starving and helpless. It takes us literally years to mature to the point where we wouldn't simply die outright if we were abandoned by others. Newborns can literally die from touch deprivation. We develop huge brains not only to allow us to come up with clever tools but also to help us build and navigate complex social relationships. We're evolved to live in tribes, yes, but we're also evolved to interact with other tribes - we created diplomacy and trading and even currency to interact with those other tribes without having to resort to violence or avoidance.

In crises, this is the behavior we fall back to. Yes, some will self-isolate and use violence to keep others away until they feel safe again. But overwhelmingly what we see after natural disasters and spaces where the formal order of civilisation and state is disrupted and leaves a vacuum is cooperation, mutual aid and people taking risks to help others - because we intrinsically know that being alone means death and being in a group means surviving. Of course the absence of state control also often enables other existing groups to assert their power, i.e. organized crime. But it shouldn't be surprising that the fledgling and atrophied ability to self-organize might not be strong enough to withstand a fast moving power grab by an existing group - what might be more surprising is that this is rarely the case and often news stories about "looting" after a natural disaster turn out to be uncharitable descriptions of self-organized rescues and searches.

I think a better analogy for human selfishness would be the mirage of "alpha wolves". As seems to be common knowledge at this point, there is no such thing as an "alpha wolf" hierarchy in groups of wolves living in nature and the phenomenon the author who coined the term (and has since regretted doing so) was mistakenly extrapolating from observations he made of wolves in captivity. But the behavior does seem to exist in captivity. Not because it's "inherent" or their natural behavior "under pressure" but because it's a maladaptation that arises from the unnatural circumstances of captivity (e.g. different wolves with no prior bonds being forced into a confined space, naturally trying to form a group but being unable to rely on natural bonds and shared trust).

Humans do not naturally form strict social hierarchies. For the longest time, Europeans would have laughed at you if you claimed the feudal system was not in the human nature - it would have literally been heresy to challenge it. Nowadays in the West most people will say capitalism or markets are human nature. Outside the West, people will still likely at least tell you that authoritarianism is human nature - whether it's the boot of a dictatorship, the boots of oligarchs or "the people's boot" that's pushing down on the unruly (yourself included).

What we do know about more egalitarian tribal societies is that they often use delegation, especially in times of war. When quick decisions need to be made, you don't have the time for lengthy discussions and consensus seeking and it can be an advantage to have one person giving orders and coordinating an attack or defense. But these systems can still be consent-based: if the war chief is reckless or seeks to take advantage of the group for his own gain, he is easily demoted and replaced. Likewise in times of unsolvable problems like droughts, spiritual leaders might be given more power by the group. Now shift from more mobile, nomadic groups to more static, agrarian groups (though it's worth pointing out the distinction here is not agriculture but more likely granaries, crop rotation and irrigation, as some nomadic tribes still engaged in forms of agriculture) and suddenly it becomes easier for that basis of consent to be forgotten and the chosen leaders to maintain that initial state of desperation and to begin justifying their status with the divine mandate. Oops, you got a monarchy going.

Capitalism freed us from the monarchy but it did not meaningfully upset the hierarchy. Aristocrats became capitalists, the absence of birthright class assignment created some social mobility but the proportions generally remained the same. You can't have a leader without followers, you can't have a ruling class without a class of those they can rule over, you can't have an owning class without a class to rent that owned property out to and to work for that owned capital to be realized into profits.

But just like a monarch despite their divine authority was still beholden to the support of the aristocracy to exert power over others and to the laborers to till the fields, build the castle and fight off foreign claims to power, the owning class too exists in a state of perpetual desperation and distrust. The absence of divine right means a billionaire must maintain their wealth and the capitalist mantra of infinite growth means anything other than growing that wealth is insufficient to maintain it. All the while they have to compete with the other billionaires above them as well as maintain control over those beneath them and especially the workers and renters whose wealth and labor they must extract from in order to grow theirs. The perverse reality of hierarchies is that even those at the top of it are crushed underneath its weight. Nobody is allowed to be happy and at peace.

I don't necessarily disagree with you, but I think the issue is a little more nuanced.

Capitalism obviously has advantages and disadvantages. Regulation can address many disadvantages if we are willing. Unfortunately, I think a particular (mostly western) fetish for privileging individuals over communities has been wrongly extended to capital itself (e.g. corporations recognised as entities with rights similar to - and sometimes over-and-above - those of a person). We have literally created monsters. There is no reason we had to go this far. Capitalism doesn't have to mean the preeminence of capital above all else. It needs to be put back in its place and not necessarily discarded. I am certain there are better ways to practice capitalism. They probably involve balancing it out with some other 'isms.

What makes you think government sponsored entities would actually stop work on machine learning?

Even if governments overtly agree to stop or pause or otherwise limit machine learning, how credible would such a "gentlemans agreement" be?

Consider the basic operations during training and inference, like matrix multiplication, derivatives, gradient descent. Which of these would be banned? All of them? None of them? Some of them? The combination of them?

How would you inspect compliance in the context of privacy?

The analogy with drugs is rather poor: people don't have general purpose laboratories in their house. People do have general purpose computational platforms in their home. Another is that nations do not prohibit each other from producing drugs, they even permit each other to research and investigate pathogens, chemical weapons in laboratories deemed sufficiently safe.

It's not even clear what you mean with "AI" does it mean all machine learning? or LLM's? Where do you draw this boundary?

What remains is the threat of punishment in your proposal, but how credible is it, wouldn't a small collective of programmers conspiring work on machine learning, predict getting paperclipped in case of arrest?

Punish them severely when detected? Nice plan. What if they aren't in your jurisdiction? Are you going to punish them severely when they're in China? North Korea? Somalia? Good luck with that.

The problem is that the information can go anywhere that has an internet connection, and the enforcement can't.

This paper should be viewed in retrospect with the present day knowledge that Deepseek exists - regulating compute in not as easy or effective as previously thought.

As for the Chinese chip industry, I don't claim to be an expert on it, but it seems the Chinese are quickly coming up with increasingly less inferior alternatives to Western tech.

“Just because we haven't seen anyone die from nuclear terrorism”

I mean, we have…for debatable definitions of “terrorism”.

There's a difference between the trolley speeding towards someone tied to the tracks, versus someone tied to the tracks but the trolley is stationary, and to someone standing at the station looking at the bare ground and saying "if we built some tracks and put a trolley on it, and then tied someone to the tracks the trolley would kill them! We need to regulate against this dangerous trolley technology before it's too late". Then instead someone builds a freeway because it turns out the area wasn't well suited to a rail trolley.

I think it's worth noting that we can't even combat the real horrors. The fox is already in the henhouse. The quote that sticks with me is:

"We've already lost our first encounter with AI" - I think Yuval Hurari.

Algorithms heavily thumbed the scales on our social contracts. Where did all of the division come from? Why is extremism blossoming everywhere? Because it gets clicks. Maybe we're just better observing what's been going on under the hood all along, but it seems like there's about 350 million little cans of gasoline dousing American eyeballs.

Make Algorithms Govern All indeed.

International systems are more organic than designed, but the problem with cooperation is that it's not a particularly stable arrangement without enforcement - sure, everybody is better off when everybody cooperates, but you can be even better off when you don't cooperate but everybody else does.

> We need more systems based on cooperation instead of competition.

That requires dissolving the anarchy of the international system. Which requires an enforcer.

I'm not certain of the balance myself. I was thinking as a counterpoint of the band The Beatles where the two song writers (McCartney and Lennon) are seen in competition. There is a balance there between their competitiveness as song writers and their cooperation in the band.

I think it is one-sided to see any situation where we want to retain balance as being significantly affected by one of the sides exclusively. If one believes that there is a balance to be maintained between cooperation and competition, I don't immediately default to believing that any perceived imbalance is due to one and not the other.

Competition is as old as time. There are single celled organisms on your skin right now competing for resources to live. There is nothing more innate to life than this.

This is a good point, and it is the reason why communists argued that the only way communism could work is if it happened globally simultaneously. You don't want to be the only non-capitalist country in a world of capitalists. Of course, when the world-wide revolution didn't happen they were forced to change their tune and adjust.

As for nuclear weapons, I mean it does kind of suck in today's age to be a country without nuclear weapons, right? Like, certain well known countries would really like to have them so they wouldn't feel bullied by the ones that have them. So, I actually think that example works against you. And we very well may end up in a similar circumstance where a few countries get super powerful AGIs and then use their advantage to prevent any other country from getting it as well. Therefore my point stands: I don't want to be in one of the countries that doesn't get to be in that exclusive club.

I live in a ruralish area. There is a lot of forested area and due to economic depression there are a lot of people living in the woods. Most live in tents but some actually cut down the trees and turn them into make-shift shacks. Using planks and nails like you suggest. They often drag propane burners into the woods which often leads to fires. Perhaps this is what you mean?

In reality, most people will continue to live the modern life where there are doctors, accountants, veterinarians, mechanics. We'll continue to enjoy food distribution and grocery stores. We'll all hope that North America gets its act together and build high speed rail so we can travel comfortably for long distances.

There was a time Canada was a big exporter of engineering technology. From mining to agriculture, satellites, and nuclear technology. I want Canada to be competitive in these ways, not making makeshift shacks out of planks and nails for junkies that have given up on life and live in the woods.

> “Competitiveness” and “innovation” are the words the unscrupulous people at the top use to instil fear on everyone else and run rampant

If a society is okay accepting a lower standard of living and sovereign subservience, then sure, competition doesn't matter. But if America and China have AI and nukes and Europe doesn't, one side gets to call the shots and the other has to listen.

This may resonate with you:

https://crimethinc.com/2018/09/03/the-mythology-of-work-eigh...

When does that competitiveness and innovation stop though? If they stopped 100 years ago where would we be today as a species and is that better or worse than today? How about 1000 years ago?

We face issues (like we always have), but I'd argue quite strongly that the competitiveness in our history and drive to invent and innovate has led to where we are today and it's a good thing.

> Maybe, but I'm not bullish on cryptology having a solution to this problem. Every consumer device that's interesting enough to be worth hacking gets hacked within a few years. Even if nobody ever steals the key there will inevitably be side-channel attacks to feed external pictures into the camera that it thinks are coming from its own sensors.

Or the front-door analog route, point a real camera at a screen showing fake images.

That said, lots of people are incompetent at forging, about knowing what "tells" each process of fakery has and how to overcome them, so I think this will still broadly work.

> We might be teetering on the edge of a dystopian techno-feudalism where a significant portion of the population languishes in slums because industry has no use for them, but that's why I said we need to adapt.

That's underestimating the impact this can have. An AI which reaches human performance and speed on 250 watt hardware, at current global average electricity prices, costs about the same to run as a human costs just to feed.

By coincidence, the global electricity supply is currently about 250 watts/capita.

Encryption doesn't need to last forever, just long enough to be scrutinized. Once a trusted individual is convinced that a certain camera took this picture at this time and location, then that authentication is forever. Maybe that trust only includes devices built in the past 5 years, as hacks and bugs are fixed. Or corroborating evidence can be gathered; say several older, "potentially untrustworthy" devices take very similar video of an event.

As with most things, the primary issue is not really a technical one. People will believe fake photos and not believe real ones based on their own biases. So even if we had the Perfect Technology, it wouldn't necessarily matter.

And this is the reason we have fallen into a dystopian feudalistic society (we aren't teetering). The weak link is our incompetent collective human brains. And a handful of people built the tools necessary to exploit that incompetence; we aren't going back.

You handle the authentication by signatures with private keys embedded in hardware modules. An AI isn't going to be able to fake that signature. Instead, the system will fail because the keys will be extracted from the hardware modules.

You might be interested to know that the Managing Director of the Fukuoka Stock Exchange was arrested yesterday[1][2] on allegations that he took upskirt shots of schoolgirls. He was caught because his tablet's camera emitted the mandatory shutter sound.

Laws like this serve primarily to deter casual criminals and catch patently stupid criminals which are the vast majority of cases. In this case it took a presumable sexual predator off the streets, which is a great application of the law.

[1]: https://www3.nhk.or.jp/news/html/20250212/k10014719841000.ht...

[2]: https://www3-nhk-or-jp.translate.goog/news/html/20250212/k10...

I think you could make it difficult for the average user, e.g. if cameras included stereo depth estimation.

Still, I can't really see it happening.

I’m not certain that premise is valid: https://thegrayzone.com/2021/04/18/at-un-aaron-mate-debunks-...

Yes, as applied to the current generation of AIs, "safety" and "alignment" refer to things like preventing the product from making jokes about women or ethnic minorities, but that is because the current generation is not powerful enough to threaten human safety and human survival. The OP in contrast is about what will happen if the labs succeed in their stated goal of creating AIs that are much more powerful.

Current AIs are safe. Will the ones in 5 years be, 20 years?

If you make it expensive enough to extract, and tie the private key to a real identity, then you can make it hard to abuse on scale.

Here I mean that at point of sale you register yourself as owner for the camera. And you make extracting a key cost about a million. Then bulk forgeries won't happen.

Shipping secure secrets is also a somewhat solved problem: TPMs ship with EKs that, AFAIK, nobody has managed to extract (yet?): https://docs.trustauthority.intel.com/main/articles/tpm-ak-p...

Not if you are making a dirty bomb . Any radioactive material even at the levels found in power reactors can be dangerous.

The point is not the usage is harmful or not, almost any tech can be used for bad purposes if you wish to do so.

You can put controls is the point , controls here could be agent Dameons monitoring the gpus and tallying usage to heat signals, or firmware etc . The controls on what is being trained would be at a higher level than just agents process on a gpu .

That's why we should ban all large training runs without trying to distinguish good ones from bad ones. People will have to be satisfied with the foundation models we have now.

Same could be said about mining and the detection works

All countermeasures have loss

This kind of hypocrisy only exists in a made up person. Anyone who is saying that autonomous vehicles are still a ways away are not taking about the very impressive but very much semi-autonomous vehicles deployed today. But instead vehicles that have no need for a human operator ever. The kind you could buy off the shelf, switch into taxi mode, and let it do its thing.

Semi-autonomous vehicles are impressive for the fact that one driver can now scale well beyond a single vehicle. Fully-autonomous vehicles are impressive because they can scale limitlessly. The former is evolutionary, the latter is revolutionary.

What did Musk's promised driverless taxis provide that existing driverless taxis don't? The tech has arrived; it's a car that drives itself while the passenger sits in the back. Is the "gotcha" that the car isn't a Tesla?

Seems like we're splitting hairs a bit here.

Ok yeah this was a treaty that came much later - didn’t realize there were multiple agreements made over the years.

> If the rich need less and less educated, healthy and well fed workers, then more and more people will get treated like shit

Maybe. I think it's a matter of culture.

Very few people mistreat their dogs and cats in wealthy countries. Why shouldn't people in power treat regular people at least as well as regular folks treat their pets?

I'm no history buff but my hunch is that mistreatment of people largely came from a fear that if I don't engage in cruelty to maximize power, my opponents will and given that they're cruel, they'll be cruel to me when they come to take over.

So we end up with this zero sum game of squeezing people, animals, resources and the planet in an arms race because everyone's afraid to lose.

In the past - you couldn't be sure if someone else was building up an army, so you had to build up an army. But now that we have satellites and we can largely track everything - we can actually agree to not engage in this zero sum dynamic.

There will be a shift from treating people as means to an end of power accumulation and containment, to treating people as something you just inherently like and would like to see prosper.

It'll be a shift away from this deeply corrosive idea of never ending competition and growth. When people's basic needs are met and no one is grouping up to take other people's goodies - why should regular people compete with one another?

They shouldn't and they won't. People who want to do good work will do so and improving the lives of people worldwide will be its own reward. Private islands, bunkers and yachts will become incomprehensible because there'll be no serf class to service any of it. We'll go back to if you want to be well liked and respected - you have to be a good person. I look forward to it :)

Come on, let’s be real: all governments are bloated with bureaucrats and what DOGE is doing, albeit in Musk style, is to trim the fat a little bit.

You can’t seriosly claim they are upending people’s jobs when those jobs were BS in the first place.

Whoever told you this is a path forward lied to you

Focusing on the clouds is no better.

Ok but what lead you to that particular belief in the first place?

Because I can think of a large number of historical scenarios where malicious people get access to certain capabilities and it absolutely does not go well and you do have to somehow account for the fact that this is a real thing that is going to happen.

Canada had a case a couple years ago where a disabled person wanted canadian-medicare to pay for a wheelchair ramp in her house and they instead referred her to their assisted suicide program.