Underwriting Superintelligence

> As insurers accurately assess risk through technical testing

If that’s not “the rest of the owl” I don’t know what is.

Let’s swap out superintelligence for something more tangible. Say, a financial crash due to systemic instability. How would you insure against such a thing? I see a few problems, which are even more of an issue for AI.

1. The premium one should pay depends on the expected risk, which is damage from the event divided by the chance of event occurring. However, quantifying the numerator is basically impossible. If you bring down the US financial system, no insurance company can cover that risk. With AI, damage might be destruction of all of humanity, if we believe the doomers.

2. Similarly, the denominator is basically impossible to quantify. What is the chance of an event which has never happened before? In fact, having “insurance” against such a thing will likely create a moral hazard, causing companies to take even bigger risks.

3. On a related point, trying to frame existential losses in financial terms doesn’t make sense. This is like trying to take out an insurance policy that will protect you from Russian roulette. No sum of cash can correct that kind of damage.

This only works if there are negative consequences faced by the insured parties when things go wrong. If all the negative consequences are faced by society and there are no regulations that incur that burden on the companies building AI, then we'll have unchecked development.

> But we don’t want medical device manufacturers or nuclear power plant operators to move fast and break things. AI will quickly get baked into critical infrastructure and could enable dangerous misuse.

nobody will put a language model in a pacemaker or a nuclear reactor, because the people who would be in a position to do such things are actual doctors or engineers aware both of their responsibilities and of the long jail term that awaits them if they neglect them

this inevitabilism, to borrow a word from another submission earlier today, about "AI" ending up in critical infrastructure and the important thing being to figure out how to do it right is really quite repugnant

sure, yes, i know about the shitty kinda-explainable statistical models that already control my insurance premiums or likelihood of getting policed or whatever

but why is it a foregone conclusion that people are going to (implicitly rightly so given the framing lets it pass unquestioned!) put llms into things that materially affect my life on the level of it ending due to a stopped heart or a lethal dose of radiation

> We’re navigating a tightrope as Superintelligence nears. If the West slows down unilaterally, China could dominate the 21st century.

I never understood this argument

as a non-USian: I'd prefer to be under the Chinese boot rather than having all of humanity under the boot of an AI

and it is certainly no reason to try to do everything we possibly can to try and summon a machine god

I'm biased because my company (Newfront) is in insurance but there are a lot of great points here. This jumped out: "By 2030, global AI data centers alone are projected to require $5 trillion of investment, while enterprise AI spend is forecast to reach $500 billion."

There's a mega trend of value concentrating in AI (and all the companies that touch/integrate it). Makes a ton of sense that insurance premiums will flow that direction as well.

This article is a bizarre mix of center-right economic ideas and completely unfounded assumptions about the nature of AI technology, to the point where I'm genuinely not sure if this is intended as parody or not.

> We’re navigating a tightrope as Superintelligence nears.

There is no evidence we're anywhere near "superintelligence" or AGI. There is no evidence any AI tools are intelligent in any sense, yet alone "superintelligence". The only reference for this, given much later, is to https://ai-2027.com/ which is no more than fan fiction. You might as well have cited Terminator or The Matrix as evidence.

The only people actually claiming any advancement towards "superintelligence" or "AGI" directly financially gain from people thinking that it's right around the corner.

> If the West slows down unilaterally, China could dominate the 21st century.

Is this casual sinophobia intended to appeal to a particular audience? I can't see what purpose this statement, and others like it, serves other than to try to frame this as "it's us or them".

> Faster than regulation: major pieces of regulation, created by bureaucrats without technical expertise, move at glacial pace.

This is a very common right-wing viewpoint. That regulation, government oversight, and "red tape" is unacceptable to business. Forgetting that building codes, public safety regulations, and workers rights all stem directly from government regulation. This article goes out of it's way to frame it as obvious, like a simple fact unworthy of introspection.

> Enterprises must adopt AI agents to maintain competitiveness domestically and internationally.

There is no evidence this is the case, and no citation is even attempted.

For this to work, large class actions are needed. If companies are liable for large judgements, companies will insure against them. If not, companies will not try to avoid harms for which they need not pay.

Is there any indication whatsoever that there's even a glimpse of artificial intelligence out there?

So far, I have seen language models that, quite impressively, translate between different languages, including programming languages and natural language specs. Yes, these models use a wast (compressed) knowledge from pretty much all of the internet.

There are also chain of thought models, yes, but what kind of actual intelligence can they achieve? Can they formulate novel algorithms? Can they formulate new physics hypotheses? Can they write a novel work of fiction?

Or aren't they actually limited by the confines of what we as a species already know?

> We’re navigating a tightrope as Superintelligence nears. If the West slows down unilaterally, China could dominate the 21st century

I stopped reading after this. First, there is no evidence of Superintelligence nearing or even any clear definition of what "Superintelligence nearing" means. This is classic "assuming the sale" gambit with fear-mongering in its appeal.

The "Incentive Flywheel" of AI: how insurance unlocks secure Al progress and enables faster AI adoption.

With no skin in the game, it will be either cool if super intelligence happens or it doesn’t and I just get to enjoy some schadenfreude. Either all of these people are geniuses or they’re Jonestown members.

I love it!

Finally some clear thinking on a very important topic.