People are losing loved ones to AI-fueled spiritual fantasies

I'm worried on a personal level that it's too easy to begin to rely on chatgpt (specifically) for questions and such that I can figure out for myself. As a time-saver when I'm doing something else.

The problem for me is -it sucks. It falls over in the most obvious ways requiring me to do a lot of tweaking to make it fit whatever task I'm doing. I don't mind (esp for free) but in my experience we're NOT in the "all the right answers all of the time" stage yet.

I can see it coming, and for good or ill the thing that will mitigate addiction is enshittification. Want the rest of the answer? Get a subscription. Hot and heavy in an intimate conversation with your dead granma wait why is she suddenly singing the praises of Turbotax (or whatever paid advert).

What I'm trying to say is that by the time it is able to be the perfect answer and companion and entertainment machine -other factors (annoyances, expense) will keep it from becoming terribly addictive.

[flagged]

The default setting on ChatGPT is to now include previous conversations as context. I disabled memories, but this new feature was enabled when I checked the settings.

An LLM trained on all other science before Copernicus or Galileo would be expected to explain as true that the world is the flat center of the universe.

[delayed]

It'd be great if it were trained on therapeutic resources, but otherwise just ends up enabling and amplifying the problem.

I knew of someone who had paranoid delusions and schizophrenia. He didn't like taking his medicine due to the side effects, but became increasingly convinced that vampires were out to kill him. Friends, family and social workers could help him get through episodes and back on the medicine before he became a danger to himself.

I'm terrified that people like him will push away friends and family because the LLM engages with their delusions.

Even when sycophantic patterns emerge?

I think the last think a delusional person needs is external confirmation, be it human or a sycophantic machine.

An LLM tuned for charisma and trained on what the power players are saying could play politics by driving a compliant actor like a bot with whispered instructions. AI politicians (etc.) may be hard to spot and impractical to prove.

You could iterate on the best prompts for cult generation as measured by social media feedback. There must be experiments like that going on.

When AI becomes better at politics than people then whatever agents control them control us. When they can make better memes, we've lost.

On what basis do you assume that that isn't exactly what "safety alignment" means, among other things?

Would you still call it a "cult" if each recruit winds up inside their own separate, personalized, ever-changing rabbit hole? Because if LLM, Inc. is trying to maximize engagement and profit, then that sounds like the way to go.

You are a conspiracy theorist and a liar! /s

The problem is inside people. I met lots of people who contributed to psychotic inducing behavior. Most of them were not in a cult. They were regular folk, who enjoy a beer, movies, music, and occasionally triggering others with mental tickles.

Very simple answer.

Is OpenAI also doing it? Well, it was trained on people.

People need to get better. Kinder. Less combative, less jokey, less provocative.

We're not gonna get there. Ever. This problem precedes AI by decades.

The article is an old recipe for dealing with this kind of realization.

Probably cheaper

To quote my favorite Smash Mouth song,

"Sister, why would I tell you my deepest, dark secrets? So you can take my diary and rip it all to pieces.

Just $6.95 for the very first minute I think you won the lottery, that's my prediction."