White House releases health report written by LLM, with hallucinated citations
56 comments
·May 30, 2025mola
TYPE_FASTER
It’s all about trust. Which sources of journalism are trustworthy? Is the government administration trustworthy?
You can’t trust an administration that would release a report that cites sources that do not exist. They are either so incompetent that they cannot perform the most basic fact checking possible, or they think we are idiots who can be easily misled.
mandmandam
> they think we are idiots who can be easily misled.
... I mean... Do you want a list of times that the American public has been easily misled?
null
Craighead
[dead]
downboots
The error was retracted. This is a clear example for an industry or branch for ensuring accuracy. We could call it the ministry of truth.
SillyUsername
It makes me remember a story I'd read, about a certain "Comrade Ogilvy", who recently died a hero whilst serving. Theres no real records of this guy, but a few lines text and a couple of faked photographs seemed easy enough to do.
kevinventullo
I don’t think eloquence or the appearance of intelligence is all that valuable for political leadership these days.
acquisitionsilk
I have entertained a similar notion when imagining the direction industries that make software might go from here. There's a possible future where some sizeable percentage of companies goes extra hard in the direction of "LLMs in, costly programmers out", and end up getting completely smashed when the LLM systems fall apart.
There might even be a couple of months of "gains" as (pointless) metrics go up, and then we might see a proper crash when stuff stops working. Especially for software which businesses rely on, surely there must be a point where they'll say enough of this crap?
Maybe not, too. Capitalism is a very surprising system, capable of absorbing shocks and morphing itself seemingly endlessly.
mrtksn
Are the countries with limited Internet better off in regards to the issues that Internet has created? Maybe, but they are also missing out on the progress of the society because despite the issues it also created a lot of opportunities and changed how we do things in general.
jimmydddd
Or maybe they will come on-line with AI after it's been sufficiently improved, and not have wasted time and resources on the early versions. Kind of like countries that never had an installed copper landline phone infrastructure, and then leap-frogged the US by going directly to a mobile infrastructure.
tokioyoyo
Tolerance for error of an average human is higher than one expects. It’s a cultural problem that’s getting more prominent in the West, because people genuinely start looking down on education. I have over-educated family members who are firm set on “my children don’t have to study hard, I’m sure they’ll smart enough to make big money at the end” idea.
conception
https://news.ycombinator.com/item?id=42707238 Nobody cares.
aorloff
I assure you Mrs. Buttle, the Ministry is very scrupulous about following up and eradicating any error.
If you have any complaints which you'd like to make, I'd be more than happy to send you the appropriate forms.
Braaaaazil......
[where my mind goes on every news from this admin]
Animats
Someone should put in an FOIA request for the prompt. Or get a congressional committee to ask for it.
chneu
Lol nobody is keeping records in this admin.
The only records we have are in Signal chats and we need people to screenshot those.
RossBencina
It would also be helpful to know which models were consulted, and whether they are certified for use in guiding national healthcare policy.
eru
In a representative democracy, elected leaders need to be able to use their judgement to decide what tools they and their lieutenants should use to come up with and implement the policies in service of what the voters' want.
Who is supposed to do this certification? How is it supposed to be binding?
MattPalmer1086
Leaders are also accountable and should be transparent about how policy is implemented, in the absence of any overriding security considerations.
llacb47
Technically, this is title editorializing, since WH denied using AI… but they are probably lying.
zombot
And when you're lying on that scale, automating it with AI is likely an actual productivity gain.
esalman
First the bogus tariff equation, now this. Certainly not the last.
Jokes aside, it's really sad to see seemingly no competent people not wanting to work with this administration.
Arainach
Your causality is wrong. This adminstration has no interest in working with competent people. They're fired huge numbers of them, gone out of their way to make life hell for those left so they quit, and are moving faster than ever imagined possible to drive competent people in non-government jobs away from this country - by deporting them, taking away their visas, attacking them, and more.
dralley
The causality is bidirectional.
Arainach
I disagree. The first Trump administration showed that not only were some competent people interested in working in the Trump cabinet, plenty of other competent people were content to keep doing their non-political jobs well for the government, running things, fighting for normal people and what they right, pushing back against overreach.
The second Trump administration took aggressive measures against all competent people - selecting a horrifying cabinet with literally no one qualified for the role they fill and working as fast as possible to fire as many as possible, even through illegal means later rejected by the courts, and punishing any who remain. It's a one-direction causation.
ksynwa
This administration is a haven for charalatans and grifters. It's no surprise that they don't have competent people. The goals of competent people are diametrically opposed to what this government wants to do.
eru
No. There's still lots of overlap in goals.
For example, most competent people I know didn't want a nuclear war to start yesterday. Lo and behold, the administration also did not start nuclear war yesterday.
chneu
While there are a lot of incompetent idiots in this admin, don't forget that a lot of them are grifters who are only trying to get rich/powerful while they can.
I don't like doing the Hitler comparison, but the similarities are definitely there. A lot of the Nazis thought Hitler was a "useful idiot" that they could use and then get rid of. Trump is very similar.
ytpete
I would imagine this is true of many other dictators and authoritarians over time too: Putin, Kim Jong Un, etc. If you are looking for non-Hitler comparisons.
duxup
These people even struggle to do a bad job well enough.
platevoltage
Next time, they'll just add "don't include citations" in the prompt. Problem solved.
rasz
Its all a big joke to these people, like asking about medical advice from Groks "not a doctor".
gloosx
At this point we can fully replace talking heads with AI. Let them generate reports, write fake citations, review each other’s work, sign it, worship it – all in imitation of what we once called a state. A huge win for humanity.
null
kcaseg
Reminds me of this “AI study” on climate change
https://factcheck.afp.com/doc.afp.com.39798G2
It is not the “minor citation errors” what are the most worrying, but rather the fact that LLMs aim to please you, even if you really believe your prompt was neutral (which I doubt when it comes to Kennedy and vaccines for example).
caseysoftware
Can we all agree to not cite any study that a) doesn't exist, b) cannot be reproduced, or c) includes fake data?
For bonus points, the participants of the b) and c) options should be forced to pay back whoever funded their research.
eru
What do you mean by 'cannot be reproduced'?
If I do a simple study where I flip a coin a million times and write down the result, you are unlikely to be able to reproduce the same result.
xboxnolifes
I'm pretty sure if we both flipped a million coins, our results would be very similar.
Y_Y
I think you and GP are using different metrics or definitions of "result".
tgv
In case you're not trolling: to reproduce a paper means to achieve the reported result within an acceptable margin through independent replication of the experiment. Since nobody is referring to a paper which reports one million coin flips, reproduction is not in question.
harvey9
In this context we would be talking about reproducing the experimental conditions and method. Obviously it is unlikely that someone else would get the same sequence you did.
eru
Basically, the whole thing is a bit more complicated.
However, it's useful to require pre-registration and sharing of data etc for studies you plan to fund.
Ideally, you give researchers an in-principle approval, but then hold the funds in escrow and only disburse them after they published their data etc.
(Financial markets can provide the bridge financing between in-principle approval and actual disbursal.)
There's many journals that have open data requirements, but more often than not they are flouted. The above suggestion would give them real teeth.
I have this pet theory that this wave of AI won't end with amazing productivity gains.
I think it'll end in mass confusion. Dumbing down of the leadership and elites that now can cosplay as smart and eloquent.
countries that don't embrace AI will have a massive edge over other countries because their population will be smarter and more capable.