Mike Lindell's lawyers used AI to write brief–judge finds nearly 30 mistakes
123 comments
·April 26, 2025seanhunter
mmcwilliams
I think it's easy to understand why people are overestimating the accuracy and performance of LLM-based output: it's currently being touted as the replacement for human labor in a large number of fields. Outside of software development there are fewer optimistic skeptics and much less nuanced takes on the tech.
Casually scrolling through TechCrunch I see over $1B in very recent investments into legal-focused startups alone. You can't push the messaging that the technology to replace humans is here and expect people will also know intrinsically that they need to do the work of checking the output. It runs counter to the massive public rollout of these products which have a simple pitch: we are going to replace the work of human employees.
gyomu
People are lazy. I’m enrolled in a language class in a foreign country right now - so presumably people taking that class want to actually get good at the language so they can actually live their life here - yet a significant portion of students just turn in ChatGPT essays.
And I don’t mean essays edited with chatGPT, but essays that are clearly verbatim output. When the teacher asks the students to read them out loud to the class, they will stumble upon words and grammar that are way obviously way beyond anything we’ve studied. The utter lack of self awareness is both funny but also really sad.
DiscourseFan
LLMs were originally designed for translation, so it makes sense. We have basically elimated the need to learn foreign languages for day to day use anyway, its only helpful for high professional tasks or close literary study or prestige.
mediumsmart
could it be that they just have to attend the class for technical reasons? Also - once the gadgets can translate for free in real time ... you can live in places you don't speak the language of, so maybe they are just prepping for that.
kevin_thibedeau
There are a lot of shit tier lawyers who are just in it for the money and just barely passed their exams. Given his notoriety, Lindell is scraping the bottom of the barrel with people willing to provide legal services.
sevensor
What this story tells us more than anything is that Lindell cannot convince a competent lawyer to defend him, so what he gets instead are clownshod phonies. Either he’s out of cash, or he’s such a terrible client that nobody with a shred of professional responsibility will take him.
kayodelycaon
I asked ChatGPT to give Wikipedia links in a table. Not one of the 50+ links was valid.
swores
Which version of GPT? I've found that 4o has actually been quite good at this lately, rarely hallucinating links any more.
Just two days ago, I gave it a list of a dozen article titles from a newspaper website (The Guardian), asked it to look up their URLs and give me a list, and to summarise each article for me, and it made no mistakes at all.
Maybe your task was more complicated to do in some way, maybe you're not paying for ChatGPT and are on a less able model, or maybe it's a question of learning how to prompt, I don't know, I just know that for me it's gone from "assume sources cited are bullshit" to "verify each one still, but they're usually correct".
kayodelycaon
Definitely more complicated. I've been playing around with using it to analyze historical data and using it to generate charts. And yes I've tried many different kinds of phrasing. I have experience working with and writing rules based "expert systems" and have a vague idea of how neural networks are used for image recognition. It's a pretty fun game to get useful information out of ChatGPT.
You cannot ask it to have crop yield as a column in a chart and get accurate information.
It only seems reasonable when doing a single list of items. Asking it for two columns of data and it starts making things up. Like bogus wikipedia links.
You could definitely make the argument I'm using it wrong but this is how people try to use it. I still find this useful because it gives me a start on where to point my research or ask clarifying questions.
It's much better at giving you a list of types of beer and wine that's been produced in history. Just don't trust any of the dates.
lolinder
> asked it to look up their URLs and give me a list
Something missing from this conversation is whether we're talking about the raw model or model+tool calls (search). This sounds like tool calls were enabled.
And I do think this is a sign that the current UX of the chatbots is deeply flawed: even on HN we don't seem to interact with the UI components to toggle these features frequently enough that they're the intuitive answer, instead we still talk about model classes as though that makes the biggest difference in accuracy.
0xFEE1DEAD
Sorry for going off topic here but I've had the same experience.
I'm not sure which update improved 4o so greatly but I get better responses from 4o than from o4-mini, o4-mini-high, and even o3. o4 and o3 have been disappointing lately - they have issues understanding intent, they have issues obeying requests, and it happened multiple times that they forgot the context even though the conversation consisted of only 4 messages without a huge number of tokens. In terms of chain-of-thought models I prefer DeepSeek over any OpenAI model (4.5 research seems great, but it’s just way too expensive).
It's rather disappointing how OpenAI releases new models that seem incredible, and then, to reduce the cost of running them, they slowly slim these models down until they're just not that good anymore.
alphan0n
Share the link to the conversation.
jedimastert
> How does anyone have the confidence to ship a legal brief that an AI produced without checking it thoroughly?
They're treating it like they would a paralegal. Typically this means giving a research task and then using their results, but sometimes lawyers will just have them write documents and ship it, so to speak.
This is making me realize that Tech Bros treat chat GPT like the 1930s secretary they never got to have
belter
Everything you’ve said is correct. Now picture a quiet spread of subtle defects seeping through countless codebases, borne on the euphoria of GenAI driven “productivity”. When those flaws surface, the coming AI winter will be long and bitter.
halgir
I use it in much the same way as you, and it's been extremely beneficial. But I also would not dream of signing my name on something that has been independently produced by AI, it's just too often blatantly wrong on specifics.
I think people who do are simply not aware that AI is not deterministic the same way a calculator is. I would feel entirely safe signing my name on a mathematical result produced by a calculator (assuming I trusted my own input).
mrob
LLMs are deterministic [0]. An LLM is a pure function that takes a list of tokens and returns a set of token probabilities. To make it "chat" you use the generated probabilities to pick a token, append that token to the list, and run the LLM again. Any randomness is introduced by the external component that picks a token using the probabilities: the sampler. Always picking the most likely token is a valid strategy.
The problem is that all output is a "hallucination", and only some of it coincidentally matches the truth. There's no internal distinction between hallucination and truth.
[0] Theoretically; race conditions in a parallel implementation could add non-determinism.
ijk
True, though in practice speed optimizations and instabilities on the GPU often lead to LLMs being very non-determanistic in practice.
Which doesn't detract from your main point: there's not a lot of distinction between hallucinations and what we'd consider to be the "real thing." There have been various attempts to measure hallucinations, and we can figure out things like how confident the model is in a particular answer...but there's nothing grounding that answer. Saturate the dataset with the wrong answer and you'll get an overconfident wrong result.
jdlshore
While this is technically correct, everyday use of LLMs involves a non-zero temperature, so they (the whole package that people think of as “AI”) are non-deterministic in practice.
koakuma-chan
No, hallucinations occur when LLM is missing information.
BlueTemplar
But then isn't this also technically true that any software including a pseudo-random number generator is deterministic ? (Starting with itself, like that sampler you mention ?)
And while it might be important in some contexts, like debugging using either the exact same or different seeds, isn't this one of them where it rather confuses the issue ?
AnimalMuppet
Lindell's lawyer claimed that somehow the preliminary copy (before human editing) got submitted to the court - that they actually did the work to fix it, but then slipped up in submitting it.
I could see that, especially with sloppy lawyers in the first place. Or, I could see it being a convenient "the dog ate my homework" excuse.
gazook89
Having not looked into it, I would guess that his lawyers know they aren’t going to get paid any time soon.
cosmicgadget
Seems like it's a fast track to not getting paid ever (disbarrment).
null
Etheryte
> Wang ordered attorneys Christopher Kachouroff and Jennifer DeMaster to show cause as to why the court should not sanction the defendants, law firm, and individual attorneys. Kachouroff and DeMaster also have to explain why they should not be referred to disciplinary proceedings for violations of the rules of professional conduct.
Glad to see that this is the outcome. Similar to bribes and other similar issues, the hammer has to be big and heavy so that people stop considering this as an option.
tzs
> "[T]he Court identified nearly thirty defective citations in the Opposition. These defects include but are not limited to misquotes of cited cases; misrepresentations of principles of law associated with cited cases, including discussions of legal principles that simply do not appear within such decisions; misstatements regarding whether case law originated from a binding authority such as the United States Court of Appeals for the Tenth Circuit; misattributions of case law to this District; and most egregiously, citation of cases that do not exist," US District Judge Nina Wang wrote in an order to show cause Wednesday
30+ years ago when I was in law school [1] I would practice legal research by debunking sovereign citizen and related claims on Usenet. The errors listed above are pretty much a catalog of common sovereign citizen legal research errors.
Just add something about gold fringed flags and Admiralty jurisdiction and it would be nearly complete.
The sovereign citizen documents I debunked were usually not written by lawyers. At best the only legal experience the authors usually had was as defendants who had represented themselves and lost.
Even they usually managed to only get a couple major errors per document. That these lawyers managed to get such a range of errors in one filing is impressive.
[1] I am not a lawyer. Do not take anything I write as legal advice. Near the end of law school I decided I'd rather be a programmer with a good knowledge of law than a lawyer with a good knowledge of programming and went back to software.
rsynnott
What is it with the American far-right and hiring the most _incompetent possible lawyers_? Like, between this and Giuliani...
Spooky23
Think about the quality of lawyer who would take Lindell as a client.
He’s a bankrupt, likely mentally ill acolyte of a dude who is infamous for stiffing his lawyers. His connection with reality is tenuous at best.
dionian
Our justice system prides itself on giving everyone due process and a fair trial, even people you hate
dymk
This guy is getting exactly the kind of lawyers he deserves, and it’s nobody’s fault but his own
foobarchu
I don't think anyone claimed he doesn't deserve due process. The only people I know of arguing against due process lately are in fact those in the Whitehouse.
cosmicgadget
This is an appeal for a civil suit he lost.
jimmydddd
This. As a junior lawyer at a large law firm, one of my jobs was checking every cited case to confirm that it was cited correctly, that it actually supported the theory that it was being used to support, and that it hadn't been overturned or qualified by subsequent case law. It's a process called Shepardizing that every law student does. So I can't fathom how fictitious cases could possibly be included in a brief. Also, just slightly mischaracterizing a case in a brief could be cause for being sanctioned. So I don't see how this type of issue would not undoubtedly result in sanctions.
zero_iq
Because competent lawyers tend to adhere to professional standards and codes of ethics, which makes them more selective in the work and clients they take on.
chneu
They think everyone is doing it and not getting caught.
Everything the right accuses anyone of, they're doing it too. That's why they don't really care about criminals and pedophiles and racists in their ranks. They think everyone is a child diddling criminal racist.
AIPedant
The problem is that Trump, Musk, Lindell, etc are all extremely arrogant and constantly disregard sound legal advice. Their lawyers aren't merely associated with a controversial client; their professional reputation is put at risk because they might lose easily winnable cases due to a client's dumb tweet. You have to be a crappy lawyer (or an unethical enforcer like Alex Spiro and Roy Cohn) to even want to work with them.
ramesh31
>The problem is that Trump, Musk, Lindell, etc are all extremely arrogant and constantly disregard sound legal advice. Their lawyers aren't merely associated with a controversial client; their professional reputation is put at risk because they might lose easily winnable cases due to a client's dumb tweet.
Bingo. This has nothing to do with ideology. Good lawyers like to win. And when a client is demonstrably too stupid to let them do that, why bother.
elsonrodriguez
Being a stupid and controversial is now a popular ideological option.
add-sub-mul-div
There's a quote I can't find right now about how fascism is associated with lower competence because it not only prioritizes but demands loyalty over all else and you get a bench made up of just the best asskissers, ideologues, extremists.
myko
If their goal is to hire people who believe in their cause, their hands are tied
AnimalMuppet
Some of the prominent people on the right have tried to ignore the law, to not let the law modify their behavior, fighting off lawsuit after lawsuit, and adverse ruling after adverse ruling. If you're going to do that, you have to file a lot of motions. That seems to drive an emphasis on volume rather than quality of motions in reply. At least, that's my perspective as an outside observer.
victorbjorklund
I dont understand how a lawyer can use AI like this and not just spend the little time required to check that the citations actually exist.
grues-dinner
I constantly see people reply to question with "I asked ChatGPT for you and this is what it says" without a hint of the shame they should feel. The willingness to just accept plausible-sounding AI spew uncritically and without further investigation seems to be baked into some people.
cogman10
I've seen this as well and I've seen pushback when pointing out it's a hallucination machine that sometimes gets good results, but not always.
Way too many people think that LLMs understand the content in their dataset.
Ruphin
That sort of response seems not too different from the classic "let me google that for you". It seems to me that it is a way to express that the answer to the question can be "trivially" obtained yourself by doing research on your own. Alternatively it can be interpreted as "I don't know anything more than Google/ChatGPT does".
What annoys me more about this type of response is that I feel there's a less rude way to express the same.
dghlsakjg
Let me google that for you is typically a sarcastic response pointing out someone’s laziness to verify something exceptionally easy to answer.
The ChatGPT responses seem to generally be in the tone of someone who has a harder question that requires a human (not googleable), and the laziness is the answer, not the question.
In my view the role of who is wasting others time with laziness is reversed.
rsynnott
It's worse, because the magic robot's output is often _wrong_.
technothrasher
At least those folks are acknowledging the source. It's the ones who ask ChatGPT and then give the answer as if it were their own that are likely to cause more of a problem.
AnimalMuppet
Go look at "The Credit Card Song" from 1974. It's intended to be humorous, but the idea of uncritically accepting anything a computer said was prevalent enough then to give the song an underlying basis.
LocalH
I downvote comments like that, regardless of platform, in almost all situations. They don't really contribute much to the majority of discussions.
sameasiteverwas
I think shame is disappearing from American culture. And that's a shame.
rokkamokka
Shame? It's often constructive! Just treat it for what it is, imperfect information.
Macha
If I wanted ChatGPT's opinion, I'd have asked ChatGPT. If I'm asking others, it's because it's too important to be left to ChatGPT's inaccuracies and I'm hoping someone has specific knowledge. If they don't, then they don't have to contribute.
distances
It's not constructive to copy-paste LLM slop to discussions. I've yet to see a context where that is welcome, and people should feel shame for doing that.
BlueTemplar
'member the moral panic when students started (often uncritically) using Wikipedia ?
Ah, we didn't knew just how good we had it...
(At least it is (was ?) real humans doing the writing, you can look at modification history, well made articles have sources, and you can debate issues with the article in the Talk page and even maybe contribute directly to it...)
eviks
It's not "a little time"
blululu
The Judge spent the time to do exactly this. Judges are busy. Their time is valuable. The lawyer used AI to make the judge do work. The lawyer was too lazy to do the verification work that they expected the judge to perform. This speaks to a profound level of disrespect.
cbfrench
I highly doubt the judge was tracking down citations or reading those cited cases herself to verify what was in them. They have law clerks for that. It doesn’t make it any less an egregious waste of the court’s time and resources, but I would be surprised if a district court judge is personally doing much, if any, of that sort of spadework.
victorbjorklund
Checking if a case exists or not is little time in the context of legal research.
dwattttt
Perhaps not, but it is the time required to discharge their obligation under Rule 11 of the Federal Rules of Civil Procedure (IANAL).
bombcar
It’s “paralegal time” which is nearly free …
dghlsakjg
Courts are not allocated an unlimited budget for clerks.
Outside of the literal dollar cost, the opportunity cost here is further delays on the docket because the clerk was unable to do something else, and the court time that must now be spent dealing with the issue.
eviks
First, you're confusing time with money
Second, the mistakes weren't just incorrect citations any paralegal could check
daymanstep
You could probably use AI to check that the citations exist
Balgair
Wait until you guys hear about how they used AI in the California bar exam.
https://www.sfgate.com/bayarea/article/controversy-californi...
The lawyer jokes aren't funny anymore...
everybodyknows
> ... response to the widespread disruptions, the Committee of Bar Examiners, or CBE, voted on April 18 to lower the raw passing score for the February exam from 560 to 534, “two standard errors of measurement below ...
Although different states are involved, perhaps this goes some way toward explaining how Lindell's lawyers could have passed their bar exams.
Balgair
I'm not too familiar with his lawyers, but I suspect they passed their exams a long time ago in another state.
Bar exams are funny things. Most states have a reciprocity with the NY bar, so when you think lawyer, think the NY bar.
But California is considered a harder bar to pass and has little reciprocity.
Somewhat surprisingly the hardest bar is Louisiana's. This is because their legal system is a crawdad fucking mess. They inherited their code based system from the French for a lot of local matters, but then also have to deal with the precedent based system the rest of the US uses. So you have to memorize two completely different types of law at a very high level. So, if you ever meet a Louisiana lawyer, you know you've met a very intelligent and dedicated person.
Obligatory IANAL here.
BlueTemplar
How ? The issue seems to have been that they had not revealed that LLMs were involved in the creation of the multiple choice questions. The questions/answers themselves seem to have passed the bar ? (no pun intended)
A much worse failure seems to have been the incompetent software to run the tests. And that for something as high level they would have decided to do it through the mediation of a computer as well as used multiple choice questions in the first place.
financetechbro
This is a really crazy story
philipwhiuk
This is just Mata v. Avianca again
ForOldHack
"You IDIOT!!! And you have IDIOT lawyers too." There. I said it. It needed to be said and I feel so much better.
LadyCailin
Everything about this entire situation is comically dumb, but shows how far the US has degraded, that this is meaningful news. If this were a fiction book, people would dismiss it as being lazy writing - an ultra conservative CEO of a pillow company spreads voting conspiracies leading to a lawsuit in which they hire lawyers that risk losing the case because they relied on AI.
cosmicgadget
Let's not forget the majestic event that was Cyber Symposium.
But here we have an example of someone not escaping justice due to his now-evaporated wealth. I'd call it a positive.
TheRealQueequeg
Quite dumb. If it were a book it would be "Infinite Jest", and the receipts of everyone who bought the pillows could be used to enter into some inane raffle.
michaelcampbell
Because this sort of thing is totally geographically bound.
cratermoon
dupe https://news.ycombinator.com/item?id=43799823
You missed this one, gnabgib
yapyap
That’s so stupid, he almost deserves to lose the case just for that
michaelcampbell
He needs punishment for himself, not for the people or entity he's representing.
emorning3
Is it possible that these AI models will tell someone what they want to hear rather than the truth?
I mean, that's always been tech's modus operandi....
What I find really strange about this is I use AI a lot as a “smart friend” to work through explanations of things I find difficult etc and I am currently preparing for some exams so I will often give the AI a document and ask for some supporting resources to take the subject further and it almost always produces something that is plausibly close to a real thing but wrong in specifics. As in when you ask for a reference it is almost invariably a hallucination. So it just amazes me that anyone would just stick that in a brief and ship it without checking it even more than they would check the work of a human underling (which they should obviously also check for something this important).
For example, yesterday I got a list of some study resources for abstract algebra. Claude referred me to a series by Benedict Gross (Which is excellent btw). It gave me a line to harvard’s website but it was a 404 and it was only with further searching that I found the real thing. It also suggested a youtube playlist by Socratica (again this exists but the url was wrong) and one by Michael Penn (same deal).
Literally every reference was almost right but actually wrong. How does anyone have the confidence to ship a legal brief that an AI produced without checking it thoroughly?