Skip to content(if available)orjump to list(if available)

A few words about FiveThirtyEight

A few words about FiveThirtyEight

92 comments

·March 6, 2025

qqtt

As someone who was super interested in the 538-style of election coverage in 2008, I've kind of fallen "out of love" so to speak with election models and forecasting in general. I'm not really convinced about what it adds to the conversation around elections. We can all look at various polls and get an assessment of who is generally ahead. Weighted polling aggregators and forecasting models just collect all these polls and spit out some data. It's easy to hand wave and think some new information is being revealed, but ultimately it is just a "garbage in garbage out" situation - you are entering polls as input, some hand waving is going on, and you get some forecast as a result.

I think part of my cynicism comes in the wake of the 2016 election, in which the forecast rightfully counted some scenarios in which either candidate could win, upon which conclusion of the model was basically "the result fits in with the forecast, because either candidate could have won according to the model" - in which case I personally concluded, if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here? We can simply look at polls and understand who is generally ahead, and not be any better or worse off.

alisonatwork

538 in its final form was not about predicting election outcomes, it was about the business and science of polling, and contrasting factual data with the way people perceived related issues. It was an interesting outlet that helped illuminate the data side of political science, and at its best also provided some insight into the disconnect between how the general public thinks about a topic versus what is actually happening.

Disney killing 538 is broadly a loss for political journalism in the US, imo, because most other American media is more interested in sensationalism and hyping imaginary culture war issues, i.e. exacerbating exactly the disconnect with reality that 538 was trying to combat with its more evidence-based reporting. From my perspective the only place still doing this kind of work outside of niche, single-topic outlets like SCOTUSblog is ProPublica, and even they don't tend to be as politics-centric as 538 was. So I definitely will miss the site, and the pod. I don't have the stomach for most other American media.

JumpCrisscross

Would note that Silver owned the election model 538 ran on. When he left, he took it with him. The recent election forecasts 538 put out were not Nate Silver’s. (In my opinion, his were more accurate. More importantly, his commentary was more informative on the model’s shortcomings and insights.)

wasabi991011

Respectfully, I disagree on both points.

> We can all look at various polls and get an assessment of who is generally ahead.

I probably could, but there's a lot of polls to look through and I don't really want to spend the time. Much rather have someone else do it for me.

> if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here?

Isn't this hypothesis testing? If you have a weighted coin and a guess as to which side is heavier and by how much, you're going to need multiple flips to see if you are right. And it doesn't even really make sense to talk about how right/wrong you are about a single flip, only on the aggregate.

It's possible someone has already compiled FiveThirtyEight's results to get some aggregate accuracy, I haven't checked. If they have and he's wrong on average and that's what you are referencing, my apologies.

tqi

I know the folks at 538 meant well, but I think the ultimate impact of their work was to accelerate the politics as entertainment, team sports-ification of elections, to our nations detriment.

sdwr

I kinda get your point - statistics suck the air out of the room. If regular people are talking about swing state poll margins of error instead of the actual issues, something's gone wrong.

538 democrasized the numbers that were the domain of political whizzes. I don't know if that's a good thing.

wslack

I think aggregators are useful to tell us what actually moved voters in a campaign - the swings are visible even if baseline error is not.

bee_rider

Eh, this seems to just promote armchair quarterbacking. What moved voters is an issue for the campaigns to track. We should listen to what the campaigns say about what we care about.

nozzlegear

I largely agree with your points. Election modelers and forecasters really don't add much to the conversation after 2016, despite their attempts and even purported success at correcting their models and mistakes. The only election forecasting model that I take seriously these days is my own vibes based forecasting.

I have enjoyed the meta-drama around forecasting and modeling that pops up every election season though. It's hard to beat "[Nate Silver] doesn't have the faintest idea how to turn the keys," or "I ran 80,000 simulations."

¹ https://knowyourmeme.com/memes/he-doesnt-have-the-faintest-i...

² https://knowyourmeme.com/memes/i-ran-80000-simulations

Gimpei

I disagree. Before 538 people were still offering lots of election predictions and it was much much worse, because it was based entirely on hunches and vibes. Silvers rates the pollsters and provides confidence intervals far better than a simple average of polls does. I’d much rather read his forecasts than any number of bloviating opeds.

mgfist

But that's because elections these days are incredibly close. It's like being upset that the best statistical answer to "who will win a coin toss" is "well it's 50/50".

genewitch

Are you sure about that?

89% of counties turned red. Looking at 1.5% total difference is missing the forest for the trees. Trump got all swing states.

To quote: "this is absolutely a mandate."

https://youtu.be/zG3n2IeaTPA

edit: facts cannot be insolent. The youtube link is a guest on the view named Stephen something, from this week, saying the words that i typed into the comment box. Not clicking it is doing yourself a disservice; as it is "source cited."

Don't get mad at me for relaying this information.

roenxi

Sure. And what was the probability of something like that happening? About a coin flip.

It is an election using First-Past-the-Post counting. A 50:50 probability doesn't mean the final count will be close, it (usually, anyway) means there is evens odds which side is about to win decisively. The county results are expected to correlate, as are the swing states in all likelihood.

aaronbrethorst

Counties don't vote and Trump's popular vote "mandate" is smaller than Clinton's popular vote win in 2016.

Here's a fun stat: literally 40 states have a population that is less than the population of Los Angeles county alone. Why doesn't Los Angeles itself have 80 senators?

https://en.wikipedia.org/wiki/2016_United_States_presidentia...

https://en.wikipedia.org/wiki/2024_United_States_presidentia...

Setting aside 2000 and 2016 (EC winners lost the popular vote), you'd have to go back to 1968 and Nixon's squeaker of an election over Humphrey to find a closer popular vote: https://en.wikipedia.org/wiki/List_of_United_States_presiden...

kemayo

I don't think someone can claim a "mandate" based on not even having won 50% of the vote. You need to at least be able to say that over half of the voters wanted you to win in order to use language like that.

That 1.5% is the only thing that mattered for "mandate" purposes. Trump won, but on a razor's edge, and that should give him some pause. (It won't.)

nobody9999

>https://youtu.be/zG3n2IeaTPA

>note: not clicking that because you disagree with me is really doing yourself a disservice.

I won't click the link because I'm here for discussion with the users of this site, not some random video you thought was interesting/useful.

Why do you find it interesting/useful? What arguments are made? How do those arguments comport with your beliefs?

Let's have a discussion. So tell me what you think, don't link to some rando on youtube.

BrenBarn

FiveThirtyEight was interesting in its time, but in the past few years I felt it ironically became exactly what it was initially trying to oppose: a site full of opinion-based punditry. All their "538 chats" were basically the same as talking heads on TV. Okay, the 538 talking heads maybe paid more attention to data, but the good part of 538 was the intent to cut to the chase, dispense with all the puffery that ordinary news sources shove at you, and just let the data speak for itself. In recent years they moved away from that and became less distinguishable from the opinion section of a mainstream news source.

red_hare

I started listening to the 538 Politics podcast a lifetime ago when they did The Gerrymandering Project. The deep intertwining of history, intentions, and statistics made the narrative compelling. I learned so much about how our democracy worked that I would never have known otherwise.

So, I kept listening and kept learning. It was sometimes difficult, not because of their storytelling skills, but because the news was hard to consume. But the cold numbers helped me manage my emotions with clarity and not disengage.

There's something wonderful about journalism backed by data. The line between news and editorial has long been blurred beyond visibility. 538 was a rare example of a place where smart people could express strong opinions but always had to show the work behind their conclusions.

I'll miss 538. They were an amazing team.

kemayo

Yeah, they've been on my podcast subscription list for at least 5 years now, and I'll miss having them around.

I had growing conflicted feelings about the site's overall impact on media, sadly. It felt like, although it was good that they existed as a dedicated organization, they contributed to (or were a symptom of) the overall media landscape's slide into politics coverage as mostly coverage of the horse-race. Sometimes I want to hear what the Scottish teens think a news story means... but more often I want something deeply reported about policy.

alisonatwork

I will miss them too. I saw that Galen is already considering starting his own politics pod, but I fear that by immediately jumping into the Substack black hole it will just end up spiraling into the usual engagement-driven slide to the right: https://www.gdpolitics.com/p/my-thoughts-on-the-end-of-fivet...

Imo part of what made 538 work post-blogosphere heyday was exactly that it had backing from legacy media and the funding to continue sharing information with the public without a paywall. As soon as sites go behind a paywall they become a personification of the "media elite" stereotype, where only rich people have the privilege of being informed. But how otherwise to fund not just a cheerful host but a team of data scientists, editors etc in this day and age? Seems like the only interested billionaires do it with strings attached.

moffers

I really enjoyed 538 in its heyday, and am glad to see Nate carry on with some of the work. I know he can be a polarizing in some circles, but keeping the data angle visible helps smooth some of the rougher edges of following politics sometimes.

rconti

I really enjoy Nate's podcast with Maria Konnikova - I read her book The Biggest Bluff a few years back and really enjoyed it, and them podcasting together is great. (Despite my feelings about never wanting to hear Malcolm Gladwell's voice ever again, as he's omnipresent in Pushkin podcast network ads, which seem to be the worst of the entire podcasting ecosystem).

I never really followed the "Nate Silver" controversy after 2016, but it basically seems to boil down to a bunch of liberals being mad because they felt lied to for no apparent reason.

nozzlegear

> I never really followed the "Nate Silver" controversy after 2016, but it basically seems to boil down to a bunch of liberals being mad because they felt lied to for no apparent reason.

FWIW I don't think he's controversial because of the 2016 polling miss, most people who follow 538 understand what 538 does and that it wasn't his "fault." He's controversial because he posts scalding hot takes on Twitter and then goes to the mat to defend them. He also has a penchant for getting into Twitter beefs with other big names in his industry.

magicalist

Yes. And "the world would be better if more people were reasonable like me, now here's my hot take...", and using angry responses to demonstrate how reasonable he is in contrast. He ran out of content and resorted to ragebait, like most internet pundits.

mmahemoff

I learned more from the reaction to Nate and 538’s forecasts than 538 itself. It helped me appreciate how journalists misunderstand and distort basic probability. If a model predicts A, B, and C as having 34%, 33%, and 33% likelihood respectively, the typical report is “538 predicts ‘A’ will win!” and they got it totally wrong when B or C is the victor. Interpretations of 538 were further fuelled by whatever political bias a pundit was coming from.

In a world where Kevin Rose can reboot Digg, Nate has every chance of acquiring and reviving 538. Good luck to Nate.

laweijfmvo

it must be a wild experience to have a megacorp buy out your brain child, burn it to ashes and throw it away, and chase after the next shiny thing to slap ads on...

prawn

Nate: "I wasn’t technically laid off — but my existing contract was set to expire in June 2023, and there was profound mutual disinterest in negotiating a new deal."

Hamuko

Seems like just the natural samsara of the news world. Small outlets are acquired by larger ones, eventually dismantled, staff moving onto build new ones and starting the cycle anew.

tekla

This is an absolute loss. 538 is amazing because it forced people to confront the cold hard data about polls surrounding politics and if you didn't like it, figure out what they did wrong or deal with it.

I'll never forget being called racist because I showed someone a 538 poll that said the presidential election was at best a toss-up to someone who was sure Kamala Harris would sweep the swing states.

nomilk

> Last night, as President Trump delivered his State of the Union address, the Wall Street Journal reported that ABC News would lay off the remaining staff at 538 as part of broader cuts within corporate parent Disney.

Did Trump's policies cause the layoffs, or does Nate just happen to mention Trump's address? (forgive my ignorance - feels odd to mention the address if it had nothing to do with the layoffs, but I'm not aware of any obvious connection)

stackskipton

No, I think Nate was pointing out while everyone was distracted by Trump State of the Union, 538 was cut so few people would notice.

chasing

I already miss the approval charts over time. Where the best place to see those, now?

bbatha

In this article Nate said his version would be up in a week or so.

relaxing

Pew Research.

dragonwriter

Pew, Gallup, and other pollsters each have charts based on their own individual poll series; 538s were a wide ranging aggregate of pretty much all the public polls which reduced the impact of noise.

MrMcCall

[flagged]

iuyhtgbd

What does that mean?

jordemort

I'm guessing the first word is either "get" or "eat"

iuyhtgbd

I don't know if I'm just dense but I continue to have no idea what the implication is.

MrMcCall

A "half Jewish" (his words) man supporting the party of neonazis is difficult for me to digest.

The last time I wept was watching the 20th part of the BBC's "The World at War", entitled "Genocide". Seeing a dumptruck full of human beings deposited into an excavated pit like so many sacks of potatoes is difficult for me to ignore or forget.

Unlike Nate, evidently.

Jewish folks have been an incredible, integral, extraordinary part of my life's strange journey.

Folks who countenance oppression so long as they're not the ones being oppressed are vile, vile creatures.

Love for the human race and respect for history dictate that their perfidy not be ignored. The negative potentials of powerful human groups are often nothing less than horrific. Being awake to those negative potentials is the most important lesson of the many recurring arcs of history.

henning

[flagged]

bcoates

Innumerates keep self identifying, even 8+ years later. Please learn anything the Fremdschämen is getting painful.

decimalenough

538's final forecast was Kamala 50%, Trump 49%.

https://projects.fivethirtyeight.com/2024-election-forecast/

MrMcCall

"There's lies, damned lies, and statistics." --Unknown

snickerbockers

Nobody who thinks they can pin a probability on an event that only gets sampled once should ever be taken seriously.

NeutralCrane

Each race is only sampled once. But he runs the same methodology on hundreds of races. If the candidates he gives an X% chance to win go on to win approximately X% of the time, the methodology is reliable.

Enginerrrd

Have you ever heard of a brier score?

Or bayes theorem, and bayseian inference?

You might change your mind if you investigate those topics.

iuyhtgbd

This is such tired debate that comes up whenever 538 is mentioned in discussion. Polls sample an election's outcome many times throughout the campaign. Statistics works. You can't know the future but you can predict it with error bars. 51/49 or 70/30 should tell you there's a very real chance of a Trump victory.

People get bent out of shape about 538 but it's usually because they're misinterpreting the prediction, not that the prediction is meaningless.

bcoates

And yet, the silver report final modal outcome was exactly the actual result.

koolba

You mean polls like this that put Clinton 15% ahead? https://www.jsonline.com/story/news/blogs/wisconsin-voter/20...

I think there’s a stronger argument that those polls were intended to suppress Trump voters into thinking they were not the majority (when in fact they were).

lurk2

Clinton won the popular vote by about 3 million votes in 2016.

mvdtnz

Are American elections decided on the popular vote?

FireBeyond

I think there's also a huge misunderstanding of statistics in the US.

More than one person I've spoken to believed that when sites said things like "Hilary has a 70% chance of winning" that that was the same as getting 70% of the vote, i.e. a landslide.

I've had literal arguments with people who can't/won't understand the difference between an average and a median even when presented with 5th grade level phrasing examples.

enjo

>I think there’s a stronger argument that those polls were intended to suppress Trump voters into thinking they were not the majority (when in fact they were).

Show your work. What evidence do you have to support that argument?

cdblades

Are you saying that Trump won the popular vote in 2016?

koolba

He won the popular vote in 2016 in States like Wisconsin that for months said he was 10-15% behind in the polls. It’s my interpretation that those polls were wrong but the majority of the media didn’t care. They just wanted to push a narrative that supporting Trump was a fringe minority in those States, rather than the true majority that it was.

doctorpangloss

Nate, no. FiveThirtyEight is being shut down because its election model is wrong.

WiSaGaN

Glad to hear they will revive the NBA part. Was using the model extensively. It was very informative.