Dumb statistical models, always making people look bad
5 comments
·April 19, 2025rawgabbit
OTOH. The blog mentions that humans excel at novel situations. Such as when there is little training data, when envisioning alternate outcomes, or when recognizing the data is wrong.
The most recent example I can think of is "Frank". In 2021, JPMorgan Chase acquired Frank, a startup founded by Charlie Javice, for $175 million. Frank claimed to simplify the FAFSA process for students. Javice asserted the platform had over 4 million users, but in reality, it had fewer than 300,000. To support her claim, she allegedly hired a data science professor to generate synthetic data, creating fake user profiles. JPMorgan later discovered the discrepancy when a marketing campaign revealed a high rate of undeliverable emails. In March 2025, Javice was convicted of defrauding JPMorgan.
IMO an data expert could have recognized the fake user profiles through the fact he has seen e.g., how messy real data is, know the demographics of would be users of a service like Frank (wealthy, time stressed families), know tell tale signs of fake data (clusters of data that follow obvious "first principles").
nitwit005
You don't even need a statistical model. We make checklists because we know we'll fail to remember to check things.
Humans are tool users. If you make a statistical table to consult for some medical issue, you've using a tool.
delichon
> why it’s often hard to demonstrate the value of human knowledge once you have a decent statistical model.
This seems to be a near restatement of the bitter lesson. It's not just that large enough statistical models outperform algorithms built from human expertise, they also outperform human expertise directly.
gopalv
> they also outperform human expertise directly
When measured statistically.
This is not a takedown of that statement, but the reason we've trouble with this idea is that it works in the lab and not always in real life.
To set up a clean experiment, you have define what success looks like before you conduct the experiment - that the output variable is defined.
Once you know what to measure ahead of time to determine success, then statistical models tend to not be as random as a group of humans in achieving that target.
The variance is bad in an experiment, but variance jitter is needed in an ever changing world even if most variants are worse off.
For example, if you can predict someone's earning potential from their birth zipcode, it is not wrong and often more right than otherwise.
And then if you base student loans and business loan interest rates on the basis of birth zipcodes, the original prediction does become more right.
The experimental version that's a win, but in real life that's a terrible loss to society.
As a matter of practicality, it seems that you professionally now want to be firmly in the tails of the data distribution for your field, e.g. expert in those things that happen rarely.
Or maybe even be in a domain which, for whatever reason, is poorly represented by a statistical model, something where data points are hard to get.