I got fooled by AI-for-science hype–here's what it taught me

Speaking out against the hype is frowned upon. I'm sure even this very measured article about "I tried it and it didn't work for me" will draw negative attention from people who think AI is the Second Coming.

It's also very hard to prove a negative. If you predict "AI will never do anything of value" people will point to literally any result to prove you wrong. TFA does a good job debunking some recent hype, but the author cannot possibly wade through every hyperbolic paper in every field to demonstrate the claims are overblown.

AlphaFold is real.

To the extent you care about chess and Go as human activities, progress there is real.

there are some other scientific computing problems where AI or neural-network-based methods do appear to be at least part of the actual state-of-the-art (weather forecasting, certain single-molecule quantum chemistry simulations).

i would like the hype of the kind described in the article to be punctured, but this is hard to do if critics make strong absolute claims ("aren't useful in any field whatsoever") which are easily disproven. it hurts credibility.

There is rarely a constructive discussion around the term “AI”. You can’t say anything useful about what it might lead to or how useful it might be, because it is purely a marketing term that does not have a specific meaning (neither do both of the words in its abbreviation).

Interesting discussions tend to avoid “AI” in favour of specific terms such as “ML”, “LLM”, “GAN”, “stable diffusion”, “chatbot”, “image generation”. These terms refer to specific tech and applications of that tech, and allow to argue about specific consequences for sciences or society (use of ML in biotech vs. proliferation of chatbots).

However, certain sub-industries prefer “AI” precisely because it’s so vague, offers seemingly unlimited potential (please give us more investment money/stonks go up), and creates a certain vibe of a conscious being useful when pretending not to be working around IP laws and creating tools based on data obtained without relevant licensing agreements (cf. the countless “if humans have the freedom to read, therefore it’s unfair to restrict the uses of a software tool” fallacies, often perpetuated even by seemingly technically literate people, in pretty much every relevant forum thread).

It’s why it keeps looking exactly like NFT’s and crypto hype cycles to me: Yes the technology has legitimate uses, but the promises of groundbreaking use cases that will change the world are obviously not materialising and to anyone that understands the tech it can’t.

It’s people making money off hype until it dies and move on to the next scam-with-some-use.

> then go on to explain how they aren't useful in any field whatsoever

> Where are the AI-driven breakthroughs

> are we just using AI to remix existing general knowledge, while making no progress of any sort in any field using it?

The obvious example of a highly significant AI-driven breakthrough is Alphafold [1]. It has already had a large impact on biotech, helping with drug discovery, computational biology, protein engineering...

[1] https://blog.google/technology/ai/google-deepmind-isomorphic...

> Where are the AI-driven breakthroughs? Or even the AI-driven incremental improvements?

literally last week

https://deepmind.google/discover/blog/alphaevolve-a-gemini-p...

An example of an "AI" incremental improvement would be Oxford Nanopore sequencing. They extrude DNA through a nanopore, measure the current, and decode the bases using recurrent neural networks.

They exist all over science, but they are just one method among many, and they do not really drive hypotheses or interpretations (even now)

Some new ish maths has been discovered. It's up to you if this is valid or impressive enough, but I think it's significant for things to come: https://youtu.be/sGCmu7YKgPA?si=EG9i0xGHhDu1Tb0O

I published my first papers a little over fifteen years ago on practical applications for AI before switching domains. Recently I've been sucked back in.

I agree it's a problem across all of science, but AI seems to attract more than it's fair share of researchers seeking fame and fortune. Exaggerated claims and cherry-picking data seem even more extreme in my limited experience, and even responsible researchers end up exaggerating a bit to try and compete.

Sure, it's not. But often on AI papers one sees remarks that actually mean: "...and if you throw in one zillion GPUs and make them run until the end of time you get {magic_benchmark}". Or "if you evaluate this very smart algo in our super-secret, real-life dataset that we claim is available on request, but we'd ghost you if you dare to ask, then you will see this chart that shows how smart we are".

Sure, it is often flag-planting, but when these papers come from big corps, you cannot "just ignore them and keep on" even when there are obvious flaws/issues.

It's a race over resources, as a (former) researcher on a low-budget university, we just cannot compete. We are coerced to believe whatever figure is passed on in the literature as "benchmark", without possibility of replication.

AI just happens to be the current hype magnet, so the cracks show more clearly

But AI makes it easier to write convincing looking papers

> Just last week i asked for a script to do image segmentation with a basic UI and claude just generated that for me in under 1 Minute.

Thing is we just see that it's copy pasting stack overflow, but now in a fancy way so this is sounding like "I asked Google for a nearby restaurant and it found it in like 500ms, my C64 couldn't do that". It sounds impressive (and it is) because it sounds like "it learned about navigating in the real world and it can now solve everything related to that" but what it actually solved is "fancy lookup in a GIS database". It's useful, damn sure it is, but once the novelty wears off you start seeing it for what it is instead of what you imagine it is.

Edit: to drive the point home.

> claude just generated that

What you think happened is AI is "thinking" and building a ontology over which it reasoned and came to the logical conclusion that this script was the right output. What actually happened is your input correlates to this output according to the trillion examples it saw. There is no ontology. There is no reasoning. There is nothing. Of course this is still impressive and useful as hell, but the novelty will wear off in time. The limitations are obvious by this point.

> “I'm not sure why people on HN (of all places) are so divided regarding the perception of AI/ML.”

Everyone is a rational actor from their individual perspective. The people hyping AI, and the people dismissing the hype both have good reasons.

The is justification to see this new tech as ground breaking. There is justification to be weary about massive theft of data and dismissiveness of privacy.

First, acknowledge and respect that there are so many opinions on any issue. Take yourself out of the equation for a minute. Understand the other side. Really understand it.

Take a long walk in other people’s shoes.

HN is always divided on "how much is the currently hype-y technology real vs just hype".

I've seen this over and over again and been on different sides of the question on different technologies at different times.

To me, this is same as it ever was!

But on the flip side, the "AI will revolutionize science" narrative feels way ahead of what the evidence supports

Google never gave a good reason for why they stopped making their cache public, but my theory is that it was because people were scraping it to train their LLMs.

>but alone for every researcher who needs to write code for their research, AI can make them already a lot more efficient.

scientists don't need to be efficient, they need to be correct. Software bugs were already a huge cause of scientific error, and responsible for lack of reproducibility, see for example cases like this (https://www.vice.com/en/article/a-code-glitch-may-have-cause...)

Programming in research environments is done with some notoriously questionably variation in quality, as is the case for the industry to be fair, but in research minor errors can ruin results of entire studies. People are fed up and come to much harsher judgements on AI because in an environment like a lab you cannot write software with the attitude of an impressionist painter or the AI equivalent, you need to actually know what you're typing.

AI can make you more efficient if you don't care if you're right, which is maybe cool if you're generating images for your summer beach volleyball event, but it's a disastrous idea if you're writing code in a scientific environment.

There are those who have realised they can make a lot of cash from it and also get funding by using the term AI. But at the end of the day what software doesn't have some machine learning built in. It's nothing new, nor is the current implementations particularly extraordinary or accurate.

I don't want to generalize because I do not know how widespread this pattern is, but my job has me hopping between a few HPC centers around Germany, and a pattern I notice is that, a lot of these places are chuck full of reject physicists, and a lot of the AI funding that gets distributed gets gobbled up by these people and the consequence of which is a lot of these ML4Science projects. I personally think it is a bit of a shame, because HPC centers are not there to only serve physicists, and especially with AI funding we in Germany should be doing more AI-core research.

Seriously don't understand what "no longer" does here.

It's more widespread than PINNs. PINNs have been widely known to be rubbish a long time ago. But the general failure of using ML for physics problems is much more widespread.

Where ML generally shines is either when you have relatively lots of experimental data with respect to a fairly narrow domain. This is the case for machine learned interatomic potentials MLIPs which have been a thing since the '90s. Also potentially the case for weather modelling (but I do not want to comment about that). Or when you have absolute insane amounts of data, and you train a really huge model. This is what we refer to as AI. This is basically why Alphafold is successful, and Alphafold still fails to produce good results when you query it on inputs that are far from any data points in its training data.

But most ML for physics problems tend to be somewhere in between. Lacking experimental data and working with not enough simulation data because it is so expensive to produce. And also training models that are not large enough, because inference would be too slow, anyway, if they were too big. And then expecting these models to learn a very wide range of physics.

And then everyone jumps in on the hype train, because it is so easy to give it a shot. And everyone gets the same dud results. But then they publish anyway. And if the lab/PI is famous enough or if they formulate the problem in a way that is unique and looks sciency or mathy, they might even get their paper in a good journal/conference and get lots of citations. But in the end, they still only end up with the same results as everyone else: replicates the training data to some extent, somebody else should work on the generalizability problem.

He published a whole paper providing a systematic analysis of a wide range of models. There's a whole section on that. So it's not specific to PINN.

Replace PINN with any "AI" solution for anything and you'll still find it overhyped.

The only realistic evaluations of "AI" so far are those that admit it's only useful for experts to skip some boring work. And triple check the output after.

I’m not a scientist or a researcher, but anything based on statistics and data interpretation is immediately subject to my skepticism.