Richard Sutton and Andrew Barto Win 2024 Turing Award

I am still hoping AI progress will get to the point where the AI can eventually create AI's that are built up out of robust and provable logic which can be read and audited. Until that time, I wouldn't trust it for risky stuff. Unfortunately, it's not my choice and within a scarily short timespan, black boxes will make painfully wrong decisions about vital things that will ruin lives.

Well, take compiler optimization for example. You can allow your AI to use correctness-preserving transformations only. This will give you correct output no matter how weird the AI behaves.

The downside is that you will sometimes not get the optimizations that you want. But, this is sort of already the case, even with human made optimization algorithms.

I was an undergrad interning in a computer vision lab in the early 2010s. During group meeting, someone presented a new paper that was using abstract machine learning like stuff to do vision. The prof was so visibly perturbed and agnostic. He could not believe that this approach was even a little bit viable, when it so clearly was.

Best lesson for me - vowed never to be the person opposed to new approaches that work.

> if you do AI research with the idea that by teaching machines how to do X, we might also be able to gain insight in how people do X, then ever more complex statistical setups will be of limited information

At the very least we know consistent language and vision abilities don't require lived experience. That is huge in itself, it was unexpected.

First there was AlphaGo, which had learnt from human games, then further improved from self-play, then there was AlphaGo Zero which taught itself from scratch just by self-play, not using any human data at all.

Game programs like AlphaGo and AlphaZero (chess) are all brute force at core - using MCTS (Monte Carlo Tree Search) to project all potential branching game continuations many moves ahead. Where the intelligence/heuristics comes to play is in pruning away unpromising branches from this expanding tree to keep the search space under control; this is done by using a board evaluation function to assess the strength of a given considered board position and assess if it is worth continuing to evaluate that potential line of play.

In DeepBlue (old IBM "chess computer" that beat Kasparov) the board evalation function was hand written using human chess expertise. In modern neural-net based engines such as AlphaGo and AlphaZero, the board evaluation function is learnt - either from human games and/or from self-play, learning what positions lead to winning outcomes.

So, not just brute force, but that (MCTS) is still the core of the algorithm.

> ... This describes Go AIs as a brute force strategy with no heuristics ...

no, not really, from the paper

>> Also important was the use of learning by self play to learn a value function (as it was in many other games and even in chess, although learning did not play a big role in the 1997 program that first beat a world champion). Learning by self play, and learning in general, is like search in that it enables massive computation to be brought to bear.

important notion here is, imho "learning by self play". required heuristics emerge out of that. they are not programmed in.

The paragraph on Go AI looked accurate to me. Go AI research spent decades trying to incorporate human-written rules about tactics and strategy. None of that is used any more, although human knowledge is leveraged a bit in the strongest programs when choosing useful features to feed into the neural nets. (Strong) Go AIs are not trained on human games anymore. Indeed they don't search the entire sample space when they perform MCTS, but I don't see Sutton claiming that they do.

that was “bitter lesson” in action.

for example there are clever ways of rewarding all the steps of a reasoning process to train a network to “think”. but deepseek found these don’t work as well as much simpler yes/no feedback on examples of reasoning.

>this fetishism to dig into another person’s personal life and find the most weird thing they said as the thing that paints over all of their life’s achievements as evil must stop.

This has nothing to do with his professional life. He has made these comments in a professional capacity at an industry AI conference... The rest of your comment is a total non sequitur.

>And worst of all internet gives your opinion the same weight as someone else (or the rest of us) who knows a lot about thing B that could change the world. From a strictly professional capacity.

I've worked professionally in the ML field for 7 years so don't try some appeal to authority bs on me. Geoff Hinton, Yoshua Bengio, Demis Hassabis, Dario Amodei and countless other leaders in the field all recognize and highlight the possible dangers of this technology.

> all of their lofty accomplishments get nullified by anyone

I don't think it's a question of whether their achievements are nullified, but as you mention, how to weight the opinions of various people. Personally, I think both a Turing award for technical achievement and a view that humanity ought to be replaced are relevant in evaluating someone's opinions on AI policy, and we shouldn't forget the latter because of the former.

(Also, this isn't about Sutton's personal life - that's a pretty bad strawman.)

>the most weird thing they said

Reminds me of a quote from Jean Cocteau, of which I could not find the exact words, but which roughly says that if the public knew what thoughts geniuses can have, it would be more terrified than admiring.

Or we could just you know, not build the thing that will probably kill us all and at minimum will obsolete all our labor value.

The video is not about his technical work but rather his view that AI will or should take over the future.

> ai succession necessarily includes replacement

How is AI going to make its own chips and energy? The supply chain for AI hardware is long an fragile. AGI will have an interest in maintaining peace for this reason.

And why would it replace us, our thoughts are like food for AI. Our bodies are very efficient and mobile, biology will certainly be an option for AGI at some point.

At his level it is much more than just being able to do what he wants, it’s about attracting resources and talent to accomplish his goals.

From that perspective location still matters if you want to maximise impact

Yeah, the formalisations can be hard to crunch through (especially because of [1]). But this book in particular is quite well laid out. I'd suggest getting a math background on the (very) basics of "contraction mappings", as this is something the book kind of assumes you have the knowledge of.

[1] There's a lot of confusing naming. For example, due to its historic ties with behavioural psychology, there are a bunch of things called "eligibility traces" and so on. Also, even more than the usual "obscurity through notation" seen in all of math and AI, early RL literature in particular has particularly bad notation. You'd see the same letter mean completely different things (sometimes even opposite!) in two different papers.

Yeah that one. Messed up the name.