Do the thinking models think?

A human who is familiar with the original surgeon riddle could also be tricked the same way that ChatGPT was tricked here. I don't think all LLMs would consistently fall for that one either.

Hmm, I don't know if the example really shows what the article argues it does.

If someone came up to me and told me the altered version of the joke I have heard many times, I might answer exactly like Chat GPT did. I would hear the beginning of the story, say "wait, i know this one", and not really pay attention to the rest because I would be all ready to respond with what I think is the right answer.

I bet if you point out the mistake, the LLM will say "oh you are right, this story actually does specify the gender of the doctor" or something to that effect... just like you or I would.

Now, I am not arguing that LLMs are really 'thinking' like humans, but I also find the second argument a bit tenuous. The article conjectures that humans reason from ideas to symbols while LLMs go from symbols to ideas, but couldn't that just be a post hoc interpretation we have for how our ideas come to our brain? We think we have ideas first, but an idea is just the result of neurons firing in our brains... and neurons are really just a probability function connected to other probability functions, just like an LLM... we give it symbols we understand to represent those probabilities, but that is just for our benefit so we can understand it.

It could be that the only difference between us and an LLM is processing power and the training data generated over billions of years of evolution.

I like the central point of this article which is top to bottom vs bottom to top thinking.

But I wonder if there is a falsifiable, formal definition to suggest that models (or anything for that matter) _do_ think.

The normal reply to chatgpt getting a question right is that it simply extrapolated what was already in the training data set. But I feel like the degree to which something "thinks" is the ability to generalise what it already knows.

This generalisation needs some formality - maybe some mathematical notation (like the opposite of overfitting). By generalisation I mean the ability to get something correct that exists pretty far from the training data.

The reason I suggest this is because GPT can solve pretty much any high school math problem you throw at it and it can do it better than 99% of humans. This is clearly not just memorising training data but doing something more. If it were not generalising, it couldn't possibly solve all new high school level mathematics.

But the extent decreases as you go higher level into undergraduate mathematics where it can still solve most problems you throw at it but not all. And still lower in PhD level mathematics. So the "thinking" ability of GPT exists somewhere in between - in some spectrum. But I don't think you can directly say that it can never generalise PhD level mathematics.. it could do it for high school so why not PhD?

If hypothetically it can solve PhD level mathematics, would people still claim that LLM's don't think?

The notion of "thinking" is not clear. I'll agree thinking with symbols is powerful and something (adult) humans and computers can do, but is is not the only way of making decisions. I'm going to suggest LLMs are not thinking this way, but that indeed "glorified auto complete" (c.f. Hinton) is far more useful than it seems. Https://arxiv.org/abs/2402.08403

Inference means synthesizing new facts from facts already known. A large language model only knows facts about language elements in its training corpus, therefore any reasoning based on such a model can only ever derive facts about language.

No. They do not.

Top-to-bottom reasons and Bottom-to-top understands.

Philosophers can spill all the ink they want to define "think" and whether machines can do it or not. Given some input, the machine takes some time, and then comes up with some output. Coloquially, the machine is thinking during that time. This has been true since there have been computers and long before LLMs. Now that computers can generate essays about anything, maybe it becomes a question that people feel is important to answer for their day to day life, but I doubt it.

This debate is a huge red herring. No one is ever going to agree on what 'thinking' means, since we can't even prove that other people are thinking, only that one's self is.

What we should concentrate on is agency. Does the system have its own desires and goals, and will it act on its own accord to achieve them? If a system demonstrates those things, we should accord it the benefit of the doubt that it should have some rights and responsibilities if it chooses to partake in society.

So far, no AI can pass the agency test -- they are all reactive such that they must be given a task before they will do anything. If one day, however, we wake up and find that an AI has written a book on its own initiative, we may have some deciding to do.

Hi. This is my company blog which I use to write about pretty much what ever happens to interest me.

Hope you like this week's post.

https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...

Thinking/intelligence is like magic, to use Dennett's analogy. Any time a machine can do it, it becomes engineering. It's just a bag of tricks. The circle of true thinking/intelligence is an ever shrinking bag of tricks that only carbon-based minds produced through sexual intercourse can do. In fact I'm surprised they call test tube babies intelligent. Do they have souls?