Cognitive Behaviors That Enable Self-Improving Reasoners

This is a well-known approach: verbalizing your thought process (either by speaking aloud, or by writing) is something that's long established as a good tactic for making sure that you're actually thinking through something, rather than glossing over it. Ironically, I've seen people bemoaning that use of AI will rob people of that.

I agree that there's potential here, though, and do genuinely hope that we find ways to make human intelligence better as we're going about AI research. Even pessimistically, I think we'll at least surface approaches that people use without thinking about, which is on its own a good thing, because once you know you're doing something, it becomes a lot easier to train yourself to do it better.

I use this method too for programming problems I would normally procrastinate on and offload to subconscious thinking.

Actually writing out all thinking steps helps with ironing out some wrong steps in my reasoning or going in circles due to having limited working memory.

I started doing this more rigorously after seeing how reasoning based AI does reasoning, because it seemed like a useful thinking technique.

These reasoning AI models help me think on a meta level about my own thinking and shows me tools I can use to improve it.

Great to see that I’m not alone in this!

Thinking out loud is an age old practice and is the equivalent of "rubber ducking" to yourself.

As someone who comes from a long ancestral line of people who talk to themselves while reasoning through problems - it would occasionally prove to be a minor handicap during proctored exams, as internal monologue isn't really the same thing.

You computer scientists cosplaying as cognitive scientists really never took single psychology class, did you?

Those four parts sound like one unified, cognitive algorithm -- having an ontology of the problem by breaking it into subgoals; checking your work properly; thinking backwards to debug a mistake and retrying; and thinking ahead and reasoning backward from the end result. It's all just one algorithm for solving hard problems. A skill that can be practiced, and then it builds on itself.

Consider the recent advancements in reasoning models, I’d say your method is a bit inefficient ;)

It’s equivalent to the LLMs reasoning in the output and not in the latent space before the final output, which gave the rise to the reasoning models we see today. So speaking out loud might not be the best reasoning method ;)

Super interesting

> As I read over practice questions, I spoke aloud

This is also something that’s expected of the applicant in technical interviews

The interviewers want to hear the applicants thought process and how they develop a strategy to solve the problems presented to them as they work them out

… some of yall

Soon enough, they'll have LLMs that reason purely from the first principles of The Subtle Art of Not Giving a Fuck.

We've had knowledge of how to eat better to not get extreme scenarios like obesity and look at the effectiveness of that. Until you have a pill that makes you think better only the motivated will do it, and in this case the motivated could already do it.

As someone with an educational background I actually often ask myself the opposite: Why don't AI techniques almost never seem to use the knowledge we have about human learning to train better AI?

So far, I don't think we found anything interesting, yet.

You never form thoughts in your mind in a linguistic way? Can you read a sentence and be aware of it as a sentence in your mind, or are you unable to do that?

I don't doubt you or anything like that, just very curious. As someone with a very strong internal monologue, it's hard for me to imagine not having one.

Genuine question, how does multi step reasoning work for you then? Like eg if you have some math problem that's trivially to solve individually but needs multiple steps, lets say 16 * 3 + 5? How does 16 * 3 = 48 land in some 'register' of your brain (short term memory), so that you can then add 5 to get to 53? Maybe 16 * 3 + 5 is to easy for you and you'll just 'see' it but the question still stands, just choose a more complex problem.

Isn't the same meta process at play when thinking about more fuzzy topics?

I do have an internal monologue. I can also think in pictures and I can also think in terms of neither, just pure raw thought.

I would say most people are like me. They have 3 modes of thinking and they probably have a primary mode which they favor. I favor none and go into all 3 depending on whether I’m reading, writing or doing something else.

The second bigger group has only one primary mode of thinking. The internal monologue. They can only think in terms of an inner voice and this inner voice is so powerful I often encountered people who think this inner voice is the definition of thought. They assumed thinking was COT.

The even rarer versions you get people who assign colors to numbers or people who can’t even perceive to think in pictures. You’re the first person I’ve encountered who can’t even have an internal monologue.

There's a fascinating thing called aphantasia where people can't picture things at all in their mind, but such people are able to lead normal lives and may never realize there's something different. This feels like a similar concept but for imaging speech.

https://en.m.wikipedia.org/wiki/Aphantasia

That said, most of my thinking is not done in the form of a linear monologue where I "talk through" steps to myself.

So you cannot think in language? Sounds kinda scary to be honest.

I’ve heard this described as talking in “Neuralese”. It seems plausible that this will be the most dense language for model-internal dialog (or presumably inter-LLM dialog assuming they share the same weights).

You will penalize this inasmuch as your alignment strategy depends on Deliberative Alignment. But at some point I assume that will come with a real capability cost as Neuralese can be more conceptually dense.

They are not going to invent a new language by themselves, they by definition can't even "think" in terms of languages they haven't seen. It does not occur to them that the language they use may be suboptimal. And surely, any better ways of thinking can still be described in English. It seems more likely there will be a gradual transition from us teaching LLMs how to reason, to LLMs being able to actually gobble and process enough data to learn more effective ways to reason, which it can then "teach" us. But that's just the LLM reflecting the way it was trained and aligned.

Once you suspend disbelief of AI all of this pseudoscience becomes just as plausible.

In my experience models aren't very good at following such prompts. Smart "non-reasoning" models like Claude 3.5 could, but would generate so much text when thinking they ran out of context.

That's LaTeX syntax for opening and closing quotes. Though their rendered paper doesn't render them that way for some reason!