Obituary for Cyc

> LLMs did scale with huge data, symbolic AI did not.

Symbolic AI have not had a privilege to be applied or "trained" with huge data. 30 millions assertions is not a big number.

LLMs did scale with huge data, symbolic AI did not. So why? [1]

Neural networks, not LLMs in particular, were just about the simplest thing that could scale - they scaled and everything else has been fine-tuning. Symbolic AI basically begins with existing mathematical models of reality and of human reason and indeed didn't scale.

The problem imo is: The standard way mathematical modeling works[2] is you have a triple of <data, model-of-data, math-formalism>. The math formalism characterizes what the data could be, how data diverges from reality etc. The trouble is that the math formalism really doesn't scale even if a given model scales[3]. So even if you were to start plugging numbers into some other math model and get a reality-approximation like an LLM, it would be a black box like an LLM because the meta-information would be just as opaque.

Consider the way Judea Pearl rejected confidence intervals and claimed probabilities were needed as the building blocks for approximate reasoning systems. But a look at human beings, animals or LLMs shows that things that "deal with reality" don't have and couldn't access to "real" probabilities.

I'd just offer that I believe that for a model to scale, the vast majority of it's parameters would have to be mathematically meaningless to us. And that's for the above reasons.

[1]. Really key point, imo [2]. That innclude symbolic and probabilistic model "at the end of the day" [3]. Contrast the simplicity of plugging data into a regression model versus the multitudes of approaches explaining regression and loss/error functions etc.

>I would be interested in reading a paper that does a good job of explaining what a parameter ends up representing in an LLM model.

https://distill.pub/2020/circuits/ https://transformer-circuits.pub/2025/attribution-graphs/bio...

>> This is exactly correct, LLMs did scale with huge data, symbolic AI did not. So why?

Like the rock salad you're mixing up two disparate contexts here. Symbolic AI like SAT solvers and planners is not trying to learn from data and there's no context in which it has to "scale with huge data".

Instead, what modern SAT solvers and planners do is even harder than "scaling with data" - which, after all, today means having imba hardware and using it well. SAT solving and planning can't do that: SAT is NP-complete and planning is PSPACE-complete so it really doesn't matter how much you "scale" your hardware, those are not problems you can solve by scaling, ever.

And yet, today both SAT and planning are solved problems. NP complete? Nowadays, that's a piece of cake. There are dedicated solvers for all the classical sub-categories of SAT and modern planners can solve planning problems that require sequences of thousands of actions. Hell, modern planners can even play Atari games from pixels alone, and do very well indeed [1].

So how did symbolic AI manage those feats? Not with bigger computers but precisely with the approach that the article above seems to think has failed to produce any results: heuristic search. In SAT solving, the dominant approach is an algorithm called "Conflict Driven Clause Learning", that is designed to exploit the special structure of SAT problems. In Planning and Scheduling, heuristic search was always used, but work really took off in the '90s when people realised that they could automatically estimate a heuristic cost function from the structure of a planning problem.

There are parallel and similar approaches everywhere you look at, in classical AI problems, like verification, theorem proving, etc, and that work has even produced a few Turing awards [2]. But do you hear about that work at all, when you hear about AI research? No, because it works, and so it's not AI.

But it works, it runs on normal hardware, it doesn't need "scale" and it doesn't need data. You're measuring the wrong thing with the wrong stick.

____________

[1] Planning with Pixels in (Almost) Real Time: https://arxiv.org/pdf/1801.03354 Competitive results with humans and RL. Bet you didn't know that.

[2] E.g. Pnueli for temporal logic in verification, or Clarke, Emerson and Sifakis, for model checking.

I think logical reasoning - so reasoning about logical problems, especially those with transitive relations like two way implication. A way round this is to get them to write prolog relations and then reason over them... with prolog. This isn't a fail - it's what things like prolog do, and not what things like nns do. If I was asked to solve these problems I would write prolog too.

I think quite a lot of planning.

I think scheduling - I tried something recently and GPT4 wrote python code which worked for very naive cases but then failed at any scale.

Basically though - trusted reasoning. Where you need a precise and correct answer LLM's aren't any good. They fail in the limit. But where you need a generally decent answer they are amazing. You just can't rely on it.

Whereas GOFAI you can, because if you couldn't the community thew it out and said it was impossible!

"Tried to address" is not the same as "can do well."

I was responding to PP, but some other (maybe obvious?) examples are logical reasoning and explainability.

As PP suggests, some of the classical symbolic ideas may be applicable or complementary to current approaches.

SAT solving, verification and model checking, automated theorem proving, planning and scheculing, knowledge representation and reasoning. Those are fields of AI research where LLMs have nothing to offer.

Theorem proving.

To be fair, the object "FredWhileShaving" is a state (of Fred), or simply an event, and as such it arguably is not a person in the first place. As persons are presumably neither states nor events.

Deleuze would argue something similar

LLMs are good at translating between languages that have significant amounts of written content on the internet. There are few languages in this category that do not already have correspondingly large Wikipedias.

There are plenty of languages with millions of speakers that are only rarely used in writing, often because some other language is enforced in education. If you try to use an LLM to translate into such a language, you'll just get garbage.

It's very easy for a hand-crafted template to beat an LLM if the LLM can't do the job at all.

Strictly speaking, Wikidata is an existing project which only provides a rather restrictive model for its assertions; they are not fully compositional, thus are quite far from being able to express general encyclopedic text, especially in a way that can be 'seamlessly' translated to natural language. It does provide a likely foundation for these further planned developments, though.

You're forcing the proud dad function: https://pubmed.ncbi.nlm.nih.gov/36995257/

Especially if you consider using a SOTA LLM to recreate and expand ResearchCyc (for fun and not profit)

It's an interesting thought experiment / philosophy / sci fi story premise though; if he spent all those years encoding his own thought processes and decision making into a program, would he have been able to create a convincing facsimile of himself and his "consciousness"? A turing test with a chatbot based on decades of self-reflection.

The Bitter Lesson says "general methods that leverage computation are ultimately the most effective". That doesn't seem to rule out symbolic approaches. It does rule out anything which relies on having humans in the loop, because terabytes of data plus a dumb learning process works better than megabytes of data plus expert instruction.

(I know your message wasn't claiming that The Bitter Lesson was explicitly a counterpoint, I just thought it was interesting.)

Imho, this is wrong. Even independent of access to vast amounts of compute, symbolic methods seem to consistently underperform statistical/numerical ones across a wide variety of domains. I can't help but think that there's more to it than just brute force.

check out Imandra's platform for neurosymbolic AI - https://www.imandra.ai/

It does seem like the Cyc people hit the wall with simply collecting facts. Having to have a human in the loop.

The problem I think is if you have LLMs figuring out the propositions, the whole system is just as prone to garbage-in-garbage-out as LLMs are.

Um, the "something" is the person reading the LLM's output. I'm afraid that you have completely missed the context and point of the discussion, which was not about LLMs understanding things--they understand nothing ("LLMs have no cognitive states"). But again, "because of how extremely well the statistical apparatus works", their outputs are useful to intelligent consumers who do have cognitive states--us.

On libgen.mx they claim to have 33,569,200 books and 84,844,242 articles

I dunno I actually think say Claude AI SOUNDS smarter than it is, right now

It has a phenomenal recall. I just asked it about "SmartOS", something I knew about, vaguely, in ~2012, and it gave me a pretty darn good answer. On that particular subject, I think it probably gave a better answer than anyone I could e-mail, call, or text right now

It was significantly more informative than wikipedia - https://en.wikipedia.org/wiki/SmartOS

But I still find it easy to stump it and get it to hallucinate, which makes it seem dumb

It is like a person with good manners, and a lot of memory, and which is quite good at comparisons (although you have to verify, which is usually fine)

But I would not say it is "smart" at coming up with new ideas or anything

I do think a key point is that a "text calculator" is doing a lot of work ... i.e. summarization and comparison are extremely useful things. They can accelerate thinking

> There’s no set definition for what a chair is.

Sure there is: a chair is anything upon which I can comfortably sit without breaking it.

So you're just ignoring all the probabilistic, fuzzy etc. Prologs etc. which do precisely that? https://github.com/lab-v2/pyreason

> Any project trying to use deterministic symbolic logic to represent the world fundamentally misunderstands cognition.

The counterposition to this is no more convincing: cognition is fuzzy, but it's not really clear at all that it's probabilistic: I don't look at a stump and ascertain its chairness with a confidence of 85%, for example. The actual meta-cognition of "can I sit on this thing" is more like "it looks sittable, and I can try to sit on it, but if it feels unstable then I shouldn't sit on it." In other words, a defeasible inference.

(There's an entire branch of symbolic logic that models fuzziness without probability: non-monotonic logic[1]. I don't think these get us to AGI either.)

[1]: https://en.wikipedia.org/wiki/Non-monotonic_logic

> The human mind is a probabilistic computer, at every level.

Fair enough, but an airplane's wing is not very similar to a bird's wing.

> There’s no way to use “rules and facts” to express concepts like “chair” or “grass”, or “face” or “justice” or really anything. Any project trying to use deterministic symbolic logic to represent the world fundamentally misunderstands cognition.

Are you sure? In terms of theoretical foundations for AGI, AIXI is probabilistic but godel-machines are proof based and I think they'd meet criteria for deterministic / symbolic. Non-monotonic and temporal logics also exist, where chairness exists as a concept that might be revoked if 2 or more legs are missing. If you really want to get technical then by allowing logics with continuous time and changing discrete truth values, then you can probably manufacture a fuzzy logic where time isn't considered but truth/certainty values are continuous. Your ideas about logic might be too simple, it's more than just Aristotle

> This kind of fuzzy reasoning is the rule, not the exception when it comes to human intuition.

That is indeed true. But we do have classic fuzzy logic, and it can be used to answer these questions. E.g. a "stool" maybe a "chair", but "automobile" is definitely not.

Maybe the symbolic logic approach could work if it's connected with ML? Maybe we can use a neural network to plot a path in the sea of assertions? Cyc really seems like something that can benefit the world if it's made open under some reasonable conditions.

"The human mind is a probabilistic computer, at every level."

We don't know that. It's mostly probabilistic. That innate behavior exists suggests some parts might be deterministic.

Words are used due to the absence of things. They fill an immediate experiential void and stand in for something else, because you want or need another person to evoke some fantasy to fill this absence and make understanding possible.

If you have a mind and it is a computer, then it is because of nurture, because the brain is nothing like a computer, and computers simulating language are nothing like brains.