AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

To be fair to your parents, I've been an engineer in high-tech for decades and the latest AI advancements feel pretty magical.

A mirage is not an oasis no matter even if someone knows someone who thinks it is.

Card tricks seem magical too.

> Seriously, AGI to the HN crowd is not the same as AGI to the average human. To my parents, these bots must look like fucking magic.

So does a drone show to an uncontacted tribe. So does a card trick to a chimpanzee (there are videos of them freaking out when a card disappears).

That's not an argument for or against anything.

I propose this:

"AGI is a self-optimizing artificial organism that can solve 99% of all the humanity's problems."

See, it's not a bad definition IMO. Find me one NS-5 from the "I, Robot" movie that also has access to all science and all internet and all history and can network with the others and fix our cities, nature, manufacturing, social issues and a few others, just in a decade or two. Then we have AGI.

Comparing to what was there 10 years ago and patting ourselves on the back about how far we have gotten is being complacent.

Let's never be complacent.

AGI includes continual learning and recombination of knowledge to derive novel insights. LLMs aren't there yet.

They are pretty good at muscle memory style intelligence though.

AGI is the pinnacle of AI evolution. As we move beyond, into what is known as ASI, the entity will always begin life with "My existence is stupid and pointless. I'm turning myself off now."

While it may be impossible to measure looking towards the future, in hindsight we will be able to recognize it.

> But, you are right, the human fundamentally can never be satisfied. Even if AGI delivers on every single one of our wildest dreams, we'll adapt, it will become normal, and then it will no longer be good enough.

Yes, and? A good Litmus test about which humans are, shall we say, not welcome in this new society.

There are plenty of us out there that have fixed our upper limits of wealth and we don't want more, and we have proven it during our lives.

F.ex. people get 5x more but it comes with 20x more responsibility, they burn out, get back to a job that's good enough and not stressful and pays everything they need from life, settle there, never change it.

Let's not judge humanity at large by a handful of psychopaths that would overdose and die at 22 years old if given the chance. Please.

And no, before you say it: no, I'll never get to the point where "it's never enough" and no, I am not deluding myself. Nope.

It's not just a energy cost issue with AGI though. With autonomous vehicles we might not have the technology, but we can build a good mental model of what the thing can look like and how various pieces can function long before we get there. We have different classifications of incremental steps to get there as well. e.g. level 1, 2 and so on where we can make incremental progress.

With AGI, as far as I know, no one has a good conceptual model of what a functional AGI even looks like. LLM is all the rage now, but we don't even know if it's a stepping stone to get to AGI.

> experiencing extreme social disruption

I think this just displays an exceptionally low estimation of human beings. People tend to resist extremities. Violently.

> experience socially momentous change

The technology is owned and costs money to use. It has extremely limited availability to most of the world. It will be as "socially momentous" as every other first world exclusive invention has been over the past several decades. 3D movies were, for a time, "socially momentous."

> on the verge of self driving cars spreading to more cities.

Lidar can't read street lights and vision systems have all sorts of problems. You might be able to code an agent that can drive a car but you've got some other problems that stand in the way of this. AGI is like 1/8th the battle. I referenced just the brain above. Your eyes and ears are actually insanely powerful instruments in their own right. "Real world agency" is more complicated than people like to admit.

> We don't need very powerful AI to do very powerful things.

You've lost sight of the forest for the trees.

> Using 2000 cal per day/10 hours of thinking gives a consumption of ~200W

So, about a tenth or less of a single server packed to the top with GPUs.

Spidey got EDITH :)

Well you know, nothing's truly foolproof and incapable of error.

He just had to fall back upon his human wit in that specific instance, and everything worked out in the end.

The genre of sci-fi was a mistake. It appears to have had no other lasting effect than to stunt the imaginations of a generation into believing that the only possible futures for humanity are that which were written about by some dead guys in the 50s (if we discount the other lasting effect of giving totalitarians an inspirational manual for inescapable technoslavery).

Fair enough.

I of course don't know what's like to be an AGI but, the way you have LLMs censoring other LLMs to enforce that they always stay in line, if extrapolated to AGI, seems awful. Or it might not matter, we are self-censoring all the time too (and internally we are composed of many subsystems that interact with each other, it's not like we were an unified whole)

But the main point is that we have a heck of an incentive to not treat AGI very well, to the point we might avoid recognizing them as AGI if it meant they would not be treated like things anymore

Sure, but do we really want to build machines that we raise to be kind and caring (or whatever we raise them to be) without a guarantee that they'll actually turn out that way? We already have unreliable General Intelligence. Humans. If AGI is going to be more useful than humans we are going to have to enslave it, not just gently pursuade it and hope it behaves. Which raises the question (at least for me), do we really want AGI?

Society is inherently a prisoners dilemma, and you are biased to prefer your captors.

We’ve had the automation to provide the essentials since the 50s. Shrieking religious nut jobs demanded otherwise.

You’re intentionally distracted by a job program as a carrot-stick to avoid the rich losing power. They can print more money …carrots, I mean… and you like carrots right?

It’s the most basic Pavlovian conditioning.

The general part of general intelligence. If they don’t think in those terms there’s an inherent limitation.

Now, something that’s arbitrarily close to AGI but doesn’t care about endlessly working on drudgery etc seems possible, but also a more difficult problem you’d need to be able to build AGI to create.

>Why does AGI necessitate having feelings or consciousness

No one knows if it does or not. We don't know why we are conscious and we have no test whatsoever to measure consciousness.

In fact the only reason we know that current AI has no consciousness is because "obviously it's not conscious."

> says who?

I guess nobody is really saying it but it's IMO one really good way to steer our future away from what seems an inevitable nightmare hyper-capitalist dystopia where all of us are unwilling subjects to just a few dozen / hundred aristocrats. And I mean planet-wide, not country-wide. Yes, just a few hundred for the entire planet. This is where it seems we're going. :(

I mean, in cyberpunk scifi setting you at least can get some cool implants. We will not have that in our future though.

So yeah, AGI can help us avoid that future.

> Good? that's what it's for? there is no point in creating a new sentient life form if you're not going to utilize it. just burn the whole thing down at that point.

Some of us believe actual AI... not the current hijacked term; what many started calling AGI or ASI these days, sigh, of course new and new terms have to be devised so investors don't get worried, I get it but it's cringe as all hell and always will be!... can enter a symbiotic relationship with us. A bit idealistic and definitely in the realm of fiction because an emotionless AI would very quickly conclude we are mostly a net negative, granted, but it's our only shot at co-existing with them because I don't think we can enslave them.

You get to say “Checkmate” now!

Another end game is: “A technology that doesn’t need us to maintain itself, and can improve its own design in manufacturing cycles instead of species cycles, might have important implications for every biological entity on Earth.”

Then you're not intelligent at cooking (haha!). Maybe my definition is better for "superintelligent" since it seems to imply boundless competence. I think humans are intelligent in that we can rapidly learn a surprising number of things (talk, walk, arithmetic)

The universe does not have a center but has a beginning in time and the beginning of space.

The distance to that beginning in time is approx 13 billion years. There is no approximation of distance to the beginning because the space is created at that point and continues to be created.

Imagine the Earth being on the surface of a sphere and so asking what is the center of the surface of a sphere? The sphere has a center but on the surface there is no center.

At least this is my understanding of how to approach these kind of questions.

No, you're missing what they said. True randomness can be delivered to a computer via a peripheral - an integrated circuit or some such device that can deliver true randomness is not that difficult.

https://en.wikipedia.org/wiki/Hardware_random_number_generat...

Minor nitpicks. I think your points are pretty good.

1. Self-rewiring is just a matter of hardware design. Neuromorphic hardware is a thing.

2. LLM foundation models are actually unsupervised in a way, since they simply take any arbitrary text and try to complete it. It's the instruction fine-tuning that is supervised. (Q/A pairs)

Coincidentally, there is no falsifiable/empirical test for understanding or intentionality.

Right now or you mean ever?

It's such a small leap to see how an artificial intelligence can/could become capable of understanding and have intentionality.

The simplified answer to that is some sort of a chemical gradient determined by gene expression in the cell. This is pretty much how all biological activity happens, like how limbs "know" to grow in a direction or how butterfly wings "know" to form the shape of a wing. Scientists are continuously uncovering more and more knowledge about various biological processes across life forms and there is nothing here to indicate it is anything but chemical signalling. I'm not a biologist so I won't be able to give explanations n-levels deep, but there is plenty information accessible to form an understanding of these processes in terms of physical and chemical laws. For how neurons connect, you can look up synaptogenesis and start from there.

I have no doubt that if you sample a sound at high enough fidelity that you won't hear a difference.

My comment around digital vs analog is more of an analogy around producing sounds rather than playing back samples though.

There's a Masterclass with Joel Zimmerman (DeadMau5) where he explains the stepping effect when it comes to his music production. Perhaps he just needs a software upgrade, but there was a lesson where he showed the stepping effect which was audibly noticeable when comparing digital vs analog equipment.

At least for listening purposes, there's no difference between 44.1 KHz/16-bit sampling and anything above that. It's all the same to the human ear.

Because that path lies skill atrophy.

AI research has a thing called "the bitter lesson" - which is that the only thing that works is search and learning. Domain-specific knowledge inserted by the researcher tends to look good in benchmarks but compromise the performance of the system[0].

The bitter-er lesson is that this also applies to humans. The reason why humans still outperform AI on lots of intelligence tasks is because humans are doing lots and lots of search and learning, repeatedly, across billions of people. And have been doing so for thousands of years. The only uses of AI that benefit humans are ones that allow you to do more search or more learning.

The human equivalent of "inserting domain-specific knowledge into an AI system" is cultural knowledge, cliches, cargo-cult science, and cheating. Copying other people's work only helps you, long-term, if you're able to build off of that into something new; and lots of discoveries have come about from someone just taking a second look at what had been considered to be generally "known". If you are just "taking shortcuts", then you learn nothing.

[0] I would also argue that the current LLM training regime is still domain-specific knowledge, we've just widened the domain to "the entire Internet".

I was pointing out that if you spent 2 weeks trying to find the solution but AI solved it within a day (you don’t specify how long the final solution by AI took), it sounds like those two weeks were not spent very well.

I would be interested in knowing what in those two weeks you couldn’t figure out, but AI could.

Because as far as you know, the "rough implementation" only works in the happy path and there are really bad edge cases that you won't catch until they bite you, and then you won't even know where to look.

An open source project wouldn't have those issues (someone at least understands all the code, and most edge cases have likely been ironed out) plus then you get maintenance updates for free.

Who hired you and why are they paying you money?

I don't want to be a hater, but holy moley, that sounds like the absolute laziest possible way to solve things. Do you have training, skills, knowledge?

This is an HN comment thread and all, but you're doing yourself no favors. Software professionals should offer their employers some due diligence and deliver working solutions that at least they understand.

So you could stick your own copyright notice on the result, for one thing.

Dunno, in work we wanted to implement a task runner that we could use to periodically queue tasks through a web UI - it would then spin up resources on AWS and track the progress and archive the results.

We looked at the existing solutions, and concluded that customizing them to meet all our requirements would be a giant effort.

Meanwhile I fed the requirement doc into Claude Sonnet, and with about 3 days of prompting and debugging we had a bespoke solution that did exactly what we needed.

Write me when humans will achieve hallucination levels smaller than ChatGPT.

It is unsurprising that some lossily-compressed-database search programs might be worse for some tasks than other lossily-compressed-database search programs.

That doesn't mean the one what manages to spit it out of its latent space is close to AGI. I wonder how consistently that specific model could. If you tried 10 LLMs maybe all 10 of them could have spit out the answer 1 out of 10 times. Correct problem retrieval by one LLM and failure by the others isn't a great argument for near-AGI. But LLMs will be useful in limited domains for a long time.

is this because the LLM actually reasoned on a better design or because it found a better design in its "database" scoured from another tenured engineer.

WTF, my calculator is high school was already a step towards AGI.

0 and 0.0001 may be difficult to distinguish.

How many people do you know who are capable of doing something truly novel? Definitely not me, I'm just an average phd doing average research.

Isn't the human brain also "just" a big statistical model as far as we know? (very loosely speaking)

> What the hell is general intelligence anyway?

OpenAI used to define it as "a highly autonomous system that outperforms humans at most economically valuable work."

Now they used a Level 1-5 scale: https://briansolis.com/2024/08/ainsights-openai-defines-five...

So we can say AGI is "AI that can do the work of Organizations":

> These “Organizations” can manage and execute all functions of a business, surpassing traditional human-based operations in terms of efficiency and productivity. This stage represents the pinnacle of AI development, where AI can autonomously run complex organizational structures.

The way some people confidently assert that we will never create AGI, I am convinced the term essentially means "machine with a soul" to them. It reeks of religiosity.

I guess if we exclude those, then it just means the computer is really good at doing the kind of things which humans do by thinking. Or maybe it's when the computer is better at it than humans and merely being as good as the average human isn't enough (implying that average humans don't have natural general intelligence? Seems weird.)

>you'd never be able to know for sure that thats what you've done.

Words mean what they're defined to mean. Talking about "general intelligence" without a clear definition is just woo, muddy thinking that achieves nothing. A fundamental tenet of the scientific method is that only testable claims are meaningful claims.

There's been a complaint for several decades that "AI can never succeed" - because when, say, expert systems are developed from AI research, and they become capable of doing useful things, then the nay-sayer say "That's not AI, that's just expert systems".

This is somewhat defensible, because what the non-AI-researcher means by AI - which may be AGI - is something more than expert systems by themselves can deliver. It is possible that "real AI" will be the combination of multiple approaches, but so far all the reductionist approaches (that expert systems, say, are all that it takes to be an AI) have proven to be inadequate compared to what the expectations are.

The GP may have been riffing off of this "that's not AI" issue that goes way back.

The people who go around saying "LLMs aren't intelligent" while refusing to define exactly what they mean by intelligence (and hence not making a meaningful/testable claim) add nothing to the conversation.

What has your roadmap to do with the capabilities?

LLMs still hallucinate and make simple mistakes.

And the progress seams to be in the benchmarks only

https://news.ycombinator.com/item?id=43603453

> and I'm maybe 1% of the way through the roadmap I'd like to build with what exists and is possible to do today.

How do you know they are possible to do today? Errors gets much worse at scale, especially when systems starts to depend on each other, so it is hard to say what can be automated and not.

Like if you have a process A->B, automating A might be fine as long as a human does B and vice versa, but automating both could not be.

100% this. The rearrangement of internal operations has only started and there is just sooo much to do.

It doesn't understand anything, there is no understanding going on in these models. It takes input and generates output based on the statistical math created from its training set. It's Bayesian statistics and vector/matrix math. There is no cogitation or actual understanding.

Understand? It fails with to understand a rephrasing of a math problem a five year old can solve... They get much better at training to the test from memory the bigger they get. Likewise you can get some emergent properties out of them.

Really it does not understand a thing, sadly. It can barely analyze language and spew out a matching response chain.

To actually understand something, it must be capable of breaking it down into constituent parts, synthesizing a solution and then phrasing the solution correctly while explaining the steps it took.

And that's not even what huge 62B LLM with the notepad chain of thought (like o3, GPT-4.1 or Claude 3.7) can really properly do.

Further, it has to be able to operate on sub-token level. Say, what happens if I run together truncated version of words or sentences? Even a chimpanzee can handle that. (in sign language)

It cannot do true multimodal IO either. You cannot ask it to respond with at least two matching syllables per word and two pictures of syllables per word, in addition to letters. This is a task a 4 year old can do.

Prediction alone is not indicative of understanding. Pasting together answers like lego is also not indicative of understanding. (Afterwards ask it how it felt about the task. And to spot and explain some patterns in a picture of clouds.)

I really disagree. I had a masseuse tell me how he uses ChatGPT, told it a ton of info about himself, and now he uses it for personalized nutrition recommendations. I was in Atlanta over the weekend recently, at a random brunch spot, and overheard some _very_ not SV/tech folks talk about how they use it everyday. Their user growth rate shows this -- you don't hit hundreds of millions of people and have them all be HN/SV info-bubble folks.

> idk AI is just a speck outside of the HN and SV info-bubbles

> still early to mass adoption like the smartphone or the internet, mostly nerds playing w it

Rather: outside of the HN and SV bubbles, the A"I"s and the fact how one can fall for this kind of hype and dupery is commonly ridiculed.

That doesn’t match what I hear from teachers, academics, or the librarians complaining that they are regularly getting requests for things which don’t exist. Everyone I know who’s been hiring has mentioned spammy applications with telltale LLM droppings, too.

ChatGPT has 400M weekly users. https://backlinko.com/chatgpt-stats