John Carmack talk at Upper Bound 2025

> If DOOM released in 1994 or 1995, would we still remember it in the same way?

Maybe. One aspect of Wolfenstein and Doom's popularity is that it was years ahead of everyone else technically on PC hardware. The other aspect is that they were genre defining titles that set the standards for gameplay design. I think Doom Deathmatch would have caught on in 1995, as there really were very few (just Command and Conquer?) standout PC network multiplayer games released between 1993 and 1995.

> So Wolfenstein and Doom were actually not really 3d games, but rather 2.5 games (you can't have rooms below other rooms). The first 3d game here is actually Quake

Ultima Underworld is a true 3D game from 1992. An incredibly impressive game, in more ways than one.

The world seems to have rewritten history, and forgotten Ultima Underworld, which shipped prior to Doom..

Hardware changes a lot in the time it takes to develop a game. When I read his plan files and interviews, I realized he seemed to spend a lot of time before developing the game thinking about what the next gen hardware was going to bring. Then design the best game they could think of whike targeting this not-yet-available hardware.

If DOOM released in 1994 or 1995, would we still remember it in the same way?

I think so, because the thing about DOOM is, it was an insanely good game. Yes, it pioneered fullscreen real-time perspective rendering on commodity hardware, instantly realigning the direction of much of the game industry, yadda yadda yadda, but at the end of the day it was a good-enough game for people to remember and respect even without considering the tech.

Minecraft would be a similar example. Minecraft looked like total ass, and games with similar rendering technology could have been (and were) made years earlier, but Minecraft was also good. And that was enough.

But also, he didn't do the technically hardest and most impressive part, Quake, on his own. IIUC he basically relied on Michael Abrash's help to get Quake done (in any reasonable amount of time).

A key aspect of the Carmack approach (or similar 'smart hacker' unconventional career approach) is avoiding that situation in the first place. However, this also carries substantial career, financial and lifestyle risks & trade-offs - especially if you're not both talented enough and lucky enough to hit a sufficiently fertile oppty in the right time window on the first few tries.

Assuming one is willing to accept the risks and has the requisite high-talent plus strong work drive, the Carmack-like career pattern is to devote great care to evaluating and selecting opptys near the edges of newly emerging 'interesting things' which also: coincide with your interests/talents, are still at a point where a small team can plausibly generate meaningful traction, and have plausible potential to grow quickly and get big.

Carmack was fortunate that his strong interest in graphics and games overlapped a time period when Moore's Law was enabling quite capable CPU, RAM and GFX hardware to hit consumer prices. But we shouldn't dismiss Carmack's success as "luck". That kind of luck is an ever-present uncontrolled variable which must be factored into your approach - not ignored. Since Carmack has since shown he can get very interested in a variety of things, I assume he filtered his strong interests to pick the one with the most near-term growth potential which also matched his skills. I suspect the most fortunate "luck" Carmack had wasn't picking game graphics in the early 90s, it was that (for whatever reasons) he wasn't already employed in a more typical "well-paying job with a big, stable company, great benefits and career growth potential" so he was free to find the oppty in the first place.

I had a similarly unconventional career path which, fortunately, turned out very well for me (although not quite at Carmack's scale :-)). The best luck I had actually looked like 'bad luck' to me and everyone else. Due to my inability to succeed in a traditional educational context (and other personal shortcomings), I didn't have a college degree or resume sufficient to get a "good job", so I had little choice but to take the high-risk road and figure out the unconventional approach as best I could - which involved teaching myself, then hiring myself (because no one else would) and then repeatedly failing my way through learning startup entrepreneurship until I got good at it. I think the reality is that few who succeed on the 'unconventional approach' consciously chose that path at the beginning over lower risk, more comfortable alternatives - we simply never had those alternatives to 'bravely' reject in pursuit of our dreams :-).

  > "makes me wonder how he would fare as an unknown Jr. developer with managers telling him (...)"

Saying we don't "need" AGI is like saying we don't need electricity. Sure life existed before we had that capability, but it would be very transformative. Of course we can make specialized tools in the mean time.

What if AGI is just a bunch of specialized AIs put together?

It would seem our own generalized intelligence is an emergent property of many, _many_ specialized processes

I wonder if AI is the same

Specialized AIs have been making an impact on society since at least the 1960s. AI has long suffered from every time they come up with something new it gets renamed and becomes important (where it makes sense) without giving AI credit.

From what I can tell most in AI are currently hoping LLMs reach that point quick just because the hype is not helping AI at all.

I think to many AI enthusiasts, we're already at the "specialized AIs" phase. The question is whether those will jump to AGI. I'm personally unconvinced but I'm not an ML researcher so my opinion is colored by what I use and what I read, not active research. I do think though that many specialized AIs is already enough to experience massive economic disruption.

Why not just hire like 100 of the smartest people across domains and give them SOTA AI, to keep the AI as accurate as possible?

Each of those 100 can hire teams or colleagues to make their domain better, so there’s always human expertise keeping the model updated.

The Wikipedia article on AGI explains it well enough.

Researchers at Google have proposed a classification scheme with multiple levels of AGI. There are different opinions in the research community.

https://arxiv.org/abs/2311.02462

In a way it sort of doesn't matter. If all we ever wind up with is unbelievably fancy chatbots, marketing teams will eventually draw their line in the sand and call it AGI.

There comes a theoretical point at which a definition is no longer relevant because it's obvious to everyone on an intuitive level. An easy lower bound for where this threshold might sit would be "when it can start and win wars unassisted under its own volition". Since, at that point no one on earth would have a need to debate it. It would simply be respected and understood for what it is without needing to define it.

Until such an obvious threshold is crossed, it will be whatever executives, product managers, and marketing teams say it is.

It doesn't really seem like there's much utility in defining it. It's like defining "heaven."

It's an ideal that some people believe in, and we're perpetually marching towards it

It is a marketing term. That's it. Trying to exhaustively define what AGI is or could be is like trying to explain what a Happy Meal is. At it's core, the Happy Meal was not invented to revolutionize food eating. It puts an attractive label on some mediocre food, a title that exists for the purpose of advertisement.

There is no point collecting definitions for AGI, it was not conceived as a description for something novel or provably existent. It is "Happy Meal marketing" but aimed for adults.

No it is not a “gotcha” and I don’t understand how you got that impression.

Carmack believes AGI systems should be able to learn new tasks in realtime alongside humans in the real world.

Just like OpenAI early on promised us an AGI and showed us how it "solved" Dota 2.

They also claimed it "learned" to play by playing itself only however it was clear that most of the advanced techniques were borrowed from existing AI and by observing humans.

No surprise they gave up on that project completely and I doubt they'll ever engage in anything like that again.

Money better spent on different marketing platforms.

The point isn't about learning video games its about learning tasks unrelated to its specific competency generally.

A human would learn it faster, and could immediately teach other humans.

AI clearly isn't at human level and it's OK to admit it.

> But why would we want robots to use an interface (especially a modern controller) heavily optimized for human hands; that seems like the definition of a horseless carriage.

Elon's response to this is that if we want these androids to replace human jobs then the lowest friction alternative is for the android to be able to do anything a human can do in a human amount of space. A specialized machine is faster and more efficient, but comes with engineering and integration costs that create a barrier to entry. Elon learned this lesson the hard way when he was building out the gigafactories and ended up having to hire a lot of people to do the work while they sorted out the issues with the robots. To someone like Elon a payroll is an ever growing parasite on a companies bottom line, far better if the entire thing is automated.

No, the joystick part is really not the limiting factor. They’ve already done this with a direct software interface. Physical interface is a new challenge. But overall you are missing the point.

Yes but it remains innate to the individual.

I'm not arguing against computational theory of mind, I'm just saying that innate behaviours don't require the same level of scale as learnt ones.

Existing at birth is not the same thing as innate. Puberty is innate but it is not present at birth.

> There is just no reason to believe that we are born with some insanely big library of knowledge, and it sounds completely impossible. How would it be stored, and how would we even evolve it?

We do have that, ever felt fear of heights? That isn't learned, we are born with it. Same with fear of small moving objects like spiders or snakes.

Such things are learned/stored very different from memories, but its certainly there and we can see animals also have those. Like cats gets very scared of objects that are long and appear suddenly, like a cucumber, since their genetic instincts thinks its a snake.

>How would it be stored, and how would we even evolve it?

DNA and the same way anything else is evolved? The body is insanely complicated, I don't see why innate knowledge is so unbelievable.

You could implement a Turing-machine with humans acting physically operating as logic gates. Then, every human is just a boolean function.

This is exactly it. Biology is making massive use of hacked real time local network communication in ways we haven´t begun to explore.

I'm not sure what the moral is from this, but if Atari games are just too easy, at the same time the response of the machine-learning guys to the challenge of the NetHack Learning Environment seems to have mostly been to quietly give up. Why is generative modeling essential to finding harder challenges when NetHack is right there ...?

Right, and the current state of tech - from accounts I’ve read, though not first hand experienced - is the “black box” methods of AI are absolutely questionable when delivering citations and factual basis for their conclusions. As in, the most real world challenge, in the basic sense, of getting facts right is still a bridge too far for OpenAI, ChatGPT, Grok, et al.

See also: specious ethics regarding the training of LLMs on copyright protected artistic works, not paying anything to the creators, and pocketing investor money while trying to legislate their way around decency in engineering as a science.

Carmack has a solid track record as an engineer, innovator, and above the board actor in the tech community. I cannot say the same for the AI cohort and I believe such a distinction is important when gauging the validity of critique or self-aggrandizement by the latter, especially at the expense of the former. I am an outlier in this community because of this perspective, but as a creator and knowledgeable enough about tech to see things through this lens, I am fine being in this position. 10 years from now will be a great time to look back on AI the way we’re looking back at Carmack’s game changing contributions 30 years ago.

That sounds like an extremely useful insight that makes this kind of research even more valuable.

x.com/... -> xcancel.com/...

Thank you for clarifying. I don't have a Twitter account, and the linked tweet genuinely looks like a standalone object. Mea culpa.

I wouldn't bet against him. "The Bitter Lesson" may imply an advantage to someone who historically has been at the tip of the spear for squeezing the most juice out of GPU hosted parallel computation.

Graphics rendering and AI live on the same pyramid of technology. A pyramid with a lot of bricks with the initials "JC" carved into them, as it turns out.

Carmack is always a genius, but like most people he requires luck, and like most people, the house always wins. Poor Armadillo Aerospace.

What has "Graphics Carmack" actually done since about 2001?

So, his initial tech was "Adaptive tile refresh" in Commander Keen, used to give it console style pixel-level scrolling. Turns out, they actually hampered themselves in Commander Keen 1 by not understanding the actual tech, and implemented "The Jolt", a feature that was not necessary. The actual hardware implemented scrolling the same way that consoles like the NES did, and did not need "the jolt", nor the limitations it imposed.

Then, Doom and Quake was mostly him writing really good optimizations of existing, known and documented algorithms and 3D techniques, usually by recognizing what assumptions they could make, what portions of the algorithm didn't need to be recalculated when, etc. Very talented at the time, but in the software development industry, making a good implementation of existing algorithms that utilize your specific requirements is called doing your job. This is still the height of his relative technical output IMO.

Fast Inverse Square Root was not invented by him, but was floating around in industry for a while. He still gets kudos for knowing about it and using it.

"Carmack's reverse" is a technique for doing stencil shadows that was a minor (but extremely clever) modification to the "standard" documented way of doing shadow buffers. There is evidence of the actual technique from a decade before Carmack put it in Doom 3 and it was outright patented by two different people the year before. There is no evidence that Carmack "stole" or anything this technique, it was independent discovery, but was clearly also just a topic in the industry at the time.

"Megatextures" from Rage didn't really go anywhere.

Did Carmack actually contribute anything to VR rendering while at Oculus?

People treat him like this programming god and I just don't understand. He was well read, had a good (maybe too good) work ethic, and was very talented at writing 386 era assembly code. These are all laudable, but doesn't in my mind imply that he's some sort of 10X programmer who could revolutionize random industries that he isn't familiar with. 3D graphics math isn't exactly difficult.

Exactly. I know him and like him. He is a genius programmer for sure BUT people forget that the last successful product that he released was Doom 3 over 20 years ago. Armadillo was a failure and Oculus went nowhere.

He's also admitted he doesn't have much of math chops, which you need if you want to make a dent in AI. (Although the same could have been said of 3D graphics when he did Wolfenstein and Doom, so perhaps he'll surprise us)

I wish him well TBH

Ever since Romero departed the id Software had shipped *checks notes* Quake II, Quake III, Doom 3 and Quake 4.

Funnily enough Romero himself didn't ship much either. IMO it's one of the most iconic "duo breakups". The whole is greater than the sum of the parts.

> People often fall into the trap of thinking that because they are highly intelligent and an expert in one domain that this makes them an expert in one or more other domains.

While this is certainly true, I'm not aware of any evidence that Carmack thinks this way about himself. I think he's been successful enough that's he's personally 'post-economic' and is choosing to spend his time working on unsolved hard problems he thinks are extremely interesting and potentially tractable. In fact, he's actively sought out domain experts to work with him and accelerate his learning.

Bayesian reasoning isn't a fallacy. A known expert in one domain is often correct about things in a related one. The post didn't claim that Carmack is right, just that that he's who they would bet on to be right, which seems perfectly reasonable to me.

Expecting an expert in one thing to also be pretty good at other domains, especially when they're relatively related, isn't a fallacy.

> But it still feels like some key ingredient is missing.

Continuous training is the key ingredient. Humans can use existing knowledge and apply it to new scenarios, and so can most AI. But AI cannot permanently remember the result of its actions in the real world, and so its body of knowledge cannot expand.

Take a toddler and an oven. The toddler has no concept of what an oven is other than maybe that it smells nice. The toddler will touch the oven, notice that it experiences pain (because the oven is hot) and learn that oven = danger. Place a current AI in a droid toddler body? It will never learn and keep touching the oven as soon as the information of "oven = danger" is out of the context window.

For some cases this inability to learn is actually desirable. You don't want anyone and everyone to be able to train ChatGPT unsupervised, otherwise you get 4chan flooding it with offensive crap like they did to Tay [1], but for AI that physically interacts with the meatspace, constant evaluation and learning is all but mandatory if it is to safely interact with its surroundings. "Dumb" robots run regular calibration cycles for their limbs to make sure they are still aligned to compensate for random deviations, and so will AI robots.

[1] https://en.wikipedia.org/wiki/Tay_(chatbot)

I watched the talk live. I felt that his main argument was that Atari _looks_ solved, but there's still plenty of value that could be gained by revisiting these "solved" games. For one, learning how to play games through a physical interface is a way to start engaging with the kinds of problems that make robotics hard (e.g., latency). They're also a good environment to study catastrophic forgetting: an hour of training on one game shouldn't erase a model's ability to play other games.

I think we could eventually saturate Atari, but for now it looks like it's still a good source of problems that are just out of reach of current methods.

My impression is that Atari was 80% solved, and then researchers and companies moved on.

A company solves self-driving 80% of the way and makes a lot of VC cash along the way. Then they solve intelligent chatbots 80% of the way and make a lot of VC cash along the way. Now they're working on solving humanoid robotics 80% of the way... I wonder why?

In the end, we have technology that can do some neat tricks, but can't be relied upon.

There are probably still some very hard problems in certain Atari games. Only the brave dare tackle these problems, because failure comes sharp and fast. Whereas, throwing more compute at a bigger LLM might not really accomplish anything, but we can make people think it accomplished something, and thus failure is not really possible.

Being highly used in the past is good, it's a benchmark to compare against.

So why can't AI learn those and reapply the same understanding to new games?

I think he argues that they would not be able to play Atari games this way (I don't know; maybe I also misunderstood).

But also, he argues a lot about sample efficiency. He wants to develop algorithms/methods/models which can learn much faster / with much fewer data.

What deepmind accomplished with suicidal Mario was so much more than you probably ever will know from outside the company.

I think a big problem with AGI discussion is sometimes people aren’t being creative enough. A 100x smarter ChatGPT means biology immortality

and build cities ...

Name any other company acting like that besides OpenAI. Or any person besides Sam Altman, the guy who screwed up OpenAI's structure/board/funding. You just have a narrative you want to be true, and one company half fits into that narrative.

Is VC really funding a trillion dollars of GPU purchases to replace labor that could instead be bid out to developing world mechanical turks for $1/hr?

No the term for that is ASI...artifical super intelligence. People in AI community have different timelines for that than AGI.

Ask Gemini to explain how it was trained

https://g.co/gemini/share/f64c3358d9fa