Skip to content(if available)orjump to list(if available)

Andrej Karpathy: Deep Dive into LLMs Like ChatGPT [video]

Andrej Karpathy: Deep Dive into LLMs Like ChatGPT [video]

15 comments

·February 5, 2025
Visit

ipsum2

He's made more than 5 videos covering basically the same topic, of transformer architecture and training. Wonder whats different about this one?

karpathy

My YouTube videos fall into two tracks:

1. technical track (all the GPT repro series) 2. general audience track

For (2), I had a 1hr video from 1 year ago, but I didn't actually expect that video to be some kind of authoritative introduction to LLMs. The history is that I was invited to give an LLM talk (to general audience), prepared some random slides for a day, gave the talk, and then re-recorded the talk in my hotel room later in a single take, and that become the video. It was quite random and haphazard. So I wanted to loop back around more formally and do a more comprehensive intro to LLMs for general audience; Something I could for example give to my parents, or a friend who uses ChatGPT all the time and is interested in it, but doesn't have the technical background to go through my videos in (1). That's this video.

gdjdkslslp

I haven’t watched this video yet, but do you plan to create any technical videos in the (1) series on RL in LLMs?

amelius

This would be very welcome as it brings us closer to understanding the secret sauce behind training a real, practical LLM.

whiplash451

3h+ is very long

Have you considered making a 1h version of it?

ks2048

From the description:

    I have one "Intro to LLMs" video already from ~year ago, but that is just a re-recording of a random talk, so I wanted to loop around and do a lot more comprehensive version.
I think he has videos on building GPT2 from scratch, but this seems more high-level.

Dinux

Thanks Andrej. I have a pretty good understanding of how LLMs work and how they are trained, but alot of my friends don't. These videos/talks give them 'some' idea.

tmp111111

Andrej, I like you much more now than when you were at Tesla. You have been adding real value to my life and many others. Thank you.

sota_pop

Really love his “let’s build” series - I end up picking up cool Python tricks along the process, even in addition to the higher level content.

brizii

[flagged]

demarq

I wish there were another way to distribute video. Content disappears from youtube eventually, for silly reasons.

I think this is important content, the more people know how ai works under the hood the more empowered society will be.

Dinux

YouTube is known for not deleting videos, and so far the never have (with some obvious exceptions)

IncreasePosts

You can just make a torrent of the video. It will then survive as long as you/others are willing to seed it.

layer8

If accessible knowledge about how LLMs work ever disappears, it won’t be due to YouTube.

m_ppp

Do you think videos disappearing is the biggest problem with YouTube from a distribution perspective?