A (Long) Peek into Reinforcement Learning
17 comments
·March 23, 2025farhanhubble
ultrasounder
I found this in the Authors GitHub repo. Looks interesting. Will use this to complement my learning resources https://github.com/farhanhubble/discover-drl/blob/master/Red...
farhanhubble
If you're interested, this is the course that I created: https://github.com/KnowchowHQ/rl-in-action/tree/master/C1-RL...
The other one you found is just one of the notebooks from the course.
ultrasounder
Thanks a lot! Looks like a lot of fun. Like I said will combine it with my Stanford online course
antman
Is anyone aware of a course that is beyond the basics and is focused on modern algorithms?
calebkaiser
It sort of depends on which direction you want to go in. If you're interested in deep RL as applied specifically to LLMs, I'd second another commenter's recommendation of Spinning Up from OpenAI. It hasn't been updated for a while and is a little outdated, but it provides a really nice introduction to some key ideas like PPO: https://spinningup.openai.com/en/latest/user/introduction.ht...
If you want to get more of a bird's eye view of modern deep RL in general, then HuggingFace's course is a good place to start. The course itself is kind of "all over the place"—not necessarily a bad thing, just maybe not what you want if you're looking to go super deep on a single topic. But if you want to get a look at some robotics stuff, deep Q-learning, multi-agent stuff, etc., it'll give you a nice sort of "tasting menu." As with most HuggingFace stuff, the format is really nice, it does a good job of introducing you to key ideas/projects in the ecosystem, and it has some cool project components: https://huggingface.co/learn/deep-rl-course/en/unit0/introdu...
avandekleut
I used SpinningUp as one of my main sources for my thesis. I also wrote some downloadable-and-runnable blog posts here: https://avandekleut.github.io/ppo/
joelthelion
This one looks pretty good: https://arxiv.org/pdf/2412.05265v2, although I've only skimmed it so far.
null
fancyfredbot
Very nice! I also recommend the CS885 course from university of Waterloo which is all on YouTube. It covers Bayesian reinforcement learning which is missing from most courses (including this one).
wiesbadener
Here a link to the lecture playlist: https://www.youtube.com/playlist?list=PLdAoL1zKcqTXFJniO3Tqq...
jxjnskkzxxhx
If you want a (long) peek at RL read Sutton and barto. Amazing book.
yamrzou
(2018)
Has RL changed since then?
fancyfredbot
This page covers the basics which are pretty timeless. You'd want to understand this before learning about any more recent advances.
It doesn't get into state of the art algorithms, for example proximal policy optimisation isn't mentioned although the paper on this was published in 2017 and is probably considered the best algorithm for at least some applications.
amelius
Curious what people here have built using RL?
dcrimp
A mate of mine built a works scheduler using RL + MCTS. It was interesting seeing the scheduler get smarter as they added in reward for real life constraints. For example, certain types of work couldn't happen on a tuesday - they add that in to the reward calculation, retrain, it now avoids Tuesdays. Build up that reward calculation based on available data, and it got to be super capable at making a workable schedule. Also orders of magnitude faster than linear solvers (albeit without guarantee of "optimality").
phyalow
Highly profitable trading systems.
Lillian's writing is really crisp without compromising the technical details.
I learnt reinforcement learning from Udacity course. It was one of the finest on the internet. Whenever I couldn't find some thing in the course I went to this website. I created my own small introductory course for RL, a pure hands-on experience for learners.