Ask HN: How to learn CUDA to professional level

45 comments

·June 8, 2025

Hi all, I was wondering what books/courses/projects one might do to learn CUDA programming.

(To be frank, the main reason is a lot of companies I'd wish to work for require CUDA experience -- this shouldn't change your answers hopefully, just wanted to provide some context )

Visit

indianmouse

As a very early CUDA programmer who participated in the cudacontest from NVidia during 2008 and I believe one of the only entries (I'm not claiming though) to be submitted from India and got a consolation and participation prize of a BlackEdition Card, I can vouch the method which I followed.

- Look up the CUDA Programming Guide from NVidia

- CUDA Programming books from NVidia from developer.nvidia.com/cuda-books-archive link

- Start creating small programs based on the existing implementations (A strong C implementation knowledge is required. So, brush up if needed.)

- Install the required Toolchains, compilers, and I am assuming you have the necessary hardware to play around

- Github links with CUDA projects. Read the code, And now you could use LLM to explain the code in the way you would need

- Start creating smaller, yet parallel programs etc., etc.,

And in about a month or two, you should have enough to start writing CUDA programs.

I'm not aware of the skill / experience levels you have, but whatever it might be, there are plenty of sources and resources available now than it was in 2007/08.

Create a 6-8 weeks of study plan and you should be flying soon!

Hope it helps.

Feel free to comment and I can share whatever I could to guide.

hiq

> I am assuming you have the necessary hardware to play around

Can you expand on that? Is it enough to have an nvidia graphic card that's like 5 year old, or do you need something more specific?

rahimnathwani

I'm not a CUDA programmer, but AIUI:

- you will want to install the latest version of CUDA Toolkit (12.9.1)

- each version of CUDA Toolkit requires the card driver to be above a certain version (e.g. toolkit depends on driver version 576 or above)

- older cards often have recent drivers, e.g. the current version of CUDA Toolkit will work with a GTX 1080, as it has a recent (576.x) driver

fifilura

I am not a CUDA programmer but when looking at this, I think I can see the parallels to Spark and SQL

https://gfxcourses.stanford.edu/cs149/fall24/lecture/datapar...

So - start getting used to programming without using for loops, would be my tip.

throwaway81523

I looked at the CUDA code for Leela Chess Zero and found it pretty understandable, though that was back when Leela used a DCNN instead of transformers. DCNN's are fairly simple and are explained in fast.ai videos that I watched a few years ago, so navigating the Leela code wasn't too difficult. Transformers are more complicated and I want to bone up on them, but I haven't managed to spend any time understanding them.

CUDA itself is just a minor departure from C++, so the language itself is no big deal if you've used C++ before. But, if you're trying to get hired programming CUDA, what that really means is they want you implementing AI stuff (unless it's game dev). AI programming is a much wider and deeper subject than CUDA itself, so be ready to spend a bunch of time studying and hacking to come up to speed in that. But if you do, you will be in high demand. As mentioned, the fast.ai videos are a great introduction.

In the case of games, that means 3D graphics which these days is another rabbit hole. I knew a bit about this back in the day, but it is fantastically more sophisticated now and I don't have any idea where to even start.

upmind

This is a great idea! This is the code right' https://github.com/leela-zero/leela-zero

I have two beginner (and probably very dumb) questions, why do they have heavy c++/cuda usage rather than using only pytorch/tensorflow. Are they too slow for training Leela? Second, why is there tensorflow code?

henrikf

That's Leela Zero (plays Go instead of Chess). It was good for its time (~2018) but it's quite outdated now. It also uses OpenCL instead of Cuda. I wrote a lot of that code including Winograd convolution routines.

Leela Chess Zero (https://github.com/LeelaChessZero/lc0) has much more optimized Cuda backend targeting modern GPU architectures and it's written by much more knowledgeable people than me. That would be a much better source to learn.

throwaway81523

As I remember, the CUDA code was about 3x faster than the tensorflow code. The tensorflow stuff is there for non-Nvidia GPU's. This was in the era of the GTX 1080 or 2080. No idea about now.

upmind

Ah I see, thanks a lot!

imjonse

These should keep you busy for months:

https://www.gpumode.com/ resources and discord community Book: Programming massively parallel processors nvidia cuda docs are very comprehensive too https://github.com/srush/GPU-Puzzles

amelius

This follows a "winner takes all" scenario. I see the differences between the submissions are not so large, often smaller than 1%. Kind of pointless to work on this, if you ask me.

lokimedes

There’s a couple of “concerns” you may separate to make this a bit more tractable:

1. Learning CUDA - the framework, libraries and high-layer wrappers. This is something that changes with times and trends.

2. Learning high-performance computing approaches. While a GPU and the Nvlink interfaces are Nvidia specific, working in a massively-parallel distributed computing environment is a general branch of knowledge that is translatable across HPC architectures.

3. Application specifics. If your thing is Transformers, you may just as well start from Torch, Tensorflow, etc. and rely on the current high-level abstractions, to inspire your learning down to the fundamentals.

I’m no longer active in any of the above, so I can’t be more specific, but if you want to master CUDA, I would say learning how massive-parallel programming works, is the foundation that may translate into transferable skills.

rramadass

This is the right approach. Without (2) trying to learn (1) will just lead to "confusion worse confounded". I also suggest a book recommendation here - https://news.ycombinator.com/item?id=44216478

jonas21

I think it depends on your learning style. For me, learning something with a concrete implementation and code that you can play around with is a lot easier than trying to study the abstract general concepts first. Once you have some experience with the code, you start asking why things are done a certain way, and that naturally leads to the more general concepts.

lokimedes

This one was my go-to for HPC, but it may be a bit dated by now: https://www.amazon.com/Introduction-Performance-Computing-Sc...

rramadass

That's a good book too (i have it) but more general than the Ridgway Scott book which uses examples from Numerical Computation domains. Here is an overview of the chapters; example domains start from chapter 10 onwards - https://www.jstor.org/stable/j.ctv1ddcxfs

These sort of books are only "dated" when it comes to specific languages/frameworks/libraries. The methods/techniques are evergreen and often conceptually better explained in these older books.

For recent up to date works on HPC, the free multi-volume The Art of High Performance Computing by Victor Eijkhout can't be beat - https://news.ycombinator.com/item?id=38815334

elashri

I will give you personal experience learning CUDA that might be helpful.

Disclaime: I don't claim that this is actually a systematic way to learn it and it is more for academic work.

I got assigned to a project that needs learning CUDA as part of my PhD. There was no one in my research group who have any experience or know CUDA. I started with standard NVIDIA courses (Getting Started with Accelerated Computing with CUDA C/C++ and there is python version too).

This gave me good introduction to the concepts and basic ideas but I think after that I did most of learning by trial and error. I tried a couple of online tutorials for specific things and some books but it was always a deprecated function there or here or a change of API that make things obsolete. Or basically things changed for your GPU and now you have to be careful because yoy might be using GPU version not compatible with what I develop for in production and you need things to work for both.

I think learning CUDA for me is an endeavor of pain and going through "compute-sanitizer" and Nsight because you will find that most of your time will go into debugging why things is running slower than you think.

Take things slowly. Take a simple project that you know how to do without CUDA then port it to CUDA ane benchmark against CPU and try to optimize different aspect of it.

The one advice that can be helpful is not to think about optimization at the beginning. Start with correct, then optimize. A working slow kernel beats a fast kernel that corrupts memory.

korbip

I can share a similar PhD story (the result being visible here: https://github.com/NX-AI/flashrnn). Back then I didn't find any tutorials that cover anything beyond the basics (which are still important). Once you have understood the principle working mode and architecture of a GPU, I would recommend the following workflow: 1. First create an environment so that you can actually test your kernels against baselines written in a higher-level language. 2. If you don't have an urgent project already, try to improve/re-implement existing problems (MatMul being the first example). Don't get caught by wanting to implement all size cases. Take an example just to learn a certain functionality, rather than solving the whole problem if it's just about learning. 3. Write the functionality you want to have in increasing complexity. Write loops first, then parallelize these loops over the grid. Use global memory first, then put things into shared memory and registers. Use plain matrix multiplication first, then use mma (TensorCore) primitives to speed things up. 4. Iterate over the CUDA C Programming Guide. It covers all (most) of the functionality that you want to learn - but can't be just read an memorized. When you apply it you learn it. 5. Might depend on you use-case but also consider using higher-level abstractions like CUTLASS or ThunderKitten. Also, if your environment is jax/torch, use triton first before going to CUDA level.

Overall, it will be some pain for sure. And to master it including PTX etc. will take a lot of time.

kevmo314

> I think learning CUDA for me is an endeavor of pain and going through "compute-sanitizer" and Nsight because you will find that most of your time will go into debugging why things is running slower than you think.

This is so true it hurts.

sputknick

I used this to teach high school students. Probably not sufficient to get what you want, but it should get you off the ground and you can run from there. https://youtu.be/86FAWCzIe_4?si=buqdqREWASNPbMQy

SoftTalker

It's 2025. Get with the times, ask Claude to do it, and then ask it to explain it to you as if you're an engineer who needs to convince a hiring manager that you understand it.

ForgotIdAgain

I have not tried it yet, but seems nice : https://leetgpu.com/

tkuraku

I think you just pick a problem you want to solve with gpu programming and go for it. Learning what you need along the way. Nvidia blog posts are great for learning things along the way such as https://devblogs.nvidia.com/cuda-pro-tip-write-flexible-kern...

mekpro

To professionals in the field, I have a question: what jobs, positions, and companies are in need of CUDA engineers? My current understanding is that while many companies use CUDA's by-products (like PyTorch), direct CUDA development seems less prevalent. I'm therefore seeking to identify more companies and roles that heavily rely on CUDA.

kloop

My team uses it for geospatial data. We rasterize slippy map tiles and then do a raster summary on the gpu.

It's a weird case, but the pixels can be processed independently for most of it, so it works pretty well. Then the rows can be summarized in parallel and rolled up at the end. The copy onto the gpu is our current bottleneck however.

math_dandy

Are there any GPU emulators you can use to run simple CUDA programs on a commodity laptops, just to get comfortable with the mechanics, the toolchain, etc.?

corysama

https://leetgpu.com/ emulates running simple CUDA programs in a web page with zero setup. It’s a good way to get your toes wet.

gkbrk

Commodity laptops can just use regular non-emulated CUDA if they have an Nvidia GPU. It's not just for datacenter GPUs, a ton of regular consumer GPUs are also supported.