Skip to content(if available)orjump to list(if available)

How to Scale Your Model: How to Think About GPUs

How to Scale Your Model: How to Think About GPUs

3 comments

·August 19, 2025
Visit

nickysielicki

The calculation under “Quiz 2: GPU nodes“ is incorrect, to the best of my knowledge. There aren’t enough ports for each GPU and/or for each switch (less the crossbar connections) to fully realize the 450GB/s that’s theoretically possible, which is why 3.2TB/s of internode bandwidth is what’s offered on all of the major cloud providers and the reference systems. If it was 3.6TB/s, this would produce internode bottlenecks in any distributed ring workload.

Shamelessly: I’m open to work if anyone is hiring.

null

[deleted]

aanet

Fantastic resource! Thanks for posting it here.