Skip to content(if available)orjump to list(if available)

DualPipe: An innovative bidirectional pipeline parallelism algorithm

DualPipe: An innovative bidirectional pipeline parallelism algorithm

9 comments

·February 27, 2025
Visit

xnhbx

> DualPipe was created and developed by Jiashi Li and Chengqi Deng and Wenfeng Liang.

A CEO who codes.

danielhanchen

I attached all 3 algorithms 1F1B (1 forward 1 backward), ZB1P (zero bubble pipeline parallelism) and DualPipe as a picture here: https://x.com/danielhanchen/status/1894937006352031832 for those interested :)

Bimos

Maybe add Chimera as well?

https://arxiv.org/pdf/2107.06925

danielhanchen

Oh more nice pictures :)

alphan0n

Off topic, but this is the Rick and Morty episode where Rick creates a perfectly level space.

The symmetry is uuugh.

danielhanchen

You'll have to refresh my memory :) Is there like a Youtube clip for it?

Cyphase

https://www.youtube.com/watch?v=-MwCJpEuC44

puppycodes

Sorry for us utter simpletons can someone explain what it do?

snake_doc

Hmm weren’t there also supposed to be the SM re-allocation, doesn’t look like it was included; I may have been mis-remembering the explanation.