How do you train a PyTorch model on multiple GPUs?

Use DDP when the model fits on one GPU, FSDP when it doesn't, and launch with torchrun. Code, memory math and a cost check for going multi-node.