DDP on a PCIe-only 4x RTX A6000 (no NVLink): plain DDP gave 2x, bf16 gradient compression plus accumulation gave 3.6x

Disclosure up front: I work on Runcrate, where these ran. Four GPUs should train about four times faster than one. On a rented 4x RTX A6000 machine this morning (2026-10-26), plain PyTorch DDP gave me 2x. This is the notebook of finding out why, and the two flags that took it to 3.6x. Setup Machine: one Runcrate 4x RTX A6000 (48 GB each) VM, US Midwest, $2.64/hour for the whole machine at launch time. Two separate machines of the same type over the morning: the first for the baseline, the sec...