Batches, and Stopping Without Losing Anything

A batch is one executor, one pass, iterations strictly in order - and the boundary where the commit happens. Cut the work into batches and you can drop it at any boundary without reconstructing anything from memory, which matters because the executor has no memory at all.