Why Meta trained a small 8B model on 15 trillion tokens, and why that's cheaper than it sounds

There's a rule of thumb from a famous paper: for the best result per dollar of training, feed a model...