I Trained a Language Model From Scratch on a 4 GB GPU

A 4 GB GPU, a small corpus and a lot of things that went wrong in instructive ways. I trained a...