What PCA is actually doing to your data

Most explanations of PCA stop at "it reduces dimensions." That is true, and it is roughly as useful...