Attention Through Arithmetic Intensity

Reduce MHA to its arithmetic intensity and you get exactly 1. GQA, MQA and MLA come out just as clean -- head counts and nothing else. That last constant is what puts MLA at odds with MTP.