Prefill vs Decode: The Two Halves of Inference

Why reading a prompt and writing an answer are different computations on different hardware bottlenecks, derived from arithmetic intensity.