Why our multi-million row join cost $4,000 in one hour

You ship the job. It passes CI. The data quality checks return green. Then, at 3:14 AM, the PagerDuty...