Alert after 1 failed check or 3? Tuning uptime alerts so on-call stops ignoring them

Alert too fast and you get 3 AM false alarms. Alert too slow and users find the outage first. A practical way to set retries, confirmation and thresholds.