My crawler said 1 in 7 websites were down. They weren't, and retrying made it worse.

I'm building Fresh Leads, a tool that finds local businesses and grades their websites so agencies...