Which AI Crawlers Do the Top 1,000 Websites Block? (robots.txt Study, October 2026)

We read the robots.txt files of the 1,000 most popular websites: 20.9% block at least one of 16 AI...