I checked what 31 big websites tell AI crawlers in robots.txt, and found a bug in my own tool

Can ChatGPT, Claude or Perplexity read your website? The first place to look is a small text file:...