Cloudflare has introduced a new feature allowing customers to block AI bots, scrapers, and crawlers with a single click. This move comes in response to the increasing demand for content used to train AI models and run inference. This post discusses the prevalence of AI bot activity, including both well-behaved and evasive bots, and Cloudflare's efforts to detect and block them using advanced machine learning models.

Key points:

• One-click option to block all AI bots

• Top AI bots by request volume: Bytespider, Amazonbot, ClaudeBot, and GPTBot

• Only 2.98% of top 1 million websites block AI bot requests

• Cloudflare's machine learning models can detect evasive AI bots

• Customers can report misbehaving AI crawlers through feedback tools