New Rules Default to Blocking Bots, Offer Pay-Per-Crawl Option. An internet infrastructure firm that routes 16% of global web traffic will block AI bots from new domains it hosts unless it has explicit permission from the owner. Cloudflare on Tuesday launched pay per crawl, a platform that allows publishers to charge AI crawlers each time they access a site.
First seen on govinfosecurity.com
Jump to article: www.govinfosecurity.com/cloudflare-aims-to-make-ai-bots-pay-for-crawling-websites-a-28873
