Patreon stops asking AI bots not to scrape — and starts blocking them

Patreon finally stopped asking. For years the platform treated AI scraping like a noise complaint — file a robots.txt, hope the bots listen, move on. That era ended Thursday when the company announced it is now actively blocking training crawlers through Cloudflare's AI Crawl Control. The polite request didn't work. Patreon's own tests showed individual AI bots making thousands of weekly attempts to scrape the site despite the robots.txt file explicitly telling them not to. Once the block went live, those attempts dropped to zero. The honor system was a fiction.

The shift matters because Patreon's content was never fully open. A paywall has long locked most creator work behind subscriber gates. But the platform's own growth strategy undercut that protection. New discovery features — a redesigned Home Feed and the tweet-like Quips — pushed more content into public view where crawlers could reach it. Patreon built the very surface area that scrapers exploited. Now it's walling that surface off.

Cloudflare's tooling makes the blockade technological rather than legal. The infrastructure provider launched a marketplace called Pay Per Crawl that lets sites charge bots for access, and it changed policy this month so that mixed-use crawlers — those that both index and train — are blocked by default on ad-supported pages. Patreon is extending its existing Cloudflare partnership to enforce a cleaner distinction: bots that organize information and send users back to the platform stay welcome. Bots that ingest work to train models without permission get stopped at the gate.

"Consent shouldn't depend on whether a scraper chooses to behave," Patreon's blog post states. That line cuts to the core. The entire architecture of web crawling has rested on voluntary compliance. Robots.txt is a gentlemen's agreement in a room full of pickpockets. Patreon's move acknowledges what every publisher knows: the scrapers with the most to gain are the least likely to ask. Active blocking replaces hope with enforcement.

Product chief Drew Rowny framed it as a creator-power play. "As AI agents become increasingly powerful and popular, creators deserve a meaningful say in how their work is used by AI companies. On most of the Internet, creators have to accept AI training on their work just to reach and grow an audience. Patreon has a different vision: creators should be able to grow their audience and control how their work is used." The rhetoric is sharp. The execution will be watched.

The compromise on indexing bots is pragmatic. Search visibility still matters. Patreon needs Google and Bing and the rest to surface creator pages. It needs social preview bots to generate link cards. Those crawlers provide a two-way value exchange: traffic for access. Training bots offer no return. They vacuum content, memorize patterns, and sell the resulting model back to the world — often in direct competition with the creators who produced the training data. Blocking them while allowing indexers draws a commercially defensible line.

Other platforms will follow or they won't. Reddit struck a licensing deal with Google. The New York Times sued OpenAI. X (formerly Twitter) rate-limited aggressively and then launched its own AI. Patreon's approach is distinct: it uses infrastructure-level blocking to enforce a policy choice without negotiating case by case. Cloudflare becomes the enforcement layer for any site that adopts its tools. That scales.

The risk is fragmentation. If every platform deploys different blocking rules, the web becomes a patchwork of semi-permeable membranes. AI companies will build better evasion — residential proxies, browser fingerprinting, distributed crawling. The arms race is inevitable. But Patreon's stance resets the default from open to closed. That shift alone changes the economics. Scrapers now need permission, not just stealth.

Creators on Patreon didn't ask for this fight. They built audiences on a platform that promised a direct relationship with supporters. AI training breaks that relationship by inserting an invisible intermediary that learns from the work and competes with the maker. Patreon's blockade restores the original compact: the platform protects the creator's control. The robots.txt experiment proved the internet won't respect that control voluntarily. Now the platform enforces it.