AI scrapers are taking over the internet. That’s where your AI model gets its knowledge from; it sticks its AI claws into any information it can find and copies the data.
People are fed up and fighting back. There are more tools than ever dedicated to feeding AI crawlers garbage instead of blocking them outright, with the idea that with enough rubbish, you can poison the AI’s output.
The reality is somewhere in between. As you’re probably aware, AI models are working just fine — but it’s causing an AI poisoning arms race that’s closer to you than you realize.
“Poisoning” sounds powerful, but it’s not that simple
There are many types of bad medicine
There are two main types of AI poisoning, and they work differently. You’ve probably already heard about the first type, content poisoning, as it has featured in the news on and off since the AI boom began.
Content poisoning involves adding a layer of corruption to actual work before anyone scrapes it, or just straight-up serving corrupted content to the model in the hope it damages it. One of the most famous poisoning tools, Nightshade, bakes invisible pixel noise into artwork so an image model learns the wrong style from it.
The second is known as “tarpitting,” which doesn’t touch your content. Instead, tarpitting is considered a defensive countermeasure against AI scraping, luring AI bots into a never-ending maze of randomly generated trash content designed to waste time and resources.
Tarpits sit between an AI crawler and your website or service, watching out for incoming AI bots, using known triggers to spot them. That could be a poorly configured browser user-agent, use patterns that no human could ever produce, and IP ranges known to send out AI crawlers to harvest data.
As you might expect, there are quite a few different tools designed to do this, given how many AI models sweep the web daily.
Nepenthes
One of the most well-known tarpits, Nepenthes, takes its name from a carnivorous plant that eats anything that falls into its clutches; in an interview with Ars Technica, designer “Aaron B” warns would-be users that it’s basically a type of malware for AI.
The tool lures bots into an endless maze of nonsense pages that link to more nonsense pages, so a crawler that wanders in just keeps clicking deeper into a dead end, feeding it more data as it goes. You can see what Nepenthes “looks like” on its demo page, though the creator warns that the page loads very slowly (pictured above if you don’t want to waste your time… like I did!).
Iocaine
Another tool, Iocaine, works as a reverse proxy that sits in front of your existing site and gives AI scraper bots a unique poisoned link the first time it’s intercepted. When it comes back, Iocaine uses a type of programmable logic to reidentify the bot, catching those repeat scrapers.
Like Nepenthes, the Iocaine site makes it clear this is considered malicious, and anyone using it should be comfortable before switching it on.
Cloudflare AI Labyrinth
Not every tarpit comes from an indie dev with a grudge, though. Cloudflare built the same idea into an actual product, AI Labyrinth, and it’s already rolled out with backing from major publishers including AP, Condé Nast, and Gannett.
Rather than blocking a bot it’s caught red-handed, which can tip off whoever’s running it that they’ve been rumbled, Cloudflare lures it into a maze of realistic, AI-generated pages instead, quietly burning its compute rather than announcing the block.
As an added benefit, AI Labyrinth also acts as a next-generation honeypot. No real human would go four links deep into a maze of AI-generated nonsense. Any visitor that does is very likely to be a bot, so this gives us a brand-new tool to identify and fingerprint bad bots, which we add to our list of known bad actors.
It’s a similar trick to Nepenthes, underneath. The difference is it’s a toggle in a dashboard rather than something you self-host and configure yourself, which is probably why it’s spread to far more sites, far faster, than any of the indie tools.
Cloudflare being one of the largest tech companies in the world also helps to add support to the idea that poisoning and trapping AI scrapers is a valid form of defense (or is the old “attack is the best form of defense?” adage at play?).
Creators and platforms aren’t always protecting the same things
Tarpitting and content poisoning target different things
While these two AI scraping defense techniques seem similar in outcome, their use is different, and mixing them up gives you very different outcomes.
If you poison content directly, the AI scraping defense is carried with the file. If it’s mirrored, reposted, scraped, stolen, or whatever, the poisoning protection travels with the file. However, in that, poisoning individual files isn’t enormously effective, because it’s limited to the files you specifically poison.
Whereas tarpitting techniques typically focus on an entire server or service, providing poisoning protection to everything hosted there. It removes the requirement to poison every file and offers broader protection — but when a file leaves the tarpitted server, it’s basically open season.
Interestingly, Cloudflare’s AI Labyrinth somewhat splits down the middle. It focuses on infrastructure-grade tarpitting, which basically puts it in the same category as Nepenthes. But Cloudflare packages Labyrinth as a toggle anyone can flip without touching a server config file.
So, does AI poisoning actually work or is it all a show?
The biggest problem: finding proof
Poisoning an AI model sounds like fighting a noble fight. Protecting content against AI scraping and the general hoovering up of all known data was important when the AI boom began. There are many reasons to mistrust and dislike generative AI, and content theft ranks high on the list.
But the problem with AI scraper poisoning is that proving it makes any difference at all is nearly impossible. Most major AI companies don’t make comments about their approaches to poisoning; OpenAI was the only respondent to Ars Technica’s request to comment on this topic.
“We’re aware of efforts to disrupt AI web crawlers,” OpenAI’s spokesperson said. “We design our systems to be resilient while respecting robots.txt and standard web practices.”
Other companies have gone the other way. Cloudflare accused Perplexity of ignoring robots.txt files to crawl sites either way, and presented plenty of evidence along with it.
The wider point is that, even with OpenAI’s admission, we don’t know how effective AI poisoning techniques are. At scale, poisoning likely makes a huge difference, degrading model quality. That’s before you get to the next point: AI companies actively clean data before adding it to the model. Scraped data isn’t just pipelined directly into an AI model for this exact reason; it’s not like throwing another trash bag into an already overflowing dumpster.
Furthermore, legally, tarpitting is a grey area. Site owners generally decide what happens on their servers, but that’s a norm, not a ruling, and one good test case (legal case, not a general test) could shift it either way.