ScraperBlocker blocks AI web bots from grabbing your content

AI bots use a technique called scraping to programatically read your website's pages and then add your content to their large language models (LLMs), the giant data sets used to generate responses to search queries or chat bot prompts. ScraperBlocker stops that.

ScraperBlocker

Why is scraping bad?

Web scraping is how search engines work. Originally, you told the search engine you wanted your site indexed (scraped) & placed into their databases. Today, web sites are crawled by searchbots & indexed if found, even without explicit permission. This model of 'scrape first, ask never' is what's led us to today.

Is scraping theft?

Taking someone's property without permission is theft. Scraping does exactly that, but its legality is mired in the concept of Fair Use.

Access isn't permission

As a website operator, you want visitors to find you. AI companies take advantage of the web's open access to scrape without permission.

What about copyright?

AI business models depend on scraping copyrighted material to generate results. Copyrighted text & source code is visible in the AI's results.

Scraping enriches others

AI companies use scraped content to generate revenue from chatbots & other services. If you haven't licensed your content to them, you get nothing.

Professional-grade features protect you. Everything is standards-based & done by the rules.

ScraperBlocker Features

Powerful features protect your web content from scraping.

Blocks AI bots

ScraperBlocker blocks all major AI scraping bots. We add more as we find them.

Web standards

We return standard HTTP response codes to all requests. Nothing we do breaks a page request.

You're in control

In order to use ScraperBlocker, a site's owner must add it to pages. Only customers who want to use this will have it.

Protection by domain

We protect pages based upon domain. Add ScraperBlocker to any page in your domain to protect that page.

Lightweight implementation

We use just two lines of code in a page to block AI bots. Processing a request takes under a second.

Protect single pages

If you have some content that you're OK with being scraped by AI bots, don't install ScraperBlocker in those pages.

Zero maintenance

ScraperBlocker requires no maintenance from you. Install it & forget it. We'll update it from here.

Works everywhere

ScraperBlocker works in any web page on the internet. If you need protection in your cloud provider, ask about our Enterprise Plan.

Sign up & start using ScraperBlocker

Choose a plan & start blocking AI scrapers.

ScraperBlocker paid plans are monthly subscriptions that require a valid credit card. Choose the plan that's right for you & begin blocking AI bot scrapers in under five minutes. You may cancel your subscription at any time.

Plans

Individual

$0/mo

  • One domain protected
  • Unlimited pages protected
  • Protection from browser-type scrapers
  • Requires installation in each page to protect
  • Email support response within 24 hours
  • Eligible for some service upgrades

Operator

$4/mo

  • Two domains protected
  • Unlimted pages protected
  • Protection from browser & command-line scrapers
  • Requires web server or CDN distribution install
  • Email support response within 12 hours
  • Eligible for most service upgrades

Enterprise

Contact us

  • Unlimited domains protected
  • Unlimited pages protected
  • Protection from browser & command-line scrapers
  • Requires web server or CDN distribution install
  • Email support response within 4 hours
  • Receive all service upgrades

Installing ScraperBlocker

Setup is as easy as adding two lines of code to each page of your web site & creating a No Scraping Allowed page to return to scraper bots. Once ScraperBlocker is on your pages, virtually all AI scraping is blocked. Attempts to retrieve your protected pages by an AI bot returns your No Scraping Allowed page instead.

Choose a plan & register above

Signup using the Plans section
Add ScraperBlocker to your pages

Insert this line into the <head> of each page
<script src="https://scraperblocker.com/sb.min.js"></script>

Invoke ScraperBlocker on page load

Add scraperBlocker's function to the <body> tag & run it onload
<body onload="scraperBlocker();">

Add the No Scraping Allowed page to your site

FAQ

These are the questions we received most frequently as we shared ScraperBlocker with our early adopters. If you have questions not answered here, email us at jbminn@jbminn.com

How does ScraperBlocker work?

ScraperBlocker requires two lines of javascript in each web page that you wish to protect from bots. You install these lines yourself, probably in the template you use to generate pages. The javascript intercepts each page request to determine its origin. If it's from an AI bot, the request to grab that page is blocked & a No Scraping Allowed page is returned instead.

Can ScraperBlocker block bots that don't process javascript?

ScraperBlocker Individual can't block curl, a command line Url retriever, as curl can't process our javascript in a web page. ScraperBlocker Operator & Enterprise can block curl scrapers when installed at your web server or in your cloud service provider's content distribution.

Which bots get blocked?

ScraperBlocker blocks AI web scraper bots that identity themselves. Most of the well-known companies that scrape appear to properly identify themselves, so they are blocked. If an unscruplous scraper misidentifies itself, we will miss that bot.

Does ScraperBlocker work with my cloud services provider?

Yes. We can protect AWS Cloudfront distributions as well as pages hosted in Google Cloud Platform. The implementations are more complex than that for individual pages, so you'll need an Enterprise Plan for this requirement.

Does ScraperBlocker remove my content from AI Large Language Models (LLMs)?

No. ScraperBlocker prevents the AI bots from grabbing your content after you install it. If an AI bot has already scraped your site, ScraperBlocker cannot remove your content from the LLM.

Who operates ScraperBlocker?

ScraperBlocker is a service of JBMINN, the technology services company run by John Minnihan. John invented hosted source control & ran Freepository, the first hosted source control service, for 17 years. Freepository securely managed over 3 billion lines of code for more than 400 thousand developers across the globe.

Is this legal?

Yes. Web site operators implement technical solutions allowing & disallowing access to their site every day. A newspaper web site paywall is an example of this. If you have concerns about using ScraperBlocker in your locale, consult with your legal representative.