WordPress.org

Pulaar

Get WordPress
WordPress.org

Plugin Directory

AI Scrape Protect

Download

Description

AI Scrape Protect is a WordPress plugin designed to protect your website from scraping for AI training purposes. It achieves this by adding opt-out instructions to the robots.txt file for the most common AI scraping bots and including meta tags to control how your content is used.

Note: These instructions are not always respected by all bots.

Features

  • Adds specific User-agent and Disallow rules to your robots.txt file to block a comprehensive list of AI scraping bots.
  • Introduces meta tags in the HTML <head> to provide additional instructions to AI bots, including new tags for Bingbot and general AI compliance.
  • Prepares for future standards with support for DisallowAITraining and noimageai meta tags.
  • Dedicated handling of specific bots like CCBot and Bingbot for better protection and compatibility.

License

This plugin is licensed under the GNU General Public License v2 or later. See the GPL v2 license for details.

Author Information

This plugin is developed by Uisce Web Development, specifically by Daan Verbaan. For support or inquiries, please visit our website or contact us directly.

Screenshots

  • robots.txt File Example: Shows how the plugin updates the robots.txt file.
  • meta tags Example: Shows an example of the Meta Tags added to the head section.

Installation

  1. Upload the ai-scrape-protect folder to the /wp-content/plugins/ directory.
  2. Activate the plugin through the ‘Plugins’ menu in WordPress.

FAQ

How does this plugin protect my site from AI scraping?

The plugin adds specific User-agent entries to your robots.txt file to instruct common AI scraping bots not to crawl or scrape your site. It also introduces meta tags in the HTML <head> to provide additional instructions to AI bots.

Will this completely stop AI scraping of my site?

While this plugin adds recommendations to the robots.txt file and includes meta tags, not all bots follow these rules. This is a measure to discourage scraping rather than a foolproof solution.

Can I add or remove bots from the list?

Currently, the plugin includes a predefined list of bots. If you need to add or remove specific bots, you would need to modify the plugin code or contact the plugin author for customization.

What happens if I deactivate the plugin?

The robots.txt file will revert to its previous state before the plugin was activated, and the meta tags added to the HTML <head> will be removed.

Reviews

There are no reviews for this plugin.

Contributors & Developers

“AI Scrape Protect” is open source software. The following people have contributed to this plugin.

Contributors

Translate “AI Scrape Protect” into your language.

Interested in development?

Browse the code, check out the SVN repository, or subscribe to the development log by RSS.

Changelog

3.0

  • Added admin bar icon functionality to indicate plugin activity.
  • Updated meta tags for compliance with official documentation and improved AI scraping protection:
    • Adjusted Bingbot tag to use nocache for better compatibility with Bing AI Chat.
    • Removed “noindex” from the ai-bot meta tag to allow search engine indexing.
    • Added DisallowAITraining and noimageai to the robots meta tag.
    • Introduced dedicated meta tags for CCBot.
  • Removed OpenAI SearchBot from the robots.txt blocklist based on OpenAI’s recommendations.

2.4

  • Updated meta tags for improved AI scraping protection.
    • Prevent Bingbot and general AI bots from using the content for AI purposes.
    • Added clear comments in the code to describe the functionality of each meta tag.

2.3

  • Added the following new AI bots to the block list: Ai2Bot-Dolma, cohere-training-data-crawler, DuckAssistBot, img2dataset, magpie-crawler, PanguBot

2.2

  • Optimized robots.txt output by consolidating multiple Disallow rules into a single Disallow: / statement.
  • Added new AI bots to the block list: StableDiffusionBot, sentibot, Grok, GrokAI, XAI, XBot.
  • Enhanced code structure for better readability and maintainability.

2.1

  • Updated Features section in readme.txt to include new functionality for meta tags in HTML <head>.
  • Updated Frequently Asked Questions section in readme.txt to include new functionality for meta tags in HTML <head>.

2.0

  • Added AI bots: AI2Bot, AmazonBot, Applebot-Extended, ClaudeBot, OAI-SearchBot, PetalBot, webzio.
  • Introduced meta tags for AI bot protection in HTML <head>.

Meta

Ratings

No reviews have been submitted yet.

Add my review

See all reviews

Support

Got something to say? Need help?

View support forum

Donate

Would you like to support the advancement of this plugin?

Donate to this plugin