Scrapfly Anti-bot Detector
See how a website defends itself against bots: which anti-bot system, which captcha, which fingerprinting scripts, and how sure each match is.
Add to Chrome · Product page · What's new · llms.txt · Report a problem
English · 简体中文 · Español · Русский · 日本語 · Português (Brasil) · Deutsch · Français · 한국어
Scrapfly Anti-bot Detector is a free Chrome extension that shows how a website protects itself from automated traffic. Browse as you normally would: the number on the toolbar icon tells you how many protections the page uses, and the popup lists each one with its confidence, its difficulty and the signals that gave it away.
It recognizes 94 technologies out of the box: 49 anti-bot systems, 24 CAPTCHAs and 21 browser fingerprinting techniques. Every detector is a plain JSON file that you can read, change or extend, and new rules ship with each release of this repository, so Rules → Update can install them before Chrome updates the extension itself.
We built it at Scrapfly because every scraping project starts with the same question: what is protecting this page? It is just as useful for checking how your own site's bot protection looks from the outside, or for security research.
Contents
Screenshots
Click an image to see it full size.
Install
Chrome Web Store
The simplest option: install it from the Chrome Web Store. Chrome keeps it up to date for you.
From a release
Each version on the Releases page has a ready-to-load zip attached.
- Download the zip and extract it into a new folder.
- Open
chrome://extensions and turn on Developer mode (top right).
- Click Load unpacked and choose that folder.
From source
There is no build step. Clone the repository and load the folder as it is:
git clone https://github.com/scrapfly/Antibot-Detector.git
Then follow steps 2 and 3 above, choosing the cloned folder. After editing a file, press the reload button on the extension's card in chrome://extensions and reload the page you are testing.
How it works
When a page loads, the extension gathers what the page reveals about itself and checks it against every enabled detector. Each detector combines up to eight methods:
| Method | What it looks at | Example signal |
|---|
| URL | The page address, script sources and every network request | challenges.cloudflare.com/turnstile/v0/api.js |
| Cookies | Cookies the site sets or sends, HttpOnly ones included | _abck (Akamai), datadome |
| Headers | Response and request headers | x-kpsdk-ct (Kasada) |
| Content | The page's HTML and the code of its inline and loaded scripts | KPSDK.configure( |
| DOM | Elements matched by a CSS selector | .cf-turnstile[data-sitekey] |
| Window properties | Objects that scripts add to window | grecaptcha.render |
| JavaScript hooks | Calls to browser APIs commonly used for fingerprinting | HTMLCanvasElement.prototype.toDataURL |
| Payload | Bodies of POST, PUT and PATCH requests, optionally filtered by URL and method, and the responses of the URLs a rule names | sensor_data (Akamai), "proofofwork":{"required":true (OpenAI Sentinel) |
Confidence. Each rule has its own confidence and detectors define combinations of rules that score higher together. A script that only mentions a vendor's name scores 5–10%; a signal only that vendor's product produces (its own script host, its own cookie, its challenge endpoint) is proof and scores 100% on its own, and so does a vendor's complete integration (its script together with its cookie, widget frame or challenge request). Shared or generic signals stay at 15–90%. The score shown for each detector is its strongest single rule or combination. The page also gets a difficulty rating (Low, Medium or High), which sets the color of the toolbar badge. Only the anti-bot and CAPTCHA protections found at 50% or more set it: fingerprinting and a script that only mentions a vendor do not.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
