REDbot

REDbot is lint for HTTP resources.

It checks HTTP resources for feature support and common protocol problems at the HTTP semantic and caching layers. You can use the public instance on https://redbot.org/, or you can install it locally.

Contributing to REDbot

Your ideas, questions and other contributions are most welcome. See CONTRIBUTING.md for details.

Setting Up Your Own REDbot

Installation

REDbot requires a current version of Python.

The recommended method for installing REDbot is using pipx. To install the latest release, do:

pipx install redbot

Or, to use the most development version of REDbot, run:

pipx install git+https://github.com/mnot/redbot.git

Both of these methods will install the following programs into your pipx binary folder:

• redbot - the command-line interface
• redbot_daemon - Web interface as a standalone daemon

By default, redbot prints a text report for the URL. Use -o to choose another format: text_verbose adds an explanation of each note, and har produces an HTTP Archive (HAR) file. With -a (--assets), text and text_verbose also list the resources linked from an HTML response. The web interface accepts the same names in its format query argument (format=text, format=text_verbose, format=har).

Running REDbot as a systemd Service

REDbot can run as a standalone service, managed by systemd. This offers a degree of sandboxing and resource management, process monitoring (including a watchdog function), and a timer that garbage-collects saved tests.

See extra/README.md for the sample units and step-by-step setup.

Running REDbot in a Container

OCI-compliant containers are available on Github, and it's easy to run REDbot one using a tool like Docker or Podman. For example:

docker run --rm -p 8000:8000 ghcr.io/mnot/redbot

or

podman run --rm -p 8000:8000 ghcr.io/mnot/redbot

The daemon runs as an unprivileged redbot user (UID 999). Saving tests is off by default. To enable it, set save_dir = /var/lib/redbot in a config file that you mount into the container and pass as the command, and mount a named volume at /var/lib/redbot to keep the saved tests. The image provides that directory, owned by the daemon's user; a bind-mounted host directory must be writable by UID 999.

Command line

The redbot command checks one URL: redbot [-a] [-o FORMAT] [-t SECONDS] URL. -a also checks the assets a page links to, and -t (default 60; 0 for no limit) stops the check after that many seconds and shows what was collected.

Web Bot Auth

REDbot can authenticate its outgoing requests using Web Bot Auth, which signs requests with an Ed25519 key using HTTP Message Signatures (RFC 9421). This lets origins verify that requests genuinely come from your REDbot instance. The implementation follows the IETF drafts and is not specific to any one verifier.

Requests are sent unsigned by default. REDbot only attaches a signature when the origin challenges for one -- that is, when it returns a 401, 403, or 429 response carrying an Accept-Signature header -- and then transparently retries the request signed. This avoids advertising the bot's identity to servers that don't ask for it.

Generating a key

Create an Ed25519 private key in PEM form:

> openssl genpkey -algorithm ed25519 -out web-bot-auth-key.pem

Keep this file private and readable by the REDbot process. The matching public key is published automatically (see below); you do not need to extract it yourself.

Configuring

Set these in config.txt (for redbot_daemon):

web_bot_auth_key = /path/to/web-bot-auth-key.pem
web_bot_auth_directory = https://your-redbot.example

web_bot_auth_directory is the HTTPS origin where your public key directory is hosted. When you run redbot_daemon, it serves that directory at /.well-known/http-message-signatures-directory (a JWKS, self-signed per the spec), so a verifier can fetch your public key. Make sure that path is reachable at the origin you configured.

web_bot_auth_validity (seconds, 1 to 86400, default 300) sets how long a signature stays valid, and web_bot_auth_key_created (a Unix timestamp) is published as the key's nbf if you set it. REDbot warns at startup if the key file is readable by other users. Only the scheme, host and port of web_bot_auth_directory are used; any path is ignored.

web_bot_auth_directory is optional: if you omit it, REDbot uses the origin of ui_uri. Because ui_uri defaults to https://redbot.org/, enabling signing requires ui_uri to be set to your instance's real public origin — otherwise you would publish the wrong bot identity. Set web_bot_auth_directory explicitly if your key directory lives on a different origin than the UI.

For the command-line tool, pass the equivalent flags (there is no ui_uri, so the directory is required):

> redbot --web-bot-auth-key web-bot-auth-key.pem --web-bot-auth-directory https://your-redbot.example https://example.com/

Registering with a verifier

To be recognised by a specific verifier, register your directory URL with them according to their process.

Credits

Icons by Font Awesome. REDbot includes code from tippy.js and prettify.js.