Chapterly

Self-hosted manager for the web novels I follow: add a novel by its table-of-contents URL, a worker checks it daily and fetches only new chapters, and the EPUB is regenerated into the Audiobookshelf library folder (its folder watcher picks it up). Scraping reuses WebToEpub's 640+ site parsers, run headless in Node (jsdom), instead of writing our own.

Project page: marvello.xyz/chapterly

Layout

shared/     db.js (the only module with SQL) + migrations/ — used by both worker and web
worker/     scraper + scheduler + CLI (cli.js, src/, test/, vendor-WebToEpub/)
web/        Next.js UI (reads the DB, records intents; never scrapes)

Setup

npm run setup       # worker/setup.sh (clones WebToEpub into worker/vendor-WebToEpub/, npm install) + web install
npm test            # worker tests (synthetic freewebnovel-shaped site, incl. full pipeline) + web tests

Re-run worker/setup.sh to pull upstream parser fixes. CLI commands below run from worker/ (cd worker && node cli.js …) or in Docker (docker compose exec worker node cli.js …).

Run with Docker

cp .env.example .env && sed -i '' "s|^AUTH_SECRET=.*|AUTH_SECRET=$(openssl rand -base64 32)|" .env
docker compose up -d --build                                  # worker + web UI on http://localhost:3000
docker compose exec worker node cli.js user:create [email protected] "Your Name"   # prompts for the password
docker compose logs -f worker web
docker compose build --no-cache && docker compose up -d       # pull WebToEpub parser fixes

The web UI never scrapes: add / check now / retry / interval / pause / delete are written to the DB and the worker picks them up within CHAPTERLY_TICK_MIN (1 min). The SQLite DB lives in the chapterly-data Docker volume (shared by both containers) and the EPUBs in ./library. Override with CHAPTERLY_DATA_DIR, CHAPTERLY_LIBRARY_DIR (point this at the Audiobookshelf library) and PUID/PGID (the user that owns that library, default 1000) in .env. Don't open the DB from the host while the containers run on Docker Desktop (SQLite WAL across the Mac/VM boundary can corrupt reads); use docker compose exec worker node cli.js list instead. On a Linux host a host path for CHAPTERLY_DATA_DIR is fine. Change the password with cli.js user:password <email> (logs out every session). The web port listens on all host interfaces (see Deploy). Every CLI command below also works via docker compose exec worker node cli.js ….

Reader (phone)

The library page is also the reader: tap a novel to read it. Use Chrome's "Install app" to put it on the home screen; it opens offline from what's saved on the phone (adding novels, check status and the novel's management page need the server and are hidden while it can't be reached). Chapters come straight from the DB. Progress is saved per novel and only moves forward on its own: opening an earlier chapter shows "Your progress is at …" with Set progress here / Go to it. The current chapter plus the next 50 unread of every novel you're reading are kept on the phone for offline reading; "Download unread" on a novel keeps all of them. EPUB building for Audiobookshelf can be switched off per novel (novel page, or node cli.js epub <id> on|off).

Login: password and/or OIDC

Controlled by env (see .env.example). With no AUTH_OIDC_* set, only the password form is shown. Set all three AUTH_OIDC_* (issuer, client id, secret) to add a "Sign in with <AUTH_OIDC_NAME>" button (password stays as a fallback unless AUTH_PASSWORD_LOGIN=false). Partial OIDC config, or AUTH_PASSWORD_LOGIN=false without OIDC, stops the web container at startup. Provider setup:

1. Create a confidential OAuth2/OpenID client with redirect URI https://<your host>/api/auth/callback/oidc, RS256 signing, scopes openid email profile. The provider must return email_verified: true for your user (the app refuses unverified emails).
2. Use the provider's issuer URL for AUTH_OIDC_ISSUER, and restrict the client to your own user/group.
3. First sign-in links the OIDC identity to the account with the same email; afterwards only that identity is accepted. cli.js user:unlink-oidc <email> resets the link.

Password sign-in is rate-limited (10 attempts/min) per client IP, plus a per-account lockout. The app only learns the client IP from Cloudflare's cf-connecting-ip header, and only trusts it with TRUST_CF_HEADER=1 — set that only behind a Cloudflare tunnel/proxy that is the sole way in. Without it (e.g. plain compose, no proxy) all clients share one rate-limit bucket; the account lockout still applies. X-Forwarded-For is never trusted. Sessions expire after 7 days without use.

Deploy on a server

Keep a git checkout of this repo on the server, next to its .env (never committed). To update, push from your machine, then pull and rebuild on the server:

git push origin main
ssh <server> 'cd <chapterly dir> && git pull --ff-only && docker compose up -d --build'

In the server's .env, point CHAPTERLY_LIBRARY_DIR at the Audiobookshelf library, PUID/PGID at its owner, and optionally set CHAPTERLY_DATA_DIR (a host path), CHAPTERLY_WEB_PORT and AUTH_URL (the URL you open the UI at). CLI commands: docker exec -it chapterly-worker node cli.js …. The web port listens on all of the host's interfaces, so it's reachable over the LAN or a private VPN. Keep it behind a firewall and never port-forward it to the internet; for public access put an authenticating tunnel / reverse proxy in front, with AUTH_URL set to that https URL. For localhost only, bind it as 127.0.0.1:${CHAPTERLY_WEB_PORT:-3000}:3000 in compose.yaml.

This is the beginning of the README. Read the rest at the original. Read the full README ↗