torrent-indexer
This is a simple torrent indexer that can be used to index torrents from HTML pages. It is written in Golang and uses Redis as a cache.
Test it
Visit https://torrent-indexer.darklyn.org/ to test it.
Supported sites
Deploy
If you have Docker + docker-compose installed, you can deploy it using the following command:
git clone https://github.com/module195/torrent-indexer.git
cd torrent-indexer
MEILI_MASTER_KEY=uma-chave-segura docker compose up -d --build
The reference docker-compose.yml builds this fork and starts Redis, FlareSolverr and Meilisearch, with RESPONSE_CACHE_TTL=5m and REQUEST_COALESCE=true enabled (the settings guindex benefits from). Remove the services you don't need.
The server will be available at http://localhost:8080/.
Configuration
You can configure the server using the following environment variables:
PORT: (optional) The port that the server will listen to. Default: 7006METRICS_PORT: (optional) The port that the metrics server will listen to. Default: 8081LOG_LEVEL: (optional) The log level. Default: 1 (info).
- Possible values:
0 (debug), 1 (info), 2 (warn), 3 (error)
LOG_FORMAT: (optional) The log format. Can be "json" or default to console logger.FLARESOLVERR_ADDRESS: (optional) The address of the FlareSolverr instance. Default: N/AFLARESOLVERR_TIMEOUT_SECONDS: (optional) Timeout for flaresolverr requests. Default: 30FLARESOLVERR_POOL_SIZE: (optional) Number of FlareSolverr sessions in the pool. Default: 5FLARESOLVERR_HEALTHCHECK_SECONDS: (optional) Interval to verify pooled FlareSolverr sessions and recreate the ones lost after a FlareSolverr restart. Default: N/A (disabled)FLARESOLVERR_SESSIONS: (optional) true/false forces whether the solver supports sessions.*. By default it is detected from GET /: FlareSolverr uses a pool of sessions; Byparr (which only speaks request.get) gets FLARESOLVERR_POOL_SIZE concurrency slots and is never sent sessions.*. Default: N/A (auto)REQUEST_COALESCE: (optional) true makes concurrent requests for the same page URL share a single fetch. Default: falseRESPONSE_CACHE_TTL: (optional) Caches indexer JSON responses (keyed by indexer + full query; a response with 0 results is kept for at most 2 minutes, and only when the site listed torrents that were all filtered out) and coalesces identical concurrent requests; the X-Cache header becomes HIT|MISS|SHARED (it is BYPASS while disabled). Duration format, e.g. 5m. Default: N/A (disabled)RESPONSE_CACHE_TTL_<NAME>: (optional) Per-indexer override of RESPONSE_CACHE_TTL, where NAME is the indexer key in uppercase with underscores (e.g. RESPONSE_CACHE_TTL_BETOR=30m, RESPONSE_CACHE_TTL_TORRENT_DOS_FILMES=2m). 0 turns the response cache off for that indexer. Default: (uses )
Extra Configuration
FALLBACK_TITLE_ENABLED: (optional) Enable the fallback title post-processor that sets the title to "[UNSAFE] {original_title}" (Page title) if the title is empty. Default: false
- This is useful for sites that do not have a title for some torrents, but can lead to misleading titles.
MAGNET_METADATA_API_ENABLED: (optional) Enable the magnet metadata API. (deploy instrucitons here) Default: falseMAGNET_METADATA_API_ADDRESS: (optional) The address of your magnet metadata API. Default: N/AMAGNET_METADATA_API_TIMEOUT_SECONDS: (optional) The timeout for the magnet metadata API requests in seconds. Default: 10INDEXER_<NAME>_URL: (optional) Set a custom URL for the indexer. Where the "NAME" will be always uppercase indexer key with underscores. ex: INDEXER_DODO_FILMES_URL=https://my-proxied-dodo-url.org
API for aggregating clients (e.g. guindex)
These additions are optional: existing routes and their JSON are unchanged.
-
X-Cache header on every /indexers/<name> route: HIT, MISS or SHARED when served through RESPONSE_CACHE_TTL (or by torrentfilmeshds' own cache), BYPASS otherwise.
-
Cache bypass: add nocache=1 to an /indexers/<name> (or /indexers/multi) request, or send Cache-Control: no-cache, to skip the cached response and run the indexer again; the fresh result replaces the cached one. nocache is not part of the cache key. It skips only the response cache, not the page cache of SHORT_LIVED_CACHE_EXPIRATION, and torrentfilmeshds keeps its own cache.
-
GET /sources?v=2: the same payload as /sources (including indexer_names) plus:
api_version: 2indexers: one entry per active indexer with name, supports_text, supports_imdb, tier (primary, secondary or reserve, see /indexers/multi), reserve_for (reserves only: the primary it mirrors), healthy (false after 3 consecutive failed runs: HTTP 5xx/403/429; true again 5 minutes after the last failure so clients retry it, doubling on every failed retry up to 1 hour; always true during the first 2 minutes after start), retry_after_s (only while unhealthy: seconds until it is offered again), consecutive_failures, last_failure_at (RFC 3339, absent until a failure), p50_ms (median of the last 50 successful runs; absent until one) and samples. Stats live in memory and reset on restart; cached replays (X-Cache: HIT) are not counted.search: {"meilisearch": true|false} (configured and answering; probed at most once a minute)features: {"multi": true, "x_cache": true, "response_cache": true|false}
An older server ignores v=2 and answers the plain payload, so clients can check api_version and fall back.
-
GET /indexers/multi?q=a&q=b&indexers=x,y&deadline_ms=8000: runs every indexer x query pair on the server (same handlers, cache and stats as ) and answers when all finish or at the deadline, whichever comes first.
Integrating with Jackett
You can integrate this indexer with Jackett by adding a new Torznab custom indexer. Here is an example of how to do it for the bludv indexer:
---
id: bludv_indexer
name: BluDV Indexer
description: "BluDV - Custom indexer on from torrent-indexer"
language: pt-BR
type: public
encoding: UTF-8
links:
- http://localhost:8080/
caps:
categorymappings:
- { id: Movie, cat: Movies, desc: "Movies" }
- { id: TV, cat: TV, desc: "TV" }
modes:
search: [q]
tv-search: [q, season, ep]
movie-search: [q]
allowrawsearch: true
settings: []
search:
paths:
- path: "indexers/bludv?filter_results=true&q={{ .Keywords }}"
response:
type: json
keywordsfilters:
- name: tolower
rows:
selector: $.results
count:
selector: $.count
fields:
_id:
selector: title
download:
selector: magnet_link
title:
selector: title
description:
selector: original_title
details:
selector: details
infohash:
selector: info_hash
date:
selector: date
size:
selector: size
seeders:
selector: seed_count
leechers:
selector: leech_count
imdb:
selector: imdb
category_is_tv_show:
selector: title
filters:
- name: regexp
args: "\\b(S\\d+(?:E\\d+)?)\\b"
category:
text: "{{ if .Result.category_is_tv_show }}TV{{ else }}Movie{{ end }}"
# json engine n/a
Integrating with Prowlarr
You can integrate this indexer with Prowlarr by adding a custom definition. See Adding a custom YML definition.
---
id: torrent-indexer
name: Torrent Indexer
description: "Indexing Brazilian Torrent websites into structured data. github.com/module195/torrent-indexer"
language: pt-BR
type: public
encoding: UTF-8
links:
- http://localhost:8080/
caps:
categories:
Movies: Movies
TV: TV
modes:
search: [q]
tv-search: [q, season]
movie-search: [q]
settings:
- name: indexer
type: select
label: Indexer
default: bludv
options:
search: Torrent-Indexer Cache
bludv: BLUDV
comando_torrents: Comando Torrents
torrent-dos-filmes: Torrent dos Filmes
starck-filmes: Starck Filmes
rede_torrent: Rede Torrent
vaca_torrent: Vaca Torrent
search:
paths:
- path: "{{ if eq .Config.indexer \"search\" }}/search{{ else }}/indexers/{{ .Config.indexer }}{{ end }}"
response:
type: json
inputs:
filter_results: "true"
q: "{{ .Keywords }}"
keywordsfilters:
- name: tolower
- name: re_replace
args: ["(?i)(S0)(\\d{1,2})$", "temporada $2"]
- name: re_replace
args: ["(?i)(S)(\\d{1,3})$", "temporada $2"]
rows:
selector: $.results
count:
selector: $.count
fields:
download:
selector: magnet_link
title:
selector: title
description:
selector: original_title
details:
selector: details
infohash:
selector: info_hash
date:
selector: date
size:
selector: size
seeders:
selector: seed_count
leechers:
selector: leech_count
imdb:
selector: imdb
category_is_tv_show:
selector: title
filters:
- name: regexp
args: "\\b(S\\d+(?:E\\d+)?)\\b"
category:
text: "{{ if .Result.category_is_tv_show }}TV{{ else }}Movies{{ end }}"
Mirrors
Community Mirrors
If you're running a public instance of torrent-indexer and would like to share it with the community, please open a Merge Request adding your instance to this list!
Requirements for community mirrors:
- Must be publicly accessible
- Should have reasonable uptime
- Include geographic location information
- Provide contact information for maintenance issues
Warning
The instance running at https://torrent-indexer.darklyn.org/ is my personal instance and it is not guaranteed to be up all the time. Also, for better availability, I recommend deploying your own instance because the Cloudflare protection may block requests from indexed sites if too many requests are made in a short period of time from the same IP.
If I notice that the instance is being used a lot, I may block requests from Jackett/Prowlarr to avoid overloading the server. I can also redirect your requests to a fake instance without prior notice. Consider yourself warned.
If you cannot self-host
If you really need to use my instance continuously without the risk of being IP blocked, please subscribe to my GitHub Sponsors (any tier) and let me know about your support ❤️ via Discord.
