🗺️ Borges

Borges (Better ORGanizations Entities mappingS) maps Autonomous Systems (ASes) to the organizations that operate them. It finds sibling ASes run by the same company, even when WHOIS lists them under different organizations. It combines CAIDA's WHOIS-based AS2Org and PeeringDB with two new signals: LLM extraction of sibling ASNs from PeeringDB's free-text fields, and website inference from redirect chains, domain similarity and favicons. It is the code of the ACM IMC 2025 paper Learning AS-to-Organization Mappings with Borges.

[!NOTE] Mappings go stale quickly. Companies merge, rebrand and disappear. Edgio vanished between the paper's submission and its presentation. The published artifacts are a September 2025 snapshot. To get current mappings, run Borges on fresh PeeringDB and AS2Org data. A full run with gpt-4o-mini costs about US$3. You can also run it with no paid API key (see below).

✨ Features

• 🏢 Organization keys: groups ASes by CAIDA AS2Org (WHOIS) organization IDs and PeeringDB organization IDs
• 🤖 LLM sibling extraction: few-shot prompting reads PeeringDB notes and aka fields and keeps only real sibling ASNs, not phone numbers, years or prefix limits
• 🔀 Redirect analysis: networks whose websites resolve to the same final URL are grouped together
• 🖼️ Favicon analysis: shared favicons plus similar domains reveal common branding; a vision LLM rejects framework defaults (WordPress, Bootstrap, …)
• 🧩 Consolidation: merges every signal into network groups, with blocklists for known false bridges
• 💾 Robust pipeline: nine stages with checkpoint/resume, parallel scraping with caching, and Parquet/JSON/CSV exports
• 💸 Free to run: tests and CI need no key. The LLM stages can be skipped or pointed at a local model

🚀 Quick Start

pip install borges                   # or: uv pip install borges
borges init                          # writes config.yaml (prompts, blocklists), .env and data/
borges download                      # latest PeeringDB + AS2Org snapshots into data/input/
borges pipeline run                  # needs an LLM key, or see below to skip it

From a clone (for development):

git clone https://github.com/NU-AquaLab/borges.git
cd borges
uv venv && source .venv/bin/activate
uv pip install -e ".[dev]"

📖 Usage

Pipeline stages

#	Stage	What it does	LLM
1	load_data	Load PeeringDB and WHOIS (AS2Org) data
2	redirect_scraping	Follow each PeeringDB website to its final URL (no HTML stored)
3	as_detection	Extract sibling ASNs from notes and aka	🤖
4	redirect_analysis	Group networks that share a final URL
5	favicon_download	Download website favicons
6	favicon_analysis	Decide whether shared favicons mean a shared company	🤖
7	whois_processing	Group ASes by AS2Org organization
8	network_consolidation	Merge groups from all sources
9	export_results	Write Parquet/JSON/CSV and reports to data/output/

Commands

borges pipeline list                                   # stages and their dependencies
borges pipeline run                                    # full run
borges pipeline run --stage redirect_scraping --stage as_detection   # selected stages
borges pipeline run --skip favicon_analysis            # skip stages
borges pipeline run --resume                           # continue from checkpoints
borges pipeline run --dry-run                          # show what would run
borges --config my-config.yaml pipeline run            # or: export BORGES_CONFIG=...
borges download --peeringdb-date 2025-07-16 --as2org-date 2025-07-01

borges --help lists the rest (report generate, favicon download, config, version).

💸 Running without a paid API key

Only stages 3 and 6 call an LLM. Pick one option:

Option	How	Trade-off
Skip the LLM stages	borges pipeline run --skip as_detection --skip favicon_analysis	No sibling extraction from notes/aka, no favicon grouping
Free local model	Any OpenAI-compatible server: vLLM on a GPU, Ollama, LM Studio. Set base_url (below)	Results differ from the paper, which used gpt-4o-mini
OpenAI	OPENAI_API_KEY=... in .env	About US$3 per full run

With Ollama (ollama pull qwen3:8b and ollama pull qwen3-vl:8b-instruct), set this in config.yaml:

api:
  llm:
    base_url: http://localhost:11434/v1
    model: qwen3:8b
    vision_model: qwen3-vl:8b-instruct
    structured_output: true              # force parseable JSON answers
    timeout: 300
    extra_body:
      reasoning_effort: none             # turn off thinking: much faster

여기까지가 README 앞부분입니다. 나머지는 원문에서 읽을 수 있습니다. README 전문 보기 ↗