SyncVey

English | 日本語

terraform plan: no changes. Reality: someone edited 3 resources in the AWS console last week. SyncVey catches what plan can't.

SyncVey organizes your AWS resources into a System → Environment → Asset hierarchy, flags configuration drift between your tfstate and live AWS state, and tracks application metadata (language, framework, dependencies) per environment. Spin it up with a single docker compose up.

Feature tour → https://mrktools.app/en/syncvey/

---

Screenshots

Dashboard — a hero-signal row answers what's happening (drift / EOL), whether it's urgent (trend vs the last scan, freshness), and what to do next

Drift report — attribute-level diff between tfstate and actual AWS state

Drift history — drift trend over time, with the field-level diff for each snapshot

---

Why SyncVey?

I'm 62, and I still write code. For years I watched infrastructure teams keep their AWS inventory in spreadsheets — and every single time, without exception, I watched those spreadsheets rot.

One day I asked the obvious question: we have tfstate, we have boto3 — why are we still doing this by hand?

The other thing nagging at me was what runs on the infrastructure: middleware nearing end-of-life, and libraries with publicly known vulnerabilities. We had lists to watch, but no alerts, no dashboard, nothing tied to a specific system or environment — just more spreadsheets quietly going stale.

So I built the tool I'd always wanted. Scheduled scans, drift detection, EOL dates and known vulnerabilities — in one place, self-hosted, with your data staying inside your own infrastructure.

---

How is SyncVey different?

Each tool in this space does one slice well. driftctl pioneered Terraform drift detection, but it's a stateless CLI — and is no longer maintained since Snyk archived it. Steampipe turns your cloud into queryable SQL, which is excellent for ad-hoc investigation, but it's a query engine you build dashboards on, not a turnkey ledger. Cloud Custodian shines at policy-as-code enforcement (tag, stop, remediate), yet it's about reacting to rules, not giving you a browsable inventory. SyncVey sits in the middle as a self-hosted web app: a persistent asset ledger with a UI, attribute-level drift between your tfstate and live AWS — including resources created by hand in the console that terraform plan never sees — plus a layer none of the others touch: per-environment application & middleware tracking, with end-of-life dates and publicly known vulnerabilities side by side. SyncVey is not a vulnerability scanner — it doesn't look inside your servers or containers. It reads what you tell it (a lockfile, a tfstate, your AWS account) and shows what is already publicly known about those versions. One docker compose up, and your data stays in your own infrastructure.

	tfstate drift	Detects console-made resources	Ledger + UI	App/middleware + EOL + known vulns	Form factor
driftctl	✅	✅	❌	❌	CLI
Steampipe	⚠️ DIY	✅ (query)	⚠️ build it	❌	CLI + SQL
Cloud Custodian	❌	⚠️ policy-dependent	❌	❌	Policy engine
SyncVey	✅	✅	✅	✅	Self-hosted web app

---

Features

Feature	Description
Asset ledger	Inventory and search EC2, ECS, Lambda, RDS, DynamoDB, ElastiCache, EFS, EKS, S3, ALB, VPC, EBS, SNS, SQS, API Gateway, Secrets Manager, CloudFront, Route 53 resources — and more
AWS scan	Auto-discover 18 resource types — compute, database, storage, network, messaging — in target accounts via AssumeRole
Terraform / OpenTofu integration	Import assets by uploading a tfstate file. OpenTofu state files use the same format and import the same way (encrypted OpenTofu state can't be read and is reported as such)
Drift detection	Spot attribute-level differences between tfstate and live AWS state
Auto Scaling-aware drift	Instances an Auto Scaling group launched or terminated are churn, not drift. SyncVey reads the aws:autoscaling:groupName tag it already scans — no extra API call or IAM — and keeps scale-out/in out of the drift count, shown transparently in their own section. Attribute changes on a persistent instance are still real drift. Toggle with DRIFT_SUPPRESS_AUTOSCALING
Deleted-resource detection	A resource that vanishes from AWS is flagged in the ledger and reported as removed drift — the row is kept, not deleted, so you can still see what was there and when it went. Marking only happens for the regions and resource types that scanned cleanly, so a throttled API call or an expired credential can never be mistaken for a mass deletion. If the resource comes back, the flag clears itself. Auto Scaling scale-in counts as churn, not drift
Drift history	Track drift over time — every scan/import records a snapshot, with a trend chart and per-snapshot diff
Drift risk & attribution	Grade drift by security impact (e.g. a security group opened to 0.0.0.0/0) and trace who changed a resource via CloudTrail
Blast radius (optional plugin)	Walk the resource reference graph outward from each drift and rank every resource it can reach by severity-weighted, distance-decayed impact. Richer with the drift-risk plugin installed; detachable — remove the app and the feature disappears
Secret rotation drift	Flag Secrets Manager secrets that should have rotated but didn't — rotation disabled, never run, or past the limit (default 90). Graded on current state, so it surfaces even when nothing changed between two scans. Metadata only: the secret value is never read or stored

What SyncVey does not do

• It does not scan inside your servers or containers.
• A lockfile is what you meant to deploy, which can differ from what runs.
• RDS results are for the upstream version; AWS may have already patched some of it inside RDS.
• It only knows publicly known vulnerabilities. "None known" is not "safe".
• Configuration checks (public databases, encryption, open security groups) are a separate matter, only partly covered by drift risk grading.

---

Tech stack

Layer	Technology
Backend	Python 3.12 / Django (server-rendered, no SPA)
Frontend	htmx 1.9 + Tailwind CSS — Django templates, no build step, no Node toolchain
Database	PostgreSQL 18.3
AWS SDK	boto3 (cross-account access via AssumeRole)
Auth	TOTP two-factor authentication (pyotp)
Scheduler	django-apscheduler (periodic scans)
Infra	Docker Compose

---

Getting started

Prerequisites

• Docker and Docker Compose

Get the code

git clone https://github.com/MR-TABATA/SyncVey
cd SyncVey

The commands below run in this folder: it holds docker-compose.yml, .env.example and the IAM policy (iam/iam-policy.json). The image itself comes from Docker Hub, so there is nothing to build.

Don't want to clone? The compose file reads nothing else from the repository, so two files are enough:

mkdir syncvey && cd syncvey
curl -O https://raw.githubusercontent.com/MR-TABATA/SyncVey/main/docker-compose.yml
curl -o .env https://raw.githubusercontent.com/MR-TABATA/SyncVey/main/.env.example

then continue with step 1 below. (docker pull jiniie/syncvey alone is not enough: the image has no start command and needs PostgreSQL, which the compose file provides. The Docker Hub page says so too.)

1. Configure environment variables

cp .env.example .env

Edit .env:

# Django
SECRET_KEY=your-secret-key
DEBUG=True

# Database (match docker-compose.yml)
DATABASE_URL=postgres://user:password@db:5432/asset_manager

# AWS — IAM user credentials for the central account
AWS_ACCOUNT_ID=123456789012
AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
AWS_SCAN_REGIONS=ap-northeast-1,ap-northeast-3

Production note: set DEBUG=False for any non-local deployment, and use a strong, unique SECRET_KEY.

2. Start the app

docker compose up -d

This pulls the published image (jiniie/syncvey, amd64 and arm64) — there is nothing to build and nothing to compile.

Building from source instead: copy docker-compose.override.yml.example to docker-compose.override.yml. It builds the image from your working tree and mounts the tree for live reload — that is the development setup.

Pinning a version: set SYNCVEY_IMAGE=jiniie/syncvey:0.2.0 in .env.

Tip: If you use VS Code, open the repo in Dev Containers — the environment is built automatically.

3. Apply migrations and (optionally) seed sample data

docker compose exec app python manage.py migrate
docker compose exec app python manage.py seed   # optional: loads sample data

4. Open in your browser

Service	URL
App	http://localhost:8000/
Django Admin	http://localhost:8000/admin/

---

Try it without an AWS account

Run a real scan — ledger, drift report, dashboard — against a local AWS emulator (LocalStack). No AWS account, no credentials, no bill.

1. Uncomment the demo block in .env:

AWS_ENDPOINT_URL=http://localstack:4566
AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test

2. Start the stack with the emulator, create some fake resources, and scan them:

docker compose --profile demo up -d
docker compose exec app python scripts/seed_localstack.py
docker compose exec app python manage.py syncvey scan --system "Demo System"

Open http://localhost:8000/ and the scanned resources are in the ledger.

3. Make something drift, then look again:

docker compose exec app python -c "
import boto3
s = boto3.Session(region_name='ap-northeast-1')
q = s.client('sqs')
q.delete_queue(QueueUrl=q.get_queue_url(QueueName='demo-jobs')['QueueUrl'])
ec2 = s.client('ec2')
ec2.stop_instances(InstanceIds=[ec2.describe_instances()['Reservations'][0]['Instances'][0]['InstanceId']])
"
docker compose exec app python manage.py syncvey scan --system "Demo System"
docker compose exec app python manage.py syncvey drift --system "Demo System"

Demo System / Production: 2 drifted (changed=1 added=0 removed=1)
    ~ EC2 i-9538ffc109d6e7b6b (i-9538ffc109d6e7b6b)
        instance_state: running -> stopped
    - SQS demo-jobs (arn:aws:sqs:ap-northeast-1:000000000000:demo-jobs)

What works, and what doesn't

LocalStack's free edition implements a subset of AWS, so 10 of the 18 scanners return data. The other 8 report an error for their service and the scan carries on — a scanner that failed is deliberately excluded from deleted-resource detection, so a service SyncVey could not read is never mistaken for a fleet someone deleted.

Scans against LocalStack	Paid LocalStack tier only
EC2, VPC, EBS, S3, Lambda, DynamoDB, SNS, SQS, Secrets Manager, Route 53	RDS, ECS, ALB, ElastiCache, EFS, EKS, API Gateway v2, CloudFront

• "Who changed this?" does not work here. Attribution reads CloudTrail event history, which the free edition does not provide, so drift rows show no actor.
• The image tag is pinned to localstack/localstack:4 on purpose. :latest now requires a LocalStack auth token and exits immediately without one.

To point SyncVey back at real AWS, comment those three lines out again and run docker compose up -d.

---

Setting up AWS scan

Create a read-only IAM Role in each target AWS account and register its ARN in SyncVey.

Full instructions → aws-setup.md

Quick overview:

1. Add your central account's IAM credentials to .env
2. Create the SyncVeyReadOnly role in each target account using the provided IAM policy (iam/iam-policy.json)
3. Register the Role ARN using the 🛡 button on the system card
4. Click ScanLine to run the first scan

---

External network calls

SyncVey is fully self-hosted. The only outbound connections it makes are listed below — there is no telemetry or usage analytics.

Destination	When	Direction	How to control
AWS APIs (boto3 / AssumeRole)	On-demand or scheduled scan	Outbound HTTPS	Only when a Role ARN is configured
AWS CloudTrail (LookupEvents)	Clicking "Who changed this?", or the weekly drift briefing	Outbound HTTPS	Lazy, never automatic — needs the role + cloudtrail:LookupEvents
hooks.slack.com	Drift detected, or the weekly drift briefing	Outbound HTTPS	Per-system Slack Webhook URL (opt-in); briefing also needs DRIFT_DIGEST_ENABLED=true
endoflife.date	Daily EOL data refresh	Outbound HTTPS	Off by default — enable with EOL_REFRESH_ENABLED=true
api.osv.dev, www.cisa.gov	Daily vulnerability check of the libraries you imported from lockfiles	Outbound HTTPS	Off by default — enable with VULN_REFRESH_ENABLED=true. Sends only (ecosystem, package name, version)
api.github.com	Daily lockfile sync of the repositories you registered (and when you press Sync now)	Outbound HTTPS	Nothing is called until you register a repository; stop it with REPO_REFRESH_ENABLED=false. Sends the owner, repository and file name; optional SYNCVEY_GITHUB_TOKEN (read-only) for private repositories, never stored
services.nvd.nist.gov	Same daily job: known vulnerabilities of your RDS PostgreSQL / MySQL engine versions	Outbound HTTPS	Off by default — same switch. Sends only the product and version (a CPE name); optional NVD_API_KEY raises NVD's rate limit

EOL refresh detail. End-of-life detection works offline using built-in data. Set EOL_REFRESH_ENABLED=true to enable a daily job that pulls fresh data from endoflife.date (falls back to built-in data on failure). By default, only dependencies you actually track are refreshed; set EOL_REFRESH_DYNAMIC=false to pin it to a fixed known set. You can also trigger a refresh manually:

docker compose exec app python manage.py refresh_eol --force

Vulnerability check detail. On the Applications & Middleware screen, each environment has an Import lockfile button. Upload requirements.txt, package-lock.json, composer.lock, Gemfile.lock or go.mod: the file is parsed in memory, the libraries are recorded, and the file is discarded. With VULN_REFRESH_ENABLED=true, a daily job looks those libraries up on OSV.dev and flags the ones also listed in CISA's Known Exploited Vulnerabilities catalog. Only the ecosystem, package name and version are sent — no organization, system or resource names. Libraries you typed in by hand (without a lockfile) are not checked.

A lockfile you upload by hand goes stale the day after, so the way in is the application's GitHub repository: under an environment, enter the repository URL (or a /tree/<branch>/<folder> URL for a monorepo) and press Register and import. From then on SyncVey re-imports the lockfiles every day (03:30; also manage.py refresh_repos --force) — requirements.txt, package-lock.json, composer.lock, Gemfile.lock, go.mod at the top level — so the library list follows the repository without anyone touching it. The screen shows when it last synced, and if a sync fails it says so and keeps the last good list rather than going quiet. A stage environment can point at another branch. It only ever talks to api.github.com; the URL you type is split into owner / repository / folder, never fetched as given. A private repository needs a read-only token (fine-grained, Contents: Read-only) in SYNCVEY_GITHUB_TOKEN; it is read from the environment and is not stored. The API host is https://api.github.com unless an administrator sets SYNCVEY_GITHUB_API (for GitHub Enterprise); it can't be changed from the screen. Without a token GitHub allows about 60 requests an hour per address (5 per environment), so set one if you register many environments. Importing a single file by hand is still there, folded away, for a repository that isn't on GitHub — it never updates again.

The same job also looks up the engine versions of your RDS PostgreSQL and MySQL instances in NVD (Aurora is skipped: its versions don't match upstream). The result is "known vulnerabilities of the upstream version" — AWS may already have fixed some of them inside RDS, and the screen says so. Without an NVD_API_KEY NVD allows about 5 requests per 30 seconds, so each engine version takes roughly 7 seconds. To run it manually:

docker compose exec app python manage.py refresh_vulns --force

---

Routes

SyncVey is a server-rendered htmx app, not a JSON REST API. All views return rendered HTML (full pages or partials).

/                                      Dashboard
/systems/                              Systems
/systems/<id>/environments/            Environments under a system
/systems/<id>/applications/            Applications under a system
/environments/<id>/scan/               Run an AWS scan
/environments/<id>/drift/              Drift report
/environments/<id>/drift/history/      Drift history (trend over time)
/drift-risk/                           Drift risk & attribution (security triage)
/blast-radius/                         Blast radius (drift impact across the reference graph; optional plugin)
/drift-digest/                         Drift briefing (weekly Slack rollup preview)
/environments/<id>/diagram/            Architecture diagram
/environments/<id>/sync-s3/            Sync remote tfstate from S3
/assets/                               Asset list
/assets/<id>/                          Asset detail
/upload-tfstate/                       Import assets from tfstate
/samples/                              Sample library
/audit-log/                            Audit log
/profile/                              Profile / 2FA settings
/admin/                                Django Admin

Full route list: asset_manager/urls.py

---

Command line (optional plugin)

The syncvey_cli plugin adds a syncvey management command so an operator — or a CI pipeline — can trigger a scan and read drift without opening the web UI. It drives the same scan/drift engine the dashboard does, so it can never disagree about what counts as drift.

# Run a live AWS scan and record a drift snapshot (all systems, or one)
docker compose exec app python manage.py syncvey scan --system e-commerce

# Print the current drift; add --format json to feed a pipeline
docker compose exec app python manage.py syncvey drift --env prod

# Fail the build on any drift — drop this into a CI step
docker compose exec app python manage.py syncvey drift --exit-code

# List systems / environments with asset counts and last-scan time
docker compose exec app python manage.py syncvey status

# Take the asset ledger out: JSON (with each asset's stored attributes) or CSV
docker compose exec app python manage.py syncvey export --system e-commerce --output ledger.json
docker compose exec app python manage.py syncvey export --format csv > ledger.csv

Exit codes are the CI contract: 0 = ok / no drift, 1 = drift found (only with drift --exit-code), 2 = a scan job failed or the selector matched nothing. Detachable like the other plugins — remove syncvey_cli from INSTALLED_APPS and the command disappears.

---

Data model

System
  └── Environment  (PROD / STG / DEV / QA)
        └── Asset  (asset_type + category; raw attributes stored in a JSON field)
  └── Application
        └── AppEnvConfig  (per-environment settings)
              └── AppDependency

---

Development

# Open a shell inside the app container
docker compose exec app bash

# Generate migrations after model changes
docker compose exec app python manage.py makemigrations

# Tail logs
docker compose logs -f app
docker compose logs -f db

---

Project

Releases and changes	CHANGELOG.md
Reporting a vulnerability	SECURITY.md — please don't use a public issue
Contributing	CONTRIBUTING.md

---

License

MIT