Arc

Open, SQL-native time-series database for telemetry you need to keep. Arc ingests 38M+ records/sec, stores data as standard Parquet on infrastructure you own, and lets you query recent and historical data together. InfluxDB Line Protocol and Telegraf compatible. Single binary. AGPL-3.0.

Prefer a UI? Arc Launchpad is a self-hosted web console for the Arc instances you run — SQL console, schema explorer, logs, monitoring, and management for tokens, retention, alerts, continuous queries, and MQTT ingestion. Deploy it alongside Arc with Docker Compose. Docs.

Prefer a terminal? arcli is the Arc command line — named connections, SQL queries with table, JSON, CSV or Arrow output, line-protocol and MessagePack writes, bulk imports, and admin for tokens, retention, continuous queries, backups and compaction. brew install basekick-labs/tap/arcli, or deb, rpm, Arch and Docker. Docs.

---

Telemetry is easy to collect and expensive to keep

Machines, services, vehicles, and devices produce data continuously. The operational problem is not only ingesting the latest readings — it is keeping the full-resolution history available for debugging, analysis, compliance, and the next question nobody has asked yet.

Teams evaluating a time-series database usually run into the same trade-offs:

• Retention cliffs: Older data is downsampled, exported, or deleted because storage costs grow too quickly.
• Split hot and cold paths: Recent data is queryable in one system while historical data waits in a warehouse or object store.
• Operational overhead: A simple workload turns into a PostgreSQL extension stack or a multi-service cluster.
• Migration friction: Existing agents, dashboards, and Line Protocol writers make changing databases risky.
• Vendor lock-in: Proprietary storage makes it difficult to use your data elsewhere or leave later.

Arc is built for teams that want to keep the data, query the whole history, and start with a small deployment. It combines high-throughput ingestion, automatic Parquet storage and compaction, analytical SQL, retention policies, and continuous queries in one binary.

Built for aerospace telemetry. Useful anywhere machines never stop producing data.

---

What Arc is (and isn't)

Arc is a complete time-series analytical database: ingestion pipeline, Parquet storage engine, compaction system, SQL query layer, retention policy manager, continuous query scheduler, and telemetry integrations — in one binary. It uses DuckDB as its query engine, while Arc adds the pieces needed to run a durable ingestion and analytics service: high-throughput writes with automatic Parquet flushing, background compaction, scheduled compute, data lifecycle management, authentication, backup and restore, and enterprise clustering.

Arc is not a wrapper. You don't bring your own ingestion, compaction, or retention policies. Arc provides the full stack.

Why teams evaluate Arc

• Keep full-resolution history instead of choosing between retention and cost.
• Use standard SQL with window functions, CTEs, joins, and analytical aggregations.
• Own the files: Arc stores data as open Apache Parquet on local disk, S3, Azure, or any S3-compatible store (SeaweedFS, R2, MinIO, ...).
• Start small: run one binary on a laptop, edge box, or server before adding enterprise clustering.
• Migrate gradually: use InfluxDB Line Protocol and Telegraf-compatible ingestion to dual-write and validate before cutover.

When Arc may not be the right choice

• If your workload is primarily transactional relational data and already fits comfortably in PostgreSQL, start by evaluating TimescaleDB.
• If you need a mature metrics-only replacement for Prometheus and depend on PromQL, evaluate VictoriaMetrics.
• If your organization cannot approve AGPL-3.0 software, use Arc Enterprise's commercial license or choose an Apache-licensed alternative.
• If you need the largest established community and the lowest adoption risk, Arc is newer than TimescaleDB, VictoriaMetrics, and QuestDB.

-- Telemetry: hourly sensor summary across a full history
SELECT
  device_id,
  DATE_TRUNC('hour', timestamp) AS hour,
  AVG(value) AS average_value,
  MIN(value) AS minimum_value,
  MAX(value) AS maximum_value
FROM telemetry.sensor_readings
WHERE timestamp > NOW() - INTERVAL '30 days'
GROUP BY device_id, hour
ORDER BY hour DESC;

-- Telemetry: correlate readings with device metadata
SELECT
  d.site,
  r.device_id,
  AVG(r.value) AS average_value
FROM telemetry.sensor_readings AS r
JOIN telemetry.devices AS d ON d.device_id = r.device_id
WHERE r.timestamp > NOW() - INTERVAL '24 hours'
GROUP BY d.site, r.device_id;

Standard SQL. Window functions, CTEs, joins, aggregations. No proprietary query language.

Start with the workload you already have

Arc accepts InfluxDB Line Protocol directly, so existing Telegraf inputs can write to Arc without changing the collection layer. A low-risk migration usually looks like this:

1. Point a small slice of ingestion at Arc, or dual-write to both systems.
2. Compare the data and query results over an overlapping time window.
3. Move one dashboard or workload at a time.
4. Decommission the old database only after a full retention cycle has passed.

See the InfluxDB migration guide, or compare Arc with TimescaleDB, InfluxDB, ClickHouse, and Elasticsearch.

---

Live Demo

See Arc in action: https://basekick.net/demos

---

Performance

Benchmarked on Apple MacBook Pro M3 Max (14 cores, 36GB RAM, 1TB NVMe). Test config: 12 concurrent workers, 1000-record batches, columnar data.

Ingestion (October 2026)

Protocol	Throughput	p50 Latency	p99 Latency
MessagePack Columnar	38.1M rec/s	0.26ms	1.19ms
MessagePack + Zstd	30.3M rec/s	0.35ms	1.23ms
MessagePack + GZIP	26.4M rec/s	0.39ms	1.36ms
Line Protocol	9.3M rec/s	1.07ms	3.38Ms

All rows measured over a 60-second sustained run. The MessagePack Columnar row is 2,288,281,000 records ingested in 60 seconds — and that's not rows streamed into a memory buffer: every record was received over HTTP, decoded, time-sorted, and durably written to disk as queryable Parquet, at 0.26ms median latency, on a laptop.

These numbers ship in 27.01.1. Since 26.09.1 (34.0M, 24.9M, 24.6M and 4.7M rec/s): line-protocol writes parse straight into typed columns (#1299), and flushes copy each column once less on the way to Parquet.

26.09.1 stopped dictionary-encoding Parquet at ingest (compaction re-encodes files anyway) and made the msgpack columnar path decode payloads directly into typed column arrays, eliminating per-value allocations — see the 26.09.1 release notes.

Compaction

Automatic background compaction merges small Parquet files into optimized larger files:

Metric	Before	After	Reduction
Files	43	1	97.7%
Size	372 MB	36 MB	90.4%

Benefits:

• 10x storage reduction via better compression and encoding
• Faster queries - scan 1 file vs 43 files
• Lower cloud costs - less storage, fewer API calls

Query (October 2026)

Arc speaks three wire formats from the same query engine. Arrow IPC is the throughput leader for analytical clients (Grafana, pyarrow, polars) that can take an Arrow dependency — zero-copy from the engine's internal columnar buffers. MessagePack (columnar, stable as of 26.09.1) is the choice for clients that don't speak Arrow but want smaller bytes and faster decode than JSON — same envelope shape as JSON, native binary types for timestamps and binary columns. JSON stays the default for ergonomic compatibility.

Benchmark: Arc 27.01.1 (main at b4d98a31, October 2026), M3 Max, query_suite from arc-benchmarks, 5 iterations per query, latency is p50 in milliseconds. One session, one dataset: a 532.6M-row cpu measurement (60 s of sustained_bench msgpack ingest spread over 24 hours, hourly-compacted: 25 hour partitions, 100 files), so the three columns are apples-to-apples.

Query	JSON (ms)	MessagePack (ms)	Arrow IPC (ms)	msgpack vs JSON	Arrow vs JSON
COUNT(*) — 532.6M rows	52.9	52.1	52.5	1.02x	1.01x
SELECT LIMIT 10K	12.3	12.5	11.9	0.99x	1.04x
SELECT LIMIT 100K	18.9	17.4	14.9	1.09x	1.27x
SELECT LIMIT 500K	43.9	38.9	27.5	1.13x	1.60x
SELECT LIMIT 1M	71.6	65.4	39.6	1.09x	1.81x
Time Range (1h) LIMIT 10K — no rows in window¹	2.6	2.7	3.1	0.96x	0.84x
Time Range (7d) LIMIT 10K	13.6	13.5	12.9	1.01x	1.05x
Time Bucket (1h, 7d)	605	606	612	1.00x	0.99x
Date Trunc (day, 30d)	605	609	608	0.99x	1.00x
GROUP BY host	502	501	500	1.00x	1.00x

Best throughput on LIMIT 1M (1M-row payload, single connection):

• Arrow IPC: 25.3M rows/sec (39.6ms)
• MessagePack: 15.3M rows/sec (65.4ms)
• JSON: 14.0M rows/sec (71.6ms)

¹ The newest data is two hours old, so the window is empty. Arc 27.01.1 answers it from the newest partition alone (it filters to zero rows) instead of reading the whole measurement, as before (49 ms); measured on main at 2eecb2d1.

Notes on the table: the wire-format differences show on response-heavy queries (≥100k rows) where encoding dominates the per-request wall time. For aggregations (Time Bucket, Date Trunc, GROUP BY) the response is a few rows and DuckDB execution is nearly all of the wall time; all three formats converge. Arrow IPC copies the engine's column buffers. Since 27.01.1 every format streams straight to the socket, and JSON and MessagePack are encoded on several cores (JSON by batch, MessagePack by column run, handed to encoder goroutines and written in order); MessagePack's typed columnar encoder keeps it ahead of JSON on large results.

The MessagePack endpoint is stable as of 26.09.1: its response shape and type vocabulary are a published contract pinned by a golden test, so clients can bind to them. It's the best general-purpose format for a client not using Arrow directly — columnar, typed, and it accepts SHOW statements, which the Arrow endpoint doesn't. See the 26.09.1 release notes for the wire-format contract.

---

Single binary. Zero dependencies.

Arc deploys as one self-contained executable: DuckDB and its dependencies are linked in, and on Linux it needs only the system C library (glibc 2.34 or newer) and libgcc_s, which every distro ships. No JVM, no Python environment, no PostgreSQL cluster to manage, no ZooKeeper ensemble to babysit. Run it on a laptop, a factory edge box, an on-premises server, or a Kubernetes cluster. Same binary, same config surface.

• Air-gap ready: No external services required at runtime. No license server, no cloud dependency.
• Edge to cloud: Deploy at the tactical edge, in a sovereign cloud, or on-premises.
• Minimal footprint: One process. Memory usage proportional to active workload, not fleet size.

---

Quick Start

# Build
make build

# Run
./arc

# Verify
curl http://localhost:8000/health

---

Installation

Docker

# Docker Hub
docker run -d \
  -p 8000:8000 \
  -v arc-data:/app/data \
  basekicklabs/arc:latest

# or GitHub Container Registry
docker run -d \
  -p 8000:8000 \
  -v arc-data:/app/data \
  ghcr.io/basekick-labs/arc:latest

Multi-arch images (linux/amd64 + linux/arm64) are published to both registries on every release.

macOS (Homebrew)

brew install basekick-labs/tap/arc

Apple Silicon. DuckDB is statically linked, so there are no runtime dependencies. (Use brew install --formula arc if you tap first, to disambiguate from the arc browser cask.)

Linux packages run on glibc 2.34 or newer: Ubuntu 22.04+, Debian 12+, RHEL/Rocky/Alma 9+, Amazon Linux 2023, Fedora 35+. Ubuntu 20.04 and RHEL 8 are not supported.

Debian/Ubuntu

wget https://github.com/basekick-labs/arc/releases/download/v26.09.3/arc_26.09.3_amd64.deb
sudo dpkg -i arc_26.09.3_amd64.deb
sudo systemctl enable arc && sudo systemctl start arc

RHEL/Fedora

wget https://github.com/basekick-labs/arc/releases/download/v26.09.3/arc-26.09.3-1.x86_64.rpm
sudo rpm -i arc-26.09.3-1.x86_64.rpm
sudo systemctl enable arc && sudo systemctl start arc

Arch Linux

Works on Arch Linux and Arch-based distros such as Omarchy (x86_64 and aarch64).

wget https://github.com/basekick-labs/arc/releases/download/v26.09.3/arc-26.09.3-1-x86_64.pkg.tar.zst
sudo pacman -U arc-26.09.3-1-x86_64.pkg.tar.zst
sudo systemctl enable arc && sudo systemctl start arc

Kubernetes (Helm)

helm install arc https://github.com/basekick-labs/arc/releases/download/v26.09.3/arc-26.09.3.tgz

Build from Source

# Prerequisites: Go 1.26+

# Clone and build
git clone https://github.com/basekick-labs/arc.git
cd arc
make build

# Or build directly with Go (the duckdb_arrow tag is required)
go build -tags=duckdb_arrow ./cmd/arc

# Run
./arc

FIPS 140-3 Build

For US defense/federal and other regulated environments, Arc ships an optional arc-fips build: the same source at the same version, compiled against the CMVP-certified Go Cryptographic Module and run in FIPS-only mode. Pick the -fips artifact instead of the standard one.

# Binary — download arc-fips-linux-amd64 (or -arm64) from the release
# Container — same repos, -fips tag suffix:
docker run -d -p 8000:8000 -v arc-data:/app/data ghcr.io/basekick-labs/arc:VERSION-fips
# or basekicklabs/arc:VERSION-fips

# Build from source:
make build-fips      # -> arc-fips (GOFIPS140=v1.0.0, -tags=duckdb_arrow,fips)

The FIPS build reports the same version as the standard build and logs "fips_mode":true at startup. Cutover note: existing bcrypt-hashed API tokens must be rotated when moving to the FIPS build (it stores new tokens with PBKDF2 and fails bcrypt verification closed). The Go Cryptographic Module is CMVP-certified; Arc itself is not a CMVP-listed module. See the FIPS 140-3 mode guide.

---

Ecosystem & Integrations

Tool	Description	Link
Arc Launchpad	Self-hosted web UI: SQL console, schema explorer, logs, monitoring, and management for tokens, retention, alerts, continuous queries, MQTT ingestion, and teams	GitHub · Docs
arcli	Command-line client: connection profiles, SQL queries (table, JSON, CSV, Arrow), writes and bulk imports, and admin for tokens, retention, continuous queries, backups, compaction and logs. Homebrew, deb/rpm/Arch, Docker	GitHub · Docs
VS Code Extension	Browse databases, run queries, visualize results	Marketplace
Grafana Data Source	Native Grafana plugin for dashboards and alerting	GitHub
Telegraf Output Plugin	Ship data from 300+ Telegraf inputs directly to Arc	Docs
Python SDK	Query and ingest from Python applications	PyPI
Superset Dialect (JSON)	Apache Superset connector using JSON transport	GitHub
Superset Dialect (Arrow)	Apache Superset connector using Arrow transport

---

Features

Core Capabilities

• Columnar storage: Parquet format with full analytical SQL engine

• Workloads: Industrial IoT, manufacturing, energy, fleet telemetry, aerospace, observability, and event analytics

• Ingestion: MessagePack columnar (fastest), InfluxDB Line Protocol, MQTT, TLE (satellite telemetry)

• Query: Full analytical SQL; JSON, columnar MessagePack, and Apache Arrow IPC responses

• Compaction: Tiered (hourly/daily) automatic Parquet file merging — 10x storage reduction; see hourly compaction tuning

• Data Lifecycle: Retention policies, continuous queries, tiered storage (hot/cold)

• Durability: Optional write-ahead log (WAL), backup and restore; WAL recovery monitoring

• Storage: Local filesystem, S3 and S3-compatible object stores (SeaweedFS, R2, MinIO, ...), Azure Blob

• Auth: Token-based authentication with in-memory caching

• Data Management: GDPR-compliant delete operations

• Observability: Prometheus metrics, structured logging, graceful shutdown

• Reliability: Circuit breakers, retry with exponential backoff

• Supply chain: SBOM (SPDX + CycloneDX), Trivy scans, cosign-signed releases, SLSA L3 provenance

• FIPS 140-3: Optional arc-fips build against the CMVP-certified Go Cryptographic Module — see Installation

• Edge Sync (new in 26.09.1): Spoke-to-hub data transport for disconnected operations, over the network or via signed air-gap bundles. Network spokes retain the manual trigger; a valid paid license also schedules sync. Configure edge_sync.spoke.sync_interval (default 5m) and edge_sync.spoke.sync_retry_interval (default ) to set the success and retry cadence.

---

Configuration

Arc uses TOML configuration with environment variable overrides.

[server]
host = "0.0.0.0"
port = 8000

[storage]
backend = "local"        # local, s3, minio
local_path = "./data/arc"

[ingest]
max_buffer_age_ms = 5000   # flush a buffer at most this old
max_buffer_size = 50000

[auth]
enabled = true

Environment variables use ARC_ prefix:

export ARC_SERVER_PORT=8000
export ARC_STORAGE_BACKEND=s3
export ARC_AUTH_ENABLED=true

See arc.toml for complete configuration reference.

---

Project Structure

arc/
├── cmd/arc/              # Application entry point
├── internal/
│   ├── api/              # HTTP handlers (Fiber) — query, write, import, TLE, admin
│   ├── audit/            # Audit logging for API operations
│   ├── auth/             # Token authentication and RBAC
│   ├── backup/           # Backup and restore (data, metadata, config)
│   ├── circuitbreaker/   # Resilience patterns (retry, backoff)
│   ├── cluster/          # Raft consensus, node roles, peer file replication, standby stream
│   ├── compaction/       # Tiered hourly/daily Parquet file merging
│   ├── config/           # TOML configuration with env var overrides
│   ├── database/         # Query engine and connection management
│   ├── governance/       # Per-token query quotas and rate limiting
│   ├── ingest/           # MessagePack, Line Protocol, TLE, Arrow writer
│   ├── license/          # License validation and feature gating
│   ├── logger/           # Structured logging (zerolog)
│   ├── metrics/          # Prometheus metrics
│   ├── mqtt/             # MQTT subscriber — topic-to-measurement ingestion
│   ├── pruning/          # Query-time partition pruning
│   ├── queryregistry/    # Active/completed query tracking
│   ├── scheduler/        # Continuous queries and retention policies
│   ├── shutdown/         # Graceful shutdown coordinator
│   ├── sql/              # SQL parsing utilities
│   ├── storage/          # Local, S3, Azure backends
│   ├── telemetry/        # Usage telemetry
│   ├── tiering/          # Hot/cold storage lifecycle management
│   └── wal/              # Write-ahead log
├── pkg/models/           # Shared data structures (Record, ColumnarRecord)
├── benchmarks/           # Performance benchmarking suites
├── deploy/               # Docker Compose and Kubernetes configs
├── helm/                 # Helm charts
├── scripts/              # Utility scripts (analysis, backfill, debugging)
├── arc.toml              # Configuration file
├── Makefile              # Build commands
└── go.mod

---

Development

make deps           # Install dependencies
make build          # Build binary
make run            # Run without building
make test           # Run tests
make test-coverage  # Run tests with coverage
make bench          # Run benchmarks
make lint           # Run linter
make fmt            # Format code
make clean          # Clean build artifacts

---

License

Arc is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0).

• Free to use, modify, and distribute
• If you modify Arc and run it as a service, you must share your changes under AGPL-3.0

For commercial licensing, contact: [email protected]

---

Contributors

Want to join this list? See CONTRIBUTING.md and the good first issue queue.

Thanks to everyone who has contributed code to Arc:

• @schotime (Adam Schroder) — Data-time partitioning, compaction API triggers, UTC fixes, bare-JOIN SQL rewriting fix
• @khalid244 — S3 partition pruning improvements, multi-line SQL query support
• @SAY-5 (Sai Asish Y) — MQTT nil-guard hardening (handlers + manager) with regression coverage
• @mvanhorn (Matt Van Horn) — backup/restore concurrency fix: atomic admission guard so competing operations get 409 instead of silently queuing
• @bferanmi806-sketch (Enoch) — Iceberg reliability series (#663–#667): version hints published only after metadata copies land, reconcile-interval validation at config load, unreadable-database skip during reconciliation, SQLite WAL/SHM 0600 hardening, empty-measurement negative caching; compaction job-history retention fix (#679); periodic file replication reconciliation with readiness-safe self-healing (#697)
• @Thundercloud12 (Keerthan Poojary) — Azure not-found error detection via errors.As, covering joined multi-errors (#670); PREPARE/EXECUTE blocked by the read-SQL validator as defense-in-depth (#767); MQTT restart TOCTOU fix with nil-placeholder slot reservation and 409 on concurrent restart (#766); subscriber install made a compare-and-swap on that reservation so a delete or shutdown racing a start can no longer leak a live subscriber, tested through an in-process MQTT broker (#788); RBAC checks every case-distinct table reference a query names (#832)
• @atirna (Atirna) — live SQLite backup/restore safety: VACUUM INTO snapshots for backups, rename-swap restores with stale sidecar removal, restart-required signaling (#678, #689); WAL chunking for oversized wide-row payloads with offset-based zero-copy splitting (#696); Helm startup probe protecting long WAL replay from liveness crash loops (#700)
• @mah1104ahm (M Ahmed) — bounded local storage directory cache (#674), tier-cache expiry pruning (#675), MQTT shutdown unsubscribe awaits with persisted subscription status (#673), pre-26.06.1 token upgrade runbook (#672)

And a thank-you to community members whose bug reports drove fixes:

• @bjarneksat — reported the line-protocol null-field handling bug fixed in 26.03.1
• @rexpository — reported the edge-sync spoke-ID namespace collision fixed in 26.09.2 (#737), and ongoing security research across the cluster, auth and edge-sync boundaries