Chargewire

---

⚡ What is Chargewire?

Chargewire is an open protocol that lets any HTTP client run SQL against a database server and get Apache Arrow data back: a browser fetch, curl, or any language's standard HTTP library. No gRPC, no protobuf, no generated code.

• 🏹 Arrow on the wire. Results, query parameters and bulk uploads are Arrow IPC streams (application/vnd.apache.arrow.stream), so every Arrow library reads them directly. Control messages are small JSON documents.
• 🚀 Short queries stay one round trip. One POST runs a query and streams its result.
• 🔀 Big results go wide. A client can ask for a large result as several partitions and read them over separate connections at once, past what one TCP stream can carry, with the order kept when the query has one. Chargewire's drivers do it by default (8 partitions).
• 📤 Big uploads go wide too. One bulk ingest can be sent as several parts over separate connections at once and committed as one transaction, all or nothing.
• 🌐 Long queries survive real networks. Submit a query, get a handle, poll it and fetch the result later. A dropped download resumes from the batch where it stopped, and no request has to outlive a proxy timeout.
• 🔐 Safe by construction. Every handle is bound to the identity that created it and checked on every use. Credentials are bearer tokens, never cookies. Results and metadata follow the caller's authorization.
• 🧩 Engine-neutral and open. Apache-2.0 licensed: any engine can implement it, and a black-box conformance suite tells you when you have.

👀 A quick look

# Get a token, open a session, run a query, read the result with pyarrow.
TOKEN=$(curl -s https://db.example.com/chargewire/v1/auth/token \
  -H 'Content-Type: application/json' \
  -d '{"grant_type":"password","username":"joe","password":"…"}' | jq -r .access_token)

SESSION=$(curl -s https://db.example.com/chargewire/v1/sessions \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{}' | jq -r .session_id)

curl -s https://db.example.com/chargewire/v1/sessions/$SESSION/statements \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -H 'Accept: application/vnd.apache.arrow.stream' \
  -d '{"sql":"SELECT 42 AS answer"}' > result.arrows

python -c "import pyarrow.ipc as ipc; print(ipc.open_stream(source='result.arrows').read_all())"

Every server describes itself at GET /chargewire, so a client knows what it may use:

{
  "protocol": "chargewire",
  "protocol_versions": ["1"],
  "server": {"name": "chargewire-duckdb", "version": "2.0.0 (DuckDB v2.0.0)"},
  "capabilities": ["prepared_statements", "transactions", "ingest", "partitioned_ingest",
                   "long_poll", "ipc_compression_zstd", "ipc_compression_lz4_frame",
                   "result_partitions", "openapi"],
  "auth": {"grant_types": ["password"]},
  "links": {"openapi": "/chargewire/v1/openapi.json"}
}

and serves the protocol's OpenAPI 3.1 description at GET /chargewire/v1/openapi.json, for API browsers and client generators.

(Both run against chargewire-duckdb, the reference server in cpp/.)

🔄 How it works

sequenceDiagram
  participant C as Client
  participant S as Chargewire server
  C->>S: POST /chargewire/v1/auth/token
  S-->>C: bearer token
  C->>S: POST /chargewire/v1/sessions
  S-->>C: session_id
  alt short query: one round trip
    C->>S: POST …/statements  (Accept: Arrow stream)
    S-->>C: Arrow IPC stream
  else long query: submit, poll, fetch
    C->>S: POST …/statements  (Prefer: respond-async)
    S-->>C: 201 statement document
    C->>S: GET …/statements/{id}  (long-poll)
    S-->>C: succeeded: result partitions
    C->>S: GET …/statements/{id}/result?from_batch=N
    S-->>C: Arrow IPC stream (resumable)
  end

📦 What the protocol covers

	Spec
🔎 Discovery, health and capabilities	§3
📜 An OpenAPI 3.1 description, served at GET /chargewire/v1/openapi.json	§3.3, openapi.yaml
🔑 Token sign-in (password, token exchange), browser sign-in	§4
🗂️ Sessions with options, and transactions	§5
▶️ Statements: streamed or submit/poll/fetch, positional and named bind parameters	§6
♻️ Retained, resumable results; errors mid-stream; idempotent submission	§6.5–6.7
🛑 Cancellation (statement and session) and timeouts	§6.8–6.9
🔀 Partitioned results, read concurrently over several connections, in order or not	§6.10
🧪 Prepared statements with JSON or Arrow parameter batches	§7
📥 Streaming bulk ingest	§8
🗜️ Arrow buffer compression, lz4_frame or zstd, negotiated by preference	§8.2

以上為 README 的開頭部分。其餘內容請閱讀原文。 檢視完整 README ↗