The Table Access Protocol is the IVOA standard for querying tabular
data — astronomical catalogues as well as ordinary database tables — over a
uniform web service. This is a complete TAP 1.1 implementation for the SKA
Regional Centre Network, built as two stateless services over one PostgreSQL
database.
Why this exists
- PostgreSQL is the only coordination point. The job queue is
FOR UPDATE SKIP LOCKED on a table, so there is no broker, no cache tier and
no service registry to operate — and executors scale out with no leases and
no split-brain.
- User ADQL never runs with service privileges. Queries are parsed as ADQL,
checked against
TAP_SCHEMA, then executed under SET LOCAL ROLE tap_reader
with a statement timeout. An ADQL query cannot write, whatever it manages to
express.
- Everything streams. Rows leave a streamed statement in server-side
chunks and become HTTP
chunks, so a ten-million-row result never exists in memory. Parquet and
Arrow are first-class formats carrying units, UCDs and descriptions as
field metadata — not exports bolted on afterwards.
- Metadata domains are plugins. A pydantic model package binds to a SQL
schema and a JSON mount point; the shared machinery generates the tables,
migrates them as the model grows, and registers them in
TAP_SCHEMA so
ingested metadata is queryable by ordinary ADQL.
Quickstart
docker compose up --build -d
./scripts/smoke_test.sh # availability, tables, sync + async round trip
Query it with curl:
curl "http://localhost:8080/tap/sync" \
--data-urlencode "LANG=ADQL" \
--data-urlencode "QUERY=SELECT TOP 5 * FROM ska.continuum_sources" \
--data-urlencode "RESPONSEFORMAT=csv"
…or with any IVOA client, which needs no configuration beyond the URL:
import pyvo
svc = pyvo.dal.TAPService("http://localhost:8080/tap")
print(svc.search("SELECT TOP 5 * FROM ska.continuum_sources").to_table())
Cone search, UWS jobs, Parquet output, TOPCAT and metadata ingest are in the
Quickstart guide.
Standards implemented
| Standard | Surface |
|---|
| TAP 1.1 | /sync, /async, TAP_SCHEMA, UPLOAD (inline multipart and http(s)) |
| ADQL 2.1 | Parsed and translated by a vendored queryparser-python3 fork; geometry becomes pg_sphere expressions |
| UWS 1.1 | Complete job lifecycle, including WAIT blocking, AFTER filtering, and ABORT that cancels the running statement |
| VOSI | /capabilities (TAPRegExt), /availability, /tables (VODataService) |
| DALI | Parameter conventions, error VOTables, /examples |
| VOTable 1.4 | hand-written streaming writer; plus CSV, TSV, JSON, Parquet, Arrow |
| VOResource | /tap/registry record for registry harvesting |
| AuthVO | Challenges naming the IAM, so a client can go and get a token |
Endpoint-by-endpoint detail with every parameter is in the API reference.
A JSON interface for machine-to-machine use lives alongside the
standards-mandated XML at /api/v1 — see the JSON API.
Deploy
helm upgrade --install egernia charts/egernia \
--namespace egernia --create-namespace \
--set tapApi.baseUrl=https://tap.example.org/tap
helm test egernia -n egernia # in-cluster VOSI + sync smoke test
The chart covers both services, an optional in-chart PostgreSQL, anti-affinity
and PodDisruptionBudgets, opt-in autoscaling and scheduled backups. External HA
PostgreSQL, PITR, restore and hardening are in the deployment guide.
[!IMPORTANT]
Authentication is off by default. A deployment that configures no IAM is
fully anonymous, including the mutating metadata endpoints, so it must not be
exposed to untrusted networks. See Authentication.
Documentation
| |
|---|
| Quickstart | Run it locally, query it from curl, PyVO and TOPCAT |
| Architecture | The two-service shape, request paths, failure behaviour |
| API reference · JSON API | Every endpoint and parameter |
| Deployment · Autoscaling | Helm values, scaling, backup, restore |
| Authentication · VO Registry | Tokens, gated operations, registration |
| Metadata plugins · Model schemas | Binding a pydantic model to a queryable schema |
| Observability · Benchmarking | Metrics, tracing, how the numbers were measured |
| Development · Roadmap |
Contributing
Contributions are welcome — see CONTRIBUTING.md for the
development loop and CODE_OF_CONDUCT.md for the standards
we hold each other to. Questions and bug reports belong in
GitHub issues.
To report a security vulnerability, follow SECURITY.md rather
than opening a public issue.
Citing
If this software supports published work, please cite it — GitHub builds the
citation from CITATION.cff via Cite this repository.
License
MIT © Michele Delli Veneri and contributors.
