Xata - Open-source Postgres platform with CoW branching

Xata is a pure open-source, cloud-native, platform for self-hosting a large number of Postgres instances on top of Kubernetes, offering the following functionality (and more) built in:

• Fast branching using Copy-on-Write at the storage level. You can “copy” TB of data in a matter of seconds
• Scale-to-zero functionality (remove compute instance on inactivity, add it back automatically on connections)
• Auto-scaling of the compute nodes, bin-packing them for cost efficiency
• High-availability / Read-replicas with automatic failover
• Upgrades, configuration edits, scale up/down with minimal downtime via switchover
• Separation of storage and compute as well as local storage if preferred
• PITR backups to object storage
• Serverless driver (SQL over HTTP/websockets)
• REST APIs and CLI for managing all “control-plane” operations, authenticated with API keys. API Keys support granular RBAC.

It is stable, actively developed, and used in production at large scale already. It is powering the Xata Cloud service, which we recommend if you prefer a managed service.

Use cases

There are two major use cases for self-hosting the Xata open source platform:

• create your own internal Postgres-as-a-Service for your company. In this case Xata provides a more opinionated and more feature-rich solution than using a K8s operator directly.
• create preview, testing, and dev environments. By taking advantage of the copy-on-write storage and automatic scale-to-zero functionality, you can create “copies” of Postgres instances with TB of data in a matter of seconds and with incredible cost efficiency.

When NOT to use Xata open source

• If you just need a single Postgres instance. Xata is running on top of a Kubernetes cluster and would be overkill in that case. You can either simply self-host Postgres on your own, or use a managed Postgres service, like Xata Cloud.
• If you want to create a public Postgres-as-a-Service to offer to end customers. While the license allows this use case (nor do we have any intention to change it in the future), we don’t recommend using it like this because we have kept closed-source some security features related to adversarial multi-tenancy. If you’d like to build a self-hosted PGaaS to offer to end customers, please contact us, likely the Xata BYOC offering is right for you.

Xata is built on top of two key open-source cloud-native technologies:

• CloudNativePG is a Postgres operator for Kubernetes. It handles most of the typical production concerns: high-availability, failover/switchover, upgrades, connection pooling, backups, etc.

On top of these two, the Xata platform adds:

• SQL gateway, responsible for routing, ip filtering, waking up scaled-to-zero clusters, serving the serverless driver over HTTP/websockets, etc.
• Branch operator managing all resources related to a branch.
• clusters and projects services which form the control-plane and serve the REST APIs.
• Auth service, based on Keycloak.
• CLI that makes use of the REST API.
• Scale-to-zero CNPG plugin for automatically hibernating branches on inactivity.

For more details about the Xata architecture, see this blog post.

Quick start guide & development

To run locally, you need the following prerequisites:

• Docker
• Kind
• Tilt

Step 1: create kind cluster

kind create cluster --wait 10m

Step 2: deploy via tilt

tilt up

Wait until all resources are up. This might take a long time when you run it for the first time, because it needs to download all the required images. Next time you start it, it should be much quicker.

Step 3: install and authenticate the xata CLI:

# Install CLI
curl -fsSL https://xata.io/install.sh | bash

# Authenticate to a local profile. Sign in with email [email protected] and password Xata1234!
xata auth login --profile local --issuer http://localhost:8080/realms/xata --api-url http://localhost:5001 --client-secret devsecret

# Set the profile
xata auth switch local

Step 4: Create the first project and branch

# Create project and main branch
$ xata project create --name my-project

# Create a child branch
$ xata branch create

License

Apache 2

Getting help

If you are considering adopting Xata in your organization, we'd love to hear from you and help you. Please open either a GitHub discussion or contact us by email.