Candybox is a distributed, S3-like object store written in Java. You create buckets and store
objects through a small TCP API, an S3-compatible HTTP gateway, or a command-line client, and Candybox
keeps them durable and replicated across a cluster.
Under the hood it is a distributed LSM tree built on Apache BookKeeper.
Object data and the index live in BookKeeper's replicated, append-only ledgers. A single fenced owner
per bucket partition keeps reads and writes consistent during failover, and partitions are spread
evenly across the cluster.
Vocabulary
A Box is a bucket,
a Candy is an object,
a CandyKey is an object key,
and a Syrup is a data ledger that holds object bytes.
(Candy in a box — that's the whole theme.)
Quick start
With Docker Compose (recommended)
The bundled docker-compose.yml starts ZooKeeper, 3 BookKeeper bookies, 3
Candybox nodes and the S3 gateway (on :9711) from the
zetaplusae/candybox image:
docker compose up -d
docker compose run --rm cli create-box photos
echo 'hello candybox' | docker compose run --rm -T cli put photos hello.txt
docker compose run --rm cli get photos hello.txt # -> hello candybox
Tear it down with docker compose down (add -v to also wipe the data volumes). The
quick start guide walks through this in more detail.
Kubernetes / Helm
The Candybox Helm chart provides demo and production
configuration examples, optional BookKeeper/ZooKeeper, and external-cluster support.
Build it with mvn -pl candybox-charts package; release charts are published to
oci://registry-1.docker.io/zetaplusae/candybox-chart with the Maven release version.
Download and Maven Central
Release 0.1.0 is out. The distribution archive is attached to the
GitHub Release, and the Java client is on
Maven Central:
<dependency>
<groupId>me.predatorray.candybox</groupId>
<artifactId>candybox-client</artifactId>
<version>0.1.0</version>
</dependency>
See Installation for all options.
Web dashboard
The compose stack also starts a stateless admin service with a dashboard at
http://localhost:9713/ui/. It shows cluster topology, a box browser,
LSM internals and time-series charts. The same process serves a JSON API under /api/*; see
Operations. To build the dashboard into the jar, add
-Pfrontend to the Maven build.
S3 compatibility
The gateway is tested against the ceph/s3-tests suite. With
SigV4 auth and ACLs enabled, 192 of 838 boto3 functional tests pass. Versioning, SSE, POST object,
lifecycle, bucket policy, CORS and conditional GET are not implemented yet. The per-family breakdown
is in compat/s3-tests/README.md, and the
S3 gateway docs cover usage, range GET and multipart upload.
Storing and retrieving objects
The zetaplusae/candybox image doubles as the command-line client: candybox <args> runs the CLI
instead of a storage node. Point it at a node with CANDYBOX_SERVER (or -s host:port):
alias candybox='docker run --rm -i --network candybox_default \
-e CANDYBOX_SERVER=candybox-1:9709 -v "$PWD:/data" -w /data zetaplusae/candybox candybox'
candybox create-box photos
candybox put photos cat.jpg cat.jpg --content-type image/jpeg
candybox get photos cat.jpg out.jpg
candybox head photos cat.jpg # size, content-type, checksum, metadata
candybox list photos # keys in the box
candybox help # full command list
From Java, use CandyboxClient in the candybox-client module. The
command-line client docs list every command.
Operations the sorted LSM tree makes cheap
Keys are stored sorted and object bytes sit behind small pointers, so a few operations that are
expensive in other S3-style stores are cheap here:
candybox list photos --start a --end m --reverse # bounded, reverse-order range scan
candybox copy photos cat.jpg cat-copy.jpg # zero-copy: shares the stored bytes
candybox rename photos cat.jpg pets/cat.jpg # zero-copy move
candybox delete-range photos thumbnails/ # one range tombstone, not N deletes
- Range scans walk a
[start, end) window in either direction.
copy and rename point a new key at the same stored bytes, even across partitions. A
same-partition rename is atomic; a cross-partition one is eventually atomic, so a reader may
briefly see both keys.delete-range removes a prefix or key window with a single tombstone, and compaction reclaims
the bytes later.
Configuration
The node reads conf/candybox.properties. Any key can be overridden by an environment variable named
CANDYBOX_<KEY> (dots become underscores, upper-cased), such as CANDYBOX_ZOOKEEPER_CONNECT; the
environment value wins. The most common keys:
| Key | Meaning | Default |
|---|
node.id | Cluster-unique node id. Falls back to the trailing number in $HOSTNAME. | — |
zookeeper.connect | ZooKeeper connect string. | 127.0.0.1:2181 |
server.bind | Address clients connect to. | 0.0.0.0:9709 |
server.advertised | Address published to the cluster for routing. | bind address |
health.port | HTTP port for /healthz (liveness), /readyz (503 while ZooKeeper is unreachable), /metrics. | 9710 |
This is the beginning of the README. Read the rest at the original. Read the full README ↗
