SeaweedFS

SeaweedFS is a simple and highly scalable distributed file system. There are two objectives:

1. to store billions of files!
2. to serve the files fast!

One SeaweedFS cluster serves an S3 object store, a POSIX file system, a lakehouse with S3 Tables and a built-in Iceberg REST Catalog, and vector search with S3 Vector Buckets, all on shared storage. Each blob is one disk read away, capacity grows by starting another volume server, and cloud storage can be cached or tiered transparently. Blob reads and writes have O(1) complexity and can run at the full speed supported by the underlying hardware.

• Download Binaries for different platforms
• Wiki Documentation
• HTTP REST API for the filer, master, and volume servers
• Community: Slack, Twitter, Telegram, Reddit, Mailing List
• SeaweedFS White Paper and introduction slides: 2025.5, 2021.5, 2019.3

Table of Contents

• Quick Start
  • One command
  • Docker
  • Docker Compose
  • Kubernetes with Helm
  • Build from source
  • Scale out
• Why SeaweedFS
  • Fast
  • Scalable
  • The most complete S3 API
  • A data warehouse with S3 Tables
  • Vector search with S3 Vector Buckets
  • A fast cache for cloud storage
  • Active-active replication and more
• Architecture
• Compared to Other Systems
  • Compared to HDFS

Quick Start

One command

Download the latest binary from the releases page and unzip the single weed (or weed.exe) file, or let the install script put it in /usr/local/bin:

curl -fsSL https://raw.githubusercontent.com/seaweedfs/seaweedfs/master/install.sh | bash

Then start a ready-to-use S3 object store:

AWS_ACCESS_KEY_ID=admin \
AWS_SECRET_ACCESS_KEY=secret \
S3_BUCKET=my-bucket \
./weed mini -dir=./data

That's it. The S3 endpoint is at http://localhost:8333, my-bucket exists, and admin/secret are valid credentials:

AWS_ACCESS_KEY_ID=admin AWS_SECRET_ACCESS_KEY=secret \
  aws --endpoint-url http://localhost:8333 s3 cp README.md s3://my-bucket/

The same process also runs the master, a volume server, the filer, WebDAV, the Iceberg REST catalog, and the Admin UI. Add S3_TABLE_BUCKET=warehouse to also create an Iceberg table bucket, or warehouse:LANCE for a Lance one. Drop the AWS keys to run without authentication for development.

Without Admin authentication or mTLS, weed mini binds the Admin UI/API and its worker gRPC control plane to loopback rather than -ip.bind. Set WEED_ADMIN_PASSWORD or configure https.admin mTLS to keep the network bind. Remote workers must opt in with -admin.worker.ip=<address> and should configure grpc.admin mTLS. -admin.allowInsecureBind restores the legacy unauthenticated network bind and should only be used on an isolated network.

This is the beginning of the README. Read the rest at the original. Read the full README ↗