ReplicaDB is a high-performance, open-source command-line tool for bulk data replication between heterogeneous databases. It enables efficient ETL/ELT workflows by transferring data in parallel between Oracle, PostgreSQL, MySQL, MongoDB, SQL Server, and other databases without requiring database agents or triggers.
ReplicaDB supports a wide range of data sources including relational databases (Oracle, PostgreSQL, MySQL, MariaDB, SQL Server, SQLite, IBM DB2 LUW and DB2 for i), NoSQL databases (MongoDB), data virtualization platforms (Denodo), file formats (CSV), cloud storage (Amazon S3), and streaming platforms (Kafka). Any JDBC-compliant database is also supported with some limitations.
The managed server provides redacted per-run diagnostics through the runs API,
including bounded multiline logs and exception stack traces. These logs are
limited to 256 KiB and are separate from standalone CLI logging; treat them as
sensitive operational data.
The tool is cross-platform compatible with Windows, Linux, and macOS, and leverages parallel data transfer for optimal performance and system utilization during large-scale data migrations and synchronization tasks.
Why ReplicaDB
ReplicaDB addresses common gaps in existing database replication tools by providing:
- Open Source: Transparent development and community-driven improvements
- Cross-Platform: Java-based solution compatible with Linux, Windows, and macOS
- Heterogeneous Support: Works with SQL, NoSQL, and persistent stores like CSV, Amazon S3, or Kafka
- Simple Architecture: Standalone command-line tool without requiring database agents
- High Performance: Optimized for bulk data transfer with large datasets
- Non-Intrusive: Focused on batch replication without requiring database triggers or CDC installation
Comparison with Alternatives
Common alternatives and how ReplicaDB differs:
- SymmetricDS: A comprehensive CDC solution with database triggers. While feature-rich, it requires installation and maintenance of capture tables in source databases, making it more intrusive for batch replication scenarios.
- Sqoop: Designed specifically for Hadoop ecosystems, limiting its use in other environments where Hadoop infrastructure is not available.
- Pentaho and Talend: Full-featured ETL platforms that require custom development for each replication job, increasing complexity and maintenance overhead for straightforward data transfer tasks.
Feature Comparison
| Feature | SymmetricDS | Sqoop | Pentaho/Talend | ReplicaDB |
|---|
| Database Agents Required | Yes | No | No | No |
| Triggers in Source DB | Yes | No | No | No |
| Heterogeneous Databases | Limited | No | Yes | Yes |
| Hadoop Requirement | No | Yes | No | No |
| Custom Development per Job | Low | Low | High | None |
| Parallel Transfer | Yes | Yes | Yes | Yes |
| Open Source | Yes | Yes | Yes | Yes |
Installation
Prerequisites
Before installing ReplicaDB, ensure you have:
- Java Runtime: Java JDK or JRE 17 or higher installed and configured
- Network Connectivity: Reliable network access to both source and sink databases
- Database Credentials: Appropriate permissions on both databases:
- Source database: SELECT permissions on tables to replicate
- Sink database: INSERT, UPDATE, DELETE, and CREATE TABLE permissions
- (Optional) Docker or Podman for containerized deployment
Choose a release
ReplicaDB has two separate releases:
| CLI | Server |
|---|
| Use it for | Direct transfers from scripts or a terminal | Shared jobs, schedules, users, audit, and run history |
| Download | ReplicaDB-1.0.3.tar.gz or .zip | ReplicaDB-server-1.0.3.tar.gz or .zip |
| PostgreSQL | Only the source and sink databases | Embedded in local; external in api and worker |
| State | REPLICADB_HOME | REPLICADB_SERVER_HOME |
| Interface | CLI | Authenticated API/frontend and private worker health endpoint |
Use the CLI for one replication at a time. It is Spring-free and keeps
its existing options-file and exit-code behavior.
Use the server for managed, shared, or distributed replication. Its
local mode is a durable single-node install; api and worker use external
PostgreSQL. It does not migrate CLI files or state.
The server package requires Java 17, but not Maven, npm, Docker, or a system
PostgreSQL installation. Extract it and start the local server:
tar -xzf ReplicaDB-server-1.0.3.tar.gz
cd ReplicaDB-server-1.0.3
./bin/replicadb-server start local
./bin/replicadb-server status
The first local start downloads and verifies the platform PostgreSQL bundle.
Set REPLICADB_BOOTSTRAP_ADMIN_USERNAME and
REPLICADB_BOOTSTRAP_ADMIN_PASSWORD for automation, or answer the hidden
prompt from an interactive terminal. The server home defaults to
~/.replicadb; set REPLICADB_SERVER_HOME to change it. Keep the keyring and
data/postgresql together when backing up. Warm-cache restarts do not need
network access.
For source builds and development profiles, see
CONTRIBUTING.md, the
server quickstart,
and replicadb-server/README.md.
Local single-node server without Docker
For a durable local installation, replicadb-server can manage a native
embedded PostgreSQL process. This mode uses the same PostgreSQL repositories,
Flyway migrations, Quartz scheduler, encrypted datasource catalog, and local
job execution as the api profile; it does not start a separate worker.
The extracted package is the recommended durable local installation:
This is the beginning of the README. Read the rest at the original. Read the full README ↗
