neuron-explorer
Qwen3.5‑4B 전용 조향형 뉴런 탐색기, 모든 MLP 뉴런을 밀어 결과를 보관·탐색
FerricStore는 소프트웨어가 다단계 작업을 추적·충돌 후 저장된 진행 상태에서 이어 실행하도록 도와줍니다
FerricStore helps software keep track of multi-step tasks and pick up from saved progress after a crash. See how it works with simple, interactive demos.
개발 도구Elixir · RustGitHub @ferricstore온라인운영자 미확인
써 보셨어요? 한 번만 눌러 주세요. 로그인은 필요 없어요.
Durable KV and workflow state engine
저장소 README에서 가져옴
README 전체를 불러오는 중입니다.
README 전문 보기 ↗FerricStore is an open-source durable workflow and queue server.
Applications run ordinary handler code in their own services. FerricStore persists one Flow for each job or process, makes named states claimable under leases and fencing, and records every transition, retry, signal, and terminal outcome.
create -> claim a state -> run handler -> transition / retry / complete / fail
Start with a durable queue. Evolve the same Flow into a multi-service state machine, then add bounded queries, schedules, policies, and governance as the workflow grows. FerricStore does not replay handler code or require application code to run inside the server.
FerricFlow runs on the same sharded, Raft-backed engine that FerricStore exposes as a durable key-value and data-structure store.
Run one development node:
docker run -p 6388:6388 -p 6380:6380 -p 6381:6381 \
-e FERRICSTORE_PROTECTED_MODE=false \
-v ferricstore_data:/data \
quay.io/ferricstore/ferricstore:0.12.15
Published SDKs connect to ferric://127.0.0.1:6388. The operations dashboard
is at http://127.0.0.1:6380/dashboard, metrics are at /metrics, and isolated
liveness/readiness probes use port 6381.
The same release also contains an opt-in HTTP/HTTPS command and invocation API. It is disabled by default; see HTTP and HTTPS API for TLS, ACL, Docker, and client configuration.
FERRICSTORE_PROTECTED_MODE=false is for local development only. With
protected mode enabled, bootstrap the first durable ACL user at
/dashboard/setup, then sign in at /dashboard/login. See
Getting Started for SDK setup and
Security before exposing a server.
The FQL1 query planner, schedules, governance primitives, and operations dashboard are included in this OSS repository and release image.
| Area | Included in FerricStore OSS 0.12.15 |
|---|---|
| Durable KV and data structures | Strings, hashes with field TTL, lists, sets, sorted sets, streams and consumer groups, Pub/Sub, transactions, blocking reads, bitmaps, HyperLogLog, GEO, Bloom and Cuckoo filters, Count-Min Sketch, TopK, and TDigest. |
| FerricStore-native helpers | Compare-and-swap, fenced distributed locks, fixed-window rate limiting, cache-aside/stampede protection, key diagnostics, quotas, and cluster inspection. |
| Durable queues and workflows | FLOW.CREATE, batched create/mutation commands, fused FLOW.START_AND_CLAIM, FLOW.STEP_CONTINUE, and FLOW.RUN_STEPS_MANY, state-specific FLOW.CLAIM_DUE, leases and fencing, transitions, retry, complete, fail, cancel, reclaim, rewind, history, and retention cleanup. |
| Query engine | FLOW.QUERY with FQL1 point, collection, exact-count, history, lineage, metadata, and lease-deadline queries; bounded indexes, field projection, cursors, statistics, EXPLAIN, EXPLAIN ANALYZE, and FLOW.QUERY.INDEXES. |
| Scheduling | Durable one-shot, delayed, interval, and cron schedules; pause/resume/delete, bounded interval catch-up, overlap policies, maximum fires, automatic due execution, and explicit administrative firing. |
| Policies and lifecycle | Type policy generations with compare-and-swap, per-state FIFO or parallel execution, fixed/exponential retry policies, exhaustion routing, maximum active lifetime, indexed attributes/state metadata, and resumable policy migrations. |
| Workflow coordination | Durable signals, parent/child fanout, lineage, named value refs, state metadata, idempotent creation, and independently claimable workflow states. |
| FlowGuard governance | Durable effect reservations, approvals, circuits, budgets, strict leased concurrency limits, per-Flow governance ledgers, global overview/list surfaces, and structured denials. |
| Operations | Browser dashboard, local operational Mix CLI, Flow query and index inspection, schedules, policies, governance, failures, retention, storage, keyspace, clients, streams, Pub/Sub, slow log, Raft/consensus, doctor diagnostics, health probes, and Prometheus metrics. |
| Security | Protected mode, named ACL users, command/key/channel rules, dashboard bootstrap/login and ACL-scoped accounts, TLS/mTLS, trusted-proxy controls, CSRF/origin validation, login throttling, session revocation, and audit logging. |
partition_key is an application routing, co-location, and bounded-query
key. It does not imply a separate HTTP security namespace.The published SDKs use the same Ferric native protocol. Their current release
lines are covered by the OSS server's 0.12.15 compatibility CI:
| Interface | Package or module | Source |
|---|---|---|
| Python | ferricstore | ferricstore/ferricstore-python |
| Go | github.com/ferricstore/ferricstore-go | ferricstore/ferricstore-go |
| Elixir native client | ferricstore_sdk | ferricstore/ferricstore-elixir |
| TypeScript / Node.js | @ferricstore/ferricstore | ferricstore/ferricstore-typescript |
| Java 17+ | io.github.ferricstore:ferricstore-java | ferricstore/ferricstore-java |
| HTTP/HTTPS API | /v1/commands, /v1/invocations, and scoped values | This repository |
| Embedded Elixir server API | ferricstore | This repository |
| Local operational CLI |
All five native SDK release lines retain FerricStore 0.11.4 as their
minimum server version and negotiate the Stream mode-34 producer fast path when
the server advertises it. Native wire protocol v1 is unchanged.
FerricStore is currently a 0.x beta release. The core durability path, Flow
commands, precompiled NIFs, Docker image, and SDKs are published and usable, but
public APIs, command details, operational defaults, and storage/projection
internals may still change before 1.0.
Beta does not mean lightly tested. The project is thoroughly tested across the durability path, FerricFlow workflow commands, FQL1 and index lifecycle, durable schedules, governance, Rust NIFs, storage behavior, security, dashboard/API surfaces, the embedded API, published SDK integration suites, restart/recovery, cluster/quorum behavior, benchmarks, and longer soak runs.
Use it today for development, benchmarks, pilots, and controlled production
experiments. For critical production workloads, pin exact versions, test
upgrades on your data model, and expect compatibility guarantees to harden with
the 1.0 release line.
A Flow is one durable workflow record:
| Field | Meaning |
|---|---|
type | Workflow or queue type, such as email or order. |
id | Application-defined Flow id. |
partition_key | Routing/co-location boundary required by bounded collection queries and FIFO lanes. |
state | Current durable state, such as queued, created, or charged. |
attributes | Small indexed metadata for Flow query, stats, and dashboard filters. |
state_meta | Metadata retained independently for each logical state; one policy-selected key can be indexed. |
payload / value refs | Small routing payload plus optional named values stored separately. |
| lease | Worker claim ownership with fencing. |
| history | State changes, signals, retries, and terminal events. |
| lineage | Parent, root, and correlation identities for fanout and cross-service inspection. |
| terminal status | Completed, failed, cancelled, or still active. |
The core loop is explicit:
FLOW.CREATE -> FLOW.CLAIM_DUE -> handler -> FLOW.TRANSITION / COMPLETE / FAIL / RETRY
Queue workers usually process one state and complete/fail/retry. Workflow workers process multiple named states and return explicit transitions.
A long workflow does not need to live in one codebase or one workflow runtime. One service can claim fraud_check, another can claim charge_card, and another can claim send_email; the work moves by durable state transition instead of ad hoc queue messages, retry tables, and status tables.
FerricStore ships operational Mix tasks for source checkouts and embedded Elixir deployments. These tasks start and operate on the default local FerricStore application; they are not a standalone shell for a remote native TCP server. Use a published SDK or the operations dashboard for remote access.
| Task | Purpose |
|---|---|
mix ferricstore.info | Show uptime, shard/key counts, memory, connection/command counters, and Raft leader status. |
mix ferricstore.keys [glob] | Stream keys with optional */? filtering through bounded cursor pages. |
mix ferricstore.config get/set ... | Inspect or update local namespace commit-window configuration. |
mix ferricstore.merge [shard] | Trigger the normal merge/compaction eligibility check for one or all local shards. |
mix ferricstore.redis_compat matrix/assess ... | Generate a Redis compatibility matrix or assess a captured workload in Markdown or JSON. |
mix ferricstore.recovery_kill9 ... | Run the manual child-process kill/restart recovery benchmark; this is a diagnostic benchmark, not routine administration. |
Run mix help ferricstore.info or the corresponding task name for its complete
arguments and behavior.
FerricFlow commands are exposed over FerricStore's native binary TCP protocol, so SDK clients can use multiplexed lanes, request ids, ACLs, TLS, and bounded backpressure. Durability is the default contract: a workflow command returns success only after the state change is accepted through the quorum path and written to disk.
Durable queue item:
FLOW.CREATE email-1 TYPE email STATE queued PAYLOAD "welcome:user-1"
FLOW.CLAIM_DUE email STATE queued WORKER worker-1 LIMIT 100
FLOW.COMPLETE email-1 <lease-token> FENCING <fencing-token> RESULT "sent"
Explicit state transition:
FLOW.CREATE order-1 TYPE order STATE created PAYLOAD "order payload"
FLOW.CLAIM_DUE order STATE created WORKER worker-1 LIMIT 1
FLOW.TRANSITION order-1 running charged LEASE_TOKEN <lease-token> FENCING <fencing-token>
FLOW.CLAIM_DUE order STATE charged WORKER worker-1 LIMIT 1
FLOW.COMPLETE order-1 <lease-token> FENCING <fencing-token> RESULT "ok"
Flow commands and normal FerricStore commands can be pipelined on the same native connection.
Latency-sensitive workers can use FLOW.START_AND_CLAIM and
FLOW.STEP_CONTINUE to combine state mutation with the next lease, while
FLOW.RUN_STEPS_MANY executes a bounded same-partition batch. These are durable
server commands, not client-side emulation.
FLOW.QUERY is the canonical read and query envelope for Flow state. FQL1 is
SQL-shaped, but it is purpose-built for bounded workflow queries and exposes
only plans advertised by the server capability manifest.
FLOW.QUERY FQL1 "FROM runs WHERE partition_key = @partition AND type = @type AND state IN ('queued', 'retrying') ORDER BY updated_at_ms DESC LIMIT 100 RETURN RECORDS (run_id, state, updated_at_ms, attribute['region'])" partition tenant-a type order
FLOW.QUERY FQL1 "FROM runs WHERE partition_key = @partition AND type = @type RETURN COUNT" partition tenant-a type order
FLOW.QUERY FQL1 "EXPLAIN ANALYZE FROM runs WHERE partition_key = @partition AND type = @type AND state = 'failed' ORDER BY updated_at_ms DESC LIMIT 50 RETURN RECORDS" partition tenant-a type order
FLOW.QUERY.INDEXES
The OSS query engine includes:
IN, BETWEEN, time windows, NULL, MISSING, typed parameters,
stable ordering, bounded limits, and opaque cursor pagination;attribute.* and
state_meta.*, to avoid decoding and transferring fields the caller did not
request;EXPLAIN, and ACL-controlled EXPLAIN ANALYZE;FLOW.QUERY.INDEXES and the dashboard; andflow_query_result_v1 compact typed-binary result codec, with
the ordinary typed-value codec retained as a lossless fallback.Current query state is checked against authoritative Flow storage where the selected plan requires hydration. Metadata-only and covered projections can be served directly from the compact QueryRow/index snapshot. Every scan, result page, decoded byte count, native response, memory reservation, and deadline is bounded.
The beta commands FLOW.LIST, FLOW.SEARCH, FLOW.TERMINALS,
FLOW.FAILURES, FLOW.STUCK, FLOW.BY_PARENT, FLOW.BY_ROOT, and
FLOW.BY_CORRELATION are not supported. SDK convenience methods compile those
use cases to FLOW.QUERY; new integrations should use FQL1 directly when they
need control over predicates, fields, order, pagination, counts, or explain
output.
See the Flow Query Guide for the grammar, result contract, index lifecycle, ACL model, diagnostics, and tuning.
Schedules are durable Flow records. FerricStore supports one-shot timestamps, relative delays, fixed intervals, and cron expressions with timezones.
FLOW.SCHEDULE.CREATE billing-sweep KIND interval EVERY_MS 60000 CATCHUP_POLICY fire_once OVERLAP_POLICY queue_after_previous TARGET <typed-map>
FLOW.SCHEDULE.GET billing-sweep
FLOW.SCHEDULE.LIST KIND interval STATE active COUNT 100
FLOW.SCHEDULE.PAUSE billing-sweep
FLOW.SCHEDULE.RESUME billing-sweep
FLOW.SCHEDULE.DELETE billing-sweep
The built-in scheduler performs automatic due execution. Interval recovery uses
bounded O(1) fire_once catch-up: it creates one target, records how many
occurrences were coalesced, and advances to the next interval without replaying
an unbounded burst. Overlap policies, optional queue-after-previous retry,
maximum fires, end times, and planning failures are persisted and inspectable.
FLOW.SCHEDULE.FIRE and FLOW.SCHEDULE.FIRE_DUE are available for explicit
administration and custom-scheduler deployments.
FLOW.POLICY.SET installs a replicated type policy. Policy updates merge by
default, return a monotonic generation, and support compare-and-swap with
EXPECTED_GENERATION; REPLACE TRUE performs an intentional full replacement.
FLOW.POLICY.SET order MAX_ACTIVE_MS 300000 INDEXED_ATTRIBUTES region \
MAX_RETRIES 5 BACKOFF exponential BASE_MS 100 MAX_MS 30000 \
STATE queued MODE FIFO STATE review MODE PARALLEL
Policies cover fixed or exponential retry and exhaustion routing, maximum non-terminal lifetime, indexed attributes, one indexed state-metadata key, and per-state FIFO or parallel execution. FIFO ordering is scoped to a type/state/partition lane, survives restart and hibernation, and cannot be overridden by an individual create or claim. Policy/index migrations are durable, bounded, resumable, and ordered by policy generation.
FlowGuard keeps workflow-side control state in FerricStore rather than an external coordination database:
FLOW.EFFECT.* reserves, confirms, fails, compensates, and reads fenced
side-effect attempts with idempotency information.FLOW.APPROVAL.* requests, approves, rejects, gets, and lists durable human or
service approvals.FLOW.CIRCUIT.* opens, closes, and reads durable circuit state.FLOW.BUDGET.* reserves, commits, releases, gets, and lists fixed-window
budgets.FLOW.LIMIT.* leases, spends, releases, gets, and lists durable strict global
concurrency credits without a cross-shard transaction on every claim.FLOW.GOVERNANCE.LEDGER and FLOW.GOVERNANCE.OVERVIEW expose per-Flow audit
and global operational summaries.Governance enforcement is fail-closed and structured, but its claim/terminal hot-path integration is opt-in: callers provide the governance limit scope and shard identity when strict running capacity should be spent and released. Workloads that do not enable governance do not pay governance work on ordinary Flow claims. See FerricFlow Governance for the implemented surface and enforcement boundaries.
Install:
pip install ferricstore
Durable queue:
from ferricstore import QueueClient
client = QueueClient.from_url("ferric://127.0.0.1:6388")
emails = client.queue(type="email")
emails.enqueue("email-1", payload=b"welcome:user-1", idempotent=True)
def send_email(job):
print(job.id, job.payload)
return b"sent"
emails.worker(concurrency=10, batch_size=100).run(send_email)
Explicit state-machine workflow:
from ferricstore import WorkflowClient, complete, transition
client = WorkflowClient.from_url("ferric://127.0.0.1:6388")
order = client.workflow(type="order", initial_state="created")
@order.state("created")
def created(job):
charge_card(job.payload)
return transition("charged")
@order.state("charged")
def charged(job):
send_receipt(job.id)
return complete(result=b"ok")
order.start("order-1", payload=b"order payload", idempotent=True)
order.worker(states=["created", "charged"]).run()
The SDK handles claim leases and fencing. Handlers return durable outcomes such as transition(...), complete(...), retry(...), or fail(...).
Python SDK links:
Signals record external events durably and can optionally move a Flow to another state.
from ferricstore import WorkflowClient
client = WorkflowClient.from_url("ferric://127.0.0.1:6388")
approval = client.workflow(type="approval", initial_state="waiting")
approval.start("approval-1", payload=b"invoice:123", idempotent=True)
approval.signal(
"approval-1",
signal="approved",
if_state="waiting",
transition_to="approved",
idempotency_key="approve-approval-1",
)
Named values let a Flow store large or optional bytes separately from hot state. Workers hydrate only the values they ask for.
from ferricstore import QueueClient
client = QueueClient.from_url("ferric://127.0.0.1:6388")
orders = client.queue(type="order")
orders.enqueue(
"order-1",
payload=b"small routing bytes",
values={"invoice": invoice_pdf_bytes, "customer": customer_snapshot_bytes},
)
orders.worker(claim_values=["customer"]).run(handle_customer_step)
A parent Flow can spawn child Flows. Children run independently with their own state, retries, leases, history, and terminal status; parent/child links are queryable later.
from ferricstore import ChildSpec, WorkflowClient, transition
client = WorkflowClient.from_url("ferric://127.0.0.1:6388")
campaign = client.workflow(type="campaign", initial_state="dispatch")
@campaign.state("dispatch")
def dispatch(job):
job.flow.spawn_children(
[
ChildSpec(
id=f"device:{device_id}:cmd:{job.id}",
type="device-command",
payload=device_id.encode(),
)
for device_id in device_ids
],
wait_state="done",
)
return transition("waiting_for_children")
FLOW.CREATE, transition, retry, complete, fail, or cancel returns success.FLOW.CLAIM_DUE grants a lease token and fencing token to a worker.The OSS server includes a browser operations dashboard on the combined HTTP port. It is an operational surface, not a separate control-plane service. Pages cover server overview, command/read activity, keyspace and prefixes, storage and merge state, clients, streams, Pub/Sub, slow log, configuration, capabilities, Raft/consensus, doctor diagnostics, and the full Flow surface: lookup, states, workers, due work, FQL queries, index projections, lineage, signals, schedules, policies, governance, failures, and retention.
The Flow query page includes Guided and FQL workbench modes. It accepts typed
JSON parameters, field projections, exact counts, EXPLAIN, and EXPLAIN ANALYZE; preserves query quality and resource usage; continues opaque cursors
through POST; keeps lifecycle state separate from the current workflow step;
discovers policy-indexed metadata and bounded top values for a selected
workflow type; samples observed type suggestions with a projection-only query;
and can visualize the ACL-filtered current page without issuing a second query.
Page charts are labeled as page-scoped and are not presented as global grouped
results.
In protected mode:
/dashboard/setup and then disables
setup;/dashboard/login with the same durable ACL identities
used by native TCP;/dashboard/security supports ACL-scoped named-account management;Remote dashboard access requires an explicitly trusted HTTPS proxy, trusted
proxy CIDRs, a shared session secret, and a file-backed bootstrap token. The
server also exposes Prometheus-compatible /metrics, combined-port dashboard
JSON endpoints, and isolated liveness/readiness probes. See
Security for ACL, dashboard, TLS/mTLS, audit, and proxy
configuration.
FerricStore also exposes a durable key-value/data-structure store through the native protocol and embedded API:
SET user:42:name alice
GET user:42:name
HSET order:1 status paid
ZADD due 1700000000000 flow-1
Writes go through Raft consensus and disk-backed storage before success is reported. There is no separate mode to turn persistence on.
The high-level data path is:
native SDK / embedded API
|
v
deterministic shard routing -> WARaft quorum commit -> append/apply storage
-> hot keydir/native indexes
-> async history + LMDB query projection
Authoritative Flow records remain in the Raft segment/apply-projection storage path. Hot keydir/native indexes serve current state. LMDB stores one compact QueryRow per current Flow plus composite index entries; it does not duplicate the full record or payload. A QueryRow contains bounded query-visible metadata and one checked physical log locator. Composite entries contain identity, version, expiry, and declared covering fields, but do not copy the payload or locator.
This layout lets covered FQL projections and counts avoid authoritative-log reads while keeping point, history, lineage, and payload-dependent reads bounded and validated. Compaction relocates the central QueryRow locator with a compare-and-swap instead of rewriting every composite index. Query backfill, hydration, native merge, and admission are limited by rows, bytes, memory, deadlines, and MemoryGuard pressure.
Rust NIFs implement bounded storage and codec paths. Portable grouped
pread/pwrite behavior remains available across supported platforms, while
Linux builds carry io_uring parity coverage for the applicable asynchronous I/O
paths.
| Property | How |
|---|---|
| Atomic | Each command is one Raft log entry, applied or not. |
| Consistent | Raft linearizability for committed writes. |
| Isolated | Single-threaded state machine per shard. |
| Durable | WAL, disk-backed storage, and Raft quorum before ack. |
The KV store is not just SET/GET: it also includes hashes, lists, sets,
sorted sets, durable streams, live Pub/Sub notifications, bitmaps, HyperLogLog,
GEO, probabilistic structures, CAS, locks, rate limits, and cache-aside helpers.
See Key-Value Store for the model, write/read path, TTL
behavior, hot/cold storage, streams vs Pub/Sub, and key design.
FerricStore can also run inside an Elixir application.
# mix.exs
{:ferricstore, "~> 0.12.15"}
:ok = FerricStore.set("user:42:name", "alice", ttl: :timer.hours(1))
{:ok, "alice"} = FerricStore.get("user:42:name")
FerricFlow is also available through embedded FerricStore.flow_* functions and the high-level Elixir Flow SDK.
Start here:
FerricFlow:
FLOW.* command syntax and FerricStore command behavior.Operations and reference:
See CONTRIBUTING.md. For security issues, see SECURITY.md.
Apache-2.0
| Durability and deployment | Sharded WARaft consensus, disk-backed authoritative records, restart recovery, compaction, memory pressure/admission controls, multi-node routing, Docker multi-arch images, ephemeral single-task and three-node AWS Fargate OSS profiles, Kubernetes, bare-metal releases, and embedded mode. |
mix ferricstore.* tasks |
| This repository |