nousergon-lib

Part of Crucible — a Nous Ergon product: a harness for rigorous AI/ML experiments in finance, an equity research-and-trading system instrumented end-to-end. Repo and S3 names use the underlying project codename alpha-engine.

Shared utility library used by all 6 modules of Nous Ergon. Cross-cutting concerns only — logging, freshness checks, trading-calendar arithmetic, ArcticDB helpers, agent-decision capture, LLM cost tracking. No proprietary trading logic, no model weights, no agent prompts.

The lib's job is to keep the same code from being maintained six times.

---

Install

# requirements.txt
nousergon-lib @ git+https://github.com/nousergon/[email protected]

Tagged releases: v0.1.0, v0.2.0, v0.3.0, v0.4.0, etc. Consumers pin to a specific tag. Breaking changes bump the minor version while Alpha Engine is in pre-1.0.

# With optional extras
pip install "nousergon-lib[arcticdb] @ git+https://github.com/nousergon/[email protected]"

Extra	Pulls in	When you need it
[arcticdb]	arcticdb, pandas	Anything that calls check_arcticdb_fresh or the ArcticDB read/write helpers
[flow_doctor]	flow-doctor	Logging integration that escalates ERROR-level events to flow-doctor
[rag]	psycopg2-binary, pgvector, numpy	The rag submodule — Neon pgvector RAG retrieval/ingestion
[rag-parquet]	pandas, pyarrow	rag.parquet_mirror — S3 parquet batch-tier mirror written at ingest time
[rag-local-ann]	pandas, pyarrow, hnswlib	rag.local_ann — in-process HNSW index over the parquet tier for batch consumers (no Neon egress)
[dev]	pytest, lint tooling	Local development

Modules

logging — structured logging + flow-doctor attach

Replaces the near-identical log_config.py copies that used to live in alpha-engine-data and alpha-engine-executor. Consumers call setup_logging once at process startup:

from nousergon_lib.logging import setup_logging

setup_logging("data-collector", flow_doctor_yaml="/path/to/flow-doctor.yaml")

• Text mode by default; JSON via ALPHA_ENGINE_JSON_LOGS=1
• Flow-doctor is default-on (0.58.0+): it attaches an ERROR-level handler whenever a flow_doctor_yaml is provided (requires [flow_doctor] extra). FLOW_DOCTOR_DISABLED=1 is the kill switch; test runs auto-disable unless FLOW_DOCTOR_ALLOW_IN_TESTS=1. Deployed runtimes (Lambda, or ALPHA_ENGINE_DEPLOYED=1) fail loud on misconfig; local dev/CI logs a WARNING and skips.
• Wrap an entrypoint in guard_entrypoint() (context manager) or @monitor_handler (Lambda decorator) so an uncaught raise is also captured — not just logger.error() records. Both no-op when flow-doctor is inactive.

preflight — fail-fast connectivity + freshness checks

Runs at the top of every entrypoint, before any real work starts. Primitives live on BasePreflight; each consumer subclasses and overrides run():

from nousergon_lib.preflight import BasePreflight

class DataPreflight(BasePreflight):
    def __init__(self, bucket, mode):
        super().__init__(bucket)
        self.mode = mode

    def run(self):
        self.check_env_vars("AWS_REGION")
        if self.mode == "phase1":
            self.check_env_vars("FRED_API_KEY", "POLYGON_API_KEY")
        self.check_s3_bucket()
        if self.mode == "daily":
            self.check_arcticdb_fresh("universe", "SPY", max_stale_days=4)

Failed checks raise RuntimeError with an explanatory message. Consumers catch nothing — the raise propagates up through main() → non-zero exit → Step Function HandleFailure → flow-doctor notification. The point is to fail before paying for any LLM calls or downstream work.

arcticdb — read/write helpers + symbol enumeration

Wrappers around the ArcticDB Python client. Standardizes the URI format, library naming, and read paths so each consumer doesn't reinvent the connection logic.

dates — trading-day arithmetic

now_dual() returns a (calendar_date, trading_day) pair following the rule trading_day = last_closed_trading_day(now). Strictly backward-looking; never ahead. session_for_timestamp(ts) resolves any timestamp to its trading session. Used at every artifact-write site to prevent calendar/trading-day drift between modules.

trading_calendar — NYSE holiday detection

Pure-Python NYSE calendar through 2030. No pandas-market-calendars dependency.

env — environment registry resolver (environment is one setting)

The only reader of the fleet environment registry (environments.yaml, a packaged copy of the governance file). Resolves the environment from NE_ENV (default prod) and returns its account, region, buckets, stores, stacks, topics, instances, SSM prefixes, LLM cap, universe and schedule times (ET) by role, so code never names a literal. Unknown environments, roles and registry keys raise. NE_ENV_REGISTRY points it at another registry file. CLI: python -m nousergon_lib.env check|parity|show.

from nousergon_lib import env
e = env.resolve()            # NE_ENV, else the registry default
e.bucket("research"); e.topic_arn("pages"); e.s3_uri("crucible")

以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗