Aquaculture Resilience Evidence Engine (AREE)

AREE is an open, reproducible evidence-generation system for aquaculture resilience biomarker discovery. The MVP focuses on Pacific oyster (Crassostrea gigas) and is structured so additional shellfish species and aquaculture organisms can be added without changing the core model.

AREE is not a static list of papers. It provides a versioned path from public study registration, through harmonized processed-result intake or raw RNA-seq reanalysis, into cross-study evidence tables, meta-analysis summaries, transparent candidate prioritization, evidence cards, and user-facing reports.

Results to date: https://sr320.github.io/AREE-co/. The site rebuilds automatically on every push to main (.github/workflows/pages.yml) from the committed registry, real data/harmonized/ evidence, versioned annotations in reports/, and a separate synthetic demo. To preview locally:

python scripts/build_site.py && quarto render _site_src

The site's Query page loads every harmonized evidence record into an in-browser DuckDB instance (from a Parquet file regenerated on each build), so visitors can search any gene, stressor or tissue across all studies without a server; the same Parquet download works in pandas, DuckDB or an AI assistant. The site leads with real-study findings and real evidence cards, derived from committed harmonized tables on every build. It displays up to ten findings per study: inference-eligible associations ordered by adjusted p-value, and exploratory PRJNA735889 effects ordered descriptively by absolute effect. Cards show all available study rows for each selected gene, including nonsignificant observations, design limitations and provenance. Full-study downloads remain available. Synthetic rankings, meta-analysis and cards are confined to the secondary demo/ section. No global biomarker ranking or cross-study pooled effect is presented on the primary site. Study detail pages expose the treatment, replication, QC and limitations. PRJNA735889's current oyster-level analysis does not model tank clustering: its uncertainty and significance are exploratory, excluded from pooling and from significance rewards in scoring until a tank-aware reanalysis is available.

For biologists

AREE asks which genes differ between oyster groups exposed to stress, or between populations with different performance. It checks the original experimental design, reprocesses public RNA-sequencing data, and puts the results into a consistent format with gene descriptions, uncertainty and limitations. This helps researchers choose genes and biological processes for follow-up experiments.

A change in RNA abundance can indicate a protective response or damage. It does not establish that a gene predicts survival or causes greater resilience. The current public findings are study-level expression results; cross-study pooling and scoring are demonstrated separately with synthetic data. See the biological overview and guide to reading results.

What Is Runnable Now

Website counts distinguish public BioProjects from registered study contrasts: one BioProject can contribute multiple contrasts, and shared controls constrain independent replication. The site build checks the registry CSV against the study YAML files to prevent stale metadata from being published.

• Study registration validation against JSON Schema.
• Registry ingestion from YAML into registry/study_registry.csv.
• Processed-result harmonization for RNA-seq, methylation, proteomics, and metabolomics-style tables.
• Demo cross-study random-effects meta-analysis.
• Transparent biomarker candidate scoring.
• Markdown evidence-card generation.
• An in-browser website query page over all harmonized evidence (DuckDB-WASM reading a build-generated Parquet; requires the site extra at build time).
• Demo report generation.
• A Streamlit browser for studies, evidence, and candidate cards.
• A Nextflow RNA-seq reanalysis pipeline (workflows/rnaseq): FastQC, Salmon, tximport/DESeq2, and export to harmonizable evidence. Methylation, proteomics and metabolomics studies currently enter as processed results.

Synthetic demo data are clearly labeled as simulated and exercise the analysis and reporting workflow alongside the separately curated real public datasets.

Install

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev,app]"

If you prefer to use an existing Python environment:

python3 -m pip install -e ".[dev,app]"

AREE supports Python 3.8 and newer; CI tests Python 3.8 with the lowest dependency versions pyproject.toml allows and Python 3.13 with the highest compatible dependency versions.

Commands read and write project files (registry/, data/, reports/). With an editable install from this checkout they use the checkout. With a regular install (pip install .), run aree from inside the project directory (any subdirectory works) or set AREE_ROOT=/path/to/project.

Demo Commands

aree validate-study registry/studies/CGIG_HEAT_RNASEQ_001.yaml
aree register-study registry/studies/CGIG_HEAT_RNASEQ_001.yaml
aree harmonize --study CGIG_HEAT_RNASEQ_001 --input data/demo/processed/CGIG_HEAT_RNASEQ_001_rnaseq.tsv
aree harmonize-demo
aree meta-analyze --phenotype thermal_tolerance --feature-type gene
aree score-candidates
aree build-evidence-cards
aree build-demo-report

For real studies, keep evidence separate from the synthetic demo table:

aree harmonize --study STUDY_ID --input data/processed/STUDY_ID_rnaseq.tsv --mapping data/mappings/MAPPING_RELEASE.tsv --output data/harmonized

--output is required for any study whose data_availability.status is not simulated_*, so real evidence cannot land in the demo table by accident. Given a directory, harmonize writes one STUDY_ID.tsv per study (rerunning a study replaces only its own file), which keeps every committed file well under GitHub's 50 MB warning and 100 MB limit. Point the downstream commands at the same directory; they concatenate every *.tsv in it (a single TSV also works). Without --evidence they read the demo data:

aree meta-analyze --evidence data/harmonized --output results/meta_analysis.tsv
aree score-candidates --evidence data/harmonized --output results/candidate_scores.tsv
aree build-evidence-cards --evidence data/harmonized --scores results/candidate_scores.tsv --output-dir results/evidence_cards

Before pooling real evidence, review biological comparability and shared samples: the engine does not model covariance from shared controls. The primary website presents study-level results without pooled effects or a global score.

Scoring recomputes heterogeneity from the evidence it scores, so meta_analysis.tsv is for inspection only and a filtered or stale copy cannot change the ranking. Effects are pooled and summarized only within one effect_size_type; candidates remain one row per feature so cross-context and multi-omics convergence still count. Use --phenotype/--stressor on score-candidates for a context-specific ranking.

Studies sharing controls must carry the same dependence_group=GROUP_ID quality flag on every evidence row. n_studies preserves the registered-study count; n_independent_studies counts these groups once for scoring and high-priority classification. Shared groups use a conservative sample-count lower bound rather than summed totals, labeled by sample_size_status. The DECICOMP temperature and starvation contrasts already carry their shared-control group.

aree build-evidence-cards --phenotype PHENOTYPE builds cards for one phenotype only. Card filenames are made safe for every OS (e.g. NCBI:LOC105317001 → NCBI_LOC105317001.md), and each build removes previously generated cards that are no longer in scope, so the directory always matches its evidence.

Rerunning aree harmonize on unchanged inputs leaves the evidence table byte-identical (date_generated is kept); it only changes when the records do.

Optional interfaces:

streamlit run app/main.py
quarto render docs/

quarto render docs/ is scaffolded for users with Quarto installed. The runnable Python report generator is aree build-demo-report.

Repository Tree

AREE/
├── README.md
├── LICENSE
├── CITATION.cff
├── CONTRIBUTING.md
├── CODE_OF_CONDUCT.md
├── docs/
├── registry/
│   ├── studies/
│   ├── controlled_vocabularies/
│   └── study_registry.csv
├── workflows/
│   └── rnaseq/        # Nextflow RNA-seq reanalysis pipeline
├── modules/
├── containers/
├── config/
├── data/
│   ├── demo/
│   ├── reference/
│   └── mappings/
├── src/
│   └── aree/
│       └── schemas/   # study and evidence JSON schemas, shipped with the package
├── app/
├── reports/
├── tests/
├── notebooks/
└── .github/
    └── workflows/

Objective 1 Component Map

AREE component	Objective 1 function
Study registry	Standardizes public dataset intake and study characterization.
Controlled vocabularies	Normalizes phenotype, stressor, assay, tissue, and quality labels.
JSON schemas (src/aree/schemas/)	Makes dataset metadata and evidence tables machine-validatable.
Nextflow RNA-seq pipeline	Reproducible raw-read reanalysis into harmonizable evidence.
Processed-result harmonizers	Allows studies without raw data to contribute transparent evidence.
Identifier mapping	Preserves original IDs while assigning comparable reference identifiers and mapping confidence.
Evidence schema	Converts assay-specific outputs into shared biomarker evidence.
Meta-analysis code	Pools comparable effects and reports heterogeneity.
Candidate scoring	Prioritizes repeated, interpretable, quality-aware biomarker associations without claiming validation.
Evidence cards	Produces database-ready summaries with limitations and recommended validation steps.
Streamlit/report outputs	Gives researchers and breeders a practical way to inspect and download evidence.

Production Status

The meta-analysis engine retains correlated feature/context groups as dependent_effects_require_covariance without a pooled estimate or heterogeneity penalty. Shared-sample covariance and biological comparability require review before real synthesis.

Area	Status
Study schema validation	Complete and runnable.
Demo registry and synthetic processed results	Complete and runnable.
Processed-result harmonization	Complete for MVP input schemas; production importers need assay-specific expansion.
Meta-analysis	Complete MVP random-effects implementation; should be cross-checked with R metafor for production.
Candidate scoring	Complete transparent MVP; weights should be reviewed by domain experts.
Evidence cards	Complete Markdown output; plots are textual summaries in MVP.
Streamlit app	Complete lightweight browser; production search and visualizations remain future work.
Raw RNA-seq Nextflow pipeline	Runnable; tested end to end on synthetic reads in CI. Other assays enter as processed results.
Containers	Scaffolded.
CI	Python compatibility tests, lint, wheel installation, demo reproducibility and a synthetic Nextflow pipeline run. A separate Pages workflow builds and deploys the website.

First Curation Targets

Suggested first public C. gigas dataset classes to curate:

1. RNA-seq thermal challenge study with survival or heat-tolerance phenotype.
2. RNA-seq ocean acidification exposure study with growth or shell-formation phenotype.
3. RNA-seq pathogen challenge study with disease resistance or pathogen-load phenotype.
4. WGBS or EM-seq temperature or acidification study with methylation changes linked to resilience.
5. Proteomics heat or pathogen challenge study with tissue-specific abundance changes.
6. Metabolomics hypoxia or salinity exposure study with survival or recovery outcomes.
7. Larval viability study under acidification or temperature stress.
8. Family or breeding-line comparison under thermal challenge.
9. Salinity or freshwater exposure study with recovery following stress.
10. Multi-stressor exposure study combining temperature with pH, hypoxia, or pathogen challenge.

Do not invent accession numbers during curation. Add only accessions verified from primary repositories or publications.