SYNAPTIK
Evidence-structure analysis for iPSC-neuron research: has this experiment been done, what does this phenotype mean, and where do the literature, sequence models, and population genetics disagree?
Why this exists
Wet-lab researchers working with iPSC-derived neurons ask the same two questions
before and after almost every experiment:
- Has this already been done? Same cell model, same maturity, same
perturbation, same readout. By whom, with what protocol, and what did they find?
- What does this phenotype mean? I see reduced SYN1/PSD95 puncta in my
mutant line. Which pathways does the literature consistently associate with
that, and how much should I trust those associations?
Today both are answered by hand: days of PubMed searching, spreadsheets, and
judgement calls about which papers to believe. The available tools sit at the
wrong level. General literature search returns papers, not experiments.
Text-mining resources map gene-disease co-occurrence but ignore the
experimental details that decide whether two results are comparable at all
(differentiation protocol, days in vitro, isogenic controls, assay). And none of
them tell you when a confident-looking association rests on a single lab.
SYNAPTIK fills that gap. Its unit of analysis is the experiment record
(paper × biological system × perturbation × assay × measured outcome), and its
core output is evidence structure rather than a single score: how many
independent research groups, how many orthogonal assay families, whether the
association survives removing the most prolific lab, whether the field agrees
on the direction of the effect, and whether any null results were ever published.
The failure mode it is built to prevent
pathway tier enrich q labs assays
neurotransmitter release strong 16.9 1.58e-06 9 2
exotic kinase cascade single-lab 225.2 3.46e-09 1 1
The second row has 13x the enrichment and a smaller q-value, and it is far
less trustworthy: one group, one assay, no null results across nine papers.
Sorting by enrichment or p-value, which is what most literature mining does,
puts it first. SYNAPTIK ranks it below independently replicated evidence and
says why.
Empirical Benchmarks & Reproducibility
Training and evaluation scripts live in scripts/ (train_splice_mlp.py, train_aso_mlp.py, train_evidence_weighter.py) with fixed seeds, dataset SHA256 logging, and 95% bootstrap confidence intervals.
| Module | Dataset & Protocol | Reproducible Command | Status |
|---|
Splice Disruption (synaptik splice) | MFASS: 30,330 mutant SNVs across 2,228 exons (5-fold exon-grouped OOF) | python scripts/train_splice_mlp.py | AUROC $= 0.8575$ (95% CI 0.8476–0.8668), AUPRC $= 0.4967$ (random 0.0654) |
Gapmer ASO Potency (v3) (synaptik aso) | ASO Atlas + Ensembl GRCh38 cDNA: 12 held-out neurogenetics genes (n=25,839, 3,312 pos) | python scripts/train_aso_mlp.py | ROC-AUC $= 0.7146$ (95% CI 0.7059–0.7229), Spearman $\rho = 0.4112$; external wet-lab (n=8) underpowered / not yet validated |
| TDP-43 Cryptic Exons | 6 published ALS/FTD loci (STMN2, UNC13A, KALRN, RANBP1, SYT7, ATG4B) | python -m synaptik.sequence.benchmark_cryptic | 6/6 cryptic splice events detected; 5/6 top-1 ASO overlaps cryptic junction |
Temporal Literature Replication (synaptik benchmark) | 1,714 OA Europe PMC full-text papers (2013–2025, 2,734 records); Cutoff 2022 (n=148: 115 pos / 33 neg) | python scripts/train_evidence_weighter.py | Underpowered / overlapping CIs (rigor_only 0.6899 [0.5807–0.7871] vs. record_count 0.7626 [0.6808–0.8376]; diff CI to ); descriptive rigor prior only |
This is the beginning of the README. Read the rest at the original. Read the full README ↗
