workshop_synthesizer
A workshop synthesizer for Claude Code that grills you first, quotes your sources word for word, and checks its own themes before you see them.
Hand it interviews, survey answers, support tickets, field notes, whiteboard photos or a FigJam board. It asks what decision the research is for, then writes themes where every claim carries a verbatim quote and a source tag. Then it loops: a script checks every quote against the sources, a second agent tries to knock each theme down, and the first agent fixes only what they caught, until the synthesis holds up.
/synthesize research/notes/
Site: bbulman.github.io/workshop_synthesizer
This is the basis for the synthesize harness in Workshopr.io: the same grill, quote gate and second-reader loop, open-sourced so you can run it in Claude Code on your own research.
Install
You need Claude Code and Python 3.
git clone https://github.com/bbulman/workshop_synthesizer.git
cd workshop_synthesizer
./install.sh
It copies the two agents to ~/.claude/agents/ and the skill to ~/.claude/skills/synthesize/, keeping any earlier copy as a backup. Open a new Claude Code session (or type /reload-skills), and run /synthesize on a folder of notes.
For whiteboard photos nothing else is needed. For FigJam boards, connect a Figma MCP server; the agents fall back to asking for an export or screenshots when none is connected.
What it does, step by step
- Looks before it asks. It counts your sources by type, notes who the participants are when the notes say so (admin, solo user, new, returning), and flags anything that isn't research, like a PRD in the pile or a FigJam link with no FigJam tool.
- Grills you. One question at a time, each with a recommended answer drawn from what it saw: the decision this informs, the research question (rephrased neutrally if it assumes its answer), who reads the report, which groups to compare, your hunches, what's out of scope. It ends with a five-line brief for you to confirm.
- Synthesizes. A fresh
workshop-synthesizer agent reads everything before coding anything, codes each source with equal care, clusters the codes into 3–7 themes, and cuts any theme that could have been written without reading the data. Your hunches become hypotheses it marks supported, contradicted or open. They never count as evidence.
- Counts instead of guessing. Each theme lists the sources behind it, as "4 of 4 participants" rather than "most users", with a confidence rating, counter-evidence and, when the notes label groups, a table of themes by group.
- Checks every quote. The agent searches the sources for each quote before it returns, and
check_quotes.py checks again: case, wording and order must match, and a [...] can't stitch two interviews together.
- Gets a second opinion. A fresh
research-verifier agent re-reads the sources and tries to prove each theme wrong: recounts support, checks the ratings, and looks for smoothed-over contradictions, missed themes and unused sources.
- Loops. If the script or the verifier objects, the synthesizer fixes only what was flagged, and both checks run again. Three passes at most; anything still open is listed plainly.
- Reports. The answer to your question, what happened to each hunch, where the report is saved, and what the checks caught on the way.
What a theme looks like
## Theme 2 — Nobody can tell how far through setup they are
**Observed:** All four participants and one ticket say setup gives no sense of how many steps remain.
**Insight:** The friction is uncertainty about the end, not the length of setup.
**Support:** P01, P02, P03, P04, T03 — 4 of 4 participants, plus 1 ticket
**Confidence:** High — every participant, both segments, no counter-evidence
**Evidence:**
> "It wasn't long exactly. It was more that I couldn't tell where the end was." — P03, 05:13
> "Not the length. A few people said they didn't know when it would end." — P04, 08:16
> "How many more steps are in setup? I have done five screens already." — T03
**Counter-evidence:** none observed
**Implication:** Show progress ("step 3 of 5") before cutting steps.
Anything one source says on its own goes under Outliers, and what the data can't answer goes under Gaps, each with a suggested follow-up.
Why it holds up
Language models are fluent synthesizers and unreliable ones. They invent or smooth quotes, over-report rare themes, give the first and last sources more weight than the middle, and drift toward whatever the research question seems to want. Four things push back:
- Quotes are checked by a script, not by trust. Self-correction without outside feedback makes models worse (Huang et al. 2023), so the loop runs on a deterministic quote gate with 12 self-test cases, plus the sources themselves.
- Counts are computed, not estimated. In a World Bank study, ChatGPT coded rare themes about three times as often as experts did, and its errors tracked who the interviewees were. Here every theme carries its source list and an "n of N", and a segment table makes a skew by group visible.
- Every source has to show up. Models attend least to the middle of long input (Liu et al. 2023). The synthesizer codes one source at a time, and any source that supports no theme is re-read and either coded or named as an outlier.
- A second reader, with fresh eyes. Model judges favour their own output (Zheng et al. 2023). The verifier never sees the synthesizer's reasoning, only its report and the sources, and has to uphold each theme before the loop stops.
여기까지가 README 앞부분입니다. 나머지는 원문에서 읽을 수 있습니다. README 전문 보기 ↗
