TS-Version-Control

Version control and promotion for ThoughtSpot content. It migrates tagged objects from a source (dev) org to a target (prod) org through Git, so every change is exported as TML, committed, reviewed and then imported, with a report for each run.

It also carries over Spotter content that TML does not include: model memory (rules and reference questions) and model-level NL instructions.

• Source is read-only, except that the migration tag is removed from objects that migrated successfully.
• Nothing is ever deleted in the target.
• Objects are matched by obj_id, so re-running updates the existing target object instead of creating a duplicate.

For setup (secrets, environments, branch protection) and troubleshooting, see docs/runbook.md. For API details, see docs/api-verification.md.

Console

site/ is a browser UI for the same workflows: start a dry or live run, follow each stage live, review and approve the promote PR, read the run's report, roll back to a restore point, and build a new org's config file. It is a static page that calls the GitHub API with a fine-grained token kept in your browser; there is no backend. pages.yml publishes it to GitHub Pages (Settings → Pages → Source: GitHub Actions). To try it locally, run python3 -m http.server -d site 8000. Demo simulates a run without a token.

How it works

 tag objects in source ──► Actions: "ThoughtSpot Migration" ──► one pipeline per org

Job	Stages	Environment	What happens
extract	1–6	dev	Validate config, authenticate, find tagged objects (and dependencies), export TML, export Spotter memory and NL instructions, sanitize, commit everything to ts-dev
promote	7, 7b	–	Open a PR from promote/<org>/<run_id> into ts-prod holding only this run's files, ask GitHub to auto-merge it, then wait until it is approved and merged. On a dry run the PR is titled [DRY RUN], auto-merge is not requested and nothing waits
import	8–9c	prod	Take the org's TML from the promote PR's merge commit on ts-prod (so reviewer edits count), then sync variables, check connections, validate, import TML, then import memory and NL instructions
untag_source	10	dev	Confirm each object exists in the target, then remove the migration tag from the successes
report	11	–	Write the summary and failure reports (runs even if earlier jobs failed)
release_tag	–	–	After a live run with no failures, tag the imported ts-prod commit migrate/<org>/<run_id>: a restore point for the redeploy workflow

Stages

Stage	Script	Purpose
1	stage1_config	Load and validate config/orgs/<org>.yml
2	stage2_auth	Trusted-auth tokens for source and target; abort on org mismatch
3	stage3_discover	Find objects with the migration tag, then what they are built on (a liveboard's model, a model's tables, joined tables), read from their TML and handled per dependency_mode. Objects without an obj_id are excluded and reported
4	stage4_export	Export TML (obj_id included, GUIDs excluded, coaching feedback included)
4b	stage4b_export_memory	Export Spotter memory for each model
4c	stage4c_export_instructions	Export NL instructions for each model
5	stage5_sanitize	Strip GUIDs and the migration tag, find ts_var(name) references (quoted or not; built-in variables like ts_user_timezone are ignored)
6	stage6_commit	Commit orgs/<org>/ to ts-dev (retries on concurrent pushes)
7	stage7_promote	PR into ts-prod with this run's files (those stamped with the run's TIMESTAMP), auto-merge requested (not on dry runs). If GitHub refuses auto-merge, for example because "Allow auto-merge" is off in the repository settings, the log shows a warning, the PR body says a person must merge it, and stage7_promote.json records auto_merge: false
7b	stage7b_wait_for_merge	Poll the promote PR until it is merged. Fails if the PR is closed unmerged or not merged within 330 minutes (MERGE_WAIT_TIMEOUT_MIN); the import is then skipped and the tag stays. Skipped on dry runs
8b	stage8b_snapshot	Before importing, save the target's current version of every object about to be overwritten (live runs only). Fails the run if a copy cannot be saved
8	stage8_prod_prep	Create or check variables, check connections exist in the target, validate-only import

Import order for Stage 9 is tables, views, models, collections, answers, liveboards, then feedback. A tagged object that no step handles is reported as a failure and keeps its tag. Stage 9 fails closed: an import counts as successful only if every object in the response is OK or WARNING, and a table whose post-import read-back cannot be done is failed rather than trusted.

Branches

Branch	Holds
ts-prod	Default branch: pipeline scripts, config, and approved TML
ts-dev	Exports only (TML, memory, instructions, reports). Scripts are overlaid from ts-prod at run time
promote/<org>/<run_id>	One short-lived branch per run, the head of that run's promote PR. Deleted automatically once the PR merges or is closed (Stage 7b, and cleanup-promote-branches.yml for PRs closed by hand)

This is the beginning of the README. Read the rest at the original. Read the full README ↗