CTM STACK
Use supported Moodle STACK questions on interactive Quarto pages. Keep the exported XML, question variables, response trees and author tests; this extension prepares chosen variants while building the page and checks learner answers in the browser. The live example site shows algebra, numbers and matrices, with optional AI hints alongside deterministic grading.
Use it for practice and for adapting supported STACK material to teaching pages. It covers a defined subset rather than Moodle's full assessment workflow; unsupported authoring features stop the build. diagnostics: true shows a diagnostic card while adapting a question.
ctm-stack is a part of CTM Assessment. Its STACK engine (the importer, parser, answer tests, response trees and question tests) is ctm_checkers.stack in ctm-checkers, CTM Assessment's checker library, which also holds the example questions, the native STACK comparison and the compliance report. Its library ctm-libs provides the browser's Python runtime and AI feedback. This repository holds the Quarto extension (the filter, the card and its grading worker) and this site.
Try the examples
The project-authored and adapted questions in ctm-checkers' examples/stack/ (this site copies them into questions/ before rendering, from the ctm-checkers commit it builds against) use the Moodle-style <quiz><question type="stack">…</question></quiz> structure, with input settings, response trees and authored <qtest> cases. Files using Python-only answer tests or resizable input settings require this port; they are not drop-in native STACK exports. upstream/ holds the original three unchanged STACK samples with provenance in sources.json. The 35-question native corpus is a cross-chapter compatibility target with a per-file manifest, unchanged exports, and authored qtests. See the corpus page for its compatibility status and original XML links.
Install in a Quarto project
ctm-stack is part of CTM Assessment, which installs and runs it: install the package (quarto add https://erasmus-ctm.github.io/ctm-assessment/ctm-assessment.tar.gz) and list filters: [ctm-assessment] in _quarto.yml. Nothing else is installed or listed for STACK cards. Without CTM Assessment (or with one that predates ctm-checkers' STACK engine, 0.2.0) the render stops with the command that installs or updates it. STACK cards are prepared at build time, so also run python -m pip install 'sympy==1.13.3' in the Python environment Quarto uses; that project does not receive this repository's requirements file. Build-time rendering needs Python 3.9+ and Quarto 1.8.27+.
For a local checkout of this example site, install the parts of CTM Assessment that it builds against, at the commits in .github/pins.json: quarto add of a ctm-libs and a ctm-checkers checkout. The site's questions/ are ctm-checkers' examples/stack/ (scripts/fetch-examples.py copies them before rendering, from _ctm-checkers-source/ or the checkout named by CTM_CHECKERS_SOURCE). Without the CTM Assessment package, render as a component development build (CTM_COMPONENT_DEV=1):
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt
git clone https://github.com/Erasmus-CTM/ctm-libs _ctm-libs-source # then check out the pinned commit
git clone https://github.com/Erasmus-CTM/ctm-checkers _ctm-checkers-source # then check out the pinned commit
quarto add ./_ctm-libs-source --no-prompt
quarto add ./_ctm-checkers-source --no-prompt
CTM_COMPONENT_DEV=1 quarto render
python -m http.server --directory _site 8000
When the filter uses a different Python interpreter, set STACK_SYMPY_PYTHON=/absolute/path/to/.venv/bin/python for the render. A plain file:// preview cannot reliably run browser workers; use an HTTP server. Browser grading downloads Pyodide and its packages on first use (ctm-libs' runtime, Pyodide 0.28.3), so a learner needs network access. The site's authored XML files are retained as downloadable resources.
This site lists ctm-stack itself in filters:, because it builds against those checkouts; a course lists only ctm-assessment.
In a .qmd file, give the card a unique label and use the exact XML question name:
```{.stack-question}
#| label: expand-a-product
#| file: questions/polynomial-expansion.xml
#| question: Expand a product (form-sensitive)
#| variants: 3
#| review: true
#| feedback: true
```
A learner enters answers, presses Validate answers to inspect how each expression was parsed, then presses Check answer to grade the validated response. Editing a field clears its validation and requires another Validate step. The preview preserves the written form, so x+x still appears as x+x; this is important when an answer test checks form. Invalid input receives a validation message and cannot be graded until corrected.
With source-panels: true (on a card, or for every card under stack: in _quarto.yml, as this site sets it) each question has closed Show source and Show STACK source panels below it. They show the authored Quarto fenced block and the original XML file, respectively, for inspection or copying. Opening either panel does not affect the question or its grading. Panels are off by default.
label, file and question are required; the label is unique on the page and keys the card's variant history, saved answers and policy overrides. variants is the number of proposals generated for review, 1–100 (default 25); only approved current variants are published (see the authoring guide). #| seed: was removed: the render stops and asks for reviewed variants instead. review: true exposes the authored qtests for inspection. diagnostics: true displays build errors in the rendered page during authoring. Remove diagnostics for publishing, so unsupported content fails the build. The full option list is in the authoring guide.
AI feedback and context
AI hints are on by default (feedback: auto) whenever the build has AI feedback; feedback: false turns them off for a card, and stack: feedback: false in _quarto.yml for every card. A build without AI feedback (ai-feedback: enabled: false or CTM_FEEDBACK=off) gives cards none: no Feedback button and no feedback data in the page; native checking is unchanged, and the build message names them. Cards use their own feedback integration, stack, whose shipped policy has math-exercise's hint sequence. context: auto is the default; context: id1,id2 references tagged blocks; context: none omits context. feedback-language chooses the explanation language, defaulting to document lang. feedback-policy selects an existing named policy. Use label for a stable exercise key in policy YAML; otherwise the card ID is used.
Automatic context comes from visible prose in the preceding section. To share a specific concept, tag it and reference its ID instead:
::: {#expansion-hint .ai-context}
Distributivity gives $a(b+c)=ab+ac$.
:::
```{.stack-question}
#| label: expand-with-context
#| file: questions/polynomial-expansion.xml
#| question: Expand a product (form-sensitive)
#| feedback: true
#| context: expansion-hint
```
context: none turns context off for that card. .ai-feedback-context and legacy .math-exercise-context are aliases of .ai-context. The live examples include all three modes. CTM Assessment's AI feedback (ctm-libs) handles explicit aliases, LaTeX and context limits; STACK does not implement a second resolver.
CTM Assessment's AI feedback loads named definitions from policy-files; loading a file alone activates nothing. Select one with policy, either by name or with a selector map for the stack integration (policy: {stack: course-mathematics}, as this site's _quarto.yml does); exercise overrides go under exercises.stack.<label>. Policies support extends, base-prompt, and a replacement list of step-prompt entries. The old authored defaults, integrations, and prompt fields are rejected. Resolution follows shipped defaults, explicit project/page selections, exercise-ID overrides, the exercise's selected policy and its ancestry, then its current step. STACK delegates this resolution to the shared runtime. See the working course policy, authoring example, and shared policy reference.
The default mathematics sequence has four steps: question, concept, procedure, and worked solution. Edits cancel stale replies and invalidate Check evidence while retaining progression. Check restarts hints when the effective reset-on-run is true; selecting another frozen variant always resets hints and evidence. Only a successful displayed reply or copy prompt advances a step. Explicit jsxgraph:diagram-id context references use the shared visual-context collector; diagrams are never attached implicitly.
The learner's current answer, visible question text and selected context form the feedback request. A preceding matching Check answer can supply minimal result status and a finite score as evidence. Original XML, source panels, teacher answers, qtests, grader configuration, feedback messages and answer notes are excluded. Feedback does not run or replace the deterministic grader. The shared settings panel defaults to copy-prompt mode and can use a learner-configured provider; provider keys are not embedded in the page. ctm-libs owns the provider client, settings, policies and feedback UI. CTM STACK supplies only its question/response adapter.
Supported subset and grading behavior
| Feature | Current handling |
|---|
| Input types | algebraic (scalar or nested list/set), scalar numerical, and bounded matrix; strict syntax, <insertstars>0</insertstars> (explicit *) or <insertstars>1</insertstars> (bounded adjacent implied multiplication), forbidden words/floats, lowest-term fractions, and validation flags. |
| Expressions | Arithmetic, powers, scalar symbols, and a constrained allowlist of Maxima-like variable assignments and functions. Seeded rand(n) works for the examples. No arbitrary Maxima evaluation. |
| Answer tests | AlgEquiv, Sets, SameType, EqualComAss, FacForm, SingleFrac, LowestTerms, NumAbsolute, NumRelative, NumDecPlaces. FacForm requires a single variable in <testoptions> and handles univariate polynomials with rational coefficients. Blank options mean a 0.05 tolerance for the two numeric tolerance tests; NumDecPlaces requires an explicit count from 1 to 50 decimal places. The numeric tests are false, with a message, for an answer that is not a finite real number (NumDecPlaces also needs a decimal literal); an answer a test cannot decide makes it false too, with the note <test>_undecided. |
| PRTs | Acyclic graphs of answer-test nodes, branch scores with =, +, or -, feedback, answer notes, and weighted aggregation across response trees. |
| Author tests | Native <qtest> inputs with expected PRT score, penalty, and answer note. Test outcomes are available in review mode and the CLI. |
| CASText | Bounded scalar, collection and matrix {@expression@} or a displayed scalar equality {@left=right@}, plus [[input:name]], [[validation:name]], and [[feedback:name]] in sanitized HTML. |
EqualComAss, FacForm, SingleFrac, and LowestTerms require sans to be the direct input name so the student's written expression form survives validation. EqualComAss checks that form up to commutative/associative reordering: cos(0) and 1 have the same value but different written forms, as do sqrt(3) and 3^(1/2). It still accepts reordered terms such as sqrt(x)+sin(x) and sin(x)+sqrt(x). AlgEquiv first screens scalar candidates at deterministic positive substitutions. Only stable, finite real evaluations at 40 and 70 digits with a relative separation greater than 1e-4 can reject a candidate early; poles, complex values, and inconclusive evaluations fall through. Agreement never establishes equivalence, and this is not a grading tolerance. Symbolic comparison uses SymPy transformations, followed by bounded exponential rewriting for elementary trigonometric and hyperbolic identities. For bounded numeric complex constants, a rectangular-form proof mirrors STACK’s rectform step and respects principal roots; symbolic complex-root and logarithm branch identities remain limited. An unresolved comparison involving newly supported functions reports an explicit unsupported/inconclusive result. FacForm checks equivalence and the student's written factorization, including whether its polynomial factors can be factored further over the rationals. It requires one polynomial variable in <testoptions> (including an author variable resolving to a symbol) and handles rational coefficients, nonnegative integer powers, and numeric denominators; it does not implement the full Maxima answer test. SingleFrac checks equivalence and a single written fraction, rejecting divisions nested through sums or products in its numerator or denominator; equivalent atomic answers are also accepted. Like the native helper, this form check does not inspect powers or function arguments for division. LowestTerms checks the student's written fractions without comparing the teacher value: it rejects reducible numeric integer fractions, a double negative, and supported irrational or complex denominators. It does not cancel symbolic polynomial factors. These two form checks cover scalar expressions in the parser's bounded syntax, not every Maxima expression or STACK option. With questionsimplify=0 (simp:false), supported CASText expressions retain their authored arithmetic form, including grouping within existing TeX math. Numerical tolerance is based on the mathematical value, while NumDecPlaces also examines the raw written decimal and its trailing zeros. The XML penalty is compared in an authored qtest as an initial-attempt expectation. Authored answer notes use STACK-style ordered subsequence matching, including anchored beginning/end notes and ; an expected terminal note can pass when earlier nodes also ran. The demo itself has no Moodle attempt state or persistence. The adapter does not claim all STACK tests, expression forms, domains, validation behaviors, question-level options, or Maxima semantics. It raises a question-and-field-specific error when it knows an author feature cannot be preserved. Authors should run the qtests, inspect rendered variants, and compare consequential questions in Moodle.
Scalar functions also include ln as an alias for log, two-argument rational-polynomial gcd, and real scalar mod, including Maxima’s mod(x,0)=x convention. Polynomial GCD retains numeric content and uses the existing degree, variable, and expansion bounds. Symbolic complex logarithms requiring branch proofs remain explicitly unsupported. Author expressions additionally support num, denom, and ev(expr) alongside ev(expr,simp); numerator/denominator extraction preserves common factors for form checks.
AlgEquiv also accepts typed propositions (true, false, and, or, not, implies, xor, xnor, and the aliases nounand, nounor, nounnot) with at most eight variables. Arithmetic on propositions is rejected. Boolean equivalence uses an exhaustive bounded truth table. Operator precedence and left-associated implication follow STACK; chained xnor uses parity equivalence. Polynomial equations and disjunctions of polynomial equations use STACK's product-of-residuals rule: a=0 or b=0 is equivalent to a*b=0. Nonzero numeric scaling is ignored; symbolic factors and root/branch multiplicity are retained. Thus a*x=a*y differs from x=y, and (x-2)^2=0 differs from x=2. Equations support rational complex coefficients, numeric denominators, total degree at most 100, and a conservative expansion bound of 2,048 terms. Variable denominators are rejected before cancellation. Conjunctions or negations of equations and mixtures of equations with propositions are unsupported. These new logical forms require AlgEquiv rather than the form-sensitive or numerical tests. See the function and logic examples.
Finite collections use the existing algebraic text input: [a,b] is an ordered list and {a,b} an unordered set. Both can nest, with scalar expression leaves only. Empty collections work. Bounds are 32 entries per container, six collection levels and 128 collection-tree nodes (containers plus scalar entries); the scalar parser bounds also apply. AlgEquiv recurses through list positions but compares normalized structural set members. Inner sets ignore order/repetition; lists inside sets retain order/repetition. Sets explains outer duplicates and missing/extra members. EqualComAss retains written arithmetic, ordered list entries and every repeated set entry, and does not flatten nested sets. Its student expression must name an input directly. Frozen held teacher/qtest alias dependencies are checked and rejected when written form cannot be preserved. Ordinary answer-test notes remain abbreviated English, while authored PRT branch notes and qtest path expectations are retained.
SameType follows STACK's applicability rule. Lists of different lengths still have the same type; equal-length lists recursively compare corresponding member types. Any two supported sets have the same type regardless of their inner member types. Thus this is not a full nested-shape validator. Equations, logical expressions and expressions are three types for it. Input checkanswertype=1 refuses an answer of another type than the teacher's answer, with a message that names the expected type. Use AlgEquiv to check actual content. Written previews retain submitted order and duplicates. Decimal arithmetic in direct nested list literals uses the scalar binary64 path; frozen collection definitions with decimal literals or aliases remain rejected in favor of exact fractions.
Collections require direct learner input names in PRT checks; derived learner collections and feedback aliases are rejected to retain decimal provenance. Sets require exact fractions, including nested members. Decimal list leaves support only literal constant arithmetic with bounded integer powers; decimal functions, symbolic leaves, author transforms and transitive frozen decimal aliases fail clearly.
Author expressions additionally support set, setify, listify, union, intersection, setdifference, member, subsetp and cardinality in normal simplification mode. Learners enter literals; these helpers are author-only. setify takes a list; member accepts a list or set; listify preserves a list or sorts rational scalar set members numerically. Symbolic/complex/nested set ordering in listify is unsupported. Set operations take sets, have bounded arity/output, and require exact fractions rather than decimal literals. Freeze their results in questionsimplify=1 question variables before using Sets or EqualComAss. Helper calls with simplification off, in feedback variables, or as direct held-test teacher expressions fail with a diagnostic. Normal AlgEquiv/SameType teacher expressions can call helpers directly. Logical collection leaves, real intervals/infinite sets, learner indexing, and EqualComAssRules remain outside this increment; bounded matrices have their own contract below. See collection semantics and the nested examples.
Matrices use a distinct rectangular value, 1–16 rows/columns and at most 64 entries. Native-compatible matrix tests are AlgEquiv, SameType and EqualComAss; general matrix arithmetic and other native matrix tests remain unsupported. The pinned standalone editor preserves raw strings and invalidates previous validation/results after edits or resizing. PRTs award scores; the editor does not invent cell partial credit. Fixed dimensions derive from the model answer; bounded resizing is a port-specific input extension. See matrix input.
The supplied matrix checker file registers five PythonChecker functions. For a basis of a null, column or row space, the PRT supplies the original matrix A and selects {"checker":"basis","options":{"space":"nullspace"}} (or columnspace or rowspace); author-only nullspace_basis(A), columnspace_basis(A) and rowspace_basis(A) supply input model answers. For integer entries, determinant or identity, choose matrix_property with an options object such as {"property":"determinant"}. The PRT teacher value is the expected property result, not a reference matrix. The file also registers is_hermitian, is_positive_definite and the combined is_hpd for exact rational real and imaginary parts. One PRT can award credit for both conditions together; two weighted PRTs can award half for each. Every function receives the learner value, teacher value and a JSON options object ({} when omitted). Select the file by filename with #| python-checkers: questions/matrix-checkers.py so Quarto includes its source for browser grading. See the authoring walkthrough and worked examples.
The native STACK comparison and the native compliance report belong to the engine and live in ctm-checkers: its native-stack.yml workflow (the reference cases against the official STACK/Maxima answer tests of the pinned STACK checkout, all 2,303 static upstream answer-test rows against authored desired results, against native STACK/Maxima and in Pyodide against CPython, and the project's port regressions) and the report's Quarto project. This site's workflow calls that workflow at the ctm-checkers commit the site builds against, and its single publication job waits for the site tests and those checks, then publishes the tested site and the report together. Main gets /compliance/; feature previews include their matching report at previews/<branch>/compliance/. Pull requests receive downloadable HTML and JSON artifacts, including failed runs with unavailable checks shown explicitly, but do not publish. An ordinary quarto render of this website does not rerun or replace the report; ctm-checkers' native STACK documentation and the report's pages say how to reproduce it. Passing these checks does not imply that all STACK expressions, question settings, or random variants agree with Maxima.
The report inventories the pinned fixture's rows and distinguishes unsupported cases, controlled errors, known native mathematical failures, reviewed Python defects, and unexplained differences. Native agreement compares CAS applicability and correctness, not whether STACK's mathematical result is correct. The report covers this adapter's bounded scalar, finite-collection and matrix answer-test subset, not Moodle's PHP controller, question state, or every STACK feature; the upstream PHP fixture is parsed as data and never executed. Its semantics and open questions page separates documented STACK conventions from observed domain differences and missing exact proofs, and its Port regressions page shows the project's own cases.
The original corpus question AlgMap-2.5 grants full credit when LowestTerms accepts the written form before its AlgEquiv node runs. A reduced fraction with the wrong value can therefore receive full credit. Its passing authored qtests demonstrate replay of those specific tests, not that the upstream grading tree enforces mathematical correctness for every response. The corpus gallery flags this behavior beside the card.
To check a file before rendering:
export PYTHONPATH=$(ls _extensions/ctm-checkers/*.whl | paste -sd: -) # ctm-checkers' two wheels
python -m ctm_checkers.stack.cli test --file questions/polynomial-expansion.xml
(The render puts the same two wheels on PYTHONPATH.) quarto render is also a validation step: errors in a selected question stop the build. The engine's tests are ctm-checkers'; this repository's tests check the extension and the site: CTM_COMPONENT_DEV=1 python -m pytest -q tests (they find ctm-libs and ctm-checkers installed here, or in the folders named by CTM_LIBS_EXTENSION and CTM_CHECKERS_EXTENSION). Browser integration checks require requirements-dev.txt and Playwright browsers; see the tests in tests/. Quarto source pages are index.qmd, examples.qmd, and the authoring guide.
Design and contribution
Student input accepts explicit arithmetic + − * / ^ (and **), parentheses, scalar symbols and, as in STACK, the constants e, pi and Pi, i and %e, %pi, %i (PI is a constant in an author's expressions, but a student writes pi or Pi: PI is rejected as the wrong case; E and I are variables, ctm-checkers 0.5.0), and the functions sqrt, sin, cos, tan, sec, csc, cot, sinh, cosh, tanh, sech, csch, coth, exp, log, abs and the inverse functions asin, acos, atan, asec, acsc, acot and their hyperbolic forms (with cosec, cosech, and STACK's arcsin-style aliases). With <insertstars>0</insertstars> the learner must write multiplication as *; <insertstars>1</insertstars> additionally accepts adjacent unambiguous products such as 8x, 2(x+1), (x+1)(x-1), and (x+1)x. A multi-letter name such as xy remains one symbol, x(y+1) is rejected as an unknown function rather than guessed as a product, and sin(x) remains a function. Further insertion modes (2–7) are unsupported. This is a bounded subset of mode 1, not every STACK token-splitting rule. An answer follows STACK's rules for its text, with STACK's messages (ctm-checkers 0.6.0, the ctm-stack audit's S-13): no space between terms (2 x: "Illegal spaces found in expression 2_x."), no decimal comma (3,5), no name of three letters or more that STACK does not know (sinx, sin*x; alpha, theta are known), names in their case (Sin(x), PI), Unicode letters and symbols replaced as STACK does (π, α, −) and other characters rejected (x²), and Maxima's error for . An answer may not name a question variable or a response tree's feedback variable: is invalid with "Forbidden variable or constant: ta." and shows no preview, so a variable never reveals its value (ctm-checkers 0.3.0, the ctm-stack audit's S-01). A question test's inputs are computed as STACK computes them (an input's name stands for its teacher's answer) and the computed value must be valid student input (ctm-checkers 0.6.0). Author assignments additionally allow lists with 1-based indexing, , , , , , , , integer , and decimal/significant-figure display functions. preserves a common-denominator form for the supported scalar expressions. The parser recognizes author , but floating-point question variables cannot be frozen into variants and are rejected at build time. General equations beyond the polynomial subset above, inequalities, equation systems, units, general matrix arithmetic, and scripts remain outside the supported subset. is accepted only when the adapter verifies that any teacher form survives frozen author variables; otherwise it fails the build. must be , and input must be empty, and branch-specific PRT penalties are unsupported. Unrecognized validation settings and unsupported HTML/CASText tags also fail the build.
The engine is ctm_checkers.stack in ctm-checkers: its importer.py turns native XML into an explicit JSON-safe question model; expressions.py, assessment.py, and engine.py implement bounded parsing, answer tests, seeded variants, PRT grading, and qtest replay; rendering.py handles CASText. Quarto's Lua filter (this repository) selects the XML question and has python -m ctm_checkers.stack.cli render generate the static card while the page builds. In the browser, a worker loads ctm-libs' Python runtime and ctm-checkers' page scripts (their URLs come from the page), installs the engine in Pyodide and grades with ctm-checkers' calls stack.validate, stack.grade and stack.tests; the calculation time limit starts once Python has loaded.
Please add an XML export and qtests with a bug report or a new feature (engine changes go to ctm-checkers, with their golden-table rows). Avoid relaxing unsupported-feature checks without a test showing the semantics. This project uses the GNU Affero General Public License version 3 or later for code. The example questions keep their licences, listed in ctm-checkers' examples/stack/ README: INGMat 3A questions are CC BY-SA 4.0; the determinant-one question originates in AGPL-3.0-or-later math-exercise. Other newly authored example questions are by the CTM team under CC BY-SA 4.0. Upstream samples retain their respective licenses: the individual provenance records distinguish the sample question license from the HELM license. The new corpus contains only samples under CC BY-SA 4.0. SymPy, Pyodide, Quarto, and Moodle STACK are independent projects with their own licenses.
The backend contract documents the version-2 card payload, adapter lifecycle and failure states.
STACK-derived native files and generated compiler data are listed in the
native NOTICE, with their upstream provenance and licences.
