shift

shift is an executable spike for a tiny agent harness whose behavior is a live Guile image. The permanent runtime owns authority, provider transport, tool execution, and atomic generation switching; the user-owned Scheme image defines the agent.

This is intentionally not a production coding agent yet. The first question is whether changing agent behavior inside a running session feels materially better than editing and restarting a conventional extension.

Install

brew install thrashr888/tap/shift
shift-agent

The command is shift-agent, because shift is a shell builtin in every POSIX shell. The formula pulls Guile 3, ripgrep and Python 3 and compiles the runtime once at install. Run it inside a project; state lives in that project's .shift/. From a checkout, ./bin/shift-agent is the same launcher.

Principle: making it mine

Shift should become the user's own instrument. Identity, appearance, layout, and behavior are editable parts of the live environment. Defaults provide a starting point; users can replace the wordmark with their own name, move panels, build a theme, or ask Shift to change its interface immediately. Changes should be inspectable, reversible, and retained at the scope the user chooses.

The runtime supports live behavior changes and a default curses terminal interface with hot-reloaded presentation packs; see the TUI guide.

Try the live loop

Requirements: Guile 3.0.9 or newer, Python 3 with curses, ripgrep, and Ollama. The checked-in image defaults to the locally installed, tool-capable qwen3.8:27b-mlx model at Ollama's native http://127.0.0.1:11434/api/chat endpoint.

make test
./bin/shift-agent
./bin/shift-agent "Give me a cookie recipe about Lisp."

A quoted positional argument is submitted immediately as the first user turn. In a terminal the process remains interactive after the answer; with redirected stdin it exits naturally at end-of-file. The initial turn uses the same tools, streaming, tracing, approvals, and durable checkpoint path as a typed prompt. Autopilot is not "allow everything": fixed rules refuse the destructive cases and a separate judge model reviews the rest against your own words, with a shadow mode that records its verdicts beside yours before you trust it. With /sandbox NAME, runs execute in an agentkernel sandbox and need neither, while run-host prefixes such as cargo tauri and codesign stay on the host.

Interactive terminals open the curses interface automatically. There is no public --repl mode. Use --print/-p for one answer, or redirect/pipeline input for the scriptable Guile command loop:

./bin/shift-agent -p "Summarize this project."
printf '/show\n/quit\n' | ./bin/shift-agent
./bin/shift-agent < demo/context-selection/session.txt

Help and session listing/forking bypass curses. Redirected stdout also stays plain text. Scripted boot is deliberately compact:

shift λ
  default · generation 1 · 0123456789ab
  qwen3.8:27b-mlx via ollama
  stream on · thinking off · watch on
  12 tools · shell ask · /help for commands

At the prompt:

shift> hello
thinking> ...reasoning streams here...
assistant> ...answer streams here...

shift> /eval (define agent-system-prompt "Reply in uppercase.")
generation 2 ...

shift> hello
...response using the patched prompt...

shift> /rollback
generation 1 ...

shift> hello
...response using the restored prompt...

In a terminal, edit agent/default.scm while the process is running and the validated file is automatically activated as another atomic generation. Existing live patches are retained, and a turn already in flight remains pinned to the generation that started it. An invalid or incomplete save is rejected once, leaves the working generation active, and is retried after the file changes again. /reload is still available explicitly; /reload-clean intentionally drops session patches. Pass --no-watch to disable source watching. Redirected/scripted input defaults to watch off; pass --watch to opt in there.

Terminal interface

./bin/shift-agent
make

The TUI uses Python 3's standard-library curses adapter around the same Guile session. Ctrl+B toggles the inspector, Ctrl+P shows commands, F2 cycles the themes, and Ctrl+C cancels the active turn. Acid keeps a split transcript/output layout; /place bottom gives any theme a full-width transcript with bottom telemetry. Tab switches the Work, Diff, Session, Model and Log pane views, where Model lists and switches the provider's models, Session lists durable sessions with a status mark, loaded skills and MCP peers and switches between sessions, Log keeps bash run output and running background jobs, and a panes preference or a project's .shift/panes.scm pane pack adds tabs of your own built from text, live fields and allowlisted commands; /copy puts a reply on the clipboard; MCP peers appear with their calls; Ctrl+W/Ctrl+O fold real work/diffs. Acid uses a lowercase pixel wordmark and deep-purple RGB palette on capable terminals, with indexed-color/ASCII/monochrome fallbacks. For example:

/name thrashr888
/brand replace
/place bottom
/theme acid
/ui save user

Ask the default agent to change its interface and it can use the ui tool. Theme, placement, identity, colors, section order, and wordmarks are live data. Custom Scheme presentation packs reload when saved; invalid updates preserve the working view. No local model is required to try the interface:

./bin/shift-agent --session ui-demo --set 'agent-model="demo"'

Matching Ghostty terminal themes are generated from the same packs into site/ghostty/ and published on the website; see the website guide.

Durable named sessions

Use a named session when the conversation and live behavior should survive a process restart:

./bin/shift-agent --session dogfood       # resume if present, otherwise create
./bin/shift-agent --session recipes "Continue our Lisp cookie recipe."
./bin/shift-agent --new-session spike-2   # fail if the name already exists
./bin/shift-agent --resume dogfood        # fail if it does not exist
./bin/shift-agent --resume 00065c45fc…    # or the id the exit line printed
./bin/shift-agent --list-sessions
./bin/shift-agent session-fork dogfood experiment

The checkpoint at .shift/sessions/NAME/session.json is atomically rewritten after successful turns and interactive mutations. It stores the provider message history, next turn number, active live-patch source, generation number, and a stable session ID. Resume rebuilds a fresh Scheme module from the current agent source plus those validated patches; it never deserializes a live Guile object. Traces before and after restart therefore share session.id and session.name while retaining the exact generation that produced each result.

/session shows the current identity and checkpoint. /reset clears the persisted conversation but intentionally keeps live behavior. The rollback stack is process-local in this version: the active patch stack resumes, but older in-memory generations do not. Checkpoints are bounded at 8 MiB and 2,000 messages; large artifacts belong in files with references in the conversation. An advisory owner lock prevents two processes from opening and overwriting the same named session concurrently; different names remain fully concurrent. session-fork copies a fixed checkpoint into a distinct child identity while preserving its generation fingerprint and durable authority ceiling.

When the process closes, Shift reports provider-reported input and output token totals for that run, followed by the named session's resume command and stable ID. Providers that omit usage are shown as unavailable instead of estimated.

Older history compacts automatically after 80 provider messages, retaining at least 24 recent messages and a boundary-safe summary. Both values are live Scheme settings (agent-compaction-threshold and agent-compaction-keep-recent). /compact runs the same traced operation on demand. A failed or cancelled summary leaves the original checkpoint intact. Compacted turn evidence remains searchable in the session trace with /traces QUERY; /trace SPAN_ID retrieves the full stored span behind a hit.

Ctrl-C cancels an in-flight model call, tool call, or compaction and returns to the prompt without adding the interrupted turn to conversation history. Before each enabled tool executes, shift atomically records its name, arguments, and generation. If the process dies or a tool is cancelled, the next boot reports the ambiguous call. /recover inspects it, /recover retry repeats it only after an explicit user action, /recover restore puts interrupted file mutations back to their recorded pre-images, and /recover discard clears it without running anything. /undo reverts the last turn's file changes while every file still matches what that turn wrote. live_eval and extension mutations are never replayed because their post-crash state must be inspected instead.

The agent has the same constrained mechanism as the user. For example, ask:

shift> Update your prompt so you remember my name is Paul.

It calls live_eval with Scheme such as (set! agent-system-prompt ...) and first explains the binding, reason, and expected effect. The tool then reports the before/after generation IDs and fingerprints; the agent explains that result before continuing. It does not need to rewrite its source through a shell. Failed patches leave the current generation intact, and /rollback undoes a successful one.

Live patches are limited to 16 KiB and 64 active patches. Their top-level forms may only be define, define*, set!, or begin, and definitions or assignments must target agent-* or extension-* names. They still run in the curated Scheme module and the complete candidate image must pass validation before it activates. Every model tool execution crosses the process-owned mode and approval policy; prompt keywords never authorize mutations. This is a constrained language surface, not a resource or semantic sandbox.

docs/index.md lists every document and the open work. See daily-driver usage for saved settings, history, Claude, approval modes, context budgets, and the live localhost MCP endpoint.

Built-in extensions

Ollama, OpenAI-compatible transport, Claude, tracing, the MCP server, and the coding tools (status, diff, apply_patch, run) are shipped as trusted built-ins under extensions/shift/. They are enabled automatically; there is no per-session load command. Provider modules load only when selected. The terminal interface serves HTTP MCP at http://127.0.0.1:7331/mcp in the same Guile process, falling forward to the next free port up to 7340 when several sessions run; --no-mcp turns it off and --mcp-port PORT pins a port.

To keep a smaller process, set an explicit built-in allowlist before launch:

SHIFT_BUILTINS=ollama,tracing ./bin/shift-agent
SHIFT_BUILTINS=ollama ./bin/shift-agent                  # no trace files/exporter
SHIFT_BUILTINS= ./bin/shift-agent --agent test/session-agent.scm "hello"

An unset variable enables ollama,openai,claude,tracing,mcp,coding; an empty value disables all six. A disabled selected provider fails explicitly. This configuration is process-owned and inherited by child sessions, so live Scheme cannot re-enable a disabled built-in. Use a shell environment setting to retain your preference. The default works without any configuration.

These built-ins contain trusted I/O code and require a restart after edits. The restricted behavior artifacts below remain a separate live patch mechanism. See the ablation and implementation report.

Persistent extension artifacts

A useful live change can become a named Scheme artifact without rewriting the base agent image. Artifacts live in extensions/*.scm, are disabled when created or exported, and load through the same validated generation boundary:

/extensions
/extension-create terse (set! agent-system-prompt "Answer in one short paragraph.")
/extension-load terse
/extension-disable terse
/extension-export repaired-session

/extension-enable is an alias for /extension-load. Export collects the active live patch stack, so it can be loaded again after restarting the process. The agent can perform the same lifecycle through its extension tool: “save your current live changes as an extension named repaired-session.” Artifacts never auto-load and existing names are never overwritten.

Memorable demo: repair bad context live

The context-selection demo starts with a deliberately stale selector, lets the agent repair its own agent-select-context Scheme function, and retries without restarting the process:

make phoenix
make demo-context

Follow the short conversation in demo/context-selection/session.txt, or run it automatically with make demo-context-scripted. The first answer comes from the 2024 runbook in generation 1; the user naturally challenges that source, and the repaired selector uses the 2026 runbook in generation 2. /traces and Phoenix show both selected paths and exactly which generation produced each answer. The demo image also supplies eight agent-context-cases; a patched candidate that misses deployment or case variations is rejected before activation. See demo/context-selection/README.md.

Ollama default

Start Ollama, confirm the model is installed, and run the harness:

ollama list
./bin/shift-agent

The selected default advertises completion, tool calling, thinking, vision, and a 262K context window in the local Ollama metadata. It was chosen over the 104 GB qwen3.8-flash-next:125b-mlx model to keep interactive iteration practical. The live harness has been exercised end to end with this model making a read tool call and incorporating the result.

Native Ollama output streams by default. Thinking is hidden by default; agent-thinking may be #t, #f, or the model-specific levels 'low, 'medium, and 'high; agent-stream? controls streaming. agent-keep-alive is sent to Ollama as keep_alive and defaults to "10m", long enough for a normal interactive pause without pinning the model indefinitely. These are all live Scheme settings:

/thinking
/thinking medium
/thinking off
/stream off
/stream on

/eval (define agent-model "your-model-id")
/eval (define agent-thinking 'medium)
/eval (define agent-stream? #f)
/eval (define agent-keep-alive "30m")

For a remote OpenAI-compatible provider, also redefine agent-base-url and the name of the environment variable holding its credential:

/eval (define agent-provider 'openai)
/eval (define agent-base-url "https://your-provider.example/v1")
/eval (define agent-api-key-environment "OPENAI_API_KEY")

Set the environment variable before launching the process. The generic OpenAI-compatible requests support SSE streaming and streamed function calls. To exercise live generations without any model, set agent-model to "demo".

For OpenAI specifically, copy .env.example to the ignored .env, then export it into the process and load the checked-in disabled extension:

set -a; source .env; set +a
./bin/shift-agent --session openai

/extension-load openai-gateway

That extension selects gpt-5.4-mini at https://api.openai.com/v1 without changing the repository's Ollama default. The session checkpoint retains the selection after restart; the environment still has to contain OPENAI_API_KEY. Never commit .env.

OpenAI cache outcomes are explicit in completed LLM spans: search traces for llm.prompt_cache.status or inspect the compact cache=hit|miss annotation. The trace also records cached and uncached prompt-token counts.

The model can call read, rg, skill, write, edit, shell, traces, recall, spawn, live_eval, extension, and the coding built-in's status, diff, apply_patch, run and job. Runs on the allowlist never ask; /allow-run persists a prefix per session, project (.shift/settings.json) or user, and the scopes union. Skills are SKILL.md folders under .shift/skills, .agents/skills or the user config; the model loads one by name and never gains authority from it. run with background: true starts a job that reports back when it finishes. spawn starts a subagent the same way: a full session in an agents/ folder under the current one, nested again for grandchildren, joined with job wait. recall searches every session's traces in the project. MCP servers declared in .shift/mcp.scm supply more tools; tool_search finds them on demand so their schemas never sit in the prompt. Plugins under plugins/ bundle servers, skills, panes and live-image patches for agentkernel, cider, cortex, alchemy, allbeads, tauri-browser, 1Password and ripwire (call-graph context over MCP), and are on by default. The traces tool searches the complete local trace file while retaining only a bounded set of current-session hits. Hits are compact and carry stable span IDs; an exact span_id lookup returns the full stored span. This lets the agent follow generation decisions and tool evidence from before compaction without reinserting the whole history. File operations are canonicalized and restricted to the process working directory. rg invokes ripgrep directly and treats queries literally by default; its explicit regex: true mode is for intentional regular expressions. write replaces a complete file atomically, and edit requires an exact unique match unless replace_all is explicit. Shell commands require a confirmation for every call; the live image can tighten that policy to deny, but cannot set ambient allow. In an interactive terminal the approval consumes one y or N keystroke immediately, without waiting for Enter. live_eval can only use the curated Scheme surface and cannot introduce process, filesystem, network, dynamic-loading, or ambient evaluation authority.

Attach to the live session

Start ./bin/shift-agent, then connect an HTTP MCP client to the endpoint shown in the Session tab, http://127.0.0.1:7331/mcp for the first session. It shares the terminal's process, settings, conversation, generation, and approval policy and dies with the session. Clients that keep the Mcp-Session-Id an initialize returns are listed as peers with their calls. Use --mcp-port PORT to pin a port or --no-mcp to disable it.

shift_status, shift_prompt, shift_inspect, and shift_cancel operate on that live session. Mutating requests serialize; status remains available.

There is no separate MCP process to launch: every session serves the endpoint itself, so clients register the URL (.codex/config.toml does) and attach to the terminal you already have open.

See daily-driver MCP details for policy, transport, and session persistence.

Skills

A skill is a folder with a SKILL.md in the Agent Skills format, read from .shift/skills and .agents/skills in the project and from the user's config. The model sees an index of names and descriptions, loads one with the read-only skill tool, and gains no authority from it. The Session tab lists them; clicking one, or /skill NAME, sends it with the next prompt.

/learn turns work the session just did into a skill. This transcript is real, with the checked-in default image on a local Ollama model:

> Check this checkout's health: run git status --short and git log --oneline -3
  with the run tool, then tell me in two sentences whether the tree is clean and
  what the last commit was.
SHIFT  The tree is clean. The last commit was 9312272 — Persist run allowlists
       per scope, add sidebar source rows, and grow skills.
> /learn checkout-health
SHIFT  Both files written. …SKILL.md (57 lines) with the frontmatter, the two
       commands, pitfalls and what to verify; reference.md (77 lines) with the
       edge cases, linked from SKILL.md.

The result is committed at .shift/skills/checkout-health/SKILL.md: the model wrote it through the ordinary write tool and its approval, and the next turn's index picked it up.

Traces and Phoenix

Every completed turn writes OpenInference-shaped AGENT, LLM, and TOOL spans to the session's own traces.jsonl, under .shift/sessions/NAME/ (a subagent's under its folder). They include hierarchy, generation and turn IDs, model name, bounded inputs/outputs/thinking, duration, status, and Ollama or OpenAI token/cache counts. This is separate from the append-only audit journal at .shift/events.scm-log.

Enter /traces for a quick local summary, /traces deployment to search all completed spans in the current session, and /trace SPAN_ID to fetch a full stored span. For a full trace UI, start the official Phoenix container (requires Docker):

make phoenix

This starts Phoenix in the background, waits for its health check, exposes it at http://localhost:6006, and stores its database in a persistent Docker volume. Then launch the harness with OTLP export enabled:

make run-traced

Open http://localhost:6006 and select the shift project. The small Python bridge is dependency-isolated by uv and sends OTLP/HTTP protobuf to Phoenix asynchronously; tracing is still fully functional as local JSONL when it is absent. make run-traced fails fast if local Phoenix is not healthy. Set SHIFT_OTEL_ENDPOINT or PHOENIX_COLLECTOR_ENDPOINT to export to a different collector.

The live image and Codex bridge expose the same two-stage traces retrieval. Search scans the complete JSONL file, filters by stable current-session ID, and returns at most 50 compact newest-first hits. Exact span lookup returns stored attributes. This is an inspection surface, not deterministic replay; individual attributes are still truncated by the trace writer.

Ollama prompt-cache behavior is treated as a measurable optimization. The system message and persisted history form the request prefix; volatile selected context comes after that prefix. Normal and mutation-intent turns are separate tool-schema cache cohorts, and a generation change or compaction is an explicit cache boundary. LLM spans record the cohort, reusable-prefix size, dynamic context size, tool count, token counts, and Ollama load/prompt/generation durations. See durability and prompt-cache strategy.

OpenAI spans additionally expose an explicit cache=hit|miss summary plus cached and uncached token counts. OpenAI requests stream content and fragmented function calls over SSE and request the final usage chunk.

To inspect Phoenix logs or stop the container without deleting its trace volume:

make phoenix-logs
make phoenix-down

Current boundary

The stable runtime provides:

• Fresh-module evaluation and validation
• Atomic activation and rollback
• An append-only S-expression event journal plus structured JSONL spans
• Native streaming Ollama and OpenAI Chat Completions transports, including streamed function calls, plus bounded tool output
• A curated live-language interface without process, filesystem, network, dynamic-loading, module-resolution, or eval functions
• Project-confined read, search, atomic write, and exact-edit capabilities
• Complete-session trace search plus bounded exact-span inspection as a model tool
• Named, non-overwriting Scheme extension artifacts that can be loaded, disabled, or exported from active live patches
• Atomic named-session checkpoints with conversation, active patches, generation continuity, and stable trace identity
• Session checkpoint forks plus one-child supervision with durable tool ceilings, cancellation, linked causal spans, deterministic tool assertions, and disabled extension proposals kept in ignored child-session state until explicit promotion
• Boundary-safe model compaction, in-flight cancellation, and explicit interrupted-tool retry/discard recovery
• Hard authority validation (agent-shell-policy may be deny or ask, never ambient allow)

The live image provides:

• Agent name, model selection, and system prompt
• Provider, base URL, API-key environment-variable name, streaming, thinking, and Ollama model keep-alive
• Enabled tool names
• Maximum tool-call rounds
• Compaction threshold and recent-message retention
• Context selection, user-input transformation, and demo behavior
• Shell policy within the authority ceiling

Each request stays pinned to the generation that began it. New code can activate while a request is running without changing that request's pinned generation. Failed evaluations and invalid watched saves leave the active generation intact.

Layout

agent/default.scm             live user-owned image
src/live-agent/generation.scm isolated module generations
src/live-agent/runtime.scm    transitions, rollback, journal
src/live-agent/session.scm    durable named-session checkpoints
src/live-agent/compaction.scm boundary-safe history reduction
src/live-agent/recovery.scm   interrupted-tool write-ahead record
src/live-agent/extensions.scm persistent artifact lifecycle
src/live-agent/provider.scm   completion data and lazy provider dispatch
src/live-agent/prompt.scm     cache-friendly request assembly and persistence split
src/live-agent/trace.scm      optional tracing boundary
src/live-agent/tools.scm      stable capability checks
src/live-agent/json.scm       dependency-free JSON codec
src/live-agent/main.scm       interactive shell
src/live-agent/builtins.scm   fixed, process-owned built-in selection
extensions/shift/mcp.scm     in-process HTTP and stdio MCP server
extensions/shift/claude.scm  native Claude provider
extensions/shift/ollama.scm    native Ollama provider
extensions/shift/openai.scm    OpenAI-compatible provider
extensions/shift/http.scm      shared bounded curl transport
extensions/shift/tracing.scm  JSONL spans and search
extensions/shift/otel.py      optional Phoenix/OTLP exporter
bin/shift-agent               the entry point (shift is a shell builtin)
.codex/config.toml            local MCP registration over HTTP
extensions/README.md          artifact contract
test/*                        Scheme runtime tests plus Python integration tests

The internal Guile module namespace remains (live-agent ...).

Journal and trace attribute values larger than 4 KiB are truncated. Tool output is bounded at 64 KiB, writes at 256 KiB, and edited source files at 512 KiB; larger artifacts should be externalized and referenced.

Gaps versus Pi

This spike is testing a narrower idea than Pi. It now has one concrete advantage to evaluate—transactional, generation-attributed live behavior—but it lacks most of the product surface of a mature coding harness. See docs/gaps-with-pi.md for the direct comparison and the gaps on both sides.

Subagents and recall

spawn forks the live image into a child session that lives in a folder under its parent, .shift/sessions/NAME/agents/CHILD/, and runs it as a background job. The child keeps the parent's generation, patches, mode and project but starts with an empty conversation and a narrower tool ceiling; job wait returns its whole answer. Several children run in parallel, and a child can spawn its own. Every one is an ordinary session: it shows nested in the Session tab and resumes with --resume NAME/agents/CHILD.

recall and /recall QUERY search every session's traces in the project, children included, and name the session behind each hit. The design and its decisions are in docs/subagents.md.

License

MIT © 2026 Paul Thrasher.