# Agent / automation control

Spiralyst Lab is **scriptable**. Every feature has a programmatic seam, so an
external agent, a MIDI/OSC bridge, a batch script, an LLM, or an automated QA
harness can drive and inspect the app without simulating mouse clicks. This
page is the public contract for that surface.

> Design principle: *no feature ships UI without a programmatic equivalent.*
> Internally this is the "agent-operability principle" (see `CLAUDE.md` +
> `docs/code-standards.md` §1.2/§6). This page is the user-facing version.
>
> Driving creative/reactive scenes? Read [`agent-creative-notes.md`](./agent-creative-notes.md)
> first — hard-won gotchas (e.g. `/api/state` is stale for animated values; use
> screenshots + `/api/audio/bands` to verify; 2D vs 3D zoom controls).

---

## Two surfaces

| Surface | Where | Best for |
|---|---|---|
| **`window.__spiral.*`** | in the WebView (Develop → Web Inspector → Console, or any JS-injecting agent / the headless harness) | synchronous, zero-latency control + inspection |
| **`/api/*` HTTP** | the app's embedded localhost server | external clients with no JS (curl, MIDI bridge, another process) |

Both read and write through the **same** `snapshotState()` / `restoreState()`
serializer the UI uses — there's no parallel state format. The JS surface is
immediate; the HTTP surface is applied within one ~250 ms poll tick.

The HTTP server binds a **random localhost port** each launch, and (Security V2)
**every `/api/*` request requires a per-launch auth token**. Both are written to
**`~/.spiralyst-lab/api-port.json`** on launch:

```json
{ "url": "http://127.0.0.1:54321", "port": 54321,
  "token": "…64 hex chars…", "pid": 1234, "version": "3.3.0", "at": 1717000000 }
```

Read `url` + `token` from that file and send the token as
`Authorization: Bearer <token>` (or `X-Spiral-Token: <token>`) on every request.
The loopback/Origin guard still applies (defense in depth); a tokenless or
wrong-token request gets **401**. The app's own WebView authenticates
transparently, and the URL + token are also shown in **About ▸ Spiralyst Lab**
(with Copy). The file is `0600` + deleted on clean exit.

---

## Quickstart

**In the console** (instant):

```js
__spiral.state.get()                                  // full scene snapshot
__spiral.state.patch({ color: { hueStart: 0.9 } })    // set one field → re-renders
__spiral.state.patch({ type: "mandelbulb",
  params: { power: 8, iterations: 10, bailout: 2, epsilon: 0.0008, maxSteps: 96 } })
__spiral.undo()  /  __spiral.redo()
__spiral.ui.tab("camera")                             // switch sidebar tab
__spiral.ui.scrollTo("color:hueStart")                // centre a control
__spiral.audio.snapshot()                             // live audio input the artwork reacts to
```

**Over HTTP** (external):

```bash
# Base URL + auth token the app writes on launch (Security V2):
P=~/.spiralyst-lab/api-port.json
BASE=$(jq -r .url "$P"); AUTH="Authorization: Bearer $(jq -r .token "$P")"
curl -s -H "$AUTH" "$BASE/api/state" | jq '{type, hueStart: .color.hueStart}'
curl -s -H "$AUTH" "$BASE/api/state" \
  | jq '.color.hueStart = 0.9' \
  | curl -s -H "$AUTH" -XPOST "$BASE/api/state/apply" -H 'Content-Type: application/json' -d @-
curl -s -H "$AUTH" -XPOST "$BASE/api/state/undo"
```

`scripts/demo-scene.sh` is a worked example (enables system audio + selects
phyllotaxis + animates Hue Start, all via curl).

---

## Reference

### `window.__spiral`

| Call | Returns | Notes |
|---|---|---|
| `.version` | string | app version |
| `.state.get()` | snapshot blob | full scene (v=10 preset shape) |
| `.state.set(snap)` | bool | replace whole state; pushes an undo entry |
| `.state.patch(partial)` | bool | deep-merge a partial then apply (set one field) |
| `.undo()` / `.redo()` | bool | |
| `.stack()` | `{undo,redo,canUndo,canRedo}` | |
| `.render()` | true | force a repaint (before reading pixels) |
| `.resetType()` | bool | reset the CURRENT fractal scene to defaults (re-seed shape params + clear color/transform/effects/layers/reactivity); one undoable step; no confirm dialog on the agent path |
| `.ui.tab(name)` | bool | `source`/`visual`/`camera`/`export` |
| `.ui.currentTab()` | string | |
| `.ui.tabs()` | string[] | |
| `.ui.scrollTo(target)` | bool | `"group:key"` (e.g. `"color:hueStart"`) or element id; auto-activates the tab |
| `.ui.math(on)` | bool | show / hide the **Math Mode** overlay (the KaTeX-rendered live formula for the current fractal). Persists across launches; syncs the File → Show Math Mode checkmark. |
| `.ui.mathVisible()` | bool | current Math Mode visibility (reads the persisted store) |
| `.ui.mathState()` | `{type,title,formula,evaluated,sweep,description,symbols[]}` | KaTeX-free snapshot of what Math Mode currently shows — usable for QA verification without a screenshot. `symbols[i] = {glyph,label,path,value,fixed}`. |
| `.ui.mathInfo(on)` | bool | show / hide the **Math Mode info card** (middle z-layer between the canvas and the formula tile — covers most of the viewport with a right gutter reserved for the formula tile). Independent of `.ui.math()` (own store key). Hosts textbook-depth content from the site's `content.json` bank. |
| `.ui.mathInfoVisible()` | bool | current info-card visibility |
| `.ui.mathInfoState()` | `{type,title,short,long[],math[{formula,note}],implementation,fact}` | KaTeX-free snapshot of the info-card content for the current type, or empty fields if no entry exists. |
| `.ui.mathAudio.play(section)` | url \| null | start narration for the current fractal's `section` (`"main"`/`"math"`/`"app"`/`"fact"`). Stops any prior playback. Returns the OGG URL, or `null` if the (type, section) has no audio entry. |
| `.ui.mathAudio.stop()` | true | halt narration |
| `.ui.mathAudio.current()` | `{typeKey,section}` \| null | what's currently playing — for icon-glow sync + QA verification without listening |
| `.audio.snapshot()` | frame | `{source,active,sampleRate,fftSize,binCount,peak,rms,pitchHz,centroidHz,flux,spectrumEnergy,spectrum[32]}` |
| `.audio.bins()` | number[] | full raw byte-FFT array the artwork samples |
| `.audio.bands()` | array | live per-reactive-slider levels (see `GET /api/audio/bands` below) |
| `.audio.testTone(opts)` | info\|`{error}` | play a calibration tone (see below) |
| `.audio.stopTone()` | true | |
| `.audio.toneInfo()` | info\|null | |
| `.audio.heal(opts)` | report (async) | auto-heal / Auto-Reactivity — retune reactive bands (see below) |
| `.audio.saveBand(name, key)` | `{ok,name,band}`\|`{error}` | save slider `key`'s current band to the library as `name` |
| `.audio.applyBand(name, toKey)` | `{ok,…}`\|`{error}` | map saved band `name` onto slider `toKey` (its own min/max range is kept) |
| `.audio.bandLibrary()` | array | the saved band-preset library |
| `.audio.pcm.enable({stereo})` | `{mode,sampleRate}` (async) | feature/scopes — start the PCM emit seam (mono if `stereo:false`, stereo if `true`). Normally driven automatically when a `scope-*` type is selected; manual calls let an agent measure bandwidth/latency in isolation |
| `.audio.pcm.disable()` | true (async) | tear the PCM emit channel down |
| `.audio.pcm.diagnose()` | `{js,rust}` (async) | bandwidth/latency telemetry: `js.{framesPerSec,bytesPerSec,callbackP95Ms,callbackMaxMs,ringFillSamples,uptimeMs}` + `rust.{mode,framesEmitted,bytesEmitted,dropped,framesPerSec,bytesPerSec}` |
| `.audio.pcm.getMode()` | `0\|1\|2` | 0 off, 1 mono, 2 stereo |
| `.audio.pcm.getSampleRate()` | Hz | from the PCM1 wire header |
| `.scope.liveMonitor(on)` | true \| false (async) | feature/scopes — RUN/STOP for the active scope. Toggle off → render loop pauses on the current frame. Harmonograph: also wires/unwires Freq A + Freq B audio bindings. Returns `false` if the active type isn't a scope; idempotent |
| `.scope.liveMonitorState()` | bool \| null | current Live monitor state, or `null` when no scope is active |
| `.scope.autoGain(on)` | true \| false (async) | feature/scopes — toggle the active scope's auto-fit normalize. Oscilloscope/Lissajous: rolling-peak amplitude normalize. Spectrum: bin-energy gain pulse. Harmonograph doesn't have it (no amplitude data in the draw) → returns `false`. Idempotent |
| `.scope.autoGainState()` | bool \| null | current Auto-gain state, or `null` when no scope active / toggle absent |
| `.scope.setToggle(key, on)` | true \| false (async) | generic setter for any per-scope `{kind:"toggle"}` control (covers `live`/`autoGain` plus any future scope toggles like spectrum's `logFreq`/`style`) |
| `.scope.getToggle(key)` | bool \| null | generic reader for any per-scope toggle |
| `.export.png(opts)` | report (async) | export the rendered viewport to PNG (exact pixels). `opts:{res?,w?,h?,ppi?,name?}`; writes to `~/.spiralyst-lab/qa/`, returns `{ok,path,w,h,ppi,bytes,at}` |
| `.eula.version` | string | the current EULA version constant |
| `.eula.status()` | acceptance (async) | the stored first-launch EULA acceptance |
| `.eula.accept()` | result (async) | persist acceptance (clears the gate for headless/CI) |
| `.feed.enable(on)` / `.enabled()` | bool | sidebar agent-activity toasts |
| `.feed.note(text)` | true | post a custom toast |

### `/api/*` routes

| Method + path | Body | Effect |
|---|---|---|
| `GET /api/state` | — | last pushed snapshot (or null) |
| `POST /api/state/apply` | **full** snapshot blob | apply (≤250 ms). Body must be a **complete** snapshot — `restoreState` rejects partials silently (a partial `{type:"X"}` no-ops). Compose with `GET /api/state` (current snapshot) → mutate → POST. ⚠ A full-snapshot apply **clobbers live audio bands** (the mirror reports active bands as `mode:"off"`), so an agent that wants to preserve reactivity must re-apply each band via `POST /api/audio/band/apply` after. The JS-side `__spiral.state.patch()` already deep-merges; the HTTP route does not. |
| `POST /api/state/undo` · `…/redo` | — | history |
| `GET /api/state/stack` | — | `{undo,redo,canUndo,canRedo}` |
| `POST /api/state/tab` | `{name}` | switch sidebar tab |
| `POST /api/state/scroll` | `{target}` | scroll a control into view |
| `POST /api/window/detach` | — | detach the control panel into its own window (the JS poll invokes the Tauri opener — window creation needs a UI-thread invoke) |
| `POST /api/window/reattach` | — | close the detached panel (reattach the sidebar) |
| `GET /api/audio/snapshot` | — | latest audio-input frame — served fresh from Rust at the FFT hop rate (~94 Hz) while the system tap is active; falls back to the JS-pushed mirror for the mic path / inactive |
| `GET /api/audio/frames` | `?since=N&max=M` | full-resolution recent frames (~94 Hz): `{frames,lastSeq,count,dropped}` — one GET covers a window with no aliasing at any tempo |
| `GET /api/audio/bands` | — | live per-reactive-slider levels: `{bands:[{key,lo,hi,signal,gain,level,peak,ema,saturated,lowDynamics,min,max}]}` |
| `GET /api/audio/pcm/diagnose` | — | feature/scopes — Rust-side counters for the PCM emit seam: `{mode,channels,sampleRate,framesEmitted,bytesEmitted,dropped,uptimeMs,framesPerSec,bytesPerSec}`. Mode `0` means PCM is off (no scope mode active or `audio.pcm.disable()` was called) |
| `POST /api/audio/heal` | `{keys?,targetPeak?,policy?,signalCandidates?}` | enqueue auto-heal (AGC); runs in-app over a few seconds |
| `GET /api/audio/heal` | — | the last heal report `{healed:[{key,before,after,note}],at}` (poll after POST) |
| `POST /api/audio/band/save` | `{name,key}` | save slider `key`'s current band to the library as `name` |
| `POST /api/audio/band/apply` | `{name,key}` | map saved band `name` onto slider `key` (its own min/max range is kept) |
| `GET /api/audio/status` | — | tap state `{active,sample_rate,frames_emitted}` |
| `GET /api/audio/diagnose` | — | one-shot tap self-test (capture diagnostics) |
| `GET /api/audio/devices` | — | list audio input devices |
| `POST /api/audio/tone` | `{freqHz,bpm,gain,…}` | start the calibration tone |
| `POST /api/audio/tone/stop` | — | stop the calibration tone |
| `POST /api/export/png` | `{res?,w?,h?,ppi?,name?}` | enqueue a viewport→PNG export the JS poll renders (native size if empty) |
| `GET /api/export/png` | — | the last export report `{path,w,h,ppi,bytes,at}` (poll after POST) |

**Reactivity observability + auto-heal (alpha.66).** A reactive slider's live
output is invisible through `/api/state` (push-on-edit, frozen during
animation). `GET /api/audio/bands` exposes each audio-mode slider's actual
level: `level`/`peak`/`ema` are post-gain, **pre-clamp** (≥1 ⇒ the band
saturates and the slider pins at its max); `saturated` and `lowDynamics`
(`peak ≈ ema` ⇒ no transient energy in the band) are precomputed. the auto-heal
(below) is built on this. Requires music playing.

**Auto-heal / "Auto-Reactivity" (alpha.67).** The easy button: one function,
shipped across the GUI (per-band "✨ Auto-React" in the reactivity popover +
"✨ Auto-Reactivity" in the Source tab), the agentic API, and the CLI — engine
in `scripts/audio-heal.js` (don't fork it). It finds the best mapping of audio
signal → visual motion across FOUR levers: **gain** (scaled so the peak lands
~0.85, iterating since the level clamps at 1.0 and hides the overshoot),
**squelch** (`noiseFloor` — raises the floor so a steady band's constant level
gates to 0 and only peaks register), **signal** (switches a flat band to
`flux`/onset energy), and **envelope** (a snappy attack/release so the level
falls between hits → more excursion). With `policy:"auto"` (default) it searches
squelch + signal + envelope for the most responsive config, then re-tunes gain.
It only ever switches the signal when the band is **steady** (no dynamics) and
another signal moves more — a band that already reacts keeps its signal. To heal
*within* a signal (e.g. stay in amplitude, just dial gain + squelch + envelope),
pass `lockSignal:true`. `gainOnly` tunes only gain. **Tone-graded by default** (alpha.76): when a band needs the config search, the
heal pulses a calibration tone in the band at an unusual tempo (211 BPM) and
scores each config by the *power of its band-level at that beat frequency*
(pulse-correlation), so live music — at its own tempo — can't skew the A/B even
without pausing. Only magnitude signals (amplitude/flux) are tone-graded; gain
converges against the real audio afterward. It plays an audible tone for a few
seconds *only* when a band is steady enough to need the search. Pass
`useTone:false` (CLI `--no-tone`) for the quick, music-only path.
Agentic mutations resync the UI (🔊 label + open popover) and the `/api/state`
mirror, so the audio band menu reflects what changed; a banner shows while a
heal runs. Drive it:

- Console / in-page: `await window.__spiral.audio.heal({ policy: "auto" })`
- HTTP: `POST /api/audio/heal {opts}` then poll `GET /api/audio/heal` for the
  report (it runs for a few seconds — settle windows).
- CLI: `node scripts/agent-heal-audio.mjs [--gain-only] [--key group:key]`
- Skill: `/spiral-heal` (see `~/.claude/skills/`).

Needs music playing; heals nothing against silence (it says so).

**Fast audio read (alpha.65).** The audio analysis runs in Rust at ~94 Hz
(one frame per FFT hop). For at-a-glance reads, `GET /api/audio/snapshot` now
serves that frame fresh (no longer the 250 ms JS mirror). For *measurement*
(beat timing, true latency), `GET /api/audio/frames?since=<seq>` returns the
whole recent window at full resolution in one call — baseline on `lastSeq`,
wait, fetch again. Each frame carries a monotonic `tMs`, so periods/intervals
come straight from the deltas. `dropped:true` means you fell behind the ~2.7 s
ring; re-baseline on `lastSeq`. The push to the Rust ring is one allocation-free
O(1) write per hop on its own mutex — never the audio-compute or render path
(CLAUDE.md §2.7). For sub-hop (<~11 ms) precision the Rust impulse generator
(`audio_impulse.rs`) remains the reference.

**Viewport PNG export.** `await __spiral.export.png({ res: "4K" })` — or
`POST /api/export/png` then `GET /api/export/png` for the path — renders the
active 2D/3D viewport straight to a PNG at any resolution via the Export-tab
engine (`recording.js exportPngAt`): exact pixels, not a screen grab, written to
`~/.spiralyst-lab/qa/`. This is the preferred "read pixels" leg for QA and for
batch art capture. `opts: { res? (a REC_PRESETS key like "4K"/"1080p"), w?, h?,
ppi?, name? }`; omit size for the live canvas resolution. Runs off the
audio→visual hot path (POST enqueues; the render happens on the JS poll).

Full HTTP contract: `docs/backend-api.md` §2.8–§2.9.

---

## Cookbook

### Automated QA loop — drive, verify, observe

The full loop has four legs, no human required:

1. **Drive** — `__spiral.state.patch(scene)` (deep-merges a partial). The HTTP analog `POST /api/state/apply` needs a **full** snapshot — `GET /api/state` → mutate → POST.
2. **Read state** — `__spiral.state.get()` / `GET /api/state` (numeric output).
3. **Read pixels** — export the rendered viewport to PNG (exact pixels, any
   resolution): `await __spiral.export.png({ res: "4K" })`, or over HTTP
   `POST /api/export/png {res:"4K"}` then `GET /api/export/png` for the path
   (written to `~/.spiralyst-lab/qa/`). Read that PNG to grade the render. (For
   multi-frame *motion* capture, `scripts/qa-snapshot.sh --frames N --interval S`
   still does a timed burst: 2 = motion, 3 = velocity, 5–10 = accurate motion.)
4. **Read audio input** — `__spiral.audio.snapshot()` / `GET /api/audio/snapshot`
   (the values the artwork reacts to).

Grade by: patch a known scene → `render()` → read all three → compare to a
baseline. This generalizes the calibration pattern (synthetic input →
measured output → grade) to any feature.

### Calibration with a ground-truth tone

`__spiral.audio.testTone()` plays a known tone to system output. With system
audio capture on, the tap processes it through the full path (FFT + MPM
pitch), so you can measure accuracy against ground truth:

```js
__spiral.audio.testTone({ freqHz: 440, bpm: 120, gain: 0.5, waveform: "sine" });
// then, over a window:
__spiral.audio.snapshot().pitchHz   // → should track ~440 (pitch accuracy)
__spiral.audio.snapshot().peak      // → pulses at 120 BPM (gain/threshold + beat)
// onset (you control the schedule) vs visual change → true end-to-end latency
__spiral.audio.stopTone();
```

`bpm: 0` gives a steady continuous tone (pure pitch/gain check, no beat).

**Automated grading harness** — `scripts/calibrate-audio.mjs` does the whole
loop and writes a graded report:

```bash
node scripts/calibrate-audio.mjs --freq 440 --bpm 120 --secs 4
#   → ~/.spiralyst-lab/calibration/audio-<ts>.{jsonl,md}
#   grades: audio active, energy response, pitch accuracy (cents),
#           beat detection (BPM), input latency (ms)
```

It enables system audio, samples a pre-tone baseline, plays the tone over
HTTP (`/api/audio/tone`), samples `/api/audio/snapshot` across the window,
stops the tone, and grades against the known input. Requires the app running
(alpha.63+) and macOS Screen-Recording permission so the tap captures the
tone. (Audible — run in a quiet room or route output to a loopback device.)

### Drive from a MIDI / OSC bridge

Map controller events → `__spiral.state.patch` (or `POST /api/state/apply` with a full snapshot — see the route's row above for the deep-merge caveat)
with the parameters you want to nudge. State round-trips through the preset
format, so a controller "scene recall" is just a `POST /api/state/apply` of a
saved preset blob.

### Feed an LLM

`__spiral.audio.snapshot()` / `.bins()` give a model the exact spectral input
the artwork uses. A digested window (mean/peak/variance per bucket) is an
ingestible format for asking a model to pick a band + threshold; apply its
answer via `state.patch`, then verify with the QA legs above.

---

## Watching what an agent does

File menu → **"Show Agent Activity"** (a native checkbox menu item; moved here
from the Export tab in alpha.86) surfaces every programmatic action as a small
toast at the bottom of the sidebar (never over the viewport). Off by default; a
per-machine UI preference (not saved in presets). Toggle it programmatically
with `__spiral.feed.enable(true)`.

File menu → **"Show Math Mode"** (a sibling CheckMenuItem) toggles the
KaTeX-rendered live formula overlay (purple-bordered, top-right corner of the
canvas). Persisted across launches in `~/.spiralyst-lab/store/spiralystMathMode.v1`.
Toggle programmatically with `__spiral.ui.math(true)`; verify without a
screenshot via `__spiral.ui.mathState()` (returns `{type, title, formula,
evaluated, symbols[{glyph,label,value}]}` — a KaTeX-free snapshot of the panel).

When Math Mode is on, the formula tile carries a **"Read more ⓘ"** button that
toggles the Math Mode **info card** — a middle-z-layer panel (between the canvas
and the formula tile) with textbook-depth content for the current fractal:
short summary, long-form paragraphs, a list of supplementary formulas with
notes, the codebase-accurate implementation paragraph, and a "did you know"
fact. Content is vendored from the site's `web/copy/fractals/content.json`
into `scripts/math-content.js`. Persistence: `~/.spiralyst-lab/store/spiralystMathInfo.v1`.
Agent toggle: `__spiral.ui.mathInfo(true)`; KaTeX-free verification:
`__spiral.ui.mathInfoState()`.

Each of the four info-card section headings carries a **purple speaker icon**
trailing the heading text. Click → that section's pre-rendered TTS narration
plays through the OS audio output; the icon flips to a filled, glowing state.
Click it again (or another section's icon) to stop / swap. One section at a
time. Closing the info card stops playback; switching the fractal type stops
it too. Agent control: `__spiral.ui.mathAudio.play("main"|"math"|"app"|"fact")`
+ `.stop()` + `.current()`. Sysaudio reactivity stays live during narration
(the CoreAudio Process Tap captures both music + narration); a v3 opt-in
toggle is on the backlog.

---

## Licensing — deliberately NOT on the agent surface

Licensing is the **one feature intentionally excluded** from the programmatic
surface — a security-sensitive exception to the agent-operability principle
(owner decision 2026-05-25). Concretely:

- **There is no `window.__spiral.license`.** The agent JS API does not expose
  license status, activation, or any license control. (The app drives licensing
  internally via `scripts/license.js`.)
- **The `/api/license/*` routes are local app plumbing, not a public agent
  API.** `GET /api/license/status`, `POST /api/license/activate`, and the
  debug-only `POST /api/license/test-clock` exist solely so the app's own JS can
  talk to the Rust verifier. Do **not** treat them as part of the scriptable
  surface above — they are not a supported automation contract and may change.
- **Serial minting is out of band.** Serials are generated only by a standalone,
  out-of-repo security tool (the license-gen CLI); the app and its agent surface
  never mint or sign keys.

See [`licensing.md`](./licensing.md) for the licensing model itself.