# AI / Overseer Audit — the terminal still behaves like the AI is in charge **Snapshot as of 2026-07-18 · branch `dev` @ `9b0d26b` · ANALYSIS ONLY — no code changed.** ARCHIVE-class planning doc (3-class library model): the reasoning is reusable, the line numbers and measurements are point-in-time and will drift — trust the code where they disagree. **Evidence basis.** The owner ran a live AI session today — the first real end-to-end Director-Link test since the app went offline-first — and sent screenshots of the actual on-screen UI. Every finding below was checked against those screenshots AND against the current code (the file/line pointers are the code confirmation), not against logs or memory. Where a finding is owner-verified from a screenshot but not independently reproduced here, that is stated. --- ## The framing (why this audit exists) This app began life as a single Gemini prompt and grew into an application. In the years since, most of the mechanical work was **converted to native, deterministic, offline code** — the combat/threat/barter/databank/medical calculators, the native command router, the trackers, level-up, the world map, and loot import all run with no AI call. But the **AI system directive** (`js/services/api-directive.js`) was written back when the AI did everything, and the **machinery around the terminal** — the state-import path, the retry surface, the ambient chatter, the post-sync auto-navigation — still behaves as though the AI is the authority. The terminal is now the primary surface. The AI is now one optional, bring-your-own-key subsystem among many. The findings below are all one theme seen from eight angles: **the plumbing hasn't caught up to that shift**, and in one case (Finding 1) that gap can silently destroy a player's items. --- ## Finding 1 — ⚠ SILENT ITEM DELETION IS REAL (confirmed real state loss, not a miscount) — this jumps the queue **Owner's report.** Change-lines across consecutive turns showed `inventory: 0→1` (a 10mm pistol looted), then on the very next turn — a *failed* armour repair and an *aborted* craft that changed nothing — `inventory: 1→0`. An item appeared to be deleted by a turn that did nothing. **Verdict: this is REAL item loss, not a wrong counter.** Both were named as candidate bugs; verification against the code says it is the worse of the two. Mechanism, confirmed in code: - **The AI-import path does a FULL REPLACE of `state.inventory`, not a merge.** `autoImportState()` (`js/services/api-import.js`, ~line 369–420): when the parsed AI response contains an `inventory` array, it runs `state.inventory = inv.map(...)` — the entire durable inventory is **overwritten** by whatever array the AI returned this turn. The only thing that "merges" is the *acquired-item animation* guard (`_invNamesBefore`, ~line 376) — the animation is suppressed for items the player already had, but the **state itself is a hard assignment**. An empty array (`inventory: []`) is truthy and `Array.isArray([])` is true (~line 370), so an AI turn that returns an empty inventory array **wipes every item**. - **The directive INSTRUCTS this full-array contract.** `api-directive.js` ~line 120: *"Inventory & Squad Persistence (CRITICAL): If the Courier loots an item or uses `> [CRAFT]`, you MUST return the ENTIRE inventory array."* The AI is told to resend the whole array on any inventory-touching turn — and a failed-repair / aborted-craft turn is exactly the kind of turn where the model can misjudge and emit a short (or empty) array. The full-replace then makes that the new truth. - **The `[DELTA]` line is telling the TRUTH, which is why it's alarming.** The `inventory: 1→0 items` line (`api-import.js` ~line 548–555) is computed by counting the real `state.inventory` length *before* vs *after* the sync. `1→0` means the array genuinely went from one item to zero. The counter is not wrong — it is faithfully reporting that the item was actually deleted from durable state. **Why this is the same class as save durability.** Protocol 24 says the AI is never the sole authority over durable state and its output must be validated before it persists. Inventory is a native-owned concept now (loot import, the manifest, equip, quantity steppers all write it natively) — yet the AI can silently overwrite the whole array with no reconciliation against what the player actually owned. This is precisely the "AI writes over state the native path owns" risk, and the damage (lost items, no recovery, no confirmation) is a data-loss event, not a cosmetic one. **Note on the existing guard.** The registry-leak hardening guard added during the FO3 work checks that the AI-import data lookup matches the campaign's *current game* before trusting it — that stops a cross-game mismatch from deleting items as "garbage." It does **not** cover this case: a same-game turn where the AI simply returns a short/empty inventory array. That path is unguarded. **Recommended fix direction (for the plan stage, not decided here).** Treat an AI inventory array as a *proposal*, reconciled against current state rather than blindly assigned: e.g. never let an AI turn remove an item the player natively holds without an explicit, narratively-grounded removal signal; or require a confirm for AI-driven net removals; or make inventory AI-read-only and drive all inventory mutation through the native path. The right shape is a plan-stage decision — but the guarding principle is Protocol 24 (validate + field-map, never blind-persist) and the regression test is mandatory (Protocol 13): a turn returning a short/empty inventory array must not delete natively-held items. **Placement: JUMPS THE QUEUE.** This is the only finding here that can silently destroy player data during *ordinary play*, with no external precondition (contrast Finding A1 in QUEUE — live-save durability — which needs a platform storage-eviction event to trigger). It belongs at the **very front of the 2.8.5 data-safety group, ahead of A1.** See the placement recommendation at the end of this doc. --- ## Finding 2 — Retry echo: one send, four copies of the message + three status lines **Owner's report (screenshot-verified).** On a network failure the app reprinted the user's message once per retry attempt, each with an extra accumulating `>` prefix (`Yo`, `> > Yo`, `> > > Yo`, `> > > > Yo`), plus a separate `[SYS] RETRANSMITTING… (attempt N/3)` line each time. One send produced four copies of the message and three status lines. **Mechanism, confirmed in code.** The transient-failure retry path (`js/services/api.js` ~line 526–543) schedules a re-send by writing the last user text back into `#chatInput` and calling `transmitMessage()` again. `transmitMessage()` unconditionally re-echoes the user line — `appendToChat(\`> ${displayUserText}\`, 'user')` (~line 236) — so the message reprints on every attempt. A `[SYS] RETRANSMITTING...` line is appended each attempt (~line 531). The accumulating `>` prefix comes from the retry reading the *already-prefixed* last-user text back out of history and re-prefixing it. **Agreed fix (owner-approved design — recorded so the plan stage doesn't relitigate it).** - Print the user's line **once** and **mutate it in place** across attempts rather than reprinting. - Keep a **single** status line underneath that counts `1/3 → 2/3 → 3/3`, replaced by the reply on success or by the error on failure. - **Keep** the accumulating `>` prefix — but make it *deliberate*: it reads diegetically as another relay hop pushing the signal further out. The counter line stays underneath because the prefix alone isn't legible as "third attempt" — the fiction carries the feel, the counter carries the meaning. **Severity:** cosmetic/UX, but it makes a routine transient failure look like the app is malfunctioning. --- ## Finding 3 — Wrong severity: a transient network blip renders as a FATAL EXCEPTION **Owner's report (screenshot-verified).** A failed fetch rendered as `⚠ FATAL EXCEPTION AT 0xA38F — MODULE: COMM_LINK — Failed to fetch`. The app was fine immediately afterward; the next message worked. **Mechanism, confirmed in code.** After the retry ladder is exhausted, a transient 5xx / network-zero failure prints the **same** `⚠ FATAL EXCEPTION AT 0x…… — MODULE: COMM_LINK — ${error.message}` framing (`api.js` ~line 552–558) that is also used for a **missing API key** (~line 268) and a **JSON parse failure** (~line 463). A recoverable, self-healing network blip is dressed in identical catastrophic language to a genuine hard fault. **Fix direction.** Distinguish transient/recoverable from fatal. A transient network failure wants a lower-severity, self-healing-flavoured line ("SIGNAL LOST — DIRECTOR LINK WILL RETRY / STILL FULLY USABLE OFFLINE"), reserving FATAL EXCEPTION for genuinely fatal conditions. Good flavour, wrong severity is the whole finding — the fix is severity-matching, not removing the flavour. **Severity:** cosmetic, but actively misleading — it reads as "the app broke" when nothing broke. --- ## Finding 4 — Persona is not game-agnostic ("Courier" in a Fallout 3 game) **Owner's report (screenshot-verified).** In a Fallout 3 campaign the AI repeatedly addressed the player as **"Courier"** — the New Vegas protagonist. Fallout 3's player character is the **Lone Wanderer**. Notably, the **ambient** text was correctly game-aware in the same session (GNR at 98.3 MHz, "Capital Wasteland uplink"), which narrows the fault precisely. **Mechanism, confirmed in code.** The system directive hardcodes the player noun as a static string literal: `api-directive.js` ~line 66 (`User = Courier (Body)`) and throughout the directive body (~lines 119, 120, 173, 179, 181, 186, 195, 202, 216 all say "Courier"). None of these read from `GAME_DEFS[ctx].identity` — they are fixed text, so every game gets "Courier." By contrast the ambient blip banks ARE per-game data (`js/core/state.js` ~line 1202–1204 for FO3's Capital Wasteland blips; ~line 1913 for FO4's Commonwealth/Mark IV blips), which is why the ambient layer got it right and the directive got it wrong. **This is Protocol 38 in spirit, in a place the rule doesn't currently reach.** Protocol 38 mandates data-driven, game-agnostic feature code — but its written scope is feature/render/state code, not the AI directive prose. The directive is a large hand-written string that predates the multi-game abstraction. The fix is to source the player noun (and any other per-game persona term) from `GAME_DEFS[ctx].identity` and interpolate it, the same way the ambient banks are per-game data. **Severity:** immersion/correctness. Low blast radius, high visibility — a player notices being called the wrong character every turn. --- ## Finding 5 — Log export is misleading: it drops everything that wasn't a chat line **Owner's report.** The downloaded conversation log contained only the narrative text and **dropped the state and modal nodes**. This caused a real false conclusion today: a log that *appeared* to show the AI silently complying with "level me up to 15" was actually the AI correctly showing a **confirmation popup** — with the popup edited out of the export. **Mechanism, confirmed in code.** `_buildHolotapeText()` (`js/ui/ui-saves.js` ~line 88–103) builds the transcript purely by iterating `chatHistory` and emitting `[COURIER]` / `[SYSTEM]` / `[DATABANK]` lines. But an AI **modal** node is rendered to a separate modal dialog element (`api.js` ~line 426–455 writes `#modalTitle` / modal body), **not** appended to `chatHistory`. So a confirmation popup — the exact thing that makes "level me up to 15" *safe* (it asked before acting) — never enters the transcript and never appears in the export. The result is a log that misrepresents what happened on screen: the safety step is invisible, so the AI looks like it silently obeyed. **The principle.** If the app offers an export at all, it must represent **what actually happened on screen** — the narrative, the state deltas, and any modal/confirmation the AI raised. An export that silently omits the confirmation step is worse than no export, because it can be used (as it was today) to reach a false conclusion about the app's behaviour. **Fix direction.** Capture modal/confirmation events and state-delta lines into the exported transcript (or a parallel structured record), so the download is a faithful account of the session, not just its prose half. **Severity:** correctness of a diagnostic surface — it actively produced a wrong conclusion during this very audit. --- ## Finding 6 — Tab-jump on AI state change → make it in-place cards instead (OWNER DIRECTIVE) **Owner's report + directive.** When the AI changes stats or inventory, the app **switches tabs** away from the terminal to the affected panel. The owner's directive: **stay on the terminal**, and surface each change as a **popup card in place**, exactly like the existing location-change card. This keeps the player where they're reading and makes changes visible at the moment they happen instead of requiring a trip to another panel to notice them. **Mechanism, confirmed in code.** After a sync, `autoImportState()` loops over every changed category and calls `expandPanelForCategory(cat)` for each (`api-import.js` ~line 828–849). `expandPanelForCategory()` routes through `switchTab()` (per `ARCHITECTURE.md` — the auto-navigate is deliberately wired to the bezel nav), so a state change yanks the view off the terminal to the panel that changed. **A primitive already exists — this is an upgrade, not a new system (Protocol 22).** Two existing pieces are the raw material: - The `[DELTA] caps: 0→45 | ticks: 3→4 | inventory: 0→1 items` line (`api-import.js` ~line 562) already computes exactly which fields changed and by how much. - The location-change **card/toast** (`#locationCard`, the sanctioned single toast component, reused by the faction ink-stamp path) is the exact in-place surface the owner wants reused. So the work is: stop the post-sync `expandPanelForCategory` loop from stealing the tab, and instead feed the already-computed deltas into the existing card/toast surface so each change pops in place on the terminal. **Bonus the owner called out:** cards make changes visible *at the moment they happen*, rather than requiring the user to go look. **Severity:** UX / player-control (the "I keep getting thrown out of the terminal" complaint), owner-directed. --- ## Finding 7 — Ambient chatter volume: thin it out **Owner's report (screenshot-verified).** `> PIP-BOY DATA SYNCED WITH ROBCO MAINFRAME <<` prints after nearly every message, plus rotating `VAULT-TEC DIAGNOSTIC: NOMINAL` / `RADIATION SENSOR: NOMINAL` / `CAPITAL WASTELAND UPLINK: HOLDING`, plus a `[DELTA] ticks: N→N+1` line that increments whether or not anything else changed. **Mechanism, confirmed in code.** - `PIP-BOY DATA SYNCED WITH ROBCO MAINFRAME <<` fires **unconditionally after every successful sync** (`api-import.js` ~line 823). - The rotating status blips come from the per-game `blipBank` (`state.js` ~line 1202–1204). - The `[DELTA] ticks: N→N+1` line is a consequence of the directive's "1 prompt = 1 tick" rule (`api-directive.js` ~line 119) combined with the DELTA line including `ticks` in its watched keys (`api-import.js` ~line 539) — so `ticks` changes **every single turn**, which means the DELTA line prints every turn even when nothing else moved. **Fix direction.** Thin the ambient layer so it reads as texture, not noise: make the "DATA SYNCED" confirmation occasional rather than every-turn (or fold it into the sync tone), throttle the blip rotation, and stop the DELTA line from firing on a lone `ticks` increment (only surface a DELTA when something the player cares about actually changed). Ties into the global Immersion dial — the volume of this chatter should respect it. **Severity:** UX polish / signal-to-noise. Compounds Finding 6 — every turn currently produces a burst of low-value lines *and* a tab-jump. --- ## Finding 8 — The directive audit proper: does `getSystemDirective()` still claim authority over native systems? **The question.** The directive (`js/services/api-directive.js`) was written when the AI ran everything. Many of those jobs are now native (calculators, command router, trackers, level-up, map, loot import). Does the directive still instruct the AI to own systems the native path now owns? **Confirmed at least partially yes — and Finding 1 is the proof.** The directive's ~line 120 "you MUST return the ENTIRE inventory array" is the directive **claiming inventory authority** — and it is the direct upstream cause of the Finding 1 item-loss path. That single instruction is doing two harmful things at once: - **It burns tokens the owner pays for.** This is a bring-your-own-Gemini-key app; every directive token is billed to the owner. Instructing the AI on jobs it no longer solely owns is paid-for waste on every call. - **It invites the AI to return state that natives own** — which is exactly the Finding 1 mechanism. **Scope of the audit unit.** Walk the whole directive and, for each instruction, ask: *is this system still AI-owned, or is it native now?* For each native-now system, decide whether the directive should (a) stop instructing it entirely, (b) be told it is read-only / must not mutate it, or (c) keep a narrow role with explicit reconciliation on import. This is the same concern as the parked "AI → native + oversight audit" (QUEUE, Closed board — it ran once and produced the 2.8.0 native conversions; this is its directive-side follow-up) and the owner's standing player-control principle. **Severity:** cost + correctness. It is the root that Finding 1 is a symptom of, which is why the two are linked: Finding 1 is the urgent fix to land now; Finding 8 is the systematic sweep that prevents the next one. --- ## Placement recommendation (for QUEUE.md) This audit becomes **one 2.8.5 unit** — "The AI / Overseer pass" — **with one carve-out that jumps ahead of it.** **The carve-out — Finding 1 (silent item deletion) jumps to the very front of the data-safety group, ahead of A1 (live-save durability).** Reasoning, stated plainly as the owner asked: Finding 1 is confirmed **real, unrecoverable item loss** that triggers during **ordinary play** with **no external precondition** and **no confirmation** — the AI need only return a short or empty inventory array on any turn. A1 (live-save durability) is also data-safety, but it requires a **platform storage-eviction event** to trigger, and its damage is **bounded** ("everything since the last rolling backup," which does reach IndexedDB). By the project's own severity ordering — the precedent that let the save-integrity pass jump ahead of cosmetics — an unguarded, unconfirmed, every-turn data-loss path outranks an eviction-conditional, recoverable one. **Finding 1 goes first in Group 1, ahead of A1.** **The rest of the unit (Findings 2–8)** sits as a normal 2.8.5-tail unit. Suggested internal ordering by severity/coupling: - **Findings 6 + 7 together** (tab-jump → in-place cards, and ambient thinning) — they compound each other and share the post-sync surface; one coherent pass on "what the terminal does after a sync." - **Findings 2 + 3 together** (retry echo + wrong severity) — both live in the same `api.js` transient-failure path; fix as one. - **Finding 4** (persona game-agnostic) — self-contained directive-data fix. - **Finding 5** (truthful log export) — self-contained. - **Finding 8** (directive authority audit) — runs **last within the unit** as the systematic sweep, because Finding 1's fix and Finding 4 both inform what "native-now, stop instructing it" means in practice; doing the sweep after those two lands means it's calibrated against real fixes, not guesses. **Where the unit sits relative to the existing 2.8.5 tail.** Finding 1 leads Group 1 (data safety) ahead of A1. The remainder (2–8) can sit alongside the other tail work — it is largely UX-truthfulness and cost work, none of which blocks the release except insofar as the owner wants a clean AI experience before `dev → main`. It does **not** need to gate the release, but Finding 1's fix **does** ride in the data-safety group that gates it. **Every fix here carries a mandatory regression test (Protocol 13)** and, for anything touching the AI contract/import (Findings 1, 4, 8), an AI-contract safety test in the same commit (Protocol 14). Finding 1's test is the load-bearing one: a sync returning a short/empty inventory array must not delete natively-held items.