Raw Twin Discipline — MCP ListTools Schema Cache Staleness Protocol

Binding for every write-path / mutation tool exposed by the lisa-memory MCP server (and any future MCP server that exposes evolving Zod schemas). Authored Dispatch 42 alongside the konnichiwagwan Phase F freshness probe and the /api/tools/list gateway advertisement endpoint.

1. The bug class

Claude Code's MCP client calls ListTools exactly once at session startup and caches every tool's inputSchema for the lifetime of the session. Any post-startup schema evolution on the server — new Zod union branches, new structured fields, new context_type variants — is invisible to the running session. New fields get silently stripped before the call leaves the process, and the gateway Zod then rejects with <field>: expected string, received undefined (or worse, the wrong branch validates and data lands in the wrong row).

This failure mode burned five consecutive dispatches (D20 → D25). It lost D22's output cache entry outright. It forced REST-via-curl fallback for every structured cache write for six weeks. D26 established the structural fix — the *_raw twin — as a hard architectural rule. D31 extended the pattern to the dispatch-lifecycle mutation family after D26 closure CONFIRMED the same bug on report_complete (domain_affinity: expected boolean, received string). D42 adds a runtime drift detector (/api/tools/list + konnichiwagwan Phase F) so stale sessions warn instead of silent-fail.

Prior art:

feedback_mcp_schema_cache_staleness.md (LISA agent memory) — the canonical one-liner
CS.AK.LISA.Intel.McpSchemaCacheFix.2026-04-08.md — full D26 forensic
CS.AK.LISA.Intel.ReportFamilyRawPayload.2026-04-08.md — D31 sweep

2. The `*_raw` twin pattern

Shape. Every affected MCP tool has a sibling tool with the suffix _raw whose inputSchema is exactly:

{
  "type": "object",
  "properties": {
    "payload": {
      "type": "string",
      "description": "JSON-serialised body — validated server-side"
    }
  },
  "required": ["payload"]
}

Semantics. The client passes the full arguments object as a single JSON-encoded string in payload. The MCP tool handler does JSON.parse(payload) and forwards the result verbatim to the REST endpoint via fetch. No destructure, no per-field stripping, no schema enforcement in the MCP layer. All validation happens server-side via the existing Zod schema at the REST route.

Why it works. A one-field payload: string schema never evolves. New fields, new branches, new enum variants, new optional flats — none of them change the inputSchema the MCP client sees at ListTools. The cache can be as stale as it likes; the transport shape is forever-stable. Schema evolution happens exclusively on the server, and every running session immediately picks up the new behaviour on the next call without restart.

Canonical example: write_psychic_cache_raw in artefacts/code/lisa/memory_gateway/server/mcp/index.ts. Five sibling twins were added D31: context_feedback_raw, report_dispatch_raw, report_progress_raw, report_complete_raw, commit_agent_memory_raw.

3. When a new `*_raw` twin is required

A new MCP tool MUST ship with a *_raw twin from day one if it meets all three criteria:

Mutation — the tool changes server state (writes, updates, deletes).
Structured OR evolving schema — the input shape contains any of: discriminated unions, non-string primitives (booleans, numbers, arrays, nested objects), enum fields, or is expected to grow new fields over time.
Server-side Zod enforcement — the REST endpoint that backs the MCP tool performs full Zod safeParse validation (not manual destructure).

If criterion 3 fails, DO NOT convert to raw — doing so moves the client-visible surface off the flat schema without strengthening server validation, which is a net regression. Instead, tech-debt-flag the REST endpoint and harden it with Zod first. Criterion 3 is what makes the raw transport safe: the server can trust the parsed payload because it owns the source-of-truth schema.

Read-only tools are out of scope. assemble_context, query-side fanout, list endpoints — their drift impact is graceful degradation (optional filter dropped), not data loss. These continue to use flat permissive schemas.

URL-path-only tools are out of scope. clear_psychic_cache takes its arguments via URL path + query string with no JSON body to parse. The raw pattern doesn't fit.

4. How to add a new `*_raw` twin

Step-by-step, against write_psychic_cache_raw as the reference implementation:

4.1 Confirm the REST endpoint has Zod validation

Open the route file (e.g. psychic-cache/routes.ts). Confirm the handler does SomeInputSchema.safeParse(req.body) and returns a Zod error on failure. If it destructures manually, stop and harden the REST endpoint first.

4.2 Add the tool registration to `server/mcp/index.ts`

In the ListTools response, add a new tool entry with the shape:

{
  name: 'mytool_raw',
  description: '<short description>. CANONICAL path (Dispatch N). See CLAUDE.md Cache Write Schema for the payload contract.',
  inputSchema: {
    type: 'object',
    properties: {
      payload: {
        type: 'string',
        description: 'JSON-serialised body — validated server-side'
      }
    },
    required: ['payload']
  }
}

Description should be long and explanatory. The tool description IS the agent-facing schema documentation — it tells the calling agent what shape to put in the payload string. Include the required fields, optional fields, per-branch contracts if applicable, and a minimal example payload.

4.3 Add the `CallTool` handler

In the CallTool switch, add:

case 'mytool_raw': {
  const { payload } = args as { payload: string };
  const body = JSON.parse(payload);
  const response = await fetch(`${GATEWAY_URL}/api/mymodule/mytool`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify(body),
  });
  return formatGatewayResponse(response);
}

Forbidden: destructuring specific fields out of the parsed body in the MCP layer, then re-assembling. This reintroduces the bug class. The full parsed object is forwarded verbatim — the server owns all field handling.

4.4 Bump `SCHEMA_VERSION` in `server/tools/schema-introspection.ts`

Add the new tool to the buildToolsList() output with its branches. Bump SCHEMA_VERSION to today's ISO date. This fires the konnichiwagwan Phase F drift warning on every stale session after the gateway restart — operators see the warning at their next konnichiwagwan invocation and restart Claude Code at a natural boundary.

4.5 Restart the gateway AND Claude Code

The MCP client caches tool lists at session startup. A gateway restart alone is not enough — Claude Code must also be restarted to pick up the new tool. Phase F will warn on the next session start of any session that was running during the deploy.

4.6 Add governance references

Add the new tool name to the table in §2 of this doc if it's a canonical replacement for a legacy tool
If the legacy tool is being deprecated, see §6 Migration workflow

5. Permissive-flat MCP layer rule

Per-field stripping in the MCP layer is forbidden. When forwarding a call from an MCP tool handler to the REST endpoint, always pass the full parsed payload via JSON.stringify(body). Never do:

// WRONG — silent-strip footgun
const { foo, bar } = args;
await fetch(url, { body: JSON.stringify({ foo, bar }) });

// RIGHT — forward everything
await fetch(url, { body: JSON.stringify(args) });

The MCP client layer is permissive-flat: it exposes every structured field for every context_type as top-level optional keys at the root of the tool's inputSchema. This is intentional — JSON Schema Draft 7 does not render Zod discriminated unions cleanly across MCP clients, so the MCP face is the lowest-common-denominator union of all branches. Per-branch validation, required-field enforcement, and .strict() rejection of unknown fields happen exclusively server-side. The MCP handler's only job is transport.

The single exception is URL path/query parameters — if the REST endpoint requires dispatch_id in the URL (e.g. /dispatch/:id/progress), the handler may destructure to read that ONE field for path construction, but must still forward the full body unchanged. See report_progress for the pattern.

When the gateway Zod gains a new field: mirror it as an optional flat key in the MCP inputSchema (for the legacy permissive-flat tool, not the raw twin) AND bump SCHEMA_VERSION in tools/schema-introspection.ts. Then restart gateway + Claude Code. The raw twin needs no change — it was always going to ship the new field in the payload string.

6. Migration workflow — deprecating a legacy tool in favour of its raw twin

Legacy tools cannot be deleted outright because in-flight agent dispatches may be holding references to them in system prompts or skill definitions. The migration is two-phase:

Phase 1 — Coexistence

Raw twin added per §4
Legacy tool retained, unchanged
CLAUDE.md updated to document the raw tool as canonical
Agent prompts updated over time to prefer the raw tool (drip migration)
Legacy tool description in MCP ListTools annotated with DEPRECATED — use {raw_tool} for new work but kept functional

Phase 2 — Retirement (only after consumption telemetry shows near-zero legacy usage)

Legacy tool removed from ListTools response
REST endpoint retained (raw twin still hits it)
Any in-flight agents that reference the legacy tool get a Tool not found error, which is preferable to a silent field-strip

Never break in-flight dispatches. The legacy tool must remain functional for at least one full dispatch cycle after the raw twin ships. The migration gate is telemetry-driven: check the MCP activity log for calls to the legacy tool name over the past N sessions. Zero calls → safe to retire.

Telemetry query (rough shape — implement against activity_log):

SELECT event_type, COUNT(*)
FROM activity_log
WHERE event_type LIKE 'mcp_call:%'
  AND created_at > datetime('now', '-7 days')
GROUP BY event_type
ORDER BY COUNT(*) DESC;

7. Runtime drift detection (Dispatch 42)

Two components added D42 to give operators a loud signal when a session's cached schema has drifted from the live gateway state.

7.1 `/api/tools/list` gateway endpoint

GET ${GATEWAY_URL}/api/tools/list returns the live gateway-side schema for every write-path / mutation tool. Source of truth lives in artefacts/code/lisa/memory_gateway/server/tools/schema-introspection.ts. Response shape:

{
  "schema_version": "2026-04-09",
  "generated_at": "2026-04-09T21:45:12.334Z",
  "tools": [
    {
      "name": "write_psychic_cache",
      "raw_tool": "write_psychic_cache_raw",
      "category": "write_path",
      "description": "...",
      "base_required": ["mission_id", "content", "context_type"],
      "base_optional": ["agent_name", "ttl_hours", "tags", "source_turn", "dispatch_id"],
      "branches": [
        { "context_type": "decision", "required": ["decision_rationale"], "optional": ["decision_alternatives"] },
        ...
      ]
    },
    ...
  ]
}

Any schema evolution (new field, new branch, new tool) MUST bump SCHEMA_VERSION in schema-introspection.ts to today's ISO date.

7.2 konnichiwagwan Phase F freshness probe

~/.claude/skills/konnichiwagwan/scripts/check_tool_freshness.sh compares the live gateway schema_version against the session's recorded baseline at ~/.claude/konnichiwagwan/session_schema.json (written by Phase A on first probe). On drift, Phase F surfaces a loud warning in the Completion Signal:

⚠️ Tool schema drift detected — session started 2026-04-09, gateway advertises 2026-04-10. Restart Claude Code to avoid silent field-strip bugs.

The operator then quits and relaunches Claude Code at a natural boundary. Full probe states and signal mappings are documented in ~/.claude/skills/konnichiwagwan/SKILL.md Phase F.

8. Invariants

Every new write-path/mutation MCP tool with structured input ships a *_raw twin from day one. No exceptions. Raiden enforces this at code review.
SCHEMA_VERSION in tools/schema-introspection.ts bumps on every schema edit. Omitting the bump silences Phase F drift detection and is a governance violation.
MCP handlers never destructure for per-field stripping. Always forward the full parsed payload via JSON.stringify(body). URL path/query extraction is the sole exception.
Legacy tools are deprecated not deleted. At least one full dispatch cycle of coexistence before retirement.
Raw twin documentation lives in the tool description string. The MCP ListTools description IS the agent-facing contract — it must include required fields, optional fields, per-branch contracts, and a minimal example payload.

9. Cross-references

CLAUDE.md → "Canonical write path — write_psychic_cache_raw" and "Cache Write Schema" sections
CS.AK.LISA.Docu.LisaOSMap.md → framework/governance matrix (raw twin discipline is a binding governance doc)
CS.AK.LISA.Docu.CodeDisciplineProtocol.md → any new MCP tool must pass this doc's Four-Gate review AND the raw twin criteria above
CS.AK.LISA.TechSpec.CleanCodePipeline → the operational pipeline that mechanically enforces this discipline. Sprint 0 ships a custom Semgrep rule cipher-shinobi.raw-twin-required (authored by Gray Fox, see TechSpec §06 Gate 1 + §09) that flags any new MCP tool registration in server/mcp/index.ts matching the mutation + structured-input + Zod-validated criteria in §3 above WITHOUT a sibling *_raw twin in the same registration block. This is the first concrete custom rule derived from a LISA governance doc — its cite_version SHA256 header pins the §3 criteria text, and Gate 3 CI re-hashes on every PR to fail-fast on doc drift (TechSpec OQ-3 resolution).
artefacts/code/lisa/memory_gateway/server/mcp/index.ts → implementation, with D26 + D31 commentary blocks
artefacts/code/lisa/memory_gateway/server/tools/schema-introspection.ts → SCHEMA_VERSION + advertised tool list
~/.claude/skills/konnichiwagwan/SKILL.md → Phase F runtime probe
feedback_mcp_schema_cache_staleness.md → LISA agent memory one-liner

10. History

D20-D25 — Bug class burns five consecutive dispatches; D22's output cache entry lost; REST fallback becomes the default
D26 — Raw twin pattern established; write_psychic_cache_raw shipped as the canonical reference implementation
D31 — Sweep of the dispatch-lifecycle mutation family: context_feedback_raw, report_dispatch_raw, report_progress_raw, report_complete_raw, commit_agent_memory_raw
D39 Bug 2 — Identified residual gap: no runtime drift detection; stale sessions still fail silently when a tool is added mid-session
D42 — This doc. /api/tools/list endpoint + konnichiwagwan Phase F freshness probe + governance consolidation

Raw Twin Discipline — MCP ListTools Schema Cache Staleness Protocol

On this page