Gateway Module — `shared/`

1. Purpose

The cross-cutting utility module — a small library of pure functions consumed by almost every other gateway module. It carries zero HTTP surface, zero database access, zero state: two files exporting deterministic helpers for FTS5 query building, token estimation, reciprocal-rank fusion, RRF normalisation, recency decay, and Zod-error formatting. It is the single source of truth for the search primitives that were originally embedded in memory/ and promoted here (D23) so psychic-cache, context, vault-index, and search could all call one implementation.

2. File Inventory

3. Public API Surface

REST Endpoints

None. shared/ mounts no router and is not reachable over HTTP.

MCP Tools

None. No MCP tool maps to this module. It is a pure-utility dependency of other modules' tools, never invoked directly.

4. Internal API

All exports are pure functions / types consumed cross-module:

• estimateTokens(text): number — Math.ceil(len/4). Used by psychic-cache, context, vault-index for token accounting.
• buildFts5Query(query): string — sanitises NL into a safe FTS5 OR expression; returns '""' for empty. Used by memory, psychic-cache, search, vault-index.
• reciprocalRankFusion(vectorResults, ftsResults, k=60): Map<string,number> — merges two ranked lists without cross-space normalisation. Used by psychic-cache, context adapters, vault-index.
• normaliseRrfScores(map): Map<string,number> — scales an RRF map to [0,1] (max-divide, 1e-10 epsilon floor). Load-bearing for context's uniform scoring funnel (D23).
• recencyBoost(createdAt, now, floor=0): number — max(floor, 1/(1+log1p(ageHours/24))). Used by context's tagAndScore.
• formatZodError(error): string — used by every route module doing safeParse (dispatch, agent-memory, psychic-cache, context routes).
• Interfaces SimilarityResult/FtsSearchResult — re-exported/mirrored by memory + psychic-cache repositories.

5. Background Services

None. No timers, no queues, no daemons, no DB. Every function is synchronous, deterministic, and side-effect-free (aside from returning fresh objects — normaliseRrfScores does not mutate its input).

6. Data Contracts

No Zod schemas of its own (it formats others' Zod errors). Behavioural contracts:

• buildFts5Query — strips " ( ) * - ^, splits on whitespace, drops ≤1-char terms, caps at 20 terms, quotes each; empty → '""'.
• reciprocalRankFusion — score(id) = Σ 1/(k + rank + 1) across both lists; k=60 standard.
• normaliseRrfScores — empty→empty; all-zero→all-zero; else divide by max (epsilon-guarded).
• recencyBoost — today ≈ 1.0, ~1 week ≈ 0.5, ~30 days ≈ 0.25; never below floor.

7. Dependencies

• Gateway modules consumed: none (leaf module — it is the bottom of the dependency graph). Depended-on-by: memory, psychic-cache, context, search, vault-index, agent-memory, and any route module using formatZodError.
• External libraries: zod (type-only import of ZodError). No runtime external deps.
• Environment variables: none.

8. Test Coverage

search-utils is directly and heavily tested (35 tests); zod-error has no dedicated suite but is covered transitively by every route module's Zod-negative tests.

9. Known Limitations

• estimateTokens is a crude heuristic — len/4 approximates OpenAI-family tokenisation only; non-English / code-heavy content mis-estimates, affecting every downstream token budget.
• buildFts5Query drops single characters — 1-char terms (e.g. a meaningful initial or CJK glyph) are filtered out; recall loss on short tokens.
• No dedicated zod-error test — regressions in error formatting would only surface via consumer suites.
• Interface duplication — SimilarityResult/FtsSearchResult are declared both here and in memory/types.ts; structurally identical but two declarations.

10. Change History