LISAOS // DOCS
GATEWAY // SEARCH

Gateway Module — `search/`

1. Purpose

The unified full-text search surface (D265). A single GET endpoint that runs the same FTS5 query against all three keyword indexes — psychic_fts, memory_fts, agent_memory_fts — in parallel via SQL, merges the results by BM25 rank, and returns a flat unified result set with a human-readable preview. It backs the session_search MCP tool used for cross-session recall. Unlike context/ (semantic RAG fanout with weighting), search/ is pure keyword BM25 with no embeddings, no scoring adaptation, and no vector search — a fast literal lookup.

2. File Inventory

FileLinesResponsibility
routes.ts272createSearchRouter(db): single GET / endpoint. q validation, limit guard (AK-362: finite/integer/positive, default 10, clamp 50), optional namespace filter (psychic only), three per-store FTS5 queries + joins, makePreview JSON-flattening helper, BM25-rank merge
Total272(single-file module — no types.ts, index.ts, or repository.ts)

3. Public API Surface

REST Endpoints

Mounted at /api/search. Auth per AK-415: GET → Bearer.

MethodPathAuthQueryResponseSide Effects
GET/Bearer?q (required, non-empty), ?limit (opt, default 10, max 50, positive-int-guarded), ?namespace (opt — psychic_cache only){data: SearchResult[], total, query} / 400 (bad q or limit) / 500none (read-only)

SearchResult = {id, source:'psychic_cache'|'memory'|'agent_memory', content, content_preview, context_type?, mission_id?, agent_name?, created_at, bm25_rank}.

MCP Tools

Tool NameRaw payload?EndpointNotes
session_searchNoGET /api/search?q=…&limit=…&namespace=…Cross-session/store keyword recall; q required

No raw twin — session_search is a read-only GET (staleness-immune bug class applies to write-path tools only).

4. Internal API

None exported for cross-module use. createSearchRouter(db) is the sole export, imported by server/index.ts. makePreview and the row-shape interfaces are module-private. SearchResult here is a distinct type from context/types SearchResult (different fields) — same name, different module namespace.

5. Background Services

None. Stateless request/response. No timers, queues, or daemons. Reads the three FTS5 virtual tables owned by memory/db.ts; writes nothing.

6. Data Contracts

No Zod schemas — validation is imperative:

  • q — required, .trim().length > 0, else 400.
  • limitundefined→10; else Number(...) must be finite, integer, >0, clamped to 50; else 400 (AK-362/D1452).
  • namespace — optional string; when present, filters the psychic_cache subquery by mission_id only (memory + agent_memory have no mission column).
  • FTS query built via shared/search-utils.buildFts5Query (word→quoted-phrase OR expression); empty query short-circuits to {data:[], total:0}.

Reads (no writes): psychic_ftspsychic_cache, memory_ftsmemory_documents, agent_memory_ftsagent_memory (joins on document_id/rowid).

7. Dependencies

  • Gateway modules consumed: shared/search-utils (buildFts5Query). Reads tables owned by memory (memory_fts/documents), psychic-cache (psychic_fts/cache), agent-memory (agent_memory_fts/agent_memory).
  • External libraries: better-sqlite3, express.
  • Environment variables: none.

8. Test Coverage

LayerFileTests
Routes (3-store merge, limit guard, namespace filter, preview, empty query)test/search-routes.test.ts39
Shared query builder (indirect)test/search-utils.test.ts35

Well-covered for a single-endpoint module; search-utils coverage is shared with psychic-cache/context/vault-index.

9. Known Limitations

  • Per-store LIMIT then merge-then-trim — each subquery pulls up to limit rows, they are concatenated (up to 3×limit), sorted by bm25_rank, then sliced to limit. BM25 ranks are not comparable across FTS5 tables (different corpora/tokenisers), so the cross-store merge ordering is approximate, not a true global ranking.
  • No paginationoffset is unsupported; only top-limit is retrievable.
  • namespace filters psychic only — memory + agent_memory results are always unfiltered even when a namespace is supplied (documented in-code).
  • Duplicate type nameSearchResult collides by name with context/types SearchResult; a reader grepping the symbol must disambiguate by module.

10. Change History

DateDispatchSummary
2026-07-042295Initial per-module spec filed (audit campaign Phase 4)
(historical)D265Module created — unified FTS5 search across 3 stores; session_search MCP wrapper
(historical)D1452 (AK-362)limit parameter hardening (finite/integer/positive guard)
(historical)D1061 (AK-341)makePreview JSON-parse graceful-fallback semgrep deferral

On this page