Prompt Interrogation Process

Status: Triggered eval tool (runs on prompt-surface change, NOT a calendar) Location: scripts/interrogate/ Cost: 1 Gemini call per agent (~$0.01 each) Output: persisted to data/interrogate/{date}/ — do NOT rely on stdout transcription

Cadence is event-driven, not periodic. The old “quarterly review” trigger never fired (v5 was 2026-03-05; the next quarterly never happened, and the 2026-04-01 consolidation merged 3 news agents → 1 News Analyst without an audit). Re-interrogation is now a hard gate on the prompt-regime bump — see the trigger table below and the “Prompt Regime Bump Rule” in CLAUDE.md.

What It Is

Prompt interrogation is how Ghost evaluates its own signal density. Instead of running a full analysis and guessing what the LLM used, we ask the LLM directly by sending it all normal inputs but replacing the output format with meta-questions about its own synthesis process.

The LLM receives exactly the same context it would in production — real KB data, real market data, real prompt instructions — but instead of producing a thesis, it classifies each input as ESSENTIAL, USEFUL, PASS-THROUGH, or NOISE.

Why It Matters

Prompt bloat is invisible. A section that was essential when written may be noise 3 months later because:

• Other sections now cover the same signal
• The output format changed and the field is no longer consumed
• The data source became unreliable and the section is always caveated away

The interrogation process makes this visible. When a director says “I receive dealer dynamics but classify it as NOISE — too speculative for my role,” that’s a concrete, evidence-based signal to cut or gate that section.

Key principle: The LLM that runs the prompt is the best judge of what it actually uses from that prompt.

The Classification Framework

Every input section and output field is classified into one of four categories:

How To Run

Prerequisites

• Recent ghost run TICKER so KB has fresh data
• Gemini API credentials configured (same as normal Ghost runs)

Commands

# Run individual director interrogation
uv run python3 scripts/interrogate/interrogate_tech_director.py
uv run python3 scripts/interrogate/interrogate_news_director.py
uv run python3 scripts/interrogate/interrogate_mkt_structure.py
uv run python3 scripts/interrogate/interrogate_strat.py

# Run all four in parallel (recommended)
uv run python3 scripts/interrogate/interrogate_tech_director.py &
uv run python3 scripts/interrogate/interrogate_news_director.py &
uv run python3 scripts/interrogate/interrogate_mkt_structure.py &
uv run python3 scripts/interrogate/interrogate_strat.py &
wait

Each script:

1. Loads the director’s real Jinja2 template
2. Populates it with real KB/market data (same as a ghost run)
3. Replaces the output format section with interrogation questions
4. Sends to Gemini (1 call, ~$0.01)
5. Prints the LLM’s self-assessment

Configuring the Ticker

Each script has TICKER = "NVDA" at the top. Change this to interrogate with a different ticker’s data.

Interrogation Structure

Each script asks 4-5 rounds of questions. The rounds follow a consistent pattern across all four directors:

Round 1: Input Triage

“For each input section, classify as ESSENTIAL / USEFUL / PASS-THROUGH / NOISE.”

This identifies what the LLM actually references when synthesizing its output.

Round 2: Deep Dive

Specific questions about the highest-priority inputs. Example (Technical Director): “How many Fib confluence zones did you USE vs IGNORE? When Fib overlaps with MA, does Fib add anything?”

Round 3: Output Attribution

“Trace each output field back to its source inputs.” This reveals pass-through fields and genuine synthesis.

Round 4: Signal Compression

“What is the MINIMUM set of inputs for 90% of the same output? What’s redundant? What’s missing?”

Round 5: Minimum Viable Output / Redesign

“If you could redesign your input, what would it look like? What output fields could be cut?”

How to Interpret Results

High-Signal Findings

• Input classified NOISE by the director that receives it — strong signal to cut or gate
• Input classified PASS-THROUGH — candidate for pre-synthesis (compute upstream, send just the result)
• Duplicate content across sections — merge into one location
• “90% of output from N inputs” — the inputs NOT in that list are compression candidates

Low-Signal Findings

• “USEFUL” — vague, doesn’t tell you what to do. Ask follow-up in Round 2.
• “I would add X” — feature requests from the LLM. Evaluate skeptically.

Cross-Director Patterns

When multiple directors classify the same data:

• All say ESSENTIAL — critical path, don’t touch
• Mixed (ESSENTIAL + NOISE) — routing problem. Maybe only one director should get it.
• All say NOISE — cut it from the pipeline entirely

When to Run Interrogation

Run interrogation after any of these changes:

No standing calendar trigger. A periodic review that never happens is worse than no policy — it reads as coverage that doesn’t exist. Interrogation fires on the events above, full stop.

Enforcement (mechanical — not a memory item)

The trigger above is enforced by code, because the 2026-04-01 miss proved that a “remember to run it” rule fails silently. ghost/signals/interrogation_currency.py records the SHA of each interrogated template at the moment it was last interrogated (in scripts/interrogate/interrogation_manifest.json). Running an interrogate script refreshes that agent’s SHA — so running IS what marks it current.

• tests/test_interrogation_currency.py fails if any template drifts from its recorded SHA (per-agent + aggregate).
• .github/workflows/deploy.yml runs scripts/ci/check_interrogation_currency.py as its FIRST step (stdlib-only, no deps — can’t flake on install). Drift → the workflow fails → no build, no deploy. The push to production is blocked until the changed prompt is re-interrogated.
• Cosmetic edits (typo/comment/whitespace, no signal-surface change) clear the gate without an LLM run, audited: uv run python -m ghost.signals.interrogation_currency --ack <agent> --reason '...' (the reason is recorded in the manifest).

Same primitive as has_thesis_changed(): hash-as-trigger. The chain that was broken at the interrogation node since 2026-04-01 is now self-healing — you physically cannot ship a prompt-surface change to the morning run without either re-interrogating it or explicitly acking it as cosmetic.

Script Architecture

Each script follows the same pattern:

# 1. Load real data (KB + market tools)
fib = get_kb_entry(TICKER, "fib_analysis")
# ...

# 2. Render the real Jinja2 prompt
prompt = load_prompt("director/technical.jinja2", ticker=TICKER, ...)

# 3. Find the output format section and replace it
marker = "## Output Format"
context_only = prompt[:prompt.find(marker)]
full_prompt = context_only + INTERROGATION

# 4. Send to Gemini (1 call)
response = client.models.generate_content(model=..., contents=full_prompt)

The key insight: the LLM sees exactly the same context it would in production. The only difference is the final instruction — “classify your inputs” instead of “produce a thesis.”

Files

scripts/interrogate/
  interrogate_tech_director.py    # Technical Director (Fib/MA/VWAP/Momentum inputs)
  interrogate_news_analyst.py     # News Analyst (merged market+company agent) — persists to data/interrogate/{date}/
  interrogate_strat.py            # Market Aggregator/Strategist (signal density)
  interrogate_strat_compact.py    # Market Aggregator/Strategist (output compactness)

Removed 2026-06-04: interrogate_news_director.py (News Director → merged News Analyst; superseded by interrogate_news_analyst.py) and interrogate_mkt_structure.py (Market Structure Director → mechanical rules, no LLM, nothing to interrogate).

Related Documentation

• ONTOLOGY.md — Signal density classification is part of the conceptual framework
• EPISTEMOLOGY.md — Interrogation is a verification mechanism (how Ghost validates its own knowledge formation)
• CONDITION_ACCURACY.md — Feedback loop measures prediction outcomes; interrogation measures prompt signal quality
• PRAXIS.md — Interrogation bridges the gap between prompt design (theory) and LLM behavior (practice)

This is the evergreen process. Specific runs are published as dated transcripts — e.g. the 2026-06-04 News Analyst run. The findings and numbers live in the essays: formalizing (II): interrogation and formalizing (III): the gate.