pam ▸ CODEX_HANDOFF_STRATEGY_FIX.md
updated 2026-03-28

Codex Handoff: Pam Research Loop Strategy Fix

Status Update (as of 2026-03-28)

This file originally captured the March 17 failure state. That diagnosis was useful, but it is now partially obsolete. The strategy surface has since been improved and the handoff for the next iteration should start from the current reality on main.

What is already fixed on main

Current persisted params

research/best_strategy_params.json currently represents the best known replay baseline:

{
  "contrarian_enabled": false,
  "economics_band_enabled": false,
  "minimum_edge_bps": 100.0,
  "politics_eventful_reversal_category": "Politics",
  "politics_eventful_reversal_enabled": true,
  "politics_eventful_reversal_max_live_price": 0.2,
  "politics_eventful_reversal_min_abs_move_bps": 150.0,
  "politics_eventful_reversal_min_tradeability": 0.9,
  "politics_midprice_enabled": false,
  "process_watch_enabled": false,
  "quantitative_eventful_enabled": true,
  "quantitative_eventful_max_live_price": 0.3,
  "quantitative_eventful_min_abs_move_bps": 100.0,
  "quantitative_eventful_min_eventful_score": 0.5,
  "quantitative_eventful_min_tradeability": 0.8,
  "quantitative_eventful_mode": "reversal"
}

Remaining issue

The replay strategy is now good enough to study, but the live operator path still does not produce deployable trades.

Current live situation after the latest fixes: - python3 main.py opportunity-scan --limit 10 --json now returns unique tickers, but the scan still surfaces mostly watch rows. - python3 main.py trade-candidates --limit 10 --json now shows review candidates instead of being suppressed by stale global gate data. - There are still 0 deployable trades. - The current global blocker is mostly stable_cycles<3. - The leading local blocker on review candidates is live_edge_bps<100.0. - The live heuristic still under-produces directional/actionable calls relative to what replay can support.

Next-phase task for Codex

  1. Read this file fully, but treat the March 17 diagnosis below as historical context, not the current state.
  2. Investigate the live heuristic path in the scanner/signal layer. - Key file: strategy/kalshi_opportunity_scanner.py - Understand why opportunity-scan still yields almost all watch rows even after the replay-side quantitative sleeve started working.
  3. Fix the live heuristic so the signal pipeline produces more directional/actionable calls. - Goal: materially reduce the share of neutral/watch scanner output. - Goal: make trade-candidates surface at least some realistic paths toward deployable_trade, not just review_candidate.
  4. If the heuristic path is fundamentally too weak, add a quantitative-only live path that can bypass heuristic_direction, analogous to how the replay-side quantitative sleeve bypasses it.
  5. Re-run: - python3 main.py opportunity-scan --limit 20 - python3 main.py trade-candidates --limit 20 --json
  6. Verify whether any names become deployable. If not, document the exact remaining blocker mix and whether the bottleneck is: - live heuristic direction - live edge estimation - stable-cycles gate - sleeve matching / semantic mismatch

Key files for the next iteration

Branch note

The earlier handoff referenced codex/pam-research-loop, but that work has already been merged. The current source of truth is main. If you need an isolated branch for the next pass, branch from main.

Context

Pam’s research loop is stuck. The confidence gate correctly identifies that the current strategy doesn’t work. This handoff covers the diagnosis and three recommended fixes, in priority order.

Current State (as of 2026-03-17)

Root Cause Analysis

Problem 1: Signal starvation

16 of 18 live candidates have heuristic_direction=neutral. Every strategy sleeve in research/strategy.py gates on heuristic_direction before producing a buy signal. When 89% of inputs are neutral, the strategy can’t act.

Evidence: latest_state.json → candidate_review.summary.top_invalidation_reasons shows heuristic_direction=neutral as #1 invalidation reason (16 candidates).

Problem 2: Too few active sleeves

research/best_strategy_params.json has disabled 3 of 6 sleeves: - contrarian_enabled: false - process_watch_enabled: false - politics_midprice_enabled: false

Only politics_eventful_reversal is firing (with threshold_review and economics_band enabled but rarely matching). This guarantees failure on unique_sleeves >= 2 and top_sleeve_share <= 0.85.

Problem 3: The active sleeve loses money

The 5 trades from politics_eventful_reversal averaged -40 bps realized. 40% hit rate. The contrarian logic (buying opposite to heuristic_direction on cheap politics markets after eventful moves) isn’t generating edge.

Fix 1: Add a quantitative sleeve that doesn’t require heuristic_direction

File: research/strategy.py Location: Inside the decide() function, add a new sleeve after the existing ones.

Concept: A sleeve that trades on eventful_score + price movement alone, without needing a directional signal. For example: - If eventful_score > 0.5 AND abs_move_bps > 150 AND live_price < 0.30 - Buy in the direction of the move (momentum) or against it (mean reversion) - Test both approaches in replay

Why this helps: - Bypasses the signal starvation problem entirely - Adds a second unique sleeve (fixes diversity gates) - Increases action count (fixes action_count gate) - Uses data that’s already available in the observation dict

Implementation notes: - The decide() function receives an obs dict. Check obs.get('eventful_score') availability in replay data — latest_state.json shows 50.8% of replay rows have eventful content. - Add the sleeve name to the return tuple: return ("buy_yes", "quantitative_momentum") or similar - Add corresponding params to DEFAULT_PARAMS dict at top of file

Fix 2: Re-enable disabled sleeves with relaxed parameters

File: research/best_strategy_params.json

The disabled sleeves were turned off based on ~5 total trades — far too small a sample for statistical significance. Re-enable them with slightly wider parameters:

{
  "contrarian_enabled": true,
  "contrarian_max_live_price": 0.05,
  "process_watch_enabled": true,
  "politics_midprice_enabled": true,
  "politics_midprice_min_tradeability": 0.85,
  "politics_eventful_reversal_enabled": true,
  "politics_eventful_reversal_max_live_price": 0.20
}

Why this helps: - More sleeves firing = more actions = better statistical sample - Diversity gates (unique_sleeves, top_sleeve_share) start passing - The search loop can then properly evaluate which sleeves work

Fix 3: Investigate signal pipeline

Files: Look upstream of strategy.py for where heuristic_direction gets set. - Check research/candidate_review.py — the _derive_heuristic_direction() or similar method - Check research/prepare_replay.py — how replay rows get their heuristic_direction field - Check core/signals.py or core/processing.py for the original signal generation

Question to answer: Why does 89% of the pipeline produce neutral? Is the heuristic too conservative? Is it missing input data? Is there a bug where a field isn’t being populated in replay rows?

This is the highest-impact fix long-term but may require more investigation. Fixes 1 and 2 are faster wins.

Success Criteria

After implementing these fixes, run the research loop:

cd /Users/joshua/Projects/Unicorn.Land/pam
source .venv/bin/activate
python3 main.py research-loop --until-confident

Target state: - Action count >= 8 - Unique sleeves >= 2 - Hit rate trending toward 55%+ - Score trending positive - Strategy should be firing on at least 2 different sleeve types

Key Files

File Purpose
research/strategy.py The strategy surface — all trading logic lives here
research/best_strategy_params.json Current tuned parameters
research/continuous_loop.py The automated research lifecycle
research/candidate_review.py Live market classification
research/latest_state.json Current research state snapshot
research/run_replay.py Replay evaluation harness
research/prepare_replay.py Builds replay pairs from DB
research/search.py Parameter grid search

Branch

Current work is on codex/pam-research-loop. There are 15 modified files not yet committed — commit those first before making new changes.

Notes

Inspired by Karpathy’s autoresearch repo. The core idea is dead simple: an AI agent modifies code, runs an experiment on a fixed time budget, checks a metric, keeps improvements and discards regressions, then repeats. You program the researcher, not the research.

This is a powerful reframe for Pam’s research loop, which currently has too many stages and gates between “try something” and “learn whether it worked.”

How to apply to Pam’s research loop

Map Pam onto the AutoResearch pattern with these choices:

AutoResearch concept Pam mapping
Single metric composite_score = hit_rate * avg_realized_bps (or the existing confidence score — pick one number)
Single file to modify research/best_strategy_params.json (the params that strategy.py reads)
Fixed evaluation budget One full replay pass via run_replay.py (~30 seconds)
Keep-or-discard Only persist params to best_strategy_params.json when score improves over baseline
Log everything Append every trial (params, score, timestamp) to research/results.tsv

The current continuous_loop.py pipeline is: backfill → prepare → replay → search → candidate review → confidence check. That’s six stages with multiple failure modes. AutoResearch says: just try things and measure. The confidence gates can be a graduation step AFTER you have a strategy that actually scores positive.

Concrete implementation: research/autoresearch.py

Create a single script that implements the loop:

#!/usr/bin/env python3
"""AutoResearch loop for Pam strategy params.

Pattern: mutate params → replay → compare to baseline → keep or discard → repeat.
Ref: github.com/karpathy/autoresearch
"""
import json, time, copy, random, csv, os
from pathlib import Path
from run_replay import run_replay  # existing replay harness

PARAMS_FILE = Path("research/best_strategy_params.json")
RESULTS_FILE = Path("research/results.tsv")

def load_params():
    return json.loads(PARAMS_FILE.read_text())

def save_params(params):
    PARAMS_FILE.write_text(json.dumps(params, indent=2) + "\n")

def score(replay_result):
    """Single number: hit_rate * avg_realized_bps."""
    hr = replay_result.get("hit_rate", 0)
    bps = replay_result.get("avg_realized_bps", 0)
    return hr * bps

def mutate(params):
    """Tweak one numeric param by ±10-30%."""
    p = copy.deepcopy(params)
    numeric_keys = [k for k, v in p.items() if isinstance(v, (int, float)) and not isinstance(v, bool)]
    if not numeric_keys:
        return p
    key = random.choice(numeric_keys)
    factor = random.uniform(0.7, 1.3)
    p[key] = round(p[key] * factor, 6) if isinstance(p[key], float) else max(1, int(p[key] * factor))
    return p

def log_trial(trial_num, params, result_score, kept):
    """Append to results.tsv."""
    file_exists = RESULTS_FILE.exists()
    with open(RESULTS_FILE, "a", newline="") as f:
        w = csv.writer(f, delimiter="\t")
        if not file_exists:
            w.writerow(["trial", "timestamp", "score", "kept", "params"])
        w.writerow([trial_num, time.strftime("%Y-%m-%dT%H:%M:%S"), result_score, kept, json.dumps(params)])

def main():
    baseline_params = load_params()
    baseline_result = run_replay(baseline_params)
    baseline_score = score(baseline_result)
    print(f"Baseline score: {baseline_score:.4f}")

    trial = 0
    while True:
        trial += 1
        candidate_params = mutate(baseline_params)
        candidate_result = run_replay(candidate_params)
        candidate_score = score(candidate_result)

        kept = candidate_score > baseline_score
        log_trial(trial, candidate_params, candidate_score, kept)

        if kept:
            print(f"  Trial {trial}: {candidate_score:.4f} > {baseline_score:.4f} — KEEPING")
            save_params(candidate_params)
            baseline_params = candidate_params
            baseline_score = candidate_score
        else:
            print(f"  Trial {trial}: {candidate_score:.4f} <= {baseline_score:.4f} — discarding")

if __name__ == "__main__":
    main()

This is a starting point — the run_replay import and score function will need to match Pam’s actual replay API. The key point is the structure: the entire loop is ~60 lines with zero ambiguity about what “better” means.

Key difference from the current approach

Current continuous_loop.py AutoResearch pattern
6 stages (backfill, prepare, replay, search, candidate review, confidence) 1 stage: mutate → evaluate → keep/discard
Multiple failure modes and intermediate state Single failure mode: replay crashes (fix and retry)
Confidence gate blocks progress early Confidence gate becomes a graduation check after positive score
Grid search explores params systematically Random mutation explores params stochastically (simpler, surprisingly effective)
Heavy orchestration in continuous_loop.py Self-contained script, runs until Ctrl+C

The current approach isn’t wrong — it’s just front-loading quality gates before the strategy has had enough iterations to improve. AutoResearch inverts this: iterate fast with a single metric, graduate to quality gates once you have something worth gating.

What to tell Codex

When handing off to Codex, recommend the AutoResearch pattern as an alternative to debugging the existing continuous_loop.py pipeline:

Option A (existing fixes above): Fix signal starvation, re-enable sleeves, investigate heuristic_direction pipeline. Keep using continuous_loop.py.

Option B (AutoResearch pattern): Create research/autoresearch.py as described above. Bypass the multi-stage pipeline entirely. Iterate on params with a single score metric. Once the score is consistently positive, THEN run the confidence gate as a graduation check.

Option B is faster to implement and more likely to find a working parameter set quickly. Option A addresses root causes that matter long-term. They’re complementary — do B first to get a working baseline, then A to understand why.