pam ▸ docs/generated/ralph_loop_2026-03-16_replay_sleeve_concentration.md
updated 2026-03-28

Ralph Loop 2026-03-16: Replay Sleeve Concentration Gate

Change

Added sleeve-family concentration metrics to the shared research evaluation path by deriving them from strategy rationale tags:

  1. unique_sleeves
  2. top_sleeve
  3. top_sleeve_action_count
  4. top_sleeve_share
  5. top_two_sleeve_share

These metrics now flow through:

  1. research/run_replay.py
  2. research/search.py
  3. research/continuous_loop.py
  4. research/status.py
  5. research/discord_status.py

The confidence gate now also blocks obviously single-sleeve replay quality with:

  1. research_loop.min_unique_sleeves
  2. research_loop.max_top_sleeve_share

Default bounds:

  1. min_unique_sleeves: 2
  2. max_top_sleeve_share: 0.85

Why

Ticker concentration was already explicit, but replay quality could still look broader than it was if many actions came from one sleeve family across several tickers.

The current persisted replay makes that visible:

  1. 72 actions
  2. 3 unique tickers
  3. 1 unique sleeve
  4. top sleeve politics_midprice_reversal
  5. top sleeve share 1.0

That is not diversified strategy quality, even before the hard deployment gate is considered.

Validation

Focused tests:

pytest -q tests/test_research_search.py tests/test_research_status.py

Result:

14 passed in 0.67s

Replay smoke check:

python3 - <<'PY'
from research.run_replay import run_replay
summary = run_replay('research/replay_rows.jsonl', 'research/strategy.py')
print(summary)
PY

Observed:

ReplaySummary(strategy_name='strategy', score=0.0193, avg_realized_bps=1.3889, hit_rate=0.0139, action_count=72, unique_tickers=3, top_ticker='KXPERSONPRESFUENTES-45', top_ticker_action_count=36, top_ticker_share=0.5, top_two_ticker_share=0.9861, unique_sleeves=1, top_sleeve='politics_midprice_reversal', top_sleeve_action_count=72, top_sleeve_share=1.0, top_two_sleeve_share=1.0, total_rows=1280)

Status smoke check:

python3 main.py research-status

Relevant line:

- current persisted replay: rows=1280 score=0.0193 hit_rate=0.0139 actions=72 tickers=3 top=KXPERSONPRESFUENTES-45 top_share=0.5 top2_share=0.9861 sleeves=1 top_sleeve=politics_midprice_reversal top_sleeve_share=1.0

Outcome

This improves replay evaluation quality by making single-sleeve behavior explicit and confidence-gated instead of letting repeated rationale-family actions overstate research progress.