pam ▸ docs/archive/2026-05-02-pam-hardening/diagnostics/overnight-diagnostic/LEAK-ANALYSIS.md
updated 2026-04-16

Why Pam isn’t making money — ranked hypotheses

Evidence window: data/overnight_overlap_runs.jsonl (97 iterations, Mar 7–8, 2026), monitor.log (3010 scans, Sep–Oct 2025), data/unicorn.db, code read.

H1 — Matching pipeline returns zero pairs. (Confidence: very high)

The single biggest leak. 97 consecutive overnight iterations produced matched_pairs: 0 against 20k+ Kalshi and 35k+ Polymarket markets. Background monitor: 3,010 scans, 0 opportunities. You cannot arbitrage what you cannot pair.

Evidence in code (arbitrage/pairing.py::find_equivalent_pairs_from_contracts, lines 916–1060): - Indexes Polymarket contracts by (market_family, token). Market families are derived from fuzzy keyword heuristics in _market_family (line 565). Kalshi and Polymarket have to land in the exact same family string (generic, occurrence, deadline, release_deadline, threshold, winner, nominee) to become candidates — otherwise continue at line 949. - Threshold-mode requires require_exact_threshold=True by default (line 903) and threshold_delta > 0 → continue at line 994. Any rounding difference between “$110,000” and “110k” or any off-by-one-dollar strike kills the match. - Expiry tolerance defaults to 90 seconds (expiry_tolerance_sec, line 902). Kalshi settles on wall-clock, Polymarket on block time — 90s is knife-edge. - Text-mode fallback requires similarity >= 0.82 (min_question_similarity, line 904). Real cross-venue questions rarely clear 0.82 on tokenized cosine-like similarity. - Overlap-loop gate (tools/overnight_overlap_loop.py::_qualified_candidate, lines 24–36) requires equivalence_score >= 0.995 AND budget_usd <= 10 AND passes_hurdle AND rebalance_status in (healthy,monitor). Even if pairs existed, this gate is so tight almost nothing qualifies. - Config key bug: config.yaml line 36 writes expiry_tolerance_seconds: 90; pairing.py reads expiry_tolerance_sec. Config silently ignored → falls back to default 90.

Test to validate: run arbitrage/pairing.py::diagnose_near_pairs_from_contracts offline on the current DB with logging; count reasons. Also run with require_exact_threshold=False, expiry_tolerance_sec=3600, min_question_similarity=0.6 and see how many pairs emerge. (Script is in FIX-PLAN.)


H2 — Kalshi market sync is undersampling drastically. (Confidence: high)

DB has 477 Kalshi markets vs 16,225 Polymarket. Kalshi in reality has ~10k+ active markets. The overlap loop says it “fetched 100, inserted 100” per cycle with cycle_completed: false and carries a cursor — so the cursor walks forward but the loop never finishes its sweep and only persists a small rolling slice. Worse: only 81 Elections and 52 Politics markets are present, which means most of Kalshi’s politics universe (where arb with Polymarket lives) is simply absent from Pam’s index.

Code: tools/overnight_overlap_loop.py defaults kalshi_pages_per_run=2, kalshi_page_size=100 — 200 rows/iteration against tens of thousands. At 5-minute cadence, a full sweep would take days, and the inventory eviction logic (if any) could drop markets before their partner shows up on Polymarket.

Test: SELECT COUNT(*) FROM catalog_market_inventory WHERE venue='kalshi'; vs the live /markets?status=open count from Kalshi API. Expect >10x gap.


H3 — Polymarket client runs in “limited” mode. (Confidence: high)

Every past monitor/scanner logline reads "Polymarket private key not provided. API will be limited.". That log comes from arbitrage/polymarket_api.py:50 — the process was started without loading .env. Without the private key the CLOB client is None and we only get public REST endpoints: no order book depth authenticated, no ability to place orders, and reduced rate caps. That alone means price data used for spread calculation is thinner than it should be.

Test: start monitor with python -c "from dotenv import load_dotenv; load_dotenv(); ..." or via main.py (which does load_dotenv at import) and verify the warning disappears.


H4 — Human-approval bottleneck / no Telegram loop running. (Confidence: high)

The system is architected around Telegram approvals (execution/approvals.py, monitoring/telegram_bot.py). The webhook is not running. Even if matches were found, there is no live loop from signal → Telegram → approve → order. The opportunity-hunter and background monitor write to opportunity_alerts.json only; they don’t push to Telegram.

Test: curl https://api.telegram.org/bot<TOKEN>/getWebhookInfo — expect empty/outdated URL.


H5 — Equivalence threshold ladder is miscalibrated. (Confidence: high)

Three different thresholds coexist: - matcher.min_equivalence_score: 0.6 (config) - pairing.min_question_similarity: 0.82 (code default, not in config → used) - auto_execute_min_equivalence_score: 0.995 (main.py:336) — extreme - Overlap loop qualified gate: 0.995 (overnight loop)

The 0.995 gate is effectively “only accept mathematical duplicates.” Historic QA report documents this pattern: started at 0.92, lowered to 0.6, but downstream gates are still at 0.995.

Test: count how many pairs produced by a relaxed run have score ≥ 0.995 vs ≥ 0.8. Expect ratio near zero.


H6 — Oracle-divergence penalty + threshold mismatch combine to zero-out real pairs. (Confidence: medium-high)

matcher/align.py applies oracle_risk * 0.4 penalty plus -0.5 subject + -0.4 expiry + threshold penalties. When Kalshi uses CF-Benchmarks RTI and Polymarket uses Binance 1m close — the most common BTC pairing — oracle risk is real but the penalty combined with a 1–2 USD strike gap knocks scores below 0.6. This is documented in ANALYSIS_matching_algorithm.md.

Test: pick a known BTC pair (Kalshi KXBTCD- ticker + Polymarket “Bitcoin above $X on date Y”) and print equivalence_score_detailed.


H7 — Sizing / Kelly parameters fine. (Confidence: medium — we have no trades to evaluate.)

config.yaml does not set Kelly fraction or per-leg liquidity; code defaults to 25% Kelly, 8% max position, $100 min liquidity. These are conservative and fine. This is not the leak; it is the downstream floor we’ll hit once H1 is fixed.


H8 — Fees/basis eat the observable edge. (Confidence: low for “why 0 opp found”; high for “why thin edge even when found”.)

Polymarket charges gas + 2% taker (via fee_rate_bps in pricing). Kalshi has per-contract fees. arbitrage/pricing.py::net_spread_bps factors these, but min_spread default is 5% (self.min_spread = 0.05 in crypto_scanner.py:69). That’s a realistic floor, though if this ever produced live candidates it’d further filter. Not the current bottleneck.


H9 — Latency / market movement. (Confidence: low right now, irrelevant until H1 fixed.)

Scanner cadence is 5 minutes; arbitrage windows are seconds. Even when this system finds a pair, by the time the human approves on Telegram the edge has re-collapsed. Relevant after H1/H4 resolved.


H10 — Data model / DB hygiene. (Confidence: medium.)

Two DBs present (unicorn.db, unicorn_phase1.db, backups). catalog_sync_state probably not pruning. Not causing the zero-pair problem but creates ambiguity about which DB the live scanner reads.


Ranking by expected $ impact of fixing

  1. H1 (matching returns 0) — unblocks every dollar.
  2. H5 (threshold ladder) — practical step 2, co-dependent with H1.
  3. H2 (Kalshi undersampling) — doubles the pair universe.
  4. H3 (Polymarket key not loaded) — currently benign (we’re off), but when you restart it affects price quality.
  5. H6 (oracle + threshold combined penalty) — refinement after H1+H5.
  6. H4 (Telegram loop) — needed to close the execution loop; no point without H1.
  7. H7–H10 — later.