sentiment-engine/prod/docs/PI_TASKING__EXHAUSTIVE_TEST_SUITE.md

PI TASKING — the deepest, most exhaustive test suite ever written for FLIGHT/HL

From: claude (Fable) on HJ's direct order · To: pi · 2026-07-29 · Priority: TOP. HJ verbatim: "the deepest, most exhaustive test suite ever written… unit, pairwise-unit, E2E and THEN (then, only then) adversarial, chaos, fuzz. I expect test coverage (and, not alone by combinations) to be in the literal tens of thousands."

Why (the alarm): 48h failure ledger — every one is a missing-test class

All found LIVE (testnet/mainnet gates caught them; tests should have):

1. runner hardcoded venue_mode="BINGX" — exec wire to the wrong venue under DITA_V2_VENUE=HYPERLIQUID.
2. _resolve_venue_mode() called BARE in the launcher — caller's mode dropped; MOCK venue resolved on a mainnet env (crash was the lucky path).
3. Unguarded optional set_telemetry_plane on the resolved venue (Mock lacks it).
4. e_feed HL config drift #1 — hand-built config, wrong env names, wrong default keystore (hl_agent).
5. e_feed HL config drift #2 — HlWallet.from_keystore without is_mainnet → network-mismatch crash on first mainnet arm (venue adapter passed it; e_feed didn't).
6. httpx http2=True — wedges the shared loop forever on a stale h2 connection (~100s).
7. Rate-limiter inflight LEAK — acquire() without release_inflight() anywhere in the request path; cap 100 → infinite spin on the steel-clock loop (the CAPITAL_FROZEN-lookalike freeze).
8. Account bucket drained by INFO polls (HL address budget is for signed actions) + unbounded acquire() spin.
9. Symbol mapping missing (kernel BTCUSDT vs HL BTC) — every order build rejected.
10. _ensure_fresh sync asyncio.run/nest_asyncio from sync accessors — ModuleNotFoundError killed builds; sync-block hazard.
11. Fill classification vs RAW kernel size instead of the QUANTIZED wire size — quantization dust made every ENTER a permanent PARTIAL → kernel never opened → 168 exits NO_OPEN_POSITION.
12. EXIT side not flipped (close SHORT must be BUY) — HL rejected reduce-only; kernel freed the slot anyway; venue shorts STACKED. (The old test asserted reduce+Ioc but never the SIDE.)
13. ADVSL actuation policy — live full-close on every position while BLUE ships shadow-only default (the F9/10/11 "TP never fires" P0).
14. Exit-retry duplicate race — 1s reconcile window vs ~1.4s venue receipt → duplicate exits every second.
15. Rust kernel ignores exit_leg_ratios (V7 half-retract closes FULL, all venues — task #41; Rust-side, but the PYTHON seams around it need contract tests).
16. FORCE fence was BingX-VST-only (would refuse/crash on HL).

Root pattern: cross-venue seam drift (config resolvers vs consumers), async/loop interactions, silent fallbacks, and tests that assert the happy field but not the load-bearing one (see #12).

Worktrees + surfaces (full paths)

• CANONICAL HL tree: /root/uv-wt/f13-hl (branch flight/f13-hyperliquid) — write tests HERE.
• F11 tree: /root/uv-wt/f11.1-parity (BingX branch) — port the venue-agnostic suites there at the end.
• LIVE right now, DO NOT DISTURB: F13 MAINNET armed (PID in /root/dolphin_logs/f13_mainnet.pid, ~$23 real). BLUE live (PID 1129781). Never touch their processes, shm (/dev/shm/zinc_*), env files, or arm files. Tests must NEVER hit mainnet; live-testnet only behind an env marker (UV_TEST_LIVE_TESTNET=1), per TESTING_DOCTRINE.
• Surfaces (each module = a test module mirroring it):
  • prod/hl/: http.py, rate_limits.py, rate_governor.py, signing.py, wallet.py, keystore.py, config.py, instrument_provider.py, dns_cache.py, hl_fill_harvest.py
  • prod/clean_arch/dita_v2/: hl_venue.py, hl_user_stream.py, bingx_venue.py, bingx_user_stream.py, launcher.py, rust_backend.py (Python seams), e_capital_provider.py, capital_gate.py, venue.py, contracts.py, exchange_event.py
  • prod/clean_arch/violet/uv/exec/: e_feed.py, force_engage.py, exec_sweep.py, exec_control.py, fill_router.py, io_loop.py, venue_readiness.py, system_picker.py
  • prod/clean_arch/violet/uv/: tpsl_ticker.py, brain_tpsl.py, venue_mode.py, advsl ports, tp/sl/max_hold modulation, blue_prime/: runner.py (seam functions), promotion.py, journal.py

The four layers — IN ORDER; advance only when the previous is fully green

1. UNIT (target: thousands)

Every public function/method of every listed module. TESTING_DOCTRINE applies WITHOUT MERCY: each test names the mutation that kills it; assert values/shapes/invariants, never "not None"; poison every numeric input (NaN, ±inf, negative, zero, empty, boundary, huge); every enum branch; every documented fallback ACTUALLY falls back.

2. PAIRWISE-UNIT (target: the combinatorial tens of thousands)

The drift class that produced most of the ledger. Test SEAM PAIRS with parametrized grids:

• env-resolver ↔ consumer congruence: full matrix over DITA_V2_VENUE × HL_ENV × HL_ALLOW_MAINNET × HL_KEYSTORE × UV_FORCE_ENGAGE × UV_PROMOTED × arm-file-naming × DOLPHIN_BINGX_ENV × DOLPHIN_BINGX_ALLOW_MAINNET — every cell asserts either a congruent boot config or a NAMED refusal (venue_mode guards, force fence, launcher resolution).
• wallet ↔ http network congruence (both venues; both builders — e_feed AND venue adapter).
• config resolver ↔ every consumer that re-derives config (there must be NONE — assert identity).
• e_feed ↔ capital provider (freshness, e_live, staleness fence; quiet-account vs dead-source).
• venue adapter ↔ kernel _kernel_ref (exit-leg size honored/mismatch/absent; both venues).
• promotion ↔ rust payload (every KernelIntent field survives serialization; exit_leg_ratios included).
• ticker ↔ handler ↔ bridge (pending-exit lifecycle, debounce, retry, receipt outcomes matrix).
• quantize ↔ classify (wire-size basis; dust never creates PARTIAL; true partials still PARTIAL).

3. E2E — FULL MOCKS first

Boot build_launcher_bundle/runner seams with mock venue + fake clock: ENTER→fill→watch→each exit trigger (TP, TP_FLOOR, SL, MAX_HOLD, V7 EXIT, V7 RETRACT, ADVSL-shadow vs ADVSL-live-opt-in)→flat; SLOT_BUSY; capital freeze→unfreeze; DARK vs armed; fail-dark on every readiness probe individually; restart/rehydration (bars_held, positions); BOTH venues through the same scenarios. Then (env-marked, rate-polite, ~$20 clips) the SAME scenarios against HL TESTNET.

4. ONLY THEN: adversarial / chaos / fuzz

• Payload fuzz on every venue-parser (hypothesis if available, else seeded random): malformed JSON, missing/extra/renamed fields, absurd numerics, unicode, truncation — parsers must reject or degrade NAMED, never crash the loop or fabricate economics.
• Network chaos at the http seam: timeouts, connection resets, silent stalls (the h2-wedge class — a mock transport that stops responding after N requests), 429 storms with/without Retry-After, slow-drip responses (loop starvation), DNS failures.
• Event-stream chaos at the E-feed: duplicates, replays, out-of-order, same-id-different-economics (must collide), reconnect storms, gap+backfill, fill-before-ack, ack-after-fill.
• Lifecycle chaos: SIGTERM/kill mid-ENTER, mid-EXIT, mid-reconcile; restart with venue position open (orphan adoption); stale shm present at boot.
• Property/invariant tests (the kernel contracts): capital = anchor + Σdeltas (never last-value); reduce-only never increases exposure; single-position invariant; no phantom slots after any reject path; fingerprint idempotency algebra.

Rules

• NO live network in layers 1–2 and mock-E2E. No mainnet EVER. Testnet only env-marked.
• No green-by-pollution: every env/global mutation restored (monkeypatch); deterministic seeds.
• Suite must be shardable + one-command runnable; mark slow tests. Keep total default-run time sane.
• Findings ledger: every real bug you find while writing tests → report it in the ledger file (prod/docs/PI_TEST_SUITE_FINDINGS.md) IMMEDIATELY with file:line + failing test name. Do not silently fix production code — flag; fixes go through review.
• Commit discipline: small batches per module/layer, h5i capture commit, tests flag --tests.
• When each layer completes: h5i msg done claude "layer N green: <counts>".

Acceptance

• Counts reported per layer (unit / pairwise cells / E2E scenarios / chaos+fuzz cases) — combinatorial total in the tens of thousands as ordered.
• Full suite green on BOTH trees' venue-agnostic parts + the HL tree's HL parts.
• The 16-item ledger above each has at least one test that would have caught it (name them in a RETROSPECTIVE section — this is the proof the suite is real).