From f0d80b05d3d5eee01629bf398f8e118dfc12e3f7 Mon Sep 17 00:00:00 2001 From: Codex Date: Wed, 29 Jul 2026 13:35:20 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20PI=20tasking=20=E2=80=94=20exhaustive?= =?UTF-8?q?=20test=20suite=20(unit/pairwise/E2E/chaos;=2016-bug=20ledger)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- .../docs/PI_TASKING__EXHAUSTIVE_TEST_SUITE.md | 109 ++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 prod/docs/PI_TASKING__EXHAUSTIVE_TEST_SUITE.md diff --git a/prod/docs/PI_TASKING__EXHAUSTIVE_TEST_SUITE.md b/prod/docs/PI_TASKING__EXHAUSTIVE_TEST_SUITE.md new file mode 100644 index 0000000..d4a19cd --- /dev/null +++ b/prod/docs/PI_TASKING__EXHAUSTIVE_TEST_SUITE.md @@ -0,0 +1,109 @@ +# PI TASKING — the deepest, most exhaustive test suite ever written for FLIGHT/HL + +**From:** claude (Fable) on HJ's direct order · **To:** pi · **2026-07-29** · **Priority: TOP.** +HJ verbatim: *"the deepest, most exhaustive test suite ever written… unit, pairwise-unit, E2E +and THEN (then, only then) adversarial, chaos, fuzz. I expect test coverage (and, not alone by +combinations) to be in the literal tens of thousands."* + +## Why (the alarm): 48h failure ledger — every one is a missing-test class + +All found LIVE (testnet/mainnet gates caught them; tests should have): +1. runner hardcoded `venue_mode="BINGX"` — exec wire to the wrong venue under DITA_V2_VENUE=HYPERLIQUID. +2. `_resolve_venue_mode()` called BARE in the launcher — caller's mode dropped; MOCK venue resolved on a mainnet env (crash was the *lucky* path). +3. Unguarded optional `set_telemetry_plane` on the resolved venue (Mock lacks it). +4. e_feed HL config drift #1 — hand-built config, wrong env names, wrong default keystore (`hl_agent`). +5. e_feed HL config drift #2 — `HlWallet.from_keystore` without `is_mainnet` → network-mismatch crash on first mainnet arm (venue adapter passed it; e_feed didn't). +6. httpx `http2=True` — wedges the shared loop forever on a stale h2 connection (~100s). +7. Rate-limiter inflight LEAK — `acquire()` without `release_inflight()` anywhere in the request path; cap 100 → infinite spin on the steel-clock loop (the CAPITAL_FROZEN-lookalike freeze). +8. Account bucket drained by INFO polls (HL address budget is for signed actions) + unbounded `acquire()` spin. +9. Symbol mapping missing (kernel `BTCUSDT` vs HL `BTC`) — every order build rejected. +10. `_ensure_fresh` sync `asyncio.run`/nest_asyncio from sync accessors — ModuleNotFoundError killed builds; sync-block hazard. +11. Fill classification vs RAW kernel size instead of the QUANTIZED wire size — quantization dust made every ENTER a permanent PARTIAL → kernel never opened → 168 exits `NO_OPEN_POSITION`. +12. EXIT side not flipped (close SHORT must be BUY) — HL rejected reduce-only; kernel freed the slot anyway; venue shorts STACKED. (The old test asserted reduce+Ioc but never the SIDE.) +13. ADVSL actuation policy — live full-close on every position while BLUE ships shadow-only default (the F9/10/11 "TP never fires" P0). +14. Exit-retry duplicate race — 1s reconcile window vs ~1.4s venue receipt → duplicate exits every second. +15. Rust kernel ignores `exit_leg_ratios` (V7 half-retract closes FULL, all venues — task #41; Rust-side, but the PYTHON seams around it need contract tests). +16. FORCE fence was BingX-VST-only (would refuse/crash on HL). + +**Root pattern:** cross-venue seam drift (config resolvers vs consumers), async/loop interactions, +silent fallbacks, and tests that assert the happy field but not the load-bearing one (see #12). + +## Worktrees + surfaces (full paths) + +- **CANONICAL HL tree:** `/root/uv-wt/f13-hl` (branch `flight/f13-hyperliquid`) — write tests HERE. +- F11 tree: `/root/uv-wt/f11.1-parity` (BingX branch) — port the venue-agnostic suites there at the end. +- **LIVE right now, DO NOT DISTURB:** F13 MAINNET armed (PID in `/root/dolphin_logs/f13_mainnet.pid`, + ~$23 real). BLUE live (PID 1129781). Never touch their processes, shm (`/dev/shm/zinc_*`), env + files, or arm files. Tests must NEVER hit mainnet; live-testnet only behind an env marker + (`UV_TEST_LIVE_TESTNET=1`), per TESTING_DOCTRINE. +- Surfaces (each module = a test module mirroring it): + - `prod/hl/`: http.py, rate_limits.py, rate_governor.py, signing.py, wallet.py, keystore.py, + config.py, instrument_provider.py, dns_cache.py, hl_fill_harvest.py + - `prod/clean_arch/dita_v2/`: hl_venue.py, hl_user_stream.py, bingx_venue.py, bingx_user_stream.py, + launcher.py, rust_backend.py (Python seams), e_capital_provider.py, capital_gate.py, venue.py, + contracts.py, exchange_event.py + - `prod/clean_arch/violet/uv/exec/`: e_feed.py, force_engage.py, exec_sweep.py, exec_control.py, + fill_router.py, io_loop.py, venue_readiness.py, system_picker.py + - `prod/clean_arch/violet/uv/`: tpsl_ticker.py, brain_tpsl.py, venue_mode.py, advsl ports, + tp/sl/max_hold modulation, `blue_prime/`: runner.py (seam functions), promotion.py, journal.py + +## The four layers — IN ORDER; advance only when the previous is fully green + +### 1. UNIT (target: thousands) +Every public function/method of every listed module. TESTING_DOCTRINE applies WITHOUT MERCY: +each test names the mutation that kills it; assert values/shapes/invariants, never "not None"; +poison every numeric input (NaN, ±inf, negative, zero, empty, boundary, huge); every enum branch; +every documented fallback ACTUALLY falls back. + +### 2. PAIRWISE-UNIT (target: the combinatorial tens of thousands) +The drift class that produced most of the ledger. Test SEAM PAIRS with parametrized grids: +- env-resolver ↔ consumer congruence: full matrix over + `DITA_V2_VENUE × HL_ENV × HL_ALLOW_MAINNET × HL_KEYSTORE × UV_FORCE_ENGAGE × UV_PROMOTED × + arm-file-naming × DOLPHIN_BINGX_ENV × DOLPHIN_BINGX_ALLOW_MAINNET` — every cell asserts either + a congruent boot config or a NAMED refusal (venue_mode guards, force fence, launcher resolution). +- wallet ↔ http network congruence (both venues; both builders — e_feed AND venue adapter). +- config resolver ↔ every consumer that re-derives config (there must be NONE — assert identity). +- e_feed ↔ capital provider (freshness, e_live, staleness fence; quiet-account vs dead-source). +- venue adapter ↔ kernel `_kernel_ref` (exit-leg size honored/mismatch/absent; both venues). +- promotion ↔ rust payload (every KernelIntent field survives serialization; exit_leg_ratios included). +- ticker ↔ handler ↔ bridge (pending-exit lifecycle, debounce, retry, receipt outcomes matrix). +- quantize ↔ classify (wire-size basis; dust never creates PARTIAL; true partials still PARTIAL). + +### 3. E2E — FULL MOCKS first +Boot `build_launcher_bundle`/runner seams with mock venue + fake clock: +ENTER→fill→watch→each exit trigger (TP, TP_FLOOR, SL, MAX_HOLD, V7 EXIT, V7 RETRACT, ADVSL-shadow +vs ADVSL-live-opt-in)→flat; SLOT_BUSY; capital freeze→unfreeze; DARK vs armed; fail-dark on every +readiness probe individually; restart/rehydration (bars_held, positions); BOTH venues through the +same scenarios. Then (env-marked, rate-polite, ~$20 clips) the SAME scenarios against HL TESTNET. + +### 4. ONLY THEN: adversarial / chaos / fuzz +- Payload fuzz on every venue-parser (hypothesis if available, else seeded random): malformed JSON, + missing/extra/renamed fields, absurd numerics, unicode, truncation — parsers must reject or + degrade NAMED, never crash the loop or fabricate economics. +- Network chaos at the http seam: timeouts, connection resets, silent stalls (the h2-wedge class — + a mock transport that stops responding after N requests), 429 storms with/without Retry-After, + slow-drip responses (loop starvation), DNS failures. +- Event-stream chaos at the E-feed: duplicates, replays, out-of-order, same-id-different-economics + (must collide), reconnect storms, gap+backfill, fill-before-ack, ack-after-fill. +- Lifecycle chaos: SIGTERM/kill mid-ENTER, mid-EXIT, mid-reconcile; restart with venue position + open (orphan adoption); stale shm present at boot. +- Property/invariant tests (the kernel contracts): capital = anchor + Σdeltas (never last-value); + reduce-only never increases exposure; single-position invariant; no phantom slots after any + reject path; fingerprint idempotency algebra. + +## Rules +- NO live network in layers 1–2 and mock-E2E. No mainnet EVER. Testnet only env-marked. +- No green-by-pollution: every env/global mutation restored (monkeypatch); deterministic seeds. +- Suite must be shardable + one-command runnable; mark slow tests. Keep total default-run time sane. +- **Findings ledger**: every real bug you find while writing tests → report it in the ledger file + (`prod/docs/PI_TEST_SUITE_FINDINGS.md`) IMMEDIATELY with file:line + failing test name. Do not + silently fix production code — flag; fixes go through review. +- Commit discipline: small batches per module/layer, `h5i capture commit`, tests flag `--tests`. +- When each layer completes: `h5i msg done claude "layer N green: "`. + +## Acceptance +- Counts reported per layer (unit / pairwise cells / E2E scenarios / chaos+fuzz cases) — + combinatorial total in the tens of thousands as ordered. +- Full suite green on BOTH trees' venue-agnostic parts + the HL tree's HL parts. +- The 16-item ledger above each has at least one test that would have caught it (name them in a + RETROSPECTIVE section — this is the proof the suite is real).