diff --git a/prod/docs/PINK_DRIVE_LOOP_PORT_INVENTORY.md b/prod/docs/PINK_DRIVE_LOOP_PORT_INVENTORY.md new file mode 100644 index 0000000..deadeef --- /dev/null +++ b/prod/docs/PINK_DRIVE_LOOP_PORT_INVENTORY.md @@ -0,0 +1,159 @@ +# PINK drive-loop — certified-learnings inventory (the §7 port checklist) + +**From**: Fable, 2026-07-14. **Source**: `prod/clean_arch/runtime/pink_direct.py` (1837 L), +read in full. **Purpose**: a file does not reach 1.8 K LoC for no reason — this catalogues +*every* execution mechanism, the incident each one encodes, and its disposition for the +Unified Execution Layer port (§7). **Reuse as much as possible; supersede only where a +newer certified learning demands it.** + +**Why the file is big (honest accounting):** ~16 execution scar-tissue mechanisms + the +kernel/account plumbing that PINK's runtime also carries. The scar tissue is the part +that ports; the plumbing stays behind the kernel seam. Almost nothing is dead weight. + +Disposition legend: **PORT** = carry into `prod/exec_unified/` warts-and-all · **SEAM** = +kernel/account concern, stays above/below the exec layer (contract §2 forbids it here) · +**AMEND** = port but update per a newer certified learning · **SUPERSEDED** = do not carry. + +> ## ⛔ ZERO SILENT SUPPRESSION (operator ruling, 2026-07-14: "we paid dearly for pink") +> +> **Nothing here vanishes.** A mechanism that is NOT ported as live code (SEAM, AMEND- +> deferred, or SUPERSEDED) must be carried into the exec-layer source as a **referenced +> comment** — the full reasoning plus a `pink_direct.py:LNNN` citation — at the seam where +> it would otherwise have lived. Every branch in PINK is a headstone bought with a real +> incident; the port preserves the *lesson* even when it relocates the *code*. A reviewer +> reading `prod/exec_unified/drive_loop.py` must be able to see, inline, why each PINK +> guard exists and where it went. Silent omission is the one unforgivable port error. + +--- + +## 1. The drive loop's heart — `_handle_expired_working` (L1089–1226). PORT, verbatim logic. + +The exact ordered sequence. **Every step exists because skipping it caused a live +incident.** This is the transcription target; do not re-derive it. + +1. **Already-resolved short-circuit** (L1091): `router.working(tid) is None` → return. A + fill/cancel notification raced the TTL sweep. *Without it: double-resolution.* +2. **Pump venue events FIRST** (L1095), then re-check working (L1096). The quote may have + filled between expiry and handling. *Without it: cancel an already-filled order.* +3. **Cancel the quote** (L1109–1128): idempotent; `CANCEL_REJECT` on a filled order is + harmless; on a *partial* entry this cancels the remainder. Injected `KernelIntent` + CANCEL, same `trade_id`. +4. **Pump again** (L1129) — the cancel round-trip may have surfaced a fill. +5. **Re-classify after the cancel** (L1132–1141): `_entry_filled()` / `_exit_done()` → + `note_fill`, pop working, emit `fill_after_ttl`, return. *The fill-races-cancel window + is real; this is the reclassification that catches it.* +6. **EXIT never strands** (L1147–1164): escalate to MARKET, **same trade_id**, sized from + the slot (`_exit_intent_from_slot`). An exit that can't maker-fill MUST cross. +7. **ENTER miss policy** (L1166): `skip | retry(bounded) | market` from config. +8. **Slot-busy guard** (L1173–1179): `size>0 or stage in _SLOT_OPENISH` → skip. *Never + double-enter on a raced remainder fill.* +9. **Venue-truth requote gate** (L1185, `_exec_safe_to_requote`): re-quote ONLY when the + venue is *provably flat*. Ambiguity → skip. **"A skipped entry is always safe, a + doubled position is not."** (live double-entry, 2026-06-10.) +10. **Requote reference price** (L1191–1205): from the working order's OWN asset history + (multi-asset), fallback to `plan.limit_price`. Rebuild intent via `replace(...)`. + +## 2. Execution mechanisms — full table + +| # | Mechanism (lines) | Incident it encodes | Disposition | +|---|---|---|---| +| 1 | **"Every await followed by a state re-check"** (L933-934 doctrine) | fills race cancels on a live venue | **PORT** — the governing invariant of the whole loop | +| 2 | `_SLOT_OPENISH` state set (L936) | classify "position exists" across FSM states incl. PARTIAL/EXIT_* | **PORT** (as typed predicate, not string tuple — V-TYPES) | +| 3 | `_exec_after_submit` — filled-now / resting / **rejected-post-only → expired deadline** (L975-1019) | one shared resolution path: a rejected GTX registers already-expired so the TTL loop resolves it through the SAME miss/fallback path | **PORT** — the "one shared path" is the gem; don't build two | +| 4 | `_exec_safe_to_requote` — **fails SAFE** (L1049-1068) | recent own fill (<5 s) OR live venue position OR probe error → False | **AMEND** — port; make the 5 s window a named, injected constant | +| 5 | `_exec_ttl_loop` — 1 s sweep (L1070) | scan cadence (~10 s) too coarse for 5-8 s TTLs → dedicated task | **AMEND** — port; inject cadence (spec: no hardcoded 1 s) | +| 6 | `pump_venue_events` — drain late fills, dedup, **resolve working quotes vs applied truth** (L1228-1311); CANCEL_ACK → pull deadline to now (route through shared path) | resting LIMIT/partials arrive after submit returns; post-only reject surfaces via reconcile | **PORT** (dedup/persist = SEAM; the working-quote resolution = PORT) | +| 7 | `_last_own_fill_mono` 5 s hot-window latch (L335, L648, L1055, L1323) | REST reconcile lags WS fills by seconds → live double-entry | **AMEND** — port; named constant, injected clock | +| 8 | `_fill_is_ours` ownership (L588-611): `p-` cid prefix → symbol membership → unattributable | shared VST account (PRODGREEN/BLUE/manual); BingX WS does NOT echo clientOrderId | **PORT** (concept) — maps to spec `u-`/`m-` clientOrderId + session symbol set | +| 9 | Single-slot ENTER block `_unsafe_entry_reason` (L1313-1356) | filled maker entry vanishes from openOrders → misread as cancel → re-enter → 2× | **SEAM** (single-slot is a kernel invariant) — but the *venue-position-is-truth* rule is **PORT** | +| 10 | `_exit_intent_from_slot` — size close from authoritative slot, cap to remaining (L1358-1380) | malformed policy size must neither strand nor overshoot | **SEAM** (kernel accounting) — exec layer receives already-sized `size` (contract §2) | +| 11 | Urgent-exit **preempts** working maker exit (L1660-1664) | cancel the resting quote FIRST so MARKET close can't double-fill | **PORT** — maps to PROTECT/CATASTROPHIC preempting a HARVEST/ROTATE working order | +| 12 | `suppress` duplicate guard (L1650-1659) | a working maker quote already represents this action → don't double-submit | **PORT** | +| 13 | **Router failure → degrade to legacy taker** (L971-973) | router bug must never block trading | **PORT** — fail-OPEN to taker is the safety default | +| 14 | `_MIN_SANE_PRICE` + fail-safe sizing refuse-the-OPEN (L81-85, L1337-1356) | price→0 overflows size=notional/price to inf; corrupt input = untrustworthy signal, skip don't clamp | **SEAM** (kernel sizing) — but the *refuse-don't-clamp* doctrine informs exec poison-input handling | +| 15 | Retry/market fallback plans, bounded (L1206-1226) | miss policy: retry once then skip/market | **PORT** — maps to urgency chase budgets (`_constants.py`) | +| 16 | Every transition `self._emit("exec_router", …)` (throughout) | before-and-after observability | **PORT** — but on the §12 side-lane, NEVER the exec path (b46ebd2) | + +## 3. Kernel / account plumbing — SEAM (does NOT enter the exec layer; contract §2 forbids it) + +Capital seeding from exchange / K≈E reconcile / `_enter_frozen` (L443-528, L717-763); fee +calibration loop + snapshot persist/restore (L129-166, L530-586); Hz writes (L906-929); +alpha sizer feedback, IRP asset picker, DC gate, ACB boost (L826-905, L1461-1492, +L1382-1459). **All of this is brain + account.** The Unified Execution Layer "knows +NOTHING about the caller's world" (spec §2) — it never sees capital, leverage, posture, +or picks an asset. These stay in the kernel seam / brain, above the `ExecutionRequest`. + +## 4. Newer certified learnings that AMEND the port + +1. **Maker fee = 2.00 bps MEASURED** (Q1, today). L440 `maker_rate: 0.0002` is CONFIRMED + (0.02% = 2 bps), not superseded. Good — the assumption the file baked was right. +2. **⚠ SIGN BUG candidate, L673**: `"fee": event.fee, # negative = rebate`. Today's Q1 + established **BingX commission negative = DEBIT (a cost), not a rebate** — the exact + trap omp fell into. If `bingx_user_stream` passes the raw venue sign into `event.fee`, + the K-fold here treats a cost as income → K optimistic vs E. Likely *bounded* (the K≈E + reconcile gate would catch the drift and freeze; calibration uses `abs()` at L549) — but + it is a real sign inconsistency worth verifying. **NOT fixing now** (BLUE/PINK live, + untouchable); flagged for the operator. The exec-layer port must get the sign right at + its own friction telemetry (§12). +3. **Hardcoded cadences** (1 s TTL L1076; 5 s hot window L1055/L1323) → inject/name with + provenance (spec NFR: cadence injectable, constants from data). +4. **`spread_bps` gate still dead** (memory `pink-exec-router` open item; spec §4-12): the + Router's spread gate was plumbed but never fed live top-of-book. The port WIRES it — + this is new work, not a transcription. +5. **Urgency generalizes PINK's binary**: PINK has entry/exit + miss(skip/retry/market). + The spec's 5-class ladder is richer and *subsumes* it: PINK "exit always escalates to + MARKET" = PROTECT/CATASTROPHIC crossing; PINK "entry miss, never chase" = ACQUIRE + abandon. `prod/exec_unified/router.py` already encodes the richer ladder; the drive + loop supplies PINK's proven *mechanics* underneath it. + +## 4b. The SECOND axis — S-grades (size), orthogonal to the T-tier (spec §15.2) + +PINK's drive loop is a **single-clip (S0/S1) machine** — it fires one order per action and +never reasons about book footprint. That is exactly correct to port, but the port must +carry the S-axis the spec added, because size that moves the book is a different pain +class the T-tier does not cover: + +| Grade | Definition | Port implication | +|---|---|---| +| **S0** | size ≪ top-of-book depth (walk < 1 tick) | any tier executes directly — PINK's loop as-is | +| **S1** | walks visible levels (expected walk ≥ spread) | T2+ prices the walk into give-up/cross (feeds the HARVEST regression rule + placer offset) | +| **S2** | exceeds book capacity in the urgency window | **REFUSED WHOLE, loudly**, below T14 — the pain-fence (NEW guard, no PINK equivalent) | +| **S3** | size IS the market | MALKHUT/VIBRASS territory — out of scope for this port | + +Two rulings the port encodes: +1. **Slicing is COMPOSABLE and lives ABOVE the contract** (`T*.s` sub-rung): TWAP/VWAP/ + participation slicers are just *callers* — they decompose one parent into N child + `ExecutionRequest`s, each inheriting urgency + S0/S1. The layer cannot tell a slicer + from a brain (agnosticism, §2). The exec layer gains only a `parent_request_id` + journal field so §12 friction telemetry aggregates families. Ships as a separate + `sizer` module — NOT inside `exec_unified`. +2. **In-order size mechanics are T14+ and NOT composable**: iceberg, live-depth-adaptive + child sizing, queue games, stealth. Each changes the working-order state machine and + carries its own weird-bug class → they may not ride in as sub-rungs. The port stops at + T13; T14 is a deliberate later escalation. +3. **The pain-fence (NEW, add to the port):** below T14 the drive loop must REFUSE an S2+ + request whole — loudly, with the computed expected-walk in the refusal — so nobody + discovers the large-size pain class by accident in production. This is a `RoutingDecision` + / pre-submit refusal path with no PINK ancestor; it is new certified doctrine, not scar + tissue. `contract.py` needs an `s_grade` field (computed at intake per Appendix E) or a + caller-supplied grade; the fence reads it. + +## 5. Port mapping → `prod/exec_unified/` + +- `router.py` (done) — the whether-maker policy; §6 urgency ladder generalizes PINK miss policy. +- `placer.py` (omp, in flight) — the where-in-book offset; wires the dead `spread_bps` gate. +- `drive_loop.py` (next, mine) — transcribe §1 sequence + table rows #1-8,11-13,15-16. + Injected seams (C1): `clock`, `venue_io`, `market_state`, `working_registry` — no ambient + state, so it can run venue-side someday. The kernel/account SEAM rows stay behind an + injected `KernelPort` — but per the ⛔ zero-suppression rule, **each SEAM row also gets a + referenced comment** at its call site (e.g. above the `KernelPort.settle` call: the K≈E + reconcile-freeze reasoning + `pink_direct.py:717-763`), so the lesson is visible inline. + Add the S2+ pain-fence (§4b.3) as a pre-submit refusal with no PINK ancestor. +- `_constants.py` — add `REQUOTE_HOT_WINDOW_MS` (=5000, prov: 2026-06-10 double-entry), + keep cadence out (injected). +- `contract.py` — add `s_grade` (S0-S3, spec §15.2) + `parent_request_id` (slicer family + aggregation, §4b.1). Both are AMEND items already documented; wire when the fence lands. + +**Golden rule for the port (operator's ruling, restated):** transcribe the mess, don't +tidy it. Each branch above is a headstone. The only edits allowed are the AMEND rows — +and every one of those cites a newer *certified* learning, not a preference.