Files
sentiment-engine/prod/docs/PINK_DRIVE_LOOP_PORT_INVENTORY.md
Codex 670b739a83 exec(unified): drive loop §7 — PINK _handle_expired_working ported + audit build items
drive_loop.py transcribes pink_direct.py:1089 (the 10-step expiry sequence) warts-and-all
against injected ports (ExecPort seam: clock+kernel+venue), so no ambient state (C1). Carries
the scar tissue: pump-before-cancel, re-classify-after-cancel (fill races cancel), EXIT never
strands→MARKET, slot-busy double-entry guard, fail-safe venue-truth requote gate. SEAM
learnings preserved as referenced comments (zero-silent-suppression rule). Decimal sizes (H1).

- working.py: WorkingRegistry + WorkingOrder, injected clock, rejected→expire_now (one shared
  path). contract.py: client_order_id_core(attempt) — unique per attempt (audit H4/FIX).
- _constants.py: REQUOTE_HOT_WINDOW_S=5.0 (prov: 2026-06-10 double-entry).
- inventory §6: audit build items BI-1..5 folded in.
- test_drive_loop.py: 15 scar-tissue tests. Mutation-verified RED under exit-no-escalate
  (kills 3) and no-hot-window (kills double-entry guard). Full exec_unified suite: 77 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 23:51:04 +02:00

14 KiB
Raw Blame History

PINK drive-loop — certified-learnings inventory (the §7 port checklist)

From: Fable, 2026-07-14. Source: prod/clean_arch/runtime/pink_direct.py (1837 L), read in full. Purpose: a file does not reach 1.8 K LoC for no reason — this catalogues every execution mechanism, the incident each one encodes, and its disposition for the Unified Execution Layer port (§7). Reuse as much as possible; supersede only where a newer certified learning demands it.

Why the file is big (honest accounting): ~16 execution scar-tissue mechanisms + the kernel/account plumbing that PINK's runtime also carries. The scar tissue is the part that ports; the plumbing stays behind the kernel seam. Almost nothing is dead weight.

Disposition legend: PORT = carry into prod/exec_unified/ warts-and-all · SEAM = kernel/account concern, stays above/below the exec layer (contract §2 forbids it here) · AMEND = port but update per a newer certified learning · SUPERSEDED = do not carry.

⛔ ZERO SILENT SUPPRESSION (operator ruling, 2026-07-14: "we paid dearly for pink")

Nothing here vanishes. A mechanism that is NOT ported as live code (SEAM, AMEND- deferred, or SUPERSEDED) must be carried into the exec-layer source as a referenced comment — the full reasoning plus a pink_direct.py:LNNN citation — at the seam where it would otherwise have lived. Every branch in PINK is a headstone bought with a real incident; the port preserves the lesson even when it relocates the code. A reviewer reading prod/exec_unified/drive_loop.py must be able to see, inline, why each PINK guard exists and where it went. Silent omission is the one unforgivable port error.


1. The drive loop's heart — _handle_expired_working (L1089–1226). PORT, verbatim logic.

The exact ordered sequence. Every step exists because skipping it caused a live incident. This is the transcription target; do not re-derive it.

  1. Already-resolved short-circuit (L1091): router.working(tid) is None → return. A fill/cancel notification raced the TTL sweep. Without it: double-resolution.
  2. Pump venue events FIRST (L1095), then re-check working (L1096). The quote may have filled between expiry and handling. Without it: cancel an already-filled order.
  3. Cancel the quote (L1109–1128): idempotent; CANCEL_REJECT on a filled order is harmless; on a partial entry this cancels the remainder. Injected KernelIntent CANCEL, same trade_id.
  4. Pump again (L1129) — the cancel round-trip may have surfaced a fill.
  5. Re-classify after the cancel (L1132–1141): _entry_filled() / _exit_done() → note_fill, pop working, emit fill_after_ttl, return. The fill-races-cancel window is real; this is the reclassification that catches it.
  6. EXIT never strands (L1147–1164): escalate to MARKET, same trade_id, sized from the slot (_exit_intent_from_slot). An exit that can't maker-fill MUST cross.
  7. ENTER miss policy (L1166): skip | retry(bounded) | market from config.
  8. Slot-busy guard (L1173–1179): size>0 or stage in _SLOT_OPENISH → skip. Never double-enter on a raced remainder fill.
  9. Venue-truth requote gate (L1185, _exec_safe_to_requote): re-quote ONLY when the venue is provably flat. Ambiguity → skip. "A skipped entry is always safe, a doubled position is not." (live double-entry, 2026-06-10.)
  10. Requote reference price (L1191–1205): from the working order's OWN asset history (multi-asset), fallback to plan.limit_price. Rebuild intent via replace(...).

2. Execution mechanisms — full table

# Mechanism (lines) Incident it encodes Disposition
1 "Every await followed by a state re-check" (L933-934 doctrine) fills race cancels on a live venue PORT — the governing invariant of the whole loop
2 _SLOT_OPENISH state set (L936) classify "position exists" across FSM states incl. PARTIAL/EXIT_* PORT (as typed predicate, not string tuple — V-TYPES)
3 _exec_after_submit — filled-now / resting / rejected-post-only → expired deadline (L975-1019) one shared resolution path: a rejected GTX registers already-expired so the TTL loop resolves it through the SAME miss/fallback path PORT — the "one shared path" is the gem; don't build two
4 _exec_safe_to_requote — fails SAFE (L1049-1068) recent own fill (<5 s) OR live venue position OR probe error → False AMEND — port; make the 5 s window a named, injected constant
5 _exec_ttl_loop — 1 s sweep (L1070) scan cadence (~10 s) too coarse for 5-8 s TTLs → dedicated task AMEND — port; inject cadence (spec: no hardcoded 1 s)
6 pump_venue_events — drain late fills, dedup, resolve working quotes vs applied truth (L1228-1311); CANCEL_ACK → pull deadline to now (route through shared path) resting LIMIT/partials arrive after submit returns; post-only reject surfaces via reconcile PORT (dedup/persist = SEAM; the working-quote resolution = PORT)
7 _last_own_fill_mono 5 s hot-window latch (L335, L648, L1055, L1323) REST reconcile lags WS fills by seconds → live double-entry AMEND — port; named constant, injected clock
8 _fill_is_ours ownership (L588-611): p- cid prefix → symbol membership → unattributable shared VST account (PRODGREEN/BLUE/manual); BingX WS does NOT echo clientOrderId PORT (concept) — maps to spec u-/m- clientOrderId + session symbol set
9 Single-slot ENTER block _unsafe_entry_reason (L1313-1356) filled maker entry vanishes from openOrders → misread as cancel → re-enter → 2× SEAM (single-slot is a kernel invariant) — but the venue-position-is-truth rule is PORT
10 _exit_intent_from_slot — size close from authoritative slot, cap to remaining (L1358-1380) malformed policy size must neither strand nor overshoot SEAM (kernel accounting) — exec layer receives already-sized size (contract §2)
11 Urgent-exit preempts working maker exit (L1660-1664) cancel the resting quote FIRST so MARKET close can't double-fill PORT — maps to PROTECT/CATASTROPHIC preempting a HARVEST/ROTATE working order
12 suppress duplicate guard (L1650-1659) a working maker quote already represents this action → don't double-submit PORT
13 Router failure → degrade to legacy taker (L971-973) router bug must never block trading PORT — fail-OPEN to taker is the safety default
14 _MIN_SANE_PRICE + fail-safe sizing refuse-the-OPEN (L81-85, L1337-1356) price→0 overflows size=notional/price to inf; corrupt input = untrustworthy signal, skip don't clamp SEAM (kernel sizing) — but the refuse-don't-clamp doctrine informs exec poison-input handling
15 Retry/market fallback plans, bounded (L1206-1226) miss policy: retry once then skip/market PORT — maps to urgency chase budgets (_constants.py)
16 Every transition self._emit("exec_router", …) (throughout) before-and-after observability PORT — but on the §12 side-lane, NEVER the exec path (b46ebd2)

3. Kernel / account plumbing — SEAM (does NOT enter the exec layer; contract §2 forbids it)

Capital seeding from exchange / K≈E reconcile / _enter_frozen (L443-528, L717-763); fee calibration loop + snapshot persist/restore (L129-166, L530-586); Hz writes (L906-929); alpha sizer feedback, IRP asset picker, DC gate, ACB boost (L826-905, L1461-1492, L1382-1459). All of this is brain + account. The Unified Execution Layer "knows NOTHING about the caller's world" (spec §2) — it never sees capital, leverage, posture, or picks an asset. These stay in the kernel seam / brain, above the ExecutionRequest.

4. Newer certified learnings that AMEND the port

  1. Maker fee = 2.00 bps MEASURED (Q1, today). L440 maker_rate: 0.0002 is CONFIRMED (0.02% = 2 bps), not superseded. Good — the assumption the file baked was right.
  2. ⚠ SIGN BUG candidate, L673: "fee": event.fee, # negative = rebate. Today's Q1 established BingX commission negative = DEBIT (a cost), not a rebate — the exact trap omp fell into. If bingx_user_stream passes the raw venue sign into event.fee, the K-fold here treats a cost as income → K optimistic vs E. Likely bounded (the K≈E reconcile gate would catch the drift and freeze; calibration uses abs() at L549) — but it is a real sign inconsistency worth verifying. NOT fixing now (BLUE/PINK live, untouchable); flagged for the operator. The exec-layer port must get the sign right at its own friction telemetry (§12).
  3. Hardcoded cadences (1 s TTL L1076; 5 s hot window L1055/L1323) → inject/name with provenance (spec NFR: cadence injectable, constants from data).
  4. spread_bps gate still dead (memory pink-exec-router open item; spec §4-12): the Router's spread gate was plumbed but never fed live top-of-book. The port WIRES it — this is new work, not a transcription.
  5. Urgency generalizes PINK's binary: PINK has entry/exit + miss(skip/retry/market). The spec's 5-class ladder is richer and subsumes it: PINK "exit always escalates to MARKET" = PROTECT/CATASTROPHIC crossing; PINK "entry miss, never chase" = ACQUIRE abandon. prod/exec_unified/router.py already encodes the richer ladder; the drive loop supplies PINK's proven mechanics underneath it.

4b. The SECOND axis — S-grades (size), orthogonal to the T-tier (spec §15.2)

PINK's drive loop is a single-clip (S0/S1) machine — it fires one order per action and never reasons about book footprint. That is exactly correct to port, but the port must carry the S-axis the spec added, because size that moves the book is a different pain class the T-tier does not cover:

Grade Definition Port implication
S0 size ≪ top-of-book depth (walk < 1 tick) any tier executes directly — PINK's loop as-is
S1 walks visible levels (expected walk ≥ spread) T2+ prices the walk into give-up/cross (feeds the HARVEST regression rule + placer offset)
S2 exceeds book capacity in the urgency window REFUSED WHOLE, loudly, below T14 — the pain-fence (NEW guard, no PINK equivalent)
S3 size IS the market MALKHUT/VIBRASS territory — out of scope for this port

Two rulings the port encodes:

  1. Slicing is COMPOSABLE and lives ABOVE the contract (T*.s sub-rung): TWAP/VWAP/ participation slicers are just callers — they decompose one parent into N child ExecutionRequests, each inheriting urgency + S0/S1. The layer cannot tell a slicer from a brain (agnosticism, §2). The exec layer gains only a parent_request_id journal field so §12 friction telemetry aggregates families. Ships as a separate sizer module — NOT inside exec_unified.
  2. In-order size mechanics are T14+ and NOT composable: iceberg, live-depth-adaptive child sizing, queue games, stealth. Each changes the working-order state machine and carries its own weird-bug class → they may not ride in as sub-rungs. The port stops at T13; T14 is a deliberate later escalation.
  3. The pain-fence (NEW, add to the port): below T14 the drive loop must REFUSE an S2+ request whole — loudly, with the computed expected-walk in the refusal — so nobody discovers the large-size pain class by accident in production. This is a RoutingDecision / pre-submit refusal path with no PINK ancestor; it is new certified doctrine, not scar tissue. contract.py needs an s_grade field (computed at intake per Appendix E) or a caller-supplied grade; the fence reads it.

5. Port mapping → prod/exec_unified/

  • router.py (done) — the whether-maker policy; §6 urgency ladder generalizes PINK miss policy.
  • placer.py (omp, in flight) — the where-in-book offset; wires the dead spread_bps gate.
  • drive_loop.py (next, mine) — transcribe §1 sequence + table rows #1-8,11-13,15-16. Injected seams (C1): clock, venue_io, market_state, working_registry — no ambient state, so it can run venue-side someday. The kernel/account SEAM rows stay behind an injected KernelPort — but per the ⛔ zero-suppression rule, each SEAM row also gets a referenced comment at its call site (e.g. above the KernelPort.settle call: the K≈E reconcile-freeze reasoning + pink_direct.py:717-763), so the lesson is visible inline. Add the S2+ pain-fence (§4b.3) as a pre-submit refusal with no PINK ancestor.
  • _constants.py — add REQUOTE_HOT_WINDOW_MS (=5000, prov: 2026-06-10 double-entry), keep cadence out (injected).
  • contract.py — add s_grade (S0-S3, spec §15.2) + parent_request_id (slicer family aggregation, §4b.1). Both are AMEND items already documented; wire when the fence lands.

Golden rule for the port (operator's ruling, restated): transcribe the mess, don't tidy it. Each branch above is a headstone. The only edits allowed are the AMEND rows — and every one of those cites a newer certified learning, not a preference.

6. Build items from the best-practice audit (EXEC_BEST_PRACTICE_AUDIT_20260714.md)

These are ADDITIONS the audit surfaced — not PINK scar tissue, so they carry their own tests. Fold into the port:

  • BI-1 (H4) — clientOrderId unique per ATTEMPT. Dialect mints u-<request_id>-<n>; a retry after INDETERMINATE gets a fresh venue id (exchanges reject duplicates) with parent linkage (FIX OrigClOrdID discipline). Add an attempt counter to the working order. Enforce BingX charset/length cap. contract.py gains client_order_id(attempt).
  • BI-2 (H5) — order-stream seqnum gap detection. WS is a delta; on a gap → forced REST resync; keep the 5 s own-fill hot-window as belt-and-braces, not the primary guard. (Lands in the venue-dialect §11, not the pure loop; referenced-comment the seam.)
  • BI-3 (M1) — log-don't-swallow. Every fail-safe guard keeps the guard, loses the silence: catch specific, log ≥debug with context. No bare except: pass in the port.
  • BI-4 (M2) — dead-man's-stop is a HARD gate. Attached STOP_MARKET must be mutation- tested for (a) auto-cancel on position close, (b) reduceOnly so a fire cannot open a reverse. Pre-live gate, not a TODO. ProtectiveSpec.reduce_only=True enforced (done).
  • BI-5 (H2) — durable atomic working-order state. Never /tmp; durable path, 0600, tmp+rename. (Venue/persistence seam; referenced-comment where state would persist.)