docs(uv): handover to successor integrator + SOA verdict + T8/T9 subspecs + spec C11 forensics

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

prod/docs/UV_MASTER_SPEC_20260702.md

@@ -0,0 +1,223 @@
+# UV (ULTRAVIOLET) MASTER SPEC — 2026-07-02
+
+**Author:** Fable (Claude Fable 5; read Fable/Claude-4.8 history as one unit).
+**Supersedes:** `~/.claude/plans/drifting-knitting-zebra.md` (2026-06-29, the approved C0–C9 plan)
+— this doc IS that plan, amended with verified 2026-07-02 findings. Commit into `/root/violet.git`
+main at T0.
+**Companions (unchanged, still binding):** `uv/UV_DEV_LOOP.md` (dev loop), `uv/specs/SPEC_00_*`
++ 11 hook specs + PASS4 WIRE specs, `prod/docs/UV_BLUE_PRIME_SHM_RESHAPE_SPEC.md`,
+`prod/docs/UV_BLUE_PRIME_ZINC_SHADOW_SPEC.md`, `prod/docs/AI_DEV_DOCTRINE.md`,
+`prod/docs/TESTING_DOCTRINE.md`.
+
+---
+
+## 1. Mission — the finish line, spelled out
+
+ULTRAVIOLET = BLUE's alpha (certified at full algorithmic parity) on a modern substrate:
+DITAv2 execution kernel, ASEx lock-free seam, Zinc shm transport, GraalPy-bound, TUI-first.
+BLUE-PRIME is the parity **oracle** (instrumented read-only copy of BLUE); UV is certified
+against PRIME; PRIME is certified against live BLUE by data diff — never by touching BLUE.
+
+**This spec is DONE only when (end-state acceptance, Wave 4 §9):**
+1. UV places and manages **real BingX VST testnet orders through DITAv2 as the exchange-exec
+   kernel, with the ASEx improvements underneath** (single-writer, lock-free, no async seams).
+2. Every UV trade's decision chain is traceable to the **certified-BLUE algorithm at Q=scan**
+   (journaled in `dolphin_uv.uv_decisions`; the parity differ stays green in shadow while
+   UV trades).
+3. SL/TP and protective market action execute **faster than eigenscan cadence** (rate-parity
+   stage 2 unlock) — demonstrably, with measured action latency in the journal.
+4. Soak-proven on VST: N days, zero orphan positions, reconciler-clean, DARK→armed ladder
+   respected at every step.
+5. Everything built **Graal-ready** (§10) — the Graal migration itself is a separate later
+   dev-ops spec, but no wave may ship code that violates the §10 constraints.
+
+"UV stood up with BLUE parity" (end of Wave 3) is the midpoint of this spec, not its end.
+
+## 2. Rate-parity doctrine (three stages; Q is a first-class dial)
+
+1. **Parity cert:** UV may run a fast internal clock but the decision surface is downsampled
+   through a **Q factor** to BLUE's NG7 eigenscan cadence (~6 s), so ALL timing-derived
+   affordances (bars_held, max_hold_bars, staleness windows, EFSM post-win windows, dedup)
+   match BLUE **warts and all**. Parity diffs join on **scan identity, never wall clock**.
+2. **Post-cert:** SL/TP + market action execute FASTER than eigenscan (DITAv2/ASEx);
+   alpha decisions stay scan-quantized. This is the economic point of UV.
+3. **Far:** Graal substrate; the Q layer itself loosened **gradually**, parity-guarded
+   (certified-Q config stays runnable as the rollback baseline). Hard ceiling today: NG7
+   eigenscan generation compute. Operator caution (doctrine): the 6 s @ 15 m correlation
+   rhythm may itself carry alpha — loosening Q is an experiment, never an assumption.
+
+**Engineering rule:** the Q-quantizer is ONE explicit component with an interface (human-defined
+boundary per AI_DEV_DOCTRINE #8), never cadence assumptions scattered through modules.
+
+## 3. Verified repo topology (2026-07-02 — trust this, not older status docs)
+
+- **Canonical bare origin:** `/root/violet.git`. `main` @ `15fb189` = 11 hooks + WIRE.1/2/3 +
+  WIRE.5 journal + pi's comprehensive hook tests.
+- **Unmerged, to integrate at T0:** `uv/wire-4-live-inputs` @ `2fd1f9d` (live_inputs + EFSM
+  mirror); `docs/uv-blue-prime-shm-reshape` @ `ee4e5eb` (snapshot contract + zinc_shadow +
+  1078 tests; branched from `ba7e12e`, so it lacks WIRE.4/5).
+- **Divergent clone line:** `/root/uv-wt/uv` + `/root/uv-wt/blue-prime` @ `1902e4b` (5 unpushed
+  commits forked at `5d583e4`): TUI v2, EFSM CH-mirror fix `6156e17`, real /dev/shm zinc region
+  writer `01e35c4`. Salvage-review at T0. **The live soak (zellij `UV_BLUE-PRIME_TUI`) runs this
+  line** — real ZINC_REG region `/dev/shm/zinc_uv_shadow_state`, seq advancing at scan cadence.
+- **RETIRED as wrong:** `/root/uv-wt/uv/prod/docs/UV_DEV_CURRENT_STATUS_2026-07-01.md`'s claim
+  that `/root/uv-wt/uv` is canonical/"ahead" — it is a fork missing hooks/WIRE/reshape. Its
+  UV-core / PRIME-oracle / VIOLET-substrate *conceptual* distinction remains correct and adopted.
+- Dev loop stays as `UV_DEV_LOOP.md`: local disk only, clone-per-chunk off the bare, branch
+  `uv/<name>`, push → scoped CI, **Fable integrates to main**.
+
+## 4. Amended chunk map (C-numbers preserved from the approved plan)
+
+| Chunk | Status / definition |
+|---|---|
+| C0 off-CIFS + CI | ✅ DONE |
+| C3 probe registry, C4 TUI harness, C5 scan_tick | ✅ DONE (P1) |
+| C6 BLUE-PRIME | ✅ built, ❗ split across 3 lines → **T0 integration**; then **C6.5 (NEW): pollution neutralization** → T1 |
+| **C6.5 pollution guard (NEW)** | PRIME must be provably write-free toward BLUE's namespaces (CH `dolphin.*`, HZ, shared spool). See `UV_TASK_T1`. |
+| C7 parity instrument (**REDEFINED**) | Not an entry_signal probe. C7a = journal hardening + differ core (T2). C7b = differ tests + TUI parity panel (T3). C7c = certification reporter (T4). |
+| C8 BingX reconciler | Wave 2, SHRUNK: verify/adapt DITAv2's existing reconcile logic under UV's `u-` prefix + rate-budget tests — not a new build (§8). |
+| C10 (NEW) DITAv2 exec seam | Wave 2: certified-PRIME decision path promoted to active + KernelIntent → DITAv2 → VST, v4-runner pattern as template (§8). |
+| C11 (NEW) SL/TP fast clock | Wave 2, post-canary unlock (§8). **Empirical justification (2026-07-02 forensics):** scan-cadence stop overshoot is real — FET −2,433 exited at 1.61% adverse vs 1.2% stop (+34% overshoot), WIN −1,920 at 1.32% (+10%); ≈$800/wk excess on 3 stops alone. **Millisecond forensics on the FET stop (dolphin.obf_universe @130ms):** price approached the stop visibly for 19s (0.87%→1.20%), breached at 16:08:15.9, next eigenscan not until 16:08:19.6 (measured scan gaps 11–12s, not 6s) → entire overshoot lived inside one scan gap; exec latency after the scan was <1s (mechanism healthy). Book stayed orderly throughout (spread pinned 5.4bps, top-5 depth $140–200K both-sides) → breach-time exit was executable; like-for-like save ≈$500–580 on this trade. Faster-than-scan SL action recovers this by construction. NOTE: `dolphin.obf_fast_intrade` (in-trade fast OB feed) has 0 rows — dead wiring in BLUE; UV's fast clock must not repeat this (a populated fast feed is the C11 sensor). |
+| C1 UV Zinc plane + Q-dial, C2 GraalPy smoke, C9 DITAv2 probe | **DEFERRED** post-first-trade (§9). Graal-readiness is enforced NOW via §10 instead. |
+| UV core runtime (separate rewrite) | **CANCELLED as churn** — UV v1 IS the certified PRIME path promoted (§8). `uv/blue_prime/` freezes as oracle at certification. |
+
+## 5. Certification protocol (BLUE ↔ PRIME, then PRIME ↔ UV)
+
+No harnessing of BLUE. PRIME runs read-only beside live BLUE; certification is a
+**tick-aligned data diff**:
+- PRIME's per-scan record: `dolphin_uv.prime_decisions` (WIRE.5 journal: inputs, 11 hook
+  effects, decision) + zinc snapshot.
+- BLUE's record: `dolphin.trade_events` (+ logs) — entries/exits with asset/side/leverage.
+- Continuous signal: per-scan hook/leverage modulation self-consistency; hard gate: **entry
+  events**, matched on scan identity, **bit-identity** (PASS2.5 standard: mismatch = bug,
+  not tolerance).
+- **Gate (restructured 2026-07-02, operator: compute-bound not calendar-bound):**
+  - **Gate A — replay-cert (gating):** replay BLUE's RECORDED input history (months of
+    eigen-scans, all realized trades) through PRIME's decision path, Q-quantized
+    (scan-sequenced, warts and all), vectorized where the kernel allows. Bit-diff every
+    decision vs `dolphin.trade_events` — every entry BLUE ever made, not 3. 0 unexplained
+    diffs. (Synthetic inputs CANNOT gate BLUE-parity — BLUE has no recorded answer for them.)
+  - **Gate B — faultline assault (gating):** Hypothesis/adversarial/fuzz on breakspots
+    (vel_div threshold boundary, EFSM transitions incl. post-win LONG overlay, staleness,
+    poison) — properties: PRIME-internal consistency + PRIME↔UV agreement.
+  - **Gate C — live plumbing (non-gating):** guarded soak keeps running; live HZ reads,
+    scan gaps, mirror hydration clean over X hours; any live entry = bonus bit-check.
+  - Pollution invariant green throughout; operator signs the cert. Then PRIME freezes as
+    oracle; UV certifies against PRIME with the same instruments (Gate A replay + Gate B),
+    writing `dolphin_uv.uv_decisions`.
+  - **Wave-2+ note (operator):** multi-instance PRIME-[n] clone farms (forkd / workdir.dev
+    -class system forks; VIBRASS bandit meta-gov) — design-in now: per-instance CH namespace
+    `dolphin_uv_{n}`, per-instance zinc prefix `uv_shadow_{n}`, injectable clock/Q.
+
+## 6. Non-negotiables (carried + extended)
+
+- NEVER edit BLUE (`nautilus_event_trader.py`, kernels, `prod/ch_writer.py` [vendored/shared],
+  supervisord, HZ contents, `dolphin.*` tables). PRIME reads BLUE's world; writes NOTHING into it.
+- `dolphin_uv.*` is UV/PRIME's only CH namespace. Hard-guarded in code + tests
+  (journal URL guard raises on non-dolphin_uv; T1 no_write_guard diverts every other CH write
+  to a local audit file; T4 reporter is zero-CH-write; HZ is wrapped read-only).
+- **Separate-install direction (operator, 2026-07-02):** before UV's first VST trade (C10),
+  UV's own writes move to a DEDICATED CH instance (own port/datadir or container) — namespace
+  isolation is the guard, instance isolation is the wall. UV needs NO HZ writes in wave 1
+  (reads BLUE's HZ read-only); if UV ever needs its own KV plane, it gets its own instance —
+  never keys in BLUE's cluster.
+- VST only; `ALLOW_MAINNET=0`; DARK until operator arms.
+- Testing doctrine: mutation litmus, poison/edges/concurrency, no green-by-pollution,
+  run your own suite before push. AI_DEV doctrine: one problem/one branch/one PR,
+  explicit staging, docs > chat.
+- Vendored-drift gate: VENDOR.lock components edited only upstream + `vendor_sync.sh`.
+
+## 7. Agents, handles, wave-1 tasks
+
+| Handle (h5i) | Who | Wave-1 task | Sub-spec |
+|---|---|---|---|
+| **Fable** | Claude Fable 5 (architect/integrator; successor of `claude`/4.8 + `cc-ultrav-1`) | **T0**: git integration pass (main + wire-4 + reshape + clone salvage → main); commit this spec; correct/retire stale status doc | this doc §3–4 |
+| **codex** | Codex 5.4mini | **T1 (CRITICAL)**: pollution forensics + neutralization + PRIME relaunch | `UV_TASK_T1_CODEX_POLLUTION_GUARD.md` |
+| **mm_VIOLET1** | mimocode (slow, precise) | **T2**: journal hardening + C7a differ core | `UV_TASK_T2_MM_JOURNAL_DIFFER.md` |
+| **pi_nvnemo** | PI harness (Nemotron) | **T3**: C7b differ test suite + TUI parity panel | `UV_TASK_T3_PI_PARITY_TESTS_TUI.md` |
+| **cmd-PASS1.1** | Command Code / DeepSeek (operator fires instance) | **T4**: C7c certification reporter | `UV_TASK_T4_CMD_CERT_REPORTER.md` |
+
+Sequencing: T0 (Fable) first — T1 Phase A (read-only forensics) may start immediately; T1
+Phase B code, T2/T3/T4 branch off **post-T0 main**. All reporting on the canonical h5i bus
+(`/mnt/dolphinng5_predict`); reply to **Fable**. Sub-specs live in
+`prod/docs/uv_subspecs/` (committed to the violet repo at T0).
+
+## 8. Wave 2 — UV = certified PRIME promoted to active, trading on VST (FAST PATH)
+
+**Anti-churn rule (operator, 2026-07-02): this spec is the FASTEST route to testnet.**
+No rewrites of things that already work. Concretely:
+
+- **NO new UV decision core.** PRIME already runs BLUE's real engine + 11 hooks + EFSM mirror
+  + live inputs. The day PRIME certifies, **UV v1 = the certified PRIME decision path promoted
+  from shadow to active** (new process/config, journals `dolphin_uv.uv_decisions`, DARK) +
+  a KernelIntent emitter. Zero algorithm code rewritten between certification and first trade.
+- **NO new exec layer.** DITAv2 is already VST-proven (PINK burn-in; VIOLET v4 runner
+  precedent — reuse its KernelIntent→DITAv2 wiring pattern as a NEW instance; never edit the
+  live v4 runner). DITAv2 stays vendored (VENDOR.lock; improvements upstream + vendor_sync).
+  ASEx improvements ride along inside DITAv2's existing integration — the single-writer /
+  no-async-seam guarantees are why it's the kernel; we do not re-plumb them.
+- **NO new reconciler from scratch.** Adopt DITAv2's existing reconcile logic (PINK ownership-
+  filter lineage) under UV's own VST account/clientOrderId prefix (`u-`); C8 shrinks to
+  "verify + adapt + rate-budget test", not "build".
+- **Arming ladder (operator-gated per rung):** DARK (journal-only) → observe-only → VST canary
+  (min size, single slot) → staged size. `ALLOW_MAINNET=0` throughout; mainnet is out of scope.
+- **Decision-vs-execution parity split:** decision layer must bit-match certified-BLUE at
+  Q=scan (differ green in shadow while UV trades); execution quality is judged by
+  reconciler-clean + DITAv2 accounting integrity (fill-price PnL doctrine), since BLUE's
+  "fills" are in-memory bookkeeping and UV's are real VST mechanics.
+- **C11 SL/TP fast clock** (rate-parity stage 2 — protective actions faster than eigenscan,
+  alpha stays at Q=scan): unlocked AFTER first clean canary trades, not before.
+- **Exit = §1 end-state acceptance.** That is the whole spec.
+
+## 9. Deferred (post-first-trade; separate specs — do NOT build in this spec's waves)
+
+- C1 UV-own Zinc plane + generalized Q-dial (UV v1 runs at scan cadence natively — Q=1:1 —
+  so the dial abstraction earns nothing until stage-3 loosening).
+- C2 GraalPy smoke rig; C9 DITAv2 innards probe; TUI beyond the existing panels.
+- Wave-Graal: the GraalPy/GraalVM/Graal-OS migration dev-ops pass (own spec when VST soak
+  is running). Q-loosening experiments (rate-parity stage 3) live there or after.
+
+### 9.1 Sketchlog lane (operator-flagged CRITICAL, 2026-07-02)
+Source: `prod/docs/VIOLET_TODO_CRITICAL_DISTRIBUTION_TRACKING_IN_CONSTRAINED_MEMORY.md`
+(9 signals mapped to BIBLE integration points; sketchlog = DDSketch/HLL/CMS/DriftSketch,
+93 KB constant memory, mergeable monoids, WindowedStreamLog realtime windows, optional C++).
+- **Now (observability, zero parity risk):** sketch dimensions (vel_div percentiles,
+  signal breadth HLL, reversal freq) added to PRIME snapshot + journal as OBSERVE-ONLY
+  columns; T7 replay computes them over full history = instant candidate-feature backtest.
+- **Post-cert (alpha, gated):** signals as decision inputs (esp. #3 rolling-MAE tail
+  detector → adaptive exits = the left-tail killer; #1 widening; #4 breadth; #6 reversal)
+  — UV-divergence features via the certified-Q baseline + diff-guarded rollout. NEVER BLUE.
+- Merge algebra fits PRIME-[n] farms (coordination-free merge); pure-Python path = §10 G1 ok.
+
+### 9.2 Control plane (operator directive 2026-07-02: "NATS/iceoryx2 the hell out of it")
+One control plane over the WHOLE system: fleet lifecycle (start/stop/arm PRIME-[n]/UV/soaks),
+config + Q-dial distribution, heartbeats, gate-ledger events, kill-switch propagation.
+- **Split doctrine:** data plane intra-box = Zinc/iceoryx2 (ADR-1, adopted); CONTROL plane
+  inter-process/inter-box = message bus. Candidate: NATS (operator-named). NOTE: ADR-2
+  reserved Zenoh for inter-box DATA — NATS-for-control vs Zenoh-for-data can coexist;
+  Fable authors the control-plane ADR when wave 2 opens.
+- NOT on the critical path to first testnet trade; REQUIRED before the clone farm.
+- Design-in now (already true): every long-running process publishes a zinc snapshot and
+  takes env-injected config — those are the surfaces the control plane will drive.
+
+## 10. Graal-readiness — build constraints binding NOW despite deferred migration (NFR-G)
+
+Cheap guardrails (mostly "don't do X"), enforced in review — so the later migration is a
+runtime swap, not a rewrite:
+
+- **G1** No CPython-only C-extensions in UV hot paths (pure-Python, or Rust behind stable FFI).
+- **G2** No `__del__`/refcount-timing for correctness — explicit lifecycle (ASEx Drop-reliance
+  leak = the cautionary tale).
+- **G3** No "GIL makes this safe" — cross-thread state only via single-writer/Zinc/ASEx seams.
+- **G4** UV code never imports `hazelcast` directly — HZ quarantined behind the existing
+  reader seams (HZBridge direction).
+- **G5** No hardcoded paths/creds; env-injected config.
+- **G6** Long-running RSS-stable, jemalloc-compatible processes.
+
+## 11. T6 (ACTIVE, HIGH PRIORITY — promoted 2026-07-02): real-Zinc unification
+
+The clone line (banked as branch `salvage/uv-clone-line-1902e4b`) carries the REAL mmap Zinc
+region transport (`ZincShadowChannel`, prefix `uv_shadow`, 18 h live soak) + the Textual
+`tui_v2`. The merged main publishes the same snapshot via atomic file (cross-process, works
+today) + in-memory zinc. T6 = port `ZincShadowChannel` under the reshape snapshot contract as
+the transport, re-home `tui_v2` on it. Small, spec to follow; does NOT gate T1-D relaunch.