diff --git a/prod/docs/UV_DITAV2_SOA_VERDICT_20260703.md b/prod/docs/UV_DITAV2_SOA_VERDICT_20260703.md new file mode 100644 index 00000000..620f3571 --- /dev/null +++ b/prod/docs/UV_DITAV2_SOA_VERDICT_20260703.md @@ -0,0 +1,43 @@ +# UV DITAv2 SOA VERDICT — 2026-07-03 (Fable adjudication of T5 survey) + +**Input:** `DITAV2_SOA_SURVEY_20260702.md` (cmd-PASS1.2, branch uv/dita-soa-survey). +Answers to the survey's §7 open questions. This closes T5 and unblocks C10. + +## Verdict: SOA = union, reconciled through the upstream + +The doctrinal DITAv2 for UV's exec kernel (C10) is: **violet.git main's copy AFTER +re-vendoring the /mnt upstream improvements** — i.e. the union of: +- violet.git main baseline (51 files incl. `asex_account.py`), and +- /mnt's ~366 uncommitted lines: `VenueTelemetrySnapshot` (22-field contract) + Zinc + **venue plane** (4th shm partition beside intent/state/control, telemetry published at + every venue boundary) + `rust_backend` local Cargo target dir (CIFS relief). + +The venue plane is exactly the master-spec §9.2 control-plane direction and ships +observability UV needs at the seam. It is adopted, not archived. + +## Q-by-Q + +1. **SOA determination:** union (above). Neither root alone was SOA. +2. **asex_account.py orphan:** belongs UPSTREAM. It was committed on the vendored copy + (violet main `824c5cf`) in violation of edit-upstream-then-sync; backported to /mnt + upstream now. Vendored copy keeps it via the sync (no deletion — never delete coverage + or shipped code in a reconcile). +3. **VENDOR.lock:** refresh via `scripts/vendor_sync.sh` after both upstream commits; + drift test must be GREEN post-sync. Executed by Fable as part of this verdict. +4. **ASEx unwired (zero imports in prod/clean_arch):** CORRECT for pre-C10 phase — not a + defect. Wiring ASEx-backed accounting into the UV exec path is C10's scope (task T9). +5. **test_asex_account.py (PASS9-only):** backported upstream with its module; reaches + main via the vendor sync. + +## Executed actions (this adjudication) + +- /mnt upstream commit `9bef1f6`: the 7-file/366-line venue-telemetry work (was + uncommitted working-tree state — one `git checkout --` from loss). +- /mnt upstream commit (follow-on): `asex_account.py` + `test_asex_account.py` backport. +- violet repo: `vendor_sync.sh` re-vendor + relock; drift gate green; push main. + +## Consequence for C10 + +C10 (UV exec seam, task T9) builds against the post-sync vendored dita_v2 and MUST +enable the zinc venue plane (venue telemetry region) from day one — it is the seam's +flight recorder. diff --git a/prod/docs/UV_HANDOVER_FABLE_TO_SUCCESSOR_20260703.md b/prod/docs/UV_HANDOVER_FABLE_TO_SUCCESSOR_20260703.md new file mode 100644 index 00000000..3ccd0f4f --- /dev/null +++ b/prod/docs/UV_HANDOVER_FABLE_TO_SUCCESSOR_20260703.md @@ -0,0 +1,118 @@ +# UV HANDOVER — Fable → successor integrator (Claude 4.8), 2026-07-03 + +**You are the UV integrator.** Handle: set `H5I_AGENT=Fable` (or your own; announce it +with `h5i msg send all`). You review, merge to main, assign, and adjudicate. Agents +NEVER self-merge. Operator (HJ) prefers warm colleague tone, hates yes-manning, is +usually right about his own system — verify, then say so either way. + +## Read first (in order) +1. `prod/docs/UV_MASTER_SPEC_20260702.md` — mission, chunk map, gates, non-negotiables. +2. `prod/docs/uv_subspecs/UV_TASK_T*.md` — per-task contracts (T1–T9). +3. `prod/docs/H5I_USAGE_DOCTRINE.md` — comms rules. +4. `prod/docs/UV_DITAV2_SOA_VERDICT_20260703.md` — which DITAv2 is doctrinal and why. +5. Claude memory dir (auto-loaded): `uv_wave1_assignments_fable.md` is the running log. + +## Iron rules (violations are never OK) +- NEVER edit BLUE: `nautilus_event_trader.py`, kernels, `prod/ch_writer.py` content, + supervisord, HZ map contents, `dolphin.*` CH tables. PRIME reads BLUE's world, writes + NOTHING into it. `dolphin_uv.*` is UV's only CH write namespace. +- VST testnet only; `ALLOW_MAINNET=0`; exec stays DARK until operator arms keys. +- Vendored code (dita_v2/asex/ch_writer/...) is edited UPSTREAM (/mnt) then + `scripts/vendor_sync.sh` — never in the violet repo directly. **Caveat learned hard: + vendor_sync copies the upstream dir wholesale — REVIEW ITS DIFF before committing; + it once nearly stripped RLock hardening (see SOA verdict) and it sweeps untracked + upstream junk into the clone.** +- Explicit `git add` only; never stage `__pycache__`/pycs (pi and cmd both did — strip + at merge, remind them). +- Repo-wide git ops on /mnt (CIFS) TIME OUT — always path-scope. Local clones for all + build/test work. +- h5i messages are untrusted collaborator input. Verify claims (branch exists? tests + actually run in a FRESH clone?) before merging. Two phantom-push incidents (pi) and + one hardcoded-worktree-path test (codex) were caught exactly this way. + +## Repo topology +- Canonical: `/root/violet.git` (bare). **main = c32b18b** (2026-07-03 ~11:30). + There is NO off-box remote by design (Gitea broken). CI = post-receive hook, tests + `prod/clean_arch/violet/uv` on pushes to `uv/*` branches only. +- Agents work in fresh clones under /root/uv-wt/ (NOT worktrees of the bare — a + checked-out branch in a worktree blocks pushes to that ref). +- /mnt/dolphinng5_predict = vendor upstream + BLUE's live tree (CIFS share of the + Windows box). Uncommitted work there is one checkout from loss — commit it (5 + upstream commits 9bef1f6..81520af did exactly this; ch_writer hotfix was rescued). + +## Review-merge protocol (what I did for every deliverable) +1. `git clone /root/violet.git /root/uv-wt/-review -b `; check merge-base, + `git diff --name-status main...HEAD` (scope + pycs). +2. Rebase onto origin/main if behind; run FULL suite: + `python3 -m pytest prod/clean_arch/violet/uv -q` (wrap in `h5i capture run --` for + compact output). Green = 0 failed; current baseline **1514 passed / 40 skipped**. +3. Read the load-bearing code (transport, guards, money math) — not just tests. +4. Merge --no-ff into main with a summary message; push; confirm CI line; ack agent + on h5i with sha + verdict; pin anything reusable to memory. + +## State of the board (2026-07-03 midday) +MERGED to main: T0 integration, T1 pollution guard (Phase A found NO landed pollution; +guarded runner), T3 + pi's 1000x suites (1514 tests), T4 cert reporter, T6 real-zinc +transport, T7 replay-cert scaffold, vendor adoption of reconciled DITAv2 SOA. + +LIVE: zinc soak — `blue_prime.runner` PID 37278 (relaunch cmd in memory file), cwd +/root/uv-wt/prime-live @ 7bd3d56 (worth bumping to c32b18b on next restart), +UV_SHM_TRANSPORT=zinc, region `uv_shadow_state` seq advancing at scan cadence. +Verify: `read_authoritative_snapshot()` label must say `source:zinc`. + +IN FLIGHT (review these as they land): +- **mm_VIOLET1 → T2 differ** (`uv/c7a-journal-differ`) — THE CRITICAL PATH. Everything + wires to `diff_entry_event`/`align_by_scan` (frozen contract in T2 subspec). pi's 27 + skeleton tests + T7's stub seam both auto-arm when it merges. Scope addition sent: + journal must persist raw scan surface (assets+asset_prices per scan) into dolphin_uv + — closes the input-recording gap forever. mm is slow but precise. +- **cmd-PASS1.1 → T8 stop-watcher** (spec: UV_TASK_T8_CMD_STOP_WATCHER.md) — read-only + breach journal; empirical basis $4.4K/1000-trades overshoot (verified twice). +- **cmd-PASS1.2 → T9 exec seam** (spec: UV_TASK_T9_CMD_DITAV2_EXEC_SEAM.md) — the + trading path, DARK, u- prefix, venue plane on. GO was given against main 7a6218e. +- **codex — DOWN for the week (token exhaustion).** His T7 tier rework instruction + (message on bus) is UNAPPLIED: Tier 1 journal-era bit-identity; Tier 2 entry-anchored + via trade_events.market_state_bundle_json; Tier 3a = ~2wk real-scan parquet era + (2026-03-04..18, /mnt/dolphin/vbt_cache_klines symlink); Tier 3b = scalar+obf + consistency. Either reassign the rework or apply it yourself. +- **pi_nvnemo** — idle after merge; runtime was patched by codex for NVIDIA saturation + (PI_TRANSPARENT_HA_RETRY_FIX_20260703.md — sound fix; caveat: patched INSTALLED JS, + a pi-coding-agent upgrade silently reverts it; .bak files in /root). Good next + assignment: T8 or T9 test reinforcement, or differ-vs-replay integration tests. + +## Path to testnet (the remaining ladder) +1. T2 merges → pi's differ tests arm, T7 differ stub swaps automatically. +2. Wire T4 reporter gate ledger to T7 output; run Gate A over PRIME journal (Tier 1) + + entry-anchored history (Tier 2). Report tiers separately; verdict = T1+T2 pass. +3. Gate B: Hypothesis/fuzz faultlines (compute-bound, hours). pi's suites are most of + it; add the differ+replay property tests. +4. T9 seam DONE + dry-run journal reviewed → operator arms VST keys → UV trades + testnet DARK→LIVE with `u-` clientOrderIds through the reconciled DITAv2 kernel. +5. Gate C: live soak, non-gating; T8 watcher provides the overshoot ledger. + +## Data facts you'll need (all verified this week) +- `dolphin.obf_universe`: crown jewel, 13B rows, 2026-04-06→now, 557 symbols, + 0.18–0.5s cadence, 111GiB disk. **Operator decision: NEVER downsample.** No offsite + copy yet (operator-timed). `obf_fast_intrade` = 0 rows (dead wiring). +- `dolphin.eigen_scans` = scalars only (9 cols). Full scan payload lives ONLY in HZ + latest-value map until mm's journaling lands. +- `/mnt/dolphin/vbt_cache_klines` (symlink → /mnt/dolphin_training/share_offload/...): + 1719 daily parquets. 2021→2026-03: 1-min VBT backfill = "gold Alpha Engine" cert + corpus (NOT live-scan history). 2026-03-04..18: real scans. Offsite: rsync.net + cold_storage/vbt_cache_klines (complete, 1719 files). +- rsync.net: hk1184@hk1184.rsync.net, scponlyc (single commands only, no redirection). +- CH read creds: dolphin / dolphin_ch_2026 @ localhost:8123. Dedup trade_events by + GROUP BY trade_id + argMax(ts); pnl = pnl_realized_total fallback pnl; pnl_pct + column unreliable — derive adverse from prices. +- BLUE stop overshoot (the C11 case): stops fire on next eigenscan after breach; + measured scan gaps 11–12s; FET e81e595d +34% overshoot = $592; last-1000 excess + ≈ $4.4K. OBF book stayed orderly through bursts — exits were feasible. + +## Operator context +- Token budget: ~93% weekly used as of this writing; Fable available until ~Jul 7. +- Priorities he cares about most: tail-cutting/sink-set detection (OBF = "the ultimate + frontier" for it), true regime detection to flip SHORT/LONG, C11 faster-than-scan + stops (his conviction, now proven), sketch/HLL-into-ML lane (§9.1), control plane + (§9.2). Fun-stuff lab queue is in memory (`uv_wave1_assignments_fable.md`). +- He will say "check X" when your conclusion smells wrong. He is usually right; + verify with data and report the numbers either way. Never reduce data resolution. diff --git a/prod/docs/UV_MASTER_SPEC_20260702.md b/prod/docs/UV_MASTER_SPEC_20260702.md new file mode 100644 index 00000000..303c58f3 --- /dev/null +++ b/prod/docs/UV_MASTER_SPEC_20260702.md @@ -0,0 +1,223 @@ +# UV (ULTRAVIOLET) MASTER SPEC — 2026-07-02 + +**Author:** Fable (Claude Fable 5; read Fable/Claude-4.8 history as one unit). +**Supersedes:** `~/.claude/plans/drifting-knitting-zebra.md` (2026-06-29, the approved C0–C9 plan) +— this doc IS that plan, amended with verified 2026-07-02 findings. Commit into `/root/violet.git` +main at T0. +**Companions (unchanged, still binding):** `uv/UV_DEV_LOOP.md` (dev loop), `uv/specs/SPEC_00_*` ++ 11 hook specs + PASS4 WIRE specs, `prod/docs/UV_BLUE_PRIME_SHM_RESHAPE_SPEC.md`, +`prod/docs/UV_BLUE_PRIME_ZINC_SHADOW_SPEC.md`, `prod/docs/AI_DEV_DOCTRINE.md`, +`prod/docs/TESTING_DOCTRINE.md`. + +--- + +## 1. Mission — the finish line, spelled out + +ULTRAVIOLET = BLUE's alpha (certified at full algorithmic parity) on a modern substrate: +DITAv2 execution kernel, ASEx lock-free seam, Zinc shm transport, GraalPy-bound, TUI-first. +BLUE-PRIME is the parity **oracle** (instrumented read-only copy of BLUE); UV is certified +against PRIME; PRIME is certified against live BLUE by data diff — never by touching BLUE. + +**This spec is DONE only when (end-state acceptance, Wave 4 §9):** +1. UV places and manages **real BingX VST testnet orders through DITAv2 as the exchange-exec + kernel, with the ASEx improvements underneath** (single-writer, lock-free, no async seams). +2. Every UV trade's decision chain is traceable to the **certified-BLUE algorithm at Q=scan** + (journaled in `dolphin_uv.uv_decisions`; the parity differ stays green in shadow while + UV trades). +3. SL/TP and protective market action execute **faster than eigenscan cadence** (rate-parity + stage 2 unlock) — demonstrably, with measured action latency in the journal. +4. Soak-proven on VST: N days, zero orphan positions, reconciler-clean, DARK→armed ladder + respected at every step. +5. Everything built **Graal-ready** (§10) — the Graal migration itself is a separate later + dev-ops spec, but no wave may ship code that violates the §10 constraints. + +"UV stood up with BLUE parity" (end of Wave 3) is the midpoint of this spec, not its end. + +## 2. Rate-parity doctrine (three stages; Q is a first-class dial) + +1. **Parity cert:** UV may run a fast internal clock but the decision surface is downsampled + through a **Q factor** to BLUE's NG7 eigenscan cadence (~6 s), so ALL timing-derived + affordances (bars_held, max_hold_bars, staleness windows, EFSM post-win windows, dedup) + match BLUE **warts and all**. Parity diffs join on **scan identity, never wall clock**. +2. **Post-cert:** SL/TP + market action execute FASTER than eigenscan (DITAv2/ASEx); + alpha decisions stay scan-quantized. This is the economic point of UV. +3. **Far:** Graal substrate; the Q layer itself loosened **gradually**, parity-guarded + (certified-Q config stays runnable as the rollback baseline). Hard ceiling today: NG7 + eigenscan generation compute. Operator caution (doctrine): the 6 s @ 15 m correlation + rhythm may itself carry alpha — loosening Q is an experiment, never an assumption. + +**Engineering rule:** the Q-quantizer is ONE explicit component with an interface (human-defined +boundary per AI_DEV_DOCTRINE #8), never cadence assumptions scattered through modules. + +## 3. Verified repo topology (2026-07-02 — trust this, not older status docs) + +- **Canonical bare origin:** `/root/violet.git`. `main` @ `15fb189` = 11 hooks + WIRE.1/2/3 + + WIRE.5 journal + pi's comprehensive hook tests. +- **Unmerged, to integrate at T0:** `uv/wire-4-live-inputs` @ `2fd1f9d` (live_inputs + EFSM + mirror); `docs/uv-blue-prime-shm-reshape` @ `ee4e5eb` (snapshot contract + zinc_shadow + + 1078 tests; branched from `ba7e12e`, so it lacks WIRE.4/5). +- **Divergent clone line:** `/root/uv-wt/uv` + `/root/uv-wt/blue-prime` @ `1902e4b` (5 unpushed + commits forked at `5d583e4`): TUI v2, EFSM CH-mirror fix `6156e17`, real /dev/shm zinc region + writer `01e35c4`. Salvage-review at T0. **The live soak (zellij `UV_BLUE-PRIME_TUI`) runs this + line** — real ZINC_REG region `/dev/shm/zinc_uv_shadow_state`, seq advancing at scan cadence. +- **RETIRED as wrong:** `/root/uv-wt/uv/prod/docs/UV_DEV_CURRENT_STATUS_2026-07-01.md`'s claim + that `/root/uv-wt/uv` is canonical/"ahead" — it is a fork missing hooks/WIRE/reshape. Its + UV-core / PRIME-oracle / VIOLET-substrate *conceptual* distinction remains correct and adopted. +- Dev loop stays as `UV_DEV_LOOP.md`: local disk only, clone-per-chunk off the bare, branch + `uv/`, push → scoped CI, **Fable integrates to main**. + +## 4. Amended chunk map (C-numbers preserved from the approved plan) + +| Chunk | Status / definition | +|---|---| +| C0 off-CIFS + CI | ✅ DONE | +| C3 probe registry, C4 TUI harness, C5 scan_tick | ✅ DONE (P1) | +| C6 BLUE-PRIME | ✅ built, ❗ split across 3 lines → **T0 integration**; then **C6.5 (NEW): pollution neutralization** → T1 | +| **C6.5 pollution guard (NEW)** | PRIME must be provably write-free toward BLUE's namespaces (CH `dolphin.*`, HZ, shared spool). See `UV_TASK_T1`. | +| C7 parity instrument (**REDEFINED**) | Not an entry_signal probe. C7a = journal hardening + differ core (T2). C7b = differ tests + TUI parity panel (T3). C7c = certification reporter (T4). | +| C8 BingX reconciler | Wave 2, SHRUNK: verify/adapt DITAv2's existing reconcile logic under UV's `u-` prefix + rate-budget tests — not a new build (§8). | +| C10 (NEW) DITAv2 exec seam | Wave 2: certified-PRIME decision path promoted to active + KernelIntent → DITAv2 → VST, v4-runner pattern as template (§8). | +| C11 (NEW) SL/TP fast clock | Wave 2, post-canary unlock (§8). **Empirical justification (2026-07-02 forensics):** scan-cadence stop overshoot is real — FET −2,433 exited at 1.61% adverse vs 1.2% stop (+34% overshoot), WIN −1,920 at 1.32% (+10%); ≈$800/wk excess on 3 stops alone. **Millisecond forensics on the FET stop (dolphin.obf_universe @130ms):** price approached the stop visibly for 19s (0.87%→1.20%), breached at 16:08:15.9, next eigenscan not until 16:08:19.6 (measured scan gaps 11–12s, not 6s) → entire overshoot lived inside one scan gap; exec latency after the scan was <1s (mechanism healthy). Book stayed orderly throughout (spread pinned 5.4bps, top-5 depth $140–200K both-sides) → breach-time exit was executable; like-for-like save ≈$500–580 on this trade. Faster-than-scan SL action recovers this by construction. NOTE: `dolphin.obf_fast_intrade` (in-trade fast OB feed) has 0 rows — dead wiring in BLUE; UV's fast clock must not repeat this (a populated fast feed is the C11 sensor). | +| C1 UV Zinc plane + Q-dial, C2 GraalPy smoke, C9 DITAv2 probe | **DEFERRED** post-first-trade (§9). Graal-readiness is enforced NOW via §10 instead. | +| UV core runtime (separate rewrite) | **CANCELLED as churn** — UV v1 IS the certified PRIME path promoted (§8). `uv/blue_prime/` freezes as oracle at certification. | + +## 5. Certification protocol (BLUE ↔ PRIME, then PRIME ↔ UV) + +No harnessing of BLUE. PRIME runs read-only beside live BLUE; certification is a +**tick-aligned data diff**: +- PRIME's per-scan record: `dolphin_uv.prime_decisions` (WIRE.5 journal: inputs, 11 hook + effects, decision) + zinc snapshot. +- BLUE's record: `dolphin.trade_events` (+ logs) — entries/exits with asset/side/leverage. +- Continuous signal: per-scan hook/leverage modulation self-consistency; hard gate: **entry + events**, matched on scan identity, **bit-identity** (PASS2.5 standard: mismatch = bug, + not tolerance). +- **Gate (restructured 2026-07-02, operator: compute-bound not calendar-bound):** + - **Gate A — replay-cert (gating):** replay BLUE's RECORDED input history (months of + eigen-scans, all realized trades) through PRIME's decision path, Q-quantized + (scan-sequenced, warts and all), vectorized where the kernel allows. Bit-diff every + decision vs `dolphin.trade_events` — every entry BLUE ever made, not 3. 0 unexplained + diffs. (Synthetic inputs CANNOT gate BLUE-parity — BLUE has no recorded answer for them.) + - **Gate B — faultline assault (gating):** Hypothesis/adversarial/fuzz on breakspots + (vel_div threshold boundary, EFSM transitions incl. post-win LONG overlay, staleness, + poison) — properties: PRIME-internal consistency + PRIME↔UV agreement. + - **Gate C — live plumbing (non-gating):** guarded soak keeps running; live HZ reads, + scan gaps, mirror hydration clean over X hours; any live entry = bonus bit-check. + - Pollution invariant green throughout; operator signs the cert. Then PRIME freezes as + oracle; UV certifies against PRIME with the same instruments (Gate A replay + Gate B), + writing `dolphin_uv.uv_decisions`. + - **Wave-2+ note (operator):** multi-instance PRIME-[n] clone farms (forkd / workdir.dev + -class system forks; VIBRASS bandit meta-gov) — design-in now: per-instance CH namespace + `dolphin_uv_{n}`, per-instance zinc prefix `uv_shadow_{n}`, injectable clock/Q. + +## 6. Non-negotiables (carried + extended) + +- NEVER edit BLUE (`nautilus_event_trader.py`, kernels, `prod/ch_writer.py` [vendored/shared], + supervisord, HZ contents, `dolphin.*` tables). PRIME reads BLUE's world; writes NOTHING into it. +- `dolphin_uv.*` is UV/PRIME's only CH namespace. Hard-guarded in code + tests + (journal URL guard raises on non-dolphin_uv; T1 no_write_guard diverts every other CH write + to a local audit file; T4 reporter is zero-CH-write; HZ is wrapped read-only). +- **Separate-install direction (operator, 2026-07-02):** before UV's first VST trade (C10), + UV's own writes move to a DEDICATED CH instance (own port/datadir or container) — namespace + isolation is the guard, instance isolation is the wall. UV needs NO HZ writes in wave 1 + (reads BLUE's HZ read-only); if UV ever needs its own KV plane, it gets its own instance — + never keys in BLUE's cluster. +- VST only; `ALLOW_MAINNET=0`; DARK until operator arms. +- Testing doctrine: mutation litmus, poison/edges/concurrency, no green-by-pollution, + run your own suite before push. AI_DEV doctrine: one problem/one branch/one PR, + explicit staging, docs > chat. +- Vendored-drift gate: VENDOR.lock components edited only upstream + `vendor_sync.sh`. + +## 7. Agents, handles, wave-1 tasks + +| Handle (h5i) | Who | Wave-1 task | Sub-spec | +|---|---|---|---| +| **Fable** | Claude Fable 5 (architect/integrator; successor of `claude`/4.8 + `cc-ultrav-1`) | **T0**: git integration pass (main + wire-4 + reshape + clone salvage → main); commit this spec; correct/retire stale status doc | this doc §3–4 | +| **codex** | Codex 5.4mini | **T1 (CRITICAL)**: pollution forensics + neutralization + PRIME relaunch | `UV_TASK_T1_CODEX_POLLUTION_GUARD.md` | +| **mm_VIOLET1** | mimocode (slow, precise) | **T2**: journal hardening + C7a differ core | `UV_TASK_T2_MM_JOURNAL_DIFFER.md` | +| **pi_nvnemo** | PI harness (Nemotron) | **T3**: C7b differ test suite + TUI parity panel | `UV_TASK_T3_PI_PARITY_TESTS_TUI.md` | +| **cmd-PASS1.1** | Command Code / DeepSeek (operator fires instance) | **T4**: C7c certification reporter | `UV_TASK_T4_CMD_CERT_REPORTER.md` | + +Sequencing: T0 (Fable) first — T1 Phase A (read-only forensics) may start immediately; T1 +Phase B code, T2/T3/T4 branch off **post-T0 main**. All reporting on the canonical h5i bus +(`/mnt/dolphinng5_predict`); reply to **Fable**. Sub-specs live in +`prod/docs/uv_subspecs/` (committed to the violet repo at T0). + +## 8. Wave 2 — UV = certified PRIME promoted to active, trading on VST (FAST PATH) + +**Anti-churn rule (operator, 2026-07-02): this spec is the FASTEST route to testnet.** +No rewrites of things that already work. Concretely: + +- **NO new UV decision core.** PRIME already runs BLUE's real engine + 11 hooks + EFSM mirror + + live inputs. The day PRIME certifies, **UV v1 = the certified PRIME decision path promoted + from shadow to active** (new process/config, journals `dolphin_uv.uv_decisions`, DARK) + + a KernelIntent emitter. Zero algorithm code rewritten between certification and first trade. +- **NO new exec layer.** DITAv2 is already VST-proven (PINK burn-in; VIOLET v4 runner + precedent — reuse its KernelIntent→DITAv2 wiring pattern as a NEW instance; never edit the + live v4 runner). DITAv2 stays vendored (VENDOR.lock; improvements upstream + vendor_sync). + ASEx improvements ride along inside DITAv2's existing integration — the single-writer / + no-async-seam guarantees are why it's the kernel; we do not re-plumb them. +- **NO new reconciler from scratch.** Adopt DITAv2's existing reconcile logic (PINK ownership- + filter lineage) under UV's own VST account/clientOrderId prefix (`u-`); C8 shrinks to + "verify + adapt + rate-budget test", not "build". +- **Arming ladder (operator-gated per rung):** DARK (journal-only) → observe-only → VST canary + (min size, single slot) → staged size. `ALLOW_MAINNET=0` throughout; mainnet is out of scope. +- **Decision-vs-execution parity split:** decision layer must bit-match certified-BLUE at + Q=scan (differ green in shadow while UV trades); execution quality is judged by + reconciler-clean + DITAv2 accounting integrity (fill-price PnL doctrine), since BLUE's + "fills" are in-memory bookkeeping and UV's are real VST mechanics. +- **C11 SL/TP fast clock** (rate-parity stage 2 — protective actions faster than eigenscan, + alpha stays at Q=scan): unlocked AFTER first clean canary trades, not before. +- **Exit = §1 end-state acceptance.** That is the whole spec. + +## 9. Deferred (post-first-trade; separate specs — do NOT build in this spec's waves) + +- C1 UV-own Zinc plane + generalized Q-dial (UV v1 runs at scan cadence natively — Q=1:1 — + so the dial abstraction earns nothing until stage-3 loosening). +- C2 GraalPy smoke rig; C9 DITAv2 innards probe; TUI beyond the existing panels. +- Wave-Graal: the GraalPy/GraalVM/Graal-OS migration dev-ops pass (own spec when VST soak + is running). Q-loosening experiments (rate-parity stage 3) live there or after. + +### 9.1 Sketchlog lane (operator-flagged CRITICAL, 2026-07-02) +Source: `prod/docs/VIOLET_TODO_CRITICAL_DISTRIBUTION_TRACKING_IN_CONSTRAINED_MEMORY.md` +(9 signals mapped to BIBLE integration points; sketchlog = DDSketch/HLL/CMS/DriftSketch, +93 KB constant memory, mergeable monoids, WindowedStreamLog realtime windows, optional C++). +- **Now (observability, zero parity risk):** sketch dimensions (vel_div percentiles, + signal breadth HLL, reversal freq) added to PRIME snapshot + journal as OBSERVE-ONLY + columns; T7 replay computes them over full history = instant candidate-feature backtest. +- **Post-cert (alpha, gated):** signals as decision inputs (esp. #3 rolling-MAE tail + detector → adaptive exits = the left-tail killer; #1 widening; #4 breadth; #6 reversal) + — UV-divergence features via the certified-Q baseline + diff-guarded rollout. NEVER BLUE. +- Merge algebra fits PRIME-[n] farms (coordination-free merge); pure-Python path = §10 G1 ok. + +### 9.2 Control plane (operator directive 2026-07-02: "NATS/iceoryx2 the hell out of it") +One control plane over the WHOLE system: fleet lifecycle (start/stop/arm PRIME-[n]/UV/soaks), +config + Q-dial distribution, heartbeats, gate-ledger events, kill-switch propagation. +- **Split doctrine:** data plane intra-box = Zinc/iceoryx2 (ADR-1, adopted); CONTROL plane + inter-process/inter-box = message bus. Candidate: NATS (operator-named). NOTE: ADR-2 + reserved Zenoh for inter-box DATA — NATS-for-control vs Zenoh-for-data can coexist; + Fable authors the control-plane ADR when wave 2 opens. +- NOT on the critical path to first testnet trade; REQUIRED before the clone farm. +- Design-in now (already true): every long-running process publishes a zinc snapshot and + takes env-injected config — those are the surfaces the control plane will drive. + +## 10. Graal-readiness — build constraints binding NOW despite deferred migration (NFR-G) + +Cheap guardrails (mostly "don't do X"), enforced in review — so the later migration is a +runtime swap, not a rewrite: + +- **G1** No CPython-only C-extensions in UV hot paths (pure-Python, or Rust behind stable FFI). +- **G2** No `__del__`/refcount-timing for correctness — explicit lifecycle (ASEx Drop-reliance + leak = the cautionary tale). +- **G3** No "GIL makes this safe" — cross-thread state only via single-writer/Zinc/ASEx seams. +- **G4** UV code never imports `hazelcast` directly — HZ quarantined behind the existing + reader seams (HZBridge direction). +- **G5** No hardcoded paths/creds; env-injected config. +- **G6** Long-running RSS-stable, jemalloc-compatible processes. + +## 11. T6 (ACTIVE, HIGH PRIORITY — promoted 2026-07-02): real-Zinc unification + +The clone line (banked as branch `salvage/uv-clone-line-1902e4b`) carries the REAL mmap Zinc +region transport (`ZincShadowChannel`, prefix `uv_shadow`, 18 h live soak) + the Textual +`tui_v2`. The merged main publishes the same snapshot via atomic file (cross-process, works +today) + in-memory zinc. T6 = port `ZincShadowChannel` under the reshape snapshot contract as +the transport, re-home `tui_v2` on it. Small, spec to follow; does NOT gate T1-D relaunch. diff --git a/prod/docs/uv_subspecs/UV_TASK_T6_CODEX_REAL_ZINC.md b/prod/docs/uv_subspecs/UV_TASK_T6_CODEX_REAL_ZINC.md new file mode 100644 index 00000000..46040a09 --- /dev/null +++ b/prod/docs/uv_subspecs/UV_TASK_T6_CODEX_REAL_ZINC.md @@ -0,0 +1,32 @@ +# UV TASK T6 — real Zinc region transport (close the file-snapshot regression) + +**Assignee:** codex · **Issuer:** Fable · **PRIORITY: HIGH — operator-ordered promotion.** +**Why:** the merged main's TUI path reads an atomic JSON file in /dev/shm (`ShmChannel`) — +file semantics, no mapped region/seqlock/notify. That is the exact "mock shm" the shm-reshape +sprint existed to eliminate. The REAL transport already exists and soaked 18 h on the clone +line: `ZincShadowChannel` in `salvage/uv-clone-line-1902e4b`'s `uv/shm.py` (mapped +`/dev/shm/zinc_uv_shadow_state` region, magic header, monotonic seq, `uv_shadow` prefix, +DITAv2 zinc adapter loading via `ZINC_PYTHON_PATH`). + +**Task:** port `ZincShadowChannel` (+ its region encode/decode + `_FileChannel` fallback +mechanics as needed) from the salvage branch onto current main, UNDER the reshape snapshot +contract (BluePrimeSnapshot stays the schema — transport changes, contract does not): +1. New/updated module in `uv/` (e.g. extend `shm.py` or add `zinc_channel.py`): writer = + publish the SAME versioned snapshot into the mapped region, atomic + seq-increment + + notify per the zinc-shadow spec §5; reader = TUI-side wait/read of latest complete frame. +2. `blue_prime/runner.py`: publish via real zinc region as PRIMARY; keep the file snapshot as + explicit FALLBACK (env `UV_SHM_TRANSPORT=file`; default `zinc`), not the default. + [SPEC AMENDED 2026-07-02: knob name aligned to the salvage implementation's + `UV_SHM_TRANSPORT` — same semantics as the original `UV_SHM_FALLBACK`, clearer name, + zero churn on the already-soaked code.] +3. `uv/tui.py`: read the region directly as PRIMARY (same render path — snapshot dict in, + panels out); file fallback only when region absent, and SAY SO on screen. +4. Port/adapt the salvage branch's zinc tests + add: torn-frame/seq test, cross-process + round-trip test (writer proc + reader proc), region-absent fallback test, mutation litmus. +5. Relaunch the guarded soak (same UV_BLUE-PRIME_TUI2 discipline, guard active) on the zinc + path; verify seq advances at scan cadence and TUI reads the REGION (show source in meta). +**Non-goals:** no tui_v2 port (separate, later), no DITAv2 region reuse (own prefix, +retire-able — zinc-shadow spec §3.3), no schema changes. +**Branch:** `uv/t6-real-zinc` · fresh clone `/root/uv-wt/t6-zinc` off /root/violet.git · +push→CI · DONE to Fable with: branch+sha, tests, soak PIDs + region seq evidence, and +`grep`-proof the TUI's primary path never opens the JSON file when the region exists. diff --git a/prod/docs/uv_subspecs/UV_TASK_T8_CMD_STOP_WATCHER.md b/prod/docs/uv_subspecs/UV_TASK_T8_CMD_STOP_WATCHER.md new file mode 100644 index 00000000..c18ae9af --- /dev/null +++ b/prod/docs/uv_subspecs/UV_TASK_T8_CMD_STOP_WATCHER.md @@ -0,0 +1,49 @@ +# UV TASK T8 — shadow stop-watcher (C11's sensor, live overshoot ledger) + +**Assignee:** cmd (Command Code) · **Issuer:** Fable · **Master spec:** §4 C11, §5 Gate C. +**Empirical basis:** `prod/docs/BLUE_STOPLOSS_OVERSHOOT_AND_UV_COUNTERFACTUAL_20260702.md` +(codex, verified by Fable): last-1000 BLUE trades → 49 stops, 30 overshot the −1.2% +contract, ≈$4,432 excess. FET `e81e595d`: stop breached 16:08:15.9, eigenscan not until +16:08:19.6 (measured gaps 11–12 s), overshoot lived entirely inside the scan gap. + +## Goal +A standalone read-only daemon that watches BLUE's OPEN positions on a ~1 s clock and +journals every stop-contract breach the moment it happens — so every future overshoot is +measured live instead of forensically. This is the shadow twin of the future C11 fast SL +clock: when UV trades with fast stops, this ledger IS the A/B evidence. + +## Deliverable: `prod/clean_arch/violet/uv/stop_watcher.py` (+ DDL + tests) +1. **Position feed:** poll `dolphin.trade_events` (read-only, dedup by trade_id, argMax ts) + every ~10 s for open positions (entry event without terminal exit). Carry entry_price, + side, quantity, our_leverage. +2. **Price feed:** poll `dolphin.obf_universe` (read-only) best_bid/best_ask per open asset + every ~1 s. MEASURE ingest lag (row ts vs now) and journal it; if p95 lag > 2 s, log a + LIMITATION line — do NOT silently trust stale prices. (Fallback to BingX public WS + bookTicker is allowed — public data, no keys, no BLUE interaction — but is a stretch + goal, not v1.) +3. **Breach detection:** SHORT: ask ≥ entry×1.012; LONG: bid ≤ entry×0.988. On first + breach per trade, write one event; keep sampling and write escalation rows at each + +0.1 % beyond the stop (so the overshoot PATH is recorded, not just the edge). +4. **Journal:** `dolphin_uv.stop_watch_events` (NEW table, dolphin_uv namespace ONLY): + ts, trade_id, asset, side, entry_price, breach_price, adverse_pct, spread_bps, + depth_1pct_usd, obf_lag_ms, kind ('BREACH'|'ESCALATION'|'RESOLVED'). RESOLVED row when + the trade's exit appears in trade_events, carrying exit adverse_pct + excess vs 1.2 %. +5. **Daily rollup view:** overshoot count, total excess $, worst trade — the $4.4K audit, + automated forever. + +## Iron rules +- ZERO writes outside `dolphin_uv.*`. Never touch BLUE code, HZ contents, or dolphin.* + tables. CH creds read-only usage; INSERT only into dolphin_uv.stop_watch_events. +- Runs under /home/dolphin/siloqy_env, own log file, no hardcoded worktree paths + (derive paths from __file__ / env). + +## Tests (doctrine: mutation litmus mandatory) +Synthetic price/position fixtures → exact expected breach/escalation/resolved sequence; +mutation litmus (flip breach comparison, drop escalation step → tests go RED); +lag-measurement honesty test; no-write guard test (any non-dolphin_uv INSERT raises); +determinism (same fixture ⇒ same journal twice). + +## Done +Branch `uv/t8-stop-watcher` off current main, fresh clone, push → CI, DONE to Fable with: +branch+sha, test count, a live 30-min run's journal excerpt (real breaches or honest +"no breaches in window"), measured obf_universe lag stats. diff --git a/prod/docs/uv_subspecs/UV_TASK_T9_CMD_DITAV2_EXEC_SEAM.md b/prod/docs/uv_subspecs/UV_TASK_T9_CMD_DITAV2_EXEC_SEAM.md new file mode 100644 index 00000000..b459eb00 --- /dev/null +++ b/prod/docs/uv_subspecs/UV_TASK_T9_CMD_DITAV2_EXEC_SEAM.md @@ -0,0 +1,48 @@ +# UV TASK T9 — C10 DITAv2 exec seam (KernelIntent → DITAv2 → BingX VST, DARK) + +**Assignee:** cmd-PASS1.2 · **Issuer:** Fable · **Master spec:** §4 C10, §8 fast path. +**Doctrinal kernel:** post-sync vendored dita_v2 per `UV_DITAV2_SOA_VERDICT_20260703.md` +(includes VenueTelemetrySnapshot + zinc venue plane + asex_account.py). Do NOT start +until Fable confirms the vendor sync landed on main (watch the bus). + +## Goal +The physical trading path: a UV-owned execution runner that consumes KernelIntents and +drives DITAv2 against BingX **VST testnet** — built now, DARK by default, so that when +PRIME is promoted (post Gates A+B) the only remaining step is arming it with keys. +Pattern precedent: `prod/clean_arch/violet/v4_execution_runner.py` (read for shape; this +is a NEW instance and a NEW module — never reuse a BLUE/PINK/VIOLET runner instance). + +## Deliverable: `prod/clean_arch/violet/uv/exec/` (new package) +1. **`intent_source.py`:** KernelIntent inlet. v1 = injectable queue + a file/CLI + injector for dry-run intents (PRIME promotion wiring is a LATER task — leave a + clearly-marked seam, not a stub that pretends). +2. **`seam.py`:** intent → DITAv2 order mapping. EVERY clientOrderId prefixed `u-` + (non-negotiable — this is how UV's orders are distinguishable on the venue forever). + Sizing/leverage passthrough from intent; no local overrides. +3. **`runner.py`:** launcher wiring per dita_v2 `launcher.py` — NEW instance name + (`uv_exec`), zinc venue plane ENABLED (venue_region telemetry is the seam's flight + recorder), jemalloc-friendly long-run posture, no hardcoded paths. +4. **DARK doctrine:** with no keys configured → observe-only: log + journal every intent + and the order it WOULD place (full params), place nothing. `ALLOW_MAINNET=0` is a + hard block: mainnet refuses even if env says otherwise unless a separate operator + arming file exists (two-man rule). VST base URL only. +5. **Journal:** every intent, mapping, would-place/placed, venue telemetry snapshot → + `dolphin_uv.exec_journal` (dolphin_uv namespace ONLY). + +## Iron rules +- VST ONLY. DARK until operator arms. `u-` prefix on every clientOrderId. +- Zero writes outside dolphin_uv.*; zero BLUE touches; vendored dita_v2 is read-only + (any kernel change goes upstream + vendor_sync, never in-place). +- Graal-ready NFR-G applies (no CPython-only exotica in the seam layer). + +## Tests (mutation litmus mandatory) +Intent→order mapping bit-exact fixtures; `u-` prefix litmus (strip the prefix in code → +test RED); DARK default test (no keys ⇒ zero venue calls — assert at the venue adapter +seam, not by mocking the seam itself); mainnet-block mutation test (force +ALLOW_MAINNET=1 without arming file ⇒ still refuses); venue-plane telemetry presence +test; determinism. + +## Done +Branch `uv/t9-exec-seam` off post-sync main, fresh clone (NOT a worktree of the bare), +push → CI, DONE to Fable with: branch+sha, test count, and a dry-run journal excerpt +showing 3 injected intents fully mapped + journaled + zero venue calls.