diff --git a/prod/docs/VIOLET_PART_SPEC_OA_TODO_PASS2.md b/prod/docs/VIOLET_PART_SPEC_OA_TODO_PASS2.md new file mode 100644 index 00000000..32c4e222 --- /dev/null +++ b/prod/docs/VIOLET_PART_SPEC_OA_TODO_PASS2.md @@ -0,0 +1,175 @@ +# VIOLET — partial spec for another agent, PASS 2 (OA TODO) + +Date: 2026-06-16. Follows `VIOLET_PART_SPEC_OA_TODO.md` (PASS 1, reported done — pending +review). PASS 2 tasks **advance the main VIOLET global spec** (`VIOLET_DEV_SPEC_AND_PLAN.md`, +the V0→V6 ladder) toward the DARK soak gate and V4 execution, while staying self-contained. + +--- + +## 0. HARD RULES (identical to PASS 1 — re-read; non-negotiable) + +1. **Never edit shared files**: `prod/nautilus_event_trader.py`, `prod/clean_arch/dita_v2/**`, + `prod/clean_arch/dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, + `prod/bingx/leverage.py`. READ only. +2. **VIOLET stays DARK** — no orders, no execution, no VST keys, no service start/stop, no HZ + restart, no PROGREEN. +3. **V-TYPES on all new code** (`StrictModel` / `Annotated[…Field]` / `@typed`); no arbitrary + caps, only faithful poison-guards. +4. **NEW-FILE-ONLY** — do NOT modify these in-flight files (owned by another agent right now): + `live_blue_source.py`, `shadow_live_factors.py`, `live_factor_source.py`, `live_factors.py`, + `decision_engine.py`, `sizing.py`, `shadow_journal.py`, `prod/clickhouse/violet/22_violet_decisions.sql`. + You may READ + IMPORT them. Wiring into them is deferred to their owner. + +## 0a. COMMIT / BRANCH POLICY (CRITICAL — 3 shared-index collisions happened 2026-06-16) + +The working tree + `.git/index` are SHARED across agents; `git commit` commits the whole index, +so a concurrent agent's staged files get swept into your commit. THEREFORE: +- **Strongly preferred:** your own worktree — `git worktree add ../vp-oa2 -b agent/oa-violet2`. +- **Otherwise:** never `git add -A`/`git add .`; commit with explicit pathspec — + `git commit -F msg.txt -- ` — and verify + `git show --stat --format="" HEAD` lists ONLY your files. +- One commit per task, prefix `VIOLET OA:`, Co-Authored-By trailer. Before "done": + `git diff --name-only HEAD~1` ∌ any forbidden path. +- Tests on `/home/dolphin/siloqy_env/bin/python3 -m pytest -q`. CIFS-slow is normal. Use + `git grep`, not recursive `grep -r`/`find`. + +## 0b. ClickHouse access (read-only) +`http://localhost:8123`, user `dolphin` / key `dolphin_ch_2026`. VIOLET data in db +`dolphin_violet`; BLUE data in db `dolphin`. **READ ONLY** — never write/alter production tables. + +--- + +## TASK 4 — Live BLUE↔VIOLET decision parity report (advances V3d → DARK-soak gate) + +**Spec link.** The ladder's V3d step requires proving the shadow DecisionEngine reproduces +BLUE's decisions. A SHADOW soak already journaled VIOLET decisions to +`dolphin_violet.violet_decisions`. This task builds the report that compares them to BLUE's +recorded decisions — the quantitative gate that makes a longer soak meaningful. + +**Affected files (NEW only):** +- `prod/clean_arch/violet/parity_report.py` (new) +- `prod/clean_arch/violet/test_violet_parity_report.py` (new) +- output → `prod/VIOLET_dev/reports/violet_parity_.{json,md}` (generated) + +**Approach.** +- Read VIOLET decisions from `dolphin_violet.violet_decisions` (asset, side, scan_number, + conviction_leverage, notional_fraction, target_exposure, vel_div, ts) and BLUE's recorded + decisions from `dolphin.trade_events` / `dolphin.v7_decision_events` (inspect their schemas + first via `DESCRIBE`). Align by nearest scan/timestamp + asset. +- Compute: pick-match rate (same asset/side chosen), sizing deltas (VIOLET vs BLUE + conviction_leverage / notional — distribution: mean/median/p95 abs err), and a + divergence-reason breakdown (no-pick, different-asset, sizing-gap > threshold). +- Pure analysis: NO writes to any production table; emit a JSON + a human-readable MD report. +- V-TYPES the report rows; reject malformed/non-finite rows to a counter (never crash). + +**Tests / pass criteria.** +- A `--self-test` / unit mode runs the comparison on a SYNTHETIC fixture (hand-built VIOLET + + BLUE rows with known overlaps) and asserts the computed pick-match / sizing-delta numbers + exactly. No live CH needed for the unit test. +- On the prod host, the script runs end-to-end against live CH and archives a report. +- DONE when: self-test passes, a real report is archived, zero production writes, no edits + outside the 2 new files. + +--- + +## TASK 5 — Full-DecisionEngine reactor latency gate (advances V0/V3 — budget proof) + +**Spec link.** V0 proved the reactor clock meets the latency budget (reaction p99 < 10ms, +jitter p99 < 25ms) with a trivial handler. V3 put the real decision brain online. This gate +proves the budget STILL holds when the reactor drives the FULL +`VioletDecisionEngine.decide(factors=SizingFactors(...))` path (the heavy 5-factor sizing), +not a stub — a prerequisite for trusting the sub-second cadence claim. + +**Affected files (NEW only):** +- `prod/clean_arch/violet/test_violet_v3_decision_latency_gate.py` (new, `@pytest.mark.gate`) +- output → `prod/VIOLET_dev/reports/violet_v3_decision_latency_.json` (generated) + +**Approach.** +- Reuse V0's `PlaneClock` / `DeadlineScheduler` (`clock.py`) and the storm pattern from + `test_violet_v0_latency_gate.py` / `test_violet_v2_exec_gate.py` (READ them for the harness). +- Build a warmed `VioletDecisionEngine` (feed enough synthetic scans to pass the IRP lookback), + then in the storm loop call `decide(now_ns=…, scan_number=…, capital=…, vel_div=…, + factors=SizingFactors(boost=…, beta=…, mc_scale=…, esof_score=…, ob_*=…, dc_status=…, + posture=…))` each tick. Measure reaction + jitter percentiles. +- Do NOT modify `decision_engine.py`; import + drive it. + +**Tests / pass criteria.** +- Gate asserts reaction p99 < 10ms and jitter p99 < 25ms (same budget as V0) over ≥ 200 cycles + with the FULL factor path; archive the percentile report. +- DONE when: the gate passes on the prod host, report archived, no edits outside the new file. + (If it FAILS, do not loosen the budget — report the regression with numbers; that is a real + finding.) + +--- + +## TASK 6 — L3 tradeability projector, standalone (advances toward V4 execution) + +**Spec link.** The 3-layer doctrine: L1 pure alpha (done), L2 parity harness (done), **L3 +tradeability** — impose conviction→exchange-leverage + maker policy on the L1 output. The +wrapper exists (`exchange_leverage.py`, bit-identity-gated). This task builds the standalone +projector that turns a `ShadowDecision` into a tradeable projection, WITHOUT wiring it into the +journal or any venue (DARK; wiring is the owner's job later). + +**Affected files (NEW only):** +- `prod/clean_arch/violet/tradeability.py` (new) +- `prod/clean_arch/violet/test_violet_tradeability.py` (new) + +**Approach.** +- Read `exchange_leverage.py` (`VioletExchangeLeverage`, `ExchangeLeverageDecision`) and + `decision_engine.py` (`ShadowDecision`) and the margin-study facts (notional = base_fraction × + conviction × capital; exchange leverage = `map_internal_conviction_to_exchange_leverage`, + max 3× cubic). +- Define `TradeabilityProjection(StrictModel)` with: internal_conviction, exchange_leverage + (int, ge=1), target_notional, est_margin (= notional / exchange_leverage), maker_policy hint + (string, e.g. "maker_both" — read existing ExecutionRouter conventions, do NOT import PINK + exec). Define `project_tradeability(decision: ShadowDecision, *, capital: float) -> + TradeabilityProjection` (`@typed`). NO orders, NO venue calls — pure projection. +- This is L1-output → L3 projection only; it must not change L1 sizing. + +**Tests / pass criteria.** +- Exchange leverage equals `exchange_leverage.py`'s mapping for the decision's + conviction_leverage (bit-identical — reuse that wrapper, don't reimplement). +- est_margin = target_notional / exchange_leverage; poison guards reject non-finite/negative. +- Hypothesis property test over conviction ∈ [0.5, 9]: projection finite, exchange_leverage ∈ + [1,3], margin ≤ notional. +- `pytest -q prod/clean_arch/violet/test_violet_tradeability.py` passes; no edits outside the 2 + new files. + +--- + +## TASK 7 — Deterministic replay + golden gate (advances V3d determinism criterion) + +**Spec link.** V3d also requires DETERMINISM: same recorded scans ⇒ same decisions twice. This +hardens soak confidence and catches accidental nondeterminism (dict ordering, RNG, time +leakage) in the decision path. + +**Affected files (NEW only):** +- `prod/clean_arch/violet/test_violet_replay_determinism_gate.py` (new) +- `prod/clean_arch/violet/fixtures/replay_scans.json` (new, small synthetic scan sequence) + +**Approach.** +- Build a fixed synthetic scan sequence (≥ 60 scans, a few assets incl. one stablecoin to prove + the exclusion gate, varied vel_div crossing the entry threshold). Store as the fixture. +- Run it through TWO fresh `VioletDecisionEngine` instances (with identical fixed + `SizingFactors`) and assert the emitted `ShadowDecision` sequences are EQUAL field-by-field. +- Add a golden assertion: the decision sequence matches a checked-in expected summary (asset, + side, conviction_leverage, scan_number per fired decision) — regenerate-on-purpose only. +- Read `decision_engine.py`; do not modify it. + +**Tests / pass criteria.** +- Two runs over the fixture produce identical `ShadowDecision` lists (`==`). +- The golden summary matches; stablecoin assets never selected. +- `pytest -q prod/clean_arch/violet/test_violet_replay_determinism_gate.py` passes; no edits + outside the 2 new files. + +--- + +## Sequencing note +Tasks 4–7 are independent and parallelizable. Recommended priority for advancing the spec: +**Task 5 (latency budget) → Task 7 (determinism) → Task 4 (parity report) → Task 6 (tradeability)** +— the first two are gates that must hold before a longer DARK soak; Task 4 makes the soak +measurable; Task 6 preps V4. None require the operator's soak/keys greenlight. + +## Still NOT in scope +DARK soak START (operator-held); V4 live execution (keys-blocked); HZ-bridge refactor (depends +on `dolphinng5_predict/hzbridge` shipping); any edit to the in-flight files in §0 rule 4.