VIOLET: master dev spec & plan + parallel L3 exchange-leverage sub-spec

VIOLET_DEV_SPEC_AND_PLAN.md: authoritative consolidated plan (mission, doctrine, V0->V6 ladder w/ status, code map, full sizing composition, next steps, vision, TODOs, operational notes). Supersedes scattered plan files. VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md: self-contained parallel-developable unit (V3.5) for an independent agent — wrap prod/bingx/leverage.py conviction->exchange mapping, V-TYPES + bit-identity gate, full file paths/tests/gates/acceptance. Zero overlap with V3.4 (DecisionEngine<->Sizing integration, lead-owned). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:14:51 +02:00
parent dc3d0970ad
commit f1ee1368d2
2 changed files with 275 additions and 0 deletions
--- a/prod/docs/VIOLET_DEV_SPEC_AND_PLAN.md
+++ b/prod/docs/VIOLET_DEV_SPEC_AND_PLAN.md
@@ -0,0 +1,128 @@
+# VIOLET — Master Dev Spec & Plan
+
+**Authoritative consolidated plan.** Supersedes the scattered plan files
+(`~/.claude/plans/harmonic-jumping-plum.md` [V2], `drifting-knitting-zebra.md` [V3]).
+Repo cwd `/mnt/dolphinng5_predict` (git root, **no remote — local-only, build on-host**).
+Branch `exp/pink-ditav2-sprint0-20260530`. Python `/home/dolphin/siloqy_env/bin/python3`.
+Last updated 2026-06-15.
+
+Cross-refs: `VIOLET_V3_FINDINGS.md` (study + 5-factor composition §8b + vision §8c),
+`VIOLET_BUILD_SPEC__SIZING_PARITY.md` (+ Annex A dev log), `FROZEN_ALGO_SPEC_GOLD_REFERENCE.md`,
+memory `violet_subsecond_rebuild_plan` / `violet_v3_alpha_doctrine` / `blue_margin_envelope_study`.
+
+---
+
+## 1. Mission
+
+Rebuild the DOLPHIN trading system (BLUE's live Alpha Engine) onto a sub-second
+event-driven reactor substrate — **bit-for-bit faithful to BLUE's alpha**, but on a
+chassis that is type-safe, observable, attributable, and trustworthy (the original's
+alpha is fund-grade; its bookkeeping is not). Stage the rebuild V0→V6; each stage is
+existing/wrapped code + new wiring, gated.
+
+## 2. Binding doctrine
+
+- **Model BLUE, not PINK.** Reference = BLUE's LIVE Alpha Engine (`prod/nautilus_event_trader.py`
+  + `nautilus_dolphin/nautilus_dolphin/nautilus/*`). Behavioural/distributional fidelity.
+- **Live BLUE code is the sole doctrine**; where it diverges from any doc/spec, BLUE wins.
+  `blue_parity.py` is a PINK-era distillation — reference only, validate don't trust.
+- **WRAP, DON'T REIMPLEMENT.** Run BLUE's real kernels; transcribe only trivial deterministic
+  arithmetic, and prove it with **Monte-Carlo → bit-identity → upstream** gates.
+- **Follow BLUE in all regards** — no filters/hygiene/bounds BLUE lacks (only V-TYPES poison
+  rejection: finite + non-negative where BLUE guarantees it).
+- **V-TYPES** (`prod/clean_arch/violet/domain.py`): refined types at boundaries, `@typed`
+  (beartype), `StrictModel` value objects, reject-at-source. Motivated by the bars_held=-106
+  poison incident.
+- **Reactor substrate, NOT a scan clone.** BLUE's scan-quantized behaviour is hosted on the
+  V0 reactor and quantized at **Q=scan initially**; per-action Q loosenable later (cadence
+  control plane). Scan-driven-ness is a quantization setting, not the architecture.
+- **Decision layer is slot-independent** — VIOLET decides every scan; the slot only gates
+  trades (execution layer). Different layers.
+- **DARK until keys** — no orders until operator provisions VST keys; ObserveOnlyVenue guard.
+
+## 3. Stage ladder (V0→V6)
+
+| Stage | Scope | Status |
+|---|---|---|
+| **V0** | reactor clock + DeadlineScheduler + harness | ✅ shipped (latency gate passed) |
+| **V1** | observe-only DARK service + divergence monitor + CH DDLs | ✅ shipped |
+| **V2** | ExecDeadlineDriver @100ms TTL + ScriptedVenue + V-TYPES domain | ✅ shipped (gate passed) |
+| **V3** | DecisionEngine SHADOW (models BLUE) | ✅ shipped |
+| ↳ V3a | alpha_wrappers (selector/sizer/exit-v7) | ✅ |
+| ↳ V3b | cadence control plane (per-action Q) | ✅ |
+| ↳ V3c | VioletDecisionEngine (reactor-resident shadow) | ✅ |
+| ↳ V3d | base-sizer parity gate | ✅ |
+| ↳ V3e | shadow journal + DDL + launcher wiring | ✅ |
+| ↳ V3.1 | BLUE stablecoin exclusion (parity fix) | ✅ |
+| ↳ V3.2 | EsoF size-modulation fold | ✅ |
+| ↳ V3.3 | **full sizing parity** (orchestrator wrap-all, 5-factor, bit-identity) | ✅ reviewed+committed |
+| **V3.4** | wire VioletSizer into VioletDecisionEngine (full conviction in shadow) | ⏳ NEXT (mine) |
+| **V3.5** | L3 exchange-leverage wrapper (conviction→exchange, bit-identity vs leverage.py) | ⏳ parallel-able (see SUB_SPEC) |
+| **V4** | execution ON — single asset, conservative caps, VST testnet → mainnet | ⏳ blocked on keys + V3.4/3.5 |
+| **V5** | IRP multi-asset + sizer | later |
+| **V6** | full bible layers (posture/vol refinements) + sub-second SL guard | later |
+
+## 4. What's shipped — code map (`prod/clean_arch/violet/`)
+
+`clock.py` (V0) · `harness.py` (V0) · `domain.py` (V-TYPES) · `divergence.py` (V1) ·
+`observe_guard.py` (V1) · `cadence.py` (V3b) · `alpha_wrappers.py` (V3a: VioletBetSizer/
+VioletAssetSelector/VioletExitEngine) · `decision_engine.py` (V3c: VioletDecisionEngine
+ STABLECOIN_SYMBOLS) · `parity_harness.py` (V3d) · `shadow_journal.py` (V3e) ·
+`modulation.py` (V3.2: VioletSizeModulation EsoF fold) · `sizing.py` (V3.3: **VioletSizer**
+— the full 5-factor conviction). DDLs in `prod/clickhouse/violet/`. Launcher
+`prod/launch_dolphin_violet.py`. Tests: `test_violet_*.py` (V0–V3.3, incl. bit-identity gates).
+
+## 5. The full sizing composition (authoritative)
+
+BLUE's conviction = five multipliers on the base cubic, composed in `esf_alpha_orchestrator`
+`:600-619` (see `VIOLET_V3_FINDINGS.md §8b`). `VioletSizer.compose` reproduces it bit-for-bit:
+```
+raw = base_leverage × dc_lev_mult × regime_size_mult[ACB_boost×(1+β·s³)×mc_scale] × market_ob_mult × esof_mult
+clamped_max = min(base_max(8) × regime × ob × esof, abs_max(9)); STALKER → min(·,2.0)
+leverage = max(min_leverage, min(raw, clamped_max));  notional = capital × fraction × leverage
+```
+Verified: 1e6 MC + 200k extreme + real-`_try_entry` bit-identity (0 mismatches). Operator's
+two recalled "factors aside ACBv6" = `dc_lev_mult` (DC boost) + `market_ob_mult` (OB consensus).
+
+## 6. Immediate next (V3.4 — mine) + parallelizable (V3.5 — agent)
+
+**V3.4 (I take this):** integrate `VioletSizer` (V3.3) into `VioletDecisionEngine` (V3c) so
+the shadow journal records the FULL conviction (currently base-only). Wire the live factor
+inputs: ACB boost/beta (`AdaptiveCircuitBreaker.get_dynamic_boost_from_hz`), mc_scale,
+esof_score, OB market consensus, dc_status, posture — sourced alongside the scan in the
+launcher's shadow path. Extend `shadow_journal` / `violet_decisions` DDL with the breakdown.
+Re-soak DARK, compare full-conviction shadow vs BLUE trades.
+
+**V3.5 (PARALLEL — independent agent):** L3 exchange-leverage wrapper — isolated, additive,
+bit-identity-gated against `prod/bingx/leverage.py`. Full sub-spec:
+`VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md`. No overlap with V3.4 (different file/concern).
+
+## 7. Long-horizon (post-V4, after live testnet→mainnet)
+
+Vision in `VIOLET_V3_FINDINGS.md §8c`: pure-dataflow-DAG → compile (separation-of-concerns
+AND FPGA-purity, bridged by bit-identity); VIBRISS millions-of-instances banditry; **LONG
+alpha** as a new pure lane; **DISTRACK** (memory-constant streaming distributions — the
+state-side enabler) sequenced AFTER VIOLET trades live.
+
+## 8. Open TODOs (memory `violet_v3_alpha_doctrine`)
+
+(a) review `FLAT_VEL_DIV_BUGFIX_CRITICAL.md` + the out-of-range-vel_div-signal research doc;
+(b) **fix Argos** (MCP disconnected all session — Grep/Read fallback in use);
+(c) DISTRACK (after live); plus the V3.3-review minor: `size()` recomputes clamp/raw outside
+`compose()` (DRY — could drift); episode-collapse trade-granularity comparison (when VIOLET
+has exec facilities); base-fraction sizing study (`VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md`).
+
+## 9. Operational notes
+
+- **Soak control:** `supervisorctl -c prod/supervisor/dolphin-supervisord.conf {start|stop} dolphin_violet`.
+  Stop is graceful (stopwaitsecs=30, stopasgroup). Shadow on via `DOLPHIN_VIOLET_DECISION_SHADOW=1`
+  in the conf env (currently set; soaker STOPPED 2026-06-15 per hygiene). **Leave no soaker
+  running unattended — bring down gracefully.**
+- **ClickHouse:** `http://localhost:8123`, user `dolphin` / key `dolphin_ch_2026`. BLUE data
+  in db `dolphin` (trade_events, eigen_scans, maras_fingerprint, exf_data, v7_decision_events);
+  VIOLET in db `dolphin_violet`.
+- **Eigenvalues data** (for live ACB): `/mnt/dolphin_training/data/eigenvalues` (auto-resolved).
+- **Gate reports:** `prod/VIOLET_dev/reports/`.
+- **Shared files — NEVER edit:** `prod/nautilus_event_trader.py`, `prod/clean_arch/dita_v2/*`,
+  `prod/clean_arch/dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`. Mechanical
+  per-commit check: `git diff --cached --name-only` ∌ those.
--- a/prod/docs/VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md
+++ b/prod/docs/VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md
@@ -0,0 +1,147 @@
+# VIOLET Sub-Spec — L3 Exchange-Leverage Wrapper (parallel-developable unit)
+
+**Status:** READY TO BUILD, independently. Self-contained brief for an autonomous agent
+running **on this host** (`/mnt/dolphinng5_predict`, no git remote). Python
+`/home/dolphin/siloqy_env/bin/python3`. Branch `exp/pink-ditav2-sprint0-20260530`.
+This is **V3.5** of `VIOLET_DEV_SPEC_AND_PLAN.md`. Develop in parallel with V3.4
+(DecisionEngine↔Sizing integration, owned by the lead) — **zero file overlap.**
+
+## 1. Objective
+
+Wrap BLUE's conviction→exchange-leverage mapping into a V-TYPES-bounded VIOLET L3
+component, **bit-identical** to the real mapping in `prod/bingx/leverage.py`. This is the
+"tradeability" side of the dual-leverage: the bet-sizer's internal **conviction leverage**
+[0.5, 9.0] sizes the QUANTITY; the **exchange leverage** [1, 3] (BingX integer, conservatively
+capped) is derived from it at the venue boundary. VIOLET needs a typed, observable wrapper
+for this so V4 (execution) can set venue leverage faithfully.
+
+## 2. Non-negotiable constraints
+
+- **WRAP, DON'T REIMPLEMENT.** Call the real `prod/bingx/leverage.py` functions; do not
+  re-derive the linear map / rounding. Bit-identity is the gate.
+- **ZERO edits to shared files** (`prod/bingx/leverage.py`, `prod/nautilus_event_trader.py`,
+  `prod/clean_arch/dita_v2/*`, `nautilus_dolphin/**`, `blue_parity.py`). Per-commit:
+  `git diff --cached --name-only` must not contain them.
+- **V-TYPES** (`prod/clean_arch/violet/domain.py`): refined types at boundaries, `@typed`
+  (beartype) on public methods, `StrictModel` value objects. Only poison guards
+  (finite + in-domain); **NO arbitrary magnitude caps BLUE/leverage.py lacks** (a prior
+  reviewer flagged exactly this liberty in the sizing layer — do not repeat it).
+- **Exchange-agnostic naming preserved**: this is the L3 boundary; keep the internal
+  (conviction) vs exchange distinction explicit in types and field names.
+
+## 3. The wrap target (authoritative — `prod/bingx/leverage.py`, 83 lines, no callers)
+
+Constants: `CONVICTION_MIN=0.5`, `CONVICTION_MAX=9.0`, `EXCHANGE_LEV_MIN=1`,
+`EXCHANGE_LEV_MAX=3`, `LEVERAGE_MAPPING_RULE="round_half_even_linear_0.5_to_9.0_to_1_to_exchange_cap"`.
+
+Functions to wrap (exact signatures):
+```
+def normalize_bingx_leverage_value(leverage, *, exchange_min=EXCHANGE_LEV_MIN,
+                                   exchange_max=EXCHANGE_LEV_MAX) -> int
+    # ROUND_HALF_EVEN(leverage) clamped to [exchange_min, exchange_max] (BingX int-only)
+def map_internal_conviction_to_exchange_leverage_target(internal, *, exchange_min=..,
+                                   exchange_max=..) -> float
+    # clamp internal to [0.5,9.0]; linear: exch_min + (internal-0.5)/(9.0-0.5) * (exch_max-exch_min)
+def map_internal_conviction_to_exchange_leverage(internal, *, exchange_min=.., exchange_max=..) -> int
+    # = normalize_bingx_leverage_value(map_..._target(internal), ...)  -> final integer exchange leverage
+```
+**Behaviours that MUST round-trip bit-identically:** the [0.5,9.0] clamp of out-of-range
+conviction; the linear interpolation; **ROUND_HALF_EVEN** (banker's rounding — x.5 cases
+round to even, e.g. 1.5→2, 2.5→2); the integer clamp to [exchange_min, exchange_max];
+non-default `exchange_min/max` args.
+
+## 4. Deliverable — files to CREATE
+
+### 4.1 `prod/clean_arch/violet/exchange_leverage.py`
+
+- **Refined types** (V-TYPES, in this file or import from domain): reuse
+  `ConvictionLeverage` from `alpha_wrappers.py` (Annotated float gt/ge 0). New:
+  `ExchangeLeverage = Annotated[int, Field(ge=1)]` (BingX integer; NO upper-cap liberty —
+  the function clamps to exchange_max itself).
+- **`ExchangeLeverageDecision(StrictModel)`**: `internal_conviction: ConvictionLeverage`,
+  `target_exchange_leverage: float` (the pre-round target, allow_inf_nan=False),
+  `exchange_leverage: ExchangeLeverage` (final int), `exchange_min: int`, `exchange_max: int`.
+  Frozen + extra=forbid.
+- **`VioletExchangeLeverage`** class:
+  - `_import_leverage()`: import `prod.bingx.leverage` (it's an in-repo module; plain
+    `from prod.bingx import leverage` should work since cwd is the root — verify; no
+    nautilus_dolphin root-injection needed).
+  - `__init__(self, *, exchange_min=1, exchange_max=3)`: store caps (defaults = gold/BingX).
+  - `@typed map_target(self, internal_conviction: float) -> float`: wraps
+    `map_internal_conviction_to_exchange_leverage_target`.
+  - `@typed normalize(self, leverage: float) -> int`: wraps `normalize_bingx_leverage_value`.
+  - `@typed to_exchange(self, internal_conviction: float) -> ExchangeLeverageDecision`:
+    wraps `map_internal_conviction_to_exchange_leverage`, returns the full decision
+    (target + final + caps) for traceability.
+- Module docstring: the dual-leverage doctrine (conviction sizes quantity; exchange leverage
+  derived at venue boundary), cite `FRACTIONAL_LEVERAGE_TO_BINGX_FIX.md` and
+  `VIOLET_V3_FINDINGS.md §2`.
+
+### 4.2 `prod/clean_arch/violet/test_violet_exchange_leverage.py`
+
+Mirror the test patterns in `test_violet_sizing.py` / `test_violet_modulation.py`
+(hypothesis + drift-guards + `@pytest.mark.gate`). Required tests:
+
+**Unit:**
+- defaults: `exchange_min=1`, `exchange_max=3`; constants match leverage.py (drift-guard:
+  import leverage.py and assert `CONVICTION_MIN/MAX/EXCHANGE_LEV_MIN/MAX` equal the wrapper's).
+- conviction 0.5 → target 1.0; conviction 9.0 → target 3.0 (endpoints).
+- **ROUND_HALF_EVEN boundary cases**: craft convictions whose target lands on x.5 and assert
+  the final int matches banker's rounding (e.g. target 1.5 → 2, 2.5 → 2). Compute the exact
+  conviction that yields target=1.5/2.5 from the linear map and verify.
+- out-of-range conviction (`<0.5`, `>9.0`, and the sizing extremes up to 9) clamps like BLUE.
+- non-default `exchange_max` (e.g. 5, 9) flows through.
+- `ExchangeLeverageDecision` frozen (pydantic raises on mutate).
+
+**Property (hypothesis):**
+- `@given` conviction ∈ floats[-5, 64] (incl. out-of-range), exchange_max ∈ ints[1,9]:
+  wrapper output `==` leverage.py output **exactly** (this is unit-level bit-identity);
+  final exchange_leverage ∈ [exchange_min, exchange_max] and is an int.
+
+**Gate (`@pytest.mark.gate`):**
+- `test_gate_exchange_leverage_bit_identity`: **N≥1e6** Monte-Carlo over the joint space
+  (conviction ∈ uniform[-1, 64] to hammer clamping + the full sizing range; exchange_min ∈
+  {1}, exchange_max ∈ {1,2,3,5,9}). Assert VIOLET `to_exchange(...).exchange_leverage` and
+  `.target_exchange_leverage` are **float/int-for-float `==`** to the real leverage.py
+  functions across every sample. `np.count_nonzero(blue != violet) == 0`. Write a gate
+  report to `prod/VIOLET_dev/reports/violet_v3_exchange_leverage_<ts>.json` (mirror
+  `_write_gate_report` in `test_violet_sizing.py`).
+
+## 5. Validation gate (BINDING)
+
+1. **MC bit-identity** (§4.2 gate) at N≥1e6, exact `==`, joint conviction×exchange-cap space
+   incl. out-of-domain conviction (clamp coverage) and x.5 rounding boundaries.
+2. Full non-gate suite green; **shared-files-clean**; the import of `prod.bingx.leverage`
+   resolves on-host.
+
+## 6. Acceptance criteria
+
+- `exchange_leverage.py` + `test_violet_exchange_leverage.py` created (no other files touched).
+- MC bit-identity gate: 0/1e6 mismatches.
+- Unit + property tests green; ROUND_HALF_EVEN boundary explicitly tested.
+- `git diff --cached --name-only` ∌ any shared file.
+- Commit message documents: wrap target, bit-identity result, V-TYPES boundary.
+
+## 7. Watch-outs (learned from the sizing review)
+
+- **No arbitrary magnitude caps** in the V-TYPES aliases (no `le=64`-style ceilings) — only
+  what leverage.py itself enforces. The function clamps; the type must not double-guard with
+  a value BLUE/leverage.py would accept.
+- **ROUND_HALF_EVEN ≠ round-half-up.** `2.5 → 2`, not 3. Test the even-rounding explicitly.
+- Bit-identity here is "trivial" by design (you call the real function) — that's the point:
+  the gate proves the V-TYPES boundary + arg passing perturb nothing. Do NOT use `approx`.
+- `target` (float, pre-round) and `exchange_leverage` (int, post-round) are BOTH part of the
+  contract — journal/return both.
+
+## 8. Integration (lead will wire; agent need not)
+
+The lead integrates `VioletExchangeLeverage` at the L3/exec boundary in V4 (venue
+leverage-set), consuming `VioletSizer`'s conviction output. The agent's deliverable is the
+standalone, gated component + tests. Hand back: the two files + the gate report path.
+
+## 9. References
+
+`prod/bingx/leverage.py` (target) · `prod/docs/FRACTIONAL_LEVERAGE_TO_BINGX_FIX.md`
+(dual-leverage origin) · `VIOLET_DEV_SPEC_AND_PLAN.md` (V3.5) · `VIOLET_V3_FINDINGS.md §2`
+(dual-leverage) · pattern refs: `prod/clean_arch/violet/modulation.py`,
+`test_violet_sizing.py` (gate + `_write_gate_report` style), `domain.py` (V-TYPES).