diff --git a/prod/docs/VIOLET_DEV_SPEC_AND_PLAN.md b/prod/docs/VIOLET_DEV_SPEC_AND_PLAN.md new file mode 100644 index 0000000..8458225 --- /dev/null +++ b/prod/docs/VIOLET_DEV_SPEC_AND_PLAN.md @@ -0,0 +1,128 @@ +# VIOLET — Master Dev Spec & Plan + +**Authoritative consolidated plan.** Supersedes the scattered plan files +(`~/.claude/plans/harmonic-jumping-plum.md` [V2], `drifting-knitting-zebra.md` [V3]). +Repo cwd `/mnt/dolphinng5_predict` (git root, **no remote — local-only, build on-host**). +Branch `exp/pink-ditav2-sprint0-20260530`. Python `/home/dolphin/siloqy_env/bin/python3`. +Last updated 2026-06-15. + +Cross-refs: `VIOLET_V3_FINDINGS.md` (study + 5-factor composition §8b + vision §8c), +`VIOLET_BUILD_SPEC__SIZING_PARITY.md` (+ Annex A dev log), `FROZEN_ALGO_SPEC_GOLD_REFERENCE.md`, +memory `violet_subsecond_rebuild_plan` / `violet_v3_alpha_doctrine` / `blue_margin_envelope_study`. + +--- + +## 1. Mission + +Rebuild the DOLPHIN trading system (BLUE's live Alpha Engine) onto a sub-second +event-driven reactor substrate — **bit-for-bit faithful to BLUE's alpha**, but on a +chassis that is type-safe, observable, attributable, and trustworthy (the original's +alpha is fund-grade; its bookkeeping is not). Stage the rebuild V0→V6; each stage is +existing/wrapped code + new wiring, gated. + +## 2. Binding doctrine + +- **Model BLUE, not PINK.** Reference = BLUE's LIVE Alpha Engine (`prod/nautilus_event_trader.py` + + `nautilus_dolphin/nautilus_dolphin/nautilus/*`). Behavioural/distributional fidelity. +- **Live BLUE code is the sole doctrine**; where it diverges from any doc/spec, BLUE wins. + `blue_parity.py` is a PINK-era distillation — reference only, validate don't trust. +- **WRAP, DON'T REIMPLEMENT.** Run BLUE's real kernels; transcribe only trivial deterministic + arithmetic, and prove it with **Monte-Carlo → bit-identity → upstream** gates. +- **Follow BLUE in all regards** — no filters/hygiene/bounds BLUE lacks (only V-TYPES poison + rejection: finite + non-negative where BLUE guarantees it). +- **V-TYPES** (`prod/clean_arch/violet/domain.py`): refined types at boundaries, `@typed` + (beartype), `StrictModel` value objects, reject-at-source. Motivated by the bars_held=-106 + poison incident. +- **Reactor substrate, NOT a scan clone.** BLUE's scan-quantized behaviour is hosted on the + V0 reactor and quantized at **Q=scan initially**; per-action Q loosenable later (cadence + control plane). Scan-driven-ness is a quantization setting, not the architecture. +- **Decision layer is slot-independent** — VIOLET decides every scan; the slot only gates + trades (execution layer). Different layers. +- **DARK until keys** — no orders until operator provisions VST keys; ObserveOnlyVenue guard. + +## 3. Stage ladder (V0→V6) + +| Stage | Scope | Status | +|---|---|---| +| **V0** | reactor clock + DeadlineScheduler + harness | ✅ shipped (latency gate passed) | +| **V1** | observe-only DARK service + divergence monitor + CH DDLs | ✅ shipped | +| **V2** | ExecDeadlineDriver @100ms TTL + ScriptedVenue + V-TYPES domain | ✅ shipped (gate passed) | +| **V3** | DecisionEngine SHADOW (models BLUE) | ✅ shipped | +| ↳ V3a | alpha_wrappers (selector/sizer/exit-v7) | ✅ | +| ↳ V3b | cadence control plane (per-action Q) | ✅ | +| ↳ V3c | VioletDecisionEngine (reactor-resident shadow) | ✅ | +| ↳ V3d | base-sizer parity gate | ✅ | +| ↳ V3e | shadow journal + DDL + launcher wiring | ✅ | +| ↳ V3.1 | BLUE stablecoin exclusion (parity fix) | ✅ | +| ↳ V3.2 | EsoF size-modulation fold | ✅ | +| ↳ V3.3 | **full sizing parity** (orchestrator wrap-all, 5-factor, bit-identity) | ✅ reviewed+committed | +| **V3.4** | wire VioletSizer into VioletDecisionEngine (full conviction in shadow) | ⏳ NEXT (mine) | +| **V3.5** | L3 exchange-leverage wrapper (conviction→exchange, bit-identity vs leverage.py) | ⏳ parallel-able (see SUB_SPEC) | +| **V4** | execution ON — single asset, conservative caps, VST testnet → mainnet | ⏳ blocked on keys + V3.4/3.5 | +| **V5** | IRP multi-asset + sizer | later | +| **V6** | full bible layers (posture/vol refinements) + sub-second SL guard | later | + +## 4. What's shipped — code map (`prod/clean_arch/violet/`) + +`clock.py` (V0) · `harness.py` (V0) · `domain.py` (V-TYPES) · `divergence.py` (V1) · +`observe_guard.py` (V1) · `cadence.py` (V3b) · `alpha_wrappers.py` (V3a: VioletBetSizer/ +VioletAssetSelector/VioletExitEngine) · `decision_engine.py` (V3c: VioletDecisionEngine ++ STABLECOIN_SYMBOLS) · `parity_harness.py` (V3d) · `shadow_journal.py` (V3e) · +`modulation.py` (V3.2: VioletSizeModulation EsoF fold) · `sizing.py` (V3.3: **VioletSizer** +— the full 5-factor conviction). DDLs in `prod/clickhouse/violet/`. Launcher +`prod/launch_dolphin_violet.py`. Tests: `test_violet_*.py` (V0–V3.3, incl. bit-identity gates). + +## 5. The full sizing composition (authoritative) + +BLUE's conviction = five multipliers on the base cubic, composed in `esf_alpha_orchestrator` +`:600-619` (see `VIOLET_V3_FINDINGS.md §8b`). `VioletSizer.compose` reproduces it bit-for-bit: +``` +raw = base_leverage × dc_lev_mult × regime_size_mult[ACB_boost×(1+β·s³)×mc_scale] × market_ob_mult × esof_mult +clamped_max = min(base_max(8) × regime × ob × esof, abs_max(9)); STALKER → min(·,2.0) +leverage = max(min_leverage, min(raw, clamped_max)); notional = capital × fraction × leverage +``` +Verified: 1e6 MC + 200k extreme + real-`_try_entry` bit-identity (0 mismatches). Operator's +two recalled "factors aside ACBv6" = `dc_lev_mult` (DC boost) + `market_ob_mult` (OB consensus). + +## 6. Immediate next (V3.4 — mine) + parallelizable (V3.5 — agent) + +**V3.4 (I take this):** integrate `VioletSizer` (V3.3) into `VioletDecisionEngine` (V3c) so +the shadow journal records the FULL conviction (currently base-only). Wire the live factor +inputs: ACB boost/beta (`AdaptiveCircuitBreaker.get_dynamic_boost_from_hz`), mc_scale, +esof_score, OB market consensus, dc_status, posture — sourced alongside the scan in the +launcher's shadow path. Extend `shadow_journal` / `violet_decisions` DDL with the breakdown. +Re-soak DARK, compare full-conviction shadow vs BLUE trades. + +**V3.5 (PARALLEL — independent agent):** L3 exchange-leverage wrapper — isolated, additive, +bit-identity-gated against `prod/bingx/leverage.py`. Full sub-spec: +`VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md`. No overlap with V3.4 (different file/concern). + +## 7. Long-horizon (post-V4, after live testnet→mainnet) + +Vision in `VIOLET_V3_FINDINGS.md §8c`: pure-dataflow-DAG → compile (separation-of-concerns +AND FPGA-purity, bridged by bit-identity); VIBRISS millions-of-instances banditry; **LONG +alpha** as a new pure lane; **DISTRACK** (memory-constant streaming distributions — the +state-side enabler) sequenced AFTER VIOLET trades live. + +## 8. Open TODOs (memory `violet_v3_alpha_doctrine`) + +(a) review `FLAT_VEL_DIV_BUGFIX_CRITICAL.md` + the out-of-range-vel_div-signal research doc; +(b) **fix Argos** (MCP disconnected all session — Grep/Read fallback in use); +(c) DISTRACK (after live); plus the V3.3-review minor: `size()` recomputes clamp/raw outside +`compose()` (DRY — could drift); episode-collapse trade-granularity comparison (when VIOLET +has exec facilities); base-fraction sizing study (`VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md`). + +## 9. Operational notes + +- **Soak control:** `supervisorctl -c prod/supervisor/dolphin-supervisord.conf {start|stop} dolphin_violet`. + Stop is graceful (stopwaitsecs=30, stopasgroup). Shadow on via `DOLPHIN_VIOLET_DECISION_SHADOW=1` + in the conf env (currently set; soaker STOPPED 2026-06-15 per hygiene). **Leave no soaker + running unattended — bring down gracefully.** +- **ClickHouse:** `http://localhost:8123`, user `dolphin` / key `dolphin_ch_2026`. BLUE data + in db `dolphin` (trade_events, eigen_scans, maras_fingerprint, exf_data, v7_decision_events); + VIOLET in db `dolphin_violet`. +- **Eigenvalues data** (for live ACB): `/mnt/dolphin_training/data/eigenvalues` (auto-resolved). +- **Gate reports:** `prod/VIOLET_dev/reports/`. +- **Shared files — NEVER edit:** `prod/nautilus_event_trader.py`, `prod/clean_arch/dita_v2/*`, + `prod/clean_arch/dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`. Mechanical + per-commit check: `git diff --cached --name-only` ∌ those. diff --git a/prod/docs/VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md b/prod/docs/VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md new file mode 100644 index 0000000..752a854 --- /dev/null +++ b/prod/docs/VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md @@ -0,0 +1,147 @@ +# VIOLET Sub-Spec — L3 Exchange-Leverage Wrapper (parallel-developable unit) + +**Status:** READY TO BUILD, independently. Self-contained brief for an autonomous agent +running **on this host** (`/mnt/dolphinng5_predict`, no git remote). Python +`/home/dolphin/siloqy_env/bin/python3`. Branch `exp/pink-ditav2-sprint0-20260530`. +This is **V3.5** of `VIOLET_DEV_SPEC_AND_PLAN.md`. Develop in parallel with V3.4 +(DecisionEngine↔Sizing integration, owned by the lead) — **zero file overlap.** + +## 1. Objective + +Wrap BLUE's conviction→exchange-leverage mapping into a V-TYPES-bounded VIOLET L3 +component, **bit-identical** to the real mapping in `prod/bingx/leverage.py`. This is the +"tradeability" side of the dual-leverage: the bet-sizer's internal **conviction leverage** +[0.5, 9.0] sizes the QUANTITY; the **exchange leverage** [1, 3] (BingX integer, conservatively +capped) is derived from it at the venue boundary. VIOLET needs a typed, observable wrapper +for this so V4 (execution) can set venue leverage faithfully. + +## 2. Non-negotiable constraints + +- **WRAP, DON'T REIMPLEMENT.** Call the real `prod/bingx/leverage.py` functions; do not + re-derive the linear map / rounding. Bit-identity is the gate. +- **ZERO edits to shared files** (`prod/bingx/leverage.py`, `prod/nautilus_event_trader.py`, + `prod/clean_arch/dita_v2/*`, `nautilus_dolphin/**`, `blue_parity.py`). Per-commit: + `git diff --cached --name-only` must not contain them. +- **V-TYPES** (`prod/clean_arch/violet/domain.py`): refined types at boundaries, `@typed` + (beartype) on public methods, `StrictModel` value objects. Only poison guards + (finite + in-domain); **NO arbitrary magnitude caps BLUE/leverage.py lacks** (a prior + reviewer flagged exactly this liberty in the sizing layer — do not repeat it). +- **Exchange-agnostic naming preserved**: this is the L3 boundary; keep the internal + (conviction) vs exchange distinction explicit in types and field names. + +## 3. The wrap target (authoritative — `prod/bingx/leverage.py`, 83 lines, no callers) + +Constants: `CONVICTION_MIN=0.5`, `CONVICTION_MAX=9.0`, `EXCHANGE_LEV_MIN=1`, +`EXCHANGE_LEV_MAX=3`, `LEVERAGE_MAPPING_RULE="round_half_even_linear_0.5_to_9.0_to_1_to_exchange_cap"`. + +Functions to wrap (exact signatures): +``` +def normalize_bingx_leverage_value(leverage, *, exchange_min=EXCHANGE_LEV_MIN, + exchange_max=EXCHANGE_LEV_MAX) -> int + # ROUND_HALF_EVEN(leverage) clamped to [exchange_min, exchange_max] (BingX int-only) +def map_internal_conviction_to_exchange_leverage_target(internal, *, exchange_min=.., + exchange_max=..) -> float + # clamp internal to [0.5,9.0]; linear: exch_min + (internal-0.5)/(9.0-0.5) * (exch_max-exch_min) +def map_internal_conviction_to_exchange_leverage(internal, *, exchange_min=.., exchange_max=..) -> int + # = normalize_bingx_leverage_value(map_..._target(internal), ...) -> final integer exchange leverage +``` +**Behaviours that MUST round-trip bit-identically:** the [0.5,9.0] clamp of out-of-range +conviction; the linear interpolation; **ROUND_HALF_EVEN** (banker's rounding — x.5 cases +round to even, e.g. 1.5→2, 2.5→2); the integer clamp to [exchange_min, exchange_max]; +non-default `exchange_min/max` args. + +## 4. Deliverable — files to CREATE + +### 4.1 `prod/clean_arch/violet/exchange_leverage.py` + +- **Refined types** (V-TYPES, in this file or import from domain): reuse + `ConvictionLeverage` from `alpha_wrappers.py` (Annotated float gt/ge 0). New: + `ExchangeLeverage = Annotated[int, Field(ge=1)]` (BingX integer; NO upper-cap liberty — + the function clamps to exchange_max itself). +- **`ExchangeLeverageDecision(StrictModel)`**: `internal_conviction: ConvictionLeverage`, + `target_exchange_leverage: float` (the pre-round target, allow_inf_nan=False), + `exchange_leverage: ExchangeLeverage` (final int), `exchange_min: int`, `exchange_max: int`. + Frozen + extra=forbid. +- **`VioletExchangeLeverage`** class: + - `_import_leverage()`: import `prod.bingx.leverage` (it's an in-repo module; plain + `from prod.bingx import leverage` should work since cwd is the root — verify; no + nautilus_dolphin root-injection needed). + - `__init__(self, *, exchange_min=1, exchange_max=3)`: store caps (defaults = gold/BingX). + - `@typed map_target(self, internal_conviction: float) -> float`: wraps + `map_internal_conviction_to_exchange_leverage_target`. + - `@typed normalize(self, leverage: float) -> int`: wraps `normalize_bingx_leverage_value`. + - `@typed to_exchange(self, internal_conviction: float) -> ExchangeLeverageDecision`: + wraps `map_internal_conviction_to_exchange_leverage`, returns the full decision + (target + final + caps) for traceability. +- Module docstring: the dual-leverage doctrine (conviction sizes quantity; exchange leverage + derived at venue boundary), cite `FRACTIONAL_LEVERAGE_TO_BINGX_FIX.md` and + `VIOLET_V3_FINDINGS.md §2`. + +### 4.2 `prod/clean_arch/violet/test_violet_exchange_leverage.py` + +Mirror the test patterns in `test_violet_sizing.py` / `test_violet_modulation.py` +(hypothesis + drift-guards + `@pytest.mark.gate`). Required tests: + +**Unit:** +- defaults: `exchange_min=1`, `exchange_max=3`; constants match leverage.py (drift-guard: + import leverage.py and assert `CONVICTION_MIN/MAX/EXCHANGE_LEV_MIN/MAX` equal the wrapper's). +- conviction 0.5 → target 1.0; conviction 9.0 → target 3.0 (endpoints). +- **ROUND_HALF_EVEN boundary cases**: craft convictions whose target lands on x.5 and assert + the final int matches banker's rounding (e.g. target 1.5 → 2, 2.5 → 2). Compute the exact + conviction that yields target=1.5/2.5 from the linear map and verify. +- out-of-range conviction (`<0.5`, `>9.0`, and the sizing extremes up to 9) clamps like BLUE. +- non-default `exchange_max` (e.g. 5, 9) flows through. +- `ExchangeLeverageDecision` frozen (pydantic raises on mutate). + +**Property (hypothesis):** +- `@given` conviction ∈ floats[-5, 64] (incl. out-of-range), exchange_max ∈ ints[1,9]: + wrapper output `==` leverage.py output **exactly** (this is unit-level bit-identity); + final exchange_leverage ∈ [exchange_min, exchange_max] and is an int. + +**Gate (`@pytest.mark.gate`):** +- `test_gate_exchange_leverage_bit_identity`: **N≥1e6** Monte-Carlo over the joint space + (conviction ∈ uniform[-1, 64] to hammer clamping + the full sizing range; exchange_min ∈ + {1}, exchange_max ∈ {1,2,3,5,9}). Assert VIOLET `to_exchange(...).exchange_leverage` and + `.target_exchange_leverage` are **float/int-for-float `==`** to the real leverage.py + functions across every sample. `np.count_nonzero(blue != violet) == 0`. Write a gate + report to `prod/VIOLET_dev/reports/violet_v3_exchange_leverage_.json` (mirror + `_write_gate_report` in `test_violet_sizing.py`). + +## 5. Validation gate (BINDING) + +1. **MC bit-identity** (§4.2 gate) at N≥1e6, exact `==`, joint conviction×exchange-cap space + incl. out-of-domain conviction (clamp coverage) and x.5 rounding boundaries. +2. Full non-gate suite green; **shared-files-clean**; the import of `prod.bingx.leverage` + resolves on-host. + +## 6. Acceptance criteria + +- `exchange_leverage.py` + `test_violet_exchange_leverage.py` created (no other files touched). +- MC bit-identity gate: 0/1e6 mismatches. +- Unit + property tests green; ROUND_HALF_EVEN boundary explicitly tested. +- `git diff --cached --name-only` ∌ any shared file. +- Commit message documents: wrap target, bit-identity result, V-TYPES boundary. + +## 7. Watch-outs (learned from the sizing review) + +- **No arbitrary magnitude caps** in the V-TYPES aliases (no `le=64`-style ceilings) — only + what leverage.py itself enforces. The function clamps; the type must not double-guard with + a value BLUE/leverage.py would accept. +- **ROUND_HALF_EVEN ≠ round-half-up.** `2.5 → 2`, not 3. Test the even-rounding explicitly. +- Bit-identity here is "trivial" by design (you call the real function) — that's the point: + the gate proves the V-TYPES boundary + arg passing perturb nothing. Do NOT use `approx`. +- `target` (float, pre-round) and `exchange_leverage` (int, post-round) are BOTH part of the + contract — journal/return both. + +## 8. Integration (lead will wire; agent need not) + +The lead integrates `VioletExchangeLeverage` at the L3/exec boundary in V4 (venue +leverage-set), consuming `VioletSizer`'s conviction output. The agent's deliverable is the +standalone, gated component + tests. Hand back: the two files + the gate report path. + +## 9. References + +`prod/bingx/leverage.py` (target) · `prod/docs/FRACTIONAL_LEVERAGE_TO_BINGX_FIX.md` +(dual-leverage origin) · `VIOLET_DEV_SPEC_AND_PLAN.md` (V3.5) · `VIOLET_V3_FINDINGS.md §2` +(dual-leverage) · pattern refs: `prod/clean_arch/violet/modulation.py`, +`test_violet_sizing.py` (gate + `_write_gate_report` style), `domain.py` (V-TYPES).