Files
siloqy/prod/docs/VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md

86 lines
5.7 KiB
Markdown
Raw Normal View History

VIOLET V3.4c review: fix mc_scale BLUE-parity bug + document re-derivation debt REVIEW of the V3.4c/d/e work (other agents) for BLUE-algo compliance found one real bug and one architectural concern. BUG FIXED — mc_scale derivation (live_blue_source.py): The adapter mapped `mc_forewarner_latest.status == "ORANGE"` → 0.5. But BLUE's sizing mc_scale is NOT the MC service's `status` label (that label, from mc_forewarner_flow.py, uses GREEN<0.10/ORANGE<0.30/RED — observability only). The live trader re-derives the haircut in begin_day (esf_alpha_orchestrator.py:956-962) from the SAME published fields `catastrophic_prob` + `envelope_score` with DIFFERENT thresholds: mc_red = cat>0.25 or env<-1.0 mc_orange = (not mc_red) and (env<0 or cat>0.10) mc_scale = 0.5 if mc_orange else 1.0 The two disagree (e.g. cat=0.05/env=-0.5 → label GREEN→1.0 but BLUE orange→0.5; cat=0.28 → label ORANGE→0.5 but BLUE red→1.0). Rewrote `_derive_mc_scale` to mirror begin_day exactly on the source fields. Per operator: there is genuine ambiguity over which surface is "nominal" — we go with the SOURCE FIELDS + begin_day formula (the path that actually drives BLUE's sizing); the ambiguity + decision are noted in the docstring. TESTS — the old fixtures were complicit: they fed `{"status":"ORANGE"}`, a payload shape BLUE never emits, so they "passed" against fiction. Replaced with BLUE's real payload (`catastrophic_prob`/`envelope_score`) and added a parametrized formula test covering the exact divergence cases the status-based code got wrong, plus missing/garbage-field neutral. 15 passed (live-HZ smoke deselected). ARCHITECTURAL DEBT documented — VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md: VIOLET imitates BLUE's computations in a DIFFERENT module/scope structure. Kernels are WRAPPED (safe), but orchestration arithmetic (compose/regime/ob/strength/mc_scale) is HAND-REPLICATED out of the monolithic NDAlphaEngine — making orderly, verifiable BLUE↔VIOLET parity comparison and refactoring hard. Doc inventories every re-derivation with BLUE's file:line authority, the drift risk, and mitigations (parity-pin tests; single canonical surface per ambiguous factor; eventual DITAv2 Rust-backplane convergence). Pointer added to live_blue_source.py's module docstring. Also reviewed (no change needed): launcher launch_dolphin_violet.py is DARK-safe (ObserveOnlyVenue + shadow gated default-OFF, no order path); shadow_live_factors.py glue is correct; trade_slot_compare.py reimplements no BLUE algo. fb34431's launcher tests — which the authoring agent never ran on the slow mount — pass (14 green). violet-only; no shared-file edits. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 16:54:35 +02:00
# VIOLET ↔ BLUE parity: structural divergence & re-derivation debt
**Date:** 2026-06-16
**Status:** ACKNOWLEDGED TRADEOFF / open architectural debt
**Raised by:** operator, during the V3.4c review.
## The problem, stated plainly
VIOLET reproduces BLUE's sizing behaviour **bit-for-bit by intent**, but it does so in a
**different module / file / scope structure** than BLUE. BLUE's logic lives in one place —
the monolithic `NDAlphaEngine` (`nautilus_dolphin/nautilus_dolphin/nautilus/esf_alpha_orchestrator.py`),
which holds day-state (`_day_base_boost`, `_day_beta`, `_day_mc_scale`, `_day_posture`),
constructs its own `signal_gen`/`bet_sizer`, and runs `begin_day` / `_try_entry` inline.
VIOLET re-expresses that same logic spread across:
`sizing.py`, `live_blue_source.py`, `live_factors.py`, `live_factor_source.py`,
`decision_engine.py`, `alpha_wrappers.py`.
**Consequence (the operator's concern, verbatim intent):** because VIOLET imitates the
computations *while* ending up with a different structure, any *orderly, systemic,
verifiable* BLUE↔VIOLET algo parity comparison — and any future refactor of either side —
is **much harder**. The surfaces do not line up 1:1, so a diff between the two engines is
not mechanical; it requires a human to know which VIOLET fragment mirrors which BLUE line.
## Two kinds of reuse — and only one is safe
1. **WRAPPED kernels (safe — single source of truth).** VIOLET imports and calls BLUE's
actual kernel objects. A BLUE change propagates automatically.
- `esof_size_mult_from_score`, `parse_esof_payload`, `esof_score_from_payload`
(`esof_size_gate.py`) — wrapped by `sizing.py` / `live_factor_source.py`.
- `OBFeatureEngine.get_market` (`ob_features.py`) — wrapped by `live_blue_source.py`.
- `AlphaSignalGenerator.generate` (`alpha_signal_generator.py`) — wrapped by `live_blue_source.py`.
- `AlphaAssetSelector` / `AlphaBetSizer` — wrapped by `alpha_wrappers.py`.
- `map_internal_conviction_to_exchange_leverage` (`bingx/leverage.py`) — wrapped by `exchange_leverage.py`.
2. **HAND-REPLICATED arithmetic (the debt — duplicated formulas, drift-prone).** VIOLET
transcribes BLUE's pure float arithmetic into its own functions. A BLUE change here is
SILENT in VIOLET until someone notices.
## Re-derivation inventory (the drift liabilities)
| Computation | BLUE authority (file:line) | VIOLET replica | Parity safety-net today | Drift risk |
|---|---|---|---|---|
| 5-factor compose + caps | `esf_alpha_orchestrator.py:600-619` | `sizing.VioletSizer.compose` | `@gate` Monte-Carlo vs REAL orchestrator (bit-identity) | LOW (gated) |
| `regime_size_mult` = boost·(1+β·s³)·mc | `esf_alpha_orchestrator.py:898-909` | `sizing.VioletSizer.regime_size_mult` | same gate | LOW-MED |
| `strength_cubic` | `esf_alpha_orchestrator.py:872-885` | `sizing.VioletSizer.strength_cubic` | same gate | LOW-MED |
| `market_ob_mult` consensus | `esf_alpha_orchestrator.py:587-595` | `sizing.VioletSizer.market_ob_mult` | same gate | MED |
| `dc_lev_mult` | `esf_alpha_orchestrator.py:575-577` | `sizing.VioletSizer.dc_lev_mult` | unit only | MED (but ≡1.0 while dc_leverage_boost=1.0) |
| **`mc_scale`** | `esf_alpha_orchestrator.py:956-962` (`begin_day`) | `live_blue_source._derive_mc_scale` | **unit only — NO pin to BLUE's fn** | **HIGH** |
| `boost`/`beta` source | trader recompute `acb.get_dynamic_boost_from_hz(exf_latest)` | reads published `DOLPHIN_FEATURES.acb_boost` (acb_processor_service) | none | MED (two surfaces may differ on dynamic-β / OB Sub-4) |
| `dc_status` config | `signal_gen` built with threaded params `:180-191` | `AlphaSignalGenerator()` **bare defaults** | none | MED (cosmetic while dc_lev_mult≡1.0) |
| OB feed shape | live OB accumulation | single-snapshot `HazelcastOBProvider` + `bar_idx=0` | none | MED |
The worst link is **`mc_scale`**: pure duplication of `begin_day`'s thresholds with no
test that pins it to BLUE's actual function (BLUE computes it inline inside `begin_day`,
which is not callable in isolation). The V3.4c bug fixed on 2026-06-16 (the adapter keyed
off the MC service's `status` label instead of BLUE's `cat`/`env` thresholds) is exactly
the failure mode this structure invites.
## Why we accept it (for now)
- The kernels that carry the heavy alpha are WRAPPED, not copied.
- The composition arithmetic IS gated bit-for-bit against the real orchestrator.
- VIOLET must stay a *read-only, DARK* mirror of a *running* BLUE; it cannot import BLUE's
live in-process day-state, so some reconstruction from published HZ surfaces is unavoidable.
## Mitigations (recommended, not yet done)
1. **Parity-pin every hand-replicated formula.** For each row above, add a test that
imports BLUE's authoritative function/constant and asserts VIOLET's replica equals it
over a sampled grid — converting silent drift into a red test. Where BLUE's logic is
trapped inside `begin_day` (mc_scale), refactor a *pure* `mc_scale_from(cat, env)` helper
**on the BLUE side** (BLUE-domain change, operator-gated) that BOTH engines call.
2. **Single ambiguity owner.** Surfaces like MC (`status` label vs `begin_day` thresholds)
and ACB (published `acb_boost` vs trader recompute) have two disagreeing sources; pick
ONE canonical per factor and document it (see `_derive_mc_scale` docstring).
3. **Backplane convergence (the real fix).** When the DITAv2 Rust middleware becomes the
shared backplane, BOTH BLUE and VIOLET should consume factors from it rather than each
computing/replicating — collapsing this divergence at the source. Until then, every new
hand-replication MUST be logged in this table.
## Maintenance rule
Any change to a BLUE formula in the left column REQUIRES a matching change + test update in
the VIOLET replica in the same PR. Any NEW hand-replication MUST add a row here.