DOLPHIN/prod/docs/SYSTEM_GOLD_SPEC_GUIDE.md

SYSTEM GOLD SPEC GUIDE

DOLPHIN NG — D_LIQ_GOLD Production Reference

Canonical document. Last updated: 2026-03-22. Purpose: Exhaustive reference for reproducing D_LIQ_GOLD (ROI=181.81%) from scratch. Covers every layer of the engine stack, every configuration constant, every file path, every known gotcha, and the complete research history that led to the gold standard.

---

TABLE OF CONTENTS

1. Gold Standard Definition
2. How to Reproduce Gold — Step by Step
3. System Architecture Overview
4. Engine Class Hierarchy (MRO)
5. Layer-by-Layer Deep Dive
6. Critical Configuration Constants
7. Data Pipeline — Input Files
8. Test Harness Anatomy
9. Known Bugs and Fixes
10. Research History (Exp1–Exp15)
11. File Map — All Critical Paths
12. Failure Mode Reference

---

1. GOLD STANDARD DEFINITION

D_LIQ_GOLD (production default since 2026-03-15)

ROI   = +181.81%
DD    = 17.65%  (max drawdown over 56-day window)
Calmar = 10.30  (ROI / max_DD)
PF    = ~1.55   (profit factor)
WR    = ~52%    (win rate)
T     = 2155    (EXACT — deterministic, any deviation is a regression)
liq_stops = 1   (0.05% rate — 1 liquidation floor stop out of 2155)
avg_leverage = 4.09x
MC    = 0 RED / 0 ORANGE across all 56 days

Engine class: LiquidationGuardEngine(soft=8x, hard=9x, mc_ref=5x, margin_buffer=0.95, adaptive_beta=True)

Factory function:

from nautilus_dolphin.nautilus.proxy_boost_engine import create_d_liq_engine
engine = create_d_liq_engine(**ENGINE_KWARGS)

Data window: 56 days, 2025-12-31 to 2026-02-26 (5-second scan data)

Baseline comparison (BRONZE regression floor): NDAlphaEngine, no boost: ROI=+88.55%, PF=1.215, DD=15.05%, Sharpe=4.38, T=2155

---

2. HOW TO REPRODUCE GOLD

Prerequisites

• Python 3.11
• SILOQY env or system Python with: numpy, pandas, numba, scipy, sklearn, pyarrow
• VBT parquet cache at C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\vbt_cache\ (56 daily parquet files, 2025-12-31 to 2026-02-26)

Step 1 — Verify the fix is applied

In esf_alpha_orchestrator.py at line ~714, confirm:

def set_esoteric_hazard_multiplier(self, hazard_score: float):
    floor_lev = 3.0
    ceiling_lev = getattr(self, '_extended_soft_cap', 6.0)   # ← MUST use getattr, not 6.0
    ...

If it says ceiling_lev = 6.0 (hardcoded), the fix has NOT been applied. Apply the fix FIRST or all D_LIQ results will be ~145.84% instead of 181.81%. See §9 for full explanation.

Step 2 — Verify leverage state after engine creation

import os; os.environ['NUMBA_DISABLE_JIT'] = '1'  # optional for faster check
from nautilus_dolphin.nautilus.proxy_boost_engine import create_d_liq_engine
from dvae.exp_shared import ENGINE_KWARGS  # load without dvae/__init__ (see §9)

eng = create_d_liq_engine(**ENGINE_KWARGS)
assert eng.base_max_leverage == 8.0
assert eng.abs_max_leverage  == 9.0
assert eng.bet_sizer.max_leverage == 8.0

eng.set_esoteric_hazard_multiplier(0.0)
assert eng.base_max_leverage == 8.0,       "FIX NOT APPLIED — stomp is still active"
assert eng.bet_sizer.max_leverage == 8.0,  "FIX NOT APPLIED — sizer stomped"
print("Leverage state OK")

Step 3 — Run the e2e test suite

cd "C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\nautilus_dolphin"
pytest -m slow tests/test_proxy_boost_production.py -v

Expected: 9/9 PASSED (runtime ~52 minutes)

• test_e2e_baseline_reproduces_gold → ROI=88.55% T=2155
• test_e2e_d_liq_gold_reproduces_exp9b → ROI=181.81% T=2155 DD≤18.15%
• test_e2e_mc_silent_all_days → 0 RED/ORANGE at d_liq leverage
• Plus 6 other mode tests

Step 4 — Run the painstaking trace backtest (for detailed analysis)

cd "C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\nautilus_dolphin"
python dvae/run_trace_backtest.py

Outputs to dvae/trace/:

• tick_trace.csv — one row per bar (~346k rows) — full system state
• trade_trace.csv — one row per trade (~2155 rows)
• daily_trace.csv — one row per day (56 rows)
• summary.json — final ROI/DD/T/Calmar

Quick Gold Reproduction (single-run, no full e2e suite)

cd "C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\nautilus_dolphin"
python dvae/test_dliq_fix_verify.py

Expected output (final 3 lines):

ROI match: ✓ PASS (diff=~0pp)
DD  match: ✓ PASS (diff=~0pp)
T   match: ✓ PASS (got 2155)

---

3. SYSTEM ARCHITECTURE OVERVIEW

┌──────────────────────────────────────────────────────────────┐
│                    DOLPHIN NG Production                     │
│                                                              │
│  VBT Parquet Cache (5s scans, 56 days)                      │
│       ↓                                                      │
│  Data Loader → float64 df + dvol per bar + vol_p60          │
│       ↓                                                      │
│  AdaptiveCircuitBreaker (ACBv6) → day_base_boost + day_beta │
│       ↓                                                      │
│  OBFeatureEngine (MockOBProvider, 48 assets)                │
│       ↓                                                      │
│  MC-Forewarner → day_mc_status (OK/ORANGE/RED)              │
│       ↓                                                      │
│  [process_day LOOP per day]                                  │
│    ├── begin_day() → ACB boost + MC gate                     │
│    ├── FOR EACH BAR:                                         │
│    │     _update_proxy(inst50, v750) → proxy_B              │
│    │     step_bar(vel_div, prices, vol_ok) →                 │
│    │       process_bar() →                                   │
│    │         ENTRY PATH: _try_entry() → size + leverage      │
│    │         EXIT PATH:  exit_manager.evaluate() → reason    │
│    │         _execute_exit() → pnl + trade record            │
│    └── end_day() → daily summary                             │
│       ↓                                                      │
│  trade_history: List[NDTradeRecord] — 2155 records          │
└──────────────────────────────────────────────────────────────┘

Signal Pathway

vel_div (eigenvalue velocity divergence):
  → vel_div < -0.020 (threshold)  → ENTRY signal  (LONG, mean reversion)
  → vel_div < -0.050 (extreme)    → max leverage region
  → vel_div ≥ -0.020              → NO ENTRY (flat or rising eigenspace)

proxy_B = instability_50 - v750_lambda_max_velocity:
  → high proxy_B = eigenspace stress (high MAE risk at entry)
  → percentile rank vs 500-bar history → day_beta
  → day_beta modulates adaptive_beta boost
  → low proxy_B → higher leverage allowed (less stress, better setup)

vol_regime_ok (dvol > vol_p60):
  → bars where BTC realized vol > 60th percentile of dataset
  → filters out ultra-low-vol bars where signal is noisy

---

4. ENGINE CLASS HIERARCHY

NDAlphaEngine                          ← base backtest engine
  └── ProxyBaseEngine                  ← adds proxy_B tracking per bar
        └── AdaptiveBoostEngine        ← scale-boost using proxy_B rank
              └── ExtendedLeverageEngine ← 8x/9x leverage + MC decoupling
                    └── LiquidationGuardEngine ← per-trade liq floor stop
                          = D_LIQ_GOLD ★

MRO (Python method resolution order): LiquidationGuardEngine → ExtendedLeverageEngine → AdaptiveBoostEngine → ProxyBaseEngine → NDAlphaEngine → object

Which class owns which method

Method	Owner class	Notes
process_day()	ProxyBaseEngine	Main backtest loop (begin_day + bar loop + end_day)
step_bar()	NDAlphaEngine (streaming API)	Per-bar state update, calls process_bar()
process_bar()	NDAlphaEngine	Core per-bar logic: entry/exit decision
_try_entry()	LiquidationGuardEngine	Sets _pending_stop_override before calling super
_execute_exit()	LiquidationGuardEngine	Counts liquidation stops, calls super
begin_day()	ExtendedLeverageEngine	MC lever-swap: saves true caps, sets mc_ref, restores after super
begin_day()	NDAlphaEngine (via super chain)	ACB boost, MC-Forewarner assessment
end_day()	NDAlphaEngine	Returns daily summary dict
_update_proxy()	ProxyBaseEngine	Updates _current_proxy_b per bar
set_esoteric_hazard_multiplier()	NDAlphaEngine	BUG FIXED 2026-03-22 — now respects _extended_soft_cap
set_acb()	NDAlphaEngine	Wires AdaptiveCircuitBreaker
set_ob_engine()	NDAlphaEngine	Wires OBFeatureEngine
set_mc_forewarner()	NDAlphaEngine	Wires DolphinForewarner

---

5. LAYER-BY-LAYER DEEP DIVE

Layer 1: NDAlphaEngine (base)

File: nautilus_dolphin/nautilus_dolphin/nautilus/esf_alpha_orchestrator.py

Core engine. All signal generation, position management, exit logic, fee/slippage.

Key parameters (from ENGINE_KWARGS in exp_shared.py):

initial_capital        = 25000.0
vel_div_threshold      = -0.020    # entry trigger: vel_div < this
vel_div_extreme        = -0.050    # extreme signal → max bet size
min_leverage           = 0.5       # minimum bet size multiplier
max_leverage           = 5.0       # base soft cap (overridden by ExtendedLeverageEngine)
leverage_convexity     = 3.0       # power in sizing curve
fraction               = 0.20      # Kelly-style fraction of capital per trade
fixed_tp_pct           = 0.0095    # take profit = 95 basis points
stop_pct               = 1.0       # stop loss at 100% of capital = effectively OFF
max_hold_bars          = 120       # max bars before forced exit (~10 minutes at 5s)
use_direction_confirm  = True      # direction confirmation gate
dc_lookback_bars       = 7         # bars for DC calculation
dc_min_magnitude_bps   = 0.75      # min move magnitude for DC confirmation
dc_skip_contradicts    = True      # skip entries with contradicting DC
dc_leverage_boost      = 1.0       # DC confirms → leverage multiplier
dc_leverage_reduce     = 0.5       # DC contradicts → reduce leverage
use_asset_selection    = True      # IRP-based asset selection (48 assets)
min_irp_alignment      = 0.45      # min fraction of assets aligned for entry
use_sp_fees            = True      # spread + maker fee model
use_sp_slippage        = True      # slippage model
sp_maker_entry_rate    = 0.62      # entry maker rate (bps)
sp_maker_exit_rate     = 0.50      # exit maker rate (bps)
use_ob_edge            = True      # order book edge gate
ob_edge_bps            = 5.0       # min OB edge required (bps)
ob_confirm_rate        = 0.40      # min OB confirm rate
lookback               = 100       # lookback for various rolling windows
use_alpha_layers       = True      # enable all alpha layers
use_dynamic_leverage   = True      # dynamic leverage calculation
seed                   = 42        # RNG seed for determinism

Leverage formula in _try_entry():

clamped_max_leverage = min(
    base_max_leverage * regime_size_mult * market_ob_mult,
    abs_max_leverage
)
raw_leverage = size_result["leverage"] * dc_lev_mult * regime_size_mult * market_ob_mult
leverage = min(raw_leverage, clamped_max_leverage)

Where:

• base_max_leverage: set by engine (5.0 base, 8.0 for D_LIQ)
• regime_size_mult: ACBv6-driven, typically 1.0–1.6
• market_ob_mult: OB-driven multiplier
• abs_max_leverage: hard cap (6.0 base, 9.0 for D_LIQ)

Exit reasons (exit_manager):

• FIXED_TP — take profit at 95bps
• MAX_HOLD — held 120+ bars
• STOP_LOSS — stop triggered (only liquidation guard stop in D_LIQ)
• HIBERNATE_HALT — MC-RED day halt
• OB_TAIL_AVOIDANCE — OB cascade/withdrawal signal (never fires with MockOBProvider)

Layer 2: ProxyBaseEngine

File: nautilus_dolphin/nautilus_dolphin/nautilus/proxy_boost_engine.py

Adds per-bar proxy_B = instability_50 - v750_lambda_max_velocity tracking.

Key attribute: self._current_proxy_b — updated by _update_proxy(inst, v750) before each step_bar() call.

Key method — process_day() loop:

def process_day(self, date_str, df, asset_columns, vol_regime_ok=None, ...):
    self.begin_day(date_str, ...)
    for ri in range(len(df)):
        row = df.iloc[ri]
        vd = row.get('vel_div')
        if vd is None or not np.isfinite(float(vd)): continue
        v50  = gf('v50_lambda_max_velocity')
        v750 = gf('v750_lambda_max_velocity')
        inst = gf('instability_50')
        self._update_proxy(inst, v750)        # ← proxy_B updated HERE
        prices = {ac: float(row[ac]) for ac in asset_columns if ...}
        vrok = bool(vol_regime_ok[ri]) if vol_regime_ok is not None else (bid >= 100)
        self.step_bar(bar_idx=ri, vel_div=float(vd), prices=prices, ...)   # ← bar processed
    return self.end_day()

Layer 3: AdaptiveBoostEngine

File: nautilus_dolphin/nautilus_dolphin/nautilus/proxy_boost_engine.py

Scale-boost mechanism. After super()._try_entry() succeeds, multiplies notional/leverage by a factor determined by proxy_B percentile rank vs 500-bar history.

With adaptive_beta=True (D_LIQ default):

alpha_eff = alpha * (1 + day_beta)     # day_beta from ACBv6 daily amplitude regime
# Lower proxy_B at entry → lower prank → more aggressive boost
# Higher proxy_B → prank near 1.0 → minimal/no boost
scale = max(1.0, 1.0 + alpha_eff * max(0, threshold - prank))

day_beta: ACBv6 outputs a beta value proportional to amplitude regime. High beta → more aggressive days get extra boost. Low beta → conservative days, less amplification.

Layer 4: ExtendedLeverageEngine

File: nautilus_dolphin/nautilus_dolphin/nautilus/proxy_boost_engine.py

Extends the leverage ceiling from 5x/6x to 8x/9x (D_LIQ config).

Critical mechanism — MC decoupling in begin_day():

def begin_day(self, date_str, posture='APEX', direction=None):
    # Save true extended caps
    _true_base  = self.base_max_leverage    # 8.0 for D_LIQ
    _true_abs   = self.abs_max_leverage     # 9.0 for D_LIQ
    _true_sizer = self.bet_sizer.max_leverage  # 8.0 for D_LIQ

    # Temporarily show MC a reference leverage (5.0)
    self.base_max_leverage      = self._mc_leverage_ref   # 5.0
    self.bet_sizer.max_leverage = self._mc_leverage_ref   # 5.0
    self.abs_max_leverage       = self._mc_leverage_ref   # 5.0

    super().begin_day(...)   # MC-Forewarner assesses at 5.0x reference

    # Restore true caps for actual trading
    self.base_max_leverage      = _true_base   # 8.0
    self.bet_sizer.max_leverage = _true_sizer  # 8.0
    self.abs_max_leverage       = _true_abs    # 9.0

Why this matters: Without MC decoupling, at 8x/9x leverage the MC-Forewarner would assess catastrophic risk at 8x and potentially return RED/ORANGE, halting trading. By showing it mc_ref=5.0 (within its trained range), MC stays GREEN every day. Empirically confirmed: 0 RED / 0 ORANGE across all 56 days at any leverage up to 10x.

Layer 5: LiquidationGuardEngine (D_LIQ_GOLD)

File: nautilus_dolphin/nautilus_dolphin/nautilus/proxy_boost_engine.py

Adds a per-trade liquidation floor stop. Before every entry:

def _try_entry(self, bar_idx, vel_div, prices, price_histories, v50_vel=0.0, v750_vel=0.0):
    self._pending_stop_override = self._liq_stop_pct   # = (1/9) * 0.95 = 10.56%
    return super()._try_entry(...)

If price moves >10.56% against the position, stop fires before exchange liquidates.

• With 9x leverage: exchange liquidates at ~11.1% adverse move
• Our stop at 10.56% → exits ~0.56% before exchange force-liquidation
• margin_buffer = 0.95 provides this safety margin

Result: 1 stop triggered across 2155 trades = 0.05% rate (negligible). The guard provides safety without materially impacting returns.

---

6. CRITICAL CONFIGURATION CONSTANTS

ENGINE_KWARGS (test harness gold standard)

File: nautilus_dolphin/dvae/exp_shared.py line 56–67

ENGINE_KWARGS = dict(
    initial_capital=25000.0, vel_div_threshold=-0.02, vel_div_extreme=-0.05,
    min_leverage=0.5, max_leverage=5.0, leverage_convexity=3.0,
    fraction=0.20, fixed_tp_pct=0.0095, stop_pct=1.0, max_hold_bars=120,
    use_direction_confirm=True, dc_lookback_bars=7, dc_min_magnitude_bps=0.75,
    dc_skip_contradicts=True, dc_leverage_boost=1.0, dc_leverage_reduce=0.5,
    use_asset_selection=True, min_irp_alignment=0.45,
    use_sp_fees=True, use_sp_slippage=True,
    sp_maker_entry_rate=0.62, sp_maker_exit_rate=0.50,
    use_ob_edge=True, ob_edge_bps=5.0, ob_confirm_rate=0.40,
    lookback=100, use_alpha_layers=True, use_dynamic_leverage=True, seed=42,
)

Note: max_leverage=5.0 is passed but IGNORED for D_LIQ — ExtendedLeverageEngine.__init__ overrides it to D_LIQ_SOFT_CAP=8.0 unconditionally.

MC_BASE_CFG (MC-Forewarner config)

File: nautilus_dolphin/dvae/exp_shared.py line 69–81

Key param: 'max_leverage': 5.00 — matches D_LIQ_MC_REF=5.0 for consistency.

D_LIQ Constants

File: nautilus_dolphin/nautilus_dolphin/nautilus/proxy_boost_engine.py line 437–440

D_LIQ_SOFT_CAP   = 8.0    # base_max_leverage: soft ceiling, ACBv6 can push toward hard cap
D_LIQ_ABS_CAP    = 9.0    # abs_max_leverage: hard ceiling, never exceeded
D_LIQ_MC_REF     = 5.0    # MC-Forewarner reference: within GOLD trained range
D_LIQ_MARGIN_BUF = 0.95   # liquidation floor = (1/9) * 0.95 = 10.56% adverse move

Vol P60 (gold calibration method)

File: nautilus_dolphin/dvae/exp_shared.py load_data() function

vol_p60 ≈ 0.00009868 (from 2 parquet files, range(60), seg-based stddev, v>0 filter)

Method:

for pf in parquet_files[:2]:   # ONLY FIRST 2 FILES
    df = pd.read_parquet(pf)
    pr = df['BTCUSDT'].values
    for i in range(60, len(pr)):   # range(60), NOT range(50)
        seg = pr[max(0, i-50):i]
        if len(seg) < 10: continue
        v = float(np.std(np.diff(seg) / seg[:-1]))
        if v > 0: all_vols.append(v)   # v>0 filter
vol_p60 = float(np.percentile(all_vols, 60))

CRITICAL: Any deviation from this exact method will change vol_p60 and alter trade timing, potentially changing T away from 2155. The run_backtest() function in exp_shared.py uses a slightly different (rolling) method that may give slightly different results — the load_data() gold method is canonical for gold reproduction.

MockOBProvider config

File: nautilus_dolphin/dvae/exp_shared.py load_data() function

MockOBProvider(
    imbalance_bias=-0.09, depth_scale=1.0, assets=OB_ASSETS,
    imbalance_biases={
        "BTCUSDT": -0.086, "ETHUSDT": -0.092,
        "BNBUSDT": +0.05,  "SOLUSDT": +0.05,
    },
)

OB_ASSETS = all 48 assets from parquet file columns (sorted alphabetically). ob_confirm_rate=0.40: with these biases, most entries pass the OB gate. OB_TAIL_AVOIDANCE (cascade/withdrawal exits) NEVER fire with MockOBProvider (mock provider generates synthetic data that never crosses the cascade threshold).

---

7. DATA PIPELINE

Primary Data Source (backtest)

Path: C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\vbt_cache\ Format: Daily Parquet files, named YYYY-MM-DD.parquet Window: 56 files, 2025-12-31 to 2026-02-26 Columns per file:

• 48 asset price columns (BTCUSDT, ETHUSDT, BNBUSDT, etc.) — float64
• vel_div — eigenvalue velocity divergence — core signal
• v50_lambda_max_velocity, v150_lambda_max_velocity, v300_lambda_max_velocity, v750_lambda_max_velocity
• instability_50, instability_150
• timestamp, scan_number (metadata, excluded from asset columns)

META_COLS (excluded from asset price list):

META_COLS = {
    'timestamp', 'scan_number', 'v50_lambda_max_velocity', 'v150_lambda_max_velocity',
    'v300_lambda_max_velocity', 'v750_lambda_max_velocity', 'vel_div',
    'instability_50', 'instability_150'
}

Bar frequency: 5 seconds (scan_number advances every 5s from NG5 scanner) Bars per day: ~6,192 (5s × 86400s / 5 ≈ 17,280, but market hours vary → ~6k active bars) Total bars: ~346,740 over 56 days

Eigenvalue Source

Path: C:\Users\Lenovo\Documents\- Dolphin NG HD (NG3)\correlation_arb512\eigenvalues\ Used by NG5 scanner to produce eigenvalue features for each 5s scan.

ExF NPZ (External Factors)

Path: C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\external_factors\eso_cache\ Latest: latest_esoteric_factors.json — lunar phase, regional session, Fibonacci time NPZ backfill: All 1710 historical dates in vbt_cache_klines/ have scan_000001__Indicators.npz

MC-Forewarner Models

Path: C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\nautilus_dolphin\mc_results\models\ Format: Pickle files (sklearn 1.7.1 — note: loading with 1.8.0 raises InconsistentVersionWarning but works) Models: OneClassSVM, DummyRegressor, DecisionTreeRegressor, GradientBoostingRegressor, StandardScaler Result: All 56 days return OK (0 RED/ORANGE) when assessed at mc_ref=5.0.

5-Year Klines Dataset (for regime research)

Path: C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\vbt_cache_klines\ Format: 1710 parquet files, 2021-06-15 to 2026-03-05 Frequency: 1-minute OHLCV + vel_div + eigenvalue features

---

8. TEST HARNESS ANATOMY

Primary e2e test

File: nautilus_dolphin/tests/test_proxy_boost_production.py (UNTRACKED)

Contains 9 e2e tests including test_e2e_d_liq_gold_reproduces_exp9b. Run: pytest -m slow tests/test_proxy_boost_production.py

_run_engine() function (critical path):

def _run_engine(eng, d, fw, MC_BASE_CFG):
    acb = AdaptiveCircuitBreaker()
    acb.preload_w750(d['date_strings'])
    eng.set_ob_engine(d['ob_eng'])
    eng.set_acb(acb)
    if fw is not None:
        eng.set_mc_forewarner(fw, MC_BASE_CFG)
    eng.set_esoteric_hazard_multiplier(0.0)   # ← CRITICAL: now correct (calls fixed function)
    for pf_file in d['parquet_files']:
        ds = pf_file.stem
        df, acols, dvol = d['pq_data'][ds]    # ← float64 pq_data from load_data()
        cap_before = eng.capital
        vol_ok = np.where(np.isfinite(dvol), dvol > d['vol_p60'], False)
        eng.process_day(ds, df, acols, vol_regime_ok=vol_ok)

exp_shared.py functions

File: nautilus_dolphin/dvae/exp_shared.py

• load_data() → gold-standard data loading (float64, seg-based vol_p60, 48 OB assets)
• run_backtest(engine_factory, name, ...) → lazy-loading backtest (float32, rolling vol_p60) Note: run_backtest() internally calls eng.set_esoteric_hazard_multiplier(0.0) — now correct after the fix. Uses slightly different vol_p60 method (rolling, not gold).
• ensure_jit() → triggers numba JIT warmup (calls all numba functions once)

Import Warning: dvae/init.py loads PyTorch

File: nautilus_dolphin/dvae/__init__.py

from .hierarchical_dvae import HierarchicalDVAE   # ← loads PyTorch!
from .corpus_builder import DolphinCorpus

CONSEQUENCE: Any from dvae.exp_shared import ... statement will import PyTorch via dvae/__init__.py, consuming ~700MB+ RAM and potentially causing OOM.

CORRECT PATTERN:

_HERE = Path(__file__).resolve().parent  # dvae/ directory
sys.path.insert(0, str(_HERE))           # add dvae/ to path
from exp_shared import run_backtest, GOLD  # direct import, no __init__.py

Trace Backtest (painstaking per-tick logger)

File: nautilus_dolphin/dvae/run_trace_backtest.py (created 2026-03-22)

Produces per-tick, per-trade, and per-day CSV trace files. See §2 Step 4 for usage and output format.

---

9. KNOWN BUGS AND FIXES

BUG 1 (CRITICAL — FIXED 2026-03-22): set_esoteric_hazard_multiplier stomps D_LIQ leverage

File: nautilus_dolphin/nautilus_dolphin/nautilus/esf_alpha_orchestrator.py line 707–720

Symptom: D_LIQ backtests give ROI=145.84% instead of gold 181.81%.

Root cause: ceiling_lev = 6.0 was hardcoded. When called with hazard_score=0.0, the function set base_max_leverage = 6.0 and bet_sizer.max_leverage = 6.0, overwriting D_LIQ's _extended_soft_cap = 8.0. On all non-ACB-boosted days (~40 of 56 days), this reduced available leverage from 8.0x to 6.0x = 33% less. The resulting ROI gap: 181.81% - 145.84% = 35.97pp.

Fix:

# BEFORE (wrong):
ceiling_lev = 6.0

# AFTER (correct):
ceiling_lev = getattr(self, '_extended_soft_cap', 6.0)

Verified: After fix, eng.set_esoteric_hazard_multiplier(0.0) on a D_LIQ engine gives base_max_leverage = 8.0 (was 6.0). NDAlphaEngine (no _extended_soft_cap) still gets 6.0 — backwards compatible.

Callers affected:

• exp_shared.py:176 → run_backtest() → now correct
• tests/test_proxy_boost_production.py → _run_engine() → now correct
• titan_stage1_run.py:130 → now correct
• dvae/exp9b_liquidation_guard.py, all exp*.py → now correct
• run_esoteric_throttled_backtest.py:142 → now correct (correctly scales from 8.0 ceiling)

Paradox resolved: titan_stage1_run.log shows ROI=181.81% WITH the stomp call. This was because when titan_stage1 ran (2026-03-15), the function either (a) didn't exist yet and was added to titan_stage1_run.py after the run, or (b) the function had a different implementation. Since titan_stage1_run.py is untracked, git history is unavailable. The 181.81% result is authentic (confirmed by 9/9 e2e tests on 2026-03-15).

BUG 2 (FIXED): dvae/init.py PyTorch OOM

See §8. Fixed by using sys.path.insert(0, str(_HERE)) before importing from exp_shared.

BUG 3 (FIXED 2026-03-09): NG5 scanner PriceData extraction

NG5 scanner had a PriceData extraction bug causing all-zero eigenvalues since 2026-03-08. Fixed in eso_cache/ and backfill scripts. All eigenvalue data now correct.

BUG 4 (KNOWN): MC model sklearn version mismatch

MC models saved with sklearn 1.7.1, loaded with 1.8.0. Produces warnings but works. Retraining models with sklearn 1.8.0 would eliminate warnings.

BUG 5 (KNOWN): OOM during numba JIT warmup on low-RAM systems

With <1.5GB free RAM, ensure_jit() (which compiles all numba functions) may fail. Workaround: Use NUMBA_DISABLE_JIT=1 for quick checks, or wait for RAM to free. Numba cache (95 .nbc files in nautilus_dolphin/nautilus_dolphin/nautilus/__pycache__/) should be warmed from previous runs.

---

10. RESEARCH HISTORY (Exp1–Exp15)

All experiments ran on the 56-day (Dec31 2025 – Feb26 2026) dataset, T=2155 baseline.

Foundation: NDAlphaEngine BRONZE baseline

• ROI=88.55%, PF=1.215, DD=15.05%, Sharpe=4.38, T=2155
• Script: test_pf_dynamic_beta_validate.py

Exp4: proxy_B Signal Characterization

• Signal: proxy_B = instability_50 - v750_lambda_max_velocity
• AUC=0.715 for eigenspace stress detection
• r=+0.42 (p=0.003) correlation with intraday MAE
• r=-0.03 (ns) with vel_div — ORTHOGONAL → pure position-sizing layer
• Results: dvae/exp4_proxy_coupling.py

Exp6: Stop Tightening Research (DEAD)

• All 8 configs fail: global stops cause re-entry cascade (1415→2916 trades)
• Best gated: ROI=38.42% DD=19.42% — far worse than baseline
• stop_pct=1.0 (effectively OFF) confirmed optimal
• Results: dvae/exp6_stop_test_results.json

Exp7: Live Coupling (scale_boost)

• scale_boost(thr=0.35, alpha=1.0): ROI=93.61% DD=14.51% T=2155 (+5.06pp ROI, -0.54pp DD)
• hold_limit and rising_exit: 91% early exit rate → cascade → DEAD
• Results: dvae/exp7_live_coupling_results.json

Exp8: Adaptive Parameterization → PREV GOLD

• adaptive_beta(thr=0.35, alpha×(1+day_beta)): ROI=96.55% DD=14.32% T=2155
• NOT overfitting: both H1+H2 temporal halves improve vs baseline
• Was GOLD from 2026-03-14 to 2026-03-15
• Results: dvae/exp8_boost_robustness_results.json

Exp9: Extended Leverage Ceiling (8 configs)

• MC-Forewarner stays GREEN at ALL leverage levels tested (5x through 10x)
• DD plateau after 7x: each +1x costs only ~+0.12pp DD (convex curve)
• Best ROI: E(9/10x)=184.00% DD=18.56% — liquidation risk unmodeled
• MC decoupling via begin_day lever swap (mc_ref=5.0) confirmed essential
• Results: dvae/exp9_leverage_ceiling_results.json

Exp9b: Liquidation Guard → D_LIQ_GOLD

• D_liq(8/9x) + liquidation floor at 10.56% (=1/9×0.95): ROI=181.81%, DD=17.65%, Calmar=10.30, T=2155, liq_stops=1
• E_liq(9/10x): 5 stops → cascade → DEAD (DD=31.79%)
• D_liq is the sweet spot: high leverage, protected against forced liquidation
• BECAME GOLD 2026-03-15
• Results: dvae/exp9b_liquidation_guard_results.json

Exp9c: Overfitting Validation

• H1/H2 split: D_LIQ wins ROI in BOTH halves; Calmar dips H2 (-0.23 margin)
• Q1–Q4 split: PASS Q1/Q2, marginal FAIL Q3/Q4 (Q2 carries most outperformance)
• Buffer sweep 0.80–1.00: 0.90–1.00 identical; 0.80 cascades (5 stops)
• Verdict: regime-conditional upgrade — works best in volatile markets (Q1/Q2)
• Results: dvae/exp9c_overfitting_results.json

Exp10: 1m Keyframe Gate (DEAD)

• i150_z1m + TITAN recon_err z-scores — all z_recon signals hurt performance
• Calmar 7.70 vs 7.82 threshold. Killed.

Exp11: z_recon_inv Baseline (DEAD)

• Noise floor, scale_mean=0.999. No signal. Killed.

Exp12: ConvNeXt z-Gate (INCONCLUSIVE)

• Used wrong ep=17 model (not v2 BOB). z10_inv +1.41pp ROI but Calmar 7.73 < 7.83.

Exp13 v2: ZLeverageGateEngine → CONFIRMED SIGNAL ⭐

• Signal: daily z[13] from 1m ConvNeXt v2 BOB (r=+0.933 with proxy_B)
• 9/20 Phase 2 configs PASS Calmar > 7.83 vs D_LIQ_GOLD baseline
• Best: A_P5_M2_W1_a0.5 → ROI=186.40% (+4.59pp), Calmar=7.87
• Compounding: +$2.38M on $25k over 1 year (+11.1%) — zero DD cost
• ⚠️ PENDING PRODUCTIONIZATION (see memory/todo_productize.md)
• Results: dvae/exp13_multiscale_sweep_results.json

Exp14: z[13]/z_post_std/delta leverage gate sweep

• Running at time of writing. Results → dvae/exp14_results.json

Exp15: z[13]-gated per-trade stop+TP extension

• Running at time of writing. Results → dvae/exp15_results.json

---

11. FILE MAP — ALL CRITICAL PATHS

Core Engine Files

nautilus_dolphin/
└── nautilus_dolphin/
    └── nautilus/
        ├── esf_alpha_orchestrator.py     ← NDAlphaEngine (BASE) — ALL core logic
        │   Lines of interest:
        │     68–83:   NDTradeRecord dataclass
        │     86–720:  NDAlphaEngine class
        │     196:     self.trade_history: List[NDTradeRecord]
        │     241–289: step_bar() — streaming API
        │     294–357: process_bar() — per-bar entry/exit
        │     358–450: _execute_exit() — exit finalization
        │     707–720: set_esoteric_hazard_multiplier() ← BUG FIXED 2026-03-22
        │     779–826: begin_day() — streaming API
        │     827–850: end_day() — streaming API
        │
        ├── proxy_boost_engine.py         ← ProxyBase + AdaptiveBoost + ExtendedLev + LiqGuard
        │   Lines of interest:
        │     1–36:    module docstring with gold numbers
        │     47–103:  create_boost_engine() factory
        │     110–203: ProxyBaseEngine — process_day() + _update_proxy()
        │     209–303: AdaptiveBoostEngine — scale-boost logic
        │     311–385: ExtendedLeverageEngine — MC decoupling begin_day()
        │     392–430: LiquidationGuardEngine — _try_entry() + _execute_exit()
        │     437–465: D_LIQ constants + create_d_liq_engine() factory
        │
        ├── adaptive_circuit_breaker.py   ← ACBv6: day_base_boost + day_beta
        ├── alpha_exit_manager.py         ← Exit logic: FIXED_TP, MAX_HOLD, STOP, OB
        ├── alpha_bet_sizer.py            ← compute_sizing_nb() — numba leverage sizing
        ├── alpha_asset_selector.py       ← compute_irp_nb() — IRP asset ranking
        ├── alpha_signal_generator.py     ← check_dc_nb() — direction confirmation
        ├── ob_features.py                ← OB feature computation (numba)
        └── ob_provider.py                ← MockOBProvider / CSVOBProvider

Test & Research Files

nautilus_dolphin/
├── tests/
│   └── test_proxy_boost_production.py    ← 9 e2e tests, 42 unit tests (UNTRACKED)
│
├── dvae/
│   ├── exp_shared.py                     ← Shared test infrastructure (gold data, run_backtest)
│   ├── run_trace_backtest.py             ← Painstaking per-tick trace logger (new 2026-03-22)
│   ├── test_dliq_fix_verify.py           ← Quick D_LIQ gold reproduction verify (new 2026-03-22)
│   ├── test_dliq_nostomp.py              ← WITH vs WITHOUT stomp comparison
│   │
│   ├── exp9b_liquidation_guard.py        ← Exp9b original research
│   ├── exp9b_retest.log                  ← Retest attempt (died with sklearn warnings)
│   ├── exp9c_overfitting_validation.py   ← Overfitting validation research
│   ├── exp13_multiscale_sweep.py         ← ConvNeXt z[13] gate sweep
│   ├── exp14_sweep.py                    ← z[13] leverage gate sweep (running)
│   ├── exp15_stop_gate.py                ← z[13] stop+TP gate (running)
│   │
│   ├── trace/                            ← Created by run_trace_backtest.py
│   │   ├── tick_trace.csv                ← Per-bar system state (~69MB)
│   │   ├── trade_trace.csv               ← Per-trade details (~320KB)
│   │   ├── daily_trace.csv               ← Per-day summary
│   │   └── summary.json                  ← Final ROI/DD/T/Calmar
│   │
│   └── convnext_model_v2.json            ← 1m ConvNeXt v2 BOB (z[13]=proxy_B, r=0.933)
│       = convnext_model_1m_bob.json      ← symlink/copy
│
└── run_logs/
    └── REGISTRY.md                       ← CANONICAL inter-agent sync file — READ FIRST

Data Paths

C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\
├── vbt_cache\                            ← 56 daily parquet files (5s scans, primary backtest)
│   └── 2025-12-31.parquet .. 2026-02-26.parquet
│
├── vbt_cache_klines\                     ← 1710 daily parquet files (1m OHLCV, 2021–2026)
│
├── external_factors\
│   ├── eso_cache\
│   │   └── latest_esoteric_factors.json  ← Current lunar/astro/session state
│   ├── realtime_exf_service.py           ← Live ExF update service
│   └── backfill_patch_npz.py             ← Historical ExF backfill
│
└── nautilus_dolphin\
    ├── mc_results\models\                ← MC-Forewarner trained models
    └── run_logs\REGISTRY.md              ← Canonical registry

Production Files (live trading)

nautilus_dolphin/
├── prod/
│   ├── dolphin_actor.py                  ← Live trading actor (reads boost_mode from config)
│   ├── paper_trade_flow.py               ← Prefect paper trading flow
│   ├── vbt_backtest_flow.py              ← Prefect backtest flow
│   ├── mc_forewarner_flow.py             ← MC update flow (every 4h)
│   └── esof_update_flow.py               ← EsoF update flow (every 6h)
│
└── dolphin_vbt_real.py                   ← (parent dir) Live VBT backtest runner

---

12. FAILURE MODE REFERENCE

T ≠ 2155 (wrong trade count)

Symptom	Likely Cause	Fix
T < 2155	Missing parquet files / wrong date range	Check vbt_cache, ensure 56 files Dec31–Feb26
T < 2155	Wrong vol_p60 (too high → too many bars filtered)	Use gold load_data() method
T ≠ 2155	Wrong seed → different DC/IRP randomness	Ensure seed=42 in ENGINE_KWARGS
T > 2155	Liquidation stop cascade (re-entries)	margin_buffer too small or abs_cap too high

ROI ≈ 145% instead of 181.81%

Cause: set_esoteric_hazard_multiplier() stomping base_max_leverage to 6.0. Fix: Apply the getattr(self, '_extended_soft_cap', 6.0) fix. See §9 Bug 1.

ROI ≈ 88–96% instead of 181.81%

Cause: Using wrong engine. create_boost_engine(mode='adaptive_beta') gives 96.55%. NDAlphaEngine gives 88.55%. Must use create_d_liq_engine() for 181.81%.

OOM during test run

Cause 1: from dvae.exp_shared import ... triggers dvae/__init__.py → PyTorch load. Fix: Use sys.path.insert(0, str(_HERE)); from exp_shared import ... (direct import).

Cause 2: ensure_jit() numba compilation with <1.5GB free RAM. Fix: Close memory-hungry apps. Numba cache (95 .nbc files) should prevent recompilation. Check: ls nautilus_dolphin/nautilus_dolphin/nautilus/__pycache__/*.nbc | wc -l → 95.

MC-Forewarner fires RED/ORANGE

Cause: set_mc_forewarner() called BEFORE set_esoteric_hazard_multiplier(). Or mc_leverage_ref not set to 5.0 (mc_ref must match MC trained range ~5x). Expected: 0 RED / 0 ORANGE when mc_ref=5.0. Any RED day halts trading → fewer trades.

sklearn InconsistentVersionWarning

Not a bug — MC models saved with 1.7.1, loaded with 1.8.0. Warnings are harmless. Suppress: import warnings; warnings.filterwarnings('ignore', category=InconsistentVersionWarning)

---

APPENDIX: Quick Commands

# Working directory for all commands:
cd "C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\nautilus_dolphin"

# Verify fix (fast, no numba):
python -c "
import os; os.environ['NUMBA_DISABLE_JIT']='1'
import sys; sys.path.insert(0, '.')
from nautilus_dolphin.nautilus.proxy_boost_engine import create_d_liq_engine
from dvae.exp_shared import ENGINE_KWARGS
e=create_d_liq_engine(**ENGINE_KWARGS)
e.set_esoteric_hazard_multiplier(0.0)
assert e.base_max_leverage==8.0, 'STOMP ACTIVE'
print('FIX OK: base_max_leverage=', e.base_max_leverage)
"

# Quick D_LIQ gold reproduction (~400s with lazy loading):
python dvae/test_dliq_fix_verify.py

# Full painstaking trace with per-tick/trade logging (~400s):
python dvae/run_trace_backtest.py

# Full e2e suite (9 tests, ~52 minutes):
pytest -m slow tests/test_proxy_boost_production.py -v

# Check REGISTRY for latest run history:
type run_logs\REGISTRY.md | more