Files
DOLPHIN/prod/docs/SYSTEM_BIBLE_v1_MIG7_20260307.md
hjnormey 01c19662cb initial: import DOLPHIN baseline 2026-04-21 from dolphinng5_predict working tree
Includes core prod + GREEN/BLUE subsystems:
- prod/ (BLUE harness, configs, scripts, docs)
- nautilus_dolphin/ (GREEN Nautilus-native impl + dvae/ preserved)
- adaptive_exit/ (AEM engine + models/bucket_assignments.pkl)
- Observability/ (EsoF advisor, TUI, dashboards)
- external_factors/ (EsoF producer)
- mc_forewarning_qlabs_fork/ (MC regime/envelope)

Excludes runtime caches, logs, backups, and reproducible artifacts per .gitignore.
2026-04-21 16:58:38 +02:00

41 KiB
Executable File
Raw Blame History

DOLPHIN-NAUTILUS SYSTEM BIBLE

Doctrinal Reference — As Running 2026-03-07

Version: MIG7 Complete (post-MIG7_MIG8_FIXES)
CI gate: 14/14 tests green
Status: Paper trading ready. NOT deployed with real capital.


TABLE OF CONTENTS

  1. System Philosophy
  2. Physical Architecture
  3. Data Layer
  4. Signal Layer — vel_div & DC
  5. Asset Selection — IRP
  6. Position Sizing — AlphaBetSizer
  7. Exit Management
  8. Fee & Slippage Model
  9. OB Intelligence Layer
  10. ACB v6 — Adaptive Circuit Breaker
  11. Survival Stack — Posture Control
  12. MC-Forewarner Envelope Gate
  13. NDAlphaEngine — Full Bar Loop
  14. DolphinActor — Nautilus Integration
  15. Hazelcast Feature Store — Sharding
  16. Production Daemon Topology
  17. CI Test Suite
  18. Parameter Reference
  19. Known Research TODOs

1. SYSTEM PHILOSOPHY

DOLPHIN-NAUTILUS is a SHORT-only (champion configuration) systematic trading engine targeting crypto perpetual futures on Binance.

Core thesis: When crypto market correlation matrices show accelerating eigenvalue-velocity divergence (vel_div < -0.02), the market is entering an instability regime. Shorting during early instability onset and exiting at fixed take-profit captures the mean-reversion from panic to normalization.

Design constraints:

  • Zero signal re-implementation in the Nautilus layer. All alpha logic lives in NDAlphaEngine.
  • 512-bit arithmetic for correlation matrix processing (separate NG3 pipeline; not in hot path of this engine).
  • Champion parameters are FROZEN. They were validated via exhaustive VBT backtest on dolphin_vbt_real.py.
  • The Nautilus actor is a thin wire, not a strategy. It routes parquet data → NDAlphaEngine → HZ result.

Champion performance (ACBv6 + IRP + DC + OB, full-stack 55-day Dec31Feb25):

  • ROI: +54.67% | PF: 1.141 | Sharpe: 2.84 | Max DD: 15.80% | WR: 49.5% | Trades: 2145
  • Log: run_logs/summary_20260307_163401.json

Data correction note (2026-03-07): An earlier reference showed ROI=+57.18%, PF=1.149, Sharpe=3.00. Those figures came from a stale vbt_cache/2026-02-25.parquet that was built mid-day — missing 435 scans and carrying corrupt vel_div on 492 rows for the final day of the window. ALGO-3 parity testing caught the mismatch (max_diff=1.22 vs tolerance 1e-10). The parquet was rebuilt from live NG3 JSON (build_parquet_cache(dates=['2026-02-25'], force=True)). The stale file is preserved as 2026-02-25.parquet.STALE_20260307 for replicability. The corrected numbers above are the canonical reference. The ~2.5pp ROI drop reflects real late-day trades on Feb 25 that the stale parquet had silently omitted.


2. PHYSICAL ARCHITECTURE

┌─────────────────────────────────────────────────────────────────────┐
│  DATA SOURCES                                                        │
│  NG3 Scanner → correlation_arb512/ (512-bit eigenvalue matrices)     │
│  CCXT Live Feed → 5s OHLCV bars (48+ USDT perpetuals on Binance)    │
│  VBT Cache → vbt_cache_klines/*.parquet (pre-built from CCXT)       │
└───────────────────┬──────────────────────────────────────────────────┘
                    │
┌───────────────────▼──────────────────────────────────────────────────┐
│  HAZELCAST IN-MEMORY GRID (localhost:5701, cluster: "dolphin")        │
│  IMaps:                                                               │
│    DOLPHIN_SAFETY       → AtomicReference / IMap (posture JSON)      │
│    DOLPHIN_FEATURES     → ACB boost + OB features per bar            │
│    DOLPHIN_PNL_BLUE     → Per-date trade results (SHORT posture)     │
│    DOLPHIN_PNL_GREEN    → Per-date trade results (LONG posture)      │
│    DOLPHIN_FEATURES_SHARD_00..09 → 400-asset sharded feature store   │
└───────────────────┬──────────────────────────────────────────────────┘
                    │
┌───────────────────▼──────────────────────────────────────────────────┐
│  PRODUCTION DAEMONS (each runs in its own process/thread)            │
│  acb_processor_service.py   — Daily ACB boost computation + HZ write │
│  ob_stream_service.py       — 500ms OB snapshot ingestion + HZ write │
│  system_watchdog_service.py — Survival Stack Rm computation          │
│  mc_forewarner_flow.py      — Prefect-orchestrated MC gate           │
└───────────────────┬──────────────────────────────────────────────────┘
                    │
┌───────────────────▼──────────────────────────────────────────────────┐
│  NAUTILUS TRADING ENGINE                                             │
│  paper_trade_flow.py → DolphinActor → NDAlphaEngine                 │
│  Receives 5s bars via Nautilus event loop                            │
│  Calls process_day() once per calendar date                          │
└─────────────────────────────────────────────────────────────────────┘

Key invariant: DolphinActor.on_bar() fires ~17,280 times per day (every 5 seconds). process_day() is called exactly once per calendar date — guarded by _processed_dates set.


3. DATA LAYER

3.1 vbt_cache_klines Parquet Schema

Location: C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\vbt_cache_klines\YYYY-MM-DD.parquet

Column Type Description
vel_div float64 Eigenvalue velocity divergence: v50_vel v750_vel (primary signal)
v50_lambda_max_velocity float64 Short-window (50-bar) lambda_max rate of change
v150_lambda_max_velocity float64 150-bar window lambda velocity
v300_lambda_max_velocity float64 300-bar window lambda velocity
v750_lambda_max_velocity float64 Long-window (750-bar) macro eigenvalue velocity
instability_50 float64 General market instability index (50-bar)
instability_150 float64 General market instability index (150-bar)
BTCUSDTSTXUSDT float64 Per-asset close prices (48 assets in current dataset)

Each file: 1,439 rows (1 per 5-second bar over 24h), 57 columns.

3.2 NG3 Eigenvalue Data

Location: C:\Users\Lenovo\Documents\- Dolphin NG HD (NG3)\correlation_arb512\

eigenvalues/
  YYYY-MM-DD/
    scan_NNNNNN__Indicators.npz   ← ACBv6 external factors (funding, dvol, fng, taker)
    scan_NNNNNN__scan_global.npz  ← lambda_vel_w750 for dynamic beta
matrices/
  YYYY-MM-DD/
    scan_NNNNNN_w50_HHMMSS.arb512.pkl.zst  ← 512-bit correlation matrix (unused in hot path)

NPZ files loaded by AdaptiveCircuitBreaker._load_external_factors() (max 10 scans per date, median-aggregated).


4. SIGNAL LAYER

4.1 Primary Signal: vel_div Threshold Gate

Source: alpha_signal_generator.py, AlphaSignalGenerator.generate()

SHORT signal condition:

vel_div < VEL_DIV_THRESHOLD (-0.02)

LONG signal condition (green posture, not champion):

vel_div > LONG_THRESHOLD (0.01)

Confidence calculation (SHORT path):

ratio = clamp((threshold - vel_div) / (threshold - extreme), 0, 1)
      = clamp((-0.02 - vel_div) / (-0.02 - (-0.05)), 0, 1)
      = clamp((-0.02 - vel_div) / 0.03, 0, 1)
confidence = 0.50 + ratio * 0.40   # range: [0.50, 0.90]

is_extreme = (vel_div <= -0.05)

4.2 Direction Confirmation (DC) — Layer 6

Source: alpha_signal_generator.py, check_dc_nb() (numba JIT)

# Looks back dc_lookback_bars (default 7) bars on the selected asset price
p0 = price[n - lookback - 1]
p1 = price[n - 1]
chg_bps = (p1 - p0) / p0 * 10000.0

if chg_bps < -min_magnitude_bps (-0.75):  return CONFIRM   # falling price → SHORT OK
if chg_bps >  min_magnitude_bps (+0.75):  return CONTRADICT
else:                                       return NEUTRAL

dc_skip_contradicts = True (champion): CONTRADICT returns null signal (skip entry).
Effect on leverage: DC has dc_leverage_boost=1.0 (no boost in champion). CONTRADICT kills entry.

4.3 OB Sub-2: Per-Asset Imbalance Confirmation

When ob_engine is wired in (use_ob_edge=True):

eff_imb = -ob_signal.imbalance_ma5  # For SHORT: sell pressure = positive eff_imb

if eff_imb > 0.10:                              # OB confirms → confidence boost ≤+15%
    ob_adj = 1 + min(0.15, eff_imb * persistence * 0.5)
    confidence *= ob_adj
elif eff_imb < -0.15 and persistence > 0.60:   # Strong persistent OB contradiction → HARD SKIP
    return null_signal
elif eff_imb < -0.10:                           # Moderate → soft dampen confidence
    ob_adj = max(0.85, 1 - |eff_imb| * persistence * 0.4)
    confidence *= ob_adj

5. ASSET SELECTION — IRP

5.1 Overview

Source: alpha_asset_selector.py, AlphaAssetSelector.rank_assets() + numba kernels

IRP = Impulse Response Profiling. Ranks all available assets by historical behavior over the last 50 bars in the regime direction. Selects the asset with the highest ARS (Asset Ranking Score) that passes all filters.

Enabled by: use_asset_selection=True (production default).

5.2 Numba Kernel: compute_irp_nb

# Input: price_segment (last 50 prices), direction (-1 or +1)
dir_returns[i] = (price[i+1] - price[i]) * direction   # directional returns

cumulative = cumsum(dir_returns)
mfe = max(cumulative)          # Maximum Favorable Excursion
mae = abs(min(cumulative, 0))  # Maximum Adverse Excursion
efficiency  = mfe / (mae + 1e-6)
alignment   = count(dir_returns > 0) / n_ret
noise       = variance(dir_returns)
latency     = bars_to_reach_10pct_of_mfe   # (default: 50 if mfe==0)

5.3 Numba Kernel: compute_ars_nb

ARS = 0.5 * log1p(efficiency) + 0.35 * alignment - 0.15 * noise * 1000

5.4 Numba Kernel: rank_assets_irp_nb

For each asset:

  1. Compute IRP in DIRECT direction (regime_direction)
  2. Compute IRP in INVERSE direction (-regime_direction)
  3. Take whichever gives higher ARS (allows inverse selection)
  4. Apply filter gates:
    • noise > 500 → skip
    • latency > 20 → skip (must reach 10% MFE within 20 bars)
    • alignment < 0.20 → skip
  5. Bubble-sort by ARS descending (numba nopython)

5.5 AlphaAssetSelector Python Wrapper

# Build 2D array (max_len × n_assets), right-aligned
valid = rank_assets_irp_nb(prices_2d, idx=max_len, regime_direction, ...)
# Walk ranked list:
for r in rankings:
    if min_irp_alignment > 0 and r.metrics.alignment < min_irp_alignment:
        continue          # alignment gate (default 0.45)
    if r.action != expected_action:
        continue          # direction gate
    if ob_engine and ob_placement.depth_quality < 0.20:
        continue          # OB depth gate (try next asset)
    trade_asset = r.asset
    break
# No match → return None (no fallback to BTCUSDT when IRP enabled)

OB Sub-1: ARS adjusted ±5%/10% by per-asset OB depth quality before sorting.


6. POSITION SIZING

6.1 Signal Strength Score

Source: alpha_bet_sizer.py, compute_sizing_nb() (numba JIT)

# SHORT path (vel_div < threshold):
if vel_div <= extreme (-0.05):
    strength_score = 1.0
else:
    strength_score = (threshold - vel_div) / (threshold - extreme)
    # = (-0.02 - vel_div) / 0.03
    strength_score = clamp(strength_score, 0.0, 1.0)

6.2 Dynamic Leverage (Cubic Convex)

scaled_score  = strength_score ** leverage_convexity (3.0)
eff_leverage  = min_leverage + scaled_score * (max_leverage - min_leverage)
              = 0.5 + scaled_score³ * 4.5   # range: [0.5, 5.0] before ACB

6.3 Signal Bucket Classification

# Numba: get_signal_bucket_nb(vel_div, threshold=-0.02, extreme=-0.05)
if vel_div <= extreme * 1.5 (-0.075):  bucket = 0  # "extreme"
elif vel_div <= extreme (-0.05):        bucket = 1  # "strong"
elif vel_div <= (threshold+extreme)/2:  bucket = 2  # "moderate"  (-0.035)
else:                                   bucket = 3  # "weak"

6.4 Alpha Layers (Layer 7)

Bucket Boost — adaptive win-rate feedback:

# get_bucket_boost_nb: per-bucket win rate → multiplier
wr > 0.60  1.3x  |  wr > 0.55  1.1x  |  wr < 0.40  0.7x  |  wr < 0.45  0.85x

Streak Multiplier — recent 5-trade loss streak:

# get_streak_mult_nb
losses_in_last_5 >= 4  0.5x  |  >= 3  0.7x  |  <= 1  1.1x

Trend Multiplier — vel_div acceleration:

# get_trend_mult_nb(vd_trend = vel_div_history[-1] - vel_div_history[-10])
vd_trend < -0.01  1.3x (deepening instability)
vd_trend >  0.01  0.7x (recovering)

Effective Fraction computation:

confidence = 0.70 if is_extreme else 0.55
conf_mult  = confidence / 0.95
extreme_boost = 2.0 if is_extreme else 1.0

base_frac   = 0.02 + strength_score * (base_fraction - 0.02)
eff_fraction = base_frac * conf_mult * extreme_boost * trend_mult * bucket_boost * streak_mult
eff_fraction = clamp(eff_fraction, 0.02, base_fraction=0.20)

Final notional:

notional = capital * eff_fraction * final_leverage

6.5 ACB + MC Size Multiplier

# regime_size_mult is recomputed every bar via _update_regime_size_mult(vel_div)
if day_beta > 0:
    strength_cubic = clamp((threshold - vel_div) / (threshold - extreme), 0, 1) ** convexity
    regime_size_mult = day_base_boost * (1.0 + day_beta * strength_cubic) * day_mc_scale
else:
    regime_size_mult = day_base_boost * day_mc_scale

# Applied to leverage ceiling:
clamped_max_leverage = min(base_max_leverage * regime_size_mult * market_ob_mult, abs_max_leverage=6.0)
raw_leverage = size_result["leverage"] * dc_lev_mult * regime_size_mult * market_ob_mult

# STALKER posture hard cap:
if posture == 'STALKER': clamped_max_leverage = min(clamped_max_leverage, 2.0)

final_leverage = clamp(raw_leverage, min_leverage=0.5, clamped_max_leverage)

7. EXIT MANAGEMENT

7.1 Exit Priority Order (champion)

Source: alpha_exit_manager.py, AlphaExitManager.evaluate()

  1. FIXED_TP: pnl_pct >= 0.0095 (95 basis points)
  2. STOP_LOSS: pnl_pct <= -1.0 (DISABLED in practice — 100% loss never triggers before TP/max_hold)
  3. OB DURESS exits (when ob_engine != None):
    • Cascade Detection: cascade_count > 0 → widen TP ×1.40, halve max_hold
    • Liquidity Withdrawal: regime_signal == 1 → hard SL 10%, TP ×0.60
  4. vel_div adverse-turn exits (vd_enabled=False by default — disabled pending calibration)
  5. MAX_HOLD: bars_held >= 120 (= 600 seconds)

7.2 OB Dynamic Exit Parameter Adjustment

if cascade_count > 0:
    dynamic_tp_pct  *= 1.40
    dynamic_max_hold = int(max_hold_bars * 0.50)    # take profit fast before snap-back

elif regime_signal == 1:   # LIQUIDITY WITHDRAWAL STRESS
    dynamic_sl_pct  = 0.10                          # hard 10% stop (tail protection)
    if pnl_pct > 0.0:
        dynamic_tp_pct *= 0.60                      # take profit sooner under stress
    if eff_imb < -0.10:                             # OB actively opposing
        dynamic_max_hold = int(max_hold_bars * 0.40)

elif regime_signal == -1 and eff_imb > 0.15:        # CALM + FAVORABLE
    dynamic_max_hold = int(max_hold_bars * 1.50)    # let winners run

# Per-asset withdrawal (micro-level):
if withdrawal_velocity < -0.20 and not in cascade/stress:
    dynamic_max_hold = min(dynamic_max_hold, int(max_hold_bars * 0.40))
    if pnl_pct > 0.0: dynamic_tp_pct *= 0.75

7.3 Sub-day ACB Force Exit

When HZ listener fires an ACB update mid-day:

# In update_acb_boost(boost, beta):
if old_boost >= 1.25 and boost < 1.10:
    evaluate_subday_exits()   # → _execute_exit("SUBDAY_ACB_NORMALIZATION", ...)

Threshold is ARBITRARY (not backtested). Marked research TODO. Safe under pending-flag pattern (fires on next bar, not mid-loop).

7.4 Slippage on Exit

# SHORT position exit:
exit_price = current_price * (1.0 + slip)   # slippage against us when covering short
# STOP_LOSS:   slip = 0.0005 (5 bps — market order fill)
# FIXED_TP:   slip = 0.0002 (2 bps — likely limit fill)
# All others: slip = 0.0002

8. FEE & SLIPPAGE MODEL

8.1 SmartPlacer Fee Model (Layer 3)

Source: esf_alpha_orchestrator.py, _execute_exit()

Blended taker/maker fee rates based on historical SP fill statistics:

# Entry fee:
entry_fee = (0.0002 * sp_maker_entry_rate + 0.0005 * (1 - sp_maker_entry_rate)) * notional
          = (0.0002 * 0.62 + 0.0005 * 0.38) * notional
          = (0.0001240 + 0.0001900) * notional
          = 0.000314 * notional   (31.4 bps)

# Exit fee (non-SL):
exit_fee  = (0.0002 * sp_maker_exit_rate + 0.0005 * (1 - sp_maker_exit_rate)) * notional
          = (0.0002 * 0.50 + 0.0005 * 0.50) * notional
          = 0.000350 * notional   (35.0 bps)

# Exit fee (SL):
exit_fee  = 0.0005 * notional    (50 bps  pure taker)

8.2 SP Slippage Refund

# Probabilistic refund for maker fills (saves the bid-ask spread):
if rng.random() < sp_maker_entry_rate (0.62):
    pnl_pct_raw += 0.0002   # +2 bps refund
if reason != STOP_LOSS and rng.random() < sp_maker_exit_rate (0.50):
    pnl_pct_raw += 0.0002   # +2 bps refund

8.3 OB Edge (Layer 4)

# With real OB engine:
if ob_placement.depth_quality > 0.5:
    pnl_pct_raw += ob_placement.fill_probability * ob_edge_bps * 1e-4

# Without OB engine (legacy Monte Carlo fallback):
if rng.random() < ob_confirm_rate (0.40):
    pnl_pct_raw += ob_edge_bps * 1e-4   # default: +5 bps

Net PnL:

gross_pnl = pnl_pct_raw * notional
net_pnl   = gross_pnl - entry_fee - exit_fee
capital  += net_pnl

9. OB INTELLIGENCE LAYER

Source: ob_features.py, ob_provider.py, hz_ob_provider.py

The OB layer is wired in via engine.set_ob_engine(ob_engine) which propagates to signal_gen, asset_selector, and exit_manager. It is OPTIONAL — the engine degrades gracefully to legacy Monte Carlo when ob_engine=None.

9.1 OB Signals Per Asset

ob_signal = ob_engine.get_signal(asset, timestamp)
# Fields:
#   imbalance_ma5        — 5-bar MA of bid/ask size imbalance ([-1, +1])
#   imbalance_persistence — fraction of last N bars sustaining sign
#   withdrawal_velocity  — rate of depth decay (negative = book thinning)

9.2 OB Macro (Market-Wide)

ob_macro = ob_engine.get_macro()
# Fields:
#   cascade_count    — number of assets in liquidation cascade
#   regime_signal    — (-1=calm/trending, 0=neutral, +1=withdrawal stress)

9.3 OB Placement Quality

ob_placement = ob_engine.get_placement(asset, timestamp)
# Fields:
#   depth_quality    — book depth score ([0, 2+]; >1 = deep book)
#   fill_probability — probability of maker fill at entry price

9.4 OB Sub-Systems Summary

Sub Location Effect
OB-1 AlphaAssetSelector ARS adjusted ±5%/10% by depth quality
OB-2 AlphaSignalGenerator Confidence boost/dampen by imbalance; hard skip if persistent contradiction
OB-3 NDAlphaEngine._try_entry Market-wide imbalance multiplier on final leverage (±20%/15%)
OB-4 AdaptiveCircuitBreaker Macro withdrawal stress modulates ACBv6 dynamic beta (+25% max)
OB-5 AlphaExitManager Dynamic TP/SL/max_hold based on cascade/withdrawal/calm regime

10. ACB v6 — ADAPTIVE CIRCUIT BREAKER

10.1 Architecture (3-Scale Confirmation)

Source: adaptive_circuit_breaker.py, AdaptiveCircuitBreaker

Scale 1 (Daily):  External macro factors → log_0.5 base_boost
Scale 2 (Per-bar): vel_div signal strength → meta-boost multiplier
Scale 3 (Macro):  w750 eigenvalue velocity → dynamic beta switch

10.2 Scale 1 — External Factor Signals

Loaded from NG3 scan_*__Indicators.npz files (median of first 10 scans per date):

Factor Source Bearish Threshold Signal Weight
funding_btc Funding rate < -0.0001 (very) / < 0.0 (mild) 1.0 / 0.5
dvol_btc BTC implied vol > 80 (extreme) / > 55 (elevated) 1.0 / 0.5
fng Fear & Greed Index < 25 (extreme fear) / < 40 (fear) 1.0 / 0.5 (requires confirmation)
taker Taker buy ratio < 0.80 (selling) / < 0.90 (mild) 1.0 / 0.5

Signal counting:

signals = sum(individual_signal_weights)   # float, e.g. 2.5

Base boost formula:

if signals >= 1.0:
    base_boost = 1.0 + 0.5 * log1p(signals)
    # signals=1 → 1.347x | signals=2 → 1.549x | signals=3 → 1.693x
else:
    base_boost = 1.0

10.3 Scale 3 — Dynamic Beta

# Preloaded: w750_threshold = np.percentile(all_w750_vels, 60)
if w750_vel >= w750_threshold:
    beta = BETA_HIGH = 0.8    # aggressive meta-boost during macro acceleration
else:
    beta = BETA_LOW  = 0.2    # conservative during calm macro

10.4 Scale 2 — Per-Bar Meta-Boost

# Computed every bar inside _update_regime_size_mult(vel_div):
strength_cubic = clamp((threshold - vel_div) / (threshold - extreme), 0, 1) ** 3
# leverage_convexity = 3 → cubic

if day_beta > 0:
    regime_size_mult = base_boost * (1.0 + beta * strength_cubic) * mc_scale
else:
    regime_size_mult = base_boost * mc_scale

10.5 Sub-Day ACB Update (HZ Listener)

The acb_processor_service.py re-runs ACB computation mid-day when new NG3 scan data arrives and writes {boost, beta} to DOLPHIN_FEATURES IMap.

_on_acb_event() in DolphinActor stores the payload in self._pending_acb (GIL-safe dict write). Applied at start of next on_bar() iteration:

# In on_bar() — BEFORE processing:
if _pending_acb is not None and engine is not None:
    engine.update_acb_boost(pending_acb['boost'], pending_acb['beta'])
    _pending_acb = None

11. SURVIVAL STACK — POSTURE CONTROL

11.1 Overview

Source: survival_stack.py, SurvivalStack

Computes a continuous Risk Multiplier Rm ∈ [0, 1] from 5 sensor categories. Maps to discrete posture {APEX, STALKER, TURTLE, HIBERNATE}.

11.2 Five Sensor Categories

Cat1 — Binary Invariant (kill switch):

if hz_nodes < 1 or heartbeat_age_s > 30:
    return 0.0   # Total system failure → HIBERNATE immediately
return 1.0

Cat2 — Structural (MC-Forewarner + data staleness):

base = {OK: 1.0, ORANGE: 0.5, RED: 0.1}[mc_status]
decay = exp(-max(0, staleness_hours - 6) / 3)
f_structural = base * decay   # Exponential decay after 6h stale

Cat3 — Microstructure (OB depth/fill quality):

if ob_stale:
    return 0.5
score = min(depth_quality, fill_prob)
return clamp(0.3 + 0.7 * score, 0.3, 1.0)

Cat4 — Environmental (DVOL spike impulse):

if dvol_spike and t_since_spike_min == 0:
    return 0.3   # Instant degradation at spike
return 0.3 + 0.7 * (1 - exp(-t_since_spike_min / 60))  # 60-min recovery tau

Cat5 — Capital (sigmoid drawdown constraint):

# Rm5 ≈ 1.0 at DD<5%, ≈ 0.5 at DD=12%, ≈ 0.1 at DD=20%
return 1 / (1 + exp(30 * (drawdown - 0.12)))

11.3 Hierarchical Combination

f_environment = min(f_structural, f_ext)     # worst of Cat2/Cat4
f_execution   = f_micro                       # Cat3
r_target      = Cat1 * Cat5 * f_environment * f_execution

# Correlated sensor collapse penalty:
degraded = count([f_structural < 0.8, f_micro < 0.8, f_ext < 0.8])
if degraded >= 2:
    r_target *= 0.5

11.4 Bounded Recovery Dynamics

# Fast attack (instant degradation), slow recovery (5%/minute max):
if r_target < last_r_total:
    r_final = r_target          # immediate drop
else:
    alpha = min(1.0, 0.02 * dt_min)
    step  = min(alpha * (r_target - last_r_total), 0.05 * dt_min)
    r_final = last_r_total + step

11.5 Posture Mapping

NOTE: Thresholds are deliberately TIGHTER than mathematical spec (safety buffer).

if Rm >= 0.90: APEX       # Full trading, no constraints
if Rm >= 0.75: STALKER    # Max leverage capped at 2.0x
if Rm >= 0.50: TURTLE     # regime_dd_halt = True (no new entries)
else:          HIBERNATE  # Force-close open positions, no new entries

11.6 Hysteresis

# Down: requires hysteresis_down=2 consecutive bars at lower level
# Up:   requires hysteresis_up=5 consecutive bars at higher level
# Prevents flip-flopping around thresholds

11.7 Posture → Engine Effect

Posture Engine Effect
APEX No constraint (max leverage = abs_max=6.0 × regime_size_mult)
STALKER clamped_max_leverage = min(..., 2.0) in _try_entry
TURTLE regime_dd_halt = Trueprocess_bar skips entry block
HIBERNATE _manage_position forces EXIT("HIBERNATE_HALT"), regime_dd_halt = True

12. MC-FOREWARNER ENVELOPE GATE

Source: Called via engine.set_mc_forewarner(forewarner, mc_base_cfg)

Runs daily at start of process_day():

mc_cfg = {**mc_base_cfg, 'max_leverage': base_max_leverage * day_base_boost}
mc_report = forewarner.assess_config_dict(mc_cfg)

mc_red    = mc_report.catastrophic_probability > 0.25 or mc_report.envelope_score < -1.0
mc_orange = (not mc_red) and (envelope_score < 0 or catastrophic_probability > 0.10)

day_mc_status = 'RED' | 'ORANGE' | 'OK'
day_mc_scale  = 0.5 if ORANGE else 1.0

if mc_red or posture in ['TURTLE', 'HIBERNATE']:
    regime_dd_halt = True   # No new entries for the entire day

13. NDAlphaEngine — FULL BAR LOOP

13.1 process_day() Flow

def process_day(date_str, df, asset_columns, vol_regime_ok, direction, posture):
    # --- Per-day initialization ---
    regime_direction = direction or -1     # -1=SHORT champion
    regime_dd_halt   = False
    day_posture      = posture
    
    # --- ACB: Scale 1 + Scale 3 (once per day) ---
    if acb:
        info = acb.get_dynamic_boost_for_date(date_str, ob_engine)
        day_base_boost = info['boost']    # log_0.5 curve
        day_beta       = info['beta']     # dynamic beta (w750 percentile switch)
    
    # --- MC-Forewarner gate (once per day) ---
    if forewarner:
        mc_report = forewarner.assess_config_dict(...)
        if mc_red or posture in [TURTLE, HIBERNATE]:
            regime_dd_halt = True   # kill new entries for day
    
    # === BAR LOOP ===
    for ri, row in df.iterrows():
        vd       = row['vel_div']            # primary signal
        v50_vel  = row['v50_lambda_max_velocity']
        v750_vel = row['v750_lambda_max_velocity']
        prices   = {col: row[col] for col in asset_columns if row[col] > 0}
        
        # Append to price_histories (capped at 200; sliding window)
        for ac, p in prices.items():
            price_histories[ac].append(p)
            if len(price_histories[ac]) > 500:
                price_histories[ac] = price_histories[ac][-200:]
        
        vol_ok = vol_regime_ok[ri] if vol_regime_ok else (bid >= 100)
        
        # ACB Scale 2: regime_size_mult updated every bar
        _update_regime_size_mult(vd)
        
        process_bar(global_bar_idx, vd, prices, vol_ok, price_histories, v50_vel, v750_vel)
        global_bar_idx += 1
    
    return {date, pnl, capital, boost, beta, mc_status, trades}

13.2 process_bar() Flow

def process_bar(bar_idx, vel_div, prices, vol_regime_ok, price_histories, v50_vel, v750_vel):
    bar_count += 1
    vel_div_history.append(vel_div)   # trimmed to 200
    
    # === EXIT MANAGEMENT (always first) ===
    if position is not None:
        exit_info = _manage_position(bar_idx, prices, vel_div, v50_vel, v750_vel)
        # → AlphaExitManager.evaluate() → if EXIT: _execute_exit()
    
    # === ENTRY (only when no position) ===
    if position is None AND bar_idx > last_exit_bar AND NOT regime_dd_halt:
        if bar_count >= lookback (100) AND vol_regime_ok:
            entry_info = _try_entry(bar_idx, vel_div, prices, price_histories, v50_vel, v750_vel)

13.3 _try_entry() Flow

def _try_entry(bar_idx, vel_div, prices, price_histories, v50_vel, v750_vel):
    if capital <= 0: return None
    
    # 1. IRP Asset Selection (Layer 2)
    if use_asset_selection:
        market_data = {a: history[-50:] for a, history in price_histories if len >= 50}
        rankings = asset_selector.rank_assets(market_data, regime_direction)
        trade_asset = first_asset_passing_all_gates(rankings)
        if trade_asset is None: return None   # strict: no fallback
    else:
        trade_asset = "BTCUSDT"   # fallback when IRP disabled
    
    # 2. Signal Generation + DC (Layer 6)
    signal = signal_gen.generate(vel_div, vel_div_history,
                                  price_histories[trade_asset],
                                  regime_direction, trade_asset)
    if not signal.is_valid: return None   # vel_div or DC killed it
    
    # 3. Position Sizing (Layers 7-8)
    size = bet_sizer.calculate_size(capital, vel_div, signal.vel_div_trend, regime_direction)
    
    # 4. OB Sub-3: Cross-asset market multiplier
    market_ob_mult = ob_engine.get_market_multiplier(...)  # ±20%
    
    # 5. ACB leverage ceiling enforcement
    clamped_max = min(base_max_leverage * regime_size_mult * market_ob_mult, abs_max_leverage=6.0)
    if posture == STALKER: clamped_max = min(clamped_max, 2.0)
    final_leverage = clamp(size.leverage * regime_size_mult * market_ob_mult, min_lev, clamped_max)
    
    # 6. Notional and entry
    notional = capital * size.fraction * final_leverage
    entry_price = prices[trade_asset]
    
    # 7. Create position
    position = NDPosition(trade_asset, regime_direction, entry_price,
                          notional, final_leverage, ...)
    exit_manager.setup_position(trade_id, entry_price, direction, bar_idx, v50_vel, v750_vel)

14. DOLPHIN ACTOR — NAUTILUS INTEGRATION

Source: dolphin_actor.py, DolphinActor(Strategy)

14.1 Lifecycle

__init__:    set defaults, _pending_acb = None
on_start:    connect HZ, read posture, setup ACB listener, instantiate NDAlphaEngine
on_bar:      (fires every 5s)
  1. Apply _pending_acb if set (thread-safe drain from HZ listener)
  2. Parse bar date
  3. On date change: re-read posture from HZ
  4. If date already processed: RETURN (strict once-per-day guard)
  5. If HIBERNATE: process_day(empty df), mark done, write HZ, RETURN
  6. Load parquet: _load_parquet_data(date_str)
  7. process_day(date_str, df, asset_cols, vol_regime_ok, direction, posture)
  8. _processed_dates.add(date_str)
  9. _write_result_to_hz(date_str, result)
on_stop:     hz_client.shutdown()

14.2 Thread Safety: ACB Pending-Flag Pattern

CRITICAL: HZ entry listeners run on HZ client pool threads, NOT the Nautilus event loop.

# HZ thread (ONLY writes _pending_acb — no engine access):
def _on_acb_event(event):
    if event.value:
        self._pending_acb = json.loads(event.value)   # GIL-safe dict assignment

# Nautilus event loop thread (only reader + engine caller):
def on_bar(bar):
    if _pending_acb is not None and engine is not None:
        engine.update_acb_boost(_pending_acb['boost'], _pending_acb['beta'])
        _pending_acb = None
    # ... rest of on_bar

Max ACB latency: 1 bar interval (5 seconds). Acceptable given ACB changes on minute-to-hour timescale.

14.3 Data Loading

def _load_parquet_data(date_str):
    path = HCM_DIR / "vbt_cache_klines" / f"{date_str}.parquet"
    df = pd.read_parquet(path)
    meta_cols = {vel_div, scan_number, v50_..., v150_..., v300_..., v750_..., instability_50, instability_150}
    asset_columns = [c for c in df.columns if c not in meta_cols]
    vol_regime_ok = None   # deferred to process_day (bar >= 100 gate)
    return df, asset_columns, None

14.4 Posture Reading

Primary: HZ CP Subsystem AtomicReference('DOLPHIN_SAFETY') — linearizable.
Fallback: HZ IMap('DOLPHIN_SAFETY').get('latest') — eventually consistent.

14.5 Result Writing

def _write_result_to_hz(date_str, result):
    imap_pnl = hz_client.get_map('DOLPHIN_PNL_BLUE').blocking()
    imap_pnl.put(date_str, json.dumps(result))
    # result: {date, pnl, capital, boost, beta, mc_status, trades}

15. HAZELCAST FEATURE STORE — SHARDING

Source: hz_sharded_feature_store.py, ShardedFeatureStore

15.1 Shard Routing

SHARD_COUNT = 10
shard = sum(ord(c) for c in symbol) % SHARD_COUNT
map_name = f"DOLPHIN_FEATURES_SHARD_{shard:02d}"   # ..._00 through ..._09

Hash is stable (sum-of-ord, not Python's hash()), deterministic across process restarts.

15.2 API

store = ShardedFeatureStore(hz_client)
store.put('BTCUSDT', 'vel_div', -0.03)   # routes to shard based on symbol hash
val  = store.get('BTCUSDT', 'vel_div')   # same shard
store.delete('BTCUSDT', 'vel_div')

# Key format inside IMap: "vel_div_BTCUSDT"
# Map handles cached after first access

15.3 Near Cache Configuration

near_cache_config = ShardedFeatureStore.near_cache_config()
# Returns dict for all 10 shards:
# { 'DOLPHIN_FEATURES_SHARD_XX': {
#     invalidate_on_change: True,
#     time_to_live_seconds: 300,
#     max_idle_seconds: 60,
#     eviction_policy: LRU,
#     max_size: 5000
#   }
# }
hz = hazelcast.HazelcastClient(near_caches=near_cache_config, ...)

15.4 Concurrency Notes

Risk Severity Mitigation
_maps dict not thread-safe LOW Only accessed from actor event loop (single thread)
Multiple ACBProcessorService writers MEDIUM CP FencedLock (only if all writers use it)
HZ Near Cache stale LOW TTL=300s, invalidate_on_change=True

16. PRODUCTION DAEMON TOPOLOGY

16.1 ACB Processor Service

Purpose: Computes ACBv6 daily boost + dynamic beta and writes to HZ DOLPHIN_FEATURES IMap.

Schedule: Runs once at market open (08:00 UTC), then re-runs whenever a new NG3 scan batch arrives. Uses CP FencedLock to prevent simultaneous writes.

Output: DOLPHIN_FEATURES.put('acb_boost', json.dumps({boost, beta}))

16.2 OB Stream Service

Purpose: Fetches live Binance order book snapshots at 500ms cadence. Computes per-asset imbalance, fills, and macro signals. Writes to HZ.

Cadence: 500ms per cycle (2 updates/second).

16.3 System Watchdog Service

Purpose: Reads HZ heartbeat, OB quality, MC status, drawdown → runs SurvivalStack.compute_rm() → writes posture to DOLPHIN_SAFETY AtomicReference via SurvivalStack.write_to_hz().

Cadence: ~60s per cycle.

16.4 MC-Forewarner Flow

Purpose: Prefect-orchestrated. Assesses current config envelope daily using correlation matrix statistics. Outcome: OK / ORANGE / RED status in HZ. ORANGE → day_mc_scale=0.5. RED → regime_dd_halt=True.

16.5 paper_trade_flow.py

Purpose: Main production entry point. Instantiates DolphinActor and wires into Nautilus BacktestEngine (paper mode). Subscribes to CCXT 5s bars. Runs indefinitely.

Direction: direction = -1 (SHORT, blue posture). GREEN (LONG) is separate configuration.

16.6 Daemon Start Sequence

1. Start Hazelcast (port 5701)
2. acb_processor_service.py    ← writes initial ACB boost before market open
3. ob_stream_service.py        ← start OB snapshots
4. system_watchdog_service.py  ← begin posture computation
5. mc_forewarner_flow.py       ← Prefect flow (port 4200 UI)
6. paper_trade_flow.py         ← Start actor LAST (reads all the above from HZ)

16.7 Monitoring Endpoints

Service URL / Path
Hazelcast MC Console http://localhost:8080
Prefect UI http://localhost:4200
Daily PnL HZ IMap: DOLPHIN_PNL_BLUE (key = date string)
Posture HZ AtomicRef: DOLPHIN_SAFETY
ACB State HZ IMap: DOLPHIN_FEATURES (key = 'acb_boost')

17. CI TEST SUITE

Location: ci/ directory. Runner: pytest ci/ -v

Passing gate: 14/14 tests in 7-10s.

File Tests What It Covers
test_13_nautilus_integration.py 6 Actor import, instantiation, on_bar, HIBERNATE posture, once-per-day guard, ACB thread safety
test_14_long_system.py 3 Multi-day run, capital persistence, trade count
test_15_acb_reactive.py 1 ACB boost update applied correctly mid-day
test_16_scaling.py 4 Memory footprint <4GB (50 assets), shard routing (400 symbols), 400-asset no-crash, 400-asset with IRP

17.1 Key Test: test_acb_listener_does_not_call_engine_directly

# Patches engine.update_acb_boost with a mock, fires _on_acb_event
# Asserts: _pending_acb is set, engine method was NOT called
with patch.object(actor.engine, 'update_acb_boost') as mock_update:
    actor._on_acb_event(event)
    assert actor._pending_acb['boost'] == 1.35
    mock_update.assert_not_called()

17.2 Key Test: test_process_day_called_once_per_day

# Fires 3 bars on same date → assert process_day.call_count == 1
# Fires 1 bar on next date  → assert process_day.call_count == 2

18. PARAMETER REFERENCE

18.1 Champion Parameters (Frozen)

Parameter Value Layer Notes
vel_div_threshold -0.02 Signal gate PRIMARY entry threshold
vel_div_extreme -0.05 Signal/sizing Extreme regime: full size
min_leverage 0.5 Sizing Floor leverage
max_leverage 5.0 Sizing Base ceiling (before ACB)
abs_max_leverage 6.0 Sizing Hard geometric ceiling
leverage_convexity 3.0 Sizing Cubic convex curve
fraction 0.20 Sizing Max capital fraction per trade
fixed_tp_pct 0.0095 Exit 95 bps take-profit
stop_pct 1.0 Exit Effectively disabled
max_hold_bars 120 Exit 600 seconds
dc_lookback_bars 7 DC 35 seconds price momentum
dc_min_magnitude_bps 0.75 DC Minimum BTC momentum
dc_skip_contradicts True DC Hard skip on contradiction
min_irp_alignment 0.45 IRP Alignment gate
sp_maker_entry_rate 0.62 Fees 62% maker fill at entry
sp_maker_exit_rate 0.50 Fees 50% maker fill at exit
ob_edge_bps 5.0 OB Legacy MC OB edge
ob_confirm_rate 0.40 OB Legacy MC confirmation rate
lookback 100 Warmup Bars before first entry allowed
seed 42 RNG Deterministic numpy RandomState

18.2 ACBv6 Parameters (Frozen — Validated)

Parameter Value Notes
BETA_HIGH 0.8 w750 above p60 threshold
BETA_LOW 0.2 w750 below p60 threshold
W750_THRESHOLD_PCT 60 Percentile switch point
FUNDING_VERY_BEARISH -0.0001 1.0 signal
DVOL_EXTREME 80 1.0 signal
FNG_EXTREME_FEAR 25 1.0 signal (needs confirmation)
TAKER_SELLING 0.8 1.0 signal

18.3 Survival Stack Thresholds (Deliberately Tight)

Posture Rm Threshold vs. Math Spec
APEX ≥ 0.90 Tighter — spec was 0.85
STALKER ≥ 0.75 Tighter — spec was 0.70
TURTLE ≥ 0.50 Tighter — spec was 0.45
HIBERNATE < 0.50

Do NOT loosen these without quantitative justification.


19. KNOWN RESEARCH TODOs

ID Description Priority
TODO-1 Calibrate vd_enabled adverse-turn exits (currently disabled). Requires analysis of trade vel_div distribution at entry vs. subsequent bars. True invalidation threshold likely ~+0.02 sustained for N=3 bars. MEDIUM
TODO-2 Validate SUBDAY_ACB force-exit threshold (old_boost >= 1.25 and boost < 1.10). Currently ARBITRARY — agent-chosen, not backtest-derived. MEDIUM
TODO-3 MIG8: Binance live adapter (real order execution). OUT OF SCOPE until after 30-day paper trading validation. LOW
TODO-4 48-hour chaos test with all daemons running simultaneously. Watch for: KeyError, stale-read anomalies, concurrent HZ writer collisions. HIGH (before live capital)
TODO-5 Memory profiler with IRP enabled at 400 assets (current 71 MB measurement was without IRP). Projected ~600 MB — verify. LOW
TODO-6 TF-spread recovery exits (tf_enabled=False). Requires sweep of tf_exhaust_ratio and tf_flip_ratio vs. champion backtest. LOW
TODO-7 GREEN (LONG) posture paper validation. LONG thresholds (long_threshold=0.01, long_extreme=0.04) not yet production-validated. MEDIUM

End of DOLPHIN-NAUTILUS System Bible v1.0 — 2026-03-07
Champion: SHORT only (APEX posture, blue configuration)
Gate: 14/14 CI tests green. Paper trading ready.
Do NOT deploy real capital until 30-day paper run is clean.