Includes core prod + GREEN/BLUE subsystems: - prod/ (BLUE harness, configs, scripts, docs) - nautilus_dolphin/ (GREEN Nautilus-native impl + dvae/ preserved) - adaptive_exit/ (AEM engine + models/bucket_assignments.pkl) - Observability/ (EsoF advisor, TUI, dashboards) - external_factors/ (EsoF producer) - mc_forewarning_qlabs_fork/ (MC regime/envelope) Excludes runtime caches, logs, backups, and reproducible artifacts per .gitignore.
41 KiB
Executable File
DOLPHIN-NAUTILUS SYSTEM BIBLE
Doctrinal Reference — As Running 2026-03-07
Version: MIG7 Complete (post-MIG7_MIG8_FIXES)
CI gate: 14/14 tests green
Status: Paper trading ready. NOT deployed with real capital.
TABLE OF CONTENTS
- System Philosophy
- Physical Architecture
- Data Layer
- Signal Layer — vel_div & DC
- Asset Selection — IRP
- Position Sizing — AlphaBetSizer
- Exit Management
- Fee & Slippage Model
- OB Intelligence Layer
- ACB v6 — Adaptive Circuit Breaker
- Survival Stack — Posture Control
- MC-Forewarner Envelope Gate
- NDAlphaEngine — Full Bar Loop
- DolphinActor — Nautilus Integration
- Hazelcast Feature Store — Sharding
- Production Daemon Topology
- CI Test Suite
- Parameter Reference
- Known Research TODOs
1. SYSTEM PHILOSOPHY
DOLPHIN-NAUTILUS is a SHORT-only (champion configuration) systematic trading engine targeting crypto perpetual futures on Binance.
Core thesis: When crypto market correlation matrices show accelerating eigenvalue-velocity divergence (vel_div < -0.02), the market is entering an instability regime. Shorting during early instability onset and exiting at fixed take-profit captures the mean-reversion from panic to normalization.
Design constraints:
- Zero signal re-implementation in the Nautilus layer. All alpha logic lives in
NDAlphaEngine. - 512-bit arithmetic for correlation matrix processing (separate NG3 pipeline; not in hot path of this engine).
- Champion parameters are FROZEN. They were validated via exhaustive VBT backtest on
dolphin_vbt_real.py. - The Nautilus actor is a thin wire, not a strategy. It routes parquet data → NDAlphaEngine → HZ result.
Champion performance (ACBv6 + IRP + DC + OB, full-stack 55-day Dec31–Feb25):
- ROI: +54.67% | PF: 1.141 | Sharpe: 2.84 | Max DD: 15.80% | WR: 49.5% | Trades: 2145
- Log:
run_logs/summary_20260307_163401.json
Data correction note (2026-03-07): An earlier reference showed ROI=+57.18%, PF=1.149, Sharpe=3.00. Those figures came from a stale
vbt_cache/2026-02-25.parquetthat was built mid-day — missing 435 scans and carrying corrupt vel_div on 492 rows for the final day of the window. ALGO-3 parity testing caught the mismatch (max_diff=1.22 vs tolerance 1e-10). The parquet was rebuilt from live NG3 JSON (build_parquet_cache(dates=['2026-02-25'], force=True)). The stale file is preserved as2026-02-25.parquet.STALE_20260307for replicability. The corrected numbers above are the canonical reference. The ~2.5pp ROI drop reflects real late-day trades on Feb 25 that the stale parquet had silently omitted.
2. PHYSICAL ARCHITECTURE
┌─────────────────────────────────────────────────────────────────────┐
│ DATA SOURCES │
│ NG3 Scanner → correlation_arb512/ (512-bit eigenvalue matrices) │
│ CCXT Live Feed → 5s OHLCV bars (48+ USDT perpetuals on Binance) │
│ VBT Cache → vbt_cache_klines/*.parquet (pre-built from CCXT) │
└───────────────────┬──────────────────────────────────────────────────┘
│
┌───────────────────▼──────────────────────────────────────────────────┐
│ HAZELCAST IN-MEMORY GRID (localhost:5701, cluster: "dolphin") │
│ IMaps: │
│ DOLPHIN_SAFETY → AtomicReference / IMap (posture JSON) │
│ DOLPHIN_FEATURES → ACB boost + OB features per bar │
│ DOLPHIN_PNL_BLUE → Per-date trade results (SHORT posture) │
│ DOLPHIN_PNL_GREEN → Per-date trade results (LONG posture) │
│ DOLPHIN_FEATURES_SHARD_00..09 → 400-asset sharded feature store │
└───────────────────┬──────────────────────────────────────────────────┘
│
┌───────────────────▼──────────────────────────────────────────────────┐
│ PRODUCTION DAEMONS (each runs in its own process/thread) │
│ acb_processor_service.py — Daily ACB boost computation + HZ write │
│ ob_stream_service.py — 500ms OB snapshot ingestion + HZ write │
│ system_watchdog_service.py — Survival Stack Rm computation │
│ mc_forewarner_flow.py — Prefect-orchestrated MC gate │
└───────────────────┬──────────────────────────────────────────────────┘
│
┌───────────────────▼──────────────────────────────────────────────────┐
│ NAUTILUS TRADING ENGINE │
│ paper_trade_flow.py → DolphinActor → NDAlphaEngine │
│ Receives 5s bars via Nautilus event loop │
│ Calls process_day() once per calendar date │
└─────────────────────────────────────────────────────────────────────┘
Key invariant: DolphinActor.on_bar() fires ~17,280 times per day (every 5 seconds). process_day() is called exactly once per calendar date — guarded by _processed_dates set.
3. DATA LAYER
3.1 vbt_cache_klines Parquet Schema
Location: C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict\vbt_cache_klines\YYYY-MM-DD.parquet
| Column | Type | Description |
|---|---|---|
vel_div |
float64 | Eigenvalue velocity divergence: v50_vel − v750_vel (primary signal) |
v50_lambda_max_velocity |
float64 | Short-window (50-bar) lambda_max rate of change |
v150_lambda_max_velocity |
float64 | 150-bar window lambda velocity |
v300_lambda_max_velocity |
float64 | 300-bar window lambda velocity |
v750_lambda_max_velocity |
float64 | Long-window (750-bar) macro eigenvalue velocity |
instability_50 |
float64 | General market instability index (50-bar) |
instability_150 |
float64 | General market instability index (150-bar) |
BTCUSDT … STXUSDT |
float64 | Per-asset close prices (48 assets in current dataset) |
Each file: 1,439 rows (1 per 5-second bar over 24h), 57 columns.
3.2 NG3 Eigenvalue Data
Location: C:\Users\Lenovo\Documents\- Dolphin NG HD (NG3)\correlation_arb512\
eigenvalues/
YYYY-MM-DD/
scan_NNNNNN__Indicators.npz ← ACBv6 external factors (funding, dvol, fng, taker)
scan_NNNNNN__scan_global.npz ← lambda_vel_w750 for dynamic beta
matrices/
YYYY-MM-DD/
scan_NNNNNN_w50_HHMMSS.arb512.pkl.zst ← 512-bit correlation matrix (unused in hot path)
NPZ files loaded by AdaptiveCircuitBreaker._load_external_factors() (max 10 scans per date, median-aggregated).
4. SIGNAL LAYER
4.1 Primary Signal: vel_div Threshold Gate
Source: alpha_signal_generator.py, AlphaSignalGenerator.generate()
SHORT signal condition:
vel_div < VEL_DIV_THRESHOLD (-0.02)
LONG signal condition (green posture, not champion):
vel_div > LONG_THRESHOLD (0.01)
Confidence calculation (SHORT path):
ratio = clamp((threshold - vel_div) / (threshold - extreme), 0, 1)
= clamp((-0.02 - vel_div) / (-0.02 - (-0.05)), 0, 1)
= clamp((-0.02 - vel_div) / 0.03, 0, 1)
confidence = 0.50 + ratio * 0.40 # range: [0.50, 0.90]
is_extreme = (vel_div <= -0.05)
4.2 Direction Confirmation (DC) — Layer 6
Source: alpha_signal_generator.py, check_dc_nb() (numba JIT)
# Looks back dc_lookback_bars (default 7) bars on the selected asset price
p0 = price[n - lookback - 1]
p1 = price[n - 1]
chg_bps = (p1 - p0) / p0 * 10000.0
if chg_bps < -min_magnitude_bps (-0.75): return CONFIRM # falling price → SHORT OK
if chg_bps > min_magnitude_bps (+0.75): return CONTRADICT
else: return NEUTRAL
dc_skip_contradicts = True (champion): CONTRADICT returns null signal (skip entry).
Effect on leverage: DC has dc_leverage_boost=1.0 (no boost in champion). CONTRADICT kills entry.
4.3 OB Sub-2: Per-Asset Imbalance Confirmation
When ob_engine is wired in (use_ob_edge=True):
eff_imb = -ob_signal.imbalance_ma5 # For SHORT: sell pressure = positive eff_imb
if eff_imb > 0.10: # OB confirms → confidence boost ≤+15%
ob_adj = 1 + min(0.15, eff_imb * persistence * 0.5)
confidence *= ob_adj
elif eff_imb < -0.15 and persistence > 0.60: # Strong persistent OB contradiction → HARD SKIP
return null_signal
elif eff_imb < -0.10: # Moderate → soft dampen confidence
ob_adj = max(0.85, 1 - |eff_imb| * persistence * 0.4)
confidence *= ob_adj
5. ASSET SELECTION — IRP
5.1 Overview
Source: alpha_asset_selector.py, AlphaAssetSelector.rank_assets() + numba kernels
IRP = Impulse Response Profiling. Ranks all available assets by historical behavior over the last 50 bars in the regime direction. Selects the asset with the highest ARS (Asset Ranking Score) that passes all filters.
Enabled by: use_asset_selection=True (production default).
5.2 Numba Kernel: compute_irp_nb
# Input: price_segment (last 50 prices), direction (-1 or +1)
dir_returns[i] = (price[i+1] - price[i]) * direction # directional returns
cumulative = cumsum(dir_returns)
mfe = max(cumulative) # Maximum Favorable Excursion
mae = abs(min(cumulative, 0)) # Maximum Adverse Excursion
efficiency = mfe / (mae + 1e-6)
alignment = count(dir_returns > 0) / n_ret
noise = variance(dir_returns)
latency = bars_to_reach_10pct_of_mfe # (default: 50 if mfe==0)
5.3 Numba Kernel: compute_ars_nb
ARS = 0.5 * log1p(efficiency) + 0.35 * alignment - 0.15 * noise * 1000
5.4 Numba Kernel: rank_assets_irp_nb
For each asset:
- Compute IRP in DIRECT direction (regime_direction)
- Compute IRP in INVERSE direction (-regime_direction)
- Take whichever gives higher ARS (allows inverse selection)
- Apply filter gates:
noise > 500→ skiplatency > 20→ skip (must reach 10% MFE within 20 bars)alignment < 0.20→ skip
- Bubble-sort by ARS descending (numba nopython)
5.5 AlphaAssetSelector Python Wrapper
# Build 2D array (max_len × n_assets), right-aligned
valid = rank_assets_irp_nb(prices_2d, idx=max_len, regime_direction, ...)
# Walk ranked list:
for r in rankings:
if min_irp_alignment > 0 and r.metrics.alignment < min_irp_alignment:
continue # alignment gate (default 0.45)
if r.action != expected_action:
continue # direction gate
if ob_engine and ob_placement.depth_quality < 0.20:
continue # OB depth gate (try next asset)
trade_asset = r.asset
break
# No match → return None (no fallback to BTCUSDT when IRP enabled)
OB Sub-1: ARS adjusted ±5%/10% by per-asset OB depth quality before sorting.
6. POSITION SIZING
6.1 Signal Strength Score
Source: alpha_bet_sizer.py, compute_sizing_nb() (numba JIT)
# SHORT path (vel_div < threshold):
if vel_div <= extreme (-0.05):
strength_score = 1.0
else:
strength_score = (threshold - vel_div) / (threshold - extreme)
# = (-0.02 - vel_div) / 0.03
strength_score = clamp(strength_score, 0.0, 1.0)
6.2 Dynamic Leverage (Cubic Convex)
scaled_score = strength_score ** leverage_convexity (3.0)
eff_leverage = min_leverage + scaled_score * (max_leverage - min_leverage)
= 0.5 + scaled_score³ * 4.5 # range: [0.5, 5.0] before ACB
6.3 Signal Bucket Classification
# Numba: get_signal_bucket_nb(vel_div, threshold=-0.02, extreme=-0.05)
if vel_div <= extreme * 1.5 (-0.075): bucket = 0 # "extreme"
elif vel_div <= extreme (-0.05): bucket = 1 # "strong"
elif vel_div <= (threshold+extreme)/2: bucket = 2 # "moderate" (-0.035)
else: bucket = 3 # "weak"
6.4 Alpha Layers (Layer 7)
Bucket Boost — adaptive win-rate feedback:
# get_bucket_boost_nb: per-bucket win rate → multiplier
wr > 0.60 → 1.3x | wr > 0.55 → 1.1x | wr < 0.40 → 0.7x | wr < 0.45 → 0.85x
Streak Multiplier — recent 5-trade loss streak:
# get_streak_mult_nb
losses_in_last_5 >= 4 → 0.5x | >= 3 → 0.7x | <= 1 → 1.1x
Trend Multiplier — vel_div acceleration:
# get_trend_mult_nb(vd_trend = vel_div_history[-1] - vel_div_history[-10])
vd_trend < -0.01 → 1.3x (deepening instability)
vd_trend > 0.01 → 0.7x (recovering)
Effective Fraction computation:
confidence = 0.70 if is_extreme else 0.55
conf_mult = confidence / 0.95
extreme_boost = 2.0 if is_extreme else 1.0
base_frac = 0.02 + strength_score * (base_fraction - 0.02)
eff_fraction = base_frac * conf_mult * extreme_boost * trend_mult * bucket_boost * streak_mult
eff_fraction = clamp(eff_fraction, 0.02, base_fraction=0.20)
Final notional:
notional = capital * eff_fraction * final_leverage
6.5 ACB + MC Size Multiplier
# regime_size_mult is recomputed every bar via _update_regime_size_mult(vel_div)
if day_beta > 0:
strength_cubic = clamp((threshold - vel_div) / (threshold - extreme), 0, 1) ** convexity
regime_size_mult = day_base_boost * (1.0 + day_beta * strength_cubic) * day_mc_scale
else:
regime_size_mult = day_base_boost * day_mc_scale
# Applied to leverage ceiling:
clamped_max_leverage = min(base_max_leverage * regime_size_mult * market_ob_mult, abs_max_leverage=6.0)
raw_leverage = size_result["leverage"] * dc_lev_mult * regime_size_mult * market_ob_mult
# STALKER posture hard cap:
if posture == 'STALKER': clamped_max_leverage = min(clamped_max_leverage, 2.0)
final_leverage = clamp(raw_leverage, min_leverage=0.5, clamped_max_leverage)
7. EXIT MANAGEMENT
7.1 Exit Priority Order (champion)
Source: alpha_exit_manager.py, AlphaExitManager.evaluate()
- FIXED_TP:
pnl_pct >= 0.0095(95 basis points) - STOP_LOSS:
pnl_pct <= -1.0(DISABLED in practice — 100% loss never triggers before TP/max_hold) - OB DURESS exits (when ob_engine != None):
- Cascade Detection:
cascade_count > 0→ widen TP ×1.40, halve max_hold - Liquidity Withdrawal:
regime_signal == 1→ hard SL 10%, TP ×0.60
- Cascade Detection:
- vel_div adverse-turn exits (
vd_enabled=Falseby default — disabled pending calibration) - MAX_HOLD:
bars_held >= 120(= 600 seconds)
7.2 OB Dynamic Exit Parameter Adjustment
if cascade_count > 0:
dynamic_tp_pct *= 1.40
dynamic_max_hold = int(max_hold_bars * 0.50) # take profit fast before snap-back
elif regime_signal == 1: # LIQUIDITY WITHDRAWAL STRESS
dynamic_sl_pct = 0.10 # hard 10% stop (tail protection)
if pnl_pct > 0.0:
dynamic_tp_pct *= 0.60 # take profit sooner under stress
if eff_imb < -0.10: # OB actively opposing
dynamic_max_hold = int(max_hold_bars * 0.40)
elif regime_signal == -1 and eff_imb > 0.15: # CALM + FAVORABLE
dynamic_max_hold = int(max_hold_bars * 1.50) # let winners run
# Per-asset withdrawal (micro-level):
if withdrawal_velocity < -0.20 and not in cascade/stress:
dynamic_max_hold = min(dynamic_max_hold, int(max_hold_bars * 0.40))
if pnl_pct > 0.0: dynamic_tp_pct *= 0.75
7.3 Sub-day ACB Force Exit
When HZ listener fires an ACB update mid-day:
# In update_acb_boost(boost, beta):
if old_boost >= 1.25 and boost < 1.10:
evaluate_subday_exits() # → _execute_exit("SUBDAY_ACB_NORMALIZATION", ...)
Threshold is ARBITRARY (not backtested). Marked research TODO. Safe under pending-flag pattern (fires on next bar, not mid-loop).
7.4 Slippage on Exit
# SHORT position exit:
exit_price = current_price * (1.0 + slip) # slippage against us when covering short
# STOP_LOSS: slip = 0.0005 (5 bps — market order fill)
# FIXED_TP: slip = 0.0002 (2 bps — likely limit fill)
# All others: slip = 0.0002
8. FEE & SLIPPAGE MODEL
8.1 SmartPlacer Fee Model (Layer 3)
Source: esf_alpha_orchestrator.py, _execute_exit()
Blended taker/maker fee rates based on historical SP fill statistics:
# Entry fee:
entry_fee = (0.0002 * sp_maker_entry_rate + 0.0005 * (1 - sp_maker_entry_rate)) * notional
= (0.0002 * 0.62 + 0.0005 * 0.38) * notional
= (0.0001240 + 0.0001900) * notional
= 0.000314 * notional (31.4 bps)
# Exit fee (non-SL):
exit_fee = (0.0002 * sp_maker_exit_rate + 0.0005 * (1 - sp_maker_exit_rate)) * notional
= (0.0002 * 0.50 + 0.0005 * 0.50) * notional
= 0.000350 * notional (35.0 bps)
# Exit fee (SL):
exit_fee = 0.0005 * notional (50 bps — pure taker)
8.2 SP Slippage Refund
# Probabilistic refund for maker fills (saves the bid-ask spread):
if rng.random() < sp_maker_entry_rate (0.62):
pnl_pct_raw += 0.0002 # +2 bps refund
if reason != STOP_LOSS and rng.random() < sp_maker_exit_rate (0.50):
pnl_pct_raw += 0.0002 # +2 bps refund
8.3 OB Edge (Layer 4)
# With real OB engine:
if ob_placement.depth_quality > 0.5:
pnl_pct_raw += ob_placement.fill_probability * ob_edge_bps * 1e-4
# Without OB engine (legacy Monte Carlo fallback):
if rng.random() < ob_confirm_rate (0.40):
pnl_pct_raw += ob_edge_bps * 1e-4 # default: +5 bps
Net PnL:
gross_pnl = pnl_pct_raw * notional
net_pnl = gross_pnl - entry_fee - exit_fee
capital += net_pnl
9. OB INTELLIGENCE LAYER
Source: ob_features.py, ob_provider.py, hz_ob_provider.py
The OB layer is wired in via engine.set_ob_engine(ob_engine) which propagates to signal_gen, asset_selector, and exit_manager. It is OPTIONAL — the engine degrades gracefully to legacy Monte Carlo when ob_engine=None.
9.1 OB Signals Per Asset
ob_signal = ob_engine.get_signal(asset, timestamp)
# Fields:
# imbalance_ma5 — 5-bar MA of bid/ask size imbalance ([-1, +1])
# imbalance_persistence — fraction of last N bars sustaining sign
# withdrawal_velocity — rate of depth decay (negative = book thinning)
9.2 OB Macro (Market-Wide)
ob_macro = ob_engine.get_macro()
# Fields:
# cascade_count — number of assets in liquidation cascade
# regime_signal — (-1=calm/trending, 0=neutral, +1=withdrawal stress)
9.3 OB Placement Quality
ob_placement = ob_engine.get_placement(asset, timestamp)
# Fields:
# depth_quality — book depth score ([0, 2+]; >1 = deep book)
# fill_probability — probability of maker fill at entry price
9.4 OB Sub-Systems Summary
| Sub | Location | Effect |
|---|---|---|
| OB-1 | AlphaAssetSelector | ARS adjusted ±5%/10% by depth quality |
| OB-2 | AlphaSignalGenerator | Confidence boost/dampen by imbalance; hard skip if persistent contradiction |
| OB-3 | NDAlphaEngine._try_entry | Market-wide imbalance multiplier on final leverage (±20%/15%) |
| OB-4 | AdaptiveCircuitBreaker | Macro withdrawal stress modulates ACBv6 dynamic beta (+25% max) |
| OB-5 | AlphaExitManager | Dynamic TP/SL/max_hold based on cascade/withdrawal/calm regime |
10. ACB v6 — ADAPTIVE CIRCUIT BREAKER
10.1 Architecture (3-Scale Confirmation)
Source: adaptive_circuit_breaker.py, AdaptiveCircuitBreaker
Scale 1 (Daily): External macro factors → log_0.5 base_boost
Scale 2 (Per-bar): vel_div signal strength → meta-boost multiplier
Scale 3 (Macro): w750 eigenvalue velocity → dynamic beta switch
10.2 Scale 1 — External Factor Signals
Loaded from NG3 scan_*__Indicators.npz files (median of first 10 scans per date):
| Factor | Source | Bearish Threshold | Signal Weight |
|---|---|---|---|
funding_btc |
Funding rate | < -0.0001 (very) / < 0.0 (mild) | 1.0 / 0.5 |
dvol_btc |
BTC implied vol | > 80 (extreme) / > 55 (elevated) | 1.0 / 0.5 |
fng |
Fear & Greed Index | < 25 (extreme fear) / < 40 (fear) | 1.0 / 0.5 (requires confirmation) |
taker |
Taker buy ratio | < 0.80 (selling) / < 0.90 (mild) | 1.0 / 0.5 |
Signal counting:
signals = sum(individual_signal_weights) # float, e.g. 2.5
Base boost formula:
if signals >= 1.0:
base_boost = 1.0 + 0.5 * log1p(signals)
# signals=1 → 1.347x | signals=2 → 1.549x | signals=3 → 1.693x
else:
base_boost = 1.0
10.3 Scale 3 — Dynamic Beta
# Preloaded: w750_threshold = np.percentile(all_w750_vels, 60)
if w750_vel >= w750_threshold:
beta = BETA_HIGH = 0.8 # aggressive meta-boost during macro acceleration
else:
beta = BETA_LOW = 0.2 # conservative during calm macro
10.4 Scale 2 — Per-Bar Meta-Boost
# Computed every bar inside _update_regime_size_mult(vel_div):
strength_cubic = clamp((threshold - vel_div) / (threshold - extreme), 0, 1) ** 3
# leverage_convexity = 3 → cubic
if day_beta > 0:
regime_size_mult = base_boost * (1.0 + beta * strength_cubic) * mc_scale
else:
regime_size_mult = base_boost * mc_scale
10.5 Sub-Day ACB Update (HZ Listener)
The acb_processor_service.py re-runs ACB computation mid-day when new NG3 scan data arrives and writes {boost, beta} to DOLPHIN_FEATURES IMap.
_on_acb_event() in DolphinActor stores the payload in self._pending_acb (GIL-safe dict write). Applied at start of next on_bar() iteration:
# In on_bar() — BEFORE processing:
if _pending_acb is not None and engine is not None:
engine.update_acb_boost(pending_acb['boost'], pending_acb['beta'])
_pending_acb = None
11. SURVIVAL STACK — POSTURE CONTROL
11.1 Overview
Source: survival_stack.py, SurvivalStack
Computes a continuous Risk Multiplier Rm ∈ [0, 1] from 5 sensor categories. Maps to discrete posture {APEX, STALKER, TURTLE, HIBERNATE}.
11.2 Five Sensor Categories
Cat1 — Binary Invariant (kill switch):
if hz_nodes < 1 or heartbeat_age_s > 30:
return 0.0 # Total system failure → HIBERNATE immediately
return 1.0
Cat2 — Structural (MC-Forewarner + data staleness):
base = {OK: 1.0, ORANGE: 0.5, RED: 0.1}[mc_status]
decay = exp(-max(0, staleness_hours - 6) / 3)
f_structural = base * decay # Exponential decay after 6h stale
Cat3 — Microstructure (OB depth/fill quality):
if ob_stale:
return 0.5
score = min(depth_quality, fill_prob)
return clamp(0.3 + 0.7 * score, 0.3, 1.0)
Cat4 — Environmental (DVOL spike impulse):
if dvol_spike and t_since_spike_min == 0:
return 0.3 # Instant degradation at spike
return 0.3 + 0.7 * (1 - exp(-t_since_spike_min / 60)) # 60-min recovery tau
Cat5 — Capital (sigmoid drawdown constraint):
# Rm5 ≈ 1.0 at DD<5%, ≈ 0.5 at DD=12%, ≈ 0.1 at DD=20%
return 1 / (1 + exp(30 * (drawdown - 0.12)))
11.3 Hierarchical Combination
f_environment = min(f_structural, f_ext) # worst of Cat2/Cat4
f_execution = f_micro # Cat3
r_target = Cat1 * Cat5 * f_environment * f_execution
# Correlated sensor collapse penalty:
degraded = count([f_structural < 0.8, f_micro < 0.8, f_ext < 0.8])
if degraded >= 2:
r_target *= 0.5
11.4 Bounded Recovery Dynamics
# Fast attack (instant degradation), slow recovery (5%/minute max):
if r_target < last_r_total:
r_final = r_target # immediate drop
else:
alpha = min(1.0, 0.02 * dt_min)
step = min(alpha * (r_target - last_r_total), 0.05 * dt_min)
r_final = last_r_total + step
11.5 Posture Mapping
NOTE: Thresholds are deliberately TIGHTER than mathematical spec (safety buffer).
if Rm >= 0.90: APEX # Full trading, no constraints
if Rm >= 0.75: STALKER # Max leverage capped at 2.0x
if Rm >= 0.50: TURTLE # regime_dd_halt = True (no new entries)
else: HIBERNATE # Force-close open positions, no new entries
11.6 Hysteresis
# Down: requires hysteresis_down=2 consecutive bars at lower level
# Up: requires hysteresis_up=5 consecutive bars at higher level
# Prevents flip-flopping around thresholds
11.7 Posture → Engine Effect
| Posture | Engine Effect |
|---|---|
| APEX | No constraint (max leverage = abs_max=6.0 × regime_size_mult) |
| STALKER | clamped_max_leverage = min(..., 2.0) in _try_entry |
| TURTLE | regime_dd_halt = True → process_bar skips entry block |
| HIBERNATE | _manage_position forces EXIT("HIBERNATE_HALT"), regime_dd_halt = True |
12. MC-FOREWARNER ENVELOPE GATE
Source: Called via engine.set_mc_forewarner(forewarner, mc_base_cfg)
Runs daily at start of process_day():
mc_cfg = {**mc_base_cfg, 'max_leverage': base_max_leverage * day_base_boost}
mc_report = forewarner.assess_config_dict(mc_cfg)
mc_red = mc_report.catastrophic_probability > 0.25 or mc_report.envelope_score < -1.0
mc_orange = (not mc_red) and (envelope_score < 0 or catastrophic_probability > 0.10)
day_mc_status = 'RED' | 'ORANGE' | 'OK'
day_mc_scale = 0.5 if ORANGE else 1.0
if mc_red or posture in ['TURTLE', 'HIBERNATE']:
regime_dd_halt = True # No new entries for the entire day
13. NDAlphaEngine — FULL BAR LOOP
13.1 process_day() Flow
def process_day(date_str, df, asset_columns, vol_regime_ok, direction, posture):
# --- Per-day initialization ---
regime_direction = direction or -1 # -1=SHORT champion
regime_dd_halt = False
day_posture = posture
# --- ACB: Scale 1 + Scale 3 (once per day) ---
if acb:
info = acb.get_dynamic_boost_for_date(date_str, ob_engine)
day_base_boost = info['boost'] # log_0.5 curve
day_beta = info['beta'] # dynamic beta (w750 percentile switch)
# --- MC-Forewarner gate (once per day) ---
if forewarner:
mc_report = forewarner.assess_config_dict(...)
if mc_red or posture in [TURTLE, HIBERNATE]:
regime_dd_halt = True # kill new entries for day
# === BAR LOOP ===
for ri, row in df.iterrows():
vd = row['vel_div'] # primary signal
v50_vel = row['v50_lambda_max_velocity']
v750_vel = row['v750_lambda_max_velocity']
prices = {col: row[col] for col in asset_columns if row[col] > 0}
# Append to price_histories (capped at 200; sliding window)
for ac, p in prices.items():
price_histories[ac].append(p)
if len(price_histories[ac]) > 500:
price_histories[ac] = price_histories[ac][-200:]
vol_ok = vol_regime_ok[ri] if vol_regime_ok else (bid >= 100)
# ACB Scale 2: regime_size_mult updated every bar
_update_regime_size_mult(vd)
process_bar(global_bar_idx, vd, prices, vol_ok, price_histories, v50_vel, v750_vel)
global_bar_idx += 1
return {date, pnl, capital, boost, beta, mc_status, trades}
13.2 process_bar() Flow
def process_bar(bar_idx, vel_div, prices, vol_regime_ok, price_histories, v50_vel, v750_vel):
bar_count += 1
vel_div_history.append(vel_div) # trimmed to 200
# === EXIT MANAGEMENT (always first) ===
if position is not None:
exit_info = _manage_position(bar_idx, prices, vel_div, v50_vel, v750_vel)
# → AlphaExitManager.evaluate() → if EXIT: _execute_exit()
# === ENTRY (only when no position) ===
if position is None AND bar_idx > last_exit_bar AND NOT regime_dd_halt:
if bar_count >= lookback (100) AND vol_regime_ok:
entry_info = _try_entry(bar_idx, vel_div, prices, price_histories, v50_vel, v750_vel)
13.3 _try_entry() Flow
def _try_entry(bar_idx, vel_div, prices, price_histories, v50_vel, v750_vel):
if capital <= 0: return None
# 1. IRP Asset Selection (Layer 2)
if use_asset_selection:
market_data = {a: history[-50:] for a, history in price_histories if len >= 50}
rankings = asset_selector.rank_assets(market_data, regime_direction)
trade_asset = first_asset_passing_all_gates(rankings)
if trade_asset is None: return None # strict: no fallback
else:
trade_asset = "BTCUSDT" # fallback when IRP disabled
# 2. Signal Generation + DC (Layer 6)
signal = signal_gen.generate(vel_div, vel_div_history,
price_histories[trade_asset],
regime_direction, trade_asset)
if not signal.is_valid: return None # vel_div or DC killed it
# 3. Position Sizing (Layers 7-8)
size = bet_sizer.calculate_size(capital, vel_div, signal.vel_div_trend, regime_direction)
# 4. OB Sub-3: Cross-asset market multiplier
market_ob_mult = ob_engine.get_market_multiplier(...) # ±20%
# 5. ACB leverage ceiling enforcement
clamped_max = min(base_max_leverage * regime_size_mult * market_ob_mult, abs_max_leverage=6.0)
if posture == STALKER: clamped_max = min(clamped_max, 2.0)
final_leverage = clamp(size.leverage * regime_size_mult * market_ob_mult, min_lev, clamped_max)
# 6. Notional and entry
notional = capital * size.fraction * final_leverage
entry_price = prices[trade_asset]
# 7. Create position
position = NDPosition(trade_asset, regime_direction, entry_price,
notional, final_leverage, ...)
exit_manager.setup_position(trade_id, entry_price, direction, bar_idx, v50_vel, v750_vel)
14. DOLPHIN ACTOR — NAUTILUS INTEGRATION
Source: dolphin_actor.py, DolphinActor(Strategy)
14.1 Lifecycle
__init__: set defaults, _pending_acb = None
on_start: connect HZ, read posture, setup ACB listener, instantiate NDAlphaEngine
on_bar: (fires every 5s)
1. Apply _pending_acb if set (thread-safe drain from HZ listener)
2. Parse bar date
3. On date change: re-read posture from HZ
4. If date already processed: RETURN (strict once-per-day guard)
5. If HIBERNATE: process_day(empty df), mark done, write HZ, RETURN
6. Load parquet: _load_parquet_data(date_str)
7. process_day(date_str, df, asset_cols, vol_regime_ok, direction, posture)
8. _processed_dates.add(date_str)
9. _write_result_to_hz(date_str, result)
on_stop: hz_client.shutdown()
14.2 Thread Safety: ACB Pending-Flag Pattern
CRITICAL: HZ entry listeners run on HZ client pool threads, NOT the Nautilus event loop.
# HZ thread (ONLY writes _pending_acb — no engine access):
def _on_acb_event(event):
if event.value:
self._pending_acb = json.loads(event.value) # GIL-safe dict assignment
# Nautilus event loop thread (only reader + engine caller):
def on_bar(bar):
if _pending_acb is not None and engine is not None:
engine.update_acb_boost(_pending_acb['boost'], _pending_acb['beta'])
_pending_acb = None
# ... rest of on_bar
Max ACB latency: 1 bar interval (5 seconds). Acceptable given ACB changes on minute-to-hour timescale.
14.3 Data Loading
def _load_parquet_data(date_str):
path = HCM_DIR / "vbt_cache_klines" / f"{date_str}.parquet"
df = pd.read_parquet(path)
meta_cols = {vel_div, scan_number, v50_..., v150_..., v300_..., v750_..., instability_50, instability_150}
asset_columns = [c for c in df.columns if c not in meta_cols]
vol_regime_ok = None # deferred to process_day (bar >= 100 gate)
return df, asset_columns, None
14.4 Posture Reading
Primary: HZ CP Subsystem AtomicReference('DOLPHIN_SAFETY') — linearizable.
Fallback: HZ IMap('DOLPHIN_SAFETY').get('latest') — eventually consistent.
14.5 Result Writing
def _write_result_to_hz(date_str, result):
imap_pnl = hz_client.get_map('DOLPHIN_PNL_BLUE').blocking()
imap_pnl.put(date_str, json.dumps(result))
# result: {date, pnl, capital, boost, beta, mc_status, trades}
15. HAZELCAST FEATURE STORE — SHARDING
Source: hz_sharded_feature_store.py, ShardedFeatureStore
15.1 Shard Routing
SHARD_COUNT = 10
shard = sum(ord(c) for c in symbol) % SHARD_COUNT
map_name = f"DOLPHIN_FEATURES_SHARD_{shard:02d}" # ..._00 through ..._09
Hash is stable (sum-of-ord, not Python's hash()), deterministic across process restarts.
15.2 API
store = ShardedFeatureStore(hz_client)
store.put('BTCUSDT', 'vel_div', -0.03) # routes to shard based on symbol hash
val = store.get('BTCUSDT', 'vel_div') # same shard
store.delete('BTCUSDT', 'vel_div')
# Key format inside IMap: "vel_div_BTCUSDT"
# Map handles cached after first access
15.3 Near Cache Configuration
near_cache_config = ShardedFeatureStore.near_cache_config()
# Returns dict for all 10 shards:
# { 'DOLPHIN_FEATURES_SHARD_XX': {
# invalidate_on_change: True,
# time_to_live_seconds: 300,
# max_idle_seconds: 60,
# eviction_policy: LRU,
# max_size: 5000
# }
# }
hz = hazelcast.HazelcastClient(near_caches=near_cache_config, ...)
15.4 Concurrency Notes
| Risk | Severity | Mitigation |
|---|---|---|
_maps dict not thread-safe |
LOW | Only accessed from actor event loop (single thread) |
| Multiple ACBProcessorService writers | MEDIUM | CP FencedLock (only if all writers use it) |
| HZ Near Cache stale | LOW | TTL=300s, invalidate_on_change=True |
16. PRODUCTION DAEMON TOPOLOGY
16.1 ACB Processor Service
Purpose: Computes ACBv6 daily boost + dynamic beta and writes to HZ DOLPHIN_FEATURES IMap.
Schedule: Runs once at market open (08:00 UTC), then re-runs whenever a new NG3 scan batch arrives. Uses CP FencedLock to prevent simultaneous writes.
Output: DOLPHIN_FEATURES.put('acb_boost', json.dumps({boost, beta}))
16.2 OB Stream Service
Purpose: Fetches live Binance order book snapshots at 500ms cadence. Computes per-asset imbalance, fills, and macro signals. Writes to HZ.
Cadence: 500ms per cycle (2 updates/second).
16.3 System Watchdog Service
Purpose: Reads HZ heartbeat, OB quality, MC status, drawdown → runs SurvivalStack.compute_rm() → writes posture to DOLPHIN_SAFETY AtomicReference via SurvivalStack.write_to_hz().
Cadence: ~60s per cycle.
16.4 MC-Forewarner Flow
Purpose: Prefect-orchestrated. Assesses current config envelope daily using correlation matrix statistics. Outcome: OK / ORANGE / RED status in HZ. ORANGE → day_mc_scale=0.5. RED → regime_dd_halt=True.
16.5 paper_trade_flow.py
Purpose: Main production entry point. Instantiates DolphinActor and wires into Nautilus BacktestEngine (paper mode). Subscribes to CCXT 5s bars. Runs indefinitely.
Direction: direction = -1 (SHORT, blue posture). GREEN (LONG) is separate configuration.
16.6 Daemon Start Sequence
1. Start Hazelcast (port 5701)
2. acb_processor_service.py ← writes initial ACB boost before market open
3. ob_stream_service.py ← start OB snapshots
4. system_watchdog_service.py ← begin posture computation
5. mc_forewarner_flow.py ← Prefect flow (port 4200 UI)
6. paper_trade_flow.py ← Start actor LAST (reads all the above from HZ)
16.7 Monitoring Endpoints
| Service | URL / Path |
|---|---|
| Hazelcast MC Console | http://localhost:8080 |
| Prefect UI | http://localhost:4200 |
| Daily PnL | HZ IMap: DOLPHIN_PNL_BLUE (key = date string) |
| Posture | HZ AtomicRef: DOLPHIN_SAFETY |
| ACB State | HZ IMap: DOLPHIN_FEATURES (key = 'acb_boost') |
17. CI TEST SUITE
Location: ci/ directory. Runner: pytest ci/ -v
Passing gate: 14/14 tests in 7-10s.
| File | Tests | What It Covers |
|---|---|---|
test_13_nautilus_integration.py |
6 | Actor import, instantiation, on_bar, HIBERNATE posture, once-per-day guard, ACB thread safety |
test_14_long_system.py |
3 | Multi-day run, capital persistence, trade count |
test_15_acb_reactive.py |
1 | ACB boost update applied correctly mid-day |
test_16_scaling.py |
4 | Memory footprint <4GB (50 assets), shard routing (400 symbols), 400-asset no-crash, 400-asset with IRP |
17.1 Key Test: test_acb_listener_does_not_call_engine_directly
# Patches engine.update_acb_boost with a mock, fires _on_acb_event
# Asserts: _pending_acb is set, engine method was NOT called
with patch.object(actor.engine, 'update_acb_boost') as mock_update:
actor._on_acb_event(event)
assert actor._pending_acb['boost'] == 1.35
mock_update.assert_not_called()
17.2 Key Test: test_process_day_called_once_per_day
# Fires 3 bars on same date → assert process_day.call_count == 1
# Fires 1 bar on next date → assert process_day.call_count == 2
18. PARAMETER REFERENCE
18.1 Champion Parameters (Frozen)
| Parameter | Value | Layer | Notes |
|---|---|---|---|
vel_div_threshold |
-0.02 | Signal gate | PRIMARY entry threshold |
vel_div_extreme |
-0.05 | Signal/sizing | Extreme regime: full size |
min_leverage |
0.5 | Sizing | Floor leverage |
max_leverage |
5.0 | Sizing | Base ceiling (before ACB) |
abs_max_leverage |
6.0 | Sizing | Hard geometric ceiling |
leverage_convexity |
3.0 | Sizing | Cubic convex curve |
fraction |
0.20 | Sizing | Max capital fraction per trade |
fixed_tp_pct |
0.0095 | Exit | 95 bps take-profit |
stop_pct |
1.0 | Exit | Effectively disabled |
max_hold_bars |
120 | Exit | 600 seconds |
dc_lookback_bars |
7 | DC | 35 seconds price momentum |
dc_min_magnitude_bps |
0.75 | DC | Minimum BTC momentum |
dc_skip_contradicts |
True | DC | Hard skip on contradiction |
min_irp_alignment |
0.45 | IRP | Alignment gate |
sp_maker_entry_rate |
0.62 | Fees | 62% maker fill at entry |
sp_maker_exit_rate |
0.50 | Fees | 50% maker fill at exit |
ob_edge_bps |
5.0 | OB | Legacy MC OB edge |
ob_confirm_rate |
0.40 | OB | Legacy MC confirmation rate |
lookback |
100 | Warmup | Bars before first entry allowed |
seed |
42 | RNG | Deterministic numpy RandomState |
18.2 ACBv6 Parameters (Frozen — Validated)
| Parameter | Value | Notes |
|---|---|---|
BETA_HIGH |
0.8 | w750 above p60 threshold |
BETA_LOW |
0.2 | w750 below p60 threshold |
W750_THRESHOLD_PCT |
60 | Percentile switch point |
FUNDING_VERY_BEARISH |
-0.0001 | 1.0 signal |
DVOL_EXTREME |
80 | 1.0 signal |
FNG_EXTREME_FEAR |
25 | 1.0 signal (needs confirmation) |
TAKER_SELLING |
0.8 | 1.0 signal |
18.3 Survival Stack Thresholds (Deliberately Tight)
| Posture | Rm Threshold | vs. Math Spec |
|---|---|---|
| APEX | ≥ 0.90 | Tighter — spec was 0.85 |
| STALKER | ≥ 0.75 | Tighter — spec was 0.70 |
| TURTLE | ≥ 0.50 | Tighter — spec was 0.45 |
| HIBERNATE | < 0.50 | — |
Do NOT loosen these without quantitative justification.
19. KNOWN RESEARCH TODOs
| ID | Description | Priority |
|---|---|---|
| TODO-1 | Calibrate vd_enabled adverse-turn exits (currently disabled). Requires analysis of trade vel_div distribution at entry vs. subsequent bars. True invalidation threshold likely ~+0.02 sustained for N=3 bars. |
MEDIUM |
| TODO-2 | Validate SUBDAY_ACB force-exit threshold (old_boost >= 1.25 and boost < 1.10). Currently ARBITRARY — agent-chosen, not backtest-derived. |
MEDIUM |
| TODO-3 | MIG8: Binance live adapter (real order execution). OUT OF SCOPE until after 30-day paper trading validation. | LOW |
| TODO-4 | 48-hour chaos test with all daemons running simultaneously. Watch for: KeyError, stale-read anomalies, concurrent HZ writer collisions. | HIGH (before live capital) |
| TODO-5 | Memory profiler with IRP enabled at 400 assets (current 71 MB measurement was without IRP). Projected ~600 MB — verify. | LOW |
| TODO-6 | TF-spread recovery exits (tf_enabled=False). Requires sweep of tf_exhaust_ratio and tf_flip_ratio vs. champion backtest. |
LOW |
| TODO-7 | GREEN (LONG) posture paper validation. LONG thresholds (long_threshold=0.01, long_extreme=0.04) not yet production-validated. | MEDIUM |
End of DOLPHIN-NAUTILUS System Bible v1.0 — 2026-03-07
Champion: SHORT only (APEX posture, blue configuration)
Gate: 14/14 CI tests green. Paper trading ready.
Do NOT deploy real capital until 30-day paper run is clean.