01c19662cb
Includes core prod + GREEN/BLUE subsystems: - prod/ (BLUE harness, configs, scripts, docs) - nautilus_dolphin/ (GREEN Nautilus-native impl + dvae/ preserved) - adaptive_exit/ (AEM engine + models/bucket_assignments.pkl) - Observability/ (EsoF advisor, TUI, dashboards) - external_factors/ (EsoF producer) - mc_forewarning_qlabs_fork/ (MC regime/envelope) Excludes runtime caches, logs, backups, and reproducible artifacts per .gitignore.
746 lines
31 KiB
Markdown
Executable File
746 lines
31 KiB
Markdown
Executable File
# Nautilus-DOLPHIN / Alpha Engine Core — Implementation Specification
|
||
|
||
**Version:** 1.0
|
||
**Date:** 2026-03-22
|
||
**Status:** Production-ready (paper trading); live deployment pending exchange integration
|
||
**Environment:** siloqy-env (`/home/dolphin/siloqy_env/bin/activate`)
|
||
**Stack:** nautilus_trader 1.219.0 · prefect 3.6.22 · hazelcast-python-client 5.6.0 · numba 0.61.2
|
||
|
||
---
|
||
|
||
## 1. System Overview
|
||
|
||
Nautilus-DOLPHIN is a production algorithmic trading system built on the NautilusTrader Rust-core HFT framework. It wraps a 7-layer alpha engine ("NDAlphaEngine") inside a NautilusTrader Strategy primitive ("DolphinActor"), supervised by Prefect for resilience and Hazelcast for distributed system memory.
|
||
|
||
### 1.1 Performance Specification (Champion — FROZEN)
|
||
|
||
| Metric | Champion Value |
|
||
|---|---|
|
||
| ROI (backtest period) | +54.67% |
|
||
| Profit Factor | 1.141 |
|
||
| Sharpe Ratio | 2.84 |
|
||
| Max Drawdown | 15.80% |
|
||
| Win Rate | 49.5% |
|
||
| Direction | SHORT only (blue deployment) |
|
||
| Bar resolution | 5-second |
|
||
| Markets | Binance Futures perpetuals (~48 assets) |
|
||
|
||
These numbers are **invariants**. Any code change that causes a statistically significant deviation must be rejected.
|
||
|
||
### 1.2 Architecture Summary
|
||
|
||
```
|
||
┌────────────────────────────────────────────────────────────────┐
|
||
│ Prefect Supervision Layer │
|
||
│ paper_trade_flow.py (00:05 UTC) nautilus_prefect_flow.py │
|
||
│ dolphin_nautilus_flow (00:10 UTC) │
|
||
└──────────────────────────────┬─────────────────────────────────┘
|
||
│
|
||
┌──────────────────────────────▼─────────────────────────────────┐
|
||
│ NautilusTrader Execution Kernel │
|
||
│ BacktestEngine (paper) / TradingNode (live) │
|
||
│ │
|
||
│ ┌─────────────────────────────────────────┐ │
|
||
│ │ DolphinActor (Strategy) │ │
|
||
│ │ on_start() → connect HZ, ACB listener │ │
|
||
│ │ on_bar() → step_bar() per 5s tick │ │
|
||
│ │ on_stop() → cleanup, HZ shutdown │ │
|
||
│ └──────────────────────┬──────────────────┘ │
|
||
│ │ │
|
||
│ ┌───────────────────────▼──────────────────────────────────┐ │
|
||
│ │ NDAlphaEngine │ │
|
||
│ │ 7-layer alpha stack (see §4) │ │
|
||
│ │ begin_day() / step_bar() / end_day() │ │
|
||
│ └───────────────────────────────────────────────────────────┘ │
|
||
└──────────────────────────────┬─────────────────────────────────┘
|
||
│
|
||
┌──────────────────────────────▼─────────────────────────────────┐
|
||
│ Hazelcast "System Memory" │
|
||
│ DOLPHIN_SAFETY → posture, Rm (survival stack) │
|
||
│ DOLPHIN_FEATURES → ACB boost, beta, eigen scan │
|
||
│ DOLPHIN_PNL_BLUE → daily trade results │
|
||
│ DOLPHIN_STATE_BLUE→ capital state (continuity) │
|
||
│ DOLPHIN_HEARTBEAT → liveness probes │
|
||
│ DOLPHIN_FEATURES_SHARD_00..09 → 400-asset feature shards │
|
||
└────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## 2. File Map
|
||
|
||
```
|
||
/mnt/dolphinng5_predict/
|
||
│
|
||
├── prod/
|
||
│ ├── paper_trade_flow.py # Primary daily Prefect flow (NDAlphaEngine direct)
|
||
│ ├── nautilus_prefect_flow.py # Nautilus BacktestEngine Prefect flow (NEW)
|
||
│ ├── run_nautilus.py # Standalone Nautilus CLI runner
|
||
│ ├── configs/
|
||
│ │ ├── blue.yml # Champion SHORT config (FROZEN)
|
||
│ │ └── green.yml # Bidirectional config (pending LONG validation)
|
||
│ └── OBF_SUBSYSTEM.md # OBF architecture reference
|
||
│
|
||
├── nautilus_dolphin/
|
||
│ └── nautilus_dolphin/
|
||
│ └── nautilus/
|
||
│ ├── dolphin_actor.py # DolphinActor(Strategy) — Nautilus wrapper
|
||
│ ├── esf_alpha_orchestrator.py # NDAlphaEngine — 7-layer core
|
||
│ ├── proxy_boost_engine.py # ProxyBoostEngine wrapper (ACBv6 pre-compute)
|
||
│ ├── adaptive_circuit_breaker.py # ACBv6 — 3-scale regime sizing
|
||
│ ├── strategy.py # DolphinExecutionStrategy (signal-level)
|
||
│ ├── strategy_config.py # DolphinStrategyConfig (StrategyConfig subclass)
|
||
│ ├── launcher.py # NautilusDolphinLauncher (TradingNode)
|
||
│ ├── ob_features.py # OBFeatureEngine — order book intelligence
|
||
│ ├── hz_ob_provider.py # HZOBProvider — HZ-backed OB data source
|
||
│ └── circuit_breaker.py # CircuitBreakerManager
|
||
│ └── tests/
|
||
│ ├── test_0_nautilus_bootstrap.py # 11 foundation tests
|
||
│ ├── test_dolphin_actor.py # 35 DolphinActor lifecycle tests (NEW)
|
||
│ ├── test_strategy.py # DolphinExecutionStrategy filter tests
|
||
│ ├── test_adaptive_circuit_breaker.py
|
||
│ ├── test_circuit_breaker.py
|
||
│ ├── test_volatility_detector.py
|
||
│ └── [12 other test files]
|
||
│
|
||
└── vbt_cache_klines/ # 5s OHLCV parquet files — daily replay source
|
||
└── YYYY-MM-DD.parquet # cols: vel_div, v50/v150/v300/v750, instability_*, 48 assets
|
||
```
|
||
|
||
---
|
||
|
||
## 3. Champion Parameters (FROZEN)
|
||
|
||
These parameters are derived from the champion backtest and **must not be altered** without a full re-validation run showing performance preservation.
|
||
|
||
| Parameter | Value | Description |
|
||
|---|---|---|
|
||
| `vel_div_threshold` | -0.02 | Primary signal gate: vd must be ≤ this to open a position |
|
||
| `vel_div_extreme` | -0.05 | Extreme signal bucket threshold (max leverage tier) |
|
||
| `fixed_tp_pct` | 0.0095 | Take-profit at 95 bps from entry (TP sweep 2026-03-06) |
|
||
| `max_hold_bars` | 120 | Maximum holding period in 5s bars (= 10 minutes) |
|
||
| `fraction` | 0.20 | Base position size fraction of capital |
|
||
| `min_leverage` | 0.5 | Floor leverage (applied by AlphaBetSizer) |
|
||
| `max_leverage` | 5.0 | Soft leverage ceiling |
|
||
| `abs_max_leverage` | 6.0 | Hard leverage ceiling (Rm-scaled by Survival Stack) |
|
||
| `leverage_convexity` | 3.0 | Cubic exponent for convex leverage scaling |
|
||
| `dc_lookback_bars` | 7 | Direction confirmation lookback window |
|
||
| `dc_min_magnitude_bps` | 0.75 | Minimum velocity magnitude for DC trigger |
|
||
| `min_irp_alignment` | 0.45 | IRP asset selection threshold |
|
||
| `sp_maker_entry_rate` | 0.62 | SmartPlacer: 62% maker fill rate at entry |
|
||
| `sp_maker_exit_rate` | 0.50 | SmartPlacer: 50% maker fill rate at exit |
|
||
| `seed` | 42 | NumPy / numba RNG seed (reproducibility invariant) |
|
||
|
||
**Verification:** `nautilus_prefect_flow._CHAMPION_HASH` is computed at import time from these values. Any config drift triggers `ValueError` and aborts the flow.
|
||
|
||
---
|
||
|
||
## 4. Alpha Engine — 7-Layer Stack
|
||
|
||
### Layer 1: Velocity Divergence Signal Gate (`AlphaSignalGenerator`)
|
||
|
||
**Input:** `vel_div = v50_lambda_max_velocity - v150_lambda_max_velocity`
|
||
|
||
The primary alpha signal. `v50` is the 50-window eigenvalue velocity; `v150` the 150-window. Negative divergence signals short momentum.
|
||
|
||
```
|
||
Buckets:
|
||
0 (extreme): vel_div ≤ -0.075 → max leverage
|
||
1 (strong): vel_div ≤ -0.050 → high leverage
|
||
2 (moderate): vel_div ≤ -0.035 → normal leverage
|
||
3 (weak): vel_div ≤ -0.020 → min leverage
|
||
```
|
||
|
||
Only fires if `vel_div ≤ vel_div_threshold (-0.02)`. Includes direction confirmation (`dc_lookback_bars=7`, `dc_min_magnitude_bps=0.75`).
|
||
|
||
### Layer 2: Volatility Regime Filter (`VolatilityRegimeDetector`)
|
||
|
||
Rolling 50-bar standard deviation of BTC returns. Positions are only opened when `dvol > vol_p60` (60th percentile threshold = 0.000099 from 55-day champion calibration). Prevents trading in dead-market microstructure.
|
||
|
||
### Layer 3: SmartPlacer Fee Model (`AlphaSignalGenerator`)
|
||
|
||
Models maker vs taker execution costs:
|
||
- `sp_maker_entry_rate=0.62`: 62% of entries assumed as maker (−0.02% fee)
|
||
- `sp_maker_exit_rate=0.50`: 50% of exits as maker
|
||
- Remaining fills incur taker fee (+0.04%)
|
||
- Net fee per round trip ≈ 0.02–0.04% depending on fill mix
|
||
|
||
Fee is charged per trade in `NDAlphaEngine.process_bar()`. No real order routing in paper mode — fee is applied analytically.
|
||
|
||
### Layer 4: OB Intelligence — 5 Sub-systems (`OBFeatureEngine` / `HZOBProvider`)
|
||
|
||
Reads from `DOLPHIN_FEATURES_SHARD_{idx}` or `ob_cache/latest_ob_features.json`:
|
||
|
||
| Sub-system | Key features | Effect |
|
||
|---|---|---|
|
||
| 1. Placement | `fill_probability`, `depth_quality`, `spread_proxy_bps` | Adjusts maker entry rate; gates entry if `fill_probability < 0.6` |
|
||
| 2. Signal | `depth_asymmetry`, `imbalance_persistence`, `withdrawal_velocity` | OB-direction confirmation layer |
|
||
| 3. Cross-asset | `agreement_pct`, `cascade_count`, `regime_signal` | Asset selection weighting in IRP |
|
||
| 4. Macro | `macro_imbalance`, `macro_spread_bps` | Long-horizon baseline normalization |
|
||
| 5. Raw depth | `bid/ask_notional_1-5pct`, `bid/ask_depth_1-5pct` | Notional depth vectors for all 5 levels |
|
||
|
||
OB edge gate: `ob_edge_bps=5.0`, `ob_confirm_rate=0.40`. Entry only if OB confirms directional signal.
|
||
|
||
### Layer 5: IRP Asset Selection (`AlphaAssetSelector`)
|
||
|
||
Inter-asset relative performance (IRP) selects which assets to trade each bar. Only assets where imbalance sign aligns with the directional view, and where `irp_alignment ≥ min_irp_alignment (0.45)`, are traded.
|
||
|
||
### Layer 6: Dynamic Cubic-Convex Leverage (`AlphaBetSizer`)
|
||
|
||
```
|
||
leverage = min_leverage + (max_leverage - min_leverage) × (signal_strength)^leverage_convexity
|
||
|
||
signal_strength = (vel_div_threshold - vel_div) / (vel_div_threshold - vel_div_extreme)
|
||
clamped to [0, 1]
|
||
```
|
||
|
||
Then scaled by `regime_size_mult` from ACBv6:
|
||
```
|
||
regime_size_mult = base_boost × (1 + beta × strength_cubic) × mc_scale
|
||
```
|
||
|
||
Final leverage clamped to `[min_leverage, abs_max_leverage × Rm]`.
|
||
|
||
### Layer 7: Exit Management (`AlphaExitManager`)
|
||
|
||
Two primary exits (no stop loss in champion):
|
||
1. **Fixed TP:** Exit when `price_change_pct ≥ fixed_tp_pct (0.0095)` = 95 bps
|
||
2. **Max hold:** Force exit at `max_hold_bars (120)` × 5s = 10 minutes
|
||
|
||
---
|
||
|
||
## 5. DolphinActor — Nautilus Strategy Wrapper
|
||
|
||
**File:** `nautilus_dolphin/nautilus/dolphin_actor.py`
|
||
**Base class:** `nautilus_trader.trading.strategy.Strategy`
|
||
**Lines:** 338
|
||
|
||
### 5.1 Initialization
|
||
|
||
```python
|
||
class DolphinActor(Strategy):
|
||
def __init__(self, config: dict):
|
||
super().__init__() # Nautilus Actor Cython init
|
||
self.dolphin_config = config # full YAML config dict
|
||
self.engine = None # NDAlphaEngine (created in on_start)
|
||
self.hz_client = None # HazelcastClient
|
||
self.current_date = None # tracks date boundary
|
||
self.posture = 'APEX' # Survival Stack posture
|
||
self._processed_dates = set()
|
||
self._pending_acb: dict | None = None # pending ACB from HZ listener
|
||
self._acb_lock = threading.Lock() # guards _pending_acb
|
||
self._stale_state_events = 0
|
||
self.last_scan_number = -1
|
||
self._day_data = None # (df, asset_columns) for replay mode
|
||
self._bar_idx_today = 0
|
||
```
|
||
|
||
### 5.2 on_start() Lifecycle
|
||
|
||
```
|
||
on_start():
|
||
1. _connect_hz() → hazelcast.HazelcastClient(cluster_name="dolphin", ...)
|
||
2. _read_posture() → reads DOLPHIN_SAFETY (CP atomic ref or map fallback)
|
||
3. _setup_acb_listener() → add_entry_listener on DOLPHIN_FEATURES["acb_boost"]
|
||
4. create_boost_engine(mode=boost_mode, **engine_kwargs) → NDAlphaEngine
|
||
5. MC-Forewarner injection (gold-performance stack — enabled by default):
|
||
mc_models_dir = config.get('mc_models_dir', _MC_MODELS_DIR_DEFAULT)
|
||
DolphinForewarner(models_dir=mc_models_dir) → engine.set_mc_forewarner(fw, _MC_BASE_CFG)
|
||
Graceful: logs warning + continues if models dir missing or import fails.
|
||
Disable: set mc_models_dir=None or mc_models_dir='' in config.
|
||
```
|
||
|
||
HZ connection failure is non-fatal: `hz_client = None`, posture defaults to APEX.
|
||
MC-Forewarner failure is non-fatal: logs warning, `_day_mc_scale` stays 1.0 (gate disabled).
|
||
|
||
### 5.3 on_bar() — Hot Loop
|
||
|
||
```
|
||
on_bar(bar: Bar):
|
||
① Apply pending ACB (under _acb_lock):
|
||
pending = _pending_acb; _pending_acb = None
|
||
if pending: engine.update_acb_boost(boost, beta)
|
||
|
||
② Date boundary detection:
|
||
date_str = datetime.fromtimestamp(bar.ts_event / 1e9, UTC).strftime('%Y-%m-%d')
|
||
if current_date != date_str:
|
||
if current_date: engine.end_day()
|
||
current_date = date_str
|
||
posture = _read_posture()
|
||
_bar_idx_today = 0
|
||
engine.begin_day(date_str, posture=posture, direction=±1)
|
||
if not live_mode: _load_parquet_data(date_str) → _day_data
|
||
|
||
③ HIBERNATE guard:
|
||
if posture == 'HIBERNATE': return # no position opened
|
||
|
||
④ Feature extraction (live HZ vs replay parquet):
|
||
live_mode=True: _get_latest_hz_scan() → scan dict
|
||
staleness check: abs(now_ns - scan_ts_ns) > 10s → warning
|
||
dedup: scan_num == last_scan_number → skip
|
||
live_mode=False: if _day_data empty → return (no step_bar with zeros)
|
||
elif bar_idx >= len(df) → return (end of day)
|
||
else: df.iloc[_bar_idx_today] → row
|
||
vol_regime_ok = bar_idx >= 100 (warmup)
|
||
|
||
⑤ Stale-state snapshot (before):
|
||
_snap = _GateSnap(acb_boost, acb_beta, posture, mc_gate_open)
|
||
|
||
⑥ Optional proxy_B pre-update (no-op for baseline engine):
|
||
if hasattr(engine, 'pre_bar_proxy_update'): engine.pre_bar_proxy_update(...)
|
||
|
||
⑦ engine.step_bar(bar_idx, vel_div, prices, v50_vel, v750_vel, vol_regime_ok)
|
||
_bar_idx_today += 1
|
||
|
||
⑧ Stale-state snapshot (after):
|
||
_snap_post = _GateSnap(acb_boost, acb_beta, _read_posture(), mc_gate_open)
|
||
if _snap != _snap_post:
|
||
stale_state_events++
|
||
log.warning("[STALE_STATE] ...")
|
||
result['stale_state'] = True
|
||
|
||
⑨ _write_result_to_hz(date_str, result)
|
||
```
|
||
|
||
### 5.4 ACB Thread Safety — Pending-Flag Pattern
|
||
|
||
```
|
||
HZ listener thread:
|
||
_on_acb_event(event):
|
||
parsed = json.loads(event.value) # parse OUTSIDE lock (pure CPU)
|
||
with _acb_lock:
|
||
_pending_acb = parsed # atomic assign under lock
|
||
|
||
on_bar() (Nautilus event thread):
|
||
with _acb_lock:
|
||
pending = _pending_acb
|
||
_pending_acb = None # consume atomically
|
||
if pending:
|
||
engine.update_acb_boost(...) # apply outside lock
|
||
```
|
||
|
||
This design minimizes lock hold time to a single pointer swap. There is no blocking I/O under the lock.
|
||
|
||
### 5.5 on_stop()
|
||
|
||
```python
|
||
def on_stop(self):
|
||
self._processed_dates.clear() # prevent stale date state on restart
|
||
self._stale_state_events = 0
|
||
if self.hz_client:
|
||
self.hz_client.shutdown()
|
||
```
|
||
|
||
---
|
||
|
||
## 6. ACBv6 — Adaptive Circuit Breaker
|
||
|
||
**File:** `nautilus_dolphin/nautilus/adaptive_circuit_breaker.py`
|
||
|
||
### 6.1 Three-Scale Architecture
|
||
|
||
```
|
||
regime_size_mult = base_boost × (1 + beta × strength_cubic) × mc_scale
|
||
|
||
Scale 1 — Daily external factors (base_boost):
|
||
preloaded from recent 60-day w750 velocity history
|
||
p60 threshold determines whether current w750 is "high regime"
|
||
base_boost ∈ [0.5, 2.0] typically
|
||
|
||
Scale 2 — Per-bar meta-boost (beta × strength_cubic):
|
||
beta: ACB sensitivity parameter from HZ DOLPHIN_FEATURES
|
||
strength_cubic: (|vel_div| / threshold)^3 — convex response to signal strength
|
||
|
||
Scale 3 — MC-Forewarner scale (mc_scale):
|
||
DolphinForewarner ML model predicts MC regime
|
||
mc_scale ∈ [0.5, 1.5]
|
||
```
|
||
|
||
### 6.2 HZ Integration
|
||
|
||
ACBv6 updates are pushed to `DOLPHIN_FEATURES["acb_boost"]` by an external Prefect flow. DolphinActor subscribes via `add_entry_listener` and receives push notifications. Updates are applied at the top of the next `on_bar()` call (pending-flag pattern, §5.4).
|
||
|
||
### 6.3 Cut-to-Position-Size API
|
||
|
||
```python
|
||
acb.apply_cut_to_position_size(position_size, cut_pct)
|
||
# cut_pct in [0.0, 0.15, 0.45, 0.55, 0.75, 0.80]
|
||
# Returns position_size × (1 - cut_pct)
|
||
```
|
||
|
||
---
|
||
|
||
## 7. Survival Stack (5-Sensor Posture)
|
||
|
||
**HZ map:** `DOLPHIN_SAFETY` (CP atomic reference preferred, map fallback)
|
||
|
||
```
|
||
Rm ∈ [0, 1] — composite risk metric from 5 sensors
|
||
|
||
Posture Rm threshold Behavior
|
||
───────── ──────────── ─────────────────────────────────────────────
|
||
APEX Rm ≥ 0.90 Full operation; abs_max_leverage unrestricted
|
||
STALKER Rm ≥ 0.75 max_leverage capped to 2.0
|
||
TURTLE Rm ≥ 0.50 position sizing reduced via abs_max_leverage × Rm
|
||
HIBERNATE Rm < 0.50 on_bar() returns immediately; no new positions
|
||
```
|
||
|
||
Posture is re-read on every date change. In `paper_trade_flow.py`, Rm is applied directly to `engine.abs_max_leverage`:
|
||
```python
|
||
engine.abs_max_leverage = max(1.0, engine.abs_max_leverage × Rm)
|
||
if posture == 'STALKER':
|
||
engine.abs_max_leverage = min(engine.abs_max_leverage, 2.0)
|
||
```
|
||
|
||
---
|
||
|
||
## 8. NDAlphaEngine API
|
||
|
||
**File:** `nautilus_dolphin/nautilus/esf_alpha_orchestrator.py`
|
||
|
||
### 8.1 Constructor Parameters
|
||
|
||
See §3 Champion Parameters. Additional non-champion params:
|
||
- `stop_pct=1.0` (effectively disabled — TP exits first)
|
||
- `lookback=100` (price history window)
|
||
- `use_alpha_layers=True` (enables OB/IRP/SP layers)
|
||
- `use_dynamic_leverage=True` (enables cubic-convex sizing)
|
||
|
||
### 8.2 Day Lifecycle API
|
||
|
||
```python
|
||
engine.begin_day(date_str: str, posture: str, direction: int)
|
||
# Sets regime_direction, reads ACB for the day, resets per-day state
|
||
|
||
for bar in bars:
|
||
result = engine.step_bar(
|
||
bar_idx=int, # 0-based index within day
|
||
vel_div=float, # primary alpha signal
|
||
prices=dict, # {symbol: float} current prices
|
||
vol_regime_ok=bool, # volatility gate
|
||
v50_vel=float, # w50 eigenvalue velocity (raw)
|
||
v750_vel=float, # w750 eigenvalue velocity (ACB scale)
|
||
) -> dict
|
||
|
||
result_dict = engine.end_day()
|
||
# Returns: {pnl, trades, capital, boost, beta, mc_status, ...}
|
||
```
|
||
|
||
### 8.3 State Fields
|
||
|
||
| Field | Type | Description |
|
||
|---|---|---|
|
||
| `capital` | float | Current equity (updated after each trade) |
|
||
| `_day_base_boost` | float | ACB base boost for today |
|
||
| `_day_beta` | float | ACB beta sensitivity for today |
|
||
| `_day_mc_scale` | float | MC-Forewarner scale for today |
|
||
| `_global_bar_idx` | int | Lifetime bar counter (persists across days) |
|
||
| `_price_histories` | dict | Per-asset price history lists (≤500 values) |
|
||
| `position` | NDPosition | Current open position (None if flat) |
|
||
| `trade_history` | list | All closed NDTradeRecord objects |
|
||
| `regime_size_mult` | float | Current ACBv6 size multiplier |
|
||
|
||
### 8.4 Setter Methods
|
||
|
||
```python
|
||
engine.set_ob_engine(ob_engine) # inject OBFeatureEngine
|
||
engine.set_acb(acb) # inject AdaptiveCircuitBreaker
|
||
engine.set_mc_forewarner(fw, base_cfg) # inject DolphinForewarner
|
||
engine.update_acb_boost(boost, beta) # called by DolphinActor from HZ events
|
||
```
|
||
|
||
---
|
||
|
||
## 9. Data Flow — Replay Mode (Paper Trading)
|
||
|
||
```
|
||
vbt_cache_klines/YYYY-MM-DD.parquet
|
||
↓ DolphinActor._load_parquet_data()
|
||
↓ pd.read_parquet() → DataFrame (1439 rows × ~57 cols)
|
||
columns: timestamp, scan_number, vel_div,
|
||
v50/v150/v300/v750_lambda_max_velocity,
|
||
instability_50, instability_150,
|
||
BTCUSDT, ETHUSDT, BNBUSDT, ... (48 assets)
|
||
↓ DolphinActor.on_bar() iterates rows via _bar_idx_today
|
||
↓ NDAlphaEngine.step_bar(bar_idx, vel_div, prices, ...)
|
||
↓ AlphaSignalGenerator → AlphaBetSizer → AlphaExitManager
|
||
↓ trade_history.append(NDTradeRecord)
|
||
↓ DolphinActor._write_result_to_hz() → DOLPHIN_PNL_BLUE[date]
|
||
```
|
||
|
||
### 9.1 Live Mode Data Flow
|
||
|
||
```
|
||
Binance Futures WS → OBF prefect flow → Hazelcast DOLPHIN_FEATURES_SHARD_*
|
||
Eigenvalue scanner → JSON scan files → Hazelcast DOLPHIN_FEATURES["latest_eigen_scan"]
|
||
|
||
DolphinActor.on_bar():
|
||
scan = _get_latest_hz_scan()
|
||
vel_div = scan["vel_div"]
|
||
prices = scan["asset_prices"]
|
||
→ engine.step_bar(...)
|
||
```
|
||
|
||
---
|
||
|
||
## 10. Hazelcast IMap Schema
|
||
|
||
| Map name | Key | Value | Writer | Reader |
|
||
|---|---|---|---|---|
|
||
| `DOLPHIN_SAFETY` | "latest" | JSON `{posture, Rm, ...}` | Survival stack flow | DolphinActor, paper_trade_flow |
|
||
| `DOLPHIN_FEATURES` | "acb_boost" | JSON `{boost, beta}` | ACB writer flow | DolphinActor (listener) |
|
||
| `DOLPHIN_FEATURES` | "latest_eigen_scan" | JSON `{vel_div, scan_number, asset_prices, ...}` | Eigenvalue scanner | DolphinActor (live mode) |
|
||
| `DOLPHIN_PNL_BLUE` | "YYYY-MM-DD" | JSON result dict | paper_trade_flow, DolphinActor | Analytics |
|
||
| `DOLPHIN_STATE_BLUE` | "latest" | JSON `{capital, date, pnl, ...}` | paper_trade_flow | paper_trade_flow (restore) |
|
||
| `DOLPHIN_STATE_BLUE` | "latest_nautilus" | JSON `{capital, param_hash, ...}` | nautilus_prefect_flow | nautilus_prefect_flow |
|
||
| `DOLPHIN_HEARTBEAT` | "nautilus_flow_heartbeat" | JSON `{ts, phase, ...}` | nautilus_prefect_flow | Monitoring |
|
||
| `DOLPHIN_FEATURES_SHARD_00..09` | symbol | JSON OB feature dict | OBF prefect flow | HZOBProvider |
|
||
|
||
**Shard routing:** `shard_idx = sum(ord(c) for c in symbol) % 10` — stable, deterministic, no config needed.
|
||
|
||
---
|
||
|
||
## 11. Prefect Integration
|
||
|
||
### 11.1 paper_trade_flow.py (Primary — 00:05 UTC)
|
||
|
||
Runs NDAlphaEngine directly (no Nautilus kernel). Tasks:
|
||
- `load_config` — YAML config with retries=0
|
||
- `load_day_scans` — parquet (preferred) or JSON fallback, retries=2
|
||
- `run_engine_day` — NDAlphaEngine.begin_day/step_bar/end_day loop
|
||
- `write_hz_state` — HZ persist, retries=3
|
||
- `log_pnl` — disk JSONL append
|
||
|
||
### 11.2 nautilus_prefect_flow.py (Nautilus Supervisor — 00:10 UTC)
|
||
|
||
Wraps BacktestEngine + DolphinActor. Tasks:
|
||
- `hz_probe_task` — verify HZ reachable, retries=3, timeout=30s
|
||
- `validate_champion_params` — SHA256 hash check vs `_CHAMPION_PARAMS`, aborts on drift
|
||
- `load_bar_data_task` — parquet load with validation, retries=2
|
||
- `read_posture_task` — DOLPHIN_SAFETY read, retries=2
|
||
- `restore_capital_task` — capital continuity from HZ state
|
||
- `run_nautilus_backtest_task` — full BacktestEngine cycle, timeout=600s
|
||
- `write_hz_result_task` — persist to DOLPHIN_PNL_BLUE + DOLPHIN_STATE_BLUE, retries=3
|
||
- `heartbeat_task` — liveness pulse at flow_start/engine_start/flow_end
|
||
|
||
### 11.3 Registration
|
||
|
||
```bash
|
||
source /home/dolphin/siloqy_env/bin/activate
|
||
PREFECT_API_URL=http://localhost:4200/api
|
||
|
||
# Primary paper trade (existing):
|
||
python prod/paper_trade_flow.py --register
|
||
|
||
# Nautilus supervisor (new):
|
||
python prod/nautilus_prefect_flow.py --register
|
||
# → dolphin-nautilus-blue, daily 00:10 UTC, work_pool=dolphin
|
||
```
|
||
|
||
---
|
||
|
||
## 12. Nautilus Kernel Backends
|
||
|
||
### 12.1 BacktestEngine (Paper / Replay)
|
||
|
||
Used in `run_nautilus.py` and `nautilus_prefect_flow.py`. Processes synthetic bars (one bar per date triggers DolphinActor which then iterates over the full parquet day internally). No real exchange connectivity.
|
||
|
||
```python
|
||
engine = BacktestEngine(config=BacktestEngineConfig(trader_id="DOLPHIN-NAUTILUS-001"))
|
||
engine.add_strategy(DolphinActor(config=config))
|
||
engine.add_venue(Venue("BINANCE"), OmsType.HEDGING, AccountType.MARGIN, ...)
|
||
engine.add_instrument(TestInstrumentProvider.default_fx_ccy("BTCUSD", venue))
|
||
engine.add_data([synthetic_bar])
|
||
engine.run()
|
||
```
|
||
|
||
### 12.2 TradingNode (Live — Future)
|
||
|
||
`NautilusDolphinLauncher` in `launcher.py` bootstraps a `TradingNode` with `BinanceExecClientConfig`. Requires Binance API keys and live WS data. Not currently active.
|
||
|
||
```python
|
||
from nautilus_dolphin.nautilus.launcher import NautilusDolphinLauncher
|
||
launcher = NautilusDolphinLauncher(config_path="prod/configs/blue.yml")
|
||
launcher.start() # blocking — runs until SIGTERM
|
||
```
|
||
|
||
### 12.3 Bar Type
|
||
|
||
```
|
||
"BTCUSD.BINANCE-5-SECOND-LAST-EXTERNAL"
|
||
```
|
||
|
||
`EXTERNAL` aggregation type: bars are not synthesized by Nautilus from ticks; they are injected directly. This is the correct type for replay from pre-aggregated parquet.
|
||
|
||
---
|
||
|
||
## 13. DolphinStrategyConfig
|
||
|
||
**File:** `nautilus_dolphin/nautilus/strategy_config.py`
|
||
|
||
```python
|
||
class DolphinStrategyConfig(StrategyConfig, kw_only=True, frozen=True):
|
||
vel_div_threshold: float = -0.02
|
||
vel_div_extreme: float = -0.05
|
||
fixed_tp_pct: float = 0.0095
|
||
max_hold_bars: int = 120
|
||
fraction: float = 0.20
|
||
min_leverage: float = 0.5
|
||
max_leverage: float = 5.0
|
||
abs_max_leverage: float = 6.0
|
||
leverage_convexity: float = 3.0
|
||
dc_lookback_bars: int = 7
|
||
dc_min_magnitude_bps: float = 0.75
|
||
min_irp_alignment: float = 0.45
|
||
sp_maker_entry_rate: float = 0.62
|
||
sp_maker_exit_rate: float = 0.50
|
||
seed: int = 42
|
||
# ...
|
||
```
|
||
|
||
Factory methods:
|
||
- `create_champion_config()` → excluded_assets=["TUSDUSDT","USDCUSDT"]
|
||
- `create_conservative_config()` → reduced leverage/fraction
|
||
- `create_growth_config()` → increased leverage
|
||
- `create_aggressive_config()` → max leverage stack
|
||
|
||
---
|
||
|
||
## 14. Test Suite Summary
|
||
|
||
| File | Tests | Coverage |
|
||
|---|---|---|
|
||
| `test_0_nautilus_bootstrap.py` | 11 | Import chain, NautilusKernelConfig, ACB, CircuitBreaker, launcher |
|
||
| `test_dolphin_actor.py` | 35 | Champion params, ACB thread-safety, HIBERNATE guard, date change, HZ degradation, replay mode, on_stop, _GateSnap |
|
||
| `test_strategy.py` | 4+ | DolphinExecutionStrategy signal filters |
|
||
| `test_adaptive_circuit_breaker.py` | ~10 | ACBv6 scale computation, cut-to-size |
|
||
| `test_circuit_breaker.py` | ~6 | CircuitBreakerManager is_tripped, can_open, status |
|
||
| `test_volatility_detector.py` | ~6 | VolatilityRegimeDetector is_high_regime |
|
||
| `test_position_manager.py` | ~5 | PositionManager state |
|
||
| `test_smart_exec_algorithm.py` | ~6 | SmartExecAlgorithm routing |
|
||
| `test_signal_bridge.py` | ~4 | SignalBridgeActor event handling |
|
||
| `test_metrics_monitor.py` | ~4 | MetricsMonitor state |
|
||
|
||
**Run all:**
|
||
```bash
|
||
source /home/dolphin/siloqy_env/bin/activate
|
||
cd /mnt/dolphinng5_predict
|
||
python -m pytest nautilus_dolphin/tests/ -v
|
||
```
|
||
|
||
**Run DolphinActor tests only:**
|
||
```bash
|
||
python -m pytest nautilus_dolphin/tests/test_dolphin_actor.py -v # 35/35
|
||
```
|
||
|
||
---
|
||
|
||
## 15. Deployment Procedures
|
||
|
||
### 15.1 siloqy-env Activation
|
||
|
||
All production and test commands must run in siloqy-env:
|
||
```bash
|
||
source /home/dolphin/siloqy_env/bin/activate
|
||
# Verify: python -c "import nautilus_trader; print(nautilus_trader.__version__)"
|
||
# Expected: 1.219.0
|
||
```
|
||
|
||
### 15.2 Daily Paper Trade (Manual)
|
||
|
||
```bash
|
||
source /home/dolphin/siloqy_env/bin/activate
|
||
cd /mnt/dolphinng5_predict
|
||
PREFECT_API_URL=http://localhost:4200/api \
|
||
python prod/paper_trade_flow.py --config prod/configs/blue.yml --date 2026-03-21
|
||
```
|
||
|
||
### 15.3 Nautilus BacktestEngine Run (Manual)
|
||
|
||
```bash
|
||
source /home/dolphin/siloqy_env/bin/activate
|
||
cd /mnt/dolphinng5_predict
|
||
python prod/run_nautilus.py --config prod/configs/blue.yml
|
||
```
|
||
|
||
### 15.4 Nautilus Prefect Flow (Manual)
|
||
|
||
```bash
|
||
source /home/dolphin/siloqy_env/bin/activate
|
||
cd /mnt/dolphinng5_predict
|
||
PREFECT_API_URL=http://localhost:4200/api \
|
||
python prod/nautilus_prefect_flow.py --date 2026-03-21
|
||
```
|
||
|
||
### 15.5 Dry Run (Data + Param Validation Only)
|
||
|
||
```bash
|
||
python prod/nautilus_prefect_flow.py --date 2026-03-21 --dry-run
|
||
```
|
||
|
||
### 15.6 Register Prefect Deployments
|
||
|
||
```bash
|
||
PREFECT_API_URL=http://localhost:4200/api \
|
||
python prod/paper_trade_flow.py --register # dolphin-paper-blue, 00:05 UTC
|
||
|
||
PREFECT_API_URL=http://localhost:4200/api \
|
||
python prod/nautilus_prefect_flow.py --register # dolphin-nautilus-blue, 00:10 UTC
|
||
```
|
||
|
||
### 15.7 Prefect Worker
|
||
|
||
```bash
|
||
source /home/dolphin/siloqy_env/bin/activate
|
||
PREFECT_API_URL=http://localhost:4200/api \
|
||
prefect worker start --pool dolphin --type process
|
||
```
|
||
|
||
---
|
||
|
||
## 16. HZ Sharded Feature Store
|
||
|
||
**Map pattern:** `DOLPHIN_FEATURES_SHARD_{shard_idx}`
|
||
**Shard count:** 10
|
||
**Routing:**
|
||
```python
|
||
shard_idx = sum(ord(c) for c in symbol) % SHARD_COUNT
|
||
imap_name = f"DOLPHIN_FEATURES_SHARD_{shard_idx:02d}"
|
||
```
|
||
|
||
The OBF flow writes per-asset OB features to the correct shard. `HZOBProvider` uses dynamic discovery (reads key_set from all 10 shards at startup) to find which assets are present.
|
||
|
||
---
|
||
|
||
## 17. Operational Invariants
|
||
|
||
1. **Champion param hash must match** at every flow start. `_CHAMPION_HASH = "..."` computed from `_CHAMPION_PARAMS` dict. Mismatch → `ValueError` → flow abort.
|
||
|
||
2. **Seed=42 is mandatory** for reproducibility. numba RNG uses Numba's internal PRNG initialized from seed. NumPy RandomState(42) used in NDAlphaEngine. Any change to seed invalidates backtest comparison.
|
||
|
||
3. **HIBERNATE is hard** — deliberately tight (Rm < 0.50). When posture=HIBERNATE, `on_bar()` returns immediately, no exceptions, no logging above WARNING.
|
||
|
||
4. **Stale-state events are logged but not fatal.** `_stale_state_events` counter increments; result dict gets `stale_state=True`. The trade result is written to HZ with a DO-NOT-USE flag in the log. Downstream systems must check this field.
|
||
|
||
5. **HZ unavailability is non-fatal.** If HZ is unreachable at on_start, `hz_client=None`, posture defaults to APEX. Flow continues with local state only. The `hz_probe_task` retries 3× before giving up with a warning (not an error).
|
||
|
||
6. **Capital continuity.** Each flow run restores capital from `DOLPHIN_STATE_BLUE["latest_nautilus"]`. If absent, falls back to `initial_capital` from config (25,000 USDT).
|
||
|
||
7. **Date boundary is ts_event-driven.** The Nautilus bar's `ts_event` nanoseconds are the authoritative source of truth for date detection. Wall clock is not used.
|
||
|
||
---
|
||
|
||
## 18. Known Limitations and Future Work
|
||
|
||
| Item | Status | Notes |
|
||
|---|---|---|
|
||
| Live TradingNode (Binance) | Pending | launcher.py exists; requires API keys + WS data integration |
|
||
| 0.1s (10 Hz) resolution | Blocked | 3 blockers: async HZ push, timeout reduction, lookback recalibration |
|
||
| LONG validation (green config) | Pending | green.yml exists; needs backtest sign-off |
|
||
| ML-MC Forewarner in Nautilus flow | **Done** | Wired in `DolphinActor.on_start()` — auto-injects for both flows; `_MC_BASE_CFG` frozen constant |
|
||
| Full end-to-end Nautilus replay parity | In progress | test_nd_vs_standalone_comparison.py exists; champion param parity confirmed |
|
||
|
||
---
|
||
|
||
*Spec version 1.0 — 2026-03-22 — Nautilus-DOLPHIN Alpha Engine Core*
|