initial: import DOLPHIN baseline 2026-04-21 from dolphinng5_predict working tree
Includes core prod + GREEN/BLUE subsystems: - prod/ (BLUE harness, configs, scripts, docs) - nautilus_dolphin/ (GREEN Nautilus-native impl + dvae/ preserved) - adaptive_exit/ (AEM engine + models/bucket_assignments.pkl) - Observability/ (EsoF advisor, TUI, dashboards) - external_factors/ (EsoF producer) - mc_forewarning_qlabs_fork/ (MC regime/envelope) Excludes runtime caches, logs, backups, and reproducible artifacts per .gitignore.
This commit is contained in:
hjnormey
395
nautilus_dolphin/IMPLEMENTATION_SUMMARY.md
Executable file
395
nautilus_dolphin/IMPLEMENTATION_SUMMARY.md
Executable file
@@ -0,0 +1,395 @@
|
||||
# DOLPHIN Nautilus Implementation Summary
|
||||
|
||||
**Date**: 2026-02-18
|
||||
**Status**: Phase 3 Complete, Phase 4 Started
|
||||
|
||||
---
|
||||
|
||||
## Implementation Status by Task
|
||||
|
||||
### Phase 1: Foundation ✅ Complete
|
||||
- ✅ 1.1 Development Environment Setup (configured in requirements.txt, config.yaml)
|
||||
- ✅ 1.2 Signal Bridge Actor (`signal_bridge.py`)
|
||||
- Redis Streams consumption with xread
|
||||
- Timestamp parsing (seconds/ms/ns)
|
||||
- Signal validation with freshness check
|
||||
- Error handling and backoff
|
||||
- ✅ 1.3 Basic Execution Strategy (`strategy.py`)
|
||||
- Signal subscription and filtering
|
||||
- Lifecycle methods (on_start/on_stop)
|
||||
- SmartExecAlgorithm registration
|
||||
|
||||
### Phase 2: Core Logic ✅ Complete
|
||||
- ✅ 2.1 Signal Filtering (`volatility_detector.py`, `strategy.py`)
|
||||
- VolatilityRegimeDetector with P50/P75 thresholds
|
||||
- IRP alignment filter (>=0.45)
|
||||
- Direction confirmation filter
|
||||
- Momentum magnitude filter (>0.75bps)
|
||||
- Asset exclusion (stablecoins)
|
||||
- Position limits (max 10 concurrent)
|
||||
- ✅ 2.2 Dynamic Leverage Calculation (`strategy.py`)
|
||||
- Base formula: `min_lev + (strength^convexity) * (max_lev - min_lev)`
|
||||
- Alpha multipliers: bucket_boost, streak_mult, trend_mult
|
||||
- Sanity check (max 50% account balance)
|
||||
- ✅ 2.3 Position Sizing (`strategy.py`)
|
||||
- Notional calculation: `balance * fraction * leverage`
|
||||
- Quantity conversion with precision
|
||||
- ✅ 2.4 Exit Logic (`position_manager.py`)
|
||||
- PositionManager class
|
||||
- TP condition (99bps for SHORT)
|
||||
- Max hold condition (120 bars)
|
||||
- Exit execution via SmartExecAlgorithm
|
||||
|
||||
### Phase 3: Execution ✅ Complete
|
||||
- ✅ 3.1 SmartExecAlgorithm Entry Orders (`smart_exec_algorithm.py`)
|
||||
- Limit order 1bps inside spread
|
||||
- 25s timeout with market fallback
|
||||
- **NEW**: Abort when price moves 5bps against
|
||||
- Fill tracking (maker/taker)
|
||||
- ✅ 3.2 SmartExecAlgorithm Exit Orders (`smart_exec_algorithm.py`)
|
||||
- TP exit: market order
|
||||
- Max hold: limit 10s → market fallback
|
||||
- Order rejection handler
|
||||
- ✅ 3.3 Fee and Slippage Measurement (`smart_exec_algorithm.py`)
|
||||
- **NEW**: Fee tracking (0.02% maker, 0.05% taker)
|
||||
- **NEW**: Slippage calculation (actual vs expected price)
|
||||
- **NEW**: Metrics collection via `get_metrics()` / `reset_metrics()`
|
||||
- ✅ 3.4 Circuit Breakers (`circuit_breaker.py`)
|
||||
- **NEW**: CircuitBreakerManager class
|
||||
- Daily loss limit (10% hard stop)
|
||||
- Max concurrent positions (10)
|
||||
- Per-asset position limit (1)
|
||||
- API failure tracking (3 consecutive)
|
||||
- Order size sanity check (50%)
|
||||
- Auto-reset after 24 hours
|
||||
- ✅ 3.5 Metrics Monitor (`metrics_monitor.py`)
|
||||
- **NEW**: MetricsMonitor class
|
||||
- Maker fill rate tracking (1-hour rolling)
|
||||
- Slippage tracking (rolling average)
|
||||
- Funding rate tracking (24-hour)
|
||||
- Threshold alerting (warning/critical)
|
||||
- Prometheus export format
|
||||
|
||||
### Phase 4: Validation 🔄 In Progress
|
||||
- ✅ 4.1 JSON Data Adapter (`data_adapter.py`)
|
||||
- **NEW**: JSONEigenvalueDataAdapter class
|
||||
- Loads eigenvalue JSON files from correlation_arb512
|
||||
- Generates synthetic bars from vel_div data
|
||||
- Signal metadata extraction
|
||||
- BacktestDataLoader for high-level loading
|
||||
- ⬜ 4.2 Validation Backtests (pending)
|
||||
- ⬜ 4.3 Validation Analysis (pending)
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
nautilus_dolphin/
|
||||
├── nautilus_dolphin/ # Main package
|
||||
│ ├── __init__.py # Package exports
|
||||
│ └── nautilus/ # Nautilus components
|
||||
│ ├── __init__.py # Component exports
|
||||
│ ├── signal_bridge.py # Redis → Nautilus bridge
|
||||
│ ├── strategy.py # Main execution strategy
|
||||
│ ├── smart_exec_algorithm.py # Entry/exit execution
|
||||
│ ├── position_manager.py # Exit management
|
||||
│ ├── volatility_detector.py # Vol regime detection
|
||||
│ ├── circuit_breaker.py # Operational safety
|
||||
│ ├── metrics_monitor.py # Stress-test metrics
|
||||
│ └── data_adapter.py # JSON backtest loader
|
||||
├── tests/ # Test suite
|
||||
│ ├── test_signal_bridge.py
|
||||
│ ├── test_strategy.py
|
||||
│ ├── test_position_manager.py
|
||||
│ ├── test_volatility_detector.py
|
||||
│ ├── test_circuit_breaker.py # NEW
|
||||
│ ├── test_metrics_monitor.py # NEW
|
||||
│ └── test_smart_exec_algorithm.py # NEW
|
||||
├── config/
|
||||
│ └── config.yaml
|
||||
├── pyproject.toml # Package config
|
||||
├── requirements.txt
|
||||
└── README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## New Components Detail
|
||||
|
||||
### 1. CircuitBreakerManager (`circuit_breaker.py`)
|
||||
|
||||
```python
|
||||
cb = CircuitBreakerManager(
|
||||
daily_loss_limit_pct=10.0,
|
||||
max_concurrent_positions=10,
|
||||
max_api_failures=3,
|
||||
max_order_size_pct=50.0
|
||||
)
|
||||
|
||||
# Check before opening position
|
||||
can_trade, reason = cb.can_open_position("BTCUSDT", balance)
|
||||
|
||||
# Track position lifecycle
|
||||
cb.on_position_opened(position_id, asset)
|
||||
cb.on_position_closed(position_id, asset, pnl)
|
||||
|
||||
# Check order size
|
||||
can_submit, reason = cb.can_submit_order(notional, balance)
|
||||
|
||||
# Monitor API health
|
||||
cb.on_api_failure(error_message)
|
||||
|
||||
# Manual control
|
||||
cb.manual_trip("Emergency stop")
|
||||
cb.reset()
|
||||
|
||||
# Get status
|
||||
status = cb.get_status()
|
||||
```
|
||||
|
||||
### 2. MetricsMonitor (`metrics_monitor.py`)
|
||||
|
||||
```python
|
||||
monitor = MetricsMonitor(
|
||||
config=ThresholdConfig(
|
||||
min_maker_fill_rate=48.0,
|
||||
max_slippage_bps=5.0
|
||||
)
|
||||
)
|
||||
|
||||
# Record fills
|
||||
monitor.record_fill('maker', slippage_bps=2.5)
|
||||
monitor.record_fill('taker', slippage_bps=4.0)
|
||||
|
||||
# Record trade results
|
||||
monitor.record_trade_result(pnl=100.0)
|
||||
|
||||
# Get metrics
|
||||
summary = monitor.get_metrics_summary()
|
||||
# Returns: maker_fill_rate_pct, avg_slippage_bps,
|
||||
# win_rate_pct, profit_factor, etc.
|
||||
|
||||
# Prometheus export
|
||||
prometheus_metrics = monitor.get_prometheus_metrics()
|
||||
```
|
||||
|
||||
### 3. SmartExecAlgorithm Enhancements
|
||||
|
||||
**Abort Logic** (Task 3.1.5):
|
||||
- Monitors quote ticks for pending entries
|
||||
- Cancels order if price moves 5bps against position
|
||||
- Tracks aborted entries in metrics
|
||||
|
||||
**Fee/Slippage Tracking** (Task 3.3):
|
||||
- Calculates fees: 0.02% maker, 0.05% taker
|
||||
- Tracks slippage: |actual - expected| / expected
|
||||
- Provides metrics via `get_metrics()` method
|
||||
|
||||
### 4. JSONEigenvalueDataAdapter (`data_adapter.py`)
|
||||
|
||||
```python
|
||||
adapter = JSONEigenvalueDataAdapter(
|
||||
eigenvalues_dir="path/to/correlation_arb512/eigenvalues",
|
||||
venue="BINANCE_FUTURES"
|
||||
)
|
||||
|
||||
# Load date range
|
||||
adapter.load_date_range(
|
||||
start_date=datetime(2026, 2, 6),
|
||||
end_date=datetime(2026, 2, 14)
|
||||
)
|
||||
|
||||
# Iterate through scans
|
||||
for bars, signals in adapter:
|
||||
# bars: List[Bar] - synthetic OHLCV
|
||||
# signals: List[dict] - signal metadata
|
||||
process_bars(bars)
|
||||
process_signals(signals)
|
||||
|
||||
# Or use high-level loader
|
||||
loader = BacktestDataLoader(eigenvalues_dir, venue)
|
||||
data = loader.load_period(start_date, end_date, assets=['BTCUSDT', 'ETHUSDT'])
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Strategy Integration
|
||||
|
||||
The `DolphinExecutionStrategy` now integrates all components:
|
||||
|
||||
```python
|
||||
class DolphinExecutionStrategy(Strategy):
|
||||
def __init__(self, config):
|
||||
# Components
|
||||
self.volatility_detector = VolatilityRegimeDetector(...)
|
||||
self.position_manager = PositionManager(...)
|
||||
self.circuit_breaker = CircuitBreakerManager(...)
|
||||
self.metrics_monitor = MetricsMonitor(...)
|
||||
|
||||
def on_signal(self, signal):
|
||||
# 1. Apply filters
|
||||
rejection = self._should_trade(signal)
|
||||
if rejection:
|
||||
return
|
||||
|
||||
# 2. Check circuit breaker
|
||||
can_trade, reason = self.circuit_breaker.can_open_position(...)
|
||||
if not can_trade:
|
||||
return
|
||||
|
||||
# 3. Execute trade
|
||||
self._execute_entry(signal)
|
||||
|
||||
def _execute_entry(self, signal):
|
||||
# Calculate size
|
||||
notional = self.calculate_position_size(signal, balance)
|
||||
|
||||
# Check order sanity
|
||||
can_submit, reason = self.circuit_breaker.can_submit_order(...)
|
||||
|
||||
# Submit via SmartExecAlgorithm
|
||||
self.submit_order(order, exec_algorithm_id="SMART_EXEC")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test Coverage
|
||||
|
||||
New tests added:
|
||||
|
||||
1. **test_circuit_breaker.py** (11 tests)
|
||||
- Position limit checks
|
||||
- Daily loss limit
|
||||
- Order size sanity
|
||||
- API failure tracking
|
||||
- Manual trip/reset
|
||||
- Auto-reset after timeout
|
||||
|
||||
2. **test_metrics_monitor.py** (13 tests)
|
||||
- Fill rate calculation
|
||||
- Slippage tracking
|
||||
- Trade result recording
|
||||
- Win rate / profit factor
|
||||
- Alert generation
|
||||
- Deduplication
|
||||
- Prometheus export
|
||||
|
||||
3. **test_smart_exec_algorithm.py** (10 tests)
|
||||
- Abort logic (price movement)
|
||||
- Fee calculation (maker/taker)
|
||||
- Slippage calculation
|
||||
- Metrics tracking
|
||||
- Category classification
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
To complete Phase 4 (Validation):
|
||||
|
||||
1. **Set up validation backtests**:
|
||||
```python
|
||||
# Load data for validation periods
|
||||
loader = BacktestDataLoader(eigenvalues_dir)
|
||||
|
||||
# High volatility: Feb 6-14, 2026
|
||||
feb_data = loader.load_period(
|
||||
datetime(2026, 2, 6),
|
||||
datetime(2026, 2, 14)
|
||||
)
|
||||
|
||||
# Low volatility: Jan 21-28, 2026
|
||||
jan_data = loader.load_period(
|
||||
datetime(2026, 1, 21),
|
||||
datetime(2026, 1, 28)
|
||||
)
|
||||
```
|
||||
|
||||
2. **Run Nautilus backtests** using the loaded data
|
||||
|
||||
3. **Compare results** to VBT baseline:
|
||||
- Total trades (±2% tolerance)
|
||||
- Win rate (±1% tolerance)
|
||||
- Profit factor (±3% tolerance)
|
||||
- Final capital (±5% tolerance)
|
||||
|
||||
---
|
||||
|
||||
## Configuration Example
|
||||
|
||||
```yaml
|
||||
# config/config.yaml
|
||||
strategy:
|
||||
venue: "BINANCE_FUTURES"
|
||||
|
||||
# Filters
|
||||
irp_alignment_min: 0.45
|
||||
momentum_magnitude_min: 0.000075
|
||||
excluded_assets: ["TUSDUSDT", "USDCUSDT"]
|
||||
|
||||
# Sizing
|
||||
min_leverage: 0.5
|
||||
max_leverage: 5.0
|
||||
leverage_convexity: 3.0
|
||||
capital_fraction: 0.20
|
||||
|
||||
# Exit
|
||||
tp_bps: 99
|
||||
max_hold_bars: 120
|
||||
|
||||
# Limits
|
||||
max_concurrent_positions: 10
|
||||
|
||||
circuit_breaker:
|
||||
daily_loss_limit_pct: 10.0
|
||||
max_api_failures: 3
|
||||
max_order_size_pct: 50.0
|
||||
auto_reset_hours: 24.0
|
||||
|
||||
smart_exec:
|
||||
entry_timeout_sec: 25
|
||||
entry_abort_threshold_bps: 5.0
|
||||
exit_timeout_sec: 10
|
||||
maker_fee_rate: 0.0002
|
||||
taker_fee_rate: 0.0005
|
||||
|
||||
metrics:
|
||||
min_maker_fill_rate: 48.0
|
||||
max_slippage_bps: 5.0
|
||||
warning_maker_fill_rate: 55.0
|
||||
critical_maker_fill_rate: 48.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Running Tests
|
||||
|
||||
```bash
|
||||
# Activate environment
|
||||
activate_siloqy.bat
|
||||
|
||||
# Install in dev mode
|
||||
pip install -e .
|
||||
|
||||
# Run all tests
|
||||
python -m pytest tests/ -v
|
||||
|
||||
# Run specific test file
|
||||
python -m pytest tests/test_circuit_breaker.py -v
|
||||
|
||||
# Run with coverage
|
||||
python -m pytest tests/ --cov=nautilus_dolphin --cov-report=html
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
1. **NautilusTrader dependency**: The implementation requires NautilusTrader 1.200.0+ which should be installed in the siloqy environment.
|
||||
|
||||
2. **Import errors in test**: Tests will fail to import if nautilus_trader is not installed. This is expected in environments without the trading dependencies.
|
||||
|
||||
3. **Data adapter**: The JSONEigenvalueDataAdapter generates synthetic prices based on vel_div. For production backtests, consider using actual price data from Binance.
|
||||
|
||||
4. **Redis**: SignalBridgeActor requires Redis 7.0+ for the Redis Streams functionality.
|
||||
Reference in New Issue
Block a user