01c19662cb
Includes core prod + GREEN/BLUE subsystems: - prod/ (BLUE harness, configs, scripts, docs) - nautilus_dolphin/ (GREEN Nautilus-native impl + dvae/ preserved) - adaptive_exit/ (AEM engine + models/bucket_assignments.pkl) - Observability/ (EsoF advisor, TUI, dashboards) - external_factors/ (EsoF producer) - mc_forewarning_qlabs_fork/ (MC regime/envelope) Excludes runtime caches, logs, backups, and reproducible artifacts per .gitignore.
396 lines
11 KiB
Markdown
Executable File
396 lines
11 KiB
Markdown
Executable File
# DOLPHIN Nautilus Implementation Summary
|
|
|
|
**Date**: 2026-02-18
|
|
**Status**: Phase 3 Complete, Phase 4 Started
|
|
|
|
---
|
|
|
|
## Implementation Status by Task
|
|
|
|
### Phase 1: Foundation ✅ Complete
|
|
- ✅ 1.1 Development Environment Setup (configured in requirements.txt, config.yaml)
|
|
- ✅ 1.2 Signal Bridge Actor (`signal_bridge.py`)
|
|
- Redis Streams consumption with xread
|
|
- Timestamp parsing (seconds/ms/ns)
|
|
- Signal validation with freshness check
|
|
- Error handling and backoff
|
|
- ✅ 1.3 Basic Execution Strategy (`strategy.py`)
|
|
- Signal subscription and filtering
|
|
- Lifecycle methods (on_start/on_stop)
|
|
- SmartExecAlgorithm registration
|
|
|
|
### Phase 2: Core Logic ✅ Complete
|
|
- ✅ 2.1 Signal Filtering (`volatility_detector.py`, `strategy.py`)
|
|
- VolatilityRegimeDetector with P50/P75 thresholds
|
|
- IRP alignment filter (>=0.45)
|
|
- Direction confirmation filter
|
|
- Momentum magnitude filter (>0.75bps)
|
|
- Asset exclusion (stablecoins)
|
|
- Position limits (max 10 concurrent)
|
|
- ✅ 2.2 Dynamic Leverage Calculation (`strategy.py`)
|
|
- Base formula: `min_lev + (strength^convexity) * (max_lev - min_lev)`
|
|
- Alpha multipliers: bucket_boost, streak_mult, trend_mult
|
|
- Sanity check (max 50% account balance)
|
|
- ✅ 2.3 Position Sizing (`strategy.py`)
|
|
- Notional calculation: `balance * fraction * leverage`
|
|
- Quantity conversion with precision
|
|
- ✅ 2.4 Exit Logic (`position_manager.py`)
|
|
- PositionManager class
|
|
- TP condition (99bps for SHORT)
|
|
- Max hold condition (120 bars)
|
|
- Exit execution via SmartExecAlgorithm
|
|
|
|
### Phase 3: Execution ✅ Complete
|
|
- ✅ 3.1 SmartExecAlgorithm Entry Orders (`smart_exec_algorithm.py`)
|
|
- Limit order 1bps inside spread
|
|
- 25s timeout with market fallback
|
|
- **NEW**: Abort when price moves 5bps against
|
|
- Fill tracking (maker/taker)
|
|
- ✅ 3.2 SmartExecAlgorithm Exit Orders (`smart_exec_algorithm.py`)
|
|
- TP exit: market order
|
|
- Max hold: limit 10s → market fallback
|
|
- Order rejection handler
|
|
- ✅ 3.3 Fee and Slippage Measurement (`smart_exec_algorithm.py`)
|
|
- **NEW**: Fee tracking (0.02% maker, 0.05% taker)
|
|
- **NEW**: Slippage calculation (actual vs expected price)
|
|
- **NEW**: Metrics collection via `get_metrics()` / `reset_metrics()`
|
|
- ✅ 3.4 Circuit Breakers (`circuit_breaker.py`)
|
|
- **NEW**: CircuitBreakerManager class
|
|
- Daily loss limit (10% hard stop)
|
|
- Max concurrent positions (10)
|
|
- Per-asset position limit (1)
|
|
- API failure tracking (3 consecutive)
|
|
- Order size sanity check (50%)
|
|
- Auto-reset after 24 hours
|
|
- ✅ 3.5 Metrics Monitor (`metrics_monitor.py`)
|
|
- **NEW**: MetricsMonitor class
|
|
- Maker fill rate tracking (1-hour rolling)
|
|
- Slippage tracking (rolling average)
|
|
- Funding rate tracking (24-hour)
|
|
- Threshold alerting (warning/critical)
|
|
- Prometheus export format
|
|
|
|
### Phase 4: Validation 🔄 In Progress
|
|
- ✅ 4.1 JSON Data Adapter (`data_adapter.py`)
|
|
- **NEW**: JSONEigenvalueDataAdapter class
|
|
- Loads eigenvalue JSON files from correlation_arb512
|
|
- Generates synthetic bars from vel_div data
|
|
- Signal metadata extraction
|
|
- BacktestDataLoader for high-level loading
|
|
- ⬜ 4.2 Validation Backtests (pending)
|
|
- ⬜ 4.3 Validation Analysis (pending)
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
```
|
|
nautilus_dolphin/
|
|
├── nautilus_dolphin/ # Main package
|
|
│ ├── __init__.py # Package exports
|
|
│ └── nautilus/ # Nautilus components
|
|
│ ├── __init__.py # Component exports
|
|
│ ├── signal_bridge.py # Redis → Nautilus bridge
|
|
│ ├── strategy.py # Main execution strategy
|
|
│ ├── smart_exec_algorithm.py # Entry/exit execution
|
|
│ ├── position_manager.py # Exit management
|
|
│ ├── volatility_detector.py # Vol regime detection
|
|
│ ├── circuit_breaker.py # Operational safety
|
|
│ ├── metrics_monitor.py # Stress-test metrics
|
|
│ └── data_adapter.py # JSON backtest loader
|
|
├── tests/ # Test suite
|
|
│ ├── test_signal_bridge.py
|
|
│ ├── test_strategy.py
|
|
│ ├── test_position_manager.py
|
|
│ ├── test_volatility_detector.py
|
|
│ ├── test_circuit_breaker.py # NEW
|
|
│ ├── test_metrics_monitor.py # NEW
|
|
│ └── test_smart_exec_algorithm.py # NEW
|
|
├── config/
|
|
│ └── config.yaml
|
|
├── pyproject.toml # Package config
|
|
├── requirements.txt
|
|
└── README.md
|
|
```
|
|
|
|
---
|
|
|
|
## New Components Detail
|
|
|
|
### 1. CircuitBreakerManager (`circuit_breaker.py`)
|
|
|
|
```python
|
|
cb = CircuitBreakerManager(
|
|
daily_loss_limit_pct=10.0,
|
|
max_concurrent_positions=10,
|
|
max_api_failures=3,
|
|
max_order_size_pct=50.0
|
|
)
|
|
|
|
# Check before opening position
|
|
can_trade, reason = cb.can_open_position("BTCUSDT", balance)
|
|
|
|
# Track position lifecycle
|
|
cb.on_position_opened(position_id, asset)
|
|
cb.on_position_closed(position_id, asset, pnl)
|
|
|
|
# Check order size
|
|
can_submit, reason = cb.can_submit_order(notional, balance)
|
|
|
|
# Monitor API health
|
|
cb.on_api_failure(error_message)
|
|
|
|
# Manual control
|
|
cb.manual_trip("Emergency stop")
|
|
cb.reset()
|
|
|
|
# Get status
|
|
status = cb.get_status()
|
|
```
|
|
|
|
### 2. MetricsMonitor (`metrics_monitor.py`)
|
|
|
|
```python
|
|
monitor = MetricsMonitor(
|
|
config=ThresholdConfig(
|
|
min_maker_fill_rate=48.0,
|
|
max_slippage_bps=5.0
|
|
)
|
|
)
|
|
|
|
# Record fills
|
|
monitor.record_fill('maker', slippage_bps=2.5)
|
|
monitor.record_fill('taker', slippage_bps=4.0)
|
|
|
|
# Record trade results
|
|
monitor.record_trade_result(pnl=100.0)
|
|
|
|
# Get metrics
|
|
summary = monitor.get_metrics_summary()
|
|
# Returns: maker_fill_rate_pct, avg_slippage_bps,
|
|
# win_rate_pct, profit_factor, etc.
|
|
|
|
# Prometheus export
|
|
prometheus_metrics = monitor.get_prometheus_metrics()
|
|
```
|
|
|
|
### 3. SmartExecAlgorithm Enhancements
|
|
|
|
**Abort Logic** (Task 3.1.5):
|
|
- Monitors quote ticks for pending entries
|
|
- Cancels order if price moves 5bps against position
|
|
- Tracks aborted entries in metrics
|
|
|
|
**Fee/Slippage Tracking** (Task 3.3):
|
|
- Calculates fees: 0.02% maker, 0.05% taker
|
|
- Tracks slippage: |actual - expected| / expected
|
|
- Provides metrics via `get_metrics()` method
|
|
|
|
### 4. JSONEigenvalueDataAdapter (`data_adapter.py`)
|
|
|
|
```python
|
|
adapter = JSONEigenvalueDataAdapter(
|
|
eigenvalues_dir="path/to/correlation_arb512/eigenvalues",
|
|
venue="BINANCE_FUTURES"
|
|
)
|
|
|
|
# Load date range
|
|
adapter.load_date_range(
|
|
start_date=datetime(2026, 2, 6),
|
|
end_date=datetime(2026, 2, 14)
|
|
)
|
|
|
|
# Iterate through scans
|
|
for bars, signals in adapter:
|
|
# bars: List[Bar] - synthetic OHLCV
|
|
# signals: List[dict] - signal metadata
|
|
process_bars(bars)
|
|
process_signals(signals)
|
|
|
|
# Or use high-level loader
|
|
loader = BacktestDataLoader(eigenvalues_dir, venue)
|
|
data = loader.load_period(start_date, end_date, assets=['BTCUSDT', 'ETHUSDT'])
|
|
```
|
|
|
|
---
|
|
|
|
## Strategy Integration
|
|
|
|
The `DolphinExecutionStrategy` now integrates all components:
|
|
|
|
```python
|
|
class DolphinExecutionStrategy(Strategy):
|
|
def __init__(self, config):
|
|
# Components
|
|
self.volatility_detector = VolatilityRegimeDetector(...)
|
|
self.position_manager = PositionManager(...)
|
|
self.circuit_breaker = CircuitBreakerManager(...)
|
|
self.metrics_monitor = MetricsMonitor(...)
|
|
|
|
def on_signal(self, signal):
|
|
# 1. Apply filters
|
|
rejection = self._should_trade(signal)
|
|
if rejection:
|
|
return
|
|
|
|
# 2. Check circuit breaker
|
|
can_trade, reason = self.circuit_breaker.can_open_position(...)
|
|
if not can_trade:
|
|
return
|
|
|
|
# 3. Execute trade
|
|
self._execute_entry(signal)
|
|
|
|
def _execute_entry(self, signal):
|
|
# Calculate size
|
|
notional = self.calculate_position_size(signal, balance)
|
|
|
|
# Check order sanity
|
|
can_submit, reason = self.circuit_breaker.can_submit_order(...)
|
|
|
|
# Submit via SmartExecAlgorithm
|
|
self.submit_order(order, exec_algorithm_id="SMART_EXEC")
|
|
```
|
|
|
|
---
|
|
|
|
## Test Coverage
|
|
|
|
New tests added:
|
|
|
|
1. **test_circuit_breaker.py** (11 tests)
|
|
- Position limit checks
|
|
- Daily loss limit
|
|
- Order size sanity
|
|
- API failure tracking
|
|
- Manual trip/reset
|
|
- Auto-reset after timeout
|
|
|
|
2. **test_metrics_monitor.py** (13 tests)
|
|
- Fill rate calculation
|
|
- Slippage tracking
|
|
- Trade result recording
|
|
- Win rate / profit factor
|
|
- Alert generation
|
|
- Deduplication
|
|
- Prometheus export
|
|
|
|
3. **test_smart_exec_algorithm.py** (10 tests)
|
|
- Abort logic (price movement)
|
|
- Fee calculation (maker/taker)
|
|
- Slippage calculation
|
|
- Metrics tracking
|
|
- Category classification
|
|
|
|
---
|
|
|
|
## Next Steps
|
|
|
|
To complete Phase 4 (Validation):
|
|
|
|
1. **Set up validation backtests**:
|
|
```python
|
|
# Load data for validation periods
|
|
loader = BacktestDataLoader(eigenvalues_dir)
|
|
|
|
# High volatility: Feb 6-14, 2026
|
|
feb_data = loader.load_period(
|
|
datetime(2026, 2, 6),
|
|
datetime(2026, 2, 14)
|
|
)
|
|
|
|
# Low volatility: Jan 21-28, 2026
|
|
jan_data = loader.load_period(
|
|
datetime(2026, 1, 21),
|
|
datetime(2026, 1, 28)
|
|
)
|
|
```
|
|
|
|
2. **Run Nautilus backtests** using the loaded data
|
|
|
|
3. **Compare results** to VBT baseline:
|
|
- Total trades (±2% tolerance)
|
|
- Win rate (±1% tolerance)
|
|
- Profit factor (±3% tolerance)
|
|
- Final capital (±5% tolerance)
|
|
|
|
---
|
|
|
|
## Configuration Example
|
|
|
|
```yaml
|
|
# config/config.yaml
|
|
strategy:
|
|
venue: "BINANCE_FUTURES"
|
|
|
|
# Filters
|
|
irp_alignment_min: 0.45
|
|
momentum_magnitude_min: 0.000075
|
|
excluded_assets: ["TUSDUSDT", "USDCUSDT"]
|
|
|
|
# Sizing
|
|
min_leverage: 0.5
|
|
max_leverage: 5.0
|
|
leverage_convexity: 3.0
|
|
capital_fraction: 0.20
|
|
|
|
# Exit
|
|
tp_bps: 99
|
|
max_hold_bars: 120
|
|
|
|
# Limits
|
|
max_concurrent_positions: 10
|
|
|
|
circuit_breaker:
|
|
daily_loss_limit_pct: 10.0
|
|
max_api_failures: 3
|
|
max_order_size_pct: 50.0
|
|
auto_reset_hours: 24.0
|
|
|
|
smart_exec:
|
|
entry_timeout_sec: 25
|
|
entry_abort_threshold_bps: 5.0
|
|
exit_timeout_sec: 10
|
|
maker_fee_rate: 0.0002
|
|
taker_fee_rate: 0.0005
|
|
|
|
metrics:
|
|
min_maker_fill_rate: 48.0
|
|
max_slippage_bps: 5.0
|
|
warning_maker_fill_rate: 55.0
|
|
critical_maker_fill_rate: 48.0
|
|
```
|
|
|
|
---
|
|
|
|
## Running Tests
|
|
|
|
```bash
|
|
# Activate environment
|
|
activate_siloqy.bat
|
|
|
|
# Install in dev mode
|
|
pip install -e .
|
|
|
|
# Run all tests
|
|
python -m pytest tests/ -v
|
|
|
|
# Run specific test file
|
|
python -m pytest tests/test_circuit_breaker.py -v
|
|
|
|
# Run with coverage
|
|
python -m pytest tests/ --cov=nautilus_dolphin --cov-report=html
|
|
```
|
|
|
|
---
|
|
|
|
## Notes
|
|
|
|
1. **NautilusTrader dependency**: The implementation requires NautilusTrader 1.200.0+ which should be installed in the siloqy environment.
|
|
|
|
2. **Import errors in test**: Tests will fail to import if nautilus_trader is not installed. This is expected in environments without the trading dependencies.
|
|
|
|
3. **Data adapter**: The JSONEigenvalueDataAdapter generates synthetic prices based on vel_div. For production backtests, consider using actual price data from Binance.
|
|
|
|
4. **Redis**: SignalBridgeActor requires Redis 7.0+ for the Redis Streams functionality.
|