DOLPHIN/prod/clean_arch/README.md

# DOLPHIN Clean Architecture - Paper Trading

## ✅ Status: OPERATIONAL

The clean hexagonal architecture paper trading system is now running with live data from Hazelcast.

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────┐
│                    PAPER TRADING SYSTEM                         │
├─────────────────────────────────────────────────────────────────┤
│  PORTS (Interfaces)                                             │
│  ├── DataFeedPort          - Abstract data feed interface        │
│  └── TradingPort           - Abstract trading interface          │
├─────────────────────────────────────────────────────────────────┤
│  ADAPTERS (Implementations)                                     │
│  ├── HazelcastDataFeed     - Reads from DolphinNG6 via Hz       │
│  └── PaperTradingExecutor  - Simulated order execution          │
├─────────────────────────────────────────────────────────────────┤
│  CORE (Business Logic)                                          │
│  ├── TradingEngine         - Position sizing, risk management   │
│  ├── SignalProcessor       - Eigenvalue-based signals           │
│  └── PortfolioManager      - Position tracking, PnL             │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  INFRASTRUCTURE                                                 │
│  ├── Hazelcast Cluster     - Single source of truth             │
│  ├── Scan Bridge Service   - Arrow → Hazelcast bridge           │
│  └── Arrow Files           - DolphinNG6 output                  │
└─────────────────────────────────────────────────────────────────┘
```

---

## Key Design Decisions

### 1. Single Source of Truth (Hazelcast)
- **Problem**: Price and eigenvalue data need to be perfectly synchronized
- **Solution**: DolphinNG6 writes both to Hazelcast atomically
- **Benefit**: No sync issues, consistent data for trading decisions

### 2. File Timestamp vs Scan Number
- **Problem**: DolphinNG6 resets scan counters on restarts
- **Solution**: Bridge uses file modification time (mtime) not scan_number
- **Benefit**: Always gets latest data even after NG6 restarts

### 3. Hexagonal Architecture
- **Benefit**: Core logic is adapter-agnostic
- **Future**: Can swap Hazelcast adapter for direct Binance WebSocket
- **Testing**: Easy to mock data feeds for unit tests

---

## Components

### DataFeedPort (`ports/data_feed.py`)
Abstract interface for market data:
```python
class DataFeedPort(ABC):
    @abstractmethod
    async def get_latest_snapshot(self, symbol: str) -> MarketSnapshot:
        ...
```

### HazelcastDataFeed (`adapters/hazelcast_feed.py`)
Implementation reading from Hazelcast:
- Connects to `DOLPHIN_FEATURES` map
- Reads `latest_eigen_scan` key
- Returns `MarketSnapshot` with price + eigenvalues

### TradingEngine (`core/trading_engine.py`)
Pure business logic:
- Position sizing based on eigenvalues
- Risk management
- ACB (Adaptive Circuit Breaker) integration

---

## Data Flow

```
┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│ DolphinNG6  │────▶│ Arrow Files │────▶│ Scan Bridge │────▶│ Hazelcast   │
│ (Trading)   │     │ (Storage)   │     │ (Service)   │     │ (SSOT)      │
└─────────────┘     └─────────────┘     └─────────────┘     └──────┬──────┘
                                                                    │
                                                                    ▼
┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Binance   │◀────│   Nautilus  │◀────│   Trading   │◀────│ Hazelcast   │
│   Futures   │     │   Trader    │     │   Engine    │     │ DataFeed    │
│  (Paper)    │     │  (Adapter)  │     │   (Core)    │     │  (Adapter)  │
└─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘
```

---

## Running the System

### 1. Start Hazelcast (if not running)
```bash
cd /mnt/dolphinng5_predict
docker-compose up -d hazelcast
```

### 2. Start Scan Bridge Service
```bash
cd /mnt/dolphinng5_predict/prod
source /home/dolphin/siloqy_env/bin/activate
python3 scan_bridge_service.py
```

### 3. Check Status
```bash
cd /mnt/dolphinng5_predict/prod/clean_arch
python3 status.py
```

### 4. Test Adapter
```bash
python3 -c "
import asyncio
import sys
sys.path.insert(0, '/mnt/dolphinng5_predict/prod/clean_arch')
from adapters.hazelcast_feed import HazelcastDataFeed

async def test():
    feed = HazelcastDataFeed({'hazelcast': {'cluster': 'dolphin', 'host': 'localhost:5701'}})
    await feed.connect()
    snap = await feed.get_latest_snapshot('BTCUSDT')
    print(f'BTC: \${snap.price:,.2f} | Eigenvalues: {len(snap.eigenvalues)}')
    await feed.disconnect()

asyncio.run(test())
"
```

---

## Current Status

| Component | Status | Notes |
|-----------|--------|-------|
| Hazelcast Cluster | ✅ Running | localhost:5701 |
| Scan Bridge | ⚠️ Manual start | Run: `python3 scan_bridge_service.py` |
| Arrow Files | ✅ Present | ~6500 files, latest #7320+ |
| Hazelcast Data | ✅ Valid | 50 assets, 50 prices |
| DataFeed Adapter | ✅ Working | BTC @ $71,281 |
| Trading Engine | 🔄 Ready | Core logic implemented |
| Nautilus Trader | 🔄 Ready | Integration pending |

---

## Evolution Path

### Phase 1: Hazelcast Feed (CURRENT)
- Uses DolphinNG6 eigenvalue calculations
- Single source of truth via Hazelcast
- Bridge service watches Arrow files

### Phase 2: Direct Binance Feed (NEXT)
- Replace Hazelcast adapter with Binance WebSocket
- Compute eigenvalues locally
- Lower latency

### Phase 3: Rust Kernel (FUTURE)
- Port core trading logic to Rust
- Python adapter layer only
- Maximum performance

---

## Troubleshooting

### No data in Hazelcast
```bash
# Check if bridge is running
ps aux | grep scan_bridge

# Check latest Arrow files
ls -lt /mnt/ng6_data/arrow_scans/$(date +%Y-%m-%d)/ | head -5

# Manually push latest
python3 << 'EOF'
# (see scan_bridge_service.py for manual push code)
EOF
```

### Hazelcast connection refused
```bash
# Check if Hazelcast is running
docker ps | grep hazelcast

# Check logs
docker logs dolphin-hazelcast
```

### Scan number mismatch
- This is normal - DolphinNG6 resets counters
- Bridge uses file timestamps, not scan numbers
- Always gets latest data

---

## File Locations

| File | Purpose |
|------|---------|
| `prod/clean_arch/ports/data_feed.py` | Abstract interfaces (PORTS) |
| `prod/clean_arch/adapters/hazelcast_feed.py` | Hazelcast adapter |
| `prod/clean_arch/core/trading_engine.py` | Business logic |
| `prod/scan_bridge_service.py` | Arrow → Hazelcast bridge |
| `prod/clean_arch/status.py` | Status check |

---

## Summary

✅ **Clean architecture implemented**  
✅ **Hazelcast data feed working**  
✅ **Live market data flowing**  
✅ **Ready for trading logic integration**  

Next step: Connect TradingEngine to Nautilus Trader for paper trading execution.