DOLPHIN/prod/clean_arch/README.md

DOLPHIN Clean Architecture - Paper Trading

✅ Status: OPERATIONAL

The clean hexagonal architecture paper trading system is now running with live data from Hazelcast.

---

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                    PAPER TRADING SYSTEM                         │
├─────────────────────────────────────────────────────────────────┤
│  PORTS (Interfaces)                                             │
│  ├── DataFeedPort          - Abstract data feed interface        │
│  └── TradingPort           - Abstract trading interface          │
├─────────────────────────────────────────────────────────────────┤
│  ADAPTERS (Implementations)                                     │
│  ├── HazelcastDataFeed     - Reads from DolphinNG6 via Hz       │
│  └── PaperTradingExecutor  - Simulated order execution          │
├─────────────────────────────────────────────────────────────────┤
│  CORE (Business Logic)                                          │
│  ├── TradingEngine         - Position sizing, risk management   │
│  ├── SignalProcessor       - Eigenvalue-based signals           │
│  └── PortfolioManager      - Position tracking, PnL             │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│  INFRASTRUCTURE                                                 │
│  ├── Hazelcast Cluster     - Single source of truth             │
│  ├── Scan Bridge Service   - Arrow → Hazelcast bridge           │
│  └── Arrow Files           - DolphinNG6 output                  │
└─────────────────────────────────────────────────────────────────┘

---

Key Design Decisions

1. Single Source of Truth (Hazelcast)

• Problem: Price and eigenvalue data need to be perfectly synchronized
• Solution: DolphinNG6 writes both to Hazelcast atomically
• Benefit: No sync issues, consistent data for trading decisions

2. File Timestamp vs Scan Number

• Problem: DolphinNG6 resets scan counters on restarts
• Solution: Bridge uses file modification time (mtime) not scan_number
• Benefit: Always gets latest data even after NG6 restarts

3. Hexagonal Architecture

• Benefit: Core logic is adapter-agnostic
• Future: Can swap Hazelcast adapter for direct Binance WebSocket
• Testing: Easy to mock data feeds for unit tests

---

Components

DataFeedPort (ports/data_feed.py)

Abstract interface for market data:

class DataFeedPort(ABC):
    @abstractmethod
    async def get_latest_snapshot(self, symbol: str) -> MarketSnapshot:
        ...

HazelcastDataFeed (adapters/hazelcast_feed.py)

Implementation reading from Hazelcast:

• Connects to DOLPHIN_FEATURES map
• Reads latest_eigen_scan key
• Returns MarketSnapshot with price + eigenvalues

TradingEngine (core/trading_engine.py)

Pure business logic:

• Position sizing based on eigenvalues
• Risk management
• ACB (Adaptive Circuit Breaker) integration

---

Data Flow

┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│ DolphinNG6  │────▶│ Arrow Files │────▶│ Scan Bridge │────▶│ Hazelcast   │
│ (Trading)   │     │ (Storage)   │     │ (Service)   │     │ (SSOT)      │
└─────────────┘     └─────────────┘     └─────────────┘     └──────┬──────┘
                                                                    │
                                                                    ▼
┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Binance   │◀────│   Nautilus  │◀────│   Trading   │◀────│ Hazelcast   │
│   Futures   │     │   Trader    │     │   Engine    │     │ DataFeed    │
│  (Paper)    │     │  (Adapter)  │     │   (Core)    │     │  (Adapter)  │
└─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘

---

Running the System

1. Start Hazelcast (if not running)

cd /mnt/dolphinng5_predict
docker-compose up -d hazelcast

2. Start Scan Bridge Service

cd /mnt/dolphinng5_predict/prod
source /home/dolphin/siloqy_env/bin/activate
python3 scan_bridge_service.py

3. Check Status

cd /mnt/dolphinng5_predict/prod/clean_arch
python3 status.py

4. Test Adapter

python3 -c "
import asyncio
import sys
sys.path.insert(0, '/mnt/dolphinng5_predict/prod/clean_arch')
from adapters.hazelcast_feed import HazelcastDataFeed

async def test():
    feed = HazelcastDataFeed({'hazelcast': {'cluster': 'dolphin', 'host': 'localhost:5701'}})
    await feed.connect()
    snap = await feed.get_latest_snapshot('BTCUSDT')
    print(f'BTC: \${snap.price:,.2f} | Eigenvalues: {len(snap.eigenvalues)}')
    await feed.disconnect()

asyncio.run(test())
"

---

Current Status

Component	Status	Notes
Hazelcast Cluster	✅ Running	localhost:5701
Scan Bridge	⚠️ Manual start	Run: python3 scan_bridge_service.py
Arrow Files	✅ Present	~6500 files, latest #7320+
Hazelcast Data	✅ Valid	50 assets, 50 prices
DataFeed Adapter	✅ Working	BTC @ $71,281
Trading Engine	🔄 Ready	Core logic implemented
Nautilus Trader	🔄 Ready	Integration pending

---

Evolution Path

Phase 1: Hazelcast Feed (CURRENT)

• Uses DolphinNG6 eigenvalue calculations
• Single source of truth via Hazelcast
• Bridge service watches Arrow files

Phase 2: Direct Binance Feed (NEXT)

• Replace Hazelcast adapter with Binance WebSocket
• Compute eigenvalues locally
• Lower latency

Phase 3: Rust Kernel (FUTURE)

• Port core trading logic to Rust
• Python adapter layer only
• Maximum performance

---

Troubleshooting

No data in Hazelcast

# Check if bridge is running
ps aux | grep scan_bridge

# Check latest Arrow files
ls -lt /mnt/ng6_data/arrow_scans/$(date +%Y-%m-%d)/ | head -5

# Manually push latest
python3 << 'EOF'
# (see scan_bridge_service.py for manual push code)
EOF

Hazelcast connection refused

# Check if Hazelcast is running
docker ps | grep hazelcast

# Check logs
docker logs dolphin-hazelcast

Scan number mismatch

• This is normal - DolphinNG6 resets counters
• Bridge uses file timestamps, not scan numbers
• Always gets latest data

---

File Locations

File	Purpose
prod/clean_arch/ports/data_feed.py	Abstract interfaces (PORTS)
prod/clean_arch/adapters/hazelcast_feed.py	Hazelcast adapter
prod/clean_arch/core/trading_engine.py	Business logic
prod/scan_bridge_service.py	Arrow → Hazelcast bridge
prod/clean_arch/status.py	Status check

---

Summary

✅ Clean architecture implemented
✅ Hazelcast data feed working
✅ Live market data flowing
✅ Ready for trading logic integration

Next step: Connect TradingEngine to Nautilus Trader for paper trading execution.