siloqy/SYSTEM_BIBLE.md

DOLPHIN Trading System Bible

Version: 1.0.0
Last Updated: 2026-03-24
Status: Production Ready - Paper Trading Operational

---

1. System Overview

The DOLPHIN Trading System is a production-grade algorithmic trading platform implementing clean hexagonal architecture for high-frequency eigenvalue-based trading strategies.

1.1 Current Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                         DOLPHIN TRADING SYSTEM                          │
│                              Version 1.0.0                               │
├─────────────────────────────────────────────────────────────────────────┤
│  PRESENTATION LAYER                                                      │
│  ├── Paper Trading CLI (paper_trade.py)                                 │
│  ├── System Monitor (status.py)                                         │
│  └── Trading Dashboard (future)                                         │
├─────────────────────────────────────────────────────────────────────────┤
│  CORE BUSINESS LOGIC (PORTS)                                             │
│  ├── DataFeedPort          - Abstract market data interface             │
│  ├── TradingPort           - Abstract execution interface               │
│  └── StrategyPort          - Abstract strategy interface                │
├─────────────────────────────────────────────────────────────────────────┤
│  ADAPTER LAYER                                                           │
│  ├── HazelcastDataFeed     - Live data from DolphinNG6                  │
│  ├── PaperTradeExecutor    - Simulated order execution                  │
│  └── (Future: BinanceDataFeed, LiveExecutor)                            │
├─────────────────────────────────────────────────────────────────────────┤
│  DOMAIN LOGIC                                                            │
│  ├── TradingEngine         - Position sizing, risk management           │
│  ├── SignalProcessor       - Eigenvalue signal generation               │
│  ├── PortfolioManager      - Position tracking, PnL calculation         │
│  └── ACBIntegration        - Adaptive Circuit Breaker                   │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  INFRASTRUCTURE                                                          │
│  ├── Hazelcast Cluster     - Single Source of Truth (localhost:5701)    │
│  ├── Scan Bridge Service   - Arrow file watcher → Hazelcast             │
│  ├── Arrow File Storage    - /mnt/ng6_data/arrow_scans/                 │
│  └── Nautilus Trader       - Execution framework (future integration)   │
└─────────────────────────────────────────────────────────────────────────┘

1.2 Key Metrics

Metric	Value
Architecture Pattern	Hexagonal (Clean Architecture)
Data Latency	~5 seconds (DolphinNG6 pulse)
Assets Tracked	50 (BTC, ETH, BNB, etc.)
Eigenvalue Windows	50, 150, 300, 750 periods
Current BTC Price	$71,281.03
Paper Trade Volume	0.001 BTC per signal

---

2. Version History

v1.0.0 (2026-03-24) - CURRENT

Codename: Clean Slate

Major Features

• ✅ Clean hexagonal architecture implementation
• ✅ Hazelcast DataFeed adapter (live data from DolphinNG6)
• ✅ Scan Bridge Service (Arrow → Hazelcast bridge)
• ✅ Paper trading engine with PnL tracking
• ✅ Mean reversion strategy on velocity divergence
• ✅ 23 round-trip trades executed in testing

Components Added

• prod/clean_arch/ports/data_feed.py - Port interfaces
• prod/clean_arch/adapters/hazelcast_feed.py - Hazelcast adapter
• prod/clean_arch/core/trading_engine.py - Business logic
• prod/scan_bridge_service.py - Infrastructure bridge
• prod/clean_arch/paper_trade.py - Trading CLI

Infrastructure

• Hazelcast cluster at localhost:5701
• Scan bridge watching /mnt/ng6_data/arrow_scans/
• 6,500+ Arrow scan files processed

---

3. Architecture Principles

3.1 Hexagonal Architecture (Ports & Adapters)

The system follows strict dependency rules:

Core (inner) → Adapters (outer)
     ↑
   Ports (interfaces)

Rule: Dependencies only point inward. Core knows nothing about Hazelcast, Arrow files, or Binance.

3.2 Single Source of Truth

All market data comes from Hazelcast DOLPHIN_FEATURES map:

• Price + eigenvalues written atomically by DolphinNG6
• No synchronization issues
• Guaranteed consistent timestamps

3.3 File Timestamp vs Scan Number

The Scan Bridge uses file modification time (mtime) instead of scan numbers because:

• DolphinNG6 resets scan counters on restarts
• mtime always correctly identifies the latest file
• Handles NG6 crashes and restarts gracefully

---

4. Subsystems

4.1 Data Flow Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  DolphinNG6 │────▶│ Arrow Files │────▶│ Scan Bridge │────▶│  Hazelcast  │
│  (Trading)  │     │  (Storage)  │     │  (Service)  │     │   (SSOT)    │
└─────────────┘     └─────────────┘     └─────────────┘     └──────┬──────┘
     5s pulse                                                          │
                                                                      ▼
┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Binance   │◀────│  Nautilus   │◀────│   Trading   │◀────│ Hazelcast   │
│  (Paper)    │     │   Trader    │     │   Engine    │     │  DataFeed   │
│             │     │  (Future)   │     │  (Core)     │     │ (Adapter)   │
└─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘

4.2 Component Details

Scan Bridge Service

File: prod/scan_bridge_service.py Purpose: Watch Arrow files and push to Hazelcast Key Features:

• Uses watchdog for file system events
• Parses JSON columns (assets_json, asset_prices_json)
• Handles numpy types in serialization
• Rolls to new day directory automatically

Startup:

cd /mnt/dolphinng5_predict/prod
source /home/dolphin/siloqy_env/bin/activate
python3 scan_bridge_service.py

Hazelcast DataFeed Adapter

File: prod/clean_arch/adapters/hazelcast_feed.py Purpose: Implement DataFeedPort using Hazelcast Key Methods:

• connect() - Connect to Hazelcast cluster
• get_latest_snapshot(symbol) - Get MarketSnapshot
• subscribe_snapshots(callback) - Subscribe to updates

Data Structure:

MarketSnapshot(
    timestamp=datetime,
    symbol="BTCUSDT",
    price=71281.03,
    eigenvalues=[...],  # 50 loadings
    velocity_divergence=-0.0058,
    scan_number=7315
)

Trading Engine

File: prod/clean_arch/core/trading_engine.py Purpose: Pure business logic, no external dependencies Responsibilities:

• Position sizing
• Signal processing
• Risk management
• PnL calculation

---

5. Trading Strategy

5.1 Current Strategy: Velocity Divergence Mean Reversion

Logic:

if velocity_divergence < BUY_THRESHOLD:  # -0.01
    BUY 0.001 BTC
elif velocity_divergence > SELL_THRESHOLD:  # 0.01
    SELL position

Rationale: Extreme negative velocity divergence suggests oversold conditions.

5.2 Signal Sources

Signal	Source	Description
velocity_divergence	DolphinNG6	Rate of change divergence
instability_composite	DolphinNG6	Market instability metric
w50_lambda_max	DolphinNG6	Max eigenvalue (50-period)
asset_loadings	DolphinNG6	Eigenvector components

---

6. Operations Guide

6.1 Starting the System

# 1. Ensure Hazelcast is running
docker ps | grep hazelcast

# 2. Start Scan Bridge
cd /mnt/dolphinng5_predict/prod
source /home/dolphin/siloqy_env/bin/activate
python3 scan_bridge_service.py &

# 3. Verify data flow
python3 clean_arch/status.py

# 4. Run paper trading
python3 clean_arch/paper_trade.py --duration 60

6.2 Monitoring

Check Status:

cd /mnt/dolphinng5_predict/prod/clean_arch
python3 status.py

Expected Output:

🐬 DOLPHIN PAPER TRADING STATUS
⚡ HAZELCAST: CONNECTED
   Scan: #7315
   Assets: 50
   Prices: 50
   BTC Price: $71,281.03

6.3 Troubleshooting

Issue	Solution
No Hazelcast connection	Check docker ps, restart container
No data in Hz	Restart scan bridge, check Arrow files
Scan number mismatch	Normal - bridge uses mtime, not scan #
Zero eigenvalues	Check asset_loadings field exists

---

7. Evolution Roadmap

Phase 1: Hazelcast Feed (COMPLETE - v1.0.0)

• ✅ Clean architecture
• ✅ Hazelcast adapter
• ✅ Paper trading
• ✅ Scan bridge service

Phase 2: Direct Market Feed (v1.1.0)

• Binance WebSocket adapter
• Local eigenvalue computation
• Reduced latency (<100ms)

Phase 3: Rust Kernel (v2.0.0)

• Port core to Rust
• Python adapter layer only
• Microsecond latency

Phase 4: Live Trading (v3.0.0)

• Live order execution
• Risk management integration
• Production deployment

---

8. File Reference

Path	Purpose	Version Added
prod/clean_arch/ports/data_feed.py	Port interfaces	v1.0.0
prod/clean_arch/adapters/hazelcast_feed.py	Hz adapter	v1.0.0
prod/clean_arch/core/trading_engine.py	Business logic	v1.0.0
prod/scan_bridge_service.py	Arrow bridge	v1.0.0
prod/clean_arch/paper_trade.py	Trading CLI	v1.0.0
prod/clean_arch/status.py	Status monitor	v1.0.0

---

9. Environment

Python Environment: /home/dolphin/siloqy_env Python Version: 3.12.12 Key Dependencies:

• nautilus_trader 1.219.0
• hazelcast-python-client 5.6.0
• pyarrow
• watchdog

Data Locations:

• Arrow files: /mnt/ng6_data/arrow_scans/YYYY-MM-DD/
• Hazelcast: localhost:5701
• Logs: /tmp/scan_bridge.log

---

10. Team & Ownership

System Architect: Clean Architecture Pattern
Data Source: DolphinNG6 (Eigenvalue Engine)
Infrastructure: Hazelcast Cluster
Trading Engine: Hexagonal Core

---

This document is the single source of truth for the DOLPHIN Trading System architecture and operations. All changes must be documented with version updates.

Document Version: 1.0.0
Last Modified: 2026-03-24
Next Review: On Phase 2 completion

---

11. DITAv2 / PINK — Accounting Architecture & Known Fixes

11.1 Fee Sign Convention (CRITICAL — 2026-06-08)

The Rust K-account uses POSITIVE = cost, NEGATIVE = rebate for all fee buckets. BingX VST uses the opposite convention: the "n" (commission) field is NEGATIVE for costs, POSITIVE for rebates.

The sign translation is applied at bingx_user_stream.py:_normalise_order():

raw_fee = _safe_float(o.get("n") or 0.0)
fee = -raw_fee   # BingX cost (neg) → kernel cost (pos)

Never apply this translation twice. The kernel's apply_fill_settled / apply_predicted_fill always expect kernel convention (positive = cost). If a second boundary layer exists, it must pass fees already in kernel convention.

11.2 Opening-Fee Settlement (Defect B — fixed 2026-06-08)

BingX ENTER fills report filled quantity only in field "z" (cumFilledQty), not "l" (lastFilledQty). The _normalise_order fallback:

_last_qty = _safe_float(o.get("l") or 0.0)
_cum_qty  = _safe_float(o.get("z") or o.get("cumFilledQty") or 0.0)
fill_qty  = _last_qty if _last_qty > 0.0 else _cum_qty

Ensures apply_predicted_fill receives non-zero fill_qty for ENTER fills, so the opening-leg fee prediction lands in k_taker_fees.

11.3 Reconcile / Capital Freeze Logic

The Rust kernel classifies K-vs-E divergence:

reconcile_status	delta range	capital_frozen
OK	< 0.01 USDT	False
WARN	0.01–20 USDT	False (unsettled but tradeable)
ERROR	≥ 20 USDT	True (unexplained gap, HALT ENTERs)

WARN does NOT freeze. During a live trade, the ENTER predicted fee temporarily pushes delta into ERROR range (seed balance not yet updated by exchange). The EXIT's realized PnL brings delta back into WARN (< 20), then the ACCOUNT_UPDATE event brings it to OK. Freezing on WARN would permanently block all trading after the first trade.

Only ERROR (≥ 20 USDT, unexplained) warrants a freeze.

11.4 Acceptance Criterion

After every round-trip trade: |k_capital − bingx_wallet_balance| < 1.0 USDT and capital_frozen = False.

Verified numerically against BingX ledger in test_fee_sign_accounting.py (34 tests, all green as of 2026-06-08).

11.5 Test File

prod/clean_arch/dita_v2/test_fee_sign_accounting.py — 34 tests covering:

• Prove-Broken: characterises the pre-fix buggy behaviour
• Prove-Fixed: T-2 and T-1 ground-truth round-trips
• Boundary layer: BingX _normalise_order sign + fill_qty
• Edge cases: rebates, partials, multiple fills