DOLPHIN/prod/docs/BRINGUP_GUIDE.md

# DOLPHIN Paper Trading — Production Bringup Guide

**Purpose**: Step-by-step ops guide for standing up the Prefect + Hazelcast paper trading stack.
**Audience**: Operations agent or junior dev. No research decisions required.
**State as of**: 2026-03-06
**Assumes**: Windows 11, Docker Desktop installed, Siloqy venv exists at `C:\Users\Lenovo\Documents\- Siloqy\`

---

## Architecture Overview

```
[ARB512 Scanner] ─► eigenvalues/YYYY-MM-DD/ ─► [paper_trade_flow.py]
                                                        |
                                           [NDAlphaEngine (Python)]
                                                        |
                                         ┌──────────────┴──────────────┐
                                    [Hazelcast IMap]            [paper_logs/*.jsonl]
                                         |
                                   [Prefect UI :4200]
                                   [HZ-MC UI   :8080]
```

**Components:**
- `docker-compose.yml`: Hazelcast 5.3 (port 5701) + HZ Management Center (port 8080) + Prefect Server (port 4200)
- `paper_trade_flow.py`: Prefect flow, runs daily at 00:05 UTC
- `configs/blue.yml`: Champion SHORT config (frozen, production)
- `configs/green.yml`: Bidirectional config (STATUS: PENDING — LONG validation still in progress)
- Python venv: `C:\Users\Lenovo\Documents\- Siloqy\`

**Data flow**: Prefect triggers daily → reads yesterday's Arrow/NPZ scans from eigenvalues dir → NDAlphaEngine processes → writes P&L to Hazelcast IMap + local JSONL log.

---

## Step 1: Prerequisites Check

Open a terminal (Git Bash or PowerShell).

```bash
# 1a. Verify Docker Desktop is installed
docker --version
# Expected: Docker version 29.x.x

# 1b. Verify Python venv
"/c/Users/Lenovo/Documents/- Siloqy/Scripts/python.exe" --version
# Expected: Python 3.11.x or 3.12.x

# 1c. Verify working directories exist
ls "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod/"
# Expected: configs/  docker-compose.yml  paper_trade_flow.py  BRINGUP_GUIDE.md

ls "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod/configs/"
# Expected: blue.yml  green.yml
```

---

## Step 2: Install Python Dependencies

Run once. Takes ~2-5 minutes.

```bash
"/c/Users/Lenovo/Documents/- Siloqy/Scripts/pip.exe" install \
    hazelcast-python-client \
    prefect \
    pyyaml \
    pyarrow \
    numpy \
    pandas
```

**Verify:**
```bash
"/c/Users/Lenovo/Documents/- Siloqy/Scripts/python.exe" -c "import hazelcast; import prefect; import yaml; print('OK')"
```

---

## Step 3: Start Docker Desktop

Docker Desktop must be running before starting containers.

**Option A (GUI):** Double-click Docker Desktop from Start menu. Wait for the whale icon in the system tray to stop animating (~30-60 seconds).

**Option B (command):**
```powershell
Start-Process "C:\Program Files\Docker\Docker\Docker Desktop.exe"
# Wait ~60 seconds, then verify:
docker ps
```

**Verify Docker is ready:**
```bash
docker info | grep "Server Version"
# Expected: Server Version: 27.x.x
```

---

## Step 4: Start the Infrastructure Stack

```bash
cd "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod"
docker compose up -d
```

**Expected output:**
```
[+] Running 3/3
 - Container dolphin-hazelcast     Started
 - Container dolphin-hazelcast-mc  Started
 - Container dolphin-prefect       Started
```

**Verify all containers healthy:**
```bash
docker compose ps
# All 3 should show "healthy" or "running"
```

**Wait ~30 seconds for Hazelcast to initialize, then verify:**
```bash
curl http://localhost:5701/hazelcast/health/ready
# Expected: {"message":"Hazelcast is ready!"}

curl http://localhost:4200/api/health
# Expected: {"status":"healthy"}
```

**UIs:**
- Prefect UI:        http://localhost:4200
- Hazelcast MC:     http://localhost:8080
  - Default cluster: `dolphin` (auto-connects to hazelcast:5701)

---

## Step 5: Register Prefect Deployments

Run once to register the blue and green scheduled deployments.

```bash
cd "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod"
"/c/Users/Lenovo/Documents/- Siloqy/Scripts/python.exe" paper_trade_flow.py --register
```

**Expected output:**
```
Registered: dolphin-paper-blue
Registered: dolphin-paper-green
```

**Verify in Prefect UI:** http://localhost:4200 → Deployments → should show 2 deployments with CronSchedule "5 0 * * *".

---

## Step 6: Start the Prefect Worker

The Prefect worker polls for scheduled runs. Run in a separate terminal (keep it open, or run as a service).

```bash
"/c/Users/Lenovo/Documents/- Siloqy/Scripts/prefect.exe" worker start --pool "dolphin"
```

**OR** (if `prefect` CLI not in PATH):
```bash
"/c/Users/Lenovo/Documents/- Siloqy/Scripts/python.exe" -m prefect worker start --pool "dolphin"
```

Leave this terminal running. It will pick up the 00:05 UTC scheduled runs.

---

## Step 7: Manual Test Run

Before relying on the schedule, test with a known good date (a date that has scan data).

```bash
cd "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod"
"/c/Users/Lenovo/Documents/- Siloqy/Scripts/python.exe" paper_trade_flow.py \
    --date 2026-03-05 \
    --config configs/blue.yml
```

**Expected output (abbreviated):**
```
=== BLUE paper trade: 2026-03-05 ===
  Loaded N scans for 2026-03-05 | cols=XX
  2026-03-05: PnL=+XX.XX  T=X  boost=1.XXx  MC=OK
  HZ write OK → DOLPHIN_PNL_BLUE[2026-03-05]
=== DONE: blue 2026-03-05 | PnL=+XX.XX | Capital=25,XXX.XX ===
```

**Verify data written to Hazelcast:**
- Open http://localhost:8080 → Maps → DOLPHIN_PNL_BLUE → should contain entry for 2026-03-05

**Verify log file written:**
```bash
ls "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod/paper_logs/blue/"
cat "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod/paper_logs/blue/paper_pnl_2026-03.jsonl"
```

---

## Step 8: Scan Data Source Verification

The flow reads scan files from:
```
C:\Users\Lenovo\Documents\- Dolphin NG HD (NG3)\correlation_arb512\eigenvalues\YYYY-MM-DD\
```

Each date directory should contain `scan_*__Indicators.npz` or `scan_*.arrow` files.

```bash
ls "/c/Users/Lenovo/Documents/- Dolphin NG HD (NG3)/correlation_arb512/eigenvalues/" | tail -5
# Expected: recent date directories like 2026-03-05, 2026-03-04, etc.

ls "/c/Users/Lenovo/Documents/- Dolphin NG HD (NG3)/correlation_arb512/eigenvalues/2026-03-05/"
# Expected: scan_NNNN__Indicators.npz files
```

If a date directory is missing, the flow logs a warning and writes pnl=0 for that day (non-critical).

---

## Step 9: Daily Operations

**Normal daily flow (automated):**
1. ARB512 scanner (extended_main.py) writes scans to eigenvalues/YYYY-MM-DD/ throughout the day
2. At 00:05 UTC, Prefect triggers dolphin-paper-blue and dolphin-paper-green
3. Each flow reads yesterday's scans, runs the engine, writes to HZ + JSONL log
4. Monitor via Prefect UI and HZ-MC

**Check today's run result:**
```bash
# Latest P&L log entry:
tail -1 "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod/paper_logs/blue/paper_pnl_$(date +%Y-%m).jsonl"
```

**Check HZ state:**
- http://localhost:8080 → Maps → DOLPHIN_STATE_BLUE → key "latest"
- Should show: `{"capital": XXXXX, "strategy": "blue", "last_date": "YYYY-MM-DD", ...}`

---

## Step 10: Restart After Reboot

After Windows restarts:

```bash
# 1. Start Docker Desktop (GUI or command — see Step 3)

# 2. Restart containers
cd "/c/Users/Lenovo/Documents/- DOLPHIN NG HD HCM TSF Predict/prod"
docker compose up -d

# 3. Restart Prefect worker (in a dedicated terminal)
"/c/Users/Lenovo/Documents/- Siloqy/Scripts/python.exe" -m prefect worker start --pool "dolphin"
```

Deployments and HZ data persist (docker volumes: hz_data, prefect_data).

---

## Troubleshooting

### "No scan dir for YYYY-MM-DD"
- The ARB512 scanner may not have run for that date
- Check: `ls "C:\Users\Lenovo\Documents\- Dolphin NG HD (NG3)\correlation_arb512\eigenvalues\YYYY-MM-DD\"`
- Non-critical: flow logs pnl=0 and continues

### "HZ write failed (not critical)"
- Hazelcast container not running or not yet healthy
- Run: `docker compose ps` → check dolphin-hazelcast shows "healthy"
- Run: `docker compose restart hazelcast`

### "ModuleNotFoundError: No module named 'hazelcast'"
- Dependencies not installed in Siloqy venv
- Rerun Step 2

### "error during connect: open //./pipe/dockerDesktopLinuxEngine"
- Docker Desktop not running
- Start Docker Desktop (see Step 3), wait 60 seconds, retry

### Prefect worker not picking up runs
- Verify worker is running with `--pool "dolphin"` (matches work_queue_name in deployments)
- Check Prefect UI → Work Pools → should show "dolphin" pool as online

### Green deployment errors on bidirectional config
- Green is PENDING LONG validation. If direction: bidirectional causes engine errors,
  temporarily set green.yml direction: short_only until LONG system is validated.

---

## Key File Locations

| File | Path |
|---|---|
| Prefect flow | `prod/paper_trade_flow.py` |
| Blue config | `prod/configs/blue.yml` |
| Green config | `prod/configs/green.yml` |
| Docker stack | `prod/docker-compose.yml` |
| Blue P&L logs | `prod/paper_logs/blue/paper_pnl_YYYY-MM.jsonl` |
| Green P&L logs | `prod/paper_logs/green/paper_pnl_YYYY-MM.jsonl` |
| Scan data source | `C:\Users\Lenovo\Documents\- Dolphin NG HD (NG3)\correlation_arb512\eigenvalues\` |
| NDAlphaEngine | `HCM\nautilus_dolphin\nautilus_dolphin\nautilus\esf_alpha_orchestrator.py` |
| MC-Forewarner models | `HCM\nautilus_dolphin\mc_results\models\` |

---

## Current Status (2026-03-06)

| Item | Status |
|---|---|
| Docker stack | Built — needs Docker Desktop running |
| Python deps (HZ + Prefect) | Installing (pip background job) |
| Blue config | Frozen champion SHORT — ready |
| Green config | PENDING — LONG validation running (b79rt78uv) |
| Prefect deployments | Not yet registered (run Step 5 after deps install) |
| Manual test run | Not yet done (run Step 7) |
| vol_p60 calibration | Hardcoded 0.000099 (pre-calibrated from 55-day window) — acceptable |
| Engine state persistence | Implemented — engine capital and open positions serialize to Hazelcast STATE IMap |

### Engine State Persistence

The NDAlphaEngine is instantiated fresh during each daily Prefect run, but its internal state is loaded from the Hazelcast `DOLPHIN_STATE_BLUE`/`GREEN` maps. Both `capital` and any active `position` spanning midnight are accurately tracked and restored.

**Impact for paper trading**: P&L and cumulative capital growth track correctly across days.

---

*Guide written 2026-03-08. Status updated.*