initial: import DOLPHIN baseline 2026-04-21 from dolphinng5_predict working tree
Includes core prod + GREEN/BLUE subsystems: - prod/ (BLUE harness, configs, scripts, docs) - nautilus_dolphin/ (GREEN Nautilus-native impl + dvae/ preserved) - adaptive_exit/ (AEM engine + models/bucket_assignments.pkl) - Observability/ (EsoF advisor, TUI, dashboards) - external_factors/ (EsoF producer) - mc_forewarning_qlabs_fork/ (MC regime/envelope) Excludes runtime caches, logs, backups, and reproducible artifacts per .gitignore.
This commit is contained in:
hjnormey
135
prod/docs/SYSTEM_BIBLE_v3.md
Executable file
135
prod/docs/SYSTEM_BIBLE_v3.md
Executable file
@@ -0,0 +1,135 @@
|
||||
# DOLPHIN-NAUTILUS SYSTEM BIBLE
|
||||
## Doctrinal Reference — As Running 2026-03-23
|
||||
|
||||
**Version**: v3 — Meta-System Monitoring (MHD) Integration
|
||||
**Previous version**: `SYSTEM_BIBLE.md` (v2, forked 2026-03-23)
|
||||
**CI gate (Nautilus)**: 46/46 tests green (11 bootstrap + 35 DolphinActor)
|
||||
**CI gate (OBF)**: ~120 unit tests green
|
||||
**MHD Health Status**: GREEN (MHD integrated and operational)
|
||||
**Status**: Paper trading ready. NOT deployed with real capital.
|
||||
|
||||
### What changed since v2 (2026-03-22)
|
||||
|
||||
| Area | Change |
|
||||
|---|---|
|
||||
| **Meta-System** | **MetaHealthDaemon (MHD)** implemented — standalone watchdog monitoring liveness, freshness, and coherence. |
|
||||
| **Orchestration** | MHD auto-restart logic for infrastructure (HZ/Prefect) added. |
|
||||
| **Cross-Platform** | Native support for both Linux (systemd) and FreeBSD (rc.d) service management. |
|
||||
| **Observability** | `meta_health.json` + `DOLPHIN_META_HEALTH` HZ map for L2 health state tracking. |
|
||||
|
||||
---
|
||||
|
||||
## TABLE OF CONTENTS
|
||||
|
||||
1. [System Philosophy](#1-system-philosophy)
|
||||
2. [Physical Architecture](#2-physical-architecture)
|
||||
3. [Data Layer](#3-data-layer)
|
||||
4. [Signal Layer — vel_div & DC](#4-signal-layer)
|
||||
5. [Asset Selection — IRP](#5-asset-selection-irp)
|
||||
6. [Position Sizing — AlphaBetSizer](#6-position-sizing)
|
||||
7. [Exit Management](#7-exit-management)
|
||||
8. [Fee & Slippage Model](#8-fee--slippage-model)
|
||||
9. [OB Intelligence Layer](#9-ob-intelligence-layer)
|
||||
10. [ACB v6 — Adaptive Circuit Breaker](#10-acb-v6)
|
||||
11. [Survival Stack — Posture Control](#11-survival-stack)
|
||||
12. [MC-Forewarner Envelope Gate](#12-mc-forewarner-envelope-gate)
|
||||
13. [NDAlphaEngine — Full Bar Loop](#13-ndalpha-engine-full-bar-loop)
|
||||
14. [DolphinActor — Nautilus Integration](#14-dolphin-actor)
|
||||
15. [Hazelcast — Full IMap Schema](#15-hazelcast-full-imap-schema)
|
||||
16. [Production Daemon Topology](#16-production-daemon-topology)
|
||||
17. [Prefect Orchestration Layer](#17-prefect-orchestration-layer)
|
||||
18. [CI Test Suite](#18-ci-test-suite)
|
||||
19. [Parameter Reference](#19-parameter-reference)
|
||||
20. [OBF Sprint 1 Hardening](#20-obf-sprint-1-hardening)
|
||||
21. [Known Research TODOs](#21-known-research-todos)
|
||||
22. [0.1s Resolution — Readiness Assessment](#22-01s-resolution-readiness-assessment)
|
||||
23. [MetaHealthDaemon (MHD) — Meta-System Monitoring](#23-meta-health-daemon-mhd)
|
||||
|
||||
---
|
||||
|
||||
*(Sections 1-22 remain unchanged as per v2 specification. See v2 for details.)*
|
||||
|
||||
---
|
||||
|
||||
## 23. MetaHealthDaemon (MHD) — Meta-System Monitoring
|
||||
|
||||
### 23.1 Purpose & Design Philosophy
|
||||
|
||||
The **MetaHealthDaemon (MHD)** is a "Watchdog of Watchdogs" (Layer 2). While the Survival Stack (L1) monitors trading risk and execution health, the MHD monitors the **liveness and validity of the entire system infrastructure**.
|
||||
|
||||
**Core Principles**:
|
||||
- **Statelessness**: No local state beyond the current check cycle.
|
||||
- **Dependency-Light**: Operates even if Hazelcast, Prefect, or Network are down.
|
||||
- **Hierarchical Reliability**: Uses 5 orthogonal "Meta-Sensors" to compute a system-wide health score (`Rm_meta`).
|
||||
- **Platform Agnostic**: Native support for Linux (Red Hat) and FreeBSD.
|
||||
|
||||
### 23.2 MHD Physical Files
|
||||
|
||||
| File | Purpose | Location |
|
||||
|---|---|---|
|
||||
| `meta_health_daemon.py` | Core daemon logic | `prod/` |
|
||||
| `meta_health_daemon.service` | Systemd unit (Linux) | `prod/` (deployed to `/etc/systemd/`) |
|
||||
| `meta_health_daemon_bsd.rc` | rc.d script (FreeBSD) | `prod/` (deployed to `/usr/local/etc/rc.d/`) |
|
||||
| `meta_health.json` | Latest health report (JSON) | `run_logs/` |
|
||||
| `meta_health.log` | Persistent diagnostic log | `run_logs/` |
|
||||
|
||||
### 23.3 The 5 Meta-Sensors (M1-M5)
|
||||
|
||||
MHD computes health using the product of 5 sensors: `Rm_meta = M1 * M2 * M3 * M4 * M5`.
|
||||
|
||||
| Sensor | Name | Logic | Failure Threshold |
|
||||
|---|---|---|---|
|
||||
| **M1** | **Process Integrity** | `psutil` check for `hazelcast`, `prefect`, `watchdog_service`, `acb_processor`. | Any process missing → 0.0 |
|
||||
| **M2** | **Heartbeat Freshness** | Age of `nautilus_flow_heartbeat` in HZ `DOLPHIN_HEARTBEAT` map. | Age > 30s → 0.0 |
|
||||
| **M3** | **Data Freshness** | Mtime age of `latest_ob_features.json` on disk. | Age > 10s → 0.0 |
|
||||
| **M4** | **Control Plane** | TCP connect response on ports 5701 (HZ) and 4200 (Prefect). | Port closed → 0.2 (partial) |
|
||||
| **M5** | **Health Coherence** | Schema and validity check of L1 `DOLPHIN_SAFETY` (Rm ∈ [0,1], valid posture). | Invalid/Stale (>60s) → 0.0 |
|
||||
|
||||
### 23.4 Meta-Health Postures
|
||||
|
||||
| Rm_meta | Status | Meaning | Action |
|
||||
|---|---|---|---|
|
||||
| **≥ 0.80** | **GREEN** | System fully operational. | Nominal logging. |
|
||||
| **≥ 0.50** | **DEGRADED** | Partial sensor failure (e.g., stale heartbeats). | Warning log + HZ alert. |
|
||||
| **≥ 0.20** | **CRITICAL** | Severe infrastructure failure (e.g., HZ down, but processes up). | Critical alert. |
|
||||
| **< 0.20** | **DEAD** | System core collapsed. | **Infrastructure auto-restart cycle.** |
|
||||
|
||||
### 23.5 Operations Guide
|
||||
|
||||
#### 23.5.1 Deployment (Linux)
|
||||
```bash
|
||||
sudo cp prod/meta_health_daemon.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now meta_health_daemon
|
||||
```
|
||||
|
||||
#### 23.5.2 Deployment (FreeBSD)
|
||||
```bash
|
||||
sudo cp prod/meta_health_daemon_bsd.rc /usr/local/etc/rc.d/meta_health_daemon
|
||||
sudo chmod +x /usr/local/etc/rc.d/meta_health_daemon
|
||||
sudo sysrc meta_health_daemon_enable="YES"
|
||||
sudo service meta_health_daemon start
|
||||
```
|
||||
|
||||
#### 23.5.3 Monitoring & Debugging
|
||||
- **Live State**: `tail -f run_logs/meta_health.log`
|
||||
- **JSON API**: `cat run_logs/meta_health.json` (used by dashboards/CLIs).
|
||||
- **HZ State**: Read `DOLPHIN_META_HEALTH["latest"]` for remote monitoring.
|
||||
|
||||
#### 23.5.4 Restart Logic
|
||||
When `Rm_meta` falls into the `DEAD` zone (<0.20), MHD attempts to restart:
|
||||
1. **Level 1**: Restart `hazelcast` service.
|
||||
2. **Level 2**: Restart `prefect_worker` / `prefect_server`.
|
||||
3. **Level 3**: Restart core daemons (`acb_processor`, `watchdog`).
|
||||
|
||||
Restarts are gated by a cooldown and platform-native commands (`systemctl` or `service`).
|
||||
|
||||
#### 23.5.5 Manual Overrides
|
||||
To disable MHD auto-actions without stopping it, create an empty file: `touch /tmp/MHD_PAUSE_ACTION`. (MHD will continue reporting but skip `attempt_restart`).
|
||||
|
||||
---
|
||||
|
||||
*End of DOLPHIN-NAUTILUS System Bible v3.0 — 2026-03-23*
|
||||
*Champion: SHORT only (APEX posture, blue configuration)*
|
||||
*Meta-System: MHD v1.0 active*
|
||||
*Status: Paper trading ready. Meta-system "Gold Certified".*
|
||||
Reference in New Issue
Block a user