chore: safety snapshot 2026-03-05 — HCM infrastructure before 2y klines experiment
Captures critical infrastructure surrounding the nautilus_dolphin core package: - dolphin_vbt_real.py: VBT vectorized backtest engine (6008 lines) - dolphin_paper_trade_adaptive_cb_v2.py: champion runner (champion_5x_f20) - _update_vbt_cache.py / update_VBT_parquet_cache.bat: cache builder - external_factors/: ExF system (all 85 indicator fetchers + NPZ cache) - mc_forewarning_qlabs_fork/: QLabs-enhanced MC-Forewarner research fork - DATA_LOCATIONS.md: source-of-truth path registry - .gitignore: excludes vbt_cache*, backfilled_data, .venv, models, etc. Note: nautilus_dolphin/ has own git repo (inner) — safety snapshot committed there separately. Champion state: WR=49.3%, ROI=+44.89%, PF=1.123, DD=14.95%, Sharpe=2.50 (55d, full-stack, abs_max_lev=6.0). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
98
DATA_LOCATIONS.md
Normal file
98
DATA_LOCATIONS.md
Normal file
@@ -0,0 +1,98 @@
|
||||
# DOLPHIN NG HD Data Locations
|
||||
|
||||
## Production Data
|
||||
|
||||
**Location**: `C:\Users\Lenovo\Documents\- Dolphin NG HD (NG3)\correlation_arb512`
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
correlation_arb512/
|
||||
├── matrices/
|
||||
│ ├── 2025-12-26_SKIP/
|
||||
│ ├── 2025-12-27_SKIP/
|
||||
│ ├── ...
|
||||
│ ├── 2025-12-31/
|
||||
│ ├── 2026-01-01/
|
||||
│ │ ├── scan_016875_w50_000003.arb512.pkl.zst
|
||||
│ │ ├── scan_016875_w150_000003.arb512.pkl.zst
|
||||
│ │ ├── scan_016875_w300_000003.arb512.pkl.zst
|
||||
│ │ ├── scan_016875_w750_000003.arb512.pkl.zst
|
||||
│ │ └── ...
|
||||
│ ├── 2026-01-02/
|
||||
│ ├── 2026-01-03/
|
||||
│ └── 2026-01-04/
|
||||
│
|
||||
├── eigenvalues/
|
||||
│ ├── 2025-12-26_SKIP/
|
||||
│ ├── ...
|
||||
│ ├── 2026-01-01/
|
||||
│ │ ├── scan_016875_000003.json
|
||||
│ │ ├── scan_016876_000014.json
|
||||
│ │ └── ...
|
||||
│ └── ...
|
||||
│
|
||||
├── eigenvectors/
|
||||
│ └── [dated directories with eigenvector data]
|
||||
│
|
||||
└── metadata/
|
||||
└── [dated directories with metadata]
|
||||
```
|
||||
|
||||
### File Naming Convention
|
||||
|
||||
**Eigenvalue JSON**: `scan_NNNNNN_HHMMSS.json`
|
||||
- `NNNNNN`: 6-digit scan number
|
||||
- `HHMMSS`: Timestamp (HHMMSS format)
|
||||
|
||||
**Matrix ZST**: `scan_NNNNNN_wWWW_HHMMSS.arb512.pkl.zst`
|
||||
- `NNNNNN`: 6-digit scan number (matches eigenvalue)
|
||||
- `WWW`: Window size (50, 150, 300, 750)
|
||||
- `HHMMSS`: Timestamp
|
||||
- `.arb512.pkl.zst`: Blosc-compressed pickle with 512-bit arb precision
|
||||
|
||||
### SKIP Directories
|
||||
|
||||
Directories with `_SKIP` suffix should be excluded from processing.
|
||||
These contain data that failed validation or is marked for exclusion.
|
||||
|
||||
---
|
||||
|
||||
## Test Data (Current Project)
|
||||
|
||||
**Location**: `C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict`
|
||||
|
||||
Test data should mirror production structure with partial data:
|
||||
```
|
||||
- DOLPHIN NG HD HCM TSF Predict/
|
||||
├── matrices/
|
||||
│ ├── [root level files - legacy format]
|
||||
│ └── 2026-01-03/
|
||||
├── eigenvalues/
|
||||
│ ├── 2026-01-01/
|
||||
│ └── 2026-01-03/
|
||||
└── ...
|
||||
```
|
||||
|
||||
**Note**: Test data scan numbers may not match between directories.
|
||||
Always verify pairing before running pipelines.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Environment | Path |
|
||||
|-------------|------|
|
||||
| **Production** | `C:\Users\Lenovo\Documents\- Dolphin NG HD (NG3)\correlation_arb512` |
|
||||
| **Test/Dev** | `C:\Users\Lenovo\Documents\- DOLPHIN NG HD HCM TSF Predict` |
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- **ZST_Compressed_Matrix_DOLPHIN_format_spec.md** - Detailed format specification for `.arb512.pkl.zst` files
|
||||
- **run_joint_encoder_pipeline.py** - Pipeline using this data
|
||||
|
||||
---
|
||||
|
||||
*Last updated: 2026-01-10*
|
||||
Reference in New Issue
Block a user