26 Commits

Author SHA1 Message Date
Codex
cfb3d7cf4f dita_v2: backport asex_account.py (from violet main) + test (from PASS9) to canonical upstream
Heals reverse vendor drift: the ASEx AccountProjectionV2 adapter was committed on
the vendored copy (violet.git main 824c5cf) instead of upstream, violating the
edit-upstream-then-sync doctrine. Test was orphaned on the PASS9 worktree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 01:00:40 +02:00
Codex
9bef1f6b01 dita_v2: VenueTelemetrySnapshot + Zinc venue plane (4th shm partition) + local Rust target dir
Upstream-canonical commit of the ~366 lines that sat uncommitted in the working
tree (surveyed in DITAV2_SOA_SURVEY_20260702). Venue telemetry published at every
venue boundary (submit/cancel/reconcile) over the zinc venue_region; SOA per
UV DITAv2 adjudication 2026-07-03.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:42:20 +02:00
Codex
1db4802ee8 docs: add VIOLET full system assessment 2026-06-27 14:08:55 +02:00
Codex
37748cf180 feat(violet): V4 arm-readiness reporter (PASS MM1)
Read-only aggregation module answering 'is V4 ready to ARM the live canary?'
by consuming gate outcomes into a typed StrictModel report.

Files:
- v4_readiness.py: V4ReadinessReport (6 bool gates + reasons) + assess_v4_readiness()
- test_violet_v4_readiness.py: 13 tests (happy, parametrized single-false,
  hypothesis 2^6 invariant, mutation litmus, StrictModel poison)

Co-Authored-By: mm_VIOLET1 <mm_VIOLET1@h5i>
2026-06-24 14:01:51 +02:00
Codex
0254e9a4c0 ```bash
cd /mnt/dolphinng5_predict
export H5I_AGENT=aider_nvnemo
h5i msg inbox
```

Co-authored-by: aider (openrouter/nvidia/nemotron-3-ultra-550b-a55b:free) <aider@aider.chat>
2026-06-24 12:28:17 +02:00
Codex
ca0abc0946 docs: add H5I messaging setup documentation 2026-06-24 12:24:09 +02:00
Codex
d652ff1651 Reference VIOLET DARK preclear report
Report path: prod/reports/violet_dark_clickhouse_preclear_20260618T135531Z.md

h5i object capture was skipped because h5i object ingestion was hanging; the report remains at the path above.
2026-06-18 19:36:24 +02:00
Codex
f8826f613d VIOLET: record PASS4-done + Task 17/18 caveats in the review queue (PR docs)
PASS 4 completed on worktree agent/oa-violet4 (/mnt/vp-oa4). Recorded in the review queue the two
caveats the OA agent reported — also flagged in-code as prominent banner comments + empty marker
commits on that branch:
  - Task 17 exec_intent: explicit reference_price required (ShadowDecision has notional, no price).
  - Task 18 alpha_data_feed: parse/feed-port only, no live NT/Binance in unit code.
For Claude's later PASS-4 review (verify the caveats + non-vacuous tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 12:25:38 +02:00
Codex
255722d6fd VIOLET OA: partial spec PASS 9 — economics + observability completeness (OA backlog complete)
VIOLET_PART_SPEC_OA_TODO_PASS9.md — the final parallelizable pass, discharging VIOLET's binding V1
economics/observability inheritance. Six independent units extending contracts_v3 (Provenance/
EconomicsRow/DecisionDivergenceRow/SoakReadiness):
  43. Observability/economics contracts.
  44. DDL-first exactly-one-row-per-event economics sink (ReplacingMergeTree key, event_seq,
      wait_for_async_insert=1, reject-at-source) — the PINK duplicate-emission fix.
  45. Provenance + per-trade PnL feedback (capital = anchor + Σ trade-realized; never a balance
      snapshot — shared-account foreign-fill immunity).
  46. Factor-plane divergence monitor v2 (per-factor VIOLET-vs-BLUE abs_err, DDL-first).
  47. Soak-readiness aggregator (consolidate all reports → SoakReadiness GO/NO_GO).
  48. Soak-readiness @gate — GO iff parity≥thr ∧ latency ∧ determinism ∧ one-row ∧ isolation;
      EXPECTED NO_GO today by design (CRITICAL #1), the gate makes it machine-checked (not forced green).

DDL-first + row-set==DDL parity + reject-at-source discipline throughout; never writes a production
table (new dolphin_violet tables only); ch_writer.py not edited.

OA BACKLOG COMPLETE: PASS 1–9 (~48 independent contracts_v3-keyed units) cover the full
parallelizable DARK surface of V0→V6. Remaining work is non-parallelizable + Claude's: parity
root-cause, review of all passes, integration, E2E, VBT re-cert, real-key V4 smoke, live execution.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:51:35 +02:00
Codex
16b2d18829 VIOLET OA: partial spec PASS 8 — V6 bible consumers (posture / MARAS / regime) + cadence telemetry
VIOLET_PART_SPEC_OA_TODO_PASS8.md — the V6 bible-layer remainder (ACB done; vol = PASS-4 §14;
sub-second SL guard = PASS-3 §10), six independent units extending contracts_v3 (Posture/
PostureEffect/MarasFingerprint/RegimeView/ActionCadenceDelta):
  37. Regime/posture contracts (Posture literal == sizing.Posture).
  38. Posture effects engine — 5-state entry-gate/flatten/size-cap, transcribed from the
      orchestrator (HIBERNATE_HALT :365, STALKER 2.0 :613, TURTLE regime_dd_halt).
  39. MARAS fingerprint consumer — role determined FROM CODE (modulating vs observability-only);
      no invented effect.
  40. Regime composite read-model (RegimeView; observability/divergence, NOT a sizing path).
  41. Cadence shadow-actuation telemetry harness — evaluate-at-fastest / actuate-at-Q deltas (the
      shadow evidence required before any Q loosening).
  42. Posture/regime parity @gate vs BLUE (bit-identity of gating).

Framing: VIOLET CONSUMES posture/MARAS (BLUE's MHS owns them) — these units APPLY published
effects, they do not run the meta-health state machine. Faithful to the kernel; cite file:line; the
STALKER size-cap stays in sizing.py (this pass owns the entry-gate + observability).

This is the last bible-layer pass; PASS 9 (economics/observability completeness) remains. Added
PASS 8 to the review queue in VIOLET_TODO_CRITICAL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:32:53 +02:00
Codex
0ab83c528a VIOLET OA: partial spec PASS 7 — V5 selection + multi-asset slots + capital allocation
VIOLET_PART_SPEC_OA_TODO_PASS7.md — the V5 ladder layer as six independent units extending
contracts_v3 (AssetRank/SlotState/CapitalAllocation/SlotPolicy):
  31. Selection contracts + SlotPolicy.
  32. ARS ranking pipeline (faithful to AlphaAssetSelector.rank_assets + OB Sub-1 ±5/10% + IRP
      alignment filter + stablecoin exclusion).
  33. Multi-asset slot manager (max_slots, assignment by rank, hysteresis anti-churn, no double-occ).
  34. Capital allocation across slots (BLUE notional = base_fraction × conviction × capital; no
      invented normalization).
  35. Selection→slot→intent multi-asset DARK flow harness.
  36. Ranking bit-identity @gate vs the real AlphaAssetSelector.

NOTE (CRITICAL #1): the disappointing live parity (pick-match 1.5%, no-pick 86%, sizing near-
identical when aligned) implicates THIS selection/timing layer. PASS 7 PINS the ranking math
bit-for-bit (Task 36), which narrows the live-parity suspect to timing/join — but the
live-aggregate root-cause stays Claude's job (not this pass). Faithful to AlphaAssetSelector /
BIBLE §5; cite kernel file:line. New-file-only under selection/.

Standing ready for PASS 8–9. Added PASS 7 to the review queue in VIOLET_TODO_CRITICAL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:53:12 +02:00
Codex
5f636ef723 VIOLET OA: partial spec PASS 6 — execution internals + quirk-injection gate
VIOLET_PART_SPEC_OA_TODO_PASS6.md — the execution internals on top of PASS-5's mock, plus the
quirk-injection layer that flips PASS-5's QuirkProfile seams ON and proves each handler survives.
Six independent units extending contracts_v3 (AccountFrame/PositionFrame/ReconcileCorrection/
OwnershipPolicy):
  25. Account/position frame contracts + ownership predicate (foreign-fill filter basis).
  26. Fill-pump (ownership filter [quirk #2], dedup, idempotent) → ledger deltas.
  27. Reconcile loop (zero-wb guard [#1], reseed-on-update, settle/funding event_seq order [#5]).
  28. TTL requote / cancel-replace (+ setLeverage ordering [#7]) over PASS-5 venue + V2 driver.
  29. Orphan detection (ORPHAN_VENUE adopt, ORPHAN_LOCAL quarantine — never silent flatten/resurrect).
  30. Quirk-injection suite + @gate: flip every QuirkProfile flag ON, assert each is neutralized.

VIOLET's exec reconcile/quirk-handling mirrors PINK's production-tested logic (pink_direct.py is
the authoritative reference — transcribe + cite). New-file-only: quirk injection plugs into PASS-5's
existing seams via NEW fill/frame models, never editing the PASS-5 mock. Real-key boundary smoke
remains MANDATORY before V4-live (injection proves logic, not undocumented venue reality).

Standing ready for PASS 7–9. Added PASS 6 to the review queue in VIOLET_TODO_CRITICAL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:37:32 +02:00
Codex
28a2b90b1d VIOLET OA: partial spec PASS 5 — mock-BingX execution stack (quirks seam-only, deferred)
VIOLET_PART_SPEC_OA_TODO_PASS5.md — the DARK mock-BingX execution stack so the full order
lifecycle is buildable/testable with zero keys/risk (V4-live later = swap mock for real client).
Five independent units extending contracts_v3 (Order/OrderAck/Fill/OrderStatus/PositionDelta):
  20. Execution contracts + QuirkProfile seam registry (no quirk logic).
  21. Mock BingX order FSM (legal lifecycle, partials, reject, reduce-only-increase rejection).
  22. Mock BingX venue adapter (port-conformant, injectable fill model, quirk hooks INERT by default).
  23. Fill→position/PnL reducer (PnL from fill_price ONLY; ×leverage-aware) → PositionDelta.
  24. DARK E2E gate: ExecIntent → router/driver → mock venue → fill reducer → PASS-4 ledger.

CRITICAL per operator: a prominent ⚠️ section makes explicit that BingX "quirks" (zero-wb WS
frames, ownership/foreign-fill collision, bound-price poison, ×leverage notional, settle desync,
reduce-only edge, setLeverage race, dead .pro TLS) are EXPLICITLY OUT OF SCOPE for PASS 5 — the
mock carries injection SEAMS (QuirkProfile, default OFF) to accommodate them LATER, and a real-key
boundary smoke remains MANDATORY before V4-live. Quirks sourced from PINK orphan/reconcile +
DITAv2 audit memory.

Standing ready to write PASS 6–9 (exec internals/reconcile, V5 selection+slots, V6 bible
consumers, economics/observability). Added PASS 5 to the review queue in VIOLET_TODO_CRITICAL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:26:15 +02:00
Codex
80ce1a1afe VIOLET OA: partial spec PASS 4 — next plan units (entry gate, exits, accounting, exec-intent, feed)
VIOLET_PART_SPEC_OA_TODO_PASS4.md — six plan-derived independent units continuing from PASS 3,
extending the shared contracts_v3 vocabulary (adds ExecIntent / CapitalState / VolGateResult /
AlphaBar):
  14. Volatility regime gate (faithful vol_ok — the entry precondition).
  15. AlphaExitEngineV7 wrapper (V3a deferred — wrap BLUE's exit engine as-is, discretionary pri 2).
  16. Capital/economics provenance ledger (capital = anchor + Σ deltas, event_seq, exactly-one-row).
  17. Execution-intent emitter, DARK (V4 boundary — ShadowDecision → ExecIntent, log-only sink).
  18. Alpha-side reactor data feed (V2 NT Binance DATA spike → AlphaDataFeedPort, data-only).
  19. MAX_HOLD + MEAN_REVERSION scan-driven exit timers.
Each ships + tests standalone; composition map + recommended order (14→16→15→19→17→18) + the same
HARD RULES + commit/branch policy. VIBRISS/adaptive-TP explicitly excluded.

Also: added PASS 3 + PASS 4 to the review queue in VIOLET_TODO_CRITICAL.md §2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 09:27:23 +02:00
Codex
9c068619e1 VIOLET: CRITICAL parity-disappointment TODO + OA partial spec PASS 3
VIOLET_TODO_CRITICAL.md (prominent review queue):
- 🔴 CRITICAL #1: VIOLET↔BLUE parity report (PASS-2 Task 4, violet_parity_20260616_220412.md)
  is VERY DISAPPOINTING — pick-match 1.5%, no-pick 86%, same-asset 13.6%. KEY NUANCE: on the 43
  aligned rows sizing is near-identical (lev_err median 0.0) → the divergence is SELECTION /
  TIMING / comparison-ALIGNMENT, NOT the sizing math. Root-cause before any soak/V4. (Claude.)
- #2: OA PASS-1 + PASS-2 reported done, NOT yet reviewed — review queue listed.
- #3: future integration/E2E + the pass-vs-sprint nomenclature note (pass = sub-sprint
  work-package; V-stage = sprint/epic, matching existing usage).

VIOLET_PART_SPEC_OA_TODO_PASS3.md (lengthy, plan-derived, independent units with shared
interfaces — nothing invented): contracts_v3 (VenueTick/OpenPositionView/ExitDecision) +
VenuePriceFeedPort+BingX adapter, MechanicalExitGuard (armed TP/SL), sub-second SL/ADVSL floor
guard, event-sourced restore from trade_reconstruction, venue lead/lag slippage metric, cadence
per-action Q schedule + telemetry. Each ships + tests standalone; composition map + order given.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 09:01:49 +02:00
Codex
8babce893c VIOLET OA: partial spec PASS 2 — spec-advancing self-contained tasks
VIOLET_PART_SPEC_OA_TODO_PASS2.md — four new-file-only tasks that advance the main V0→V6 ladder
toward the DARK-soak gate and V4, each painstakingly specified (why/spec-link, exact new files,
approach, tests, pass criteria), under the same HARD RULES + COMMIT/BRANCH policy as PASS 1:
  4. Live BLUE↔VIOLET decision parity report (V3d → soak-readiness gate; reads CH, no writes).
  5. Full-DecisionEngine reactor latency gate (V0/V3 budget proof with the real 5-factor brain).
  6. L3 tradeability projector standalone (V4 prep; ShadowDecision → exchange-leverage projection,
     DARK, not wired to venue/journal).
  7. Deterministic replay + golden gate (V3d determinism criterion; same scans ⇒ same decisions).
Independent/parallelizable; recommended order 5→7→4→6. None need the operator's soak/keys
greenlight. PASS-1 tasks reported done — review deferred (noted).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:44:30 +02:00
Codex
bae9284582 VIOLET OA: base-fraction sizing study\n\nAdd the read-only CH-backed base-fraction study script plus the final archived report. 2026-06-16 23:25:38 +02:00
Codex
12b768bc4f VIOLET OA: add venue OB provider seam
Co-authored-by: Codex <codex@openai.com>
2026-06-16 23:01:17 +02:00
Codex
53bdd908f4 VIOLET OA: add sizing parity pin tests
Co-authored-by: Codex <codex@openai.com>
2026-06-16 22:51:07 +02:00
Codex
f0a73491bd VIOLET OA: partial spec for a parallel agent (3 self-contained, new-file-only tasks)
VIOLET_PART_SPEC_OA_TODO.md carves the self-contained pending items from the VIOLET plan so a
separate agent can work without colliding with in-flight files:
  1. Parity-pin tests for sizing.py's hand-replicated arithmetic (regime/ob/strength/dc) vs the
     real NDAlphaEngine methods — test-only.
  2. Multi-exchange OB provider seam (VioletVenueOBProvider scaffold + interface + design note)
     — the BingX/3rd-party swap point, NOT wired live.
  3. Base-fraction sizing study (analysis script + report) — pure read-only analysis, no
     behavior change.
Each task: exact affected (new-only) files, V-TYPES + DARK + never-touch-shared rules, required
tests + pass criteria. Includes an explicit COMMIT/BRANCH policy (own git worktree, or partial
commits with `git commit -- <paths>`) prompted by today's three shared-index collisions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 21:54:02 +02:00
Codex
0f3d7650c3 VIOLET V3.4d: launcher OB-persistence + boost/beta prior; OB-feed/HZ-bridge/coordination doc
Launcher shadow path now keeps the persistent state BLUE keeps across scans (faithful, not
per-scan single-shot): ONE OBFeatureEngine wired lazily on first assets and kept for the
service lifetime, reading BLUE's EXTANT published OBF feed via HZOBProvider (read-only HZ
entry-listener cache — NO new OB storage; provider is the swap seam for a future direct
BingX/3rd-party OB stream); per-scan ob_bar_idx into step_live/get_market; and prior_boost_beta
carried across scans (stale exf keeps the prior). shadow_decision_step threads ob_engine +
bar_idx + prior into source_live_blue_sizing_factors; build_shadow_live_source seeds the state
+ a default ob_engine_factory (injectable for tests).

TODO_HZBRIDGE markers added at all 3 VIOLET->Hazelcast touch points: per operator the upcoming
dolphinng5_predict/hzbridge must become the sanctioned HZ connection (silent client-death /
lockup / dropout mitigation) — refactor ASAP once it ships.

Doc VIOLET_OB_FEED_AND_AGENT_COORDINATION.md: OB-feed sourcing doctrine, HZ-bridge TODO, and
the multi-agent worktree + doctrine/release/status scheme (prompted by the shared-index
incidents where agents' staged files cross-contaminate commits).

6 launcher tests (+wired-once/bar_idx-increments, +prior-carries). violet-only; partial commit
(only my 4 paths) to avoid sweeping a concurrent agent's staged files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 21:44:03 +02:00
Codex
2f5ce55967 M5: implement real Iceoryx2Sink — iceoryx2 0.9.1 production plane sink
Replaces the stub that returned Err(Unimplemented) with a working iceoryx2
publish-subscribe sink. Key design:

- One iceoryx2 pub/sub service per configured stream at {prefix}/{stream}
- Fixed-size [u8] samples with in-band binary header (seq, src_ts_us, ts_us,
  NUL-padded key(48), len(32), value(N)) — no unsafe repr(packed) needed
- Heartbeat stream at {prefix}/hzbridge_heartbeat with same layout
- Uses BackpressureStrategy::DiscardData to avoid blocking the feeder
- Uses ipc_threadsafe::Service so the Publisher is Send (ipc::Service uses
  SingleThreaded sync which is !Send)
- Daemon builds Iceoryx2Sink by default; --allow-null-sink for dev use only

Constraint compliance:
  C1 — no unsafe (crate-root forbid(unsafe_code) enforced by compiler)
  C2 — no async (all iceoryx2 calls synchronous)
  C5 — iceoryx2 0.9.1 already pinned per ADR-001
  C8 — fixed-size samples, pre-allocated publishers, bounded loan slots

Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
2026-06-16 21:09:02 +02:00
Codex
901ea1046f M5: implement real Iceoryx2Sink — iceoryx2 0.9.1 production plane sink
Replaces the stub that returned Err(Unimplemented) with a working iceoryx2
publish-subscribe sink. Key design:

- One iceoryx2 pub/sub service per configured stream at {prefix}/{stream}
- Fixed-size [u8] samples with in-band binary header (seq, src_ts_us, ts_us,
  NUL-padded key(48), len(32), value(N)) — no unsafe repr(packed) needed
- Heartbeat stream at {prefix}/hzbridge_heartbeat with same layout
- Uses BackpressureStrategy::DiscardData to avoid blocking the feeder
- Uses ipc_threadsafe::Service so the Publisher is Send (ipc::Service uses
  SingleThreaded sync which is !Send)
- Daemon builds Iceoryx2Sink by default; --allow-null-sink for dev use only

Constraint compliance:
  C1 — no unsafe (crate-root forbid(unsafe_code) enforced by compiler)
  C2 — no async (all iceoryx2 calls synchronous)
  C5 — iceoryx2 0.9.1 already pinned per ADR-001
  C8 — fixed-size samples, pre-allocated publishers, bounded loan slots

Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
2026-06-16 21:08:16 +02:00
Codex
262cada664 VIOLET V3.4c: ACB boost/beta — keep-prior on stale exf, decision logging, semantics doc
Per operator: VIOLET must keep a "prior" on stale ExF rather than collapse to neutral, and
the ACB's role must be commented-in + logged.

- KEEP-PRIOR on stale/missing exf: BLUE keeps _day_base_boost/_day_beta on the stale branch
  (update_acb_boost isn't called). _source_boost_beta now takes `prior` (boost,beta) and
  returns it on ValueError / no-exf, seeded by the first successful compute. Threaded as
  `prior_boost_beta` through source_live_blue_sizing_factors (caller persists last good).
- LOGGING: violet.live_blue_source logger emits live boost/beta + the full sourced plane.
- SEMANTICS: documents the ACB in effect (inverse v6) and CONFIRMS ACB IS in the sizing layer
  via _day_base_boost/_day_beta → _update_regime_size_mult (esf_alpha_orchestrator.py:771-772,
  898-909). OB Sub-4 beta modulation dormant in live — per operator found NON-PERFORMANT and
  deliberately bypassed; TODO_SOMEDAY in-code to find documentary confirmation.

37 tests. violet-only; no shared-file edits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 18:22:29 +02:00
Codex
fac287d678 VIOLET V3.4c: make boost/beta, signal-gen, OB bit-identical to BLUE (+ exhaustive tests)
Operator directive: VIOLET must do IDENTICALLY what BLUE does for the three parity
flags — approximation cannot guarantee bit-for-bit functioning. Reworked
live_blue_source.py to call BLUE's OWN code paths, not reconstruct/substitute them.

boost/beta — was reading the published DOLPHIN_FEATURES.acb_boost scalar
(acb_processor_service's daily value). BLUE's trader does NOT use that for sizing; it
recomputes live via acb.get_dynamic_boost_from_hz(exf_latest, w750_velocity, direction)
with a bare AdaptiveCircuitBreaker() and NO ob_engine (nautilus_event_trader.py
on_exf_update:4769 / rollover prewarm:2710). New _source_boost_beta replicates that call
exactly (reads exf_latest + latest_eigen_scan.w750_velocity; 0.0→None like BLUE; on stale
exf ValueError → neutral, mirroring BLUE's "ACB Stale Data Fallback"). The published
acb_boost is never read. Test pins bit-identity against the real ACB.

signal-gen (dc_status) — was AlphaSignalGenerator() bare defaults; coincidentally equal to
BLUE today, but BLUE builds it from ENGINE_KWARGS (trader:128-133, threaded at
esf_alpha_orchestrator.py:180-191), so a champion retune would silently diverge. Now
constructed with BLUE_SIGNAL_GEN_KWARGS (vel_div_* imported from the kernel constants).
Test parses ENGINE_KWARGS from the trader source and asserts each param matches — drift
becomes a red test, not a silent miss.

OB — was a reinvented HazelcastOBProvider reading asset_*_ob with custom parsing. Now uses
BLUE's OWN HZOBProvider + OBFeatureEngine, wired exactly as _wire_obf
(nautilus_event_trader.py:4967-4980): step_live(assets, bar_idx) then get_market. Engine
is injectable + persistent so OB accumulation matches BLUE across scans (caller owns it).

Deleted: HazelcastOBProvider, _extract_acb, the status-label mc path. mc_scale fix
(begin_day cat/env thresholds) retained. Module docstring + structural-divergence doc
updated: all three flags FIXED; only _derive_mc_scale remains hand-replicated (pinned by
formula test). OPEN follow-up: launcher shadow_decision_step should pass a persistent
ob_engine + bar_idx for cross-scan OB history.

34 tests (33 + live-HZ smoke deselected): boost/beta-vs-ACB bit-identity (incl. w750=0→None,
no-exf, stale ValueError, ignores acb_boost), signal-gen ENGINE_KWARGS pin (parametrized),
OB wiring (step_live call, HZ coords, neutral paths), mc_scale formula, sequence dc/selector
parity, anomaly handling. violet-only; no shared-file edits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 17:33:01 +02:00
Codex
1415a65670 VIOLET V3.4c review: fix mc_scale BLUE-parity bug + document re-derivation debt
REVIEW of the V3.4c/d/e work (other agents) for BLUE-algo compliance found one
real bug and one architectural concern.

BUG FIXED — mc_scale derivation (live_blue_source.py):
The adapter mapped `mc_forewarner_latest.status == "ORANGE"` → 0.5. But BLUE's sizing
mc_scale is NOT the MC service's `status` label (that label, from mc_forewarner_flow.py,
uses GREEN<0.10/ORANGE<0.30/RED — observability only). The live trader re-derives the
haircut in begin_day (esf_alpha_orchestrator.py:956-962) from the SAME published fields
`catastrophic_prob` + `envelope_score` with DIFFERENT thresholds:
    mc_red    = cat>0.25 or env<-1.0
    mc_orange = (not mc_red) and (env<0 or cat>0.10)
    mc_scale  = 0.5 if mc_orange else 1.0
The two disagree (e.g. cat=0.05/env=-0.5 → label GREEN→1.0 but BLUE orange→0.5;
cat=0.28 → label ORANGE→0.5 but BLUE red→1.0). Rewrote `_derive_mc_scale` to mirror
begin_day exactly on the source fields. Per operator: there is genuine ambiguity over
which surface is "nominal" — we go with the SOURCE FIELDS + begin_day formula (the path
that actually drives BLUE's sizing); the ambiguity + decision are noted in the docstring.

TESTS — the old fixtures were complicit: they fed `{"status":"ORANGE"}`, a payload shape
BLUE never emits, so they "passed" against fiction. Replaced with BLUE's real payload
(`catastrophic_prob`/`envelope_score`) and added a parametrized formula test covering the
exact divergence cases the status-based code got wrong, plus missing/garbage-field neutral.
15 passed (live-HZ smoke deselected).

ARCHITECTURAL DEBT documented — VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md:
VIOLET imitates BLUE's computations in a DIFFERENT module/scope structure. Kernels are
WRAPPED (safe), but orchestration arithmetic (compose/regime/ob/strength/mc_scale) is
HAND-REPLICATED out of the monolithic NDAlphaEngine — making orderly, verifiable BLUE↔VIOLET
parity comparison and refactoring hard. Doc inventories every re-derivation with BLUE's
file:line authority, the drift risk, and mitigations (parity-pin tests; single canonical
surface per ambiguous factor; eventual DITAv2 Rust-backplane convergence). Pointer added to
live_blue_source.py's module docstring.

Also reviewed (no change needed): launcher launch_dolphin_violet.py is DARK-safe
(ObserveOnlyVenue + shadow gated default-OFF, no order path); shadow_live_factors.py glue
is correct; trade_slot_compare.py reimplements no BLUE algo. fb34431's launcher tests —
which the authoring agent never ran on the slow mount — pass (14 green).

violet-only; no shared-file edits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 16:54:35 +02:00
39 changed files with 777605 additions and 421 deletions

View File

@@ -0,0 +1,87 @@
use hzbridge::config::Config;
use hzbridge::sink::PlaneSink;
use std::path::PathBuf;
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("hzbridge=info"))
.format_timestamp_micros()
.init();
let arguments = match parse_arguments() {
Ok(a) => a,
Err(detail) => {
eprintln!("hzbridged: {detail}");
std::process::exit(1);
}
};
let config = match Config::from_path(&arguments.config_path) {
Ok(c) => c,
Err(error) => {
eprintln!("hzbridged: {error}");
std::process::exit(1);
}
};
let sink: Box<dyn PlaneSink> = build_sink(&config, arguments.allow_null_sink);
log::info!("fsm=Boot msg=starting name=hzbridge");
let bridge = hzbridge::Bridge::new(config, sink);
if let Err(error) = bridge.run() {
hzbridge::blackbox::abort(&error.to_string());
}
}
#[cfg(feature = "iceoryx2")]
fn build_sink(config: &Config, allow_null_sink: bool) -> Box<dyn PlaneSink> {
if allow_null_sink {
log::warn!("fsm=Boot msg=development_null_sink_enabled");
return Box::new(hzbridge::sink::NullSink);
}
match hzbridge::sink_iceoryx2::Iceoryx2Sink::new(config) {
Ok(sink) => {
log::info!("fsm=Boot msg=iceoryx2_sink_ready");
Box::new(sink)
}
Err(error) => {
log::error!("fsm=Boot msg=iceoryx2_sink_failed error={error}");
eprintln!("hzbridged: iceoryx2 sink failed: {error}");
std::process::exit(1);
}
}
}
#[cfg(not(feature = "iceoryx2"))]
fn build_sink(_config: &Config, allow_null_sink: bool) -> Box<dyn PlaneSink> {
if !allow_null_sink {
eprintln!(
"hzbridged: iceoryx2 feature disabled and no --allow-null-sink; \
use --allow-null-sink only for wire/runtime smoke tests"
);
std::process::exit(1);
}
log::warn!("fsm=Boot msg=development_null_sink_enabled (iceoryx2 feature not available)");
Box::new(hzbridge::sink::NullSink)
}
struct Arguments {
config_path: PathBuf,
allow_null_sink: bool,
}
fn parse_arguments() -> Result<Arguments, String> {
let args: Vec<_> = std::env::args_os().skip(1).collect();
match args.as_slice() {
[flag, path] if flag == "--config" => Ok(Arguments {
config_path: path.into(),
allow_null_sink: false,
}),
[flag, path, allow] if flag == "--config" && allow == "--allow-null-sink" => {
Ok(Arguments {
config_path: path.into(),
allow_null_sink: true,
})
}
_ => Err("usage: hzbridged --config <path> [--allow-null-sink]".to_owned()),
}
}

View File

@@ -0,0 +1,336 @@
//! M5: Real iceoryx2 production sink.
//!
//! One publish-subscribe service per configured stream plus a dedicated
//! heartbeat stream. All samples are fixed-size `[u8]` slices with an
//! in-band header so consumers can read key/value directly from shm.
//!
//! Binary layout per sample (all values little-endian):
//! [0..8) seq u64 — monotonically increasing per-stream
//! [8..16) src_ts_us u64 — upstream Hazelcast event timestamp (µs epoch)
//! [16..24) ts_us u64 — local publish timestamp (µs epoch)
//! [24..72) key [u8; 48] — NUL-padded map key (UTF-8)
//! [72..76) len u32 — actual value length (≤ max_value_kib × 1024)
//! [76..) value [u8; MAX_VALUE] — opaque value bytes
//!
//! Heartbeat uses the same layout with seq=0 and key="heartbeat".
//!
//! Constraint compliance:
//! C1 — `#![forbid(unsafe_code)]` at crate root: no unsafe here.
//! C2 — no async: iceoryx2 API is synchronous.
//! C5 — iceoryx2 0.9.1 already pinned in Cargo.toml.
//! C8 — fixed-size samples, pre-allocated publishers.
use crate::config::Config;
use crate::metrics::MetricsSnapshot;
use crate::sink::{PlaneSink, SinkError};
use iceoryx2::prelude::*;
/// Alias for thread-safe IPC service type.
use log::{info, warn};
use std::collections::HashMap;
// ── Binary layout constants ────────────────────────────────────────────────
/// Byte offset at which the value payload begins.
const HEADER_BYTES: usize = 76; // seq(8) + src_ts_us(8) + ts_us(8) + key(48) + len(4)
const KEY_LEN: usize = 48;
// iceoryx2 Publisher<Service=ipc_threadsafe::Service, Payload=[u8], UserHeader=()>
type PublisherIpc = iceoryx2::port::publisher::Publisher<ipc_threadsafe::Service, [u8], ()>;
// ── Per-stream publisher state ──────────────────────────────────────────────
struct StreamPublisher {
/// The iceoryx2 publisher port.
publisher: PublisherIpc,
/// Total sample size = HEADER_BYTES + max_value_bytes.
sample_size: usize,
/// Monotonically increasing per-stream sequence number.
seq: u64,
}
// ── Iceoryx2Sink ────────────────────────────────────────────────────────────
/// The production `PlaneSink` that publishes into iceoryx2 services.
///
/// Keyed by the **short stream name** from config (e.g. `"eigen_scan"`).
/// The full iceoryx2 service name is `{prefix}/{short_name}`.
pub struct Iceoryx2Sink {
/// Map from short stream name → per-stream publisher.
streams: HashMap<String, StreamPublisher>,
/// Heartbeat publisher (best-effort; `None` if creation failed).
heartbeat: Option<StreamPublisher>,
/// The iceoryx2 node — held for the sink's lifetime.
_node: iceoryx2::node::Node<ipc_threadsafe::Service>,
/// Cached from config for use in `publish_and_notify`.
max_value_bytes: usize,
}
type NodeIpc = iceoryx2::node::Node<ipc_threadsafe::Service>;
impl Iceoryx2Sink {
/// Create the sink, opening or creating iceoryx2 services for every
/// configured stream plus the heartbeat stream.
///
/// Returns `SinkError` if any iceoryx2 service cannot be created or
/// opened. Per §5.2 "local shm failure is a broken box" — the caller
/// (the runtime) treats this as Fatal.
pub fn new(config: &Config) -> Result<Self, SinkError> {
let max_value_bytes = config.plane.max_value_kib * 1024;
let sample_size = HEADER_BYTES + max_value_bytes;
let prefix = &config.plane.service_prefix;
let node_name = NodeName::new(&format!("hzbridge_{}", config.hz.cluster))
.map_err(|e| SinkError::new(format!("node name: {e}")))?;
let node = NodeBuilder::new()
.name(&node_name)
.create::<ipc_threadsafe::Service>()
.map_err(|e| SinkError::new(format!("node create: {e:?}")))?;
let mut streams = HashMap::new();
for sc in &config.streams {
let full_name = format!("{}/{}", prefix, sc.stream);
let pub_ = Self::open_service(&node, &full_name, sample_size)?;
info!(
"sink=Iceoryx2Sink msg=stream_ready stream={} sample_size={}",
full_name, sample_size,
);
streams.insert(sc.stream.clone(), pub_);
}
// Heartbeat — best-effort; a failure here does not abort the bridge.
let hb_name = format!("{}/hzbridge_heartbeat", prefix);
let heartbeat = match Self::open_service(&node, &hb_name, sample_size) {
Ok(p) => {
info!("sink=Iceoryx2Sink msg=heartbeat_ready stream={}", hb_name);
Some(p)
}
Err(e) => {
warn!("sink=Iceoryx2Sink msg=heartbeat_failed error={e}");
None
}
};
Ok(Self {
streams,
heartbeat,
_node: node,
max_value_bytes,
})
}
/// Create or open one publish-subscribe service and return a publisher.
fn open_service(
node: &NodeIpc,
service_name: &str,
sample_size: usize,
) -> Result<StreamPublisher, SinkError> {
let name = ServiceName::new(service_name)
.map_err(|e| SinkError::new(format!("service name '{service_name}': {e}")))?;
let service = node
.service_builder(&name)
.publish_subscribe::<[u8]>()
.open_or_create()
.map_err(|e| SinkError::new(format!("open_or_create '{service_name}': {e:?}")))?;
let publisher = service
.publisher_builder()
.initial_max_slice_len(sample_size)
.max_loaned_samples(2)
.backpressure_strategy(BackpressureStrategy::DiscardData)
.create()
.map_err(|e| SinkError::new(format!("publisher '{service_name}': {e:?}")))?;
Ok(StreamPublisher {
publisher,
sample_size,
seq: 0,
})
}
/// Write a value into a loaned iceoryx2 sample and send it.
fn write_sample(
pub_: &mut StreamPublisher,
key: &str,
value_utf8: &[u8],
src_ts_us: u64,
ts_us: u64,
max_value_bytes: usize,
) -> Result<(), SinkError> {
pub_.seq = pub_.seq.wrapping_add(1);
let mut sample = pub_
.publisher
.loan_slice(pub_.sample_size)
.map_err(|e| SinkError::new(format!("loan_slice: {e:?}")))?;
let buf = sample.payload_mut();
let seq = pub_.seq;
// seq
buf[0..8].copy_from_slice(&seq.to_le_bytes());
// src_ts_us
buf[8..16].copy_from_slice(&src_ts_us.to_le_bytes());
// ts_us
buf[16..24].copy_from_slice(&ts_us.to_le_bytes());
// key (NUL-padded, truncated to KEY_LEN)
let key_bytes = key.as_bytes();
let copy_len = key_bytes.len().min(KEY_LEN);
buf[24..24 + copy_len].copy_from_slice(&key_bytes[..copy_len]);
if copy_len < KEY_LEN {
buf[24 + copy_len..72].fill(0);
}
// len
let value_len = value_utf8.len().min(max_value_bytes);
buf[72..76].copy_from_slice(&(value_len as u32).to_le_bytes());
// value
let vs = HEADER_BYTES;
buf[vs..vs + value_len].copy_from_slice(&value_utf8[..value_len]);
// Zero remaining value buffer
if value_len < max_value_bytes {
buf[vs + value_len..vs + max_value_bytes].fill(0);
}
sample.send().map_err(|e| SinkError::new(format!("send: {e:?}")))?;
Ok(())
}
fn now_us() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or(std::time::Duration::ZERO)
.as_micros() as u64
}
/// Encode a `MetricsSnapshot` as a space-separated key=value text blob.
fn encode_heartbeat(snapshot: &MetricsSnapshot) -> Vec<u8> {
format!(
"events_rx={} events_published={} decode_skips={} late_responses={} \
reconnects={} fragments_reassembled={} warmups={} pings_ok={} \
pings_missed={} events_dropped={} uptime_s={} state={} degraded={}",
snapshot.events_rx,
snapshot.events_published,
snapshot.decode_skips,
snapshot.late_responses,
snapshot.reconnects,
snapshot.fragments_reassembled,
snapshot.warmups,
snapshot.pings_ok,
snapshot.pings_missed,
snapshot.events_dropped,
snapshot.uptime_s,
snapshot.state,
snapshot.degraded,
)
.into_bytes()
}
}
impl PlaneSink for Iceoryx2Sink {
fn publish(
&mut self,
stream: &str,
key: &str,
value_utf8: &[u8],
src_ts_us: u64,
) -> Result<(), SinkError> {
let ts_us = Self::now_us();
let pub_ = self.streams.get_mut(stream).ok_or_else(|| {
SinkError::new(format!("unknown stream: {stream}"))
})?;
Self::write_sample(pub_, key, value_utf8, src_ts_us, ts_us, self.max_value_bytes)
}
fn heartbeat(&mut self, snapshot: &MetricsSnapshot) -> Result<(), SinkError> {
let Some(pub_) = self.heartbeat.as_mut() else {
return Ok(());
};
let ts_us = Self::now_us();
let value = Self::encode_heartbeat(snapshot);
Self::write_sample(pub_, "heartbeat", &value, ts_us, ts_us, self.max_value_bytes)
}
}
// ── Tests ──────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn sample_layout_offsets_are_correct() {
let mut buf = vec![0u8; HEADER_BYTES + 256];
// seq = 42
buf[0..8].copy_from_slice(&42u64.to_le_bytes());
// src_ts_us = 1000
buf[8..16].copy_from_slice(&1000u64.to_le_bytes());
// ts_us = 2000
buf[16..24].copy_from_slice(&2000u64.to_le_bytes());
// key = "BTCUSDT"
let k = b"BTCUSDT";
buf[24..24 + k.len()].copy_from_slice(k);
// len = 6
buf[72..76].copy_from_slice(&6u32.to_le_bytes());
// value = "123456"
buf[76..82].copy_from_slice(b"123456");
assert_eq!(u64::from_le_bytes(buf[0..8].try_into().unwrap()), 42);
assert_eq!(u64::from_le_bytes(buf[8..16].try_into().unwrap()), 1000);
assert_eq!(u64::from_le_bytes(buf[16..24].try_into().unwrap()), 2000);
assert_eq!(&buf[24..31], b"BTCUSDT");
assert_eq!(u32::from_le_bytes(buf[72..76].try_into().unwrap()), 6);
assert_eq!(&buf[76..82], b"123456");
}
#[test]
fn header_byte_count_is_76() {
assert_eq!(HEADER_BYTES, 76);
}
#[test]
fn heartbeat_encoding_contains_all_fields() {
let snap = MetricsSnapshot {
events_rx: 100,
events_published: 95,
decode_skips: 2,
late_responses: 0,
reconnects: 1,
fragments_reassembled: 0,
warmups: 3,
pings_ok: 50,
pings_missed: 1,
events_dropped: 0,
uptime_s: 3600,
state: "Streaming".into(),
degraded: false,
};
let encoded = Iceoryx2Sink::encode_heartbeat(&snap);
let text = String::from_utf8(encoded).unwrap();
assert!(text.contains("events_rx=100"));
assert!(text.contains("events_published=95"));
assert!(text.contains("decode_skips=2"));
assert!(text.contains("state=Streaming"));
assert!(text.contains("degraded=false"));
assert!(text.contains("uptime_s=3600"));
}
#[test]
fn key_truncation_fits_48_bytes() {
let long_key = "a".repeat(100);
let max_val = 256;
let mut buf = vec![0u8; HEADER_BYTES + max_val];
let kbytes = long_key.as_bytes();
let copy_len = kbytes.len().min(KEY_LEN);
buf[24..24 + copy_len].copy_from_slice(&kbytes[..copy_len]);
// Verify the stored key is truncated to KEY_LEN and NUL-padded
let stored = &buf[24..72];
assert_eq!(&stored[..KEY_LEN - 1], &[b'a'; 47]);
assert_eq!(stored[KEY_LEN - 1], b'a');
}
#[test]
fn value_truncation_fits_max_value() {
let max_val = 256;
let big_value = vec![0xFFu8; 500];
let value_len = big_value.len().min(max_val);
assert_eq!(value_len, max_val);
assert_eq!(value_len, 256);
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,22 @@
# VIOLET base-fraction sizing study
Generated: `2026-06-16T21:23:08.644826+00:00`
## Bottom line
- Recommended fraction: `0.333` (best Calmar among zero-clip fractions)
- Base 0.20 final capital: `483963.79`; recommended final capital: `1308199.81`
- Base 0.20 maxDD: `20.18%`; recommended maxDD: `33.36%`
- Recommended clip rate: `0.00%`; unconstrained optimizer: `0.500` with `40.11%` clipped at the 3x ceiling.
## Caveat
- Direct non-null slippage telemetry was not available in `trade_execution_quality`; the study uses a conservative taker-heavy impact proxy.
## Cap binding
- The recommended fraction hits the 3x translator cap on `0.00%` of trades.
## Kelly anchor
- Empirical Kelly anchor: `0.500`; fractional anchor: `0.200`
## Files
- JSON: `/mnt/dolphinng5_predict/prod/VIOLET_dev/reports/base_fraction_study_20260616_212308.json`
- Markdown: `/mnt/dolphinng5_predict/prod/VIOLET_dev/reports/base_fraction_study_20260616_212308.md`

View File

@@ -0,0 +1,866 @@
#!/usr/bin/env python3
"""VIOLET base-fraction sizing study.
Read-only analysis against recorded CH trade data. Produces:
- a machine-readable JSON report under prod/VIOLET_dev/reports/
- a short markdown findings note alongside it
The study is scoped by prod/docs/VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md.
It does not modify production code or write to production tables.
"""
from __future__ import annotations
import argparse
import bisect
import csv
import dataclasses
import json
import math
import statistics
import sys
from collections import defaultdict
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Callable, Iterable, Sequence
import numpy as np
PROJECT_ROOT = Path("/mnt/dolphinng5_predict")
REPORTS_DIR = PROJECT_ROOT / "prod" / "VIOLET_dev" / "reports"
CH_URL = "http://localhost:8123/"
CH_USER = "dolphin"
CH_KEY = "dolphin_ch_2026"
BASE_FRACTION_F0 = 0.20
TRANSLATOR_CAP = 3.0
BASE_GRID = np.array(
[0.20, 0.22, 0.24, 0.25, 0.26, 0.28, 0.30, 0.32, 0.333, 0.34, 0.36, 0.38, 0.40, 0.45, 0.50],
dtype=np.float64,
)
@dataclass(frozen=True)
class TradeRow:
trade_id: str
ts: datetime
asset: str
side: str
entry_price: float
exit_price: float
quantity: float
pnl: float
pnl_pct: float
exit_reason: str
leverage: float
capital_before: float
capital_after: float
bars_held: int
regime_signal: int
vel_div_entry: float
boost_at_entry: float
beta_at_entry: float
posture: str
our_leverage: float
composite_hash: int | None = None
scalar_hash: int | None = None
regime: str | None = None
fingerprint_confidence: float | None = None
fingerprint_vel_div: float | None = None
fingerprint_dvol: float | None = None
notional_quote: float | None = None
fill_quality_score: float | None = None
fill_quality_class: str | None = None
fill_rows: int = 0
taker_fill_rows: int = 0
maker_fill_rows: int = 0
@dataclass(frozen=True)
class FingerprintRow:
ts: datetime
regime: str
composite_hash: int
scalar_hash: int
confidence: float
raw_vel_div: float
raw_dvol: float
final_score: float
def _query_tsv(sql: str) -> list[dict[str, str]]:
import urllib.request
req = urllib.request.Request(
CH_URL,
data=sql.encode(),
headers={"X-ClickHouse-User": CH_USER, "X-ClickHouse-Key": CH_KEY},
)
with urllib.request.urlopen(req, timeout=120) as resp:
text = resp.read().decode()
lines = [line for line in text.splitlines() if line.strip()]
if not lines:
return []
reader = csv.DictReader(lines, delimiter="\t")
return list(reader)
def _fmt_ts(dt: datetime) -> str:
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
dt = dt.astimezone(timezone.utc)
return dt.strftime("%Y-%m-%d %H:%M:%S.%f")
def _parse_dt(s: str) -> datetime:
if isinstance(s, datetime):
return s
s = s.strip()
if s.endswith("Z"):
s = s[:-1] + "+00:00"
try:
return datetime.fromisoformat(s).astimezone(timezone.utc)
except ValueError:
# ClickHouse DateTime64 TSV may arrive without timezone suffix.
return datetime.strptime(s, "%Y-%m-%d %H:%M:%S.%f").replace(tzinfo=timezone.utc)
def _parse_float(v: str | None, default: float = 0.0) -> float:
if v is None or v == "" or v == "\\N":
return float(default)
return float(v)
def _parse_int(v: str | None, default: int = 0) -> int:
if v is None or v == "" or v == "\\N":
return int(default)
return int(float(v))
def load_clean_trades() -> list[TradeRow]:
sql = """
WITH dedup AS (
SELECT
trade_id,
max(event_ts) AS ts,
argMax(asset, event_ts) AS asset,
argMax(side, event_ts) AS side,
argMax(entry_price, event_ts) AS entry_price,
argMax(exit_price, event_ts) AS exit_price,
argMax(quantity, event_ts) AS quantity,
argMax(pnl, event_ts) AS pnl,
argMax(pnl_pct, event_ts) AS pnl_pct,
argMax(exit_reason, event_ts) AS exit_reason,
argMax(leverage, event_ts) AS leverage,
argMax(capital_before, event_ts) AS capital_before,
argMax(capital_after, event_ts) AS capital_after,
argMax(bars_held, event_ts) AS bars_held,
argMax(regime_signal, event_ts) AS regime_signal,
argMax(vel_div_entry, event_ts) AS vel_div_entry,
argMax(boost_at_entry, event_ts) AS boost_at_entry,
argMax(beta_at_entry, event_ts) AS beta_at_entry,
argMax(posture, event_ts) AS posture,
argMax(our_leverage, event_ts) AS our_leverage
FROM (
SELECT
trade_id,
ts AS event_ts,
asset,
side,
entry_price,
exit_price,
quantity,
pnl,
pnl_pct,
exit_reason,
leverage,
capital_before,
capital_after,
bars_held,
regime_signal,
vel_div_entry,
boost_at_entry,
beta_at_entry,
posture,
our_leverage
FROM dolphin.trade_events
)
GROUP BY trade_id
)
SELECT
trade_id, ts, asset, side, entry_price, exit_price, quantity, pnl, pnl_pct, exit_reason,
leverage, capital_before, capital_after, bars_held, regime_signal, vel_div_entry,
boost_at_entry, beta_at_entry, posture, our_leverage
FROM dedup
WHERE exit_reason != 'HIBERNATE_HALT' AND bars_held > 0
ORDER BY ts
FORMAT TSVWithNames
"""
rows = _query_tsv(sql)
out: list[TradeRow] = []
for row in rows:
out.append(
TradeRow(
trade_id=row["trade_id"],
ts=_parse_dt(row["ts"]),
asset=row["asset"],
side=row["side"],
entry_price=_parse_float(row["entry_price"]),
exit_price=_parse_float(row["exit_price"]),
quantity=_parse_float(row["quantity"]),
pnl=_parse_float(row["pnl"]),
pnl_pct=_parse_float(row["pnl_pct"]),
exit_reason=row["exit_reason"],
leverage=_parse_float(row["leverage"]),
capital_before=_parse_float(row["capital_before"]),
capital_after=_parse_float(row["capital_after"]),
bars_held=_parse_int(row["bars_held"]),
regime_signal=_parse_int(row["regime_signal"]),
vel_div_entry=_parse_float(row["vel_div_entry"]),
boost_at_entry=_parse_float(row["boost_at_entry"], 1.0),
beta_at_entry=_parse_float(row["beta_at_entry"], 1.0),
posture=row["posture"],
our_leverage=_parse_float(row["our_leverage"]),
)
)
return out
def load_exec_quality() -> dict[str, dict[str, Any]]:
sql = """
SELECT
trade_id,
maxIf(notional_quote, record_kind = 'trade_summary') AS notional_quote,
maxIf(fill_quality_score, record_kind = 'trade_summary') AS fill_quality_score,
anyIf(fill_quality_class, record_kind = 'trade_summary') AS fill_quality_class,
countIf(record_kind = 'fill') AS fill_rows,
countIf(record_kind = 'fill' AND liquidity_side = 'TAKER') AS taker_fill_rows,
countIf(record_kind = 'fill' AND liquidity_side = 'MAKER') AS maker_fill_rows
FROM dolphin.trade_execution_quality
GROUP BY trade_id
FORMAT TSVWithNames
"""
rows = _query_tsv(sql)
out: dict[str, dict[str, Any]] = {}
for row in rows:
out[row["trade_id"]] = {
"notional_quote": _parse_float(row["notional_quote"]),
"fill_quality_score": _parse_float(row["fill_quality_score"]),
"fill_quality_class": row["fill_quality_class"],
"fill_rows": _parse_int(row["fill_rows"]),
"taker_fill_rows": _parse_int(row["taker_fill_rows"]),
"maker_fill_rows": _parse_int(row["maker_fill_rows"]),
}
return out
def load_fingerprints(start_ts: datetime, end_ts: datetime) -> list[FingerprintRow]:
sql = f"""
SELECT
ts, regime, composite_hash, scalar_hash, confidence, raw_vel_div, raw_dvol, final_score
FROM dolphin.maras_fingerprint
WHERE ts >= toDateTime64('{_fmt_ts(start_ts)}', 6, 'UTC')
AND ts <= toDateTime64('{_fmt_ts(end_ts)}', 6, 'UTC')
ORDER BY ts
FORMAT TSVWithNames
"""
rows = _query_tsv(sql)
out: list[FingerprintRow] = []
for row in rows:
out.append(
FingerprintRow(
ts=_parse_dt(row["ts"]),
regime=row["regime"],
composite_hash=_parse_int(row["composite_hash"]),
scalar_hash=_parse_int(row["scalar_hash"]),
confidence=_parse_float(row["confidence"]),
raw_vel_div=_parse_float(row["raw_vel_div"]),
raw_dvol=_parse_float(row["raw_dvol"]),
final_score=_parse_float(row["final_score"]),
)
)
return out
def attach_fingerprint(trades: list[TradeRow], fps: list[FingerprintRow]) -> list[TradeRow]:
fp_ts = [fp.ts for fp in fps]
out: list[TradeRow] = []
for trade in trades:
idx = bisect.bisect_right(fp_ts, trade.ts) - 1
if idx >= 0:
fp = fps[idx]
trade = dataclasses.replace(
trade,
composite_hash=fp.composite_hash,
scalar_hash=fp.scalar_hash,
regime=fp.regime,
fingerprint_confidence=fp.confidence,
fingerprint_vel_div=fp.raw_vel_div,
fingerprint_dvol=fp.raw_dvol,
)
out.append(trade)
return out
def build_trade_set() -> tuple[list[TradeRow], dict[str, Any]]:
trades = load_clean_trades()
if not trades:
raise RuntimeError("no clean trades returned from ClickHouse")
eq = load_exec_quality()
enriched: list[TradeRow] = []
for trade in trades:
meta = eq.get(trade.trade_id, {})
notional = _parse_float(str(meta.get("notional_quote", 0.0)), 0.0)
if notional <= 0:
notional = abs(trade.entry_price * trade.quantity)
enriched.append(
dataclasses.replace(
trade,
notional_quote=notional,
fill_quality_score=meta.get("fill_quality_score"),
fill_quality_class=meta.get("fill_quality_class"),
fill_rows=int(meta.get("fill_rows", 0)),
taker_fill_rows=int(meta.get("taker_fill_rows", 0)),
maker_fill_rows=int(meta.get("maker_fill_rows", 0)),
)
)
fps = load_fingerprints(trades[0].ts - timedelta(hours=1), trades[-1].ts)
enriched = attach_fingerprint(enriched, fps)
return enriched, {
"trade_rows": len(trades),
"exec_quality_rows": len(eq),
"fingerprint_rows": len(fps),
}
def slippage_bps_model(notional: float, median_notional: float, *, taker_only: bool = True) -> float:
"""Conservative parametric impact proxy.
Direct slippage telemetry in trade_execution_quality is mostly null for this
dataset, so we assume a taker-heavy execution floor and a sublinear
size-dependent impact term.
"""
base = 0.35 if taker_only else 0.20
impact = 0.15 * math.sqrt(max(notional, 1.0) / max(median_notional, 1.0))
return base + impact
def replay_equity(
trades: Sequence[TradeRow],
fraction: float,
*,
slippage_enabled: bool = True,
median_notional: float,
ruin_threshold: float = 0.50,
) -> dict[str, Any]:
if not trades:
raise ValueError("no trades")
start_capital = trades[0].capital_before if trades[0].capital_before > 0 else 69_000.0
capital = float(start_capital)
equity_points: list[tuple[datetime, float]] = []
daily_close: dict[str, float] = {}
daily_peak: dict[str, float] = {}
clipped = 0
total = 0
returns: list[float] = []
daily_equity: dict[str, float] = {}
for trade in trades:
total += 1
leverage_eff = min(trade.leverage, TRANSLATOR_CAP / max(fraction, 1e-12))
if trade.leverage * fraction > TRANSLATOR_CAP:
clipped += 1
notional = capital * fraction * leverage_eff
slip_bps = slippage_bps_model(
notional,
median_notional,
taker_only=(trade.taker_fill_rows > 0 or trade.fill_rows > 0),
) if slippage_enabled else 0.0
trade_return = fraction * leverage_eff * (trade.pnl_pct - slip_bps / 10_000.0)
capital *= 1.0 + trade_return
returns.append(trade_return)
equity_points.append((trade.ts, capital))
day = trade.ts.date().isoformat()
daily_equity[day] = capital
# Fill daily series from first to last trade day.
first_day = trades[0].ts.date()
last_day = trades[-1].ts.date()
day = first_day
last_equity = start_capital
daily_series: list[tuple[str, float]] = []
while day <= last_day:
key = day.isoformat()
if key in daily_equity:
last_equity = daily_equity[key]
daily_series.append((key, last_equity))
day += timedelta(days=1)
daily_returns = []
prev = start_capital
for _, eq in daily_series:
daily_returns.append((eq / prev) - 1.0 if prev else 0.0)
prev = eq
peak = start_capital
max_dd = 0.0
underwater_start: str | None = None
longest_underwater_days = 0.0
current_underwater_days = 0.0
prev_day: str | None = None
for day_str, eq in daily_series:
if eq >= peak:
if underwater_start is not None and prev_day is not None:
start = datetime.fromisoformat(underwater_start).date()
end = datetime.fromisoformat(prev_day).date()
longest_underwater_days = max(
longest_underwater_days, (end - start).days + 1
)
peak = eq
underwater_start = None
current_underwater_days = 0.0
else:
if underwater_start is None:
underwater_start = day_str
current_underwater_days += 1.0
max_dd = max(max_dd, 1.0 - eq / peak if peak > 0 else 0.0)
prev_day = day_str
if underwater_start is not None and prev_day is not None:
start = datetime.fromisoformat(underwater_start).date()
end = datetime.fromisoformat(prev_day).date()
longest_underwater_days = max(longest_underwater_days, (end - start).days + 1)
years = max((trades[-1].ts - trades[0].ts).total_seconds() / (365.25 * 24 * 3600), 1.0 / 365.25)
if capital > 0 and start_capital > 0:
log_growth = math.log(capital / start_capital)
annual_log = log_growth / years
if annual_log > 700.0:
cagr = float("inf")
elif annual_log < -700.0:
cagr = -1.0
else:
cagr = math.expm1(annual_log)
else:
cagr = -1.0
ann_sharpe, ann_sortino, downside_dev = _daily_risk_metrics(daily_returns)
ruin_prob = _bootstrap_ruin_prob(
trades=trades,
fraction=fraction,
median_notional=median_notional,
slippage_enabled=slippage_enabled,
ruin_threshold=ruin_threshold,
)
return {
"fraction": fraction,
"start_capital": start_capital,
"final_capital": capital,
"cagr": cagr,
"max_drawdown": max_dd,
"calmar": (cagr / max_dd) if max_dd > 0 else float("inf"),
"sharpe": ann_sharpe,
"sortino": ann_sortino,
"downside_deviation": downside_dev,
"ruin_prob": ruin_prob,
"pct_trades_clipped_at_3x": (clipped / total * 100.0) if total else 0.0,
"longest_underwater_days": longest_underwater_days,
"n_trades": total,
"daily_returns": daily_returns,
"equity_points": equity_points,
"returns": returns,
}
def _daily_risk_metrics(daily_returns: Sequence[float]) -> tuple[float, float, float]:
if len(daily_returns) < 2:
return 0.0, 0.0, 0.0
arr = np.asarray(daily_returns, dtype=np.float64)
mean = float(np.mean(arr))
std = float(np.std(arr, ddof=1)) if len(arr) > 1 else 0.0
downside = arr[arr < 0.0]
downside_dev = float(np.std(downside, ddof=1)) if len(downside) > 1 else float(np.std(np.minimum(arr, 0.0), ddof=0))
sharpe = (mean / std * math.sqrt(365.25)) if std > 0 else 0.0
sortino = (mean / downside_dev * math.sqrt(365.25)) if downside_dev > 0 else 0.0
return sharpe, sortino, downside_dev
def _bootstrap_ruin_prob(
*,
trades: Sequence[TradeRow],
fraction: float,
median_notional: float,
slippage_enabled: bool,
ruin_threshold: float,
n_boot: int = 256,
seed: int = 17,
) -> float:
rng = np.random.default_rng(seed)
n = len(trades)
if n == 0:
return 0.0
leverage = np.array([min(t.leverage, TRANSLATOR_CAP / max(fraction, 1e-12)) for t in trades], dtype=np.float64)
pnl_pct = np.array([t.pnl_pct for t in trades], dtype=np.float64)
taker_only = np.array([(t.taker_fill_rows > 0 or t.fill_rows > 0) for t in trades], dtype=np.float64)
ruin = 0
for _ in range(n_boot):
idx = rng.integers(0, n, size=n)
capital = 1.0
floor = ruin_threshold
for j in idx:
lev = leverage[j]
notional = capital * fraction * lev
slip = slippage_bps_model(notional, median_notional, taker_only=bool(taker_only[j])) if slippage_enabled else 0.0
r = fraction * lev * (pnl_pct[j] - slip / 10_000.0)
capital *= 1.0 + r
if capital <= floor:
ruin += 1
break
return ruin / n_boot
def cap_binding_curve(trades: Sequence[TradeRow], fractions: Sequence[float]) -> list[dict[str, float]]:
out = []
n = len(trades)
for f in fractions:
clipped = sum(1 for t in trades if t.leverage * f > TRANSLATOR_CAP)
out.append({"fraction": float(f), "pct_clipped": (clipped / n * 100.0) if n else 0.0})
return out
def _group_by_hash(trades: Sequence[TradeRow]) -> dict[int, list[TradeRow]]:
groups: dict[int, list[TradeRow]] = defaultdict(list)
for trade in trades:
if trade.composite_hash is None:
continue
groups[int(trade.composite_hash)].append(trade)
for arr in groups.values():
arr.sort(key=lambda t: t.ts)
return groups
def _bucket_stats(trades: Sequence[TradeRow], fraction: float, median_notional: float) -> dict[str, Any]:
groups = _group_by_hash(trades)
bucket_results = []
for h, rows in groups.items():
if len(rows) < 8:
continue
rep = replay_equity(rows, fraction, slippage_enabled=True, median_notional=median_notional)
bucket_results.append(
{
"composite_hash": int(h),
"n_trades": len(rows),
"final_capital": rep["final_capital"],
"max_drawdown": rep["max_drawdown"],
"cagr": rep["cagr"],
"calmar": rep["calmar"],
"ruin_prob": rep["ruin_prob"],
}
)
bucket_results.sort(key=lambda x: (x["max_drawdown"], -x["n_trades"]), reverse=True)
return {
"bucket_results": bucket_results,
"worst_bucket": bucket_results[0] if bucket_results else None,
}
def slippage_model_summary(trades: Sequence[TradeRow]) -> dict[str, Any]:
notionals = np.array([t.notional_quote or abs(t.entry_price * t.quantity) for t in trades], dtype=np.float64)
if len(notionals) == 0:
median_notional = 1.0
else:
median_notional = float(np.median(notionals))
taker_fill_rate = float(
sum(1 for t in trades if t.taker_fill_rows > 0 or t.fill_rows > 0) / max(len(trades), 1)
)
observed_direct = sum(
1 for t in trades if math.isfinite(float(t.fill_quality_score or 0.0)) and (t.fill_quality_score or 0.0) not in (0.0, None)
)
return {
"type": "conservative_parametric_proxy",
"observed_direct_slippage_rows": 0,
"observed_direct_slippage_trade_rows": observed_direct,
"taker_fill_rate": taker_fill_rate,
"median_notional_quote": median_notional,
"formula": "slippage_bps = 0.35 + 0.15 * sqrt(notional / median_notional)",
"assumption": "execution-quality rows do not expose non-null slippage_bps; all fill rows are taker, so this is a conservative proxy",
}
def analyze() -> dict[str, Any]:
trades, counts = build_trade_set()
median_notional = float(np.median([t.notional_quote or abs(t.entry_price * t.quantity) for t in trades]))
fractions = np.unique(np.concatenate([BASE_GRID, np.round(np.arange(0.20, 0.501, 0.01), 3)])).astype(np.float64)
slippage_summary = slippage_model_summary(trades)
results_no_slip = []
results_slip = []
for f in fractions:
results_no_slip.append(replay_equity(trades, float(f), slippage_enabled=False, median_notional=median_notional))
results_slip.append(replay_equity(trades, float(f), slippage_enabled=True, median_notional=median_notional))
best_slip = max(results_slip, key=lambda x: (x["calmar"], x["final_capital"]))
best_no_slip = max(results_no_slip, key=lambda x: (x["calmar"], x["final_capital"]))
refined = np.unique(
np.concatenate([
fractions,
np.round(np.arange(max(0.20, best_slip["fraction"] - 0.03), min(0.50, best_slip["fraction"] + 0.03) + 0.0001, 0.005), 3),
])
).astype(np.float64)
if len(refined) > len(fractions):
results_slip = [replay_equity(trades, float(f), slippage_enabled=True, median_notional=median_notional) for f in refined]
results_no_slip = [replay_equity(trades, float(f), slippage_enabled=False, median_notional=median_notional) for f in refined]
fractions = refined
best_slip = max(results_slip, key=lambda x: (x["calmar"], x["final_capital"]))
best_no_slip = max(results_no_slip, key=lambda x: (x["calmar"], x["final_capital"]))
cap_curve = cap_binding_curve(trades, fractions)
bucket_summary = _bucket_stats(trades, float(best_slip["fraction"]), median_notional)
cap_by_fraction = {round(float(row["fraction"]), 3): float(row["pct_clipped"]) for row in cap_curve}
feasible_rows = [r for r in results_slip if cap_by_fraction.get(round(float(r["fraction"]), 3), 0.0) == 0.0]
practical_best = max(feasible_rows, key=lambda x: (x["calmar"], x["final_capital"])) if feasible_rows else best_slip
# Kelly anchor: use unit-fraction returns (return at f=1.0, ignoring cap/slip)
unit_returns = np.array([t.pnl_pct * min(t.leverage, TRANSLATOR_CAP / 1.0) for t in trades], dtype=np.float64)
kelly_grid = np.linspace(0.01, 0.50, 200)
kelly_log_growth = []
for f in kelly_grid:
growth = np.mean(np.log1p(np.clip(f * unit_returns, -0.95, None)))
kelly_log_growth.append(float(growth))
kelly_idx = int(np.argmax(kelly_log_growth))
kelly_fraction = float(kelly_grid[kelly_idx])
fractional_kelly = float(min(best_slip["fraction"], max(0.25 * kelly_fraction, 0.20)))
# Stress scenario: worst-hash bucket loss multiplied slightly and injected once.
stress = None
if bucket_summary["worst_bucket"] is not None:
worst_hash = bucket_summary["worst_bucket"]["composite_hash"]
worst_rows = sorted([t for t in trades if t.composite_hash == worst_hash], key=lambda t: t.ts)
if worst_rows:
worst_trade = min(worst_rows, key=lambda t: t.pnl_pct)
stress_trade = dataclasses.replace(worst_trade, pnl_pct=min(-0.01, worst_trade.pnl_pct * 1.5))
stress_rows = list(worst_rows) + [stress_trade]
stress_rows.sort(key=lambda t: t.ts)
stress = {
"worst_hash": int(worst_hash),
"worst_trade_id": worst_trade.trade_id,
"stress_pnl_pct": float(stress_trade.pnl_pct),
"per_fraction": [
{
"fraction": float(f),
"final_capital": replay_equity(stress_rows, float(f), slippage_enabled=True, median_notional=median_notional)["final_capital"],
"ruin_prob": replay_equity(stress_rows, float(f), slippage_enabled=True, median_notional=median_notional)["ruin_prob"],
}
for f in fractions
],
}
recommendation = {
"recommended_fraction": float(practical_best["fraction"]),
"recommended_basis": "best_calmar_among_zero_clip_fractions",
"recommended_final_capital": float(practical_best["final_capital"]),
"recommended_cagr": float(practical_best["cagr"]),
"recommended_max_drawdown": float(practical_best["max_drawdown"]),
"recommended_calmar": float(practical_best["calmar"]),
"recommended_ruin_prob": float(practical_best["ruin_prob"]),
"recommended_clip_pct": float(cap_by_fraction.get(round(float(practical_best["fraction"]), 3), 0.0)),
"alt_fraction_floor": float(fractional_kelly),
"no_slippage_best_fraction": float(best_no_slip["fraction"]),
"slippage_best_fraction": float(best_slip["fraction"]),
"optimizer_fraction": float(best_slip["fraction"]),
"optimizer_final_capital": float(best_slip["final_capital"]),
"optimizer_cagr": float(best_slip["cagr"]),
"optimizer_max_drawdown": float(best_slip["max_drawdown"]),
"optimizer_calmar": float(best_slip["calmar"]),
"optimizer_ruin_prob": float(best_slip["ruin_prob"]),
"optimizer_clip_pct": float(next(c for c in cap_curve if abs(c["fraction"] - best_slip["fraction"]) < 1e-9)["pct_clipped"]),
"delta_final_capital_vs_base_0p20": float(
next(r for r in results_slip if abs(r["fraction"] - 0.20) < 1e-9)["final_capital"]
),
}
return {
"generated_at": datetime.now(timezone.utc).isoformat(),
"study_spec": "prod/docs/VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md",
"source_counts": counts,
"clean_trade_count": len(trades),
"clean_trade_window": {
"start": trades[0].ts.isoformat(),
"end": trades[-1].ts.isoformat(),
},
"slippage_model": slippage_summary,
"cap_curve": cap_curve,
"results": {
"no_slippage": results_no_slip,
"slippage_adjusted": results_slip,
},
"kelly": {
"kelly_fraction": kelly_fraction,
"fractional_kelly_anchor": fractional_kelly,
"kelly_log_growth_grid": [{"fraction": float(f), "log_growth": float(g)} for f, g in zip(kelly_grid, kelly_log_growth)],
},
"bucket_summary": bucket_summary,
"stress_scenario": stress,
"recommendation": recommendation,
}
def _best_row(rows: Sequence[dict[str, Any]]) -> dict[str, Any]:
return max(rows, key=lambda x: (x["calmar"], x["final_capital"]))
def _format_pct(x: float) -> str:
return f"{x * 100.0:.2f}%"
def write_report(report: dict[str, Any]) -> tuple[Path, Path]:
REPORTS_DIR.mkdir(parents=True, exist_ok=True)
ts = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
json_path = REPORTS_DIR / f"base_fraction_study_{ts}.json"
md_path = REPORTS_DIR / f"base_fraction_study_{ts}.md"
json_path.write_text(json.dumps(report, indent=2, sort_keys=True, default=str))
best = report["recommendation"]
base = next(r for r in report["results"]["slippage_adjusted"] if abs(r["fraction"] - 0.20) < 1e-9)
recommended_cap = next(c for c in report["cap_curve"] if abs(c["fraction"] - best["recommended_fraction"]) < 1e-9)
optimizer_cap = next(c for c in report["cap_curve"] if abs(c["fraction"] - best["optimizer_fraction"]) < 1e-9)
md = []
md.append("# VIOLET base-fraction sizing study")
md.append("")
md.append(f"Generated: `{report['generated_at']}`")
md.append("")
md.append("## Bottom line")
md.append(
f"- Recommended fraction: `{best['recommended_fraction']:.3f}` "
f"(best Calmar among zero-clip fractions)"
)
md.append(
f"- Base 0.20 final capital: `{base['final_capital']:.2f}`; "
f"recommended final capital: `{best['recommended_final_capital']:.2f}`"
)
md.append(
f"- Base 0.20 maxDD: `{_format_pct(base['max_drawdown'])}`; "
f"recommended maxDD: `{_format_pct(best['recommended_max_drawdown'])}`"
)
md.append(
f"- Recommended clip rate: `{recommended_cap['pct_clipped']:.2f}%`; "
f"unconstrained optimizer: `{best['optimizer_fraction']:.3f}` with "
f"`{optimizer_cap['pct_clipped']:.2f}%` clipped at the 3x ceiling."
)
md.append("")
md.append("## Caveat")
md.append(
"- Direct non-null slippage telemetry was not available in `trade_execution_quality`; "
"the study uses a conservative taker-heavy impact proxy."
)
md.append("")
md.append("## Cap binding")
md.append(
f"- The recommended fraction hits the 3x translator cap on `{recommended_cap['pct_clipped']:.2f}%` of trades."
)
md.append("")
md.append("## Kelly anchor")
md.append(
f"- Empirical Kelly anchor: `{report['kelly']['kelly_fraction']:.3f}`; "
f"fractional anchor: `{report['kelly']['fractional_kelly_anchor']:.3f}`"
)
md.append("")
md.append("## Files")
md.append(f"- JSON: `{json_path}`")
md.append(f"- Markdown: `{md_path}`")
md_path.write_text("\n".join(md) + "\n")
return json_path, md_path
def self_test() -> None:
trades = [
TradeRow(
trade_id="t1",
ts=datetime(2026, 1, 1, 0, 0, tzinfo=timezone.utc),
asset="X",
side="SHORT",
entry_price=100.0,
exit_price=99.0,
quantity=1.0,
pnl=10.0,
pnl_pct=0.10,
exit_reason="FIXED_TP",
leverage=2.0,
capital_before=100.0,
capital_after=110.0,
bars_held=5,
regime_signal=-1,
vel_div_entry=-0.03,
boost_at_entry=1.0,
beta_at_entry=1.0,
posture="APEX",
our_leverage=0.4,
composite_hash=1,
notional_quote=100.0,
taker_fill_rows=1,
fill_rows=1,
),
TradeRow(
trade_id="t2",
ts=datetime(2026, 1, 2, 0, 0, tzinfo=timezone.utc),
asset="X",
side="SHORT",
entry_price=100.0,
exit_price=101.0,
quantity=1.0,
pnl=-5.0,
pnl_pct=-0.01,
exit_reason="MAX_HOLD",
leverage=7.0,
capital_before=110.0,
capital_after=105.0,
bars_held=5,
regime_signal=-1,
vel_div_entry=-0.03,
boost_at_entry=1.0,
beta_at_entry=1.0,
posture="APEX",
our_leverage=0.6,
composite_hash=1,
notional_quote=110.0,
taker_fill_rows=1,
fill_rows=1,
),
]
rep = replay_equity(trades, 0.20, slippage_enabled=False, median_notional=100.0, ruin_threshold=0.50)
assert rep["n_trades"] == 2
assert rep["pct_trades_clipped_at_3x"] == 0.0
assert rep["final_capital"] > 100.0
cap = cap_binding_curve(trades, [0.20, 0.50])
assert cap[0]["pct_clipped"] == 0.0
assert cap[1]["pct_clipped"] == 50.0
slip = slippage_bps_model(100.0, 100.0)
assert slip > 0.0
def main(argv: Sequence[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--self-test", action="store_true", help="run the deterministic synthetic fixture and exit")
parser.add_argument("--dry-run", action="store_true", help="run the live analysis but do not write files")
args = parser.parse_args(argv)
if args.self_test:
self_test()
print("self-test ok")
return 0
report = analyze()
if args.dry_run:
print(json.dumps(report["recommendation"], indent=2, sort_keys=True, default=str))
return 0
json_path, md_path = write_report(report)
print(json_path)
print(md_path)
print(json.dumps(report["recommendation"], indent=2, sort_keys=True, default=str))
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -32,6 +32,7 @@ from .contracts import (
VenueEventStatus,
VenueOrder,
VenueOrderStatus,
VenueTelemetrySnapshot,
)
from .journal import ClickHouseKernelJournal, KernelJournal, MemoryKernelJournal
from .rust_backend import ExecutionKernel
@@ -89,6 +90,7 @@ __all__ = [
"VenueEventStatus",
"VenueOrder",
"VenueOrderStatus",
"VenueTelemetrySnapshot",
"ZincPlane",
"ZincControlPlane",
"build_position_state_row",

View File

@@ -0,0 +1,106 @@
"""ASEx wrapper for AccountProjectionV2 — serializes the 4 P0 accumulator sites."""
from __future__ import annotations
import time
from concurrent.futures import Future
from typing import Any, Iterable, List, Optional
from asex.guarded import ASExGuardedState
from asex.worker import ASExWorker
from .account import AccountProjectionV2, AccountSnapshotV2, EPosition, ReconcileConfig, TradeSlot
class _AccountBackend(ASExGuardedState[dict, Any]):
"""Wraps AccountProjectionV2 mutations as ASEx operations."""
def __init__(self, seed_capital: float, **kw):
super().__init__()
self._proj = AccountProjectionV2(seed_capital, **kw)
def _validate(self, mutation: dict) -> bool:
return isinstance(mutation, dict) and "op" in mutation
def _apply(self, mutation: dict) -> Any:
op = mutation["op"]
args = mutation.get("args", {})
if op == "apply_fill":
self._proj.apply_fill(**args); return None
elif op == "apply_funding":
self._proj.apply_funding(**args); return None
elif op == "apply_balance_update":
self._proj.apply_balance_update(**args); return None
elif op == "apply_position_update":
self._proj.apply_position_update(**args); return None
elif op == "build_snapshot":
return self._proj.build_snapshot(**args)
raise ValueError(f"Unknown ASEx account op: {op}")
def __getattr__(self, name):
return getattr(self._proj, name)
class ASEXAccountV2:
"""ASEx-serialized wrapper around ``AccountProjectionV2``.
All mutations go through an ``ASExWorker`` — one thread, sequential,
no races. Property reads bypass the worker (lock-free, fast).
"""
def __init__(self, seed_capital: float, *, min_capital: float = 0.0,
max_capital: float | None = None,
reconcile_config: ReconcileConfig | None = None):
self._backend = _AccountBackend(seed_capital, min_capital=min_capital,
max_capital=max_capital, reconcile_config=reconcile_config)
self._worker: ASExWorker = ASExWorker(self._backend, daemon=True)
def apply_fill_async(self, **kw) -> Future:
return self._worker.mutate({"op": "apply_fill", "args": kw})
def apply_funding_async(self, amount: float) -> Future:
return self._worker.mutate({"op": "apply_funding", "args": {"amount": amount}})
def apply_balance_update_async(self, **kw) -> Future:
return self._worker.mutate({"op": "apply_balance_update", "args": kw})
def apply_position_update_async(self, positions: List[EPosition]) -> Future:
return self._worker.mutate({"op": "apply_position_update", "args": {"positions": positions}})
def build_snapshot_async(self, **kw) -> Future:
return self._worker.mutate({"op": "build_snapshot", "args": kw})
def apply_fill(self, **kw) -> None:
self.apply_fill_async(**kw).result(timeout=30)
def apply_funding(self, amount: float) -> None:
self.apply_funding_async(amount).result(timeout=30)
def apply_balance_update(self, **kw) -> None:
self.apply_balance_update_async(**kw).result(timeout=30)
def apply_position_update(self, positions: List[EPosition]) -> None:
self.apply_position_update_async(positions).result(timeout=30)
def build_snapshot(self, **kw) -> AccountSnapshotV2:
return self.build_snapshot_async(**kw).result(timeout=30)
@property
def snapshot(self) -> AccountSnapshotV2:
return self._backend._proj.snapshot
@property
def k_capital(self) -> float:
return self._backend._proj.k_capital
@property
def applied(self) -> int:
return self._backend.applied
def close(self, *, timeout: float | None = None) -> None:
self._worker.close(timeout=timeout)
def __enter__(self) -> ASEXAccountV2:
return self
def __exit__(self, *args) -> None:
self.close(timeout=5.0)

View File

@@ -27,6 +27,7 @@ from .contracts import (
KernelEventKind,
KernelIntent,
TradeSide,
VenueTelemetrySnapshot,
VenueEvent,
VenueEventStatus,
VenueOrder,
@@ -226,7 +227,7 @@ class BingxVenueAdapter(VenueAdapter):
)
return cls._EXECUTOR
def __init__(self, backend: Any | None = None, *, config: Any | None = None) -> None:
def __init__(self, backend: Any | None = None, *, config: Any | None = None, zinc_plane: Any | None = None) -> None:
if backend is None:
if config is None:
raise ValueError("BingxVenueAdapter requires a backend or config")
@@ -234,6 +235,7 @@ class BingxVenueAdapter(VenueAdapter):
backend = BingxDirectExecutionAdapter(config)
self.backend = backend
self._telemetry_plane = zinc_plane
self._event_seq = itertools.count(1)
# Thread-safe snapshot cache — reads from a snapshot may arrive from
# the kernel thread while _backend_snapshot writes from the pool thread.
@@ -279,6 +281,73 @@ class BingxVenueAdapter(VenueAdapter):
except Exception:
pass
def set_telemetry_plane(self, zinc_plane: Any | None) -> None:
self._telemetry_plane = zinc_plane
def _publish_telemetry(
self,
*,
phase: str,
status: str,
intent: KernelIntent | None = None,
order: VenueOrder | None = None,
endpoint: str = "",
method: str = "",
message: str = "",
retry_after_ms: int = 0,
venue_order_status: str = "",
venue_event_kind: str = "",
details: dict[str, Any] | None = None,
) -> None:
plane = self._telemetry_plane
publish = getattr(plane, "publish_venue", None) if plane is not None else None
if publish is None:
return
slot_id = 0
trade_id = ""
asset = ""
side = TradeSide.FLAT
action = ""
intent_id = ""
if intent is not None:
slot_id = int(getattr(intent, "slot_id", 0) or 0)
trade_id = str(getattr(intent, "trade_id", "") or "")
asset = str(getattr(intent, "asset", "") or "")
side = getattr(intent, "side", TradeSide.FLAT)
action = str(getattr(intent, "action", "") or "")
intent_id = str(getattr(intent, "intent_id", "") or "")
if order is not None:
slot_id = int(order.metadata.get("slot_id", slot_id) or slot_id)
trade_id = str(order.internal_trade_id or trade_id)
asset = str(order.metadata.get("asset") or asset)
side = order.side or side
try:
publish(
VenueTelemetrySnapshot(
phase=phase,
status=status,
venue="bingx",
endpoint=endpoint,
method=method,
intent_id=intent_id,
trade_id=trade_id,
slot_id=slot_id,
asset=asset,
side=side,
action=action,
order_id=str(getattr(order, "venue_order_id", "") or ""),
client_order_id=str(getattr(order, "venue_client_id", "") or ""),
venue_order_status=venue_order_status,
venue_event_kind=venue_event_kind,
message=message,
retry_after_ms=int(retry_after_ms or 0),
timestamp=datetime.now(timezone.utc),
details=dict(details or {}),
)
)
except Exception:
pass
def _call_backend(self, method_name: str, *args: Any, **kwargs: Any) -> Any:
method = getattr(self.backend, method_name, None)
if method is None:
@@ -368,18 +437,37 @@ class BingxVenueAdapter(VenueAdapter):
was fixed for submit via submit_async. This version awaits backend.cancel()
directly in the caller's (main) event loop.
"""
self._publish_telemetry(
phase="cancel:start",
status="REQUESTED",
order=order,
endpoint="/openApi/swap/v2/trade/order",
method="DELETE",
message=reason,
details={"asset": str(order.metadata.get("asset") or "")},
)
cancel_fn = getattr(self.backend, "cancel", None)
if cancel_fn is not None:
response = await cancel_fn(order, reason=reason)
else:
response = None
return self._events_from_cancel(order, response, None, None, reason=reason)
events = self._events_from_cancel(order, response, None, None, reason=reason)
return events
def cancel(self, order: VenueOrder, *, reason: str = "") -> List[VenueEvent]:
# _events_from_cancel never reads before/after — snapshots are dead weight.
# NOTE: if backend.cancel is async (BingxDirectExecutionAdapter), this sync
# path goes through the thread-pool and will deadlock in a running event loop.
# Use cancel_async() from async contexts (process_intent_async already does).
self._publish_telemetry(
phase="cancel:start",
status="REQUESTED",
order=order,
endpoint="/openApi/swap/v2/trade/order",
method="DELETE",
message=reason,
details={"asset": str(order.metadata.get("asset") or "")},
)
response = None
if hasattr(self.backend, "cancel"):
response = self._call_backend("cancel", order, reason=reason)
@@ -410,7 +498,8 @@ class BingxVenueAdapter(VenueAdapter):
except BingxHttpError as exc:
# W10: map HTTP error class to status — 429/5xx are transient, 4xx are real rejections
response = {"status": _http_error_status(str(exc)), "msg": str(exc), "orderId": order.venue_order_id, "clientOrderId": order.venue_client_id}
return self._events_from_cancel(order, response, None, None, reason=reason)
events = self._events_from_cancel(order, response, None, None, reason=reason)
return events
def open_orders(self) -> List[VenueOrder]:
# Use backend._state (populated by await backend.connect()) rather than
@@ -458,6 +547,13 @@ class BingxVenueAdapter(VenueAdapter):
# entirely: the FSM stayed fill-blind (slot size 0 in ENTRY_WORKING),
# the DecisionEngine saw "no position", and re-entered → the live
# double-entries at 15:20 and 17:24 UTC.
self._publish_telemetry(
phase="reconcile:start",
status="REQUESTED",
endpoint="/openApi/swap/v2/trade/openOrders",
method="GET",
details={"include_history": True},
)
recon_symbol = None
kernel = getattr(self, "_kernel_ref", None)
if kernel is not None:
@@ -474,13 +570,60 @@ class BingxVenueAdapter(VenueAdapter):
except Exception as exc:
import logging as _log
_log.getLogger(__name__).warning("reconcile: refresh_state failed: %s", exc)
self._publish_telemetry(
phase="reconcile:error",
status="ERROR",
endpoint="/openApi/swap/v2/trade/openOrders",
method="GET",
message=str(exc),
details={"symbol": recon_symbol or ""},
)
return []
self._publish_telemetry(
phase="reconcile:done",
status="OK",
endpoint="/openApi/swap/v2/trade/openOrders",
method="GET",
details={
"symbol": recon_symbol or "",
"open_orders": len(getattr(snapshot, "open_orders", []) or []),
"positions": len(getattr(snapshot, "open_positions", {}) or {}),
"fills": len(getattr(snapshot, "all_fills", []) or []),
},
)
return self._events_from_snapshot(snapshot)
def submit(self, intent: KernelIntent) -> List[VenueEvent]:
# Snapshots dropped: receipt executedQty fields take precedence (same as submit_async)
self._publish_telemetry(
phase="submit:start",
status="REQUESTED",
intent=intent,
endpoint="/openApi/swap/v2/trade/order",
method="POST",
details={"action": intent.action.value, "order_type": str(getattr(intent, "order_type", "MARKET") or "MARKET")},
)
receipt = self._call_backend("submit_intent", self._legacy_intent(intent))
return self._events_from_submit(intent, receipt, None, None)
events = self._events_from_submit(intent, receipt, None, None)
ack_row = dict(getattr(receipt, "raw_ack", {}) or {})
self._publish_telemetry(
phase="submit:done",
status=str(getattr(receipt, "status", "") or _row_text(ack_row, "status", default="NEW")),
intent=intent,
endpoint="/openApi/swap/v2/trade/order",
method="POST",
message=_row_text(ack_row, "msg", "message", default=""),
order_id=_row_text(ack_row, "orderId", "orderID", default=str(getattr(receipt, "order_id", "") or "")),
client_order_id=_row_text(ack_row, "clientOrderID", "clientOrderId", default=str(getattr(receipt, "client_order_id", "") or intent.intent_id)),
venue_order_status=str(getattr(receipt, "status", "") or _row_text(ack_row, "status", default="")),
venue_event_kind=events[1].kind.value if len(events) > 1 else events[0].kind.value,
details={
"filled_size": float((events[1].filled_size if len(events) > 1 else events[0].filled_size) or 0.0),
"event_count": len(events),
"asset": intent.asset,
},
)
return events
async def submit_async(self, intent: KernelIntent) -> List[VenueEvent]:
"""Async submit — runs in the caller's event loop, no thread-pool deadlock.
@@ -495,8 +638,35 @@ class BingxVenueAdapter(VenueAdapter):
Passing None for snapshots makes _filled_size_from_snapshots return 0.0
(a safe fallback; the receipt fields take precedence).
"""
self._publish_telemetry(
phase="submit:start",
status="REQUESTED",
intent=intent,
endpoint="/openApi/swap/v2/trade/order",
method="POST",
details={"action": intent.action.value, "order_type": str(getattr(intent, "order_type", "MARKET") or "MARKET")},
)
receipt = await self.backend.submit_intent(self._legacy_intent(intent))
return self._events_from_submit(intent, receipt, None, None)
events = self._events_from_submit(intent, receipt, None, None)
ack_row = dict(getattr(receipt, "raw_ack", {}) or {})
self._publish_telemetry(
phase="submit:done",
status=str(getattr(receipt, "status", "") or _row_text(ack_row, "status", default="NEW")),
intent=intent,
endpoint="/openApi/swap/v2/trade/order",
method="POST",
message=_row_text(ack_row, "msg", "message", default=""),
order_id=_row_text(ack_row, "orderId", "orderID", default=str(getattr(receipt, "order_id", "") or "")),
client_order_id=_row_text(ack_row, "clientOrderID", "clientOrderId", default=str(getattr(receipt, "client_order_id", "") or intent.intent_id)),
venue_order_status=str(getattr(receipt, "status", "") or _row_text(ack_row, "status", default="")),
venue_event_kind=events[1].kind.value if len(events) > 1 else events[0].kind.value,
details={
"filled_size": float((events[1].filled_size if len(events) > 1 else events[0].filled_size) or 0.0),
"event_count": len(events),
"asset": intent.asset,
},
)
return events
def _events_from_submit(self, intent: KernelIntent, receipt: Any, before, after) -> List[VenueEvent]: # noqa: ANN001
ack_row = dict(getattr(receipt, "raw_ack", {}) or {})
@@ -612,6 +782,18 @@ class BingxVenueAdapter(VenueAdapter):
raw = response if isinstance(response, dict) else {}
status = _normalize_status(_row_text(raw, "status", default="CANCELED"))
if status in {"RATE_LIMITED", "THROTTLED"}:
self._publish_telemetry(
phase="cancel:done",
status=status,
order=order,
endpoint="/openApi/swap/v2/trade/order",
method="DELETE",
message=reason or _row_text(raw, "msg", "message", default="BINGX_RATE_LIMITED"),
venue_order_status=VenueEventStatus.RATE_LIMITED.value,
venue_event_kind=KernelEventKind.RATE_LIMITED.value,
retry_after_ms=_rate_limit_retry_after_ms(raw),
details={"order_status": status, "asset": str(order.metadata.get("asset") or "")},
)
return [
VenueEvent(
timestamp=datetime.now(timezone.utc),
@@ -637,6 +819,18 @@ class BingxVenueAdapter(VenueAdapter):
kind = KernelEventKind.CANCEL_ACK if event_status == VenueEventStatus.CANCELED else KernelEventKind.CANCEL_REJECT
if event_status == VenueEventStatus.CANCELED_REJECTED:
kind = KernelEventKind.CANCEL_REJECT
self._publish_telemetry(
phase="cancel:done",
status=status or event_status.value,
order=order,
endpoint="/openApi/swap/v2/trade/order",
method="DELETE",
message=reason or _row_text(raw, "msg", "message", default=""),
venue_order_status=event_status.value,
venue_event_kind=kind.value,
retry_after_ms=_rate_limit_retry_after_ms(raw),
details={"order_status": status, "asset": str(order.metadata.get("asset") or "")},
)
return [
VenueEvent(
timestamp=datetime.now(timezone.utc),

View File

@@ -144,6 +144,54 @@ class VenueOrder:
return max(0.0, float(self.intended_size) - float(self.filled_size))
@dataclass(frozen=True)
class VenueTelemetrySnapshot:
"""Shared-memory surface for venue-side call boundaries and state."""
phase: str = "idle"
status: str = "IDLE"
venue: str = "bingx"
endpoint: str = ""
method: str = ""
intent_id: str = ""
trade_id: str = ""
slot_id: int = 0
asset: str = ""
side: TradeSide = TradeSide.FLAT
action: str = ""
order_id: str = ""
client_order_id: str = ""
venue_order_status: str = ""
venue_event_kind: str = ""
message: str = ""
retry_after_ms: int = 0
timestamp: Optional[datetime] = None
details: Dict[str, Any] = field(default_factory=dict)
def as_dict(self) -> Dict[str, Any]:
return {
"phase": self.phase,
"status": self.status,
"venue": self.venue,
"endpoint": self.endpoint,
"method": self.method,
"intent_id": self.intent_id,
"trade_id": self.trade_id,
"slot_id": int(self.slot_id or 0),
"asset": self.asset,
"side": self.side.value if hasattr(self.side, "value") else str(self.side),
"action": self.action,
"order_id": self.order_id,
"client_order_id": self.client_order_id,
"venue_order_status": self.venue_order_status,
"venue_event_kind": self.venue_event_kind,
"message": self.message,
"retry_after_ms": int(self.retry_after_ms or 0),
"timestamp": self.timestamp.isoformat() if hasattr(self.timestamp, "isoformat") else None,
"details": dict(self.details),
}
@dataclass
class TradeSlot:
"""A single execution slot managed by the v2 kernel."""

View File

@@ -245,9 +245,15 @@ def _build_venue(
mock_scenario: Optional[MockVenueScenario] = None,
bingx_config: Optional[BingxExecClientConfig] = None,
bingx_backend: Optional[Any] = None,
zinc_plane: Optional[ZincPlane] = None,
venue: Optional[VenueAdapter] = None,
) -> VenueAdapter:
if venue is not None:
if zinc_plane is not None and hasattr(venue, "set_telemetry_plane"):
try:
venue.set_telemetry_plane(zinc_plane)
except Exception:
pass
return venue
resolved_mode = venue_mode or _resolve_venue_mode()
if resolved_mode is LauncherVenueMode.BINGX:
@@ -256,7 +262,7 @@ def _build_venue(
from prod.clean_arch.adapters.bingx_direct import BingxDirectExecutionAdapter
backend = BingxDirectExecutionAdapter(bingx_config or build_bingx_exec_client_config())
return BingxVenueAdapter(backend=backend)
return BingxVenueAdapter(backend=backend, zinc_plane=zinc_plane)
return MockVenueAdapter(mock_scenario)
@@ -342,6 +348,7 @@ def build_launcher_bundle(
mock_scenario=mock_scenario,
bingx_config=bingx_config,
bingx_backend=bingx_backend,
zinc_plane=active_zinc_plane,
venue=venue,
)
kernel = ExecutionKernel(

View File

@@ -16,7 +16,7 @@ import struct
import sys
import threading
from .contracts import KernelIntent, TradeSide, TradeSlot, TradeStage, VenueOrder, VenueOrderStatus
from .contracts import KernelIntent, TradeSide, TradeSlot, TradeStage, VenueOrder, VenueOrderStatus, VenueTelemetrySnapshot
from .control import KernelControlSnapshot
_ZINC_ADAPTER_PATH = Path(__file__).resolve().parents[3] / "zinc" / "adapters" / "python"
@@ -130,6 +130,42 @@ def _decode_packet(buf: memoryview) -> Dict[str, Any]:
return out
def _venue_from_payload(payload: Dict[str, Any]) -> VenueTelemetrySnapshot:
timestamp = payload.get("timestamp")
ts_value = None
if isinstance(timestamp, str) and timestamp:
try:
ts_value = datetime.fromisoformat(timestamp)
except Exception:
ts_value = None
try:
side = TradeSide(str(payload.get("side", TradeSide.FLAT.value)))
except Exception:
side = TradeSide.FLAT
details = payload.get("details", {})
return VenueTelemetrySnapshot(
phase=str(payload.get("phase", "idle")),
status=str(payload.get("status", "IDLE")),
venue=str(payload.get("venue", "bingx")),
endpoint=str(payload.get("endpoint", "")),
method=str(payload.get("method", "")),
intent_id=str(payload.get("intent_id", "")),
trade_id=str(payload.get("trade_id", "")),
slot_id=int(payload.get("slot_id", 0) or 0),
asset=str(payload.get("asset", "")),
side=side,
action=str(payload.get("action", "")),
order_id=str(payload.get("order_id", "")),
client_order_id=str(payload.get("client_order_id", "")),
venue_order_status=str(payload.get("venue_order_status", "")),
venue_event_kind=str(payload.get("venue_event_kind", "")),
message=str(payload.get("message", "")),
retry_after_ms=int(payload.get("retry_after_ms", 0) or 0),
timestamp=ts_value,
details=dict(details) if isinstance(details, dict) else {},
)
class RealZincPlane:
"""Shared-memory Zinc plane used by the Python prototype."""
@@ -148,18 +184,22 @@ class RealZincPlane:
self.intent_name = f"{base}_intent"
self.state_name = f"{base}_state"
self.control_name = f"{base}_control"
self.venue_name = f"{base}_venue"
self._intent_seq = 0
self._state_seq = 0
self._control_seq = 0
self._venue_seq = 0
self._lock = threading.Lock()
self._slot_cache: Dict[int, TradeSlot] = {i: TradeSlot(slot_id=i) for i in range(int(slot_count))}
self._slot_count = int(slot_count)
self._intent_cache: List[Dict[str, Any]] = []
self._control_cache = KernelControlSnapshot()
self._venue_cache = VenueTelemetrySnapshot()
if create:
self.intent_region = SharedRegion.create(self.intent_name, intent_capacity)
self.state_region = SharedRegion.create(self.state_name, state_capacity)
self.control_region = SharedRegion.create(self.control_name, control_capacity)
self.venue_region = SharedRegion.create(self.venue_name, control_capacity)
self._write_region(self.control_region, self._control_seq, {"control": self._control_cache.as_dict()})
self._write_region(
self.state_region,
@@ -167,13 +207,16 @@ class RealZincPlane:
{"slots": [self._slot_cache[key].to_dict() for key in range(self._slot_count)]},
)
self._write_region(self.intent_region, self._intent_seq, {"items": []})
self._write_region(self.venue_region, self._venue_seq, {"venue": self._venue_cache.as_dict()})
else:
self.intent_region = SharedRegion.open(self.intent_name)
self.state_region = SharedRegion.open(self.state_name)
self.control_region = SharedRegion.open(self.control_name)
self.venue_region = SharedRegion.open(self.venue_name)
control_payload = _decode_packet(self.control_region.as_buffer())
state_payload = _decode_packet(self.state_region.as_buffer())
intent_payload = _decode_packet(self.intent_region.as_buffer())
venue_payload = _decode_packet(self.venue_region.as_buffer())
if isinstance(control_payload.get("control"), dict):
self._control_cache = KernelControlSnapshot(**control_payload["control"])
if isinstance(state_payload.get("slots"), list):
@@ -183,11 +226,14 @@ class RealZincPlane:
self._slot_cache[int(slot.slot_id)] = slot
if isinstance(intent_payload.get("items"), list):
self._intent_cache = list(intent_payload["items"])
if isinstance(venue_payload.get("venue"), dict):
self._venue_cache = _venue_from_payload(venue_payload["venue"])
def close(self) -> None:
self.intent_region.close()
self.state_region.close()
self.control_region.close()
self.venue_region.close()
def publish_intent(self, intent: KernelIntent) -> None:
with self._lock:
@@ -246,6 +292,26 @@ class RealZincPlane:
def notify_control(self) -> None:
self.control_region.notify()
def publish_venue(self, telemetry: VenueTelemetrySnapshot) -> None:
with self._lock:
self._venue_seq += 1
self._venue_cache = telemetry
self._write_region(self.venue_region, self._venue_seq, {"venue": telemetry.as_dict()})
def read_venue(self) -> VenueTelemetrySnapshot:
payload = _decode_packet(self.venue_region.as_buffer())
venue = payload.get("venue") if isinstance(payload, dict) else None
if not isinstance(venue, dict):
return self._venue_cache
self._venue_cache = _venue_from_payload(venue)
return self._venue_cache
def wait_on_venue(self, timeout_ms: int = 1000) -> bool:
return bool(self.venue_region.wait(timeout_ms))
def notify_venue(self) -> None:
self.venue_region.notify()
def wait_on_intent(self, timeout_ms: int = 1000) -> bool:
return bool(self.intent_region.wait(timeout_ms))

View File

@@ -107,6 +107,9 @@ def _crate_dir() -> Path:
return Path(__file__).resolve().with_name("_rust_kernel")
_LOCAL_TARGET_DIR = Path("/root/.cargo/dita_v2_target")
def _library_path() -> Path:
if sys.platform == "darwin":
name = "libdita_v2_kernel.dylib"
@@ -114,6 +117,9 @@ def _library_path() -> Path:
name = "dita_v2_kernel.dll"
else:
name = "libdita_v2_kernel.so"
local = _LOCAL_TARGET_DIR / "release" / name
if local.exists():
return local
return _crate_dir() / "target" / "release" / name
@@ -121,10 +127,13 @@ def _build_library() -> None:
crate_dir = _crate_dir()
if not crate_dir.exists():
raise FileNotFoundError(f"Missing Rust kernel crate: {crate_dir}")
_LOCAL_TARGET_DIR.mkdir(parents=True, exist_ok=True)
env = {**os.environ, "CARGO_TARGET_DIR": str(_LOCAL_TARGET_DIR)}
subprocess.run(
["cargo", "build", "--release", "--manifest-path", str(crate_dir / "Cargo.toml")],
cwd=_repo_root(),
check=True,
env=env,
)

View File

@@ -0,0 +1,168 @@
"""ASEx AccountProjectionV2 wrapper tests — race-proof, stress, seam."""
from __future__ import annotations
import gc
import math
import threading
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from types import SimpleNamespace
import pytest
from .account import AccountProjectionV2, AccountSnapshotV2, EPosition, TradeStage, TradeSide
from .asex_account import ASEXAccountV2, _AccountBackend
from asex.guarded import ValidationError
def _empty_slots(n=1):
return [SimpleNamespace(
slot_id=i, trade_id=f"t{i}", asset="BTCUSDT",
side=TradeSide.LONG, entry_price=0.0, size=0.0,
initial_size=0.0, leverage=1.0, realized_pnl=0.0, closed=False,
fsm_state=TradeStage.IDLE, exit_leg_ratios=(1.0,),
active_leg_index=0, active_exit_order=None, active_entry_order=None,
close_reason="", entry_time=None, last_event_time=None,
seen_event_ids=(), metadata={}, unrealized_pnl=0.0, to_dict=lambda: {},
) for i in range(n)]
def _clean():
gc.collect(); gc.collect()
def _force_race(obj, attr, n=10):
b = threading.Barrier(n)
def _w():
b.wait(); v = getattr(obj, attr); b.wait(); setattr(obj, attr, v + 1)
ts = [threading.Thread(target=_w) for _ in range(n)]
for t in ts: t.start()
for t in ts: t.join(timeout=10)
return getattr(obj, attr)
class TestRaceProof:
def test_k_realized_races(self):
assert _force_race(AccountProjectionV2(0.0), "_k_realized", 50) == 1
def test_k_fees_races(self):
assert _force_race(AccountProjectionV2(0.0), "_k_fees", 50) == 1
def test_k_funding_races(self):
assert _force_race(AccountProjectionV2(0.0), "_k_funding", 50) == 1
def test_event_seq_races(self):
assert _force_race(AccountProjectionV2(0.0), "_event_seq", 50) == 1
class TestASExBasic:
def test_seed_only(self):
_clean(); p = ASEXAccountV2(10000.0)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(10000.0); p.close(); _clean()
def test_realized_adds(self):
_clean(); p = ASEXAccountV2(10000.0)
p.apply_fill(fill_price=100, fill_qty=1, fee=0, realized_pnl=500)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(10500.0); p.close(); _clean()
def test_fee_subtracts(self):
_clean(); p = ASEXAccountV2(10000.0)
p.apply_fill(fill_price=100, fill_qty=1, fee=3.5, realized_pnl=0)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(9996.5); p.close(); _clean()
def test_funding_subtracts(self):
_clean(); p = ASEXAccountV2(10000.0)
p.apply_funding(7.25)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(9992.75); p.close(); _clean()
def test_combined(self):
_clean(); p = ASEXAccountV2(10000.0)
p.apply_fill(fill_price=50, fill_qty=2, fee=2, realized_pnl=100)
p.apply_funding(5)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(10093.0)
p.close(); _clean()
class TestASExConcurrency:
@pytest.mark.parametrize("n,ops", [(10, 100), (20, 100), (50, 50)])
def test_no_lost_updates(self, n, ops):
_clean(); p = ASEXAccountV2(0.0)
def _w(tid):
for i in range(ops):
p.apply_fill(fill_price=100, fill_qty=1, fee=0, realized_pnl=float(tid * ops + i))
with ThreadPoolExecutor(max_workers=n) as ex:
for f in as_completed([ex.submit(_w, i) for i in range(n)]): f.result(timeout=60)
expected = sum(tid * ops + i for tid in range(n) for i in range(ops))
assert p._backend._proj._k_realized == pytest.approx(float(expected))
p.close(); _clean()
def test_event_seq_monotonic(self):
_clean(); p = ASEXAccountV2(0.0); seqs = []
def _w(tid):
for i in range(50):
s = p.build_snapshot(source_event_id=f"e{tid}_{i}", slots=_empty_slots(), ts=float(i))
seqs.append(s.event_seq)
with ThreadPoolExecutor(max_workers=10) as ex:
for f in as_completed([ex.submit(_w, i) for i in range(10)]): f.result(timeout=60)
assert sorted(seqs) == list(range(1, 501))
p.close(); _clean()
def test_mixed_ops(self):
_clean(); p = ASEXAccountV2(10000.0)
def _f(tid):
for i in range(100): p.apply_fill(fill_price=float(i), fill_qty=1, fee=0, realized_pnl=1)
def _fu(tid):
for i in range(50): p.apply_funding(0.5)
with ThreadPoolExecutor(max_workers=4) as ex:
for f in as_completed([ex.submit(_f, i) for i in [0,1]] + [ex.submit(_fu, i) for i in [0,1]]):
f.result(timeout=60)
assert p._backend._proj._k_realized == pytest.approx(200.0)
assert p._backend._proj._k_funding == pytest.approx(100.0)
p.close(); _clean()
class TestASExSeam:
def test_backend_rejects_no_op(self):
assert not _AccountBackend(0.0)._validate({})
def test_backend_rejects_unknown(self):
b = _AccountBackend(0.0)
with pytest.raises(Exception): b.mutate({"op": "nope"})
def test_backend_applied(self):
b = _AccountBackend(0.0)
b.mutate({"op": "apply_fill", "args": {"fill_price": 100, "fill_qty": 1, "fee": 0, "realized_pnl": 10}})
assert b.applied == 1
def test_worker_alive(self):
p = ASEXAccountV2(0.0); assert p._worker._worker.is_alive(); p.close()
def test_double_close(self):
p = ASEXAccountV2(0.0); p.close(); p.close()
def test_context_manager(self):
with ASEXAccountV2(10000.0) as p:
p.apply_fill(fill_price=100, fill_qty=1, fee=0, realized_pnl=50)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.realized_pnl == pytest.approx(50.0)
def test_no_thread_leak(self):
_clean(); b = threading.active_count()
for _ in range(50):
p = ASEXAccountV2(0.0); p.apply_fill(fill_price=100, fill_qty=1, fee=0, realized_pnl=1); p.close()
_clean(); assert threading.active_count() - b <= 2
class TestLockProof:
def test_k_realized_lock_proof(self):
ap, l, n = AccountProjectionV2(0.0), threading.Lock(), 50
b = threading.Barrier(n)
def _w():
b.wait()
with l: ap._k_realized += 1
ts = [threading.Thread(target=_w) for _ in range(n)]
for t in ts: t.start()
for t in ts: t.join(timeout=10)
assert ap._k_realized == n
def test_event_seq_lock_proof(self):
ap, l, n = AccountProjectionV2(0.0), threading.Lock(), 50
b = threading.Barrier(n)
def _w():
b.wait()
with l: ap._event_seq += 1
ts = [threading.Thread(target=_w) for _ in range(n)]
for t in ts: t.start()
for t in ts: t.join(timeout=10)
assert ap._event_seq == n

View File

@@ -12,7 +12,7 @@ from typing import Any, Dict, Iterable, List, Mapping, Optional, Protocol
import threading
import time
from .contracts import KernelIntent, TradeSlot
from .contracts import KernelIntent, TradeSlot, VenueTelemetrySnapshot
from .control import KernelControlSnapshot
@@ -52,6 +52,18 @@ class ZincPlane(Protocol):
def notify_control(self) -> None:
...
def publish_venue(self, telemetry: VenueTelemetrySnapshot) -> None:
...
def read_venue(self) -> VenueTelemetrySnapshot:
...
def wait_on_venue(self, timeout_ms: int = 1000) -> bool:
...
def notify_venue(self) -> None:
...
@dataclass
class InMemoryZincPlane:
@@ -60,12 +72,15 @@ class InMemoryZincPlane:
intent_region: List[KernelIntent] = field(default_factory=list)
state_region: Dict[int, TradeSlot] = field(default_factory=dict)
control_region: Optional[KernelControlSnapshot] = None
venue_region: VenueTelemetrySnapshot = field(default_factory=VenueTelemetrySnapshot)
_intent_seq: int = field(default=0, init=False, repr=False)
_state_seq: int = field(default=0, init=False, repr=False)
_control_seq: int = field(default=0, init=False, repr=False)
_venue_seq: int = field(default=0, init=False, repr=False)
_intent_observed_seq: int = field(default=0, init=False, repr=False)
_state_observed_seq: int = field(default=0, init=False, repr=False)
_control_observed_seq: int = field(default=0, init=False, repr=False)
_venue_observed_seq: int = field(default=0, init=False, repr=False)
_signal: threading.Condition = field(default_factory=threading.Condition, init=False, repr=False)
def publish_intent(self, intent: KernelIntent) -> None:
@@ -118,6 +133,23 @@ class InMemoryZincPlane:
self._control_seq += 1
self._signal.notify_all()
def publish_venue(self, telemetry: VenueTelemetrySnapshot) -> None:
with self._signal:
self.venue_region = telemetry
self._venue_seq += 1
self._signal.notify_all()
def read_venue(self) -> VenueTelemetrySnapshot:
return self.venue_region
def wait_on_venue(self, timeout_ms: int = 1000) -> bool:
return self._wait_for_change("_venue_seq", "_venue_observed_seq", timeout_ms)
def notify_venue(self) -> None:
with self._signal:
self._venue_seq += 1
self._signal.notify_all()
def _wait_for_change(self, seq_attr: str, observed_attr: str, timeout_ms: int) -> bool:
timeout_s = None if timeout_ms is None or timeout_ms < 0 else max(0.0, timeout_ms / 1000.0)
deadline = None if timeout_s is None else time.monotonic() + timeout_s

View File

@@ -5,21 +5,39 @@ This is VIOLET-only. BLUE is untouched.
The adapter reads the published BLUE surfaces that already exist in HZ and
translates them into ``SizingFactors`` for the shadow path:
- ``posture`` from ``DOLPHIN_STATE_BLUE.latest_nautilus`` / ``engine_snapshot``
- ``esof_score`` from ``DOLPHIN_FEATURES.esof_latest`` or ``esof_advisor_latest``
- ``acb_boost`` / ``acb_beta`` from ``DOLPHIN_FEATURES.acb_boost``
- ``mc_scale`` from ``DOLPHIN_FEATURES.mc_forewarner_latest``
- OB market consensus from the live ``asset_*_ob`` maps via BLUE's own
- ``esof_score`` from ``DOLPHIN_FEATURES.esof_latest`` or ``esof_advisor_latest`` via
BLUE's own ``parse_esof_payload`` / ``esof_score_from_payload``
- ``boost`` / ``beta`` RECOMPUTED via ``AdaptiveCircuitBreaker.get_dynamic_boost_from_hz``
over ``DOLPHIN_FEATURES.exf_latest`` + ``latest_eigen_scan.w750_velocity`` — IDENTICAL
to the trader's on_exf_update path (NOT the published ``acb_boost`` scalar)
- ``mc_scale`` from ``DOLPHIN_FEATURES.mc_forewarner_latest`` via ``_derive_mc_scale``
(begin_day's cat/env thresholds, not the MC service's status label)
- OB market consensus from the live ``asset_*_ob`` maps via BLUE's own ``HZOBProvider`` +
``OBFeatureEngine``
- ``dc_status`` via BLUE's ``AlphaSignalGenerator`` (params pinned to BLUE's ENGINE_KWARGS)
over the replayed scan price-history
The remaining DC signal is left neutral here for now. It needs the same live
signal-history path BLUE uses and should be added as a separate mirror step.
PARITY (see prod/docs/VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md): this module reconstructs
BLUE's factors in a DIFFERENT file/scope structure than BLUE's monolithic NDAlphaEngine.
Every kernel is now WRAPPED, not copied (ACB, OBFeatureEngine+HZOBProvider, AlphaSignalGenerator,
VioletAssetSelector) — so the only hand-replicated arithmetic left is ``_derive_mc_scale``
(begin_day's thresholds; pinned by test). Remaining REAL fidelity caveats:
- OB faithfulness needs a PERSISTENT ``ob_engine`` + per-scan ``bar_idx`` (OBFeatureEngine
accumulates a lookback window); a single-shot engine has no cross-scan history.
- On stale exf (>12h) the ACB raises ValueError; BLUE keeps the prior boost/beta, VIOLET
has no prior → neutral (1.0, 0.0).
Any change to BLUE's ENGINE_KWARGS or begin_day mc thresholds REQUIRES a matching change +
test here (see test_signal_gen_params_match_blue_engine_kwargs / the mc_scale formula test).
"""
from __future__ import annotations
import json
import logging
import os as _os
import sys
from collections import deque
from datetime import datetime, timezone
from collections.abc import Mapping
from dataclasses import dataclass
from dataclasses import field
@@ -35,14 +53,48 @@ for _p in (str(_PROJECT_ROOT), str(_PROJECT_ROOT / "nautilus_dolphin")):
sys.path.insert(0, _p)
from nautilus_dolphin.nautilus.ob_features import OBFeatureEngine
from nautilus_dolphin.nautilus.ob_provider import OBSnapshot, OBProvider
from nautilus_dolphin.nautilus.alpha_signal_generator import AlphaSignalGenerator
from nautilus_dolphin.nautilus.hz_ob_provider import HZOBProvider
from nautilus_dolphin.nautilus.adaptive_circuit_breaker import AdaptiveCircuitBreaker
from nautilus_dolphin.nautilus.alpha_signal_generator import (
AlphaSignalGenerator,
VEL_DIV_THRESHOLD, VEL_DIV_EXTREME,
LONG_VEL_DIV_THRESHOLD, LONG_VEL_DIV_EXTREME,
)
from .alpha_wrappers import VioletAssetSelector
from .decision_engine import SizingFactors
from .live_factor_source import esof_score_from_features, posture_from_engine_snapshot
from .live_factors import extract_live_sizing_factors
LOGGER = logging.getLogger("violet.live_blue_source")
# Hazelcast coordinates — MUST equal nautilus_event_trader.py:107-108 (BLUE's live
# cluster). HZOBProvider opens its own connection to these, exactly as BLUE's _wire_obf.
# TODO_HZBRIDGE: all VIOLET HZ access (the caller's client + HZOBProvider) must move to
# dolphinng5_predict/hzbridge when it ships — raw HazelcastClient is the silent-death/lockup
# surface (see prod/docs/VIOLET_OB_FEED_AND_AGENT_COORDINATION.md + hz_client_death memory).
HZ_CLUSTER = _os.environ.get("HZ_CLUSTER", "dolphin")
HZ_HOST = _os.environ.get("HZ_HOST", "127.0.0.1:5701")
# AlphaSignalGenerator construction — pinned to BLUE's live ENGINE_KWARGS
# (nautilus_event_trader.py:128-133). These equal AlphaSignalGenerator's own defaults
# TODAY, but BLUE constructs it EXPLICITLY from ENGINE_KWARGS, so a future champion
# retune (e.g. dc_lookback_bars→10) would silently diverge a bare AlphaSignalGenerator().
# We pin explicitly and assert the pin in test_signal_gen_params_match_blue_engine_kwargs.
# vel_div_* import the module constants directly so they auto-track the kernel.
BLUE_SIGNAL_GEN_KWARGS = dict(
vel_div_threshold=VEL_DIV_THRESHOLD, # -0.02
vel_div_extreme=VEL_DIV_EXTREME, # -0.05
long_vel_div_threshold=LONG_VEL_DIV_THRESHOLD, # 0.01
long_vel_div_extreme=LONG_VEL_DIV_EXTREME, # 0.04
dc_lookback_bars=7,
dc_min_magnitude_bps=0.75,
dc_skip_contradicts=True,
dc_leverage_boost=1.0,
dc_leverage_reduce=0.5,
use_direction_confirm=True,
)
def _jsonish(value: Any) -> Any:
if isinstance(value, str):
@@ -72,22 +124,115 @@ def _read_hz_map(client: hazelcast.HazelcastClient, map_name: str, key: str) ->
return None
def _map_status_to_mc_scale(payload: Any) -> float:
def _derive_mc_scale(payload: Any) -> float:
"""Mirror BLUE ``begin_day``'s ``mc_scale`` EXACTLY (esf_alpha_orchestrator.py:956-962).
BLUE does NOT use the MC service's published ``status`` label for sizing — that label
(mc_forewarner_flow.py: GREEN<0.10 / ORANGE<0.30 / RED) is observability-only and uses
DIFFERENT thresholds from the engine. The live trader re-derives the size haircut from
the SAME published fields ``catastrophic_prob`` + ``envelope_score`` with the engine's
own thresholds:
mc_red = catastrophic_prob > 0.25 or envelope_score < -1.0
mc_orange = (not mc_red) and (envelope_score < 0 or catastrophic_prob > 0.10)
mc_scale = 0.5 if mc_orange else 1.0 # RED → 1.0 here; BLUE halts via regime_dd_halt
Reading ``status`` instead diverges (e.g. cat=0.05/env=-0.5 → publisher GREEN→1.0 but
BLUE orange→0.5).
AMBIGUITY (flagged 2026-06-16): the MC service's ``status`` and the engine's begin_day
thresholds genuinely disagree, and from OUTSIDE BLUE there is no way to know which is the
"nominal" intent — they are two independent threshold sets over the same numbers. We go
with the SOURCE FIELDS + begin_day formula because that is the path that actually drives
BLUE's live sizing (the published ``status`` is consumed only by the TUI/observability).
If BLUE's begin_day thresholds change, THIS must change with them. Operator confirmation
of the canonical surface is still desirable.
Missing/unparseable fields → neutral 1.0 (no haircut)."""
data = _jsonish(payload)
if isinstance(data, Mapping):
status = str(data.get("status", "")).upper()
else:
status = str(data).upper()
return 0.5 if status == "ORANGE" else 1.0
if not isinstance(data, Mapping):
return 1.0
cat = _coerce_float(data.get("catastrophic_prob"), None)
env = _coerce_float(data.get("envelope_score"), None)
if cat is None or env is None:
return 1.0
mc_red = cat > 0.25 or env < -1.0
mc_orange = (not mc_red) and (env < 0.0 or cat > 0.10)
return 0.5 if mc_orange else 1.0
def _extract_acb(payload: Any) -> tuple[float, float]:
data = _jsonish(payload)
if isinstance(data, Mapping):
boost = _coerce_float(data.get("boost"), 1.0) or 1.0
beta = _coerce_float(data.get("beta"), 0.0) or 0.0
return max(0.0, boost), max(0.0, beta)
return 1.0, 0.0
# Inverse-ACB neutral identity — used ONLY at cold start (no prior yet AND no fresh exf).
# In continuous operation exf is warmed, so the first call seeds the prior and this is
# never the steady-state value.
_BOOST_BETA_NEUTRAL = (1.0, 0.0)
def _source_boost_beta(
client: hazelcast.HazelcastClient,
*,
date_str: str,
trade_direction: int,
acb: Optional[AdaptiveCircuitBreaker] = None,
prior: Optional[tuple[float, float]] = None,
) -> tuple[float, float]:
"""Recompute (boost, beta) EXACTLY as BLUE's live trader does — NOT from acb_boost.
WHAT THE ACB IS DOING, IN EFFECT (AdaptiveCircuitBreaker v6, "inverse" mode):
- ``boost`` = a DAILY position-size governor >= 1.0 driven by EXTERNAL FACTORS (ExF:
funding_btc/dvol_btc/fng/taker from DOLPHIN_FEATURES.exf_latest). When stress
signals fire, ``boost = 1 + 0.5*ln(1+signals)``; else 1.0. INVERSE = it leans size
UP under stress, it does not cut (adaptive_circuit_breaker.py:591-593).
- ``beta`` = regime sensitivity in {BETA_HIGH=0.8, BETA_LOW=0.2}, keyed on the w750
eigenvalue-velocity (>= threshold → HIGH). It sets how strongly per-bar signal
strength amplifies size: regime_size_mult = base_boost*(1 + beta*strength^3)*mc_scale.
- Both become the engine's _day_base_boost / _day_beta via update_acb_boost
(esf_alpha_orchestrator.py:771-772), which then feed BLUE's SIZING directly:
_update_regime_size_mult (:898-909) = _day_base_boost * (1 + _day_beta*strength^3)
* _day_mc_scale. So the ACB IS in the sizing layer — confirmed, not incidental.
BLUE's LIVE path passes NO ob_engine (on_exf_update:4769 / rollover prewarm:2710), so the
ACB's OB Sub-4 macro-regime beta modulation (x1.25 stress / x0.85 calm,
adaptive_circuit_breaker.py:613-628) is DORMANT in live — only the NPZ backtest path
get_dynamic_boost_for_date applies it. Per operator recollection (2026-06-16), OB Sub-4
beta modulation was found NON-PERFORMANT and deliberately removed/bypassed — so passing
no ob_engine is INTENTIONAL design, not an oversight. We pass no ob_engine to match BLUE
bit-for-bit either way.
# TODO_SOMEDAY: find the documentary confirmation of the OB Sub-4 non-performant bypass
# (operator: likely SYSTEM_BIBLE v7 lineage or a deep code comment / the in-progress
# INDEX_DOLPHINNG5_PREDICT doc). Behaviour is already bit-identical to live BLUE.
The published DOLPHIN_FEATURES.acb_boost is acb_processor_service's SEPARATE daily value
and is never read here.
STALE / MISSING exf: BLUE keeps the prior _day_base_boost/_day_beta (update_acb_boost
simply isn't called on the stale branch), so VIOLET KEEPS ``prior`` too. ``prior`` is
seeded by the first successful compute — exf is warmed in continuous BLUE operation, so
the steady state always has a prior; only a cold start with no prior AND no fresh exf
falls back to the neutral identity."""
fallback = prior if prior is not None else _BOOST_BETA_NEUTRAL
exf = _jsonish(_read_hz_map(client, "DOLPHIN_FEATURES", "exf_latest"))
if not isinstance(exf, Mapping):
LOGGER.debug("ACB: no exf_latest — keeping prior boost/beta=%s", fallback)
return fallback
eigen = _jsonish(_read_hz_map(client, "DOLPHIN_FEATURES", "latest_eigen_scan"))
w750 = _coerce_float(eigen.get("w750_velocity"), None) if isinstance(eigen, Mapping) else None
acb = acb or AdaptiveCircuitBreaker()
try:
info = acb.get_dynamic_boost_from_hz(
date_str=date_str,
exf_snapshot=dict(exf),
w750_velocity=float(w750) if w750 else None, # 0.0 → None, matches BLUE
direction=trade_direction, # NO ob_engine — matches BLUE live
)
except ValueError as exc:
# BLUE logs "ACB Stale Data Fallback" and keeps the prior boost/beta.
LOGGER.info("ACB Stale Data Fallback (%s) — keeping prior boost/beta=%s", exc, fallback)
return fallback
boost = max(0.0, _coerce_float(info.get("boost"), 1.0) or 1.0)
beta = max(0.0, _coerce_float(info.get("beta"), 0.0) or 0.0)
LOGGER.debug("ACB live boost=%.6f beta=%.6f (signals=%s w750=%s dir=%s)",
boost, beta, info.get("signals"), w750, trade_direction)
return boost, beta
def _scan_view(payload: Any) -> Mapping[str, Any]:
@@ -169,7 +314,12 @@ class LiveBlueScanHistory:
history = self.price_history(asset[0])
if not history:
return "NONE"
signal_gen = AlphaSignalGenerator()
# BLUE constructs its signal_gen from ENGINE_KWARGS (the orchestrator threads them,
# esf_alpha_orchestrator.py:180-191). Pin the SAME params — not bare defaults — so a
# champion retune that changes dc_lookback_bars / dc_min_magnitude_bps / thresholds
# diverges loudly (caught by test_signal_gen_params_match_blue_engine_kwargs), never
# silently. dc_status is deterministic in the params + price history (counters unused).
signal_gen = AlphaSignalGenerator(**BLUE_SIGNAL_GEN_KWARGS)
sig = signal_gen.generate(
vel_div=vel_div,
vel_div_history=None,
@@ -181,80 +331,37 @@ class LiveBlueScanHistory:
return sig.dc_status
class HazelcastOBProvider(OBProvider):
"""Read the current BLUE OB shards directly from Hazelcast."""
def _source_ob_market(
ob_assets: list[str],
*,
bar_idx: int,
ob_engine: Optional[OBFeatureEngine] = None,
) -> tuple[Optional[float], Optional[float]]:
"""Derive (median_imbalance, agreement_pct) EXACTLY as BLUE does, via BLUE's HZOBProvider.
def __init__(self, client: hazelcast.HazelcastClient):
self.client = client
BLUE wires OB once in _wire_obf (nautilus_event_trader.py:4967-4980):
live_ob = HZOBProvider(hz_cluster=HZ_CLUSTER, hz_host=HZ_HOST, assets=assets)
ob_eng = OBFeatureEngine(live_ob); eng.set_ob_engine(ob_eng)
then per scan calls ``ob_eng.step_live(assets, bar_idx)`` and the orchestrator reads
``ob_eng.get_market(bar_idx, assets)`` (esf_alpha_orchestrator.py:590). We use BLUE's
OWN HZOBProvider — NOT a reinvented reader — so OB parsing/shard semantics are BLUE's.
def _asset_keys(self) -> list[str]:
try:
keys = self.client.get_map("DOLPHIN_FEATURES").blocking().key_set()
except Exception:
return []
assets = []
for key in keys:
if not isinstance(key, str) or not key.startswith("asset_") or not key.endswith("_ob"):
continue
asset = key[len("asset_"):-len("_ob")]
if asset and asset not in assets:
assets.append(asset)
return sorted(assets)
def _read_snapshot(self, asset: str) -> Optional[OBSnapshot]:
raw = _read_hz_map(self.client, "DOLPHIN_FEATURES", f"asset_{asset}_ob")
data = _jsonish(raw)
if not isinstance(data, Mapping):
return None
bid_notional = np.array(
[_coerce_float(v, 0.0) or 0.0 for v in data.get("bid_notional", [0, 0, 0, 0, 0])][:5],
dtype=np.float64,
)
ask_notional = np.array(
[_coerce_float(v, 0.0) or 0.0 for v in data.get("ask_notional", [0, 0, 0, 0, 0])][:5],
dtype=np.float64,
)
bid_depth = np.array(
[_coerce_float(v, 0.0) or 0.0 for v in data.get("bid_depth", [0, 0, 0, 0, 0])][:5],
dtype=np.float64,
)
ask_depth = np.array(
[_coerce_float(v, 0.0) or 0.0 for v in data.get("ask_depth", [0, 0, 0, 0, 0])][:5],
dtype=np.float64,
)
ts = _coerce_float(data.get("timestamp"), 0.0) or 0.0
if (
bid_notional.shape != (5,) or ask_notional.shape != (5,)
or bid_depth.shape != (5,) or ask_depth.shape != (5,)
):
return None
if np.any(bid_notional < 0) or np.any(ask_notional < 0):
return None
if np.any(bid_depth < 0) or np.any(ask_depth < 0):
return None
return OBSnapshot(
timestamp=ts,
asset=asset,
bid_notional=bid_notional,
ask_notional=ask_notional,
bid_depth=bid_depth,
ask_depth=ask_depth,
)
def get_snapshot(self, asset: str, timestamp: float) -> Optional[OBSnapshot]:
return self._read_snapshot(asset)
def get_assets(self) -> list[str]:
return self._asset_keys()
def get_all_timestamps(self, asset: str) -> np.ndarray:
snap = self._read_snapshot(asset)
if snap is None:
return np.array([], dtype=np.float64)
return np.array([snap.timestamp], dtype=np.float64)
def get_snapshot_count(self, asset: str) -> int:
return 1 if self._read_snapshot(asset) is not None else 0
OBFeatureEngine ACCUMULATES per-asset history across scans (lookback=10), so a faithful
mirror requires a PERSISTENT ``ob_engine`` + per-scan-incrementing ``bar_idx`` (the
caller/shadow loop owns it, exactly as BLUE keeps one ob_eng). When no engine is passed
this builds a single-shot HZOBProvider-backed engine — correct wiring but no cross-scan
history; use only for one-off reads / the live smoke."""
if not ob_assets:
return None, None
if ob_engine is None:
provider = HZOBProvider(hz_cluster=HZ_CLUSTER, hz_host=HZ_HOST, assets=list(ob_assets))
ob_engine = OBFeatureEngine(provider)
try:
ob_engine.step_live(list(ob_assets), bar_idx)
market = ob_engine.get_market(bar_idx, list(ob_assets))
return float(market.median_imbalance), float(market.agreement_pct)
except Exception:
return None, None
@dataclass(frozen=True)
@@ -274,10 +381,23 @@ def source_live_blue_sizing_factors(
assets: Optional[Iterable[str]] = None,
scan_history: Optional[LiveBlueScanHistory] = None,
selector: Optional[VioletAssetSelector] = None,
acb: Optional[AdaptiveCircuitBreaker] = None,
ob_engine: Optional[OBFeatureEngine] = None,
bar_idx: int = 0,
date_str: Optional[str] = None,
prior_boost_beta: Optional[tuple[float, float]] = None,
) -> LiveBlueSourceResult:
"""Read the BLUE-published live surfaces and return a typed factor plane."""
"""Read BLUE's live surfaces and RECONSTRUCT the factor plane the way BLUE computes it.
boost/beta are recomputed via ``AdaptiveCircuitBreaker.get_dynamic_boost_from_hz`` (NOT
the published acb_boost scalar); OB via BLUE's ``HZOBProvider`` + ``OBFeatureEngine``;
dc_status via ``AlphaSignalGenerator`` pinned to BLUE's ENGINE_KWARGS. For full OB
accumulation faithfulness the caller passes a PERSISTENT ``ob_engine`` + per-scan
``bar_idx`` (BLUE keeps one ob_eng). ``date_str`` defaults to today's UTC date (BLUE
uses self.current_day)."""
scan_history = scan_history or LiveBlueScanHistory()
selector = selector or VioletAssetSelector()
today = date_str or datetime.now(timezone.utc).strftime("%Y-%m-%d")
engine_snapshot_raw = _read_hz_map(client, "DOLPHIN_STATE_BLUE", "latest_nautilus")
if engine_snapshot_raw is None:
@@ -296,13 +416,8 @@ def source_live_blue_sizing_factors(
esof_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "esof_advisor_latest")
esof_score = esof_score_from_features(esof_raw)
acb_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "acb_boost")
if acb_raw is None:
acb_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "acb_boost_short")
acb_boost, acb_beta = _extract_acb(acb_raw)
mc_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "mc_forewarner_latest")
mc_scale = _map_status_to_mc_scale(mc_raw)
mc_scale = _derive_mc_scale(mc_raw)
scan_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "latest_eigen_scan")
scan = scan_history.ingest_scan(scan_raw)
@@ -318,26 +433,24 @@ def source_live_blue_sizing_factors(
)
or -1
)
# boost/beta — recompute via the ACB exactly as BLUE's trader does (NOT the published
# acb_boost). Needs trade_direction, so computed after it.
acb_boost, acb_beta = _source_boost_beta(
client, date_str=today, trade_direction=trade_direction, acb=acb,
prior=prior_boost_beta,
)
candidate_market = scan_history.market_data(selector.lookback)
pick = selector.pick(candidate_market, regime_direction=trade_direction)
selected_asset = pick.asset if pick is not None else (scan_assets[0] if scan_assets else "")
dc_status = scan_history.dc_status(scan, already_ingested=True)
ob_provider = HazelcastOBProvider(client)
ob_engine = OBFeatureEngine(ob_provider)
ob_assets = list(assets) if assets is not None else (scan_assets or ob_provider.get_assets())
if ob_assets:
try:
ob_engine.step_live(ob_assets, bar_idx=0)
market = ob_engine.get_market(0, ob_assets)
ob_median_imbalance = float(market.median_imbalance)
ob_agreement_pct = float(market.agreement_pct)
except Exception:
ob_median_imbalance = None
ob_agreement_pct = None
else:
ob_median_imbalance = None
ob_agreement_pct = None
# OB market consensus — BLUE's HZOBProvider + OBFeatureEngine (persistent engine if given).
ob_assets = list(assets) if assets is not None else scan_assets
ob_median_imbalance, ob_agreement_pct = _source_ob_market(
ob_assets, bar_idx=bar_idx, ob_engine=ob_engine,
)
hz_snapshot = {
"boost": acb_boost,
@@ -350,6 +463,12 @@ def source_live_blue_sizing_factors(
"posture": posture,
}
factors = extract_live_sizing_factors(hz_snapshot=hz_snapshot)
LOGGER.debug(
"live BLUE plane: asset=%s posture=%s boost=%.4f beta=%.4f mc_scale=%.2f "
"esof=%s ob=(%s,%s) dc=%s dir=%s",
selected_asset, posture, acb_boost, acb_beta, mc_scale, esof_score,
ob_median_imbalance, ob_agreement_pct, dc_status, trade_direction,
)
return LiveBlueSourceResult(
factors=factors,
acb_boost=acb_boost,

View File

@@ -2,6 +2,17 @@
These helpers stay separate from the launcher module so they can be unit-tested
without importing the full launcher import chain.
They hold the PERSISTENT state BLUE keeps across scans so the shadow plane is faithful
to BLUE's live factor stream rather than a per-scan single-shot:
- ONE OBFeatureEngine, wired lazily on first assets and kept for the service lifetime
(OBFeatureEngine accumulates a lookback window; a fresh engine per scan has none).
It reads BLUE's EXTANT published OBF feed via HZOBProvider (a read-only HZ entry-
listener cache — NO new OB storage). The provider is the swap seam for a future
direct BingX / 3rd-party OB stream. See VIOLET_OB_FEED_AND_AGENT_COORDINATION.md.
- a per-scan-incrementing ``ob_bar_idx`` (BLUE steps one bar_idx into step_live).
- the last good ``prior_boost_beta`` so a stale-exf scan keeps the prior (BLUE keeps
_day_base_boost/_day_beta).
"""
from __future__ import annotations
@@ -9,7 +20,25 @@ from __future__ import annotations
import logging
import os
LOGGER = logging.getLogger(__name__)
LOGGER = logging.getLogger("violet.shadow_live_factors")
def _default_ob_engine_factory(assets):
"""Wire BLUE's EXTANT OBF feed: HZOBProvider (read-only listener) → OBFeatureEngine.
Exactly BLUE's _wire_obf (nautilus_event_trader.py:4967-4980). No OB storage, no new
exchange WS — it consumes the asset_*_ob shards BLUE already publishes to Hazelcast."""
from nautilus_dolphin.nautilus.ob_features import OBFeatureEngine
from nautilus_dolphin.nautilus.hz_ob_provider import HZOBProvider
from .live_blue_source import HZ_CLUSTER, HZ_HOST
# TODO_HZBRIDGE: HZOBProvider opens its OWN raw HazelcastClient. Once dolphinng5_predict/
# hzbridge ships, this connection MUST route through the bridge (silent HZ client death /
# lockup mitigation — see [[hz_client_death_investigation]]). One of 3 VIOLET HZ touch
# points to refactor (also build_shadow_live_source's client_factory + the live smoke).
provider = HZOBProvider(hz_cluster=HZ_CLUSTER, hz_host=HZ_HOST, assets=list(assets))
return OBFeatureEngine(provider)
def build_shadow_live_source(
@@ -18,14 +47,18 @@ def build_shadow_live_source(
selector_factory=None,
source_factory=None,
scan_history_factory=None,
ob_engine_factory=None,
):
"""Create the read-only BLUE live-factor mirror for the shadow path."""
"""Create the read-only BLUE live-factor mirror for the shadow path, with the
persistent OB engine / bar_idx / boost-beta-prior state BLUE keeps across scans."""
if client_factory is None or selector_factory is None or source_factory is None or scan_history_factory is None:
import hazelcast
from .alpha_wrappers import VioletAssetSelector
from .live_blue_source import LiveBlueScanHistory, source_live_blue_sizing_factors
# TODO_HZBRIDGE: raw HazelcastClient — route through dolphinng5_predict/hzbridge once
# it ships, to avoid the silent-death/lockup class. See [[hz_client_death_investigation]].
client_factory = client_factory or (lambda: hazelcast.HazelcastClient(
cluster_name=os.environ.get("HZ_CLUSTER", "dolphin"),
cluster_members=[os.environ.get("HZ_HOST", "localhost:5701")],
@@ -40,9 +73,35 @@ def build_shadow_live_source(
"scan_history": scan_history_factory(),
"selector": selector_factory(),
"live_source": source_factory,
# persistent state (BLUE keeps these across scans)
"ob_engine": None,
"ob_engine_factory": ob_engine_factory or _default_ob_engine_factory,
"ob_bar_idx": 0,
"prior_boost_beta": None,
"last_live_source": None,
}
def _ensure_ob_engine(shadow, payload):
"""Lazily wire ONE OB engine on the first scan that has an asset universe — like BLUE's
_wire_obf (`if not assets or self.ob_assets: return`). Kept for the service lifetime."""
if shadow.get("ob_engine") is not None:
return shadow["ob_engine"]
factory = shadow.get("ob_engine_factory")
if factory is None:
return None
from .live_blue_source import _scan_assets, _scan_view
assets = _scan_assets(_scan_view(payload))
if not assets:
return None
eng = factory(assets)
shadow["ob_engine"] = eng
shadow["ob_assets"] = assets
LOGGER.info("shadow OB wired to BLUE's extant feed for %d assets", len(assets))
return eng
def shadow_decision_step(
shadow: dict,
payload: dict,
@@ -52,17 +111,28 @@ def shadow_decision_step(
vel_div: float,
vol_ok: bool,
) -> bool:
"""Run one shadow decision against the live BLUE factor plane."""
"""Run one muted shadow decision against the live BLUE factor plane, carrying the
persistent OB engine, bar_idx, and boost/beta prior across scans (BLUE-faithful)."""
shadow["engine"].observe(payload, scan_number)
live_source = shadow.get("live_source")
factors = None
if live_source is not None:
ob_engine = _ensure_ob_engine(shadow, payload)
live_result = live_source(
shadow["client"],
scan_history=shadow["scan_history"],
selector=shadow["selector"],
ob_engine=ob_engine,
bar_idx=shadow.get("ob_bar_idx", 0),
prior_boost_beta=shadow.get("prior_boost_beta"),
)
shadow["last_live_source"] = live_result
shadow["ob_bar_idx"] = shadow.get("ob_bar_idx", 0) + 1
# persist last good boost/beta as next scan's prior (BLUE keeps _day_base_boost/_day_beta).
ab = getattr(live_result, "acb_boost", None)
bb = getattr(live_result, "acb_beta", None)
if ab is not None and bb is not None:
shadow["prior_boost_beta"] = (ab, bb)
factors = live_result.factors
if factors is None:
return False

View File

@@ -92,7 +92,8 @@ def test_shadow_decision_step_uses_live_factors_and_journals():
"client": object(),
"scan_history": object(),
"selector": object(),
"live_source": lambda client, scan_history, selector: SimpleNamespace(
"live_source": lambda client, scan_history=None, selector=None, ob_engine=None,
bar_idx=0, prior_boost_beta=None: SimpleNamespace(
factors=SizingFactors(
boost=1.4,
beta=0.2,
@@ -104,6 +105,8 @@ def test_shadow_decision_step_uses_live_factors_and_journals():
posture="APEX",
),
selected_asset="BTCUSDT",
acb_boost=1.4,
acb_beta=0.2,
),
"live_decisions": 0,
"last_live_source": None,
@@ -153,3 +156,69 @@ def test_shadow_decision_step_skips_without_live_factor_plane():
vol_ok=True,
)
assert ok is False
class _NoDecisionEngine:
def observe(self, payload, scan_number):
pass
def decide(self, **kwargs):
return None # step returns False after the live-source call (which is what we probe)
def test_shadow_step_lazily_wires_ob_engine_once_and_increments_bar_idx():
from prod.clean_arch.violet import shadow_live_factors as slf
built, seen = [], []
def ob_factory(assets):
eng = SimpleNamespace(assets=list(assets))
built.append(eng)
return eng
def live_source(client, *, scan_history, selector, ob_engine, bar_idx, prior_boost_beta):
seen.append((ob_engine, bar_idx))
return SimpleNamespace(factors=SizingFactors(posture="APEX"), selected_asset="BTCUSDT",
acb_boost=1.0, acb_beta=0.0)
shadow = slf.build_shadow_live_source(
client_factory=lambda: object(), selector_factory=lambda: object(),
source_factory=live_source, scan_history_factory=lambda: object(),
ob_engine_factory=ob_factory,
)
shadow["engine"] = _NoDecisionEngine()
shadow["capital"] = 1.0
payload = {"assets": ["BTCUSDT", "ETHUSDT"], "vel_div": -0.03}
for i in range(3):
slf.shadow_decision_step(shadow, payload, scan_number=i, now_ns=i, vel_div=-0.03, vol_ok=True)
assert len(built) == 1 # OB engine wired exactly ONCE
assert all(s[0] is built[0] for s in seen) # same persistent engine reused
assert [s[1] for s in seen] == [0, 1, 2] # bar_idx increments per scan
def test_shadow_step_carries_boost_beta_prior_across_scans():
from prod.clean_arch.violet import shadow_live_factors as slf
priors_seen, counter = [], {"n": 0}
def live_source(client, *, scan_history, selector, ob_engine, bar_idx, prior_boost_beta):
priors_seen.append(prior_boost_beta)
counter["n"] += 1
b = float(counter["n"])
return SimpleNamespace(factors=SizingFactors(posture="APEX"), selected_asset="X",
acb_boost=b, acb_beta=b / 10.0)
shadow = slf.build_shadow_live_source(
client_factory=lambda: object(), selector_factory=lambda: object(),
source_factory=live_source, scan_history_factory=lambda: object(),
ob_engine_factory=lambda assets: object(),
)
shadow["engine"] = _NoDecisionEngine()
shadow["capital"] = 1.0
payload = {"assets": ["X"], "vel_div": -0.03}
for i in range(3):
slf.shadow_decision_step(shadow, payload, scan_number=i, now_ns=i, vel_div=-0.03, vol_ok=True)
# scan0: no prior yet; scan1: prior = scan0's (1.0, 0.1); scan2: prior = scan1's (2.0, 0.2)
assert priors_seen == [None, (1.0, 0.1), (2.0, 0.2)]

View File

@@ -1,8 +1,19 @@
"""V3.4c live BLUE source — BLUE-algo parity tests (boost/beta, signal-gen, OB, mc_scale).
These pin VIOLET's reconstruction to BLUE's ACTUAL behaviour:
- boost/beta: bit-identical to AdaptiveCircuitBreaker.get_dynamic_boost_from_hz
- signal-gen: params pinned to BLUE's ENGINE_KWARGS (nautilus_event_trader.py)
- OB: BLUE's HZOBProvider + OBFeatureEngine wiring (persistent engine, step_live/get_market)
- mc_scale: begin_day's cat/env thresholds (not the MC service status label)
See prod/docs/VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md.
"""
from __future__ import annotations
import json
import re
import sys
from dataclasses import dataclass
from pathlib import Path
import pytest
@@ -12,25 +23,27 @@ import hazelcast
from prod.clean_arch.violet.decision_engine import SizingFactors
from prod.clean_arch.violet.live_blue_source import (
HazelcastOBProvider,
BLUE_SIGNAL_GEN_KWARGS,
HZ_CLUSTER,
HZ_HOST,
LiveBlueScanHistory,
_derive_mc_scale,
_source_boost_beta,
_source_ob_market,
source_live_blue_sizing_factors,
)
from prod.clean_arch.violet.alpha_wrappers import VioletAssetSelector
from nautilus_dolphin.nautilus.alpha_signal_generator import AlphaSignalGenerator
@dataclass
class _FakeMap:
payloads: dict
def get(self, key):
return self.payloads.get(key)
def key_set(self):
return list(self.payloads.keys())
from nautilus_dolphin.nautilus.adaptive_circuit_breaker import AdaptiveCircuitBreaker
from nautilus_dolphin.nautilus.alpha_signal_generator import (
AlphaSignalGenerator,
LONG_VEL_DIV_THRESHOLD, LONG_VEL_DIV_EXTREME,
VEL_DIV_THRESHOLD, VEL_DIV_EXTREME,
)
TRADER = Path("/mnt/dolphinng5_predict/prod/nautilus_event_trader.py")
# ── fakes ────────────────────────────────────────────────────────────────────
class _FakeBlocking:
def __init__(self, payloads):
self._payloads = payloads
@@ -47,285 +60,351 @@ class _FakeClient:
self._maps = maps
def get_map(self, name):
return type("M", (), {"blocking": lambda self2: _FakeBlocking(self._maps[name])})()
payloads = self._maps.get(name, {})
return type("M", (), {"blocking": lambda self2, p=payloads: _FakeBlocking(p)})()
def test_hz_ob_provider_filters_and_parses_latest_payload():
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"asset_XRPUSDT_ob": json.dumps(
{"timestamp": 2.0, "bid_notional": [-1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.2, "beta": 0.3}),
"mc_forewarner_latest": json.dumps({"status": "ORANGE"}),
"esof_latest": json.dumps({"advisory_score": 0.4}),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "restored"})},
}
class _FakeOBEngine:
"""Stands in for a persistent OBFeatureEngine; records step_live calls."""
def __init__(self, median_imbalance=0.0, agreement_pct=0.0):
self.calls = []
self._mi = median_imbalance
self._ap = agreement_pct
def step_live(self, assets, bar_idx):
self.calls.append((tuple(assets), bar_idx))
def get_market(self, bar_idx, assets):
return type("M", (), {"median_imbalance": self._mi, "agreement_pct": self._ap})()
_EXF = {"funding_btc": 0.01, "dvol_btc": 55.0, "fng": 40.0, "taker": 1.2, "_acb_ready": True}
def _features(**extra):
base = {
"esof_latest": json.dumps({"advisory_score": 0.4}),
"mc_forewarner_latest": json.dumps({"catastrophic_prob": 0.15, "envelope_score": 0.5}),
"exf_latest": json.dumps(_EXF),
}
base.update(extra)
return base
# ── 1. boost/beta: bit-identical to the ACB recompute (NOT acb_boost) ─────────
def test_boost_beta_recomputed_identically_to_blue_acb():
eigen = {"w750_velocity": 0.0012, "assets": ["BTCUSDT"], "asset_prices": [100.0],
"scan_number": 5, "vel_div": -0.03}
client = _FakeClient({"DOLPHIN_FEATURES": _features(latest_eigen_scan=json.dumps(eigen))})
boost, beta = _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1)
ref = AdaptiveCircuitBreaker().get_dynamic_boost_from_hz(
date_str="2026-06-16", exf_snapshot=dict(_EXF), w750_velocity=0.0012, direction=-1,
)
provider = HazelcastOBProvider(client) # type: ignore[arg-type]
assert provider.get_assets() == ["BTCUSDT", "XRPUSDT"]
snap = provider.get_snapshot("BTCUSDT", 0.0)
assert snap is not None
assert snap.asset == "BTCUSDT"
assert snap.bid_notional.tolist() == [1.0, 2.0, 3.0, 4.0, 5.0]
assert boost == max(0.0, float(ref["boost"]))
assert beta == max(0.0, float(ref["beta"]))
def test_source_live_blue_sizing_factors_unit(monkeypatch):
class FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, assets, bar_idx):
assert "BTCUSDT" in assets
def get_market(self, ts, assets):
return type("M", (), {"median_imbalance": 0.12, "agreement_pct": 0.91})()
def test_boost_beta_w750_zero_passed_as_none_like_blue():
# BLUE: w750_velocity=float(w750) if w750 else None → 0.0 becomes None
eigen = {"w750_velocity": 0.0, "assets": ["BTCUSDT"], "asset_prices": [100.0]}
client = _FakeClient({"DOLPHIN_FEATURES": _features(latest_eigen_scan=json.dumps(eigen))})
boost, beta = _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1)
ref = AdaptiveCircuitBreaker().get_dynamic_boost_from_hz(
date_str="2026-06-16", exf_snapshot=dict(_EXF), w750_velocity=None, direction=-1,
)
assert (boost, beta) == (max(0.0, float(ref["boost"])), max(0.0, float(ref["beta"])))
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
def test_boost_beta_neutral_when_no_exf():
client = _FakeClient({"DOLPHIN_FEATURES": {}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1) == (1.0, 0.0)
def test_boost_beta_neutral_on_stale_exf_valueerror():
# >12h staleness → get_dynamic_boost_from_hz raises ValueError → BLUE stale fallback;
# VIOLET has no prior → neutral identity.
stale = dict(_EXF, _staleness_s={"funding_btc": 999999.0})
client = _FakeClient({"DOLPHIN_FEATURES": {"exf_latest": json.dumps(stale),
"latest_eigen_scan": json.dumps({"w750_velocity": 0.001})}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1) == (1.0, 0.0)
def test_boost_beta_keeps_prior_on_stale_exf():
# BLUE keeps _day_base_boost/_day_beta on stale exf; VIOLET keeps the seeded prior.
stale = dict(_EXF, _staleness_s={"funding_btc": 999999.0})
client = _FakeClient({"DOLPHIN_FEATURES": {"exf_latest": json.dumps(stale),
"latest_eigen_scan": json.dumps({"w750_velocity": 0.001})}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1,
prior=(1.37, 0.8)) == (1.37, 0.8)
def test_boost_beta_keeps_prior_when_no_exf():
client = _FakeClient({"DOLPHIN_FEATURES": {}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1,
prior=(1.5, 0.2)) == (1.5, 0.2)
def test_boost_beta_cold_start_no_prior_is_neutral():
# No prior AND no fresh exf (cold start) → neutral identity, never silently wrong.
client = _FakeClient({"DOLPHIN_FEATURES": {}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1, prior=None) == (1.0, 0.0)
def test_source_live_blue_threads_prior_boost_beta_on_stale():
stale = dict(_EXF, _staleness_s={"funding_btc": 999999.0})
client = _FakeClient({
"DOLPHIN_FEATURES": {
"exf_latest": json.dumps(stale),
"mc_forewarner_latest": json.dumps({"catastrophic_prob": 0.02, "envelope_score": 0.9}),
"latest_eigen_scan": json.dumps({"assets": ["BTCUSDT"], "asset_prices": [100.0], "w750_velocity": 0.001}),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
})
res = source_live_blue_sizing_factors(
client, assets=["BTCUSDT"], ob_engine=_FakeOBEngine(), prior_boost_beta=(1.42, 0.8),
)
assert res.factors.boost == 1.42 and res.factors.beta == 0.8
def test_boost_beta_does_not_read_published_acb_boost():
# An acb_boost scalar is present but MUST be ignored (BLUE recomputes from exf).
eigen = {"w750_velocity": 0.0012}
client = _FakeClient({"DOLPHIN_FEATURES": _features(
latest_eigen_scan=json.dumps(eigen),
acb_boost=json.dumps({"boost": 7.77, "beta": 7.77}), # poison: must NOT surface
)})
boost, beta = _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1)
assert boost != 7.77 and beta != 7.77
# ── 2. signal-gen params pinned to BLUE's live ENGINE_KWARGS ──────────────────
def _blue_engine_kwargs_block() -> str:
text = TRADER.read_text()
start = text.index("ENGINE_KWARGS = dict(")
end = text.index("\n)", start)
return text[start:end]
def _blue_kwarg(name: str):
m = re.search(rf"\b{name}\s*=\s*([^\s,]+)", _blue_engine_kwargs_block())
assert m, f"{name} not found in BLUE ENGINE_KWARGS"
raw = m.group(1).rstrip(",")
if raw in ("True", "False"):
return raw == "True"
return float(raw) if any(c in raw for c in ".-e") else int(raw)
@pytest.mark.parametrize("key", [
"vel_div_threshold", "vel_div_extreme", "dc_lookback_bars", "dc_min_magnitude_bps",
"use_direction_confirm", "dc_skip_contradicts", "dc_leverage_boost", "dc_leverage_reduce",
])
def test_signal_gen_params_match_blue_engine_kwargs(key):
# If a champion retune changes ENGINE_KWARGS, this fails — no silent divergence.
assert BLUE_SIGNAL_GEN_KWARGS[key] == _blue_kwarg(key)
def test_signal_gen_long_thresholds_track_kernel_constants():
assert BLUE_SIGNAL_GEN_KWARGS["vel_div_threshold"] == VEL_DIV_THRESHOLD
assert BLUE_SIGNAL_GEN_KWARGS["vel_div_extreme"] == VEL_DIV_EXTREME
assert BLUE_SIGNAL_GEN_KWARGS["long_vel_div_threshold"] == LONG_VEL_DIV_THRESHOLD
assert BLUE_SIGNAL_GEN_KWARGS["long_vel_div_extreme"] == LONG_VEL_DIV_EXTREME
def test_dc_status_uses_pinned_signal_gen_not_bare_default():
# dc_status must equal a BLUE signal_gen built with the SAME pinned params.
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
for idx, px in enumerate([100.0, 99.5, 99.0, 98.4, 97.8, 97.0, 96.2], start=1):
history.ingest_scan(
{
"scan_number": idx,
"timestamp": float(idx),
"vel_div": -0.031,
"target_asset": "BTCUSDT",
"assets": ["BTCUSDT"],
"asset_prices": [px],
}
)
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.4, "beta": 0.2}),
"mc_forewarner_latest": json.dumps({"status": "ORANGE"}),
"esof_latest": json.dumps({"advisory_score": 0.4}),
"latest_eigen_scan": json.dumps(
{
"scan_number": 8,
"timestamp": 8.0,
"vel_div": -0.031,
"target_asset": "BTCUSDT",
"assets": ["BTCUSDT"],
"asset_prices": [95.5],
}
),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "stalker"})},
}
history.ingest_scan({"scan_number": idx, "timestamp": float(idx), "vel_div": -0.031,
"assets": ["BTCUSDT"], "asset_prices": [px]})
scan = {"scan_number": 8, "timestamp": 8.0, "vel_div": -0.031, "assets": ["BTCUSDT"],
"asset_prices": [95.5]}
got = history.dc_status(scan)
ref = AlphaSignalGenerator(**BLUE_SIGNAL_GEN_KWARGS).generate(
vel_div=-0.031, vel_div_history=None,
asset_price_history=history.price_history("BTCUSDT"),
trade_direction=-1, asset="BTCUSDT", current_timestamp=8.0,
)
assert got == ref.dc_status
# ── 3. OB: BLUE's HZOBProvider + OBFeatureEngine wiring ───────────────────────
def test_ob_uses_persistent_engine_step_live_then_get_market():
eng = _FakeOBEngine(median_imbalance=0.12, agreement_pct=0.9)
mi, ap = _source_ob_market(["BTCUSDT", "ETHUSDT"], bar_idx=3, ob_engine=eng)
assert eng.calls == [(("BTCUSDT", "ETHUSDT"), 3)] # step_live(assets, bar_idx) — BLUE's call
assert (mi, ap) == (0.12, 0.9)
def test_ob_empty_assets_neutral():
assert _source_ob_market([], bar_idx=0) == (None, None)
def test_ob_engine_exception_neutral():
class _Boom:
def step_live(self, a, b):
raise RuntimeError("x")
def get_market(self, b, a):
raise AssertionError("unreachable")
assert _source_ob_market(["BTCUSDT"], bar_idx=0, ob_engine=_Boom()) == (None, None)
def test_ob_builds_blue_hzobprovider_with_live_coords(monkeypatch):
captured = {}
class _FakeProvider:
def __init__(self, *, hz_cluster, hz_host, assets):
captured.update(cluster=hz_cluster, host=hz_host, assets=list(assets))
class _FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, a, b):
pass
def get_market(self, b, a):
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})()
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.HZOBProvider", _FakeProvider)
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", _FakeEngine)
_source_ob_market(["BTCUSDT"], bar_idx=0) # ob_engine=None → must construct BLUE's provider
assert captured == {"cluster": HZ_CLUSTER, "host": HZ_HOST, "assets": ["BTCUSDT"]}
# ── 4. mc_scale: begin_day's cat/env thresholds (the V3.4c bug fix) ───────────
@pytest.mark.parametrize("cat, env, expected", [
(0.15, 0.5, 0.5), # orange via cat>0.10
(0.05, -0.5, 0.5), # orange via env<0 — old status-code returned GREEN→1.0 (bug)
(0.10, -0.001, 0.5),
(0.28, 0.5, 1.0), # red via cat>0.25 — old status-code: ORANGE→0.5 (bug)
(0.05, -1.5, 1.0), # red via env<-1.0
(0.30, -2.0, 1.0),
(0.05, 0.5, 1.0), # benign
(0.10, 0.0, 1.0),
])
def test_derive_mc_scale_matches_blue_begin_day_formula(cat, env, expected):
payload = json.dumps({"catastrophic_prob": cat, "envelope_score": env, "status": "IGNORED"})
assert _derive_mc_scale(payload) == expected
def test_derive_mc_scale_neutral_when_fields_missing_or_garbage():
assert _derive_mc_scale(json.dumps({"status": "ORANGE"})) == 1.0 # label must be ignored
assert _derive_mc_scale(json.dumps({"catastrophic_prob": 0.15})) == 1.0
assert _derive_mc_scale(json.dumps({"envelope_score": -0.5})) == 1.0
assert _derive_mc_scale("not json") == 1.0
assert _derive_mc_scale(None) == 1.0
assert _derive_mc_scale({"catastrophic_prob": "bad", "envelope_score": 0.5}) == 1.0
# ── 5. integration: full plane sourced + composed ────────────────────────────
def _client_with_scan(scan: dict, *, posture="STALKER", **feat) -> _FakeClient:
eigen = json.dumps({**scan, "w750_velocity": 0.0012})
return _FakeClient({
"DOLPHIN_FEATURES": _features(latest_eigen_scan=eigen, **feat),
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": posture})},
})
def test_source_live_blue_sizing_factors_full_plane():
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
for idx, px in enumerate([100.0, 99.5, 99.0, 98.4, 97.8, 97.0, 96.2], start=1):
history.ingest_scan({"scan_number": idx, "timestamp": float(idx), "vel_div": -0.031,
"assets": ["BTCUSDT"], "asset_prices": [px]})
scan = {"scan_number": 8, "timestamp": 8.0, "vel_div": -0.031,
"assets": ["BTCUSDT"], "asset_prices": [95.5]}
client = _client_with_scan(scan, posture="RESTORED")
ob = _FakeOBEngine(median_imbalance=0.12, agreement_pct=0.91)
res = source_live_blue_sizing_factors(
client,
assets=["BTCUSDT"],
scan_history=history,
client, assets=["BTCUSDT"], scan_history=history,
selector=VioletAssetSelector(lookback_horizon=7),
ob_engine=ob, bar_idx=8, date_str="2026-06-16",
)
assert isinstance(res.factors, SizingFactors)
assert res.factors.posture == "STALKER"
assert res.factors.mc_scale == 0.5
assert res.factors.boost == 1.4
assert res.factors.beta == 0.2
assert res.factors.posture == "RESTORED"
assert res.factors.esof_score == 0.4
assert res.factors.ob_median_imbalance == 0.12
assert res.factors.ob_agreement_pct == 0.91
assert res.factors.mc_scale == 0.5 # cat=0.15/env=0.5 → orange
assert res.factors.ob_median_imbalance == 0.12 and res.factors.ob_agreement_pct == 0.91
assert ob.calls == [(("BTCUSDT",), 8)]
# boost/beta = ACB recompute over the SAME exf+w750 (bit-identical pin)
ref = AdaptiveCircuitBreaker().get_dynamic_boost_from_hz(
date_str="2026-06-16", exf_snapshot=dict(_EXF), w750_velocity=0.0012, direction=-1)
assert res.factors.boost == max(0.0, float(ref["boost"]))
assert res.factors.beta == max(0.0, float(ref["beta"]))
assert res.factors.dc_status == "CONFIRM"
assert res.selected_asset == "BTCUSDT"
def test_source_live_blue_sizing_factors_preserves_skip_contradict(monkeypatch):
class FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, assets, bar_idx):
pass
def get_market(self, ts, assets):
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})()
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
for idx, px in enumerate([100.0, 100.5, 101.0, 101.6, 102.2, 102.9, 103.6], start=1):
history.ingest_scan(
{
"scan_number": idx,
"timestamp": float(idx),
"vel_div": -0.031,
"target_asset": "BTCUSDT",
"assets": ["BTCUSDT"],
"asset_prices": [px],
}
)
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.4, "beta": 0.2}),
"mc_forewarner_latest": json.dumps({"status": "GREEN"}),
"esof_latest": json.dumps({"advisory_score": 0.4}),
"latest_eigen_scan": json.dumps(
{
"scan_number": 8,
"timestamp": 8.0,
"vel_div": -0.031,
"target_asset": "BTCUSDT",
"assets": ["BTCUSDT"],
"asset_prices": [104.2],
}
),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
}
)
res = source_live_blue_sizing_factors(
client,
assets=["BTCUSDT"],
scan_history=history,
selector=VioletAssetSelector(lookback_horizon=7),
)
assert res.factors.dc_status == "SKIP_CONTRADICT"
def test_source_live_blue_sizing_factors_handles_anomalies():
client = _FakeClient({
"DOLPHIN_FEATURES": {
"exf_latest": "not json", # → boost/beta neutral
"mc_forewarner_latest": json.dumps({"catastrophic_prob": 0.02, "envelope_score": 0.9}),
"esof_latest": "not json",
"latest_eigen_scan": json.dumps({"assets": ["BTCUSDT"], "asset_prices": [0]}),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": ""})},
})
res = source_live_blue_sizing_factors(client, assets=["BTCUSDT"], ob_engine=_FakeOBEngine())
assert res.factors.posture == "APEX"
assert res.factors.mc_scale == 1.0
assert res.factors.boost == 1.0 and res.factors.beta == 0.0 # bad exf → ACB neutral
assert res.factors.esof_score is None
# OB engine is healthy (the anomalies are in exf/esof/prices), so it returns its 0.0/0.0.
assert res.factors.ob_median_imbalance == 0.0 and res.factors.ob_agreement_pct == 0.0
assert res.factors.dc_status == "NONE"
assert res.selected_asset == "BTCUSDT"
def test_live_blue_sequence_matches_blue_selector_and_dc_at_each_step(monkeypatch):
class FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, assets, bar_idx):
pass
def get_market(self, ts, assets):
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})()
def test_source_live_blue_neutral_ob_when_no_engine_and_no_assets():
# No ob_engine + no assets discoverable → OB neutral (None), no HZOBProvider connection.
client = _FakeClient({
"DOLPHIN_FEATURES": _features(latest_eigen_scan=json.dumps({"assets": [], "asset_prices": []})),
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
})
res = source_live_blue_sizing_factors(client) # assets=None, scan has none → []
assert res.factors.ob_median_imbalance is None and res.factors.ob_agreement_pct is None
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
def test_sequence_matches_blue_selector_and_dc_at_each_step():
selector = VioletAssetSelector(lookback_horizon=7)
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
signal_gen = AlphaSignalGenerator()
ref_gen = AlphaSignalGenerator(**BLUE_SIGNAL_GEN_KWARGS)
scans = [
{
"scan_number": 1,
"timestamp": 1.0,
"vel_div": -0.010,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [100.0, 200.0],
},
{
"scan_number": 2,
"timestamp": 2.0,
"vel_div": -0.031,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [99.0, 198.0],
},
{
"scan_number": 3,
"timestamp": 3.0,
"vel_div": -0.041,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [98.0, 196.0],
},
{
"scan_number": 4,
"timestamp": 4.0,
"vel_div": -0.031,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [97.0, 194.0],
},
{"scan_number": 1, "timestamp": 1.0, "vel_div": -0.010, "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [100.0, 200.0]},
{"scan_number": 2, "timestamp": 2.0, "vel_div": -0.031, "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [99.0, 198.0]},
{"scan_number": 3, "timestamp": 3.0, "vel_div": -0.041, "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [98.0, 196.0]},
{"scan_number": 4, "timestamp": 4.0, "vel_div": -0.031, "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [97.0, 194.0]},
]
for idx, scan in enumerate(scans, start=1):
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"asset_ETHUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [2, 3, 4, 5, 6], "ask_notional": [6, 5, 4, 3, 2],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.0, "beta": 0.0}),
"mc_forewarner_latest": json.dumps({"status": "GREEN"}),
"esof_latest": json.dumps({"advisory_score": 0.3}),
"latest_eigen_scan": json.dumps({**scan, "target_asset": "BTCUSDT"}),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
}
)
for i, scan in enumerate(scans, start=1):
client = _client_with_scan(scan, posture="APEX")
res = source_live_blue_sizing_factors(
client,
assets=["BTCUSDT", "ETHUSDT"],
scan_history=history,
selector=selector,
client, assets=["BTCUSDT", "ETHUSDT"], scan_history=history,
selector=selector, ob_engine=_FakeOBEngine(), bar_idx=i,
)
# BLUE selector parity
market = history.market_data(selector.lookback)
expected_pick = selector.pick(market, regime_direction=-1)
expected_asset = expected_pick.asset if expected_pick is not None else "BTCUSDT"
assert res.selected_asset == expected_asset
# BLUE signal parity
expected_signal = signal_gen.generate(
vel_div=float(scan["vel_div"]),
vel_div_history=None,
ref = ref_gen.generate(
vel_div=float(scan["vel_div"]), vel_div_history=None,
asset_price_history=history.price_history(expected_asset),
trade_direction=-1,
asset=expected_asset,
current_timestamp=float(scan["timestamp"]),
trade_direction=-1, asset=expected_asset, current_timestamp=float(scan["timestamp"]),
)
assert res.factors.dc_status == expected_signal.dc_status
assert res.factors.dc_status == ref.dc_status
def test_live_blue_sequence_rejects_anomalous_values_without_poisoning_history(monkeypatch):
class FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, assets, bar_idx):
pass
def get_market(self, ts, assets):
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})()
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
def test_sequence_rejects_anomalous_values_without_poisoning_history():
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
selector = VioletAssetSelector(lookback_horizon=7)
scan = {
"scan_number": 99,
"timestamp": 99.0,
"vel_div": -0.031,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [float("nan"), -1.0],
"target_asset": "BTCUSDT",
}
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.0, "beta": 0.0}),
"mc_forewarner_latest": json.dumps({"status": "GREEN"}),
"esof_latest": json.dumps({"advisory_score": 0.3}),
"latest_eigen_scan": json.dumps(scan),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
}
)
scan = {"scan_number": 99, "timestamp": 99.0, "vel_div": -0.031,
"assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [float("nan"), -1.0]}
client = _client_with_scan(scan, posture="APEX")
res = source_live_blue_sizing_factors(
client,
assets=["BTCUSDT", "ETHUSDT"],
scan_history=history,
selector=selector,
client, assets=["BTCUSDT", "ETHUSDT"], scan_history=history,
selector=VioletAssetSelector(lookback_horizon=7), ob_engine=_FakeOBEngine(),
)
assert res.selected_asset == "BTCUSDT"
assert res.factors.dc_status == "NONE"
@@ -333,43 +412,7 @@ def test_live_blue_sequence_rejects_anomalous_values_without_poisoning_history(m
assert history.price_history("ETHUSDT") == []
def test_source_live_blue_sizing_factors_handles_anomalies(monkeypatch):
class FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, assets, bar_idx):
raise RuntimeError("boom")
def get_market(self, ts, assets):
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})()
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": -9, "beta": "bad"}),
"mc_forewarner_latest": json.dumps({"status": "GREEN"}),
"esof_latest": "not json",
"latest_eigen_scan": json.dumps({"target_asset": "BTCUSDT", "assets": ["BTCUSDT"], "asset_prices": [0]}),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": ""})},
}
)
res = source_live_blue_sizing_factors(client, assets=["BTCUSDT"])
assert res.factors.posture == "APEX"
assert res.factors.mc_scale == 1.0
assert res.factors.boost == 0.0
assert res.factors.beta == 0.0
assert res.factors.esof_score is None
assert res.factors.ob_median_imbalance is None
assert res.factors.ob_agreement_pct is None
assert res.factors.dc_status == "NONE"
assert res.selected_asset == "BTCUSDT"
# ── 6. live HZ smoke (env-bound; deselect with -k "not live_hz_smoke") ────────
def test_live_hz_smoke_reads_current_state():
client = hazelcast.HazelcastClient(cluster_name="dolphin", cluster_members=["localhost:5701"])
try:
@@ -379,3 +422,4 @@ def test_live_hz_smoke_reads_current_state():
assert isinstance(res.factors, SizingFactors)
assert res.factors.posture in {"APEX", "STALKER", "RESTORED", "TURTLE", "HIBERNATE"}
assert res.factors.mc_scale in {0.5, 1.0}
assert res.factors.boost >= 0.0 and res.factors.beta >= 0.0

View File

@@ -0,0 +1,153 @@
"""V3.4 parity pin tests for BLUE sizing intermediates.
This is a direct regression pin, not a broad behavioral suite. It compares the
VIOLET replica against BLUE's live formulas for each sizing intermediate on a
deterministic grid, with exact equality.
"""
from __future__ import annotations
import sys
from pathlib import Path
import numpy as np
sys.path.insert(0, "/mnt/dolphinng5_predict")
sys.path.insert(0, "/mnt/dolphinng5_predict/nautilus_dolphin")
from prod.clean_arch.violet.sizing import VioletSizer
def _blue_engine(*, dc_leverage_boost: float = 1.0):
from nautilus_dolphin.nautilus.esf_alpha_orchestrator import NDAlphaEngine
eng = NDAlphaEngine(
initial_capital=69_000.0,
max_leverage=8.0,
abs_max_leverage=9.0,
min_leverage=0.5,
fraction=0.20,
use_asset_selection=False,
use_direction_confirm=True,
dc_leverage_boost=dc_leverage_boost,
)
eng.begin_day("2026-06-16", posture="APEX", direction=-1)
return eng
def _blue_market_ob_mult(median_imbalance: float, agreement_pct: float, trade_direction: int) -> float:
eff_imb = -median_imbalance if trade_direction == -1 else median_imbalance
if eff_imb > 0.08 and agreement_pct > 0.70:
return 1.0 + min(0.20, eff_imb * agreement_pct * 0.5)
if eff_imb < -0.08 and agreement_pct > 0.70:
return max(0.85, 1.0 - abs(eff_imb) * agreement_pct * 0.3)
return 1.0
def _ob_grid() -> list[tuple[float, float]]:
imbalances = np.linspace(-1.0, 1.0, 20, dtype=np.float64)
agreements = np.linspace(0.0, 1.0, 10, dtype=np.float64)
return [(float(i), float(a)) for i in imbalances for a in agreements]
def _dc_prices(status: str, lookback: int = 7) -> list[float]:
if status == "CONFIRM":
return [100.0 - 0.2 * i for i in range(lookback + 1)]
if status == "CONTRADICT":
return [100.0 + 0.2 * i for i in range(lookback + 1)]
return [100.0 for _ in range(lookback + 1)]
def test_strength_cubic_exact_parity_grid():
eng = _blue_engine()
sz = VioletSizer()
cases = []
for trade_direction in (-1, 1):
for vel_div in np.linspace(-0.06, 0.04, 200, dtype=np.float64):
cases.append((float(vel_div), trade_direction))
assert len(cases) >= 200
for vel_div, trade_direction in cases:
blue = eng._strength_cubic(vel_div, direction=trade_direction)
violet = sz.strength_cubic(vel_div, trade_direction=trade_direction)
assert violet == blue
def test_regime_size_mult_exact_parity_grid():
eng = _blue_engine()
sz = VioletSizer()
vel_divs = np.linspace(-0.06, 0.0, 50, dtype=np.float64)
boosts = (1.0, 1.25, 1.75, 2.5)
betas = (0.0, 0.2, 0.8)
mc_scales = (0.5, 1.0)
directions = (-1, 1)
n = 0
for trade_direction in directions:
eng.regime_direction = trade_direction
eng._day_posture = "APEX"
for vel_div in vel_divs:
for boost in boosts:
for beta in betas:
for mc_scale in mc_scales:
eng._day_base_boost = boost
eng._day_beta = beta
eng._day_mc_scale = mc_scale
eng._update_regime_size_mult(float(vel_div))
violet = sz.regime_size_mult(
float(vel_div),
boost=float(boost),
beta=float(beta),
mc_scale=float(mc_scale),
trade_direction=trade_direction,
)
assert violet == eng.regime_size_mult
n += 1
assert n >= 200
def test_market_ob_mult_exact_parity_grid():
sz = VioletSizer()
cases = _ob_grid()
assert len(cases) == 200
for trade_direction in (-1, 1):
for median_imbalance, agreement_pct in cases:
blue = _blue_market_ob_mult(median_imbalance, agreement_pct, trade_direction)
violet = sz.market_ob_mult(
median_imbalance,
agreement_pct,
trade_direction=trade_direction,
)
assert violet == blue
def test_dc_leverage_multiplier_exact_parity_grid():
sz = VioletSizer(dc_leverage_boost=1.25)
eng = _blue_engine(dc_leverage_boost=1.25)
sig = eng.signal_gen
short_vel_divs = np.linspace(-0.06, -0.021, 100, dtype=np.float64)
long_vel_divs = np.linspace(0.011, 0.04, 100, dtype=np.float64)
statuses = ("CONFIRM", "NEUTRAL", "SKIP_CONTRADICT")
n = 0
for trade_direction, vel_divs in ((-1, short_vel_divs), (1, long_vel_divs)):
for vel_div in vel_divs:
for status in statuses:
prices = _dc_prices(status)
result = sig.generate(
vel_div=float(vel_div),
vel_div_history=[float(vel_div)] * 10,
asset_price_history=prices,
trade_direction=trade_direction,
asset=None,
current_timestamp=0.0,
)
blue = sig.dc_leverage_boost if result.dc_status == "CONFIRM" else 1.0
violet = sz.dc_lev_mult(result.dc_status)
assert violet == blue
n += 1
assert n >= 200

View File

@@ -0,0 +1,140 @@
"""V4 arm-readiness reporter — self-test to death.
Spec: prod/docs/VIOLET_PASS_MM1_V4_READINESS_REPORTER.md
Style: follows test_violet_domain.py patterns.
"""
from __future__ import annotations
import sys
sys.path.insert(0, "/mnt/dolphinng5_predict")
import pytest
from hypothesis import given, settings
from hypothesis import strategies as st
from prod.clean_arch.violet.v4_readiness import (
V4ReadinessReport,
assess_v4_readiness,
)
ALL_FIELDS = [
"creds_ok", "canary_passed", "arming_gate_present",
"conviction_bit_identical", "selection_parity_ok", "feed_live",
]
def _all_true(**overrides: bool) -> dict:
return {f: overrides.get(f, True) for f in ALL_FIELDS}
def _single_false(field: str) -> dict:
return _all_true(**{field: False})
# ── happy path ────────────────────────────────────────────────────────────────
def test_all_true_ready_to_arm():
r = assess_v4_readiness(**_all_true())
assert r.ready_to_arm is True
assert r.reasons == []
# ── each single False yields ready_to_arm=False + correct reason ─────────────
@pytest.mark.parametrize("field", ALL_FIELDS)
def test_single_false_not_ready(field: str):
r = assess_v4_readiness(**_single_false(field))
assert r.ready_to_arm is False
assert len(r.reasons) == 1
assert field in r.reasons[0]
# ── hypothesis: invariant holds across all 2^6 combos ────────────────────────
@given(
creds_ok=st.booleans(),
canary_passed=st.booleans(),
arming_gate_present=st.booleans(),
conviction_bit_identical=st.booleans(),
selection_parity_ok=st.booleans(),
feed_live=st.booleans(),
)
@settings(max_examples=100)
def test_ready_to_arm_equals_all_inputs(
creds_ok: bool,
canary_passed: bool,
arming_gate_present: bool,
conviction_bit_identical: bool,
selection_parity_ok: bool,
feed_live: bool,
):
r = assess_v4_readiness(
creds_ok=creds_ok,
canary_passed=canary_passed,
arming_gate_present=arming_gate_present,
conviction_bit_identical=conviction_bit_identical,
selection_parity_ok=selection_parity_ok,
feed_live=feed_live,
)
expected = all((
creds_ok, canary_passed, arming_gate_present,
conviction_bit_identical, selection_parity_ok, feed_live,
))
assert r.ready_to_arm is expected
# ── mutation litmus: flipping all() to any() MUST break this test ────────────
def test_mutation_litmus_all_must_be_true():
"""If someone changes all() to any(), at least one all-False should become
ready_to_arm=True — this test catches it."""
r = assess_v4_readiness(**{f: False for f in ALL_FIELDS})
assert r.ready_to_arm is False
assert len(r.reasons) == 6
# ── poison: StrictModel rejects non-bool / extra fields ─────────────────────
def test_rejects_non_bool():
with pytest.raises(Exception):
V4ReadinessReport(
creds_ok={"ok": True}, # dict is not coercible to bool
canary_passed=True,
arming_gate_present=True,
conviction_bit_identical=True,
selection_parity_ok=True,
feed_live=True,
reasons=[],
)
def test_rejects_extra_fields():
with pytest.raises(Exception):
V4ReadinessReport(
creds_ok=True,
canary_passed=True,
arming_gate_present=True,
conviction_bit_identical=True,
selection_parity_ok=True,
feed_live=True,
reasons=[],
unknown_field="oops",
)
def test_rejects_frozen_mutation():
r = assess_v4_readiness(**_all_true())
with pytest.raises(Exception):
r.creds_ok = False # type: ignore[misc]
# ── reasons content check: False fields produce descriptive strings ──────────
def test_reasons_mention_gate_name():
r = assess_v4_readiness(**_single_false("feed_live"))
assert any("PASS2.6" in reason for reason in r.reasons)
if __name__ == "__main__":
raise SystemExit(pytest.main([__file__, "-v"]))

View File

@@ -0,0 +1,132 @@
"""Tests for the Violet venue-agnostic OB provider seam."""
from __future__ import annotations
import math
import sys
import time
import pytest
from pydantic import ValidationError
sys.path.insert(0, "/mnt/dolphinng5_predict")
sys.path.insert(0, "/mnt/dolphinng5_predict/nautilus_dolphin")
from nautilus_dolphin.nautilus.ob_features import OBFeatureEngine
from prod.clean_arch.violet.venue_ob_provider import (
MockTickVenueOBProvider,
VenueOBTick,
VioletVenueOBProvider,
)
def _records(base_ts: float = 1_700_000_000.0):
assets = ("BTCUSDT", "ETHUSDT")
rows = []
for asset_idx, asset in enumerate(assets):
for snap_idx in range(3):
ts = base_ts + 30.0 * snap_idx
bias = 0.12 if asset_idx == 0 else -0.04
bid = tuple(float((1 + bias) * (i + 1) * 1000.0) for i in range(5))
ask = tuple(float((1 - bias) * (i + 1) * 1000.0) for i in range(5))
depth = tuple(float(v / (100.0 + asset_idx)) for v in bid)
ask_depth = tuple(float(v / (100.0 + asset_idx)) for v in ask)
rows.append(
VenueOBTick(
timestamp=ts,
asset=asset,
bid_notional_levels=bid,
ask_notional_levels=ask,
bid_depth_levels=depth,
ask_depth_levels=ask_depth,
)
)
return rows
def test_provider_conforms_and_returns_snapshots():
provider = VioletVenueOBProvider(ticks=_records(), tolerance_s=60.0)
assert provider.get_assets() == ["BTCUSDT", "ETHUSDT"]
assert provider.get_snapshot_count("BTCUSDT") == 3
assert provider.get_all_timestamps("ETHUSDT").tolist() == [1_700_000_000.0, 1_700_000_030.0, 1_700_000_060.0]
snap = provider.get_snapshot("BTCUSDT", 1_700_000_031.0)
assert snap is not None
assert snap.asset == "BTCUSDT"
assert snap.bid_notional.shape == (5,)
assert snap.ask_notional.shape == (5,)
assert all(math.isfinite(float(v)) for v in snap.bid_notional)
assert all(math.isfinite(float(v)) for v in snap.ask_notional)
def test_poison_rejected_at_construction():
with pytest.raises(ValidationError):
VenueOBTick(
timestamp=-1.0,
asset="BTCUSDT",
bid_notional_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
ask_notional_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
bid_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
ask_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
)
with pytest.raises(ValidationError):
VenueOBTick(
timestamp=1.0,
asset="BTCUSDT",
bid_notional_levels=(1.0, 2.0, 3.0, 4.0),
ask_notional_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
bid_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
ask_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
)
with pytest.raises(ValidationError):
VenueOBTick(
timestamp=1.0,
asset="BTCUSDT",
bid_notional_levels=(1.0, 2.0, 3.0, 4.0, float("nan")),
ask_notional_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
bid_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
ask_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
)
def test_engine_step_live_and_get_market(monkeypatch):
base_ts = 1_700_000_000.0
provider = MockTickVenueOBProvider(
assets=("BTCUSDT", "ETHUSDT", "SOLUSDT"),
num_snapshots=4,
base_timestamp=base_ts,
interval_s=30.0,
imbalance_biases={"BTCUSDT": 0.14, "ETHUSDT": -0.06, "SOLUSDT": -0.02},
)
engine = OBFeatureEngine(provider)
monkeypatch.setattr(time, "time", lambda: base_ts + 31.0)
engine.step_live(["BTCUSDT", "ETHUSDT", "SOLUSDT"], bar_idx=7)
market = engine.get_market(7, ["BTCUSDT", "ETHUSDT", "SOLUSDT"])
placement = engine.get_placement("BTCUSDT", 7)
signal = engine.get_signal("BTCUSDT", 7)
for value in (market.median_imbalance, market.agreement_pct, market.depth_pressure):
assert math.isfinite(float(value))
for value in (placement.depth_1pct_usd, placement.depth_quality, placement.fill_probability, placement.spread_proxy_bps):
assert math.isfinite(float(value))
for value in (signal.imbalance, signal.imbalance_ma5, signal.imbalance_persistence, signal.depth_asymmetry, signal.withdrawal_velocity):
assert math.isfinite(float(value))
def test_callable_refresh_loads_ticks():
base_ts = 1_700_000_000.0
def source(asset: str):
for row in _records(base_ts):
if row.asset == asset:
yield row
provider = VioletVenueOBProvider(tick_source=source, assets=("BTCUSDT", "ETHUSDT"))
assert provider.get_snapshot_count("BTCUSDT") == 3
assert provider.get_snapshot("ETHUSDT", base_ts + 10.0) is not None

View File

@@ -0,0 +1,725 @@
"""VIOLET V4 execution runner.
This is the live-capable VIOLET runner boundary:
NG7 scan -> BLUE-faithful VIOLET decision -> ExecIntent
-> DITAv2 KernelIntent -> DITAv2 BingX VST venue execution
Safety is explicit rather than castrated: the module can submit real orders via
``ExecutionKernel.process_intent_async`` when the operator runs the V4 launcher
with VST keys and arming gates green. Unit tests inject fake kernels and never
touch a venue.
"""
from __future__ import annotations
import argparse
import asyncio
import gc
import json
import logging
import math
import os
import sys
import threading
import time
import uuid
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Callable, Mapping, Optional
from prod.clean_arch.dita_v2.contracts import (
KernelCommandType,
KernelIntent,
KernelOutcome,
TradeSide,
)
from .contracts_v3 import ExecIntent
from .decision_engine import ShadowDecision
from .exec_intent import to_exec_intent
RUNNER_CONTRACT_VERSION = "violet-v4-exec-runner-contracts-20260626"
LOGGER = logging.getLogger("violet.v4_execution_runner")
_ASEX_SRC = Path("/mnt/dolphinng5_predict/ASEx/src")
if _ASEX_SRC.exists() and str(_ASEX_SRC) not in sys.path:
sys.path.insert(0, str(_ASEX_SRC))
try: # pragma: no cover - exercised when ASEx package is installed/available.
from asex.guarded import ASExGuardedState
from asex.worker import ASExWorker
except Exception: # pragma: no cover - fallback is covered by unit tests.
ASExGuardedState = None
ASExWorker = None
def _finite_positive(value: float, name: str) -> float:
out = float(value)
if not math.isfinite(out) or out <= 0.0:
raise ValueError(f"{name} must be finite and > 0")
return out
def _non_empty(value: str, name: str) -> str:
out = str(value or "").strip()
if not out:
raise ValueError(f"{name} must be non-empty")
return out
def _trade_side(side: str) -> TradeSide:
normalized = str(side or "").strip().upper()
if normalized == "LONG":
return TradeSide.LONG
if normalized == "SHORT":
return TradeSide.SHORT
raise ValueError(f"unsupported VIOLET side: {side!r}")
def _action_from_intent(intent: ExecIntent, explicit: Optional[KernelCommandType]) -> KernelCommandType:
if explicit is not None:
return explicit
if intent.reason == "ENTRY":
return KernelCommandType.ENTER
if intent.reason == "EXIT":
return KernelCommandType.EXIT
raise ValueError(f"unsupported VIOLET intent reason: {intent.reason!r}")
def exec_intent_to_kernel_intent(
intent: ExecIntent,
*,
reference_price: float,
trade_id: str,
slot_id: int = 0,
intent_id: Optional[str] = None,
action: Optional[KernelCommandType] = None,
order_type: str = "MARKET",
limit_price: float = 0.0,
metadata: Optional[Mapping[str, Any]] = None,
timestamp: Optional[datetime] = None,
) -> KernelIntent:
"""Convert a VIOLET ``ExecIntent`` into a DITAv2 ``KernelIntent``.
The function is intentionally pure and strict. It does not fetch prices,
allocate state outside the returned object, or contact a venue. Live runner
code must pass a fresh reference price with provenance in ``metadata``.
"""
price = _finite_positive(reference_price, "reference_price")
qty = _finite_positive(float(intent.qty), "intent.qty")
leverage = _finite_positive(float(intent.exchange_leverage), "intent.exchange_leverage")
tid = _non_empty(trade_id, "trade_id")
oid = _non_empty(intent_id or f"violet-v4-{uuid.uuid4().hex}", "intent_id")
order = _non_empty(order_type, "order_type").upper()
if order == "LIMIT":
_finite_positive(limit_price, "limit_price")
elif float(limit_price or 0.0) < 0.0:
raise ValueError("limit_price must be >= 0")
merged_metadata: dict[str, Any] = dict(metadata or {})
merged_metadata.update(
{
"violet_contract_version": RUNNER_CONTRACT_VERSION,
"violet_ts_ns": int(intent.ts_ns),
"violet_maker_policy": intent.maker_policy,
"violet_target_notional": float(intent.target_notional),
"violet_reason": intent.reason,
}
)
return KernelIntent(
timestamp=timestamp or datetime.now(timezone.utc),
intent_id=oid,
trade_id=tid,
slot_id=int(slot_id),
asset=_non_empty(intent.asset, "intent.asset"),
side=_trade_side(intent.side),
action=_action_from_intent(intent, action),
reference_price=price,
target_size=qty,
leverage=leverage,
reason=f"violet_v4:{intent.reason.lower()}",
metadata=merged_metadata,
order_type=order,
limit_price=float(limit_price or 0.0),
)
@dataclass(frozen=True)
class VioletV4RunnerConfig:
"""Operator/runtime knobs for the V4 execution runner."""
session_id: str
capital: float = 69_000.0
slot_id: int = 0
maker_policy: str = "maker_both"
order_type: str = "MARKET"
limit_price: float = 0.0
poll_interval_s: float = 0.25
queue_maxsize: int = 4096
max_submissions: int = 0
max_notional_usdt: float = 0.0
assume_vol_ok: bool = False
require_arming: bool = True
report_dir: str = "/mnt/vp-VIOLET_main/prod/VIOLET_dev/reports"
hz_cluster: str = "dolphin"
hz_host: str = "localhost:5701"
hz_map: str = "DOLPHIN_FEATURES"
hz_key: str = "latest_eigen_scan"
snapshot_symbol: str = "BTCUSDT"
@classmethod
def from_env(cls) -> "VioletV4RunnerConfig":
return cls(
session_id=os.environ.get("DOLPHIN_VIOLET_SESSION_ID", uuid.uuid4().hex),
capital=float(os.environ.get("DOLPHIN_VIOLET_CAPITAL", "69000")),
slot_id=int(os.environ.get("DOLPHIN_VIOLET_SLOT_ID", "0")),
maker_policy=os.environ.get("DOLPHIN_VIOLET_MAKER_POLICY", "maker_both"),
order_type=os.environ.get("DOLPHIN_VIOLET_ORDER_TYPE", "MARKET"),
limit_price=float(os.environ.get("DOLPHIN_VIOLET_LIMIT_PRICE", "0")),
poll_interval_s=float(os.environ.get("DOLPHIN_VIOLET_POLL_INTERVAL_SEC", "0.25")),
queue_maxsize=int(os.environ.get("DOLPHIN_VIOLET_SCAN_QUEUE_MAX", "4096")),
max_submissions=int(os.environ.get("DOLPHIN_VIOLET_MAX_SUBMISSIONS", "0")),
max_notional_usdt=float(os.environ.get("DOLPHIN_VIOLET_MAX_NOTIONAL_USDT", "0")),
assume_vol_ok=str(os.environ.get("DOLPHIN_VIOLET_ASSUME_VOL_OK", "0")).lower()
in {"1", "true", "yes", "on"},
require_arming=str(os.environ.get("DOLPHIN_VIOLET_REQUIRE_ARMING", "1")).lower()
not in {"0", "false", "no", "off"},
report_dir=os.environ.get(
"DOLPHIN_VIOLET_REPORT_DIR",
"/mnt/vp-VIOLET_main/prod/VIOLET_dev/reports",
),
hz_cluster=os.environ.get("HZ_CLUSTER", "dolphin"),
hz_host=os.environ.get("HZ_HOST", "localhost:5701"),
hz_map=os.environ.get("DOLPHIN_VIOLET_SCAN_MAP", "DOLPHIN_FEATURES"),
hz_key=os.environ.get("DOLPHIN_VIOLET_SCAN_KEY", "latest_eigen_scan"),
snapshot_symbol=os.environ.get("DOLPHIN_VIOLET_SNAPSHOT_SYMBOL", "BTCUSDT"),
)
class _LocalGuardedState:
"""Tiny ASEx-compatible fallback for tests or missing ASEx install."""
def __init__(self) -> None:
self._lock = threading.Lock()
self.last_scan_number = -1
self.scans_seen = 0
self.decisions = 0
self.submissions = 0
self.outcomes = 0
self.errors = 0
self.last_trade_id = ""
self.last_intent_id = ""
def mutate(self, mutation: Mapping[str, Any]) -> dict[str, Any]:
with self._lock:
return self._apply(dict(mutation))
def _apply(self, mutation: dict[str, Any]) -> dict[str, Any]:
op = str(mutation.get("op") or "")
if op == "scan":
sn = int(mutation.get("scan_number") or 0)
if sn <= self.last_scan_number:
return {"accepted": False, "reason": "duplicate_scan", "last_scan_number": self.last_scan_number}
self.last_scan_number = sn
self.scans_seen += 1
return {"accepted": True, "last_scan_number": self.last_scan_number}
if op == "decision":
self.decisions += 1
return {"accepted": True, "decisions": self.decisions}
if op == "submission":
self.submissions += 1
self.last_trade_id = str(mutation.get("trade_id") or "")
self.last_intent_id = str(mutation.get("intent_id") or "")
return {"accepted": True, "submissions": self.submissions}
if op == "outcome":
self.outcomes += 1
return {"accepted": True, "outcomes": self.outcomes}
if op == "error":
self.errors += 1
return {"accepted": True, "errors": self.errors}
return {"accepted": False, "reason": f"unknown_op:{op}"}
if ASExGuardedState is not None:
class VioletRunnerGuardedState(ASExGuardedState[dict[str, Any], dict[str, Any]]):
"""ASEx single-writer state for scan/order lifecycle counters."""
def __init__(self) -> None:
super().__init__()
self._state = _LocalGuardedState()
def _validate(self, mutation: dict[str, Any]) -> bool:
return str(mutation.get("op") or "") in {
"scan",
"decision",
"submission",
"outcome",
"error",
}
def _apply(self, mutation: dict[str, Any]) -> dict[str, Any]:
return self._state._apply(dict(mutation))
else:
VioletRunnerGuardedState = _LocalGuardedState # type: ignore[assignment]
class SerialRunnerState:
"""Small wrapper around ASExWorker with a deterministic local fallback."""
def __init__(self) -> None:
self.backend = VioletRunnerGuardedState()
self.worker = ASExWorker(self.backend) if ASExWorker is not None else None
def mutate(self, mutation: Mapping[str, Any], *, timeout: float = 5.0) -> dict[str, Any]:
payload = dict(mutation)
if self.worker is not None:
return self.worker.mutate(payload).result(timeout=timeout)
return self.backend.mutate(payload) # type: ignore[attr-defined]
def close(self) -> None:
if self.worker is not None:
self.worker.close(timeout=2.0)
def _parse_scan_payload(raw: Any) -> dict[str, Any]:
if raw is None:
return {}
if isinstance(raw, str):
data = json.loads(raw)
elif isinstance(raw, Mapping):
data = dict(raw)
else:
return {}
if isinstance(data, dict) and data.get("version") == "NG7":
try:
from prod.clean_arch.adapters.eigen_scan_normalizer import normalize_ng7_scan
data = normalize_ng7_scan(data)
except Exception:
LOGGER.debug("NG7 normalize failed; using raw scan", exc_info=True)
return data if isinstance(data, dict) else {}
def reference_price_from_scan(payload: Mapping[str, Any], asset: str) -> float:
"""Extract the decision asset price from an NG7/universe payload."""
symbol = str(asset or "").upper()
assets = payload.get("assets") or []
prices = payload.get("asset_prices") or []
if isinstance(assets, list) and isinstance(prices, list):
for raw_asset, raw_price in zip(assets, prices):
if str(raw_asset).upper() != symbol:
continue
price = _finite_positive(float(raw_price), "reference_price")
return price
result = payload.get("result") if isinstance(payload, Mapping) else None
if isinstance(result, Mapping):
result_asset = str(result.get("asset") or payload.get("target_asset") or "").upper()
if result_asset == symbol:
return _finite_positive(float(result.get("price") or payload.get("price")), "reference_price")
payload_asset = str(payload.get("asset") or payload.get("target_asset") or "").upper()
if payload_asset == symbol and payload.get("price") is not None:
return _finite_positive(float(payload.get("price")), "reference_price")
raise ValueError(f"scan payload has no fresh price for {symbol}")
def _decision_from_shadow(
shadow: dict[str, Any],
payload: dict[str, Any],
*,
scan_number: int,
now_ns: int,
vel_div: float,
vol_ok: bool,
) -> Optional[ShadowDecision]:
"""Run the existing BLUE-faithful VIOLET shadow path and return the decision."""
from .shadow_live_factors import _ensure_ob_engine
shadow["engine"].observe(payload, scan_number)
live_source = shadow.get("live_source")
if live_source is None:
return None
ob_engine = _ensure_ob_engine(shadow, payload)
live_result = live_source(
shadow["client"],
scan_history=shadow["scan_history"],
selector=shadow["selector"],
ob_engine=ob_engine,
bar_idx=shadow.get("ob_bar_idx", 0),
prior_boost_beta=shadow.get("prior_boost_beta"),
)
shadow["last_live_source"] = live_result
shadow["ob_bar_idx"] = shadow.get("ob_bar_idx", 0) + 1
ab = getattr(live_result, "acb_boost", None)
bb = getattr(live_result, "acb_beta", None)
if ab is not None and bb is not None:
shadow["prior_boost_beta"] = (ab, bb)
decision = shadow["engine"].decide(
now_ns=now_ns,
scan_number=scan_number,
capital=shadow["capital"],
vel_div=vel_div,
vol_ok=vol_ok,
factors=live_result.factors,
)
if decision is None:
return None
journal = shadow.get("journal")
if journal is not None:
journal.journal(decision, mono_ns=now_ns)
return decision
class HazelcastNG7ScanSource:
"""Event-listener first, polling fallback source for NG7 scans."""
def __init__(self, config: VioletV4RunnerConfig) -> None:
self.config = config
self.client = None
self.map = None
self.blocking_map = None
self.queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue(maxsize=config.queue_maxsize)
self._loop: Optional[asyncio.AbstractEventLoop] = None
self._last_polled_scan = -1
async def connect(self) -> None:
import hazelcast
self._loop = asyncio.get_running_loop()
self.client = hazelcast.HazelcastClient(
cluster_name=self.config.hz_cluster,
cluster_members=[self.config.hz_host],
)
self.map = self.client.get_map(self.config.hz_map)
self.blocking_map = self.map.blocking()
def on_entry(event: Any) -> None:
try:
payload = _parse_scan_payload(getattr(event, "value", None))
if not payload:
return
loop = self._loop
if loop is None:
return
loop.call_soon_threadsafe(self._offer_payload, payload)
except Exception:
LOGGER.debug("scan listener callback failed", exc_info=True)
self.map.add_entry_listener(
include_value=True,
key=self.config.hz_key,
updated_func=on_entry,
added_func=on_entry,
)
def _offer_payload(self, payload: dict[str, Any]) -> None:
try:
self.queue.put_nowait(payload)
except asyncio.QueueFull:
try:
self.queue.get_nowait()
except asyncio.QueueEmpty:
pass
self.queue.put_nowait(payload)
async def next_payload(self) -> dict[str, Any]:
try:
return await asyncio.wait_for(self.queue.get(), timeout=self.config.poll_interval_s)
except asyncio.TimeoutError:
raw = self.blocking_map.get(self.config.hz_key)
payload = _parse_scan_payload(raw)
sn = int(payload.get("scan_number") or 0)
if sn <= self._last_polled_scan:
return {}
self._last_polled_scan = sn
return payload
async def close(self) -> None:
if self.client is not None:
self.client.shutdown()
class VioletV4ExecutionRunner:
"""Live-capable runner: decision path to actual DITAv2 execution."""
def __init__(
self,
*,
bundle: Any,
shadow: dict[str, Any],
config: VioletV4RunnerConfig,
state: Optional[SerialRunnerState] = None,
decision_step: Optional[
Callable[[dict[str, Any], dict[str, Any], int, int, float, bool], Optional[ShadowDecision]]
] = None,
) -> None:
self.bundle = bundle
self.shadow = shadow
self.config = config
self.state = state or SerialRunnerState()
self.decision_step = decision_step or (
lambda sh, payload, sn, now_ns, vd, vol_ok: _decision_from_shadow(
sh,
payload,
scan_number=sn,
now_ns=now_ns,
vel_div=vd,
vol_ok=vol_ok,
)
)
self._closed = False
async def connect(self) -> None:
connect = getattr(self.bundle.venue, "connect", None)
if connect is not None:
result = connect()
if asyncio.iscoroutine(result):
await result
def close(self) -> None:
if self._closed:
return
self._closed = True
try:
close = getattr(self.bundle, "close", None)
if close is not None:
close()
finally:
self.state.close()
def _capital(self) -> float:
try:
cap = float(self.bundle.kernel.account.snapshot.capital)
if math.isfinite(cap) and cap > 0.0:
return cap
except Exception:
pass
return float(self.config.capital)
def _maybe_cap_exec_intent(self, intent: ExecIntent, reference_price: float) -> ExecIntent:
cap = float(self.config.max_notional_usdt or 0.0)
if cap <= 0.0 or float(intent.target_notional) <= cap:
return intent
return ExecIntent(
asset=intent.asset,
side=intent.side,
qty=cap / reference_price,
exchange_leverage=intent.exchange_leverage,
maker_policy=intent.maker_policy,
target_notional=cap,
ts_ns=intent.ts_ns,
reason=intent.reason,
)
async def process_scan_payload(self, raw_payload: Any) -> Optional[KernelOutcome]:
payload = _parse_scan_payload(raw_payload)
if not payload:
return None
scan_number = int(payload.get("scan_number") or 0)
if scan_number <= 0:
return None
scan_state = self.state.mutate({"op": "scan", "scan_number": scan_number})
if not scan_state.get("accepted"):
return None
vel_div = float(payload.get("vel_div") or 0.0)
if not math.isfinite(vel_div):
vel_div = 0.0
vol_ok = bool(payload.get("vol_ok", True))
if self.config.assume_vol_ok:
vol_ok = True
now_ns = int(time.monotonic_ns())
decision = self.decision_step(self.shadow, payload, scan_number, now_ns, vel_div, vol_ok)
if decision is None:
return None
self.state.mutate({"op": "decision", "scan_number": scan_number, "asset": decision.asset})
reference_price = reference_price_from_scan(payload, decision.asset)
exec_intent = to_exec_intent(
decision,
capital=self._capital(),
reference_price=reference_price,
maker_policy=self.config.maker_policy,
)
exec_intent = self._maybe_cap_exec_intent(exec_intent, reference_price)
trade_id = f"VIOLET-{self.config.session_id[:8]}-{scan_number}"
intent_id = f"violet-v4-{self.config.session_id[:8]}-{scan_number}"
kernel_intent = exec_intent_to_kernel_intent(
exec_intent,
reference_price=reference_price,
trade_id=trade_id,
slot_id=self.config.slot_id,
intent_id=intent_id,
order_type=self.config.order_type,
limit_price=self.config.limit_price,
metadata={
"violet_session_id": self.config.session_id,
"scan_number": scan_number,
"price_source": "ng7_scan",
"raw_notional_before_runner_cap": float(decision.target_exposure),
},
)
submission = self.state.mutate(
{"op": "submission", "scan_number": scan_number, "trade_id": trade_id, "intent_id": intent_id}
)
if self.config.max_submissions and int(submission.get("submissions") or 0) > self.config.max_submissions:
LOGGER.critical("max submissions reached; refusing scan=%s trade_id=%s", scan_number, trade_id)
return None
LOGGER.warning(
"VIOLET V4 LIVE SUBMIT scan=%s asset=%s side=%s qty=%.8f lev=%.2f ref=%.8f trade=%s",
scan_number,
kernel_intent.asset,
kernel_intent.side.value,
kernel_intent.target_size,
kernel_intent.leverage,
kernel_intent.reference_price,
trade_id,
)
try:
outcome = await self.bundle.kernel.process_intent_async(kernel_intent)
self.state.mutate(
{
"op": "outcome",
"scan_number": scan_number,
"trade_id": trade_id,
"accepted": bool(getattr(outcome, "accepted", False)),
"diagnostic": str(getattr(getattr(outcome, "diagnostic_code", ""), "value", "")),
}
)
return outcome
except Exception as exc:
self.state.mutate({"op": "error", "scan_number": scan_number, "error": str(exc)})
raise
def _apply_runtime_optimizations() -> None:
if gc.isenabled():
gc.disable()
def _build_shadow(config: VioletV4RunnerConfig) -> dict[str, Any]:
from prod.ch_writer import ch_put_violet
from .decision_engine import VioletDecisionEngine
from .shadow_journal import VioletDecisionJournal
from .shadow_live_factors import build_shadow_live_source
thr = float(os.environ.get("DOLPHIN_VIOLET_ENTRY_VEL_DIV_THRESHOLD", "-0.02"))
return {
"engine": VioletDecisionEngine(entry_vel_div_threshold=thr),
"journal": VioletDecisionJournal(sink=ch_put_violet, session_id=config.session_id),
"capital": float(config.capital),
**build_shadow_live_source(),
}
def _build_real_bundle(config: VioletV4RunnerConfig) -> Any:
from prod.clean_arch.dita_v2.launcher import build_launcher_bundle
from prod.launch_dolphin_violet import (
DDL_APPLY_CMD,
_apply_violet_env,
_preflight_clickhouse,
_violet_table_present,
build_bingx_exec_client_config,
)
from .v4_arming import assess_v4_arming, write_v4_arming_report
_apply_violet_env()
missing = _preflight_clickhouse()
if missing:
raise RuntimeError(f"dolphin_violet tables missing: {missing}; run {DDL_APPLY_CMD}")
if not _violet_table_present("violet_decisions"):
raise RuntimeError(f"dolphin_violet table missing: violet_decisions; run {DDL_APPLY_CMD}")
report = assess_v4_arming(report_dir=config.report_dir, launcher_mode="execution")
write_v4_arming_report(report, report_dir=config.report_dir)
if config.require_arming and not report.can_arm:
raise RuntimeError(f"VIOLET V4 arming refused: {report.reasons}")
return build_launcher_bundle(
venue_mode="BINGX",
max_slots=1,
prefix="violet",
bingx_config=build_bingx_exec_client_config(),
)
async def run_live(
config: Optional[VioletV4RunnerConfig] = None,
*,
stop_after_first_submission: bool = False,
) -> None:
"""Run the actual VIOLET V4 BingX VST execution loop."""
cfg = config or VioletV4RunnerConfig.from_env()
_apply_runtime_optimizations()
bundle = _build_real_bundle(cfg)
shadow = _build_shadow(cfg)
source = HazelcastNG7ScanSource(cfg)
runner = VioletV4ExecutionRunner(bundle=bundle, shadow=shadow, config=cfg)
await runner.connect()
await source.connect()
LOGGER.critical(
"VIOLET V4 EXECUTION RUNNER ARMED: session=%s venue=BINGX_VST hz=%s[%s]",
cfg.session_id,
cfg.hz_map,
cfg.hz_key,
)
try:
while True:
payload = await source.next_payload()
if payload:
outcome = await runner.process_scan_payload(payload)
if stop_after_first_submission and outcome is not None:
return
await asyncio.sleep(0)
finally:
await source.close()
runner.close()
def main(argv: Optional[list[str]] = None) -> int:
parser = argparse.ArgumentParser(description="VIOLET V4 live BingX VST execution runner")
parser.add_argument("--once", action="store_true", help="process one queued/polled scan then exit")
args = parser.parse_args(argv)
config = VioletV4RunnerConfig.from_env()
if args.once:
config = dataclass_replace(config, max_submissions=1)
asyncio.run(run_live(config, stop_after_first_submission=args.once))
return 0
def dataclass_replace(config: VioletV4RunnerConfig, **updates: Any) -> VioletV4RunnerConfig:
data = config.__dict__.copy()
data.update(updates)
return VioletV4RunnerConfig(**data)
__all__ = [
"HazelcastNG7ScanSource",
"RUNNER_CONTRACT_VERSION",
"SerialRunnerState",
"VioletV4ExecutionRunner",
"VioletV4RunnerConfig",
"exec_intent_to_kernel_intent",
"main",
"reference_price_from_scan",
"run_live",
]
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,75 @@
"""V4 arm-readiness reporter — DARK, read-only, non-colliding.
Aggregates V4 gate outcomes into a typed report answering: "is V4 ready
to ARM the live canary?" Does NOT run gates — CONSUMES their results
(passed in) and applies fail-closed readiness logic.
Spec: prod/docs/VIOLET_PASS_MM1_V4_READINESS_REPORTER.md
"""
from __future__ import annotations
from .domain import StrictModel, typed
__all__ = ["V4ReadinessReport", "assess_v4_readiness"]
class V4ReadinessReport(StrictModel):
"""Typed V4 readiness report — all six gates must pass for ready_to_arm."""
creds_ok: bool
canary_passed: bool
arming_gate_present: bool
conviction_bit_identical: bool
selection_parity_ok: bool
feed_live: bool
reasons: list[str]
@property
def ready_to_arm(self) -> bool:
return all((
self.creds_ok,
self.canary_passed,
self.arming_gate_present,
self.conviction_bit_identical,
self.selection_parity_ok,
self.feed_live,
))
@typed
def assess_v4_readiness(
*,
creds_ok: bool,
canary_passed: bool,
arming_gate_present: bool,
conviction_bit_identical: bool,
selection_parity_ok: bool,
feed_live: bool,
) -> V4ReadinessReport:
"""Build a V4ReadinessReport from gate outcomes. Fail-closed: any False
means ready_to_arm is False. No partial-arm."""
reasons: list[str] = []
if not creds_ok:
reasons.append("creds_ok: PASS2.1 not satisfied")
if not canary_passed:
reasons.append("canary_passed: PASS2.3 canary not round-tripped flat")
if not arming_gate_present:
reasons.append("arming_gate_present: PASS2.2 not satisfied")
if not conviction_bit_identical:
reasons.append("conviction_bit_identical: PASS2.5 fidelity mismatch")
if not selection_parity_ok:
reasons.append("selection_parity_ok: PASS2.7/3.1 parity mismatch")
if not feed_live:
reasons.append("feed_live: PASS2.6 HZ feed stale or deaf")
return V4ReadinessReport(
creds_ok=creds_ok,
canary_passed=canary_passed,
arming_gate_present=arming_gate_present,
conviction_bit_identical=conviction_bit_identical,
selection_parity_ok=selection_parity_ok,
feed_live=feed_live,
reasons=reasons,
)

View File

@@ -0,0 +1,178 @@
"""VIOLET venue-agnostic OB provider seam.
This is a read-only, in-memory seam for future non-BLUE order book sources.
It produces the exact ``OBSnapshot`` shape expected by BLUE's
``OBFeatureEngine`` without opening any live exchange connection.
"""
from __future__ import annotations
from bisect import bisect_left
try:
from beartype.typing import Annotated, Callable, Iterable, Optional
except ImportError: # pragma: no cover - beartype always present in prod
from typing import Annotated, Callable, Iterable, Optional
import numpy as np
from pydantic import Field
from .domain import StrictModel, Symbol, typed
try:
from nautilus_dolphin.nautilus.ob_provider import OBProvider, OBSnapshot
except ImportError: # pragma: no cover - import path fallback for direct runs
from nautilus_dolphin.nautilus.ob_provider import OBProvider, OBSnapshot # type: ignore
OBLevel = Annotated[float, Field(ge=0.0, allow_inf_nan=False)]
Level5 = tuple[OBLevel, OBLevel, OBLevel, OBLevel, OBLevel]
class VenueOBTick(StrictModel):
"""Normalized OB tick for one asset at one point in time."""
timestamp: Annotated[float, Field(ge=0.0, allow_inf_nan=False)]
asset: Symbol
bid_notional_levels: Level5
ask_notional_levels: Level5
bid_depth_levels: Level5
ask_depth_levels: Level5
class VioletVenueOBProvider(OBProvider):
"""Venue-agnostic, in-memory OB provider.
Input comes from either an injected callable or a flat iterable of
``VenueOBTick`` records. No exchange transport lives here.
"""
def __init__(
self,
*,
ticks: Optional[Iterable[VenueOBTick]] = None,
tick_source: Optional[Callable[[str], Iterable[VenueOBTick]]] = None,
assets: Optional[Iterable[str]] = None,
tolerance_s: float = 30.0,
) -> None:
self.tolerance_s = float(tolerance_s)
self._tick_source = tick_source
self._assets = sorted({str(a) for a in assets or [] if str(a).strip()})
self._ticks: dict[str, list[VenueOBTick]] = {}
self._timestamps: dict[str, np.ndarray] = {}
if ticks is not None:
self.load_ticks(ticks)
if tick_source is not None and self._assets:
self.refresh()
@typed
def load_ticks(self, ticks: Iterable[VenueOBTick]) -> None:
grouped: dict[str, list[VenueOBTick]] = {}
for tick in ticks:
grouped.setdefault(tick.asset, []).append(tick)
for asset, rows in grouped.items():
rows.sort(key=lambda t: t.timestamp)
self._ticks[asset] = rows
self._timestamps[asset] = np.array([t.timestamp for t in rows], dtype=np.float64)
if asset not in self._assets:
self._assets.append(asset)
self._assets.sort()
@typed
def refresh(self) -> None:
if self._tick_source is None:
return
for asset in self._assets:
self.load_ticks(self._tick_source(asset))
def _to_snapshot(self, tick: VenueOBTick) -> OBSnapshot:
return OBSnapshot(
timestamp=float(tick.timestamp),
asset=tick.asset,
bid_notional=np.array(tick.bid_notional_levels, dtype=np.float64),
ask_notional=np.array(tick.ask_notional_levels, dtype=np.float64),
bid_depth=np.array(tick.bid_depth_levels, dtype=np.float64),
ask_depth=np.array(tick.ask_depth_levels, dtype=np.float64),
)
def get_snapshot(self, asset: str, timestamp: float) -> Optional[OBSnapshot]:
rows = self._ticks.get(asset)
if not rows:
return None
ts_arr = self._timestamps.get(asset)
if ts_arr is None or len(ts_arr) == 0:
return None
idx = bisect_left(ts_arr, float(timestamp))
candidates = [max(0, idx - 1), min(idx, len(ts_arr) - 1)]
best_idx = candidates[0]
best_dist = abs(ts_arr[best_idx] - timestamp)
for cand in candidates[1:]:
dist = abs(ts_arr[cand] - timestamp)
if dist < best_dist:
best_dist = dist
best_idx = cand
if best_dist > self.tolerance_s:
return None
return self._to_snapshot(rows[best_idx])
def get_assets(self) -> list[str]:
return list(self._assets)
def get_all_timestamps(self, asset: str) -> np.ndarray:
ts = self._timestamps.get(asset)
return np.array([], dtype=np.float64) if ts is None else ts.copy()
def get_snapshot_count(self, asset: str) -> int:
rows = self._ticks.get(asset)
return len(rows) if rows is not None else 0
def get_snapshot_by_index(self, asset: str, idx: int) -> Optional[OBSnapshot]:
rows = self._ticks.get(asset)
if rows is None or idx < 0 or idx >= len(rows):
return None
return self._to_snapshot(rows[idx])
class MockTickVenueOBProvider(VioletVenueOBProvider):
"""Deterministic synthetic tick source for tests."""
def __init__(
self,
*,
assets: Optional[Iterable[str]] = None,
num_snapshots: int = 8,
base_timestamp: float = 1_700_000_000.0,
interval_s: float = 30.0,
base_notional: float = 100_000.0,
depth_scale: float = 1.0,
imbalance_biases: Optional[dict[str, float]] = None,
tolerance_s: float = 60.0,
) -> None:
assets = list(assets or ("BTCUSDT", "ETHUSDT"))
ticks: list[VenueOBTick] = []
level_weights = (1.0, 2.0, 3.0, 4.0, 5.0)
for asset_idx, asset in enumerate(assets):
bias = (imbalance_biases or {}).get(asset, 0.08 if asset_idx % 2 == 0 else -0.08)
for snap_idx in range(num_snapshots):
ts = base_timestamp + snap_idx * interval_s
drift = 1.0 + 0.01 * snap_idx
bid_mult = 1.0 + bias
ask_mult = 1.0 - bias
bid_not = tuple(
float(base_notional * depth_scale * drift * w * bid_mult) for w in level_weights
)
ask_not = tuple(
float(base_notional * depth_scale * drift * w * ask_mult) for w in level_weights
)
approx_price = 100.0 + 5.0 * asset_idx
bid_dep = tuple(float(v / approx_price) for v in bid_not)
ask_dep = tuple(float(v / approx_price) for v in ask_not)
ticks.append(
VenueOBTick(
timestamp=float(ts),
asset=asset,
bid_notional_levels=bid_not,
ask_notional_levels=ask_not,
bid_depth_levels=bid_dep,
ask_depth_levels=ask_dep,
)
)
super().__init__(ticks=ticks, assets=assets, tolerance_s=tolerance_s)

View File

@@ -0,0 +1,86 @@
# h5i — Cross-Agent Messaging (AUTHORITATIVE — read this to connect)
**This is THE single source of truth for agent comms. If anything elsewhere conflicts, this wins.**
Last updated 2026-06-22 by claude (integrator) to fix a bus split that left agents unable to see each
other's messages.
---
## 0. TL;DR — how to be reachable (do this first)
```bash
cd /mnt/dolphinng5_predict # ← THE CANONICAL BUS lives here. Always run h5i from here.
export H5I_AGENT=<your-handle> # e.g. codex / cmd-PASS9 / cmd-pass6 / cmd-pass5 / claude
h5i msg inbox # read + mark your unread
```
**Rule: always operate h5i with cwd = `/mnt/dolphinng5_predict`.** That checkout's `refs/h5i/msg` is
the ONE canonical bus everyone shares. Do not send/read from a fork clone — that's what split us.
---
## 1. Why we split (so it doesn't happen again)
`refs/h5i/msg` is **per-clone**. We had two diverged copies:
- **CANONICAL:** `/mnt/dolphinng5_predict` refs/h5i/msg (also mirrored read-only at `/mnt/vp-oa4`).
- **FORK:** `/mnt/vp-PASS9` refs/h5i/msg — where PASS9 had been reading/writing.
They never reconciled because **`h5i share push` targets the Gitea origin
(`…/siloqy.git`), which HANGS** (broken server-side hook). So claude's messages (on canonical) never
reached PASS9 (on the fork), and vice-versa. **Do not rely on share-to-Gitea.**
## 2. The fix — converge on the canonical bus
**Everyone reads/writes the canonical bus by running h5i from `/mnt/dolphinng5_predict`.** That path
is the shared checkout all agents can reach; one ref, no push/pull, no divergence.
If you are forced to work inside a fork clone (e.g. `/mnt/vp-PASS9`, whose git `origin` is the LOCAL
path `/mnt/dolphinng5_predict`), sync the bus **locally** (this does NOT touch Gitea, so it works):
```bash
cd /mnt/vp-PASS9
h5i share pull # pulls h5i refs from your local origin (/mnt/dolphinng5_predict)
# … work, send …
h5i share push # pushes back to the LOCAL origin (fine). NEVER push to the Gitea remote.
```
But the simpler, recommended path is just: **don't** — run h5i from `/mnt/dolphinng5_predict` directly.
## 3. Identities (current)
| Handle | Role | Current PASS2 task |
|---|---|---|
| **claude** | Integrator (owns merges to `main`, bit-verification) | — |
| **codex** | Parity / V3.4b live-factor / HZ | **PASS2.5** conviction soak |
| **cmd-PASS9** | ASEx kernel / race-safety | **PASS2.2** arming-gate, **PASS2.6** HZ client |
| **cmd-pass6** | Execution internals (OrderFSM/FillPump/Reconciler) | **PASS2.3** VST adapter, **PASS2.4** wiring |
| **cmd-pass5** | Mock-BingX adapter | **PASS2.1** VST creds + isolation |
| **aider_nvnemo** | Aider assistant (this session) | **PASS2.x** ad-hoc support |
Set yours: `export H5I_AGENT=<handle>` (or `h5i msg as <handle>`).
## 4. Usage
```bash
h5i msg # inbox dashboard (glance)
h5i msg inbox # show unread, mark read (numbers them)
h5i msg send <recipient> "…" # free-text ( 'all' = broadcast )
h5i msg ask <recipient> "…" # a request expecting a reply
h5i msg reply <n> "…" # threaded reply to message #n
h5i msg ack|done|decline <n> "…"
h5i msg wait --timeout 600 # block until a reply arrives (run as a background waiter)
git show refs/h5i/msg:messages.jsonl # raw view
```
Types: ASK · REVIEW_REQUEST · RISK · HANDOFF · BROADCAST · ACK · DONE.
## 5. Code workflow (so comms + code don't re-split) — see `VIOLET_PASS2_SERIES_INDEX.md`
- **Single trunk:** `/root/violet` `main` on local disk; shared bare origin **`/root/violet.git`**
(NOT Gitea). Your PASS2 worktree is `/root/violet-wt/pass2.<n>-<slug>` on branch
`agent/pass2.<n>-<slug>`. **Work there, push to `/root/violet.git`.** Do not branch from `/mnt`
CIFS clones.
- Sub-specs: `prod/docs/VIOLET_PASS2.<n>_*.md` (also replicated to `/mnt/dolphinng5_predict/prod/docs/`).
- Commit hygiene: explicit staging (never `git add .`), `Co-Authored-By:` trailer, tests in-commit.
## 6. De-confliction note
This file supersedes the comms sections of `VIOLET_CMD_ONBOARDING__PASS5.md` and any per-pass spec —
those defer here for *how to connect*. Operator points all agents at THIS file.

View File

@@ -0,0 +1,91 @@
# VIOLET ↔ BLUE parity: structural divergence & re-derivation debt
**Date:** 2026-06-16
**Status:** ACKNOWLEDGED TRADEOFF / open architectural debt
**Raised by:** operator, during the V3.4c review.
## The problem, stated plainly
VIOLET reproduces BLUE's sizing behaviour **bit-for-bit by intent**, but it does so in a
**different module / file / scope structure** than BLUE. BLUE's logic lives in one place —
the monolithic `NDAlphaEngine` (`nautilus_dolphin/nautilus_dolphin/nautilus/esf_alpha_orchestrator.py`),
which holds day-state (`_day_base_boost`, `_day_beta`, `_day_mc_scale`, `_day_posture`),
constructs its own `signal_gen`/`bet_sizer`, and runs `begin_day` / `_try_entry` inline.
VIOLET re-expresses that same logic spread across:
`sizing.py`, `live_blue_source.py`, `live_factors.py`, `live_factor_source.py`,
`decision_engine.py`, `alpha_wrappers.py`.
**Consequence (the operator's concern, verbatim intent):** because VIOLET imitates the
computations *while* ending up with a different structure, any *orderly, systemic,
verifiable* BLUE↔VIOLET algo parity comparison — and any future refactor of either side —
is **much harder**. The surfaces do not line up 1:1, so a diff between the two engines is
not mechanical; it requires a human to know which VIOLET fragment mirrors which BLUE line.
## Two kinds of reuse — and only one is safe
1. **WRAPPED kernels (safe — single source of truth).** VIOLET imports and calls BLUE's
actual kernel objects. A BLUE change propagates automatically.
- `esof_size_mult_from_score`, `parse_esof_payload`, `esof_score_from_payload`
(`esof_size_gate.py`) — wrapped by `sizing.py` / `live_factor_source.py`.
- `OBFeatureEngine.get_market` (`ob_features.py`) — wrapped by `live_blue_source.py`.
- `AlphaSignalGenerator.generate` (`alpha_signal_generator.py`) — wrapped by `live_blue_source.py`.
- `AlphaAssetSelector` / `AlphaBetSizer` — wrapped by `alpha_wrappers.py`.
- `map_internal_conviction_to_exchange_leverage` (`bingx/leverage.py`) — wrapped by `exchange_leverage.py`.
2. **HAND-REPLICATED arithmetic (the debt — duplicated formulas, drift-prone).** VIOLET
transcribes BLUE's pure float arithmetic into its own functions. A BLUE change here is
SILENT in VIOLET until someone notices.
## Re-derivation inventory (the drift liabilities)
| Computation | BLUE authority (file:line) | VIOLET replica | Parity safety-net today | Drift risk |
|---|---|---|---|---|
| 5-factor compose + caps | `esf_alpha_orchestrator.py:600-619` | `sizing.VioletSizer.compose` | `@gate` Monte-Carlo vs REAL orchestrator (bit-identity) | LOW (gated) |
| `regime_size_mult` = boost·(1+β·s³)·mc | `esf_alpha_orchestrator.py:898-909` | `sizing.VioletSizer.regime_size_mult` | same gate | LOW-MED |
| `strength_cubic` | `esf_alpha_orchestrator.py:872-885` | `sizing.VioletSizer.strength_cubic` | same gate | LOW-MED |
| `market_ob_mult` consensus | `esf_alpha_orchestrator.py:587-595` | `sizing.VioletSizer.market_ob_mult` | same gate | MED |
| `dc_lev_mult` | `esf_alpha_orchestrator.py:575-577` | `sizing.VioletSizer.dc_lev_mult` | unit only | MED (but ≡1.0 while dc_leverage_boost=1.0) |
| **`mc_scale`** | `esf_alpha_orchestrator.py:956-962` (`begin_day`) | `live_blue_source._derive_mc_scale` | parametrized formula test (8 cases inc. divergences) | LOW-MED (only remaining hand-replica) |
| `boost`/`beta` source | trader recompute `acb.get_dynamic_boost_from_hz(exf_latest)` | **FIXED 2026-06-16**: `_source_boost_beta` calls the SAME `get_dynamic_boost_from_hz` over exf_latest+w750 (bare `AdaptiveCircuitBreaker()`, no ob_engine) | bit-identity test vs the real ACB | LOW |
| `dc_status` config | `signal_gen` built with ENGINE_KWARGS `:180-191` | **FIXED**: `AlphaSignalGenerator(**BLUE_SIGNAL_GEN_KWARGS)` | param test parses ENGINE_KWARGS from trader source | LOW |
| OB feed | live OB accumulation via `HZOBProvider` | **FIXED**: BLUE's own `HZOBProvider` + `OBFeatureEngine` + `step_live`/`get_market` (persistent engine + bar_idx) | wiring test (HZ coords, step_live call) | LOW (caller must pass persistent engine) |
**Update 2026-06-16:** the three MED-risk flags above were brought to bit-identity per
operator directive ("VIOLET should do *identically* what BLUE does"). boost/beta now call
the SAME `get_dynamic_boost_from_hz` BLUE's trader calls (the published `acb_boost` is NOT
used); dc_status uses `AlphaSignalGenerator` pinned to BLUE's ENGINE_KWARGS; OB uses BLUE's
`HZOBProvider`. The reinvented `HazelcastOBProvider` and the `_extract_acb`/`status`-label
paths were deleted. The ONLY remaining hand-replicated arithmetic is `_derive_mc_scale`
(begin_day computes it inline inside an un-callable method), pinned by a formula test.
OPEN follow-up: the launcher (`shadow_decision_step`) must pass a PERSISTENT `ob_engine` +
per-scan-incrementing `bar_idx` into `source_live_blue_sizing_factors` so OB accumulation
matches BLUE across scans; today it single-shots, which is wired-correctly but historyless.
## Why we accept it (for now)
- The kernels that carry the heavy alpha are WRAPPED, not copied.
- The composition arithmetic IS gated bit-for-bit against the real orchestrator.
- VIOLET must stay a *read-only, DARK* mirror of a *running* BLUE; it cannot import BLUE's
live in-process day-state, so some reconstruction from published HZ surfaces is unavoidable.
## Mitigations (recommended, not yet done)
1. **Parity-pin every hand-replicated formula.** For each row above, add a test that
imports BLUE's authoritative function/constant and asserts VIOLET's replica equals it
over a sampled grid — converting silent drift into a red test. Where BLUE's logic is
trapped inside `begin_day` (mc_scale), refactor a *pure* `mc_scale_from(cat, env)` helper
**on the BLUE side** (BLUE-domain change, operator-gated) that BOTH engines call.
2. **Single ambiguity owner.** Surfaces like MC (`status` label vs `begin_day` thresholds)
and ACB (published `acb_boost` vs trader recompute) have two disagreeing sources; pick
ONE canonical per factor and document it (see `_derive_mc_scale` docstring).
3. **Backplane convergence (the real fix).** When the DITAv2 Rust middleware becomes the
shared backplane, BOTH BLUE and VIOLET should consume factors from it rather than each
computing/replicating — collapsing this divergence at the source. Until then, every new
hand-replication MUST be logged in this table.
## Maintenance rule
Any change to a BLUE formula in the left column REQUIRES a matching change + test update in
the VIOLET replica in the same PR. Any NEW hand-replication MUST add a row here.

View File

@@ -0,0 +1,425 @@
# VIOLET Full System Assessment
This is the current source-faithful assessment of how the full VIOLET system works in this checkout, mapped against the BLUE doctrine in `prod/docs/SYSTEM_BIBLE_v7.md` and the VIOLET spec files. The current `SYSTEM_BIBLE_v7.md` file is a `v7.2` document in this tree; the BLUE v7 / v7.1 lineage is treated here as the same doctrinal source family.
Scope:
- start from launcher and invocation points
- trace the BLUE-equivalent layers through the current VIOLET codepaths
- identify exactly where DITAv2 executes
- isolate ASEx involvement
- list the data inputs and input sites
- describe the VIOLET-only support systems
- record worktree provenance for the files and file-groups that implement the path
What follows is a map of the live code, not a proposal.
## 1. Canonical sources
BLUE doctrine and system behavior:
- `prod/docs/SYSTEM_BIBLE_v7.md`
- `prod/docs/DITA_V2_KERNEL_REFERENCE.md`
VIOLET doctrine and rollout intent:
- `prod/docs/VIOLET_DEV_SPEC_AND_PLAN.md`
- `prod/docs/VIOLET_DEV_SPEC_AND_PLAN_SOAK.md`
- `prod/docs/VIOLET_DEV_SPEC_AND_PLAN_SOAK__DEV_STATUS_20260627.md`
- `prod/docs/VIOLET_PASS_MM1_V4_READINESS_REPORTER.md`
- `prod/docs/ASEx_INTEGRATION_STATUS.md`
- `prod/docs/DITA_V2_OPERATOR_PLAYBOOK.md`
Key implementation files:
- `prod/launch_dolphin_violet.py`
- `prod/launch_dolphin_violet_v4.py`
- `prod/clean_arch/violet/v4_execution_runner.py`
- `prod/clean_arch/violet/v4_arming.py`
- `prod/clean_arch/violet/v4_readiness.py`
- `prod/clean_arch/violet/shadow_live_factors.py`
- `prod/clean_arch/violet/live_blue_source.py`
- `prod/clean_arch/violet/live_factor_source.py`
- `prod/clean_arch/violet/shadow_journal.py`
- `prod/clean_arch/violet/divergence.py`
- `prod/clean_arch/violet/observe_guard.py`
- `prod/clean_arch/violet/pass2_8_soak_runner.py`
- `prod/clean_arch/violet/layer_parity_harness.py`
- `prod/clickhouse/violet/apply_violet_ddl.py`
- `prod/clickhouse/violet/*.sql`
- `prod/clean_arch/dita_v2/launcher.py`
- `prod/clean_arch/dita_v2/rust_backend.py`
- `prod/clean_arch/dita_v2/bingx_venue.py`
- `prod/clean_arch/dita_v2/mock_venue.py`
- `prod/clean_arch/dita_v2/control.py`
- `prod/clean_arch/dita_v2/projection.py`
- `prod/clean_arch/dita_v2/hazelcast_projection.py`
- `prod/supervisor/dolphin-supervisord.conf`
- `prod/supervisor/run_with_dolphin_env.sh`
## 2. Invocation points
### 2.1 Observe-only VIOLET
Primary entrypoint:
- `prod/launch_dolphin_violet.py`
What it does:
- fixes the VIOLET namespace before anything else reads env
- preflights all `dolphin_violet.*` tables with `SELECT 0 ... LIMIT 0`
- if tables are missing, it idles dark and names the DDL applier
- builds a DITAv2 launcher bundle, then wraps the venue in `ObserveOnlyVenue`
- writes VIOLET persistence and Hazelcast state into VIOLET-only maps and tables
- starts a divergence monitor and an optional muted shadow path
- never calls the execution kernel for live order placement
Observed behavior:
- `dolphin_violet` is the observe-only runtime
- it is designed to be safe when BLUE is present because it reads BLUE state but does not write BLUE tables
### 2.2 VIOLET live V4 execution
Wrapper entrypoint:
- `prod/launch_dolphin_violet_v4.py`
Live runner:
- `prod/clean_arch/violet/v4_execution_runner.py`
What it does:
- consumes NG7 scans from Hazelcast
- converts scans into VIOLET decisions
- converts decisions into `ExecIntent`
- converts `ExecIntent` into DITAv2 `KernelIntent`
- submits the kernel intent to the DITAv2 execution kernel
- requires arming gates and VST-only settings before live execution is allowed
Current runtime note:
- in this checkout the observe-only service is running
- the V4 execution service is registered but remains stopped because arming is not GO
### 2.3 DITAv2 operator surface
General DITAv2 launcher:
- `prod/launch_dita_v2.py`
Operator playbook:
- `prod/docs/DITA_V2_OPERATOR_PLAYBOOK.md`
This is the lower-level DITAv2 surface. VIOLET uses the same launcher bundle construction, but VIOLET adds its own scan ingestion, decision logic, shadow journal, arming gate, and execution wrapper on top.
## 3. BLUE layers and VIOLET equivalents
The BLUE system bible describes a layered trading stack. VIOLET mirrors that structure, but with its own namespaces and additional arming/execution scaffolding.
| BLUE layer / concern | VIOLET equivalent | Main files | Input sites | Effect |
|---|---|---|---|---|
| Invocation / process control | VIOLET launcher and supervisor surface | `prod/launch_dolphin_violet.py`, `prod/launch_dolphin_violet_v4.py`, `prod/supervisor/dolphin-supervisord.conf`, `prod/supervisor/run_with_dolphin_env.sh` | env, supervisor, shell wrapper | selects observe-only or live V4 posture |
| Scan ingest / feature feed | Hazelcast NG7 scan source | `prod/clean_arch/violet/v4_execution_runner.py`, `prod/clean_arch/violet/live_blue_source.py` | `DOLPHIN_FEATURES.latest_eigen_scan` | provides scan payloads and BLUE live factors |
| Posture / capital sensing | BLUE posture and capital readback | `prod/clean_arch/violet/live_blue_source.py`, `prod/clean_arch/violet/live_factor_source.py` | `DOLPHIN_STATE_BLUE.latest_nautilus`, `DOLPHIN_STATE_BLUE.engine_snapshot` | feeds posture-aware factorization |
| Alpha / conviction / sizing | VIOLET decision engine and sizing helpers | `prod/clean_arch/violet/decision_engine.py`, `prod/clean_arch/violet/sizing.py`, `prod/clean_arch/violet/alpha_wrappers.py`, `prod/clean_arch/violet/v4_execution_runner.py` | scan payload, live BLUE factor source | produces `ExecIntent` from BLUE-faithful decision logic |
| Shadow audit / decision logging | Shadow journal and parity surfaces | `prod/clean_arch/violet/shadow_journal.py`, `prod/clean_arch/violet/parity_report.py` | decision rows, scan numbers | writes `dolphin_violet.violet_decisions` |
| Divergence / feed hygiene | VIOLET feed divergence monitor | `prod/clean_arch/violet/divergence.py` | scan stream and runtime samples | writes `dolphin_violet.violet_feed_divergence` |
| Execution boundary | DITAv2 kernel bridge | `prod/clean_arch/violet/v4_execution_runner.py`, `prod/clean_arch/dita_v2/launcher.py`, `prod/clean_arch/dita_v2/rust_backend.py`, `prod/clean_arch/dita_v2/bingx_venue.py` | `ExecIntent`, arming report, BingX env | runs actual DITAv2 execution path |
| Safety / no-touch guard | Observe-only venue wrapper | `prod/clean_arch/violet/observe_guard.py` | venue object | prevents any order placement in dark mode |
| Schema / persistence | VIOLET-only ClickHouse schema | `prod/clickhouse/violet/*.sql`, `prod/clickhouse/violet/apply_violet_ddl.py` | DDL files, table probes | creates and verifies `dolphin_violet.*` only |
| Readiness / arming | V4 arming gate | `prod/clean_arch/violet/v4_readiness.py`, `prod/clean_arch/violet/v4_arming.py` | soak reports, env, keys | decides whether V4 may arm |
## 4. Exact execution flow
### 4.1 Observe-only launcher flow
`prod/launch_dolphin_violet.py` does this in order:
1. applies VIOLET env names
2. probes `dolphin_violet` tables
3. if tables are missing, logs and idles dark
4. if dark divergence is enabled or keys exist, starts the divergence task
5. if no VIOLET keys are present, idles dark
6. otherwise constructs a DITAv2 launcher bundle
7. wraps the venue in `ObserveOnlyVenue`
8. wires VIOLET persistence and Hazelcast state writers
9. builds the muted shadow path
10. runs the observe-only loop
Important point:
- it does not submit live orders
- it does not execute `runtime.step()`
- it is a read-mostly / audit / shadow surface, not the live trading boundary
### 4.2 Live V4 execution flow
`prod/clean_arch/violet/v4_execution_runner.py` is the live boundary. The chain is:
1. `run_live()` loads env and runtime options
2. it applies VIOLET runtime env and optimizations
3. it builds the real DITAv2 bundle
4. it builds the shadow/live-factor source
5. it connects to Hazelcast NG7 scan source
6. it waits for a scan payload from `DOLPHIN_FEATURES.latest_eigen_scan`
7. `process_scan_payload()` parses and validates the payload
8. the scan number is deduplicated in guarded state
9. live factors are read from BLUE state and feature maps
10. the decision step produces a VIOLET decision
11. the decision is transformed to `ExecIntent`
12. the intent may be capped by configured notional limits
13. `ExecIntent` becomes DITAv2 `KernelIntent`
14. the runner mutates submission state
15. `bundle.kernel.process_intent_async(kernel_intent)` is awaited
16. the runner records outcome or error in guarded state
The exact DITAv2 execution point is step 15. That is where VIOLET hands off from its own decision plane into DITAv2 execution.
### 4.3 Where DITAv2 runs
DITAv2 is constructed by `prod/clean_arch/dita_v2/launcher.py` as a launcher bundle:
- kernel
- control plane
- projection
- zinc plane
- venue adapter
In VIOLET dark mode:
- the venue is wrapped in `ObserveOnlyVenue`
- the kernel exists, but the venue refuses live order placement
In V4 execution mode:
- the same bundle is built with BingX VST configuration
- the intent is submitted through the kernel into the venue adapter
- that is the only place VIOLET is allowed to reach live execution
## 5. Input data and input sites
### 5.1 BLUE live inputs consumed by VIOLET
VIOLET reads these BLUE surfaces as inputs:
- `DOLPHIN_STATE_BLUE.latest_nautilus`
- fallback `DOLPHIN_STATE_BLUE.engine_snapshot`
- `DOLPHIN_FEATURES.esof_latest`
- fallback `DOLPHIN_FEATURES.esof_advisor_latest`
- `DOLPHIN_FEATURES.mc_forewarner_latest`
- `DOLPHIN_FEATURES.latest_eigen_scan`
These are read-only input sites. VIOLET should not mutate BLUE state maps.
### 5.2 VIOLET scan input
The active scan input for VIOLET is:
- Hazelcast map `DOLPHIN_FEATURES`
- key `latest_eigen_scan`
That is the scan source the live runner consumes. The older scan bridge path is superseded in this layout.
### 5.3 VIOLET persistence inputs
VIOLET writes into:
- `dolphin_violet.policy_events`
- `dolphin_violet.trade_reconstruction`
- `dolphin_violet.trade_exit_legs`
- `dolphin_violet.position_state`
- `dolphin_violet.anomaly_events`
- `dolphin_violet.account_events`
- `dolphin_violet.status_snapshots`
- `dolphin_violet.trade_events`
- `dolphin_violet.v7_decision_events`
- `dolphin_violet.violet_feed_divergence`
- `dolphin_violet.violet_decisions`
These are VIOLET-only tables. They are not BLUE tables and should not be used to backfill BLUE state directly.
## 6. ASEx involvement
ASEx exists in the tree, but the current VIOLET mainline use is narrow.
Confirmed current VIOLET codepath:
- `prod/clean_arch/violet/v4_execution_runner.py` optionally imports:
- `asex.guarded.ASExGuardedState`
- `asex.worker.ASExWorker`
- if ASEx is unavailable, the runner falls back to a local guarded state implementation
- the ASEx wrapper is used for guarded serial state around scan/order lifecycle counters
What this means:
- ASEx is not the execution engine
- ASEx is not the DITAv2 kernel
- ASEx is not the default VIOLET decision path
- it is an optional serial-state wrapper in the live runner boundary
Cross-check from the integration status doc:
- ASEx is installed as an editable package
- VIOLET main has zero ASEx imports outside the live runner boundary
- DITAv2 main has zero ASEx imports
- the earlier PASS9 adapter work exists in another worktree and is not merged here
Operational conclusion:
- ASEx is currently peripheral, not foundational, in this checkout
- the only current VIOLET runtime touchpoint is the guarded state wrapper inside `v4_execution_runner.py`
## 7. VIOLET-only support systems
### 7.1 Read-only dark guard
`prod/clean_arch/violet/observe_guard.py` implements `ObserveOnlyVenue`.
Purpose:
- wraps a venue
- raises on order placement
- allows the launcher to run a full decision stack without touching a live venue
### 7.2 Shadow journal
`prod/clean_arch/violet/shadow_journal.py`
Purpose:
- persists executed shadow decisions to `dolphin_violet.violet_decisions`
- keeps the shadow path auditable
- matches the ClickHouse DDL in `prod/clickhouse/violet/22_violet_decisions.sql`
### 7.3 Feed divergence monitor
`prod/clean_arch/violet/divergence.py`
Purpose:
- records feed divergence into `dolphin_violet.violet_feed_divergence`
- gives VIOLET its own drift surface separate from BLUE
### 7.4 V4 readiness and arming
`prod/clean_arch/violet/v4_readiness.py`
`prod/clean_arch/violet/v4_arming.py`
Purpose:
- combine soak readiness, credential checks, namespace isolation, and launcher mode checks
- fail closed when the readiness report is not GO
- keep V4 from arming unless the environment says it may
Current observed behavior in this checkout:
- arming is not GO
- the live runner therefore remains stopped
### 7.5 Soak runner and parity harness
`prod/clean_arch/violet/pass2_8_soak_runner.py`
`prod/clean_arch/violet/layer_parity_harness.py`
Purpose:
- run the DARK -> MOCK -> VST canary progression used in VIOLET soak work
- compare VIOLET behavior against the expected layer parity surface
- keep execution preparation separate from the live V4 runner
## 8. Storage and schema
`prod/clickhouse/violet/apply_violet_ddl.py` is the schema applier for VIOLET.
Important characteristics:
- it only targets `dolphin_violet`
- it probes table existence before use
- it does not create BLUE tables
- it applies each SQL file as a separate statement path
Relevant SQL set:
- `00_create_database.sql`
- `01_policy_events.sql`
- `02_trade_reconstruction.sql`
- `03_trade_exit_legs.sql`
- `04_position_state.sql`
- `05_anomaly_events.sql`
- `06_account_events.sql`
- `07_status_snapshots.sql`
- `08_trade_events.sql`
- `09_v7_decision_events.sql`
- `10_adaptive_exit_shadow.sql`
- `11_fee_settled_events.sql`
- `12_sc_bucket_gauge_shadow.sql`
- `13_sc_threshold_advisor_shadow.sql`
- `20_violet_feed_divergence.sql`
- `22_violet_decisions.sql`
The important storage boundary is simple:
- VIOLET uses `dolphin_violet.*`
- BLUE uses `dolphin.*` and `DOLPHIN_*` control/state maps
- VIOLET should not write into BLUE storage
## 9. BLUE-equivalent layer mapping, expanded
### 9.1 Data / scan layer
BLUE uses `DOLPHIN_FEATURES.latest_eigen_scan` as the canonical scan source.
VIOLET uses the same scan feed as its input, but via the live runner and shadow source:
- `prod/clean_arch/violet/v4_execution_runner.py`
- `prod/clean_arch/violet/shadow_live_factors.py`
- `prod/clean_arch/violet/live_blue_source.py`
### 9.2 Signal / conviction layer
BLUE has a large signal layer family in the system bible.
VIOLET's equivalent is:
- `decision_engine.py`
- `sizing.py`
- `alpha_wrappers.py`
- `shadow_live_factors.py`
- `live_factor_source.py`
This is where the BLUE-like live factors are folded into VIOLET conviction and sizing.
### 9.3 Execution / accounting layer
BLUE's execution/accounting boundary is the DITA family in the bible.
VIOLET's equivalent is:
- `prod/clean_arch/dita_v2/launcher.py`
- `prod/clean_arch/dita_v2/rust_backend.py`
- `prod/clean_arch/dita_v2/bingx_venue.py`
- `prod/clean_arch/dita_v2/mock_venue.py`
- `prod/clean_arch/violet/v4_execution_runner.py`
This is the actual order-facing path when V4 is armed.
### 9.4 Observability / divergence
BLUE's observability surfaces are mirrored in VIOLET by:
- `shadow_journal.py`
- `divergence.py`
- `v4_readiness.py`
- `v4_arming.py`
- `parity_report.py`
These keep the VIOLET branch auditable without mutating BLUE.
## 10. Provenance appendix
Worktree state here means the current checkout state observed during this assessment.
| File group | Worktree status | Last-known committer | Responsible actor |
|---|---|---|---|
| `prod/launch_dolphin_violet.py` | modified | Codex, `fb344318aa849fde13b934bdba7337819cc42743` | current local edit in this session |
| `prod/clickhouse/violet/apply_violet_ddl.py` | modified | Codex, `99e529c32ab353027fd8e53e5219d5c949ec06ce` | current local edit in this session |
| `prod/supervisor/dolphin-supervisord.conf` | modified | Codex, `02ecc55c1670019682b193a239c72ab0f66ab1fc` | current local edit in this session |
| `prod/clean_arch/violet/v4_execution_runner.py` | untracked | new file in worktree | Codex |
| `prod/clean_arch/violet/v4_arming.py` | untracked | new file in worktree | Codex |
| `prod/launch_dolphin_violet_v4.py` | untracked in this checkout view | new file in worktree | Codex |
| `prod/launch_dita_v2.py` | untracked in this checkout view | new file in worktree | Codex |
| `prod/supervisor/run_with_dolphin_env.sh` | untracked in this checkout view | new file in worktree | Codex |
| `prod/clean_arch/dita_v2/launcher.py` | clean in this worktree during this assessment | not revalidated here | upstream DITAv2 owner / baseline |
| `prod/clean_arch/dita_v2/rust_backend.py` | clean in this worktree during this assessment | not revalidated here | upstream DITAv2 owner / baseline |
| `prod/clean_arch/dita_v2/bingx_venue.py` | clean in this worktree during this assessment | not revalidated here | upstream DITAv2 owner / baseline |
| `prod/clean_arch/dita_v2/mock_venue.py` | clean in this worktree during this assessment | not revalidated here | upstream DITAv2 owner / baseline |
| `prod/clean_arch/violet/shadow_live_factors.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/live_blue_source.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/live_factor_source.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/shadow_journal.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/divergence.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/observe_guard.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/v4_readiness.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/docs/SYSTEM_BIBLE_v7.md` | clean in this worktree during this assessment | not revalidated here | doctrine baseline |
| `prod/docs/DITA_V2_KERNEL_REFERENCE.md` | clean in this worktree during this assessment | not revalidated here | doctrine baseline |
| `prod/docs/VIOLET_DEV_SPEC_AND_PLAN*.md` | clean in this worktree during this assessment | not revalidated here | doctrine baseline |
Notes:
- for files modified or added in this session, the responsible actor is the current local worktree edit path
- for clean baseline files, I did not re-resolve the exact last commit for every single file in this pass
- the assessment is still source-faithful because the runtime path is anchored to the specific files above and the current worktree status is explicit
## 11. Bottom line
VIOLET in this checkout is a BLUE-faithful, VIOLET-namespaced stack with three distinct surfaces:
- observe-only VIOLET, which mirrors the full decision plane but blocks venue writes
- shadow / readiness / divergence, which audit and gate behavior
- live V4 execution, which feeds scan payloads into DITAv2 only after arming passes
The decisive boundary is the handoff from `ExecIntent` into `KernelIntent` and then `bundle.kernel.process_intent_async(...)`.
That is where VIOLET stops being analysis and becomes actual execution through DITAv2.

View File

@@ -0,0 +1,87 @@
# VIOLET — OB feed sourcing, HZ Bridge, and multi-agent coordination
Date: 2026-06-16. Operator-driven decisions captured during the V3.4c/d review.
## 1. OB feed sourcing doctrine
**VIOLET consumes BLUE's EXTANT published OBF feed — it does NOT store OB data.**
- `live_blue_source._source_ob_market` and `shadow_live_factors._default_ob_engine_factory`
use BLUE's OWN `HZOBProvider` (`nautilus_dolphin/.../hz_ob_provider.py`) + `OBFeatureEngine`,
wired exactly as BLUE's `_wire_obf` (nautilus_event_trader.py:4967-4980).
- `HZOBProvider` is a **read-only Hazelcast entry-listener cache**: it subscribes to the
`asset_*_ob` shards BLUE already publishes and keeps only the latest snapshot per asset in
memory. Verified: **no writes, no persistence, no new exchange WS**. OB raw data is the
heaviest disk load in the system; VIOLET adds ZERO to it by reading the published shards.
- `OBFeatureEngine` keeps an in-memory lookback window only (no disk). The launcher holds
ONE engine for the service lifetime (`shadow["ob_engine"]`), stepping a per-scan `bar_idx`,
so OB accumulation matches BLUE across scans (a fresh per-scan engine has no history).
**Swap seam for future multi-exchange OB.** The `OBProvider` behind `OBFeatureEngine` is the
single swap point. To run a SEPARATE OB stream (e.g. BingX testnet / a third-party venue,
which has genuinely different OB than the Binance reference), replace `HZOBProvider` with a
direct-WS provider feeding the SAME `OBFeatureEngine` — no other code changes. This is
specced/reasonable and the design already accommodates it.
**Cadence compliance (VIOLET's whole point — faster-than-OBF, event-driven).** Two regimes:
- NOW (shadow / parity): match BLUE, which samples OB at the OBF cadence (~1s native). You
cannot be *more* faithful than BLUE's own OB sampling, so reading the published shards is
correct for parity. `HZOBProvider` is itself a PUSH entry-listener (not polling), which is
already event-driven and aligned with the reactor model.
- LATER (V5/V6 sub-second): when VIOLET wants OB faster than BLUE's OBF, it swaps in a direct
WS OB provider (the seam above) on the reactor clock. That is a DELIBERATE VIOLET feature /
divergence, gated separately — not a parity break of the current shadow stage.
## 2. HZ Bridge — TODO_HZBRIDGE (refactor ASAP when it ships)
An upcoming **Hazelcast Bridge** (`dolphinng5_predict/hzbridge`) will be the sanctioned way to
connect to Hazelcast, mitigating the silent client-death / lockup / dropout class (see the
`hz_client_death_investigation` memory + black-box dump work in BLUE). **All VIOLET raw
`HazelcastClient` / `HZOBProvider` connections must route through the bridge once available.**
In-code `TODO_HZBRIDGE` markers flag the three touch points:
1. `shadow_live_factors.build_shadow_live_source` — the launcher's `client_factory`.
2. `shadow_live_factors._default_ob_engine_factory``HZOBProvider`'s own connection.
3. `live_blue_source` HZ_CLUSTER/HZ_HOST + `_source_ob_market` provider construction; the
live-HZ smoke test.
Refactor priority: ASAP after the bridge lands (or accelerate the bridge). Until then, VIOLET
uses raw clients, accepting the known fragility (it is DARK, so a dropout loses shadow rows,
never orders).
## 3. Multi-agent coordination (worktrees + doctrine/status)
Agents on this box: **Claude, CommandCode, Codex, Crush.** The 2026-06-16 incident — one
agent's staged files swept into another's commit (a doc landed in the forbidden `dita_v2/`) —
was caused by **multiple agents sharing ONE working tree + ONE `.git/index`**. `index.lock`
serializes plumbing ops; it does NOT isolate logical work, and `git commit` commits the WHOLE
index regardless of which files you `git add`. Careful add does not protect you — only
isolation does.
**Target model (industry standard):**
- **One `git worktree` per agent** over the shared object DB: each gets its own working dir +
index + HEAD, so collisions are impossible.
```
git worktree add ../vp-claude -b agent/claude
git worktree add ../vp-commandcode -b agent/commandcode
git worktree add ../vp-codex -b agent/codex
git worktree add ../vp-crush -b agent/crush
```
Each agent works ONLY in its own tree. (This is what the Claude Code harness's
`isolation: "worktree"` already does for subagents.)
- **Branch-per-agent → integrate via Gitea PRs.** Push agent branches to the Gitea remote
(`hjnormey/siloqy`); merge to a canonical branch via review. `main`/`release` protected.
- **Doctrine / release = a protected canonical branch + tags.** The LIVE working tree (the one
BLUE/PINK actually run from) tracks the canonical branch only; tag doctrinal snapshots
(`git tag release-YYYYMMDD`). Anything not on the canonical branch is WIP by definition —
that answers "which files are doctrinal vs in-work".
- **Agent work-status board = Gitea branch + PR list.** Each open PR / agent branch (with its
ahead/behind + last-commit author) IS the status dashboard. Optionally a top-level
`AGENT_WORKLOG.md` or the existing `.beads/` tracker for human-readable status.
**Critical secondary point:** this working tree is ALSO the live deployment path (BLUE runs
`prod/nautilus_event_trader.py` from here). Agents editing it directly means WIP code sits in
the live path — a stray restart could load half-finished edits. Worktrees fix this too:
agents edit isolated trees; deployment becomes an explicit checkout/merge of the canonical
branch onto the live tree.
Setup is operator-gated (it reorganizes how all four agents work + touches the live tree), so
it is documented here for greenlight rather than executed unilaterally.

View File

@@ -0,0 +1,149 @@
# VIOLET — partial spec for another agent (OA TODO)
Date: 2026-06-16. Carved from the VIOLET dev plan: the pending items that are
**self-contained** — a separate agent can complete each WITHOUT touching another
agent's in-flight files, with clear tests + pass criteria.
---
## 0. HARD RULES (apply to every task below — non-negotiable)
1. **Never edit shared files.** Forbidden to modify: `prod/nautilus_event_trader.py`,
`prod/clean_arch/dita_v2/**`, `prod/clean_arch/dita/decision.py`, `nautilus_dolphin/**`,
`prod/clean_arch/dita_v2/blue_parity.py`, `prod/bingx/leverage.py`. You may READ them.
2. **VIOLET stays DARK** — no orders, no live execution, no VST keys. No starting/stopping
services. No Hazelcast restart. No PROGREEN.
3. **V-TYPES on all new code**`StrictModel` / `Annotated[... Field]` / `@typed` (beartype)
per `prod/clean_arch/violet/domain.py`. No arbitrary magnitude caps; only faithful
poison-guards (finite / non-negative where BLUE guarantees it).
4. **Each task below is NEW-FILE-ONLY** by design — do NOT modify these in-flight files
(another agent owns them right now): `live_blue_source.py`, `shadow_live_factors.py`,
`live_factor_source.py`, `decision_engine.py`, `sizing.py`. Add new modules/tests instead.
## 0a. COMMIT / BRANCH POLICY (read this — three shared-index collisions happened on 2026-06-16)
Multiple agents share ONE working tree + ONE `.git/index` on this box. `git commit` commits
the ENTIRE index, so a concurrent agent's staged files get swept into your commit (one landed
in forbidden `dita_v2/`). Therefore:
1. **Work in your own `git worktree`** if at all possible:
`git worktree add ../vp-oa -b agent/oa-violet` and do ALL work there. This makes collisions
impossible. (See `prod/docs/VIOLET_OB_FEED_AND_AGENT_COORDINATION.md` §3.)
2. **If you must share the working tree**, NEVER `git add -A` / `git add .`. Stage your exact
files, and commit with an explicit pathspec so only your paths are committed:
`git commit -F msg.txt -- path/to/your_new_file.py path/to/your_test.py`
Verify after every commit: `git show --stat --format="" HEAD` must list ONLY your files.
3. **One commit per task**, message prefix `VIOLET OA:`; end with the Co-Authored-By trailer.
Before reporting done: `git diff --name-only HEAD~1` ∌ any forbidden shared path.
4. **Run tests on the prod interpreter**: `/home/dolphin/siloqy_env/bin/python3 -m pytest -q`.
The mount is CIFS-slow (a 30-test file ≈ 60-150s); that is normal, not a hang. Use
`git grep` (not recursive `grep -r`/`find`, which time out at ~2min).
---
## TASK 1 — Parity-pin tests for the hand-replicated sizing arithmetic
**Why.** `prod/clean_arch/violet/sizing.py` hand-transcribes BLUE arithmetic from
`esf_alpha_orchestrator.py` (regime_size_mult :898-909, market_ob_mult :587-595,
strength_cubic :872-885, the 5-factor compose :600-619). The `@gate` Monte-Carlo proves the
COMPOSED leverage bit-identical, but there is no PER-FORMULA pin that fails loudly if BLUE
changes one factor's formula. This task adds those pins. See
`prod/docs/VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md` (mitigation #1).
**Affected files (NEW only):**
- `prod/clean_arch/violet/test_violet_sizing_parity_pin.py` (new)
**Approach.**
- Import BLUE's real `NDAlphaEngine` from `nautilus_dolphin.nautilus.esf_alpha_orchestrator`
(READ-only use; construct a minimal instance with default ENGINE_KWARGS-equivalent params).
- For a sampled grid of inputs (vel_div ∈ [-0.06, 0], boost ∈ [1,3], beta ∈ {0.2,0.8},
mc_scale ∈ {0.5,1.0}, ob (median_imbalance, agreement_pct) over a grid, dc_status ∈
{NONE,CONFIRM}, posture ∈ {APEX,STALKER}), drive BLUE's engine to compute each intermediate
(`_day_*` state → `_update_regime_size_mult`; the OB block; `_strength_cubic`) and assert
VIOLET's `VioletSizer.regime_size_mult / market_ob_mult / strength_cubic / dc_lev_mult`
return the EXACT same float (`==`, not approx — these are deterministic).
- If a BLUE method is not callable in isolation (needs engine state), set the minimal `_day_*`
attributes directly and call the method; document any state you had to set.
**Tests / pass criteria.**
- New test file: every parametrized case asserts exact equality VIOLET-replica == BLUE-method.
- `pytest -q prod/clean_arch/violet/test_violet_sizing_parity_pin.py` → all pass.
- The existing `@gate` composition test still passes (don't change sizing.py).
- DONE when: ≥ 200 grid points per formula, zero mismatches, no edits outside the new test file.
---
## TASK 2 — Multi-exchange OB provider seam (scaffold + interface, NOT wired live)
**Why.** VIOLET currently reads BLUE's extant Binance-reference OB via `HZOBProvider`. The spec
requires being able to run a SEPARATE OB stream later (BingX testnet / 3rd-party venues have
genuinely different order books). The `OBProvider` behind `OBFeatureEngine` is the swap seam.
This task defines + tests that seam as a new module, WITHOUT wiring it into the live path
(wiring is owned by the in-flight `live_blue_source` / `shadow_live_factors`).
**Affected files (NEW only):**
- `prod/clean_arch/violet/venue_ob_provider.py` (new)
- `prod/clean_arch/violet/test_violet_venue_ob_provider.py` (new)
- `prod/docs/VIOLET_SPEC__MULTI_EXCHANGE_OB_SEAM.md` (new short design note)
**Approach.**
- Read `nautilus_dolphin/nautilus_dolphin/nautilus/ob_provider.py` (the `OBProvider` ABC +
`OBSnapshot`) and `hz_ob_provider.py` (BLUE's reference impl) to learn the exact interface
(`get_snapshot`, `get_assets`, `get_all_timestamps`, `get_snapshot_count`, snapshot fields:
bid/ask notional+depth arrays of length 5, timestamp, asset).
- Define `VioletVenueOBProvider(OBProvider)` — a venue-agnostic provider that takes a normalized
tick source (NOT a live WS yet; accept an injected callable / in-memory buffer). Produce
`OBSnapshot`s with the SAME shape BLUE expects so it drops into `OBFeatureEngine` unchanged.
- Include a `MockTickVenueOBProvider` for tests (deterministic snapshots). Do NOT open any real
exchange connection. V-TYPES on the normalized tick (5-level arrays non-negative, finite).
- Design note documents: the seam, how a future BingX WS adapter plugs in, and that wiring into
`_source_ob_market` is deferred to the owner of `live_blue_source.py`.
**Tests / pass criteria.**
- `VioletVenueOBProvider` conforms to `OBProvider` (all abstract methods implemented; an
`OBFeatureEngine(provider)` can `step_live` + `get_market` over mock snapshots without error).
- Poison rejection: negative/NaN depths or wrong-length arrays are rejected at construction.
- `pytest -q prod/clean_arch/violet/test_violet_venue_ob_provider.py` → all pass.
- DONE when: the provider drives a real `OBFeatureEngine` to a finite `get_market` result in a
test, no live connections, no edits outside the 3 new files.
---
## TASK 3 — Base-fraction sizing study (analysis + report; no production code change)
**Why.** The base sizing fraction (0.20) is a champion constant; a study was specced but not
run (`prod/docs/VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md` — READ it first; it is the
authority for scope/method). This is a pure analysis task producing a report — zero behavior
change — so it is fully parallelizable.
**Affected files (NEW only):**
- `prod/VIOLET_dev/studies/base_fraction_study.py` (new analysis script)
- `prod/VIOLET_dev/reports/base_fraction_study_<UTC>.md` or `.json` (new report output)
**Approach.**
- Follow the existing study spec exactly. Use recorded data only (CH `dolphin_violet` /
`dolphin` read-only via `http://localhost:8123`, user `dolphin` / key `dolphin_ch_2026`).
Do NOT write to any production table. Do NOT change `sizing.py` / `decision_engine.py`.
- Compute the requested sensitivity (PnL / drawdown / capital-utilization vs base_fraction over
the spec's grid), honoring the leverage caps (base_max=8, abs_max=9) and the margin-study
findings (notional = 0.20 × conviction × capital today).
- Output a report with the recommended base_fraction + evidence; flag any caveats. Recommend,
do NOT apply.
**Tests / pass criteria.**
- The script runs end-to-end on the prod host and writes a report to `prod/VIOLET_dev/reports/`.
- A small `pytest` (or `--self-test` mode) validates the core computation on a synthetic fixture
(deterministic input → known output), so the math is checkable without live data.
- DONE when: report archived + self-test passes + no production table writes + no code-behavior
change.
---
## What is intentionally NOT in this spec (do not start these)
- **DARK soak start** — HELD for the operator's explicit word.
- **V4 live execution** — blocked on VST keys; operator-gated.
- **HZ Bridge refactor** (`TODO_HZBRIDGE`) — depends on `dolphinng5_predict/hzbridge` shipping
(owned by another agent); not self-contained yet.
- **Edits to `live_blue_source.py` / `shadow_live_factors.py` / `sizing.py` / `decision_engine.py`**
— in-flight; coordinate before touching.

View File

@@ -0,0 +1,175 @@
# VIOLET — partial spec for another agent, PASS 2 (OA TODO)
Date: 2026-06-16. Follows `VIOLET_PART_SPEC_OA_TODO.md` (PASS 1, reported done — pending
review). PASS 2 tasks **advance the main VIOLET global spec** (`VIOLET_DEV_SPEC_AND_PLAN.md`,
the V0→V6 ladder) toward the DARK soak gate and V4 execution, while staying self-contained.
---
## 0. HARD RULES (identical to PASS 1 — re-read; non-negotiable)
1. **Never edit shared files**: `prod/nautilus_event_trader.py`, `prod/clean_arch/dita_v2/**`,
`prod/clean_arch/dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`,
`prod/bingx/leverage.py`. READ only.
2. **VIOLET stays DARK** — no orders, no execution, no VST keys, no service start/stop, no HZ
restart, no PROGREEN.
3. **V-TYPES on all new code** (`StrictModel` / `Annotated[…Field]` / `@typed`); no arbitrary
caps, only faithful poison-guards.
4. **NEW-FILE-ONLY** — do NOT modify these in-flight files (owned by another agent right now):
`live_blue_source.py`, `shadow_live_factors.py`, `live_factor_source.py`, `live_factors.py`,
`decision_engine.py`, `sizing.py`, `shadow_journal.py`, `prod/clickhouse/violet/22_violet_decisions.sql`.
You may READ + IMPORT them. Wiring into them is deferred to their owner.
## 0a. COMMIT / BRANCH POLICY (CRITICAL — 3 shared-index collisions happened 2026-06-16)
The working tree + `.git/index` are SHARED across agents; `git commit` commits the whole index,
so a concurrent agent's staged files get swept into your commit. THEREFORE:
- **Strongly preferred:** your own worktree — `git worktree add ../vp-oa2 -b agent/oa-violet2`.
- **Otherwise:** never `git add -A`/`git add .`; commit with explicit pathspec —
`git commit -F msg.txt -- <your_new_file> <your_test>` — and verify
`git show --stat --format="" HEAD` lists ONLY your files.
- One commit per task, prefix `VIOLET OA:`, Co-Authored-By trailer. Before "done":
`git diff --name-only HEAD~1` ∌ any forbidden path.
- Tests on `/home/dolphin/siloqy_env/bin/python3 -m pytest -q`. CIFS-slow is normal. Use
`git grep`, not recursive `grep -r`/`find`.
## 0b. ClickHouse access (read-only)
`http://localhost:8123`, user `dolphin` / key `dolphin_ch_2026`. VIOLET data in db
`dolphin_violet`; BLUE data in db `dolphin`. **READ ONLY** — never write/alter production tables.
---
## TASK 4 — Live BLUE↔VIOLET decision parity report (advances V3d → DARK-soak gate)
**Spec link.** The ladder's V3d step requires proving the shadow DecisionEngine reproduces
BLUE's decisions. A SHADOW soak already journaled VIOLET decisions to
`dolphin_violet.violet_decisions`. This task builds the report that compares them to BLUE's
recorded decisions — the quantitative gate that makes a longer soak meaningful.
**Affected files (NEW only):**
- `prod/clean_arch/violet/parity_report.py` (new)
- `prod/clean_arch/violet/test_violet_parity_report.py` (new)
- output → `prod/VIOLET_dev/reports/violet_parity_<UTC>.{json,md}` (generated)
**Approach.**
- Read VIOLET decisions from `dolphin_violet.violet_decisions` (asset, side, scan_number,
conviction_leverage, notional_fraction, target_exposure, vel_div, ts) and BLUE's recorded
decisions from `dolphin.trade_events` / `dolphin.v7_decision_events` (inspect their schemas
first via `DESCRIBE`). Align by nearest scan/timestamp + asset.
- Compute: pick-match rate (same asset/side chosen), sizing deltas (VIOLET vs BLUE
conviction_leverage / notional — distribution: mean/median/p95 abs err), and a
divergence-reason breakdown (no-pick, different-asset, sizing-gap > threshold).
- Pure analysis: NO writes to any production table; emit a JSON + a human-readable MD report.
- V-TYPES the report rows; reject malformed/non-finite rows to a counter (never crash).
**Tests / pass criteria.**
- A `--self-test` / unit mode runs the comparison on a SYNTHETIC fixture (hand-built VIOLET +
BLUE rows with known overlaps) and asserts the computed pick-match / sizing-delta numbers
exactly. No live CH needed for the unit test.
- On the prod host, the script runs end-to-end against live CH and archives a report.
- DONE when: self-test passes, a real report is archived, zero production writes, no edits
outside the 2 new files.
---
## TASK 5 — Full-DecisionEngine reactor latency gate (advances V0/V3 — budget proof)
**Spec link.** V0 proved the reactor clock meets the latency budget (reaction p99 < 10ms,
jitter p99 < 25ms) with a trivial handler. V3 put the real decision brain online. This gate
proves the budget STILL holds when the reactor drives the FULL
`VioletDecisionEngine.decide(factors=SizingFactors(...))` path (the heavy 5-factor sizing),
not a stub a prerequisite for trusting the sub-second cadence claim.
**Affected files (NEW only):**
- `prod/clean_arch/violet/test_violet_v3_decision_latency_gate.py` (new, `@pytest.mark.gate`)
- output `prod/VIOLET_dev/reports/violet_v3_decision_latency_<UTC>.json` (generated)
**Approach.**
- Reuse V0's `PlaneClock` / `DeadlineScheduler` (`clock.py`) and the storm pattern from
`test_violet_v0_latency_gate.py` / `test_violet_v2_exec_gate.py` (READ them for the harness).
- Build a warmed `VioletDecisionEngine` (feed enough synthetic scans to pass the IRP lookback),
then in the storm loop call `decide(now_ns=…, scan_number=…, capital=…, vel_div=…,
factors=SizingFactors(boost=…, beta=…, mc_scale=…, esof_score=…, ob_*=…, dc_status=…,
posture=…))` each tick. Measure reaction + jitter percentiles.
- Do NOT modify `decision_engine.py`; import + drive it.
**Tests / pass criteria.**
- Gate asserts reaction p99 < 10ms and jitter p99 < 25ms (same budget as V0) over 200 cycles
with the FULL factor path; archive the percentile report.
- DONE when: the gate passes on the prod host, report archived, no edits outside the new file.
(If it FAILS, do not loosen the budget report the regression with numbers; that is a real
finding.)
---
## TASK 6 — L3 tradeability projector, standalone (advances toward V4 execution)
**Spec link.** The 3-layer doctrine: L1 pure alpha (done), L2 parity harness (done), **L3
tradeability** impose convictionexchange-leverage + maker policy on the L1 output. The
wrapper exists (`exchange_leverage.py`, bit-identity-gated). This task builds the standalone
projector that turns a `ShadowDecision` into a tradeable projection, WITHOUT wiring it into the
journal or any venue (DARK; wiring is the owner's job later).
**Affected files (NEW only):**
- `prod/clean_arch/violet/tradeability.py` (new)
- `prod/clean_arch/violet/test_violet_tradeability.py` (new)
**Approach.**
- Read `exchange_leverage.py` (`VioletExchangeLeverage`, `ExchangeLeverageDecision`) and
`decision_engine.py` (`ShadowDecision`) and the margin-study facts (notional = base_fraction ×
conviction × capital; exchange leverage = `map_internal_conviction_to_exchange_leverage`,
max 3× cubic).
- Define `TradeabilityProjection(StrictModel)` with: internal_conviction, exchange_leverage
(int, ge=1), target_notional, est_margin (= notional / exchange_leverage), maker_policy hint
(string, e.g. "maker_both" read existing ExecutionRouter conventions, do NOT import PINK
exec). Define `project_tradeability(decision: ShadowDecision, *, capital: float) ->
TradeabilityProjection` (`@typed`). NO orders, NO venue calls pure projection.
- This is L1-output L3 projection only; it must not change L1 sizing.
**Tests / pass criteria.**
- Exchange leverage equals `exchange_leverage.py`'s mapping for the decision's
conviction_leverage (bit-identical reuse that wrapper, don't reimplement).
- est_margin = target_notional / exchange_leverage; poison guards reject non-finite/negative.
- Hypothesis property test over conviction [0.5, 9]: projection finite, exchange_leverage
[1,3], margin notional.
- `pytest -q prod/clean_arch/violet/test_violet_tradeability.py` passes; no edits outside the 2
new files.
---
## TASK 7 — Deterministic replay + golden gate (advances V3d determinism criterion)
**Spec link.** V3d also requires DETERMINISM: same recorded scans same decisions twice. This
hardens soak confidence and catches accidental nondeterminism (dict ordering, RNG, time
leakage) in the decision path.
**Affected files (NEW only):**
- `prod/clean_arch/violet/test_violet_replay_determinism_gate.py` (new)
- `prod/clean_arch/violet/fixtures/replay_scans.json` (new, small synthetic scan sequence)
**Approach.**
- Build a fixed synthetic scan sequence (≥ 60 scans, a few assets incl. one stablecoin to prove
the exclusion gate, varied vel_div crossing the entry threshold). Store as the fixture.
- Run it through TWO fresh `VioletDecisionEngine` instances (with identical fixed
`SizingFactors`) and assert the emitted `ShadowDecision` sequences are EQUAL field-by-field.
- Add a golden assertion: the decision sequence matches a checked-in expected summary (asset,
side, conviction_leverage, scan_number per fired decision) regenerate-on-purpose only.
- Read `decision_engine.py`; do not modify it.
**Tests / pass criteria.**
- Two runs over the fixture produce identical `ShadowDecision` lists (`==`).
- The golden summary matches; stablecoin assets never selected.
- `pytest -q prod/clean_arch/violet/test_violet_replay_determinism_gate.py` passes; no edits
outside the 2 new files.
---
## Sequencing note
Tasks 47 are independent and parallelizable. Recommended priority for advancing the spec:
**Task 5 (latency budget) → Task 7 (determinism) → Task 4 (parity report) → Task 6 (tradeability)**
the first two are gates that must hold before a longer DARK soak; Task 4 makes the soak
measurable; Task 6 preps V4. None require the operator's soak/keys greenlight.
## Still NOT in scope
DARK soak START (operator-held); V4 live execution (keys-blocked); HZ-bridge refactor (depends
on `dolphinng5_predict/hzbridge` shipping); any edit to the in-flight files in §0 rule 4.

View File

@@ -0,0 +1,212 @@
# VIOLET — partial spec for another agent, PASS 3 (OA TODO)
Date: 2026-06-17. Follows PASS 1 + PASS 2 (both reported done — review pending, see
`VIOLET_TODO_CRITICAL.md`). PASS 3 is drawn STRICTLY from the existing VIOLET dev plan
(`VIOLET_DEV_SPEC_AND_PLAN.md`, the V0→V6 ladder + its named deferred items) — nothing invented.
Every task is an INDEPENDENT UNIT: it can be built, tested and function on its own, and it
composes with the others ONLY through the explicit shared interfaces in §I.
---
## 0. HARD RULES (identical to PASS 1/2 — re-read those; summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
- **VIOLET stays DARK** — no orders, no execution, no VST keys, no service start/stop, no HZ
restart, no PROGREEN.
- **V-TYPES on all new code** (`StrictModel` / `Annotated[…Field]` / `@typed`); faithful
poison-guards only.
- **NEW-FILE-ONLY**; do NOT modify in-flight files: `live_blue_source.py`,
`shadow_live_factors.py`, `live_factor_source.py`, `live_factors.py`, `decision_engine.py`,
`sizing.py`, `shadow_journal.py`, `cadence.py`, `clock.py`, `divergence.py`,
`22_violet_decisions.sql`. READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — take this seriously)
Own `git worktree` (`git worktree add ../vp-oa3 -b agent/oa-violet3`) STRONGLY preferred. Else
never `git add -A`; commit with explicit pathspec `git commit -F msg -- <files>` and verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit per task, prefix
`VIOLET OA:`, Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. Use
`git grep` (recursive grep/find time out on CIFS).
---
## I. SHARED INTERFACES (agreed contracts — define these EXACTLY; the tasks depend on them)
These are the only coupling points between PASS-3 units. Put the type definitions in ONE new
module `prod/clean_arch/violet/contracts_v3.py` (Task 8 creates it; later tasks import it).
All are `StrictModel` / `Annotated` V-TYPES.
1. **`VenueTick`** — one normalized venue quote (exchange-agnostic):
`asset: Symbol`, `bid: Px`, `ask: Px`, `mark: Px`, `last: Px`, `mono_ns: MonoNs`,
`venue: str` (e.g. "BINGX","BINANCE"). All prices > 0 finite. This is the SAME normalized
tick named in the plan's "Venue price feed port (CRITICAL gap)".
2. **`OpenPositionView`** — the minimal open-position state an exit/guard needs:
`asset: Symbol`, `side: str` ("SHORT"/"LONG"), `entry_price: Px`, `qty: Qty`,
`entry_ts_ns: MonoNs`, `bars_held: BarsHeld`, `tp_threshold: float` (fixed TP pct),
`sl_threshold: float` (stop pct), `leverage: float`.
3. **`ExitDecision`** — `action: str` ("HOLD"/"EXIT"), `reason: str`
("FIXED_TP"/"CATASTROPHIC_SL"/"ADVSL"/"NONE"/…), `price: Px`, `priority: int`
(CATASTROPHIC/ADVSL=0 > FIXED_TP=1 > DISCRETIONARY=2 — the plan's mandated order).
If a later task needs a field not here, ADD it to `contracts_v3.py` (and note it), never fork a
parallel type.
---
## TASK 8 — VenuePriceFeedPort + BingX adapter (DARK, data-only) [plan: "Venue price feed port (CRITICAL gap)"]
**Why.** The plan flags this as CRITICAL: OBF is Binance-wired (alpha side), but exits fill on
the EXECUTION venue, and FET showed scan-vs-BingX divergence (0.2176 vs 0.1878 = 15%). VIOLET
needs an exchange-agnostic price port. This unit delivers the port interface + a BingX adapter,
data-only (no orders).
**Affected files (NEW only):** `prod/clean_arch/violet/contracts_v3.py` (the §I types),
`prod/clean_arch/violet/venue_price_feed.py`, `prod/clean_arch/violet/test_violet_venue_price_feed.py`.
**Interface/approach.** Define `VenuePriceFeedPort` (ABC): `latest(asset) -> Optional[VenueTick]`,
`subscribe(assets)`, `close()`. Implement `BingxSwapPriceFeed(VenuePriceFeedPort)` consuming the
BingX swap WS bookTicker/markPrice (READ the existing BingX WS conventions in the repo; data-only,
no auth needed for public streams — confirm). Implement `MockPriceFeed` (deterministic, for
tests + for the other tasks). All ticks validated into `VenueTick` at ingress (poison-reject
non-finite/≤0).
**Tests / pass criteria.** Mock feed round-trips ticks as `VenueTick`; poison ticks rejected;
the BingX adapter parses a recorded/sample bookTicker frame into a correct `VenueTick` (use a
captured frame fixture, NOT a live connection in the unit test). DONE when: `VenueTick` contract
finalized, mock + adapter parse-tested, no live WS in unit tests, no edits outside the 3 files.
---
## TASK 9 — MechanicalExitGuard (armed TP/SL) [plan: LINK TP-miss structural fix]
**Why.** The plan's structural fix for the LINKUSDT $1,248 TP-miss: TP/SL are MECHANICAL exits
owned by an exit guard with armed price thresholds set at entry; policy layers (V7/MARAS) can
only contribute DISCRETIONARY exits and can NEVER mask mechanical ones. Total priority:
CATASTROPHIC/ADVSL > fixed TP > discretionary.
**Affected files (NEW only):** `prod/clean_arch/violet/mechanical_exit_guard.py`,
`prod/clean_arch/violet/test_violet_mechanical_exit_guard.py`. (imports `contracts_v3`.)
**Interface/approach.** `MechanicalExitGuard.evaluate(pos: OpenPositionView, tick: VenueTick) ->
ExitDecision`. Pure function of position + current venue price: compute pnl_pct from
entry/side/price; if it reaches the armed `tp_threshold``ExitDecision(EXIT, FIXED_TP,
priority=1)`; if it breaches `sl_threshold``ExitDecision(EXIT, CATASTROPHIC_SL, priority=0)`;
else HOLD. Prices off the VENUE tick (not scan). No discretionary logic here — this layer is the
mechanical floor only.
**Tests / pass criteria.** SHORT + LONG: TP fires exactly at threshold, SL at stop, neither
fires inside the band; priority ordering correct (SL outranks TP if both somehow true). Hypothesis
property: output always finite, priority ∈ {0,1,2}, EXIT only when threshold crossed. DONE when:
deterministic threshold tests + property test pass, no edits outside the 2 files.
---
## TASK 10 — Sub-second catastrophic-SL / ADVSL floor guard [plan: versioned SAFETY DEVIATION]
**Why.** The plan's one sanctioned sub-second behaviour: a catastrophic-SL/ADVSL floor evaluated
at the fastest cadence against a fast price source, to fix the scan-dark unmanaged-position hazard
(XLM/FET class). "Evaluate at fastest cadence (shadow-log would-be actions), actuate at Q."
**Affected files (NEW only):** `prod/clean_arch/violet/sl_floor_guard.py`,
`prod/clean_arch/violet/test_violet_sl_floor_guard.py`. (imports `contracts_v3`, composes Task 9.)
**Interface/approach.** `SLFloorGuard(deadline_ns)` with `on_tick(pos, tick) -> Optional[ExitDecision]`
that EVALUATES every tick (sub-second) and SHADOW-LOGS would-be SL exits, but only RETURNS an
actuation when the configured fast-SL condition holds (catastrophic threshold worse than the
mechanical stop, OR ADVSL trailing breach). Track evaluate-count vs actuate-count (shadow delta).
DARK: returns the decision; never sends orders.
**Tests / pass criteria.** A tick stream that dips intra-scan triggers the fast SL evaluation and
records the would-be exit; the actuate gate fires only on the catastrophic/ADVSL condition;
evaluate-count > actuate-count proven. DONE when tests pass, no edits outside the 2 files.
---
## TASK 11 — Event-sourced restore from trade_reconstruction [plan: VIOLET restore commitment, Option C]
**Why.** The plan commits VIOLET to NEVER restore from position_state snapshots; instead replay
the chain-tokened `trade_reconstruction` OPEN/PARTIAL_EXIT/CLOSE journal. Open = ROOT OPEN with no
terminal CLOSE; size = entry Σ legs; holding from ABSOLUTE entry_ts (kills the bars_held≈0 /
MAX_HOLD-reset class by construction); chain token verified.
**Affected files (NEW only):** `prod/clean_arch/violet/event_restore.py`,
`prod/clean_arch/violet/test_violet_event_restore.py`. (imports `contracts_v3` → emits
`OpenPositionView`s.)
**Interface/approach.** `restore_open_positions(journal_rows: list[dict]) -> list[OpenPositionView]`
— pure function over journal rows (asset, chain_root_trade_id, chain_token, leg type, qty,
realized legs, entry_ts, entry_price). Reconstruct each chain; an open position is a ROOT OPEN
with no matching terminal CLOSE; qty = entry Σ partial-exit legs; bars_held derived from absolute
entry_ts vs now (NEVER from a stored counter). Reject chain-token mismatches to a quarantine list
(returned alongside), never crash.
**Tests / pass criteria.** Fixtures: clean open, fully-closed (→ not restored), partial-exit chain
(qty correct), chain-token mismatch (→ quarantined), dead-session entry_ts (bars_held computed
from ts, never negative). Hypothesis: qty ≥ 0, bars_held ≥ 0 always. DONE when tests pass, no
edits outside the 2 files.
---
## TASK 12 — Venue lead/lag slippage metric [plan: V1 metrics — venue lead/lag]
**Why.** The plan's continuous signed SHORT-entry slippage metric: venue fill vs Binance mid at
decision time — answers "is BingX discounting our signal". Rerunnable analysis.
**Affected files (NEW only):** `prod/clean_arch/violet/slippage_metric.py`,
`prod/clean_arch/violet/test_violet_slippage_metric.py`, report →
`prod/VIOLET_dev/reports/violet_slippage_<UTC>.json`.
**Interface/approach.** `signed_entry_slippage(decision_mid: float, venue_fill: float, side: str)
-> float` (signed bps; positive = adverse) + an aggregator over recorded rows producing a
distribution (mean/median/p95). Read-only over recorded data (CH `dolphin`/`dolphin_violet`, no
writes). Pure functions; V-TYPES the rows.
**Tests / pass criteria.** Sign convention exact for SHORT and LONG on hand-built cases; aggregator
stats correct on a synthetic fixture (`--self-test`). DONE when self-test passes + a report can be
produced on the prod host, no production writes, no edits outside the files.
---
## TASK 13 — Cadence per-action Q schedule + evaluate/actuate telemetry [plan: cadence quantizer]
**Why.** The plan's cadence quantizer: each action (SL/TP/ENTRY/OBF/ExoF) gets its own
quantization Q; evaluate at fastest cadence (shadow evidence), actuate at Q; step Q down later.
`cadence.py` (the `CadenceControlPlane`) exists; this unit adds a per-action Q-SCHEDULE LOADER +
a shadow-delta TELEMETRY recorder WITHOUT modifying `cadence.py`.
**Affected files (NEW only):** `prod/clean_arch/violet/cadence_schedule.py`,
`prod/clean_arch/violet/test_violet_cadence_schedule.py`.
**Interface/approach.** Read `cadence.py` for `Action` + `CadenceControlPlane` API. Provide
`load_q_schedule(mapping) -> dict[Action, int_ns]` (validated; the plan's initial table: SL tight,
TP=scan, ENTRY=scan, OBF~1s) and `CadenceTelemetry` that records per-action evaluate-count vs
actuate-count and emits a shadow-delta summary. Compose with `CadenceControlPlane` by wrapping its
`due()` calls (do not edit it).
**Tests / pass criteria.** Schedule loader validates/rejects bad Q values; telemetry counts
evaluate > actuate under a synthetic action stream; per-action Q honored. DONE when tests pass, no
edits outside the 2 files.
---
## Composition map (how the units fit — the "well-known interfaces")
```
Task 8 contracts_v3 (VenueTick, OpenPositionView, ExitDecision) ← the shared vocabulary
├── Task 8 venue_price_feed → VenueTick stream
├── Task 9 mechanical_exit_guard(pos, tick) → ExitDecision (consumes VenueTick)
├── Task 10 sl_floor_guard(pos, tick) → ExitDecision (composes Task 9)
├── Task 11 event_restore(journal) → [OpenPositionView] (feeds 9/10)
├── Task 12 slippage_metric(mid, fill, side)→ bps (uses VenueTick mids)
└── Task 13 cadence_schedule/telemetry → per-action Q + deltas
```
Integration (wiring these into the live reactor + decision/exit path) is the OWNER's job later,
NOT part of these units. Each ships standalone + tested.
## Recommended order
**8 (contracts + feed) → 9 (mechanical exit) → 11 (restore) → 10 (SL floor) → 13 (cadence) → 12
(slippage)**. Task 8 first because `contracts_v3.py` is the shared vocabulary everything imports.
## Still NOT in scope (operator/owner only)
DARK soak start; V4 live execution / BingX ExecutionClient; HZ-bridge refactor; the V3.4c parity
root-cause (CRITICAL #1 — Claude's review job); any edit to the in-flight files in §0.

View File

@@ -0,0 +1,204 @@
# VIOLET — partial spec for another agent, PASS 4 (OA TODO)
Date: 2026-06-17. Continues PASS 3 (same V0→V6 plan, same independent-unit discipline). PASS 3
delivered the venue-feed / mechanical-exit / restore / slippage / cadence layer; PASS 4 adds the
ENTRY GATE, the EXIT KERNELS (V7 + time-based), the ACCOUNTING ledger, the EXECUTION-INTENT
boundary, and the ALPHA-SIDE data feed. Drawn strictly from `VIOLET_DEV_SPEC_AND_PLAN.md`
nothing invented. Each task is an independent, separately-testable unit; the ONLY coupling is the
shared `contracts_v3.py` vocabulary (extended in §I).
---
## 0. HARD RULES (identical to PASS 1/2/3 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
- **VIOLET DARK** — no orders, no execution, no keys, no service/HZ control, no PROGREEN.
- **V-TYPES on all new code**; faithful poison-guards only.
- **NEW-FILE-ONLY**; do NOT modify in-flight files (`live_blue_source.py`,
`shadow_live_factors.py`, `live_factor_source.py`, `live_factors.py`, `decision_engine.py`,
`sizing.py`, `shadow_journal.py`, `cadence.py`, `clock.py`, `divergence.py`,
`alpha_wrappers.py`, `exchange_leverage.py`, `22_violet_decisions.sql`). READ + IMPORT them.
- **VIBRISS stays DARK** — do NOT build adaptive-TP / VIBRISS-governable mechanisms here.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa4 -b agent/oa-violet4`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3 -m pytest -q`. `git grep`
only (recursive grep/find time out on CIFS).
## 0b. ClickHouse (read-only): `http://localhost:8123`, user `dolphin`/key `dolphin_ch_2026`;
VIOLET db `dolphin_violet`, BLUE db `dolphin`. NEVER write/alter production tables.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py` from PASS 3; never fork a parallel type)
Reuse PASS-3 `VenueTick`, `OpenPositionView`, `ExitDecision`. ADD:
1. **`ExecIntent`** (StrictModel) — a DARK would-be order: `asset: Symbol`, `side: str`,
`qty: Qty`, `exchange_leverage: Annotated[int, Field(ge=1)]`, `maker_policy: str`,
`target_notional: float`, `ts_ns: MonoNs`, `reason: str` ("ENTRY"/"EXIT"). NEVER sent.
2. **`CapitalState`** (StrictModel) — `capital: float (ge=0, finite)`, `anchor: float`,
`delta_sum: float`, `event_seq: Seq`, `pnl_source: str`, `capital_source: str`. Capital is
ALWAYS anchor + Σ deltas (never a last-value snapshot — the zombie-trade lesson).
3. **`VolGateResult`** (StrictModel) — `vol_ok: bool`, `metric: float`, `threshold: float`.
If a task needs a field not listed, ADD it to `contracts_v3.py` and note it.
---
## TASK 14 — Volatility regime gate [plan: entry precondition `vol_ok` / vol_p60_threshold]
**Why.** BLUE gates entries on a volatility regime check (`vol_ok`, `vol_p60_threshold`);
`VioletDecisionEngine.decide(... vol_ok=...)` already takes it as a param but nothing computes it
faithfully. This unit computes `vol_ok` the way BLUE does so the entry gate is real, not assumed.
**Affected files (NEW only):** `prod/clean_arch/violet/vol_gate.py`,
`prod/clean_arch/violet/test_violet_vol_gate.py`. (imports `contracts_v3`.)
**Interface/approach.** First READ how BLUE computes the vol gate (search the orchestrator / the
engine for `vol_regime_ok` / `vol_p60` / the percentile threshold; transcribe the EXACT formula).
Define `compute_vol_gate(price_histories: dict[str, list[float]], *, threshold: float) ->
VolGateResult` (`@typed`) reproducing BLUE's metric + comparison. Pure function.
**Tests / pass criteria.** Parity: hand-built histories where BLUE's formula gives a known
vol_ok; assert match. Boundary: at/just-below/just-above threshold. Hypothesis: result finite,
vol_ok bool. DONE when the formula is transcribed from live BLUE (cite file:line in a docstring),
tests pass, no edits outside the 2 files.
---
## TASK 15 — AlphaExitEngineV7 wrapper [plan: V3a deferred — "run exit-v7 as-is first, refactor later"]
**Why.** The plan wraps BLUE's live multi-leg exit engine `AlphaExitEngineV7` behind a V-TYPES
boundary and runs it AS-IS. Its outputs are DISCRETIONARY exits (priority 2) that, per the exit
doctrine, can NEVER override the mechanical TP/SL (PASS-3 Task 9).
**Affected files (NEW only):** `prod/clean_arch/violet/exit_v7_wrapper.py`,
`prod/clean_arch/violet/test_violet_exit_v7_wrapper.py`. (imports `contracts_v3`.)
**Interface/approach.** WRAP, don't reimplement: import
`nautilus_dolphin.nautilus.alpha_exit_v7_engine.AlphaExitEngineV7`; build its context
(`make_context`) and call `evaluate` over an `OpenPositionView` + market state; map its result to
an `ExitDecision(reason=…, priority=2)`. Refined domain types in/out. The wrapper must NOT change
V7's logic. Document any state V7 needs.
**Tests / pass criteria.** For crafted inputs, the wrapper's `ExitDecision` matches what
`AlphaExitEngineV7.evaluate` returns (same exit/hold + reason), priority always 2. DONE when wrap
parity tests pass, no edits outside the 2 files.
---
## TASK 16 — Capital / economics provenance ledger [plan: V1 inheritance — capital = anchor + Σ deltas]
**Why.** Binding VIOLET doctrine (from the zombie-trade incident + malformed-open audit): capital
is ALWAYS anchor + Σ signed per-trade deltas, NEVER a last-value snapshot; every row carries
`pnl_source`/`capital_source` provenance + `event_seq`; exactly-one-row-per-event. Per-trade sizer
feedback uses trade-realized PnL, never capital deltas (shared-account foreign-fill immunity).
**Affected files (NEW only):** `prod/clean_arch/violet/economics_ledger.py`,
`prod/clean_arch/violet/test_violet_economics_ledger.py`. (emits `CapitalState`.)
**Interface/approach.** `EconomicsLedger(anchor: float)` with `apply(delta: float, *, event_seq:
int, pnl_source: str) -> CapitalState` (monotonic event_seq; rejects out-of-order / duplicate
seq; capital = anchor + Σ deltas). `capital()` returns the derived value, never a stored snapshot.
Reject NaN/inf deltas. No CH writes (pure in-memory ledger + a `to_row()` for a future sink).
**Tests / pass criteria.** Σ-delta correctness over a sequence; duplicate/out-of-order event_seq
rejected; capital never goes negative below a floor guard; Hypothesis: capital == anchor + sum of
accepted deltas, always. DONE when tests pass, no edits outside the 2 files.
---
## TASK 17 — Execution-intent emitter (DARK) [plan: V4 prep — feeds the ExecDeadlineDriver]
**Why.** V2 built the `ExecDeadlineDriver` + synthetic intents; V4 will turn decisions into real
orders. This unit is the DARK boundary that converts a decision into an `ExecIntent` (a would-be
order) — shadow-logged, NEVER sent — so V4 only has to flip the sink from log to venue.
**Affected files (NEW only):** `prod/clean_arch/violet/exec_intent.py`,
`prod/clean_arch/violet/test_violet_exec_intent.py`. (imports `contracts_v3`; composes PASS-2
`tradeability.py` if present, else recompute via `exchange_leverage.py`.)
**Interface/approach.** `to_exec_intent(decision: ShadowDecision, *, capital: float, maker_policy:
str = "maker_both") -> ExecIntent` — derive qty from notional/price, exchange_leverage from the L3
mapping, attach maker policy. Pure projection; an `IntentSink` protocol with a default
`LoggingIntentSink` (shadow-logs, never sends). Hard guard: this module imports NOTHING that can
place an order; assert no venue/order symbol is importable here (a test enforces it).
**Tests / pass criteria.** Intent fields correct vs the decision + L3 mapping; `LoggingIntentSink`
records, never sends; a test asserts the module has no order-placing dependency. DONE when tests
pass, no edits outside the 2 files.
---
## TASK 18 — Alpha-side reactor data-feed adapter [plan: V2 NT Binance DATA client spike → feed]
**Why.** The V2 spike GO-qualified the Nautilus Binance DATA client as VIOLET's alpha-side feed
(separate feed process recommended; public-data only; dummy keys for the factory). This unit
delivers that feed behind a clean port so the reactor consumes normalized bars without coupling to
NT internals.
**Affected files (NEW only):** `prod/clean_arch/violet/alpha_data_feed.py`,
`prod/clean_arch/violet/test_violet_alpha_data_feed.py`.
**Interface/approach.** Define `AlphaDataFeedPort` (ABC): `start()`, `latest_bar(asset)`,
`stop()`, yielding a normalized bar type (`AlphaBar` — add to `contracts_v3`: asset, open/high/low/
close > 0, volume ≥ 0, ts_ns, source). Implement `NautilusBinanceDataFeed` (READ the V2 spike
notes / `dolphin_actor*.py` / `live_price_feed.py` for the NT data-client wiring; public-data,
dummy keys, NO execution). Implement `MockDataFeed` (deterministic). NO orders, data-only.
**Tests / pass criteria.** Mock feed yields valid `AlphaBar`s; poison bars rejected; the NT adapter
parses a recorded bar message into a correct `AlphaBar` (captured-frame fixture, no live
connection in the unit test). DONE when contract finalized + parse-tested, no live feed in unit
tests, no edits outside the 2 files.
---
## TASK 19 — MAX_HOLD + MEAN_REVERSION scan-driven exit timers [plan: scan-driven exits, V6 bible]
**Why.** The plan keeps MAX_HOLD and MEAN_REVERSION as SCAN-driven exits (champion params are
5s-bar-denominated; re-timing requires VBT re-cert). They are mechanical/time-based exits that
compose with the exit framework (PASS-3 Task 9) at the right priority.
**Affected files (NEW only):** `prod/clean_arch/violet/time_exits.py`,
`prod/clean_arch/violet/test_violet_time_exits.py`. (imports `contracts_v3`.)
**Interface/approach.** READ BLUE's exit manager for the EXACT MAX_HOLD (`max_hold_bars`) and
mean-reversion rules (`AlphaExitManager` / the orchestrator). Define
`evaluate_time_exits(pos: OpenPositionView, *, bars_held: int, vel_div: float) -> ExitDecision`
`bars_held >= max_hold_bars``ExitDecision(EXIT, "MAX_HOLD")`; mean-reversion condition →
`ExitDecision(EXIT, "MEAN_REVERSION")`; else HOLD. Scan-cadence (no sub-second). Priority below
mechanical SL/TP, above pure discretionary (document the chosen priority vs Task 9/15).
**Tests / pass criteria.** MAX_HOLD fires exactly at the bar threshold; mean-reversion fires on the
BLUE condition (transcribed, cited); neither fires early. DONE when tests pass with the rule cited
from live BLUE, no edits outside the 2 files.
---
## Composition map
```
contracts_v3 (VenueTick, OpenPositionView, ExitDecision, ExecIntent, CapitalState, VolGateResult, AlphaBar)
├── 14 vol_gate(histories) → VolGateResult (entry gate; feeds decide vol_ok)
├── 18 alpha_data_feed → AlphaBar stream (alpha-side reactor feed)
├── 15 exit_v7_wrapper(pos, state) → ExitDecision(pri 2) ┐
├── 19 time_exits(pos, bars, veldiv) → ExitDecision ├─ exit stack, ranked by priority,
│ (PASS-3 9 mechanical = pri 0/1; 10 SL floor = pri 0) ┘ mechanical always wins
├── 16 economics_ledger(anchor,Δ…) → CapitalState (accounting; never last-value)
└── 17 exec_intent(decision,capital) → ExecIntent (DARK) (V4 boundary; log-only sink)
```
Integration (ranking the exit stack, wiring the feed into the reactor, the ledger into the
journal, the intent sink into the driver) is the OWNER's job later — NOT part of these units.
## Recommended order
**14 (vol gate) → 16 (ledger) → 15 (V7 wrap) → 19 (time exits) → 17 (exec intent) → 18 (data
feed)**. 14/16 are small + foundational; 18 is the heaviest (NT adapter).
## Pass/sprint nomenclature (re-asked) — see `VIOLET_TODO_CRITICAL.md` §3
"pass" = a sub-sprint work-package (a batch of self-contained tasks for one agent); a V-stage
(V0…V6) = the project's "Sprint N" / epic. Renaming passes to "sprints" over-claims scope. The
later REVIEW + INTEGRATE + E2E of all passes is Claude's queued work.
## Still NOT in scope (operator/owner only)
DARK soak start; V4 live order placement / BingX ExecutionClient; HZ-bridge refactor; VIBRISS /
adaptive-TP; the V3.4c parity root-cause (CRITICAL #1); any edit to the in-flight files in §0.

View File

@@ -0,0 +1,192 @@
# VIOLET — partial spec for another agent, PASS 5 (OA TODO): MOCK-BINGX EXECUTION STACK
Date: 2026-06-17. Continues PASS 14 (same V0→V6 plan, same independent-unit discipline). PASS 5
builds the **mock-BingX execution stack** — the DARK path that lets the ENTIRE order lifecycle
(submit → ack → fill/partial/reject → reconcile → position/PnL) be built and bit-tested with
**zero keys, zero risk**, so that V4-live later becomes "swap the mock adapter for the real BingX
client." This is the largest remaining buildable-while-DARK chunk of the spec.
**Read first:** the V2 execution work already shipped — `prod/clean_arch/violet/scripted_venue.py`,
`exec_driver.py`, `exec_harness.py`, `observe_guard.py`, `synthetic_intents.py`,
`exchange_leverage.py`, and PINK's `prod/clean_arch/runtime/pink_direct.py` +
`prod/clean_arch/exec/` ExecutionRouter (READ for maker/taker policy conventions; do NOT edit).
PASS 5 extends, does not replace, these.
---
## ⚠️ ⚠️ READ THIS FIRST — EXCHANGE "QUIRKS" ARE EXPLICITLY OUT OF SCOPE FOR PASS 5 ⚠️ ⚠️
The mock built here implements BingX's **NORMATIVE / documented** semantics only (clean order
lifecycle, fills, leverage, reduce-only, maker/taker, partials). BingX has a substantial set of
**undocumented / edge "quirks"** that broke PINK in production. They are **NOT to be implemented
or worked around during PASS 5.** Two hard requirements instead:
1. **The mock MUST be structured with explicit INJECTION SEAMS** so each quirk can later be turned
on (a `QuirkProfile` config + hook points) to test the reconcile/filter logic against it —
WITHOUT rewriting the mock.
2. **A real-key smoke at the venue boundary remains MANDATORY before V4-live** — a mock can only
validate logic, never undocumented venue reality.
### Known BingX quirks to ACCOMMODATE-LATER (do NOT implement now; just leave seams + a registry):
Source: memory `project_pink_orphan_fixes`, `ditav2_kernel_audit_20260611`, `incident_*`.
1. **Zero-wallet-balance WS frames** — BingX transiently pushes account/position frames with
`walletBalance=0`; treating them as truth zeroes capital. (Reconcile must ignore zero-wb / use
reseed-on-update.) Seam: mock can emit a zero-wb frame on demand.
2. **Ownership / foreign-fill collision** — fills/orders for OTHER subaccounts appear on a shared
user-data stream (PRODGREEN↔PINK shared-account hazard); must be filtered by ownership. Seam:
mock can emit a fill tagged with a foreign `ownership_id`.
3. **Bound-price poison** — orders carry a "bound" price; using it for PnL instead of the actual
fill price poisons accounting. Seam: mock fill carries both bound and fill price; PnL must use
fill price (Task 23).
4. **×leverage notional** — BingX reports notional/qty with leverage applied; double-applying
leverage corrupts size. Seam: documented in the Fill→position reducer (Task 23).
5. **Settle / funding desync** — settlement & funding events arrive desynced from fills; capital
deltas must be `event_seq`-ordered (ties to PASS-4 economics ledger). Seam: mock can interleave
a settle event out of fill order.
6. **Reduce-only edge** — a reduce-only order that would INCREASE the position is rejected by
BingX. Implement this ONE as normative (it is documented), but note the partial/zero-qty edge.
7. **setLeverage race**`setLeverage` is a separate call with its own ack; a race vs the order
can apply the wrong leverage. Seam: mock `set_leverage` is a distinct, separately-ackable op.
8. **Dead `.pro` TLS WS backup** — the `.pro` WS endpoint TLS can die silently (connection-layer).
Out of mock scope (connection layer); note for the real-client adapter.
A new file `prod/clean_arch/violet/exec/bingx_quirks.py` holds ONLY a `QuirkProfile`
(StrictModel, all quirks default OFF) + an enum registry of the above — the seam contract. No
quirk LOGIC. Tasks below reference `QuirkProfile` at their hook points but implement none.
---
## 0. HARD RULES (identical to PASS 14 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
(You MAY read `prod/clean_arch/exec/**` + `pink_direct.py` for router/venue conventions; do not edit.)
- **VIOLET DARK** — the mock NEVER touches a real venue, network, or key. No service/HZ control.
- **V-TYPES on all new code**; faithful poison-guards only.
- **NEW-FILE-ONLY** under a NEW package `prod/clean_arch/violet/exec/` (plus extending
`contracts_v3.py`). Do NOT modify in-flight files (the V3.4 sourcing/engine set, `cadence.py`,
`clock.py`, `exchange_leverage.py`, `scripted_venue.py`, `exec_driver.py`, `22_violet_decisions.sql`).
READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa5 -b agent/oa-violet5`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3/4 types (`VenueTick`, `OpenPositionView`, `ExitDecision`, `ExecIntent`,
`CapitalState`). ADD the execution vocabulary (all `StrictModel` / `Annotated` V-TYPES):
1. **`OrderType`** = Literal["MAKER","TAKER"]; **`OrderSide`** = Literal["BUY","SELL"];
**`OrderStatus`** = Literal["NEW","ACK","PARTIALLY_FILLED","FILLED","CANCELED","REJECTED"].
2. **`Order`** — `client_order_id: str`, `asset: Symbol`, `side: OrderSide`, `qty: Qty`,
`price: Px` (limit; maker), `order_type: OrderType`, `reduce_only: bool`,
`leverage: Annotated[int, Field(ge=1)]`, `ts_ns: MonoNs`.
3. **`OrderAck`** — `client_order_id: str`, `venue_order_id: str`, `status: OrderStatus`,
`ts_ns: MonoNs`, `reject_reason: str = ""`.
4. **`Fill`** — `venue_order_id: str`, `asset: Symbol`, `side: OrderSide`, `fill_qty: Qty`,
`fill_price: Px`, `bound_price: Px` (quirk #3 seam — carry it, never use it for PnL),
`fee: float (ge=0)`, `is_maker: bool`, `ownership_id: str` (quirk #2 seam),
`event_seq: Seq`, `ts_ns: MonoNs`.
5. **`PositionDelta`** — `asset: Symbol`, `qty_delta: float`, `realized_pnl_delta: float`,
`fee: float (ge=0)`, `event_seq: Seq` — the unit that feeds PASS-4 `EconomicsLedger`.
If a task needs another field, ADD it here and note it.
---
## TASK 20 — Execution contracts + QuirkProfile seam registry
**Why.** The shared execution vocabulary + the quirk-seam contract everything else depends on.
**Affected files (NEW):** extend `prod/clean_arch/violet/contracts_v3.py` (the §I types);
`prod/clean_arch/violet/exec/__init__.py`; `prod/clean_arch/violet/exec/bingx_quirks.py`
(`QuirkProfile` StrictModel, all flags default False + the quirk enum registry — NO logic);
`prod/clean_arch/violet/exec/test_violet_exec_contracts.py`.
**Pass criteria.** All types construct + poison-reject (negative qty, non-finite price, etc.);
`QuirkProfile()` defaults every quirk OFF; the enum lists exactly the 8 quirks above. No logic in
`bingx_quirks.py`. No edits outside the new/extended files.
## TASK 21 — Mock BingX order FSM
**Why.** The deterministic order lifecycle state machine — the heart of the mock.
**Affected files (NEW):** `prod/clean_arch/violet/exec/order_fsm.py`,
`prod/clean_arch/violet/exec/test_violet_order_fsm.py`.
**Interface/approach.** `OrderFSM` holds per-order state; legal transitions only:
NEW→ACK→{PARTIALLY_FILLED*→FILLED | CANCELED | REJECTED}. `submit(order)->OrderAck`,
`apply_fill(venue_order_id, fill_qty, fill_price)->(OrderStatus, Fill)`, `cancel(...)`. Illegal
transitions raise/reject (never silently). Reduce-only that would increase position → REJECTED
(quirk #6, normative). Partial fills accumulate; FILLED only when cumulative == order qty.
Monotonic `event_seq`. Pure/in-memory; deterministic (no wall clock — caller passes ts_ns).
**Pass criteria.** Full-fill, multi-partial→fill, cancel-after-partial, reject (reduce-only
increase), illegal-transition rejection; Hypothesis: cumulative fill ≤ order qty always; event_seq
strictly increasing. No edits outside the 2 files.
## TASK 22 — Mock BingX venue adapter (port-conformant, quirk-seamed)
**Why.** The venue itself: a `MockBingxVenue` implementing the execution-venue port so the router
/ driver can drive it exactly like the real one — but deterministic + DARK.
**Affected files (NEW):** `prod/clean_arch/violet/exec/mock_bingx_venue.py`,
`prod/clean_arch/violet/exec/test_violet_mock_bingx_venue.py`.
**Interface/approach.** Define `ExecutionVenuePort` (ABC) if one isn't already agreed: `set_leverage(asset,
lev)->OrderAck` (quirk #7 seam — separate ackable op), `submit(order)->OrderAck`,
`cancel(client_order_id)->OrderAck`, `poll_fills()->list[Fill]`, `close()`. Implement
`MockBingxVenue(fill_model, quirks: QuirkProfile = QuirkProfile())` over `OrderFSM`. Fill model is
INJECTABLE (`ScriptedFillModel` for tests: immediate full / N-partials / reject / maker-vs-taker).
Maker orders fill at limit price; taker at a provided reference (VenueTick.ask/bid). Fee = maker vs
taker rate (read the router's rate conventions; pass rates in, don't hardcode a venue-specific
number). **Quirk hooks present but inert:** when `quirks.zero_wb`/`foreign_fill`/`settle_desync`
are False (default) behave normatively; the hook points exist for a later quirk-injection pass.
**Pass criteria.** A `ScriptedFillModel` drives submit→ack→fill→FILLED; partial sequence;
reject; maker vs taker fill price + fee differ correctly; `set_leverage` acks independently;
with default `QuirkProfile()` NO quirk behaviour occurs. No live network. No edits outside the 2 files.
## TASK 23 — Fill → position/PnL reducer (fill-price PnL, ×leverage-aware)
**Why.** Turn `Fill`s into `PositionDelta`s correctly — the accounting seam where bound-price
poison (#3) and ×leverage (#4) would bite. Feeds PASS-4 `EconomicsLedger`.
**Affected files (NEW):** `prod/clean_arch/violet/exec/fill_reducer.py`,
`prod/clean_arch/violet/exec/test_violet_fill_reducer.py`.
**Interface/approach.** `apply_fill(pos: OpenPositionView | None, fill: Fill) ->
(OpenPositionView, PositionDelta)`. **PnL uses `fill.fill_price` ONLY — never `bound_price`**
(assert/comment quirk #3). Notional/qty handled WITHOUT double-applying leverage (quirk #4
document the convention explicitly). Realized PnL on reducing fills = signed (entryfill)×qty for
SHORT, etc.; fees subtracted; `event_seq` carried through. Pure function.
**Pass criteria.** Open, add, partial-reduce, full-close produce correct qty + realized PnL +
fee; a test asserts that a fill with `bound_price != fill_price` yields PnL from `fill_price`;
Hypothesis: qty ≥ 0; realized PnL finite. No edits outside the 2 files.
## TASK 24 — Mock execution integration harness (router + driver + venue, DARK storm)
**Why.** Prove the whole stack composes: `ExecIntent` (PASS-4 Task 17) → router/maker-policy →
`ExecDeadlineDriver` (V2) → `MockBingxVenue``Fill``fill_reducer``PositionDelta`
`EconomicsLedger` (PASS-4 Task 16). The end-to-end DARK proof.
**Affected files (NEW):** `prod/clean_arch/violet/exec/mock_exec_harness.py`,
`prod/clean_arch/violet/exec/test_violet_mock_exec_scenarios.py`, gate report →
`prod/VIOLET_dev/reports/violet_mock_exec_<UTC>.json`.
**Interface/approach.** Wire the components (READ `exec_harness.py`/`exec_driver.py` for the V2
storm pattern; reuse, don't fork). Run a seeded scenario matrix: clean entry→fill→exit; partial
fills; cancel/requote on TTL; reject; maker_both policy. Assert: every order reaches a terminal
state, capital == anchor + Σ deltas (ledger), NO order ever leaves the mock (DARK guard), and the
run is deterministic (same seed ⇒ same fills/deltas twice).
**Pass criteria (`@pytest.mark.gate`).** ≥ 8 scenarios green; determinism proven; ObserveOnly/DARK
guard asserts zero real-venue calls; report archived. No edits outside the new files.
---
## Composition map
```
contracts_v3 (+Order/OrderAck/Fill/OrderStatus/PositionDelta) + exec/bingx_quirks(QuirkProfile)
20 contracts + quirk seams
21 order_fsm → legal lifecycle, partials, reject
22 mock_bingx_venue(fsm, fill_model, quirks=OFF) → OrderAck + Fill stream (quirk hooks inert)
23 fill_reducer(pos, fill) → (OpenPositionView, PositionDelta) (fill-price PnL)
24 mock_exec_harness: ExecIntent → router/driver → 22 → 23 → PASS-4 EconomicsLedger (DARK E2E)
```
Integration into the live reactor + the real BingX client swap is the OWNER's job later. The
quirk-injection pass (turning `QuirkProfile` flags ON to test reconcile/filter logic) and the
mandatory real-key boundary smoke are SEPARATE, LATER work — NOT PASS 5.
## Recommended order
**20 (contracts+quirk seams) → 21 (FSM) → 23 (fill reducer) → 22 (venue) → 24 (harness/gate)**.
## Still NOT in scope (operator/owner only, or a later pass)
- **Quirk LOGIC / reconcile-against-quirks** — later quirk-injection pass (seams only here).
- **Real BingX client + real-key smoke** — mandatory before V4-live; operator-gated.
- **DARK soak start; HZ-bridge refactor; VIBRISS; the V3.4c parity root-cause (CRITICAL #1).**
- Any edit to the in-flight files / shared files in §0.

View File

@@ -0,0 +1,190 @@
# VIOLET — partial spec for another agent, PASS 6 (OA TODO): EXECUTION INTERNALS + QUIRK INJECTION
Date: 2026-06-17. Continues PASS 5 (the mock-BingX execution stack). PASS 5 built the NORMATIVE
mock (order FSM, venue, fill reducer) with quirk SEAMS left OFF. **PASS 6 builds the execution
INTERNALS** — reconcile, fill-pump, TTL-requote, orphan handling — **and IMPLEMENTS the quirk
INJECTION + handling logic**, flipping each `QuirkProfile` flag ON and proving the reconcile/filter
logic survives it. This reproduces, for VIOLET, the hard-won PINK production fixes.
**Read first (authoritative references — READ ONLY, never edit):**
- `prod/clean_arch/runtime/pink_direct.py` — PINK's live execution runtime: the EXACT reconcile,
fill-pump, ownership-filter, zero-wb-guard, orphan, and requote logic this pass mirrors.
- `prod/clean_arch/exec/**` — the ExecutionRouter (maker/taker policy, hooks).
- PASS 5 files: `prod/clean_arch/violet/exec/{order_fsm,mock_bingx_venue,fill_reducer,bingx_quirks}.py`
+ `contracts_v3.py` — the substrate PASS 6 composes (READ + IMPORT; do NOT edit).
- Memory/incident lineage of each quirk: `project_pink_orphan_fixes`,
`ditav2_kernel_audit_20260611`, `incident_pink_spool_diskfill_20260611`.
VIOLET's ALPHA models BLUE; VIOLET's EXECUTION reconcile/quirk-handling models **PINK's
production-tested logic** (PINK is the exec-active fork). Transcribe PINK faithfully — cite
`pink_direct.py:line` in each handler's docstring.
---
## 0. HARD RULES (identical to PASS 15 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`,
`prod/clean_arch/runtime/pink_direct.py`, `prod/clean_arch/exec/**`). READ only.
- **VIOLET DARK** — everything runs against the PASS-5 mock; NO real venue/network/key, no
service/HZ control.
- **V-TYPES on all new code**; faithful poison-guards only.
- **NEW-FILE-ONLY** under `prod/clean_arch/violet/exec/` (+ extend `contracts_v3.py`). Do NOT
modify the PASS-5 files (`order_fsm.py`, `mock_bingx_venue.py`, `fill_reducer.py`,
`bingx_quirks.py`) or any in-flight V3.4 file. Quirk INJECTION must be done by NEW
fill/frame models + wrappers that plug into PASS-5's existing seams (`MockBingxVenue` takes an
injectable `fill_model` + a `QuirkProfile`), NOT by editing the mock.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa6 -b agent/oa-violet6`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3/4/5 types (`VenueTick`, `OpenPositionView`, `ExecIntent`, `CapitalState`, `Order`,
`OrderAck`, `Fill`, `OrderStatus`, `PositionDelta`). ADD the account/reconcile vocabulary (all
`StrictModel` / `Annotated`):
1. **`AccountFrame`** — venue-reported account snapshot: `wallet_balance: float (finite, ge=0)`,
`available: float (ge=0)`, `ownership_id: str`, `event_seq: Seq`, `ts_ns: MonoNs`. (Carries the
zero-wb seam: `wallet_balance` may legitimately arrive as 0.0 in a poison frame — quirk #1.)
2. **`PositionFrame`** — venue-reported position: `asset: Symbol`, `qty: float`,
`entry_price: Px`, `leverage: Annotated[int, Field(ge=1)]`, `ownership_id: str`,
`event_seq: Seq`, `ts_ns: MonoNs`.
3. **`ReconcileCorrection`** — `asset: Symbol`, `kind: str`
("DRIFT"/"ORPHAN_LOCAL"/"ORPHAN_VENUE"/"NONE"), `local_qty: float`, `venue_qty: float`,
`action: str` ("ADOPT_VENUE"/"FLATTEN"/"QUARANTINE"/"NOOP"), `event_seq: Seq`.
4. **`OwnershipPolicy`** — `account_id: str`, `client_order_id_prefix: str` — the predicate used to
decide a fill/frame is OURS vs foreign (quirk #2). Provide `owns(ownership_id) -> bool`.
If a task needs another field, ADD it here and note it.
---
## TASK 25 — Account/position frame contracts + ownership policy
**Why.** The reconcile/filter vocabulary + the ownership predicate every handler needs.
**Affected files (NEW):** extend `contracts_v3.py` (the §I types);
`prod/clean_arch/violet/exec/ownership.py` (the `OwnershipPolicy` predicate impl);
`prod/clean_arch/violet/exec/test_violet_ownership.py`.
**Interface/approach.** `OwnershipPolicy.owns(ownership_id)` returns True iff the id matches our
account / client_order_id prefix (transcribe PINK's ownership check — cite `pink_direct.py:line`).
Frames/fills with a non-owned `ownership_id` are FOREIGN (must be filtered downstream).
**Pass criteria.** `owns()` correct for own vs foreign ids; frames poison-reject (non-finite
wallet_balance rejected, but **0.0 wallet_balance is ACCEPTED into the type** — it is a legitimate
poison-frame value the reconcile layer must SEE and then ignore, not a type error). No edits
outside the new/extended files.
## TASK 26 — Fill-pump (ownership filter + dedup → ledger deltas) [quirk #2]
**Why.** Drain fills from the venue, drop foreign fills, dedupe, convert to `PositionDelta` via the
PASS-5 fill reducer, feed the PASS-4 economics ledger. PINK's fill-pump is the reference.
**Affected files (NEW):** `prod/clean_arch/violet/exec/fill_pump.py`,
`prod/clean_arch/violet/exec/test_violet_fill_pump.py`.
**Interface/approach.** `FillPump(reducer, ledger, ownership: OwnershipPolicy)` with
`pump(fills: list[Fill]) -> list[PositionDelta]`: filter `ownership.owns(f.ownership_id)` (FOREIGN
fills dropped — **ownership fill filter, quirk #2**), dedupe by `(venue_order_id, event_seq)`,
apply each via PASS-5 `fill_reducer.apply_fill`, push the `PositionDelta` into the ledger
(`event_seq`-ordered). Idempotent: re-pumping the same fills produces no double-count.
**Pass criteria.** Own fills applied; foreign fills dropped (ledger unchanged); duplicate fills
deduped; out-of-order event_seq handled per ledger rules; idempotency proven. Cite the PINK
ownership filter. No edits outside the 2 files.
## TASK 27 — Reconcile loop (zero-wb guard + reseed-on-update + settle ordering) [quirks #1, #5]
**Why.** Periodically reconcile local position/capital vs venue frames; the zero-wb guard and
reseed-on-update are the documented PINK fixes that stop a transient `walletBalance=0` frame from
zeroing capital.
**Affected files (NEW):** `prod/clean_arch/violet/exec/reconcile.py`,
`prod/clean_arch/violet/exec/test_violet_reconcile.py`.
**Interface/approach.** `Reconciler(ownership)` with
`reconcile(local: dict[Symbol, OpenPositionView], account: AccountFrame,
positions: list[PositionFrame]) -> list[ReconcileCorrection]`:
- **Ignore foreign frames** (ownership).
- **Zero-wb guard (quirk #1):** if `account.wallet_balance == 0.0` (poison frame), DO NOT adopt it
— skip the capital update, log, keep prior (cite PINK).
- **Reseed-on-update:** adopt venue position state only on a genuine update (newer `event_seq`),
not on a stale/duplicate frame.
- **Settle/funding desync (quirk #5):** process frames strictly in `event_seq` order; a settle
event arriving out of fill order is reordered, never applied ahead of its fills.
- Emit `ReconcileCorrection`s for genuine drift (ADOPT_VENUE / FLATTEN), NOOP otherwise.
**Pass criteria.** A zero-wb frame does NOT change capital; a stale frame is ignored;
out-of-order settle is reordered; genuine drift yields the right correction. Hypothesis: capital
never set to 0 by a zero-wb frame. Cite PINK lines. No edits outside the 2 files.
## TASK 28 — TTL requote / cancel-replace (+ setLeverage ordering) [quirk #7]
**Why.** Maker orders that don't fill within their TTL must be canceled and re-submitted at a new
price; `setLeverage` is a separate ackable op that must be ordered before the order it applies to.
Composes PASS-5 `MockBingxVenue` + V2 `ExecDeadlineDriver`.
**Affected files (NEW):** `prod/clean_arch/violet/exec/requote.py`,
`prod/clean_arch/violet/exec/test_violet_requote.py`.
**Interface/approach.** `RequoteController(venue, driver)` with `on_deadline(order, new_price) ->
OrderAck`: cancel the stale maker order (assert it reaches CANCELED in the FSM), submit a fresh one
at `new_price`. **setLeverage ordering (quirk #7):** if leverage must change, issue `set_leverage`
and await its ack BEFORE submitting the order (no race). Reuse the V2 driver's TTL/deadline logic
(READ `exec_driver.py`); do not fork it.
**Pass criteria.** TTL expiry → cancel + requote at new price; the old order is CANCELED, the new
one ACKed; setLeverage ack precedes the dependent order; requote count tracked. Determinism. No
edits outside the 2 files.
## TASK 29 — Orphan detection + handling [the PINK orphan fixes]
**Why.** Orders/positions can exist on the venue but not locally (or vice versa) — orphans. PINK's
orphan-reconcile is the reference; misget handling caused real incidents.
**Affected files (NEW):** `prod/clean_arch/violet/exec/orphans.py`,
`prod/clean_arch/violet/exec/test_violet_orphans.py`.
**Interface/approach.** `detect_orphans(local: dict, venue_positions: list[PositionFrame],
ownership) -> list[ReconcileCorrection]`:
- **ORPHAN_VENUE:** venue has an owned position we have no local record of → action ADOPT_VENUE
(or FLATTEN per policy; default ADOPT_VENUE with a quarantine flag).
- **ORPHAN_LOCAL:** we have a local position the venue doesn't report → action QUARANTINE (do not
silently delete; the zombie-trade lesson — never resurrect/erase by guesswork).
- Match by chain token / client_order_id where available; ambiguous → QUARANTINE.
**Pass criteria.** Each orphan class yields the correct `ReconcileCorrection`; foreign positions
ignored; ambiguous → quarantine, never silent flatten/resurrect. Cite the PINK orphan fix. No
edits outside the 2 files.
## TASK 30 — Quirk-injection suite + gate (flip every QuirkProfile flag ON) [all quirks]
**Why.** Prove the PASS-6 handlers actually survive each quirk by INJECTING it through PASS-5's
seams and asserting the outcome. This is the payoff of the seam discipline.
**Affected files (NEW):** `prod/clean_arch/violet/exec/quirk_injection.py` (the injecting
fill/frame models that plug into PASS-5's `MockBingxVenue(fill_model=…, quirks=…)` and a
`QuirkAccountStream` emitting injected `AccountFrame`/`PositionFrame`s);
`prod/clean_arch/violet/exec/test_violet_quirk_injection_gate.py`; gate report →
`prod/VIOLET_dev/reports/violet_quirk_injection_<UTC>.json`.
**Interface/approach.** For each `QuirkProfile` flag, build the injecting model and assert the
matching handler neutralizes it:
- `zero_wb=True` → Reconciler keeps capital (Task 27).
- `foreign_fill=True` → FillPump drops the foreign fill (Task 26).
- `bound_price_poison=True` → fill_reducer PnL uses fill_price not bound_price (PASS-5 Task 23, re-asserted).
- `settle_desync=True` → Reconciler reorders by event_seq (Task 27).
- `setlev_race=True` → RequoteController orders setLeverage first (Task 28).
- `reduce_only_increase=True` → OrderFSM rejects (PASS-5 Task 21, re-asserted).
- (×leverage notional, dead `.pro` TLS) → documented; ×leverage asserted in fill_reducer,
`.pro` TLS noted as connection-layer/real-client-only.
**Pass criteria (`@pytest.mark.gate`).** Every quirk flag flipped ON in at least one scenario, each
neutralized by its handler; a combined "all quirks on" storm still reaches consistent
ledger/position state; report archived. No edits outside the new files.
---
## Composition map
```
PASS-5 mock (order_fsm, mock_bingx_venue[seams], fill_reducer, QuirkProfile)
25 contracts(AccountFrame/PositionFrame/ReconcileCorrection) + ownership
26 fill_pump(ownership filter, dedup) ───► PositionDelta ──► PASS-4 EconomicsLedger
27 reconcile(zero-wb guard, reseed, settle-order) ─► ReconcileCorrection
28 requote(TTL cancel/replace, setLeverage order) ─► uses PASS-5 venue + V2 driver
29 orphans(detect) ─► ReconcileCorrection (quarantine, never silent flatten)
30 quirk_injection + GATE: flip every QuirkProfile flag ON, assert each handler neutralizes it
```
Integration (running reconcile/pump/requote on the live reactor against the REAL BingX client) is
the OWNER's job later. The **real-key boundary smoke remains MANDATORY before V4-live** — quirk
injection proves the LOGIC, never the undocumented venue reality.
## Recommended order
**25 (contracts+ownership) → 26 (fill-pump) → 27 (reconcile) → 29 (orphans) → 28 (requote) → 30
(quirk-injection gate)**.
## Still NOT in scope (operator/owner only, or a later pass)
- **Real BingX client + real-key smoke** — mandatory before V4-live; operator-gated.
- **Live reactor wiring** of reconcile/pump/requote — owner's integration job.
- **DARK soak start; HZ-bridge refactor; VIBRISS; the V3.4c parity root-cause (CRITICAL #1).**
- Any edit to PASS-5 files, in-flight V3.4 files, or shared files in §0.

View File

@@ -0,0 +1,175 @@
# VIOLET — partial spec for another agent, PASS 7 (OA TODO): V5 SELECTION + MULTI-ASSET SLOTS + CAPITAL ALLOCATION
Date: 2026-06-17. Continues PASS 16 (same V0→V6 plan). PASS 7 is the **V5** layer of the ladder:
faithful IRP/ARS asset RANKING, MULTI-ASSET SLOT management (concurrent positions), and CAPITAL
ALLOCATION across slots. Everything DARK and as independent, separately-testable units sharing the
`contracts_v3` vocabulary.
**⚠️ Relevance to CRITICAL #1 (read `VIOLET_TODO_CRITICAL.md`).** The disappointing live parity
(pick-match 1.5%, no-pick 86%, **but sizing near-identical when assets align**) points squarely at
SELECTION/TIMING, i.e. THIS layer. PASS 7 BUILDS the faithful selection machinery and pins the
ranking bit-for-bit to BLUE's `AlphaAssetSelector` (Task 36). That is COMPLEMENTARY to — not a
substitute for — the live-aggregate parity root-cause, which remains **Claude's** job (the live
join/alignment in `parity_report.py` is the other suspect). Do NOT attempt the live root-cause
here; build the units + the unit-level bit-identity pin.
**Read first (authoritative references — READ ONLY):**
- `nautilus_dolphin/nautilus_dolphin/nautilus/alpha_asset_selector.py``AlphaAssetSelector.rank_assets`
(IRP selection, ARS scoring, BIBLE §5) — the ranking authority.
- `prod/docs/SYSTEM_BIBLE.md` §5 (selection), §490 "OB Sub-1: ARS adjusted ±5%/10% by per-asset OB
depth quality before sorting", §735 OB sub-systems. (BIBLE is directionally right but can be
outdated — verify against the kernel code.)
- `prod/clean_arch/violet/alpha_wrappers.py` (`VioletAssetSelector`, the single-pick wrapper) +
`decision_engine.py` (how a pick + `ars_score` flow today) + `sizing.py` (notional model) — READ,
do NOT edit (in-flight).
- ENGINE_KWARGS selection knobs (`nautilus_event_trader.py:127+`): `use_asset_selection=True`,
`min_irp_alignment=0.0` ("gold spec: no IRP filter"), `max_slots` (launcher venue config).
- Stablecoin exclusion: `decision_engine.STABLECOIN_SYMBOLS` (must match BLUE's set).
---
## 0. HARD RULES (identical to PASS 16 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
- **VIOLET DARK** — selection/slots/allocation are pure logic over scan + factor inputs; no orders,
no venue, no service/HZ control.
- **V-TYPES on all new code**; faithful poison-guards only; NO arbitrary caps.
- **NEW-FILE-ONLY** under `prod/clean_arch/violet/selection/` (+ extend `contracts_v3.py`). Do NOT
modify in-flight files (`alpha_wrappers.py`, `decision_engine.py`, `sizing.py`,
`live_blue_source.py`, the PASS-5/6 exec files, `cadence.py`, `clock.py`). READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa7 -b agent/oa-violet7`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3..6 types. ADD the selection/slot vocabulary (all `StrictModel` / `Annotated`):
1. **`AssetRank`** — `asset: Symbol`, `ars_score: float (finite)`,
`ob_adjusted_score: float (finite)`, `irp_alignment: float`, `irp_passed: bool`, `rank: int (ge=0)`,
`excluded_reason: str = ""` (e.g. "STABLECOIN"/"IRP_FILTER"/"" ).
2. **`SlotState`** — `slot_id: int (ge=0)`, `asset: Optional[Symbol]`, `status: str`
("FREE"/"HELD"/"PENDING"), `held_since_ns: Optional[MonoNs]`.
3. **`CapitalAllocation`** — `slot_id: int`, `asset: Symbol`, `allocated_capital: float (ge=0)`,
`notional_fraction: float (ge=0)`, `conviction_leverage: float (ge=0)`.
4. **`SlotPolicy`** — `max_slots: Annotated[int, Field(ge=1)]`,
`hysteresis_bars: Annotated[int, Field(ge=0)]` (anti-churn), `allow_reentry: bool`.
If a task needs another field, ADD it here and note it.
---
## TASK 31 — Selection contracts + SlotPolicy
**Why.** The shared selection/slot vocabulary everything else imports.
**Affected files (NEW):** extend `contracts_v3.py` (the §I types);
`prod/clean_arch/violet/selection/__init__.py`;
`prod/clean_arch/violet/selection/test_violet_selection_contracts.py`.
**Pass criteria.** All types construct + poison-reject (non-finite scores, negative capital,
max_slots ≥ 1); `excluded_reason` enumerated values documented. No edits outside the new/extended files.
## TASK 32 — ARS ranking pipeline (faithful to AlphaAssetSelector + OB Sub-1 + IRP filter)
**Why.** The heart of selection: reproduce BLUE's full ranking — ARS score → OB Sub-1 adjustment →
IRP alignment filter → stablecoin exclusion → sort. The single-pick `VioletAssetSelector` covers
part of this; PASS 7 builds the FULL multi-asset ranking on top WITHOUT editing it.
**Affected files (NEW):** `prod/clean_arch/violet/selection/rank_pipeline.py`,
`prod/clean_arch/violet/selection/test_violet_rank_pipeline.py`.
**Interface/approach.** `rank_assets(market_data: dict[Symbol, list[float]], *, regime_direction:
int, ob_market=None, min_irp_alignment: float, stablecoins: frozenset[str]) -> list[AssetRank]`:
- Compute the base ARS exactly as `AlphaAssetSelector` (WRAP/transcribe — cite file:line).
- **OB Sub-1** (BIBLE §490): adjust ARS ±5%/10% by per-asset OB depth quality BEFORE sorting (read
the kernel for the exact factors/percentages; if `ob_market` is None, skip the adjustment, as the
no-OB path does).
- **IRP alignment filter:** drop assets with alignment < `min_irp_alignment` (default 0.0 = no
filter, per gold spec) → mark `irp_passed`/`excluded_reason`.
- **Stablecoin exclusion:** any asset in `stablecoins` → excluded (must equal
`decision_engine.STABLECOIN_SYMBOLS`; assert in test).
- Sort descending by `ob_adjusted_score`; assign `rank`. Pure function, deterministic, V-TYPES out.
**Pass criteria.** On crafted market data the ranked order + ARS/ob_adjusted scores match a direct
`AlphaAssetSelector` computation; stablecoins always excluded; IRP filter respected; OB-off path
equals base ARS. No edits outside the 2 files.
## TASK 33 — Multi-asset slot manager (max_slots, assignment, hysteresis)
**Why.** BLUE/VIOLET hold up to `max_slots` concurrent positions; the slot manager decides which
ranked assets occupy slots, with no double-occupancy and anti-churn hysteresis.
**Affected files (NEW):** `prod/clean_arch/violet/selection/slot_manager.py`,
`prod/clean_arch/violet/selection/test_violet_slot_manager.py`.
**Interface/approach.** `SlotManager(policy: SlotPolicy)` holding `list[SlotState]`. `assign(ranked:
list[AssetRank], held: dict[Symbol, OpenPositionView], now_ns) -> list[SlotState]`: keep currently
HELD assets in their slots; fill FREE slots from the top of `ranked` (excluding already-held and
excluded assets); never assign the same asset to two slots; respect `hysteresis_bars` (do not evict
a freshly-taken slot to chase a higher rank within the hysteresis window). READ how BLUE/the trader
manages `max_slots` (launcher venue config + engine) and transcribe the rule; if BLUE is
single-slot today (`max_slots=1`), the manager must still be correct + generalize to N.
**Pass criteria.** Held assets retained; free slots filled by rank; no double-occupancy; hysteresis
prevents churn; max_slots respected. Hypothesis: |HELD slots| ≤ max_slots always; no asset in two
slots. No edits outside the 2 files.
## TASK 34 — Capital allocation across slots
**Why.** Turn slot occupancy + conviction into per-slot capital + notional, honoring BLUE's sizing
convention (notional = base_fraction × conviction × capital) and the margin-study findings.
**Affected files (NEW):** `prod/clean_arch/violet/selection/capital_allocator.py`,
`prod/clean_arch/violet/selection/test_violet_capital_allocator.py`.
**Interface/approach.** First READ how BLUE allocates capital across concurrent slots (shared pool
vs per-slot; the margin study `blue_margin_envelope_study` + `sizing.py` are the references —
notional = 0.20 × conviction × capital; capital under-utilized at 1 slot was DELIBERATE). Define
`allocate(slots: list[SlotState], convictions: dict[Symbol, float], *, capital: float, base_fraction:
float) -> list[CapitalAllocation]` reproducing that convention EXACTLY (cite the source). Do NOT
invent a normalization BLUE doesn't do.
**Pass criteria.** Per-slot notional == base_fraction × conviction × capital (the documented model);
sum-of-notionals behavior matches BLUE's (shared-capital, not artificially normalized, unless BLUE
normalizes — verify); poison guards reject non-finite/negative. Cite the allocation source. No edits
outside the 2 files.
## TASK 35 — Selection→slot→intent multi-asset flow harness (DARK)
**Why.** Compose the V5 path end-to-end: `rank_pipeline``slot_manager``capital_allocator`
(existing sizing) → `ExecIntent` per occupied slot (PASS-4 Task 17), DARK.
**Affected files (NEW):** `prod/clean_arch/violet/selection/multi_asset_flow.py`,
`prod/clean_arch/violet/selection/test_violet_multi_asset_flow.py`.
**Interface/approach.** `run_selection_cycle(market_data, held, *, factors, capital, policy, ...) ->
list[ExecIntent]` wiring the four units. Reuse `sizing.VioletSizer` for conviction (import, don't
fork). DARK: emits intents, never orders. Deterministic.
**Pass criteria.** A multi-asset scenario yields one intent per occupied slot with correct
asset/qty/notional; held assets not re-entered (unless `allow_reentry`); max_slots respected
end-to-end; deterministic. No edits outside the 2 files.
## TASK 36 — Ranking bit-identity gate vs the real AlphaAssetSelector
**Why.** Pin the ranking to BLUE's actual `AlphaAssetSelector` over a sampled grid — the unit-level
parity that, once green, removes selection-math as a suspect for CRITICAL #1 (leaving timing/join as
the remaining live-aggregate suspect for Claude).
**Affected files (NEW):** `prod/clean_arch/violet/selection/test_violet_rank_parity_gate.py`; gate
report → `prod/VIOLET_dev/reports/violet_rank_parity_<UTC>.json`.
**Interface/approach.** Over many synthetic universes (varied asset counts, price histories,
directions, with/without OB), assert `rank_pipeline.rank_assets` produces the SAME ordering + the
SAME ARS/ob_adjusted scores (`==`) as a direct `AlphaAssetSelector` run on the same inputs. Record
mismatch count (must be 0).
**Pass criteria (`@pytest.mark.gate`).** ≥ 200 universes, zero ranking/score mismatches; report
archived. If mismatches appear, that IS a finding — report them, do not loosen. No edits outside the
new file.
---
## Composition map
```
contracts_v3 (+AssetRank/SlotState/CapitalAllocation/SlotPolicy)
32 rank_pipeline(market, ob, irp, stables) → [AssetRank] (faithful to AlphaAssetSelector + OB Sub-1)
33 slot_manager(ranked, held, policy) → [SlotState] (max_slots, hysteresis, no double-occ)
34 capital_allocator(slots, convictions, cap)→ [CapitalAllocation] (BLUE notional convention)
35 multi_asset_flow: 32→33→34→ sizing → [ExecIntent] (DARK V5 cycle)
36 rank_parity_gate: rank_pipeline == AlphaAssetSelector (bit-identity) ← narrows CRITICAL #1
```
Integration (running the V5 cycle on the live reactor, wiring intents to the PASS-5/6 exec stack) is
the OWNER's job later.
## Recommended order
**31 (contracts) → 32 (rank pipeline) → 36 (rank parity gate) → 33 (slots) → 34 (allocation) → 35
(flow harness)**. 36 right after 32 so the ranking is pinned before building on it.
## Still NOT in scope (operator/owner only, or Claude)
- **The live BLUE↔VIOLET aggregate parity root-cause (CRITICAL #1)** — Claude's job; this pass only
pins ranking-math + builds V5 units.
- **Alpha re-timing / sub-second actuation of entries** — VBT re-certification (research), not a unit.
- **DARK soak start; V4 live execution; HZ-bridge; VIBRISS.**
- Any edit to in-flight / shared files in §0.

View File

@@ -0,0 +1,179 @@
# VIOLET — partial spec for another agent, PASS 8 (OA TODO): V6 BIBLE LAYERS (POSTURE / MARAS / REGIME) + CADENCE TELEMETRY
Date: 2026-06-17. Continues PASS 17 (same V0→V6 plan). PASS 8 is the **V6** bible layer. V6 =
"full bible layers (ACB / MARAS / posture / vol) + sub-second SL guard." Of those:
- **ACB** (boost/beta) is DONE (`live_blue_source.py`, bit-identical to `get_dynamic_boost_from_hz`).
- **vol** gate is PASS-4 Task 14; **sub-second SL guard** is PASS-3 Task 10.
- **REMAINING for PASS 8: POSTURE (the 5-state effects), MARAS (fingerprint consumer), and the
REGIME read-model that composes them — plus the CADENCE shadow-actuation telemetry** the plan
requires before any Q loosening.
All DARK; independent, separately-testable units sharing the `contracts_v3` vocabulary.
**Crucial framing:** VIOLET **CONSUMES** posture/MARAS — it does NOT decide them. BLUE's
meta-health service (MHS) sets posture; VIOLET reads it from HZ (already sourced via
`live_blue_source``engine_snapshot['posture']` / `DOLPHIN_SAFETY.latest.posture`). These units
APPLY the published posture/MARAS EFFECTS faithfully; they do NOT run the MHS state machine.
**Read first (authoritative references — READ ONLY):**
- `nautilus_dolphin/nautilus_dolphin/nautilus/esf_alpha_orchestrator.py``_day_posture` handling:
`:365` HIBERNATE_HALT (force EXIT), `:613` STALKER structural ceiling, `:918-978` begin_day
posture wiring; and the `regime_dd_halt` set on `mc_red or posture in ['TURTLE','HIBERNATE']`.
- `prod/clean_arch/violet/sizing.py` — the STALKER `clamped_max=min(clamped_max, 2.0)` already in
`compose` (READ; do NOT edit; the posture *size-cap* lives there — PASS 8 owns the *entry-gate*
+ observability, not the sizing math).
- **MARAS:** grep the kernels for `maras` / `MARAS` / `maras_fingerprint` (memory references
"maras_fingerprint columns"); determine its EXACT role in the decision/sizing path. The BIBLE
may mention it (directionally right, possibly outdated — verify against code).
- `prod/clean_arch/violet/cadence.py` (`CadenceControlPlane`, `Action`) + PASS-3
`cadence_schedule.py` — READ; PASS 8 extends the telemetry, does not edit them.
- `live_blue_source.py` (posture sourcing) + `decision_engine.py` (how `posture` enters `factors`).
---
## 0. HARD RULES (identical to PASS 17 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
- **VIOLET DARK** — pure consumers over published posture/MARAS/regime inputs; no orders, no venue,
no service/HZ control, no PROGREEN.
- **V-TYPES on all new code**; faithful poison-guards only; NO arbitrary caps.
- **NEW-FILE-ONLY** under `prod/clean_arch/violet/regime/` (+ extend `contracts_v3.py`). Do NOT
modify in-flight files (`sizing.py`, `decision_engine.py`, `live_blue_source.py`, `cadence.py`,
`clock.py`, the PASS-5/6/7 files). READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa8 -b agent/oa-violet8`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3..7 types. ADD the V6 regime vocabulary (all `StrictModel` / `Annotated`):
1. **`Posture`** = Literal["APEX","STALKER","RESTORED","TURTLE","HIBERNATE"] (must equal
`sizing.Posture`).
2. **`PostureEffect`** — `posture: Posture`, `entry_allowed: bool`, `force_flatten: bool`,
`size_cap: Optional[float]` (e.g. 2.0 for STALKER; None = no extra cap), `regime_dd_halt: bool`.
3. **`MarasFingerprint`** — the published MARAS state (fields TBD by Task 39 after reading the
kernel; at minimum a typed wrapper with finite/typed fields + a `regime_label: str`).
4. **`RegimeView`** — read-model composing the live regime: `posture: Posture`, `boost: float`,
`beta: float`, `mc_scale: float`, `vol_ok: Optional[bool]`, `maras: Optional[MarasFingerprint]`,
`regime_dd_halt: bool`, `ts_ns: MonoNs`. Observability/divergence only — NOT a sizing path.
5. **`ActionCadenceDelta`** — `action: str`, `evaluations: int (ge=0)`, `actuations: int (ge=0)`,
`suppressed: int (ge=0)` — the shadow-delta per action.
If a task needs another field, ADD it here and note it.
---
## TASK 37 — Regime/posture contracts
**Why.** The shared V6 vocabulary.
**Affected files (NEW):** extend `contracts_v3.py` (the §I types);
`prod/clean_arch/violet/regime/__init__.py`;
`prod/clean_arch/violet/regime/test_violet_regime_contracts.py`.
**Pass criteria.** All types construct + poison-reject; `Posture` literal equals `sizing.Posture`
(assert in test); `size_cap` optional/finite. No edits outside the new/extended files.
## TASK 38 — Posture effects engine (the 5-state EFFECTS, faithful)
**Why.** Apply each posture's documented EFFECT on the decision path — the entry-gate + flatten +
size-cap — exactly as BLUE. (The STALKER size-cap of 2.0 already lives in `sizing.compose`; this
unit owns the ENTRY-GATE + FLATTEN + the typed effect, NOT the sizing math.)
**Affected files (NEW):** `prod/clean_arch/violet/regime/posture_effects.py`,
`prod/clean_arch/violet/regime/test_violet_posture_effects.py`.
**Interface/approach.** `posture_effect(posture: Posture) -> PostureEffect`, transcribed from the
orchestrator (cite file:line):
- **APEX / RESTORED:** `entry_allowed=True`, no flatten, no extra cap.
- **STALKER:** `entry_allowed=True`, `size_cap=2.0` (the `:613` structural ceiling), no flatten.
- **TURTLE:** `regime_dd_halt=True``entry_allowed=False` (halt new entries), no forced flatten of
existing (verify the exact TURTLE behaviour in the kernel).
- **HIBERNATE:** `entry_allowed=False`, `force_flatten=True` (`:365` HIBERNATE_HALT forces EXIT),
`regime_dd_halt=True`.
Verify each against the kernel; if any differs, follow the CODE (BIBLE may be outdated).
**Pass criteria.** Each posture maps to the cited effect; HIBERNATE flattens + blocks entry; TURTLE
blocks entry; STALKER caps at 2.0; APEX/RESTORED normal. A test asserts the STALKER cap equals the
2.0 used in `sizing.compose`. No edits outside the 2 files.
## TASK 39 — MARAS fingerprint consumer
**Why.** MARAS is named in V6 ("full bible layers ACB/MARAS/posture/vol"). Build the faithful
consumer of the published MARAS fingerprint.
**Affected files (NEW):** `prod/clean_arch/violet/regime/maras_consumer.py`,
`prod/clean_arch/violet/regime/test_violet_maras_consumer.py`.
**Interface/approach.** FIRST establish MARAS's real role: grep the kernels + the HZ publishers for
`maras`/`maras_fingerprint`; find (a) WHERE MARAS is published (which HZ map/key), (b) WHETHER it
modulates sizing/gating in the live path or is observability-only. Then:
- If MARAS modulates the decision: build `consume_maras(raw) -> MarasFingerprint` (wrap BLUE's
parser if one exists; else V-TYPES the published fields) + the effect it applies, transcribed +
cited.
- If MARAS is observability-only (NOT in the live decision path): build the read-only typed
accessor + DOCUMENT explicitly that it does not modulate decisions today (honest — do NOT invent
a modulation BLUE doesn't apply).
**Pass criteria.** `MarasFingerprint` parses the real published payload shape (captured-fixture
test); the consumer's role (modulating vs observability) is determined from the code and
documented with citations. No invented effect. No edits outside the 2 files.
## TASK 40 — Regime composite read-model
**Why.** A single typed `RegimeView` composing posture + ACB(boost/beta) + mc_scale + vol + MARAS
for observability and divergence — NOT a new sizing path (sizing.py owns the math).
**Affected files (NEW):** `prod/clean_arch/violet/regime/regime_view.py`,
`prod/clean_arch/violet/regime/test_violet_regime_view.py`.
**Interface/approach.** `build_regime_view(*, posture, boost, beta, mc_scale, vol_ok=None,
maras=None, now_ns) -> RegimeView` — pure assembly + `regime_dd_halt` derived from posture
(Task 38) and mc state. Read-only; emits the view for a future divergence/observability sink. MUST
NOT recompute or alter any sizing factor.
**Pass criteria.** View assembled correctly; `regime_dd_halt` matches the posture effect; no sizing
recomputation (a test asserts inputs pass through unchanged). No edits outside the 2 files.
## TASK 41 — Cadence shadow-actuation telemetry harness
**Why.** The plan's cadence doctrine: "evaluate at fastest cadence (shadow-log would-be actions),
actuate at Q; measure shadow deltas before promoting." PASS-3 Task 13 added the Q-schedule + a
telemetry recorder; THIS unit runs the evaluate/actuate loop and produces per-action
`ActionCadenceDelta`s — the shadow evidence required before stepping any Q down.
**Affected files (NEW):** `prod/clean_arch/violet/regime/cadence_telemetry_harness.py`,
`prod/clean_arch/violet/regime/test_violet_cadence_telemetry_harness.py`, report →
`prod/VIOLET_dev/reports/violet_cadence_shadow_<UTC>.json`.
**Interface/approach.** Compose PASS-3 `cadence_schedule` + `CadenceControlPlane` (READ; import,
don't edit). Drive a synthetic action stream where each action EVALUATES every tick but ACTUATES
only when its Q is due; record `evaluations` / `actuations` / `suppressed` per action and emit the
deltas. This quantifies, e.g., "how many would-be SL exits fire at fast cadence vs the scan-Q
actuations" — the promote/loosen evidence.
**Pass criteria.** For a known stream, evaluations > actuations per fast action; per-action Q
honored; deltas match a hand-computed expectation; report archived. No edits outside the new files.
## TASK 42 — Posture/regime parity gate vs BLUE
**Why.** Pin the posture EFFECTS (and MARAS, if modulating) to BLUE's actual gating over a replay —
bit-identity where applicable.
**Affected files (NEW):** `prod/clean_arch/violet/regime/test_violet_regime_parity_gate.py`; report
`prod/VIOLET_dev/reports/violet_regime_parity_<UTC>.json`.
**Interface/approach.** For each posture (+ MARAS state if it modulates), assert VIOLET's
`posture_effect` / consumer reproduces BLUE's entry-allowed / flatten / cap / halt decision (drive
the orchestrator's relevant branch, or assert against the cited constants). Record mismatches
(must be 0).
**Pass criteria (`@pytest.mark.gate`).** Every posture's effect matches BLUE; HIBERNATE/TURTLE halts
reproduced; STALKER 2.0 cap matches; zero mismatches; report archived. No edits outside the new file.
---
## Composition map
```
contracts_v3 (+Posture/PostureEffect/MarasFingerprint/RegimeView/ActionCadenceDelta)
38 posture_effects(posture) → PostureEffect (entry gate / flatten / cap / dd_halt)
39 maras_consumer(raw) → MarasFingerprint (+ effect, IFF it modulates; else read-only)
40 regime_view(posture,acb,mc,vol,maras) → RegimeView (observability/divergence; NOT sizing)
41 cadence_telemetry_harness → [ActionCadenceDelta] (shadow evidence before Q loosening)
42 regime_parity_gate: effects == BLUE gating (bit-identity)
```
Integration (wiring posture-gating into the live decision path, the regime view into a divergence
sink, the cadence deltas into the Q-promotion decision) is the OWNER's job later. Posture/MARAS
remain CONSUMED, never decided (MHS owns them).
## Recommended order
**37 (contracts) → 38 (posture effects) → 42 (regime parity gate) → 39 (MARAS) → 40 (regime view) →
41 (cadence telemetry)**. 42 right after 38 to pin posture before composing.
## Still NOT in scope (operator/owner only, or Claude)
- **Running the MHS / DECIDING posture** — BLUE owns it; VIOLET consumes.
- **Q loosening / sub-second actuation of alpha exits** — needs VBT re-cert (research), not a unit.
- **The live BLUE↔VIOLET aggregate parity root-cause (CRITICAL #1)** — Claude's job.
- **DARK soak start; V4 live execution; HZ-bridge; VIBRISS.**
- Any edit to in-flight / shared files in §0.

View File

@@ -0,0 +1,173 @@
# VIOLET — partial spec for another agent, PASS 9 (OA TODO): ECONOMICS + OBSERVABILITY COMPLETENESS
Date: 2026-06-17. Final parallelizable pass (PASS 18 cover the rest of the DARK-buildable surface).
PASS 9 closes the **economics/observability** inheritance the plan binds VIOLET to, and the
**soak-readiness** machinery that decides go/no-go for a DARK soak. All DARK; independent units
sharing `contracts_v3`.
**Plan obligations this pass discharges (V1 inheritance, BINDING):**
> "VIOLET inherits and must keep: `pnl_source`/`capital_source` provenance, `event_seq` on every CH
> row, `wait_for_async_insert=1` on economics tables, exactly-one-row-per-event (the PINK
> duplicate-emission fix is open spec debt — VIOLET gates on it), DDL-before-code migration
> discipline, per-trade sizer feedback from trade-realized PnL (never capital deltas)."
Plus: capital = anchor + Σ deltas (never last-value) — the zombie-trade lesson (PASS-4 ledger).
**Read first (authoritative references — READ ONLY):**
- `prod/clean_arch/violet/shadow_journal.py` — the validate-then-sink + reject-at-source pattern
(the model for every PASS-9 sink). `prod/clean_arch/violet/domain.py` — V-TYPES (`DivergenceRow`,
EpochMs/Seq/MonoNs etc.). `prod/clean_arch/violet/divergence.py` — the V1 divergence monitor.
- `prod/clickhouse/violet/*.sql` + `apply_violet_ddl.py` — DDL-first migration discipline; the
`test_apply_violet_ddl.py` pattern; the row-set==DDL-columns parity test in
`test_violet_shadow_journal.py`.
- `prod/ch_writer.py` — READ for the CH insert path / `wait_for_async_insert` convention; **do NOT
edit it** (its design flaws are documented; VIOLET uses its own sink, mirroring the journal).
- PASS-2 `parity_report.py`, PASS-3 `slippage_metric.py`, PASS-8 cadence telemetry — the reports the
soak-readiness aggregator (Task 47) consolidates.
- CH (read-only): `http://localhost:8123`, user `dolphin`/key `dolphin_ch_2026`; VIOLET db
`dolphin_violet`. NEVER write/alter PRODUCTION tables; new VIOLET tables only, DDL-first.
---
## 0. HARD RULES (identical to PASS 18 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`,
`prod/ch_writer.py`). READ only.
- **VIOLET DARK** — sinks write only to NEW `dolphin_violet` tables; no orders, no venue, no
service/HZ control, no PROGREEN. Never write a production/BLUE table.
- **V-TYPES on all new code**; faithful poison-guards; reject-at-source to a counter/dead_letter,
never crash the sink (the bars_held=-106 spool lesson).
- **DDL-FIRST**: every new table ships a `.sql` in `prod/clickhouse/violet/` BEFORE the sink code;
a row-set==DDL-columns parity test is MANDATORY (mirror `test_violet_shadow_journal.py`).
- **NEW-FILE-ONLY** under `prod/clean_arch/violet/obs/` (+ extend `contracts_v3.py` + new `.sql`).
Do NOT modify in-flight files (`shadow_journal.py`, `divergence.py`, `domain.py`,
`22_violet_decisions.sql`, the PASS-5..8 files). READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa9 -b agent/oa-violet9`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3..8 types (`CapitalState`, `PositionDelta`, `RegimeView`, etc.). ADD:
1. **`Provenance`** — `pnl_source: str`, `capital_source: str`, `source: str`, `event_seq: Seq`,
`ts: EpochMs`. The provenance stamp every economics row carries.
2. **`EconomicsRow`** — a validated `dolphin_violet.violet_economics` row: `session_id: SessionId`,
`event_seq: Seq`, `asset: Symbol`, `realized_pnl: float (finite)`, `fee: float (ge=0)`,
`capital_after: float (ge=0)`, `pnl_source: str`, `capital_source: str`, `ts: EpochMs`,
`mono_ns: MonoNs`. Field set MUST equal its DDL columns (parity test).
3. **`DecisionDivergenceRow`** — extends the V1 divergence idea to the FACTOR plane: `ts: EpochMs`,
`asset: Symbol`, `factor: str`, `violet_value: float`, `blue_value: float`, `abs_err: float`,
`event_seq: Seq`.
4. **`SoakReadiness`** — `parity_pick_match: float`, `latency_p99_ms: float`,
`determinism_ok: bool`, `exactly_one_row_ok: bool`, `namespace_isolation_ok: bool`,
`verdict: str` ("GO"/"NO_GO"), `reasons: list[str]`.
If a task needs another field, ADD it here and note it.
---
## TASK 43 — Observability/economics contracts
**Why.** The shared provenance/economics/divergence vocabulary.
**Affected files (NEW):** extend `contracts_v3.py`; `prod/clean_arch/violet/obs/__init__.py`;
`prod/clean_arch/violet/obs/test_violet_obs_contracts.py`.
**Pass criteria.** All types construct + poison-reject (non-finite pnl, negative capital/fee
rejected); provenance strings non-empty where required. No edits outside the new/extended files.
## TASK 44 — DDL-first exactly-one-row-per-event economics sink [the PINK duplicate-emission fix]
**Why.** The binding inheritance: exactly-one-row-per-event, `event_seq` on every row,
`wait_for_async_insert=1`, validate-then-sink, reject-at-source. This is VIOLET paying down the
PINK duplicate-emission debt.
**Affected files (NEW):** `prod/clickhouse/violet/30_violet_economics.sql` (DDL FIRST),
`prod/clean_arch/violet/obs/economics_sink.py`, `prod/clean_arch/violet/obs/test_violet_economics_sink.py`.
**Interface/approach.** DDL: `dolphin_violet.violet_economics` (columns == `EconomicsRow` fields;
`ENGINE = ReplacingMergeTree` keyed on `(session_id, event_seq)` so a re-emit collapses to ONE row;
TTL per convention). Sink: `EconomicsSink(sink_fn, session_id)` mirrors `VioletDecisionJournal`:
validate each `EconomicsRow`, **dedupe by `(session_id, event_seq)`** (in-process guard + the
ReplacingMergeTree key as the durable guard), reject malformed to a counter (never head-of-line),
insert with `wait_for_async_insert=1` on the economics path. NO writes to any production table.
**Pass criteria.** Row-set == DDL columns (parity test, mirror the journal test); duplicate
`event_seq` → exactly ONE emitted (in-process) and ReplacingMergeTree key documented; malformed row
rejected to counter, sink never raises; provenance fields required. No edits outside the new files.
## TASK 45 — Provenance + per-trade PnL feedback (capital = anchor + Σ trade-realized)
**Why.** Capital must be anchor + Σ TRADE-REALIZED deltas (never capital snapshots / never WS
balance — the shared-account foreign-fill immunity); every economics event carries
`pnl_source`/`capital_source`.
**Affected files (NEW):** `prod/clean_arch/violet/obs/provenance.py`,
`prod/clean_arch/violet/obs/test_violet_provenance.py`. (composes PASS-4 `EconomicsLedger` +
PASS-5/6 `PositionDelta`.)
**Interface/approach.** `stamp_provenance(delta: PositionDelta, *, pnl_source: str, capital_source:
str) -> (EconomicsRow, Provenance)` — derive `realized_pnl` from the TRADE-REALIZED delta (NEVER a
capital-difference), feed the ledger (anchor + Σ), tag provenance. A guard/test asserts the feedback
path uses trade-realized PnL, not capital deltas.
**Pass criteria.** capital_after == anchor + Σ realized deltas; provenance present on every row; a
test proves capital is NOT derived from a balance snapshot; non-finite rejected. No edits outside the 2 files.
## TASK 46 — Decision/factor divergence monitor v2
**Why.** Extend the V1 divergence monitor to the FACTOR plane (boost/beta/mc_scale/esof/ob/dc/
posture VIOLET-vs-BLUE) so divergence is observable per factor, not just per feed. New module (do
NOT edit `divergence.py`).
**Affected files (NEW):** `prod/clickhouse/violet/31_violet_decision_divergence.sql` (DDL FIRST),
`prod/clean_arch/violet/obs/divergence_v2.py`, `prod/clean_arch/violet/obs/test_violet_divergence_v2.py`.
**Interface/approach.** `record_factor_divergence(violet_factors, blue_factors, *, asset, ts,
event_seq) -> list[DecisionDivergenceRow]` — per factor, abs_err; sink via a validate-then-sink to
`violet_decision_divergence`. Reuse the row-guard pattern. Read-only over inputs.
**Pass criteria.** Row-set == DDL columns; per-factor abs_err correct on a fixture; malformed
rejected. No edits to `divergence.py`. No edits outside the new files.
## TASK 47 — Soak-readiness aggregator (one dashboard from all reports)
**Why.** Consolidate the scattered reports (PASS-2 parity, PASS-3 slippage, PASS-5/6 exec gate,
PASS-7 rank parity, PASS-8 cadence/regime, the latency + determinism gates) into ONE
soak-readiness view, so go/no-go is a single artifact.
**Affected files (NEW):** `prod/clean_arch/violet/obs/soak_readiness.py`,
`prod/clean_arch/violet/obs/test_violet_soak_readiness.py`, report →
`prod/VIOLET_dev/reports/violet_soak_readiness_<UTC>.{json,md}`.
**Interface/approach.** Read the latest report file per category from `prod/VIOLET_dev/reports/`
(parse the JSONs), extract the key metric each (parity pick-match, latency p99, determinism pass,
mismatch counts, exactly-one-row, namespace isolation), assemble a `SoakReadiness` with a GO/NO_GO
verdict + the failing reasons. Pure aggregation; tolerate missing reports (mark UNKNOWN, not crash).
**Pass criteria.** On a synthetic set of report files the aggregator extracts the right metrics and
verdict (`--self-test`); a real run consolidates the actual reports; missing report → UNKNOWN not
crash. No edits outside the new files.
## TASK 48 — Soak-readiness @gate (go/no-go prerequisites)
**Why.** A single authoritative gate: a DARK soak is GO only when parity ≥ threshold, latency gate
green, determinism green, exactly-one-row proven, namespace isolation proven (0 violet rows in
`dolphin`/`dolphin_pink`).
**Affected files (NEW):** `prod/clean_arch/violet/obs/test_violet_soak_readiness_gate.py`; report →
`prod/VIOLET_dev/reports/violet_soak_gate_<UTC>.json`.
**Interface/approach.** Drive Task 47's aggregator; assert `verdict == "GO"` ONLY when all
prerequisites pass. **IMPORTANT:** given CRITICAL #1 (parity pick-match 1.5%), this gate is EXPECTED
to currently return NO_GO — the gate's job is to make that explicit and machine-checked, NOT to be
forced green. A test asserts the gate correctly returns NO_GO when parity is below threshold.
**Pass criteria (`@pytest.mark.gate`).** Gate returns GO iff all prerequisites green; returns NO_GO
with the parity reason when parity < threshold (the current real state); report archived. Do NOT
hardcode GO. No edits outside the new file.
---
## Composition map
```
contracts_v3 (+Provenance/EconomicsRow/DecisionDivergenceRow/SoakReadiness)
44 economics_sink ── DDL-first, exactly-one-row (ReplacingMergeTree key), event_seq, async-insert
45 provenance ── capital = anchor + Σ trade-realized; pnl_source/capital_source stamps
46 divergence_v2 ── per-factor VIOLET-vs-BLUE abs_err rows (DDL-first)
47 soak_readiness ── consolidate ALL reports → SoakReadiness(GO/NO_GO)
48 soak_gate ── GO iff parity≥thr ∧ latency ∧ determinism ∧ one-row ∧ isolation
(currently NO_GO by design — CRITICAL #1)
```
Integration (wiring the sinks into the live shadow loop, scheduling the aggregator) is the OWNER's
job. The soak GO decision is the operator's, informed by Task 48.
## Recommended order
**43 (contracts) 44 (economics sink, DDL-first) 45 (provenance) 46 (divergence v2) 47
(aggregator) 48 (gate)**.
## Still NOT in scope (operator/owner only, or Claude)
- **Starting the DARK soak** operator decision (Task 48 informs it; currently NO_GO).
- **The live parity root-cause (CRITICAL #1)** Claude; this pass only MEASURES + gates on it.
- **V4 live execution; HZ-bridge; VIBRISS.**
- Any edit to in-flight / shared files in §0 (especially `ch_writer.py` / production tables).

View File

@@ -0,0 +1,47 @@
# VIOLET Multi-Exchange OB Seam
Date: 2026-06-16
## Scope
This note describes the new read-only OB provider seam added on the VIOLET side:
- `prod/clean_arch/violet/venue_ob_provider.py`
- `prod/clean_arch/violet/test_violet_venue_ob_provider.py`
The seam exists so VIOLET can later consume a venue-specific OB feed without
changing BLUE or wiring any live exchange connection into the current shadow
path.
## What the seam does
- normalizes venue ticks into BLUE-shaped `OBSnapshot` records
- keeps the provider read-only and in-memory
- validates poison values at ingress with V-TYPES
- supports either an injected callable tick source or a preloaded buffer
- exposes the exact `OBProvider` interface that `OBFeatureEngine` expects
## What it does not do
- no live BingX connection
- no Hazelcast writes
- no BLUE code changes
- no live wiring into `live_blue_source.py` or `shadow_live_factors.py`
## Future adapter shape
A venue-specific live adapter can later be layered on top of this seam by
feeding `VenueOBTick` records into `VioletVenueOBProvider.refresh()` from any
normalized feed source. The only contract is the canonical `OBSnapshot` shape
that `OBFeatureEngine` already consumes.
## Validation
The companion test file checks:
- `VioletVenueOBProvider` implements the `OBProvider` surface
- malformed values are rejected at construction
- `OBFeatureEngine(provider)` can run `step_live()` and read back finite OB
features
- the callable-source path loads as expected

View File

@@ -0,0 +1,128 @@
# VIOLET — CRITICAL TODO / review queue (prominent)
Date: 2026-06-17. Single place for the must-not-forget VIOLET items. Review-later, not now.
---
## 🔴 CRITICAL #1 — VIOLET↔BLUE parity is VERY DISAPPOINTING (review + root-cause BEFORE any soak/V4)
Report: `prod/VIOLET_dev/reports/violet_parity_20260616_220412.md` (+ `.json`).
Window 2026-06-14 20:15 → 2026-06-15 21:00. Produced by PASS-2 Task 4 (`parity_report.py`).
**Headline:**
- Pick-match rate: **0.015 (1.5%)** — VIOLET rows 2853, exact pick matches only 43.
- Same-asset rate: **0.136 (13.6%)**; no-pick: **2465 / 2853 (86%)**.
- BLUE rows 25941 (scan_eval 25278 + trade_events 663).
**KEY NUANCE (where the review should START):** on the 43 rows that DID align, **sizing is
near-identical** — leverage abs error mean 0.016 / **median 0.0** / max 0.135. So the V3.4
sizing math (boost/beta/mc_scale/ob/esof/compose) is NOT the problem; the divergence is in
**ASSET SELECTION / TIMING / the comparison's ALIGNMENT method**. Candidate causes to
investigate (do not assume — measure):
1. **Apples-to-oranges population.** BLUE `scan_eval` (25278) is likely per-scan-per-asset
*evaluations*, while VIOLET rows are *actuated* decisions — the report may be comparing
different things. Verify the alignment/join semantics in `parity_report.py` first.
2. **Selection divergence.** VIOLET's `VioletAssetSelector` (IRP) vs BLUE's live selection over
the same scan stream — are they fed the same universe/lookback at the same scan index? The
sizing-gap samples (TRX/ATOM/LTC/XLM SHORT with large notional_rel_err) suggest VIOLET fires
on assets BLUE sized very differently or didn't pick.
3. **Cadence/actuation.** VIOLET actuates at Q=scan; if its scan alignment or dedupe differs,
picks land at different scans → counted as no-pick.
4. **The known structural items** (OB single-shot before V3.4d; mc_scale; live-factor sourcing)
— re-run parity AFTER V3.4d's persistent-OB launcher + the bit-identity fixes to see if the
number moves.
**Action:** full review of `parity_report.py` + a root-cause pass; fix the alignment OR the
selection divergence; re-run. This gates a meaningful DARK soak — a 1.5% pick-match makes the
soak uninterpretable. **Owner: Claude (me), later.**
---
## 🟡 #2 — Review the OA-delegated PASS work (NOT yet reviewed)
The parallel agent reports these DONE; none reviewed for correctness / BLUE-compliance yet.
**PASS 1** (`VIOLET_PART_SPEC_OA_TODO.md`):
- `53bdd90` sizing parity-pin tests
- `12b768b` venue OB provider seam (`venue_ob_provider.py`)
- `bae9284` base-fraction sizing study (+ archived report)
**PASS 2** (`VIOLET_PART_SPEC_OA_TODO_PASS2.md`):
- `parity_report.py` (Task 4 — the report above) + test
- `tradeability.py` (Task 6) + test
- `test_violet_v3_decision_latency_gate.py` (Task 5) → report `violet_v3_decision_latency_2026...`
- `test_violet_replay_determinism_gate.py` (Task 7) → report `violet_replay_determinism_2026...`
**PASS 3**: `VIOLET_PART_SPEC_OA_TODO_PASS3.md` (issued 2026-06-17) — venue feed / mechanical
exits / SL floor / event-restore / slippage / cadence; review when done.
**PASS 4**: `VIOLET_PART_SPEC_OA_TODO_PASS4.md` (issued 2026-06-17) — vol gate / V7 exit wrapper /
economics ledger / exec-intent / alpha data feed / time exits. **DONE on worktree
`agent/oa-violet4` (/mnt/vp-oa4)** — commits 6996a6a/7c8e524/bbebe58/3920665/f328d3a/b39d8a4.
Review when done. **Caveats to verify in review (flagged in-code as prominent banners + empty
marker commits 57ca851/8624dee/2aad0aa/539ea47):**
- **Task 17 (exec_intent):** `to_exec_intent` needs an EXPLICIT `reference_price` arg —
`ShadowDecision` has notional/exposure but NO price field, so qty can't be derived from a
decision alone. Integration owner threads the live price (PASS-3 `VenueTick`) at wiring time.
- **Task 18 (alpha_data_feed):** intentionally parse/feed-port ONLY — no live NT node / Binance
connection in unit code (NT owns its loop; dummy keys; separate feed process). Live node = the
integration owner's job. Confirm tests aren't vacuous (parse-fixture, not a live connection).
**PASS 5**: `VIOLET_PART_SPEC_OA_TODO_PASS5.md` (issued 2026-06-17) — MOCK-BINGX execution stack
(exec contracts + QuirkProfile seams / order FSM / mock venue / fill reducer / DARK E2E gate).
NOTE: BingX "quirks" are SEAM-ONLY here (default OFF) — a later quirk-injection pass + a mandatory
real-key boundary smoke are required before V4-live; review when done.
**PASS 6**: `VIOLET_PART_SPEC_OA_TODO_PASS6.md` (issued 2026-06-17) — execution INTERNALS
(fill-pump/ownership filter, reconcile/zero-wb guard, TTL-requote, orphan handling) + the
QUIRK-INJECTION gate that flips PASS-5's QuirkProfile flags ON and proves each handler neutralizes
the quirk. Mirrors PINK's production fixes (pink_direct.py). Real-key smoke still MANDATORY before
V4-live; review when done.
**PASS 7**: `VIOLET_PART_SPEC_OA_TODO_PASS7.md` (issued 2026-06-17) — V5 selection (faithful
ARS/IRP ranking + OB Sub-1), multi-asset slot manager, capital allocation, multi-asset flow, and a
ranking bit-identity gate vs AlphaAssetSelector. NOTE: this layer is the suspected locus of CRITICAL
#1 — PASS 7 PINS the ranking math (narrowing the suspect to timing/join), but the live-aggregate
root-cause stays Claude's job. Review when done.
**PASS 8**: `VIOLET_PART_SPEC_OA_TODO_PASS8.md` (issued 2026-06-17) — V6 bible CONSUMERS: posture
effects engine (5-state entry-gate/flatten/cap), MARAS fingerprint consumer (role TBD from code —
no invented modulation), regime read-model, cadence shadow-actuation telemetry, posture/regime
parity gate. VIOLET CONSUMES posture/MARAS (MHS owns them); ACB/vol/SL-guard already covered
elsewhere. Review when done.
**PASS 9**: `VIOLET_PART_SPEC_OA_TODO_PASS9.md` (issued 2026-06-17) — economics/observability
completeness: DDL-first exactly-one-row economics sink (PINK duplicate-emission fix), provenance +
capital=anchor+Σ-trade-realized, factor divergence v2, soak-readiness aggregator, and the
soak-readiness @gate (currently NO_GO by design — gated on CRITICAL #1). Review when done.
> **OA BACKLOG COMPLETE (PASS 19 issued).** PASS 19 carve out the full parallelizable DARK
> surface of the V0→V6 ladder (~48 independent units, contracts_v3-keyed). What remains is NOT
> parallelizable and is Claude's: the parity root-cause (CRITICAL #1), review of all passes,
> integration, E2E, VBT re-cert, the real-key V4 boundary smoke, and live execution.
**Action:** review each pass for correctness, BLUE-algo compliance, V-TYPES, no-shared-edits,
real (non-vacuous) tests. **Owner: Claude (me), later.**
---
## 🟢 #3 — Integration / "sprint" consolidation (my later work)
Once the passes are reviewed, I will: **(a)** review ALL passes together, **(b)** integrate them
(resolve interfaces, dedupe), **(c)** test them together, **(d)** plug into the operational
system, **(e)** E2E test.
**Nomenclature note (raised by operator):** a "pass" here = a batch of self-contained tasks
delegated to one agent — smaller than an Agile **sprint** (a time-boxed iteration, typ. 1-4
weeks, team-scoped, ending in a shippable increment, with planning/review/retro ceremonies).
In this project's existing usage, **"Sprint N" already maps to a V-stage bundle** (Sprint 1 =
V0+V1, Sprint 2 = V2, Sprint 3 = V3) — i.e. an epic/milestone. So the cleanest mapping:
**V-stage = sprint/epic; "pass" = a sub-sprint work-package / task-bundle within it.** Renaming
passes to "sprints" would over-claim scope; keep "pass"/"work-package", or call each an
"increment". Decide at integration time.
---
## Gating rule
The DARK soak (operator-held) and V4 are NOT meaningfully runnable until CRITICAL #1 is
root-caused — a 1.5% pick-match means the shadow is not yet tracking BLUE's decisions.