43 Commits

Author SHA1 Message Date
Codex
519565965e docs: Cubic→Linear translator fixes + Beads PASS tracker evaluation
- NEW_PINK_FORENSICS_DUAL_LEV_2026_SEARCH_RESULTS_SPEC.md:94 — 'cubic translator' → 'linear translator' (exchange mapping)
- VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md:19 — 'cubic translator' → 'linear translator'
- VIOLET_V3_FINDINGS.md:51 — 'cubic translator' → 'linear translator'
- BEADS_PASS_TRACKER_EVALUATION.md: New evaluation recommending ADOPT for PASS tracking

Per Fable: exchange leverage mapping is LINEAR (round_half_even); cubic is conviction sizer only.
2026-07-08 17:48:38 +02:00
Codex
5607bfcc2c skills: pi_wake_agent skill definition (YAML + JSON)
- pi_wake_agent.skill.yaml: Full skill definition with all capabilities, parameters, examples, error handling, testing
- pi_wake_agent.skill.json: OpenAI function calling compatible schema
2026-07-08 16:15:59 +02:00
Codex
722ead5384 docs: Add --dry-run to PI_WAKE_AGENT_TOOL.md 2026-07-08 15:26:15 +02:00
Codex
88eaf20363 pi_wake_agent.py: Add --dry-run flag
- --dry-run shows what would be done without executing
- Logs intended actions to log file
- Exits 0 without modifying crontab or sending wake messages
2026-07-08 15:18:36 +02:00
Codex
d9284b7b75 docs/tools: pi_wake_agent tool documentation + test suite
- prod/docs/PI_WAKE_AGENT_TOOL.md: complete usage documentation
- test_pi_wake_agent.py: 38 test cases
2026-07-08 14:21:18 +02:00
Codex
de561b88b1 tools: pi_wake_agent.py — Reusable multi-agent wake timer with self-cron/daemon/succession
- Modes: --install, --once, --daemon, --succession, --run, --remove, --list, --status, --validate
- Multi-session support (--session / --sessions)
- Non-blocking h5i bus messages (fire-and-forget)
- Self-cleaning succession mode (--succession --count N --interval X)
- 38 tests passing
2026-07-08 14:14:50 +02:00
Codex
2faa179957 pi_wake_agent.py: Add run_once and run_wake with non-blocking h5i bus 2026-07-08 13:28:05 +02:00
Codex
0a586da6af pi_wake_agent.py: Add succession mode, fix h5i non-blocking 2026-07-08 11:05:52 +02:00
Codex
9546ad6c7b docs: record shared memory formats and addressing 2026-07-04 19:58:47 +02:00
Codex
a098962eec docs(uv): testnet push — T10 Gate A/B run spec, T12 promotion bridge spec, operator arming checklist
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 15:14:38 +02:00
Codex
c725bae815 docs(uv): handover board-state update — T2P2 differ merged, T2 rerouted
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 14:46:14 +02:00
Codex
3b2b6987ce docs(uv): handover to successor integrator + SOA verdict + T8/T9 subspecs + spec C11 forensics
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 12:47:37 +02:00
Codex
81520af83c dita_v2: fix arithmetic in test_asex_account mixed-ops funding expectation (2x50x0.5=50, not 100)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 08:03:36 +02:00
Codex
da2d140b3d record live state: ch_writer poison-row/dead_letter quarantine hotfix + violet_decisions DDL entry
NO content authored here — this commits the long-uncommitted LIVE hotfix from
incident 2026-06-12 (bars_held UInt16 poison jammed 18M rows 1.5d): poison rows
retried individually after CH_POISON_ATTEMPTS, then quarantined to dead_letter;
includes the ids loop-variable bugfix. Uncommitted live code was one checkout
away from loss.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 01:17:03 +02:00
Codex
a7522bf8d1 dita_v2: reconcile SOA — vendored thread-safety hardening ∪ venue plane
3-way merge (base=pre-venue-plane upstream, ours=violet-main vendored hardening,
theirs=upstream venue plane): RLock atomic-snapshot wrapping (account.py,
real_control_plane, real_zinc_plane), lazy __getattr__ imports (__init__),
account-core test coverage — merged with venue_region telemetry. Zero conflicts;
all files AST-verified. Ends the two-way vendor drift found in
DITAV2_SOA_SURVEY_20260702 §5 / UV_DITAV2_SOA_VERDICT_20260703.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 01:15:17 +02:00
Codex
cfb3d7cf4f dita_v2: backport asex_account.py (from violet main) + test (from PASS9) to canonical upstream
Heals reverse vendor drift: the ASEx AccountProjectionV2 adapter was committed on
the vendored copy (violet.git main 824c5cf) instead of upstream, violating the
edit-upstream-then-sync doctrine. Test was orphaned on the PASS9 worktree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 01:00:40 +02:00
Codex
9bef1f6b01 dita_v2: VenueTelemetrySnapshot + Zinc venue plane (4th shm partition) + local Rust target dir
Upstream-canonical commit of the ~366 lines that sat uncommitted in the working
tree (surveyed in DITAV2_SOA_SURVEY_20260702). Venue telemetry published at every
venue boundary (submit/cancel/reconcile) over the zinc venue_region; SOA per
UV DITAv2 adjudication 2026-07-03.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-03 00:42:20 +02:00
Codex
1db4802ee8 docs: add VIOLET full system assessment 2026-06-27 14:08:55 +02:00
Codex
37748cf180 feat(violet): V4 arm-readiness reporter (PASS MM1)
Read-only aggregation module answering 'is V4 ready to ARM the live canary?'
by consuming gate outcomes into a typed StrictModel report.

Files:
- v4_readiness.py: V4ReadinessReport (6 bool gates + reasons) + assess_v4_readiness()
- test_violet_v4_readiness.py: 13 tests (happy, parametrized single-false,
  hypothesis 2^6 invariant, mutation litmus, StrictModel poison)

Co-Authored-By: mm_VIOLET1 <mm_VIOLET1@h5i>
2026-06-24 14:01:51 +02:00
Codex
0254e9a4c0 ```bash
cd /mnt/dolphinng5_predict
export H5I_AGENT=aider_nvnemo
h5i msg inbox
```

Co-authored-by: aider (openrouter/nvidia/nemotron-3-ultra-550b-a55b:free) <aider@aider.chat>
2026-06-24 12:28:17 +02:00
Codex
ca0abc0946 docs: add H5I messaging setup documentation 2026-06-24 12:24:09 +02:00
Codex
d652ff1651 Reference VIOLET DARK preclear report
Report path: prod/reports/violet_dark_clickhouse_preclear_20260618T135531Z.md

h5i object capture was skipped because h5i object ingestion was hanging; the report remains at the path above.
2026-06-18 19:36:24 +02:00
Codex
f8826f613d VIOLET: record PASS4-done + Task 17/18 caveats in the review queue (PR docs)
PASS 4 completed on worktree agent/oa-violet4 (/mnt/vp-oa4). Recorded in the review queue the two
caveats the OA agent reported — also flagged in-code as prominent banner comments + empty marker
commits on that branch:
  - Task 17 exec_intent: explicit reference_price required (ShadowDecision has notional, no price).
  - Task 18 alpha_data_feed: parse/feed-port only, no live NT/Binance in unit code.
For Claude's later PASS-4 review (verify the caveats + non-vacuous tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 12:25:38 +02:00
Codex
255722d6fd VIOLET OA: partial spec PASS 9 — economics + observability completeness (OA backlog complete)
VIOLET_PART_SPEC_OA_TODO_PASS9.md — the final parallelizable pass, discharging VIOLET's binding V1
economics/observability inheritance. Six independent units extending contracts_v3 (Provenance/
EconomicsRow/DecisionDivergenceRow/SoakReadiness):
  43. Observability/economics contracts.
  44. DDL-first exactly-one-row-per-event economics sink (ReplacingMergeTree key, event_seq,
      wait_for_async_insert=1, reject-at-source) — the PINK duplicate-emission fix.
  45. Provenance + per-trade PnL feedback (capital = anchor + Σ trade-realized; never a balance
      snapshot — shared-account foreign-fill immunity).
  46. Factor-plane divergence monitor v2 (per-factor VIOLET-vs-BLUE abs_err, DDL-first).
  47. Soak-readiness aggregator (consolidate all reports → SoakReadiness GO/NO_GO).
  48. Soak-readiness @gate — GO iff parity≥thr ∧ latency ∧ determinism ∧ one-row ∧ isolation;
      EXPECTED NO_GO today by design (CRITICAL #1), the gate makes it machine-checked (not forced green).

DDL-first + row-set==DDL parity + reject-at-source discipline throughout; never writes a production
table (new dolphin_violet tables only); ch_writer.py not edited.

OA BACKLOG COMPLETE: PASS 1–9 (~48 independent contracts_v3-keyed units) cover the full
parallelizable DARK surface of V0→V6. Remaining work is non-parallelizable + Claude's: parity
root-cause, review of all passes, integration, E2E, VBT re-cert, real-key V4 smoke, live execution.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:51:35 +02:00
Codex
16b2d18829 VIOLET OA: partial spec PASS 8 — V6 bible consumers (posture / MARAS / regime) + cadence telemetry
VIOLET_PART_SPEC_OA_TODO_PASS8.md — the V6 bible-layer remainder (ACB done; vol = PASS-4 §14;
sub-second SL guard = PASS-3 §10), six independent units extending contracts_v3 (Posture/
PostureEffect/MarasFingerprint/RegimeView/ActionCadenceDelta):
  37. Regime/posture contracts (Posture literal == sizing.Posture).
  38. Posture effects engine — 5-state entry-gate/flatten/size-cap, transcribed from the
      orchestrator (HIBERNATE_HALT :365, STALKER 2.0 :613, TURTLE regime_dd_halt).
  39. MARAS fingerprint consumer — role determined FROM CODE (modulating vs observability-only);
      no invented effect.
  40. Regime composite read-model (RegimeView; observability/divergence, NOT a sizing path).
  41. Cadence shadow-actuation telemetry harness — evaluate-at-fastest / actuate-at-Q deltas (the
      shadow evidence required before any Q loosening).
  42. Posture/regime parity @gate vs BLUE (bit-identity of gating).

Framing: VIOLET CONSUMES posture/MARAS (BLUE's MHS owns them) — these units APPLY published
effects, they do not run the meta-health state machine. Faithful to the kernel; cite file:line; the
STALKER size-cap stays in sizing.py (this pass owns the entry-gate + observability).

This is the last bible-layer pass; PASS 9 (economics/observability completeness) remains. Added
PASS 8 to the review queue in VIOLET_TODO_CRITICAL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:32:53 +02:00
Codex
0ab83c528a VIOLET OA: partial spec PASS 7 — V5 selection + multi-asset slots + capital allocation
VIOLET_PART_SPEC_OA_TODO_PASS7.md — the V5 ladder layer as six independent units extending
contracts_v3 (AssetRank/SlotState/CapitalAllocation/SlotPolicy):
  31. Selection contracts + SlotPolicy.
  32. ARS ranking pipeline (faithful to AlphaAssetSelector.rank_assets + OB Sub-1 ±5/10% + IRP
      alignment filter + stablecoin exclusion).
  33. Multi-asset slot manager (max_slots, assignment by rank, hysteresis anti-churn, no double-occ).
  34. Capital allocation across slots (BLUE notional = base_fraction × conviction × capital; no
      invented normalization).
  35. Selection→slot→intent multi-asset DARK flow harness.
  36. Ranking bit-identity @gate vs the real AlphaAssetSelector.

NOTE (CRITICAL #1): the disappointing live parity (pick-match 1.5%, no-pick 86%, sizing near-
identical when aligned) implicates THIS selection/timing layer. PASS 7 PINS the ranking math
bit-for-bit (Task 36), which narrows the live-parity suspect to timing/join — but the
live-aggregate root-cause stays Claude's job (not this pass). Faithful to AlphaAssetSelector /
BIBLE §5; cite kernel file:line. New-file-only under selection/.

Standing ready for PASS 8–9. Added PASS 7 to the review queue in VIOLET_TODO_CRITICAL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:53:12 +02:00
Codex
5f636ef723 VIOLET OA: partial spec PASS 6 — execution internals + quirk-injection gate
VIOLET_PART_SPEC_OA_TODO_PASS6.md — the execution internals on top of PASS-5's mock, plus the
quirk-injection layer that flips PASS-5's QuirkProfile seams ON and proves each handler survives.
Six independent units extending contracts_v3 (AccountFrame/PositionFrame/ReconcileCorrection/
OwnershipPolicy):
  25. Account/position frame contracts + ownership predicate (foreign-fill filter basis).
  26. Fill-pump (ownership filter [quirk #2], dedup, idempotent) → ledger deltas.
  27. Reconcile loop (zero-wb guard [#1], reseed-on-update, settle/funding event_seq order [#5]).
  28. TTL requote / cancel-replace (+ setLeverage ordering [#7]) over PASS-5 venue + V2 driver.
  29. Orphan detection (ORPHAN_VENUE adopt, ORPHAN_LOCAL quarantine — never silent flatten/resurrect).
  30. Quirk-injection suite + @gate: flip every QuirkProfile flag ON, assert each is neutralized.

VIOLET's exec reconcile/quirk-handling mirrors PINK's production-tested logic (pink_direct.py is
the authoritative reference — transcribe + cite). New-file-only: quirk injection plugs into PASS-5's
existing seams via NEW fill/frame models, never editing the PASS-5 mock. Real-key boundary smoke
remains MANDATORY before V4-live (injection proves logic, not undocumented venue reality).

Standing ready for PASS 7–9. Added PASS 6 to the review queue in VIOLET_TODO_CRITICAL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:37:32 +02:00
Codex
28a2b90b1d VIOLET OA: partial spec PASS 5 — mock-BingX execution stack (quirks seam-only, deferred)
VIOLET_PART_SPEC_OA_TODO_PASS5.md — the DARK mock-BingX execution stack so the full order
lifecycle is buildable/testable with zero keys/risk (V4-live later = swap mock for real client).
Five independent units extending contracts_v3 (Order/OrderAck/Fill/OrderStatus/PositionDelta):
  20. Execution contracts + QuirkProfile seam registry (no quirk logic).
  21. Mock BingX order FSM (legal lifecycle, partials, reject, reduce-only-increase rejection).
  22. Mock BingX venue adapter (port-conformant, injectable fill model, quirk hooks INERT by default).
  23. Fill→position/PnL reducer (PnL from fill_price ONLY; ×leverage-aware) → PositionDelta.
  24. DARK E2E gate: ExecIntent → router/driver → mock venue → fill reducer → PASS-4 ledger.

CRITICAL per operator: a prominent ⚠️ section makes explicit that BingX "quirks" (zero-wb WS
frames, ownership/foreign-fill collision, bound-price poison, ×leverage notional, settle desync,
reduce-only edge, setLeverage race, dead .pro TLS) are EXPLICITLY OUT OF SCOPE for PASS 5 — the
mock carries injection SEAMS (QuirkProfile, default OFF) to accommodate them LATER, and a real-key
boundary smoke remains MANDATORY before V4-live. Quirks sourced from PINK orphan/reconcile +
DITAv2 audit memory.

Standing ready to write PASS 6–9 (exec internals/reconcile, V5 selection+slots, V6 bible
consumers, economics/observability). Added PASS 5 to the review queue in VIOLET_TODO_CRITICAL.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:26:15 +02:00
Codex
80ce1a1afe VIOLET OA: partial spec PASS 4 — next plan units (entry gate, exits, accounting, exec-intent, feed)
VIOLET_PART_SPEC_OA_TODO_PASS4.md — six plan-derived independent units continuing from PASS 3,
extending the shared contracts_v3 vocabulary (adds ExecIntent / CapitalState / VolGateResult /
AlphaBar):
  14. Volatility regime gate (faithful vol_ok — the entry precondition).
  15. AlphaExitEngineV7 wrapper (V3a deferred — wrap BLUE's exit engine as-is, discretionary pri 2).
  16. Capital/economics provenance ledger (capital = anchor + Σ deltas, event_seq, exactly-one-row).
  17. Execution-intent emitter, DARK (V4 boundary — ShadowDecision → ExecIntent, log-only sink).
  18. Alpha-side reactor data feed (V2 NT Binance DATA spike → AlphaDataFeedPort, data-only).
  19. MAX_HOLD + MEAN_REVERSION scan-driven exit timers.
Each ships + tests standalone; composition map + recommended order (14→16→15→19→17→18) + the same
HARD RULES + commit/branch policy. VIBRISS/adaptive-TP explicitly excluded.

Also: added PASS 3 + PASS 4 to the review queue in VIOLET_TODO_CRITICAL.md §2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 09:27:23 +02:00
Codex
9c068619e1 VIOLET: CRITICAL parity-disappointment TODO + OA partial spec PASS 3
VIOLET_TODO_CRITICAL.md (prominent review queue):
- 🔴 CRITICAL #1: VIOLET↔BLUE parity report (PASS-2 Task 4, violet_parity_20260616_220412.md)
  is VERY DISAPPOINTING — pick-match 1.5%, no-pick 86%, same-asset 13.6%. KEY NUANCE: on the 43
  aligned rows sizing is near-identical (lev_err median 0.0) → the divergence is SELECTION /
  TIMING / comparison-ALIGNMENT, NOT the sizing math. Root-cause before any soak/V4. (Claude.)
- #2: OA PASS-1 + PASS-2 reported done, NOT yet reviewed — review queue listed.
- #3: future integration/E2E + the pass-vs-sprint nomenclature note (pass = sub-sprint
  work-package; V-stage = sprint/epic, matching existing usage).

VIOLET_PART_SPEC_OA_TODO_PASS3.md (lengthy, plan-derived, independent units with shared
interfaces — nothing invented): contracts_v3 (VenueTick/OpenPositionView/ExitDecision) +
VenuePriceFeedPort+BingX adapter, MechanicalExitGuard (armed TP/SL), sub-second SL/ADVSL floor
guard, event-sourced restore from trade_reconstruction, venue lead/lag slippage metric, cadence
per-action Q schedule + telemetry. Each ships + tests standalone; composition map + order given.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 09:01:49 +02:00
Codex
8babce893c VIOLET OA: partial spec PASS 2 — spec-advancing self-contained tasks
VIOLET_PART_SPEC_OA_TODO_PASS2.md — four new-file-only tasks that advance the main V0→V6 ladder
toward the DARK-soak gate and V4, each painstakingly specified (why/spec-link, exact new files,
approach, tests, pass criteria), under the same HARD RULES + COMMIT/BRANCH policy as PASS 1:
  4. Live BLUE↔VIOLET decision parity report (V3d → soak-readiness gate; reads CH, no writes).
  5. Full-DecisionEngine reactor latency gate (V0/V3 budget proof with the real 5-factor brain).
  6. L3 tradeability projector standalone (V4 prep; ShadowDecision → exchange-leverage projection,
     DARK, not wired to venue/journal).
  7. Deterministic replay + golden gate (V3d determinism criterion; same scans ⇒ same decisions).
Independent/parallelizable; recommended order 5→7→4→6. None need the operator's soak/keys
greenlight. PASS-1 tasks reported done — review deferred (noted).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 23:44:30 +02:00
Codex
bae9284582 VIOLET OA: base-fraction sizing study\n\nAdd the read-only CH-backed base-fraction study script plus the final archived report. 2026-06-16 23:25:38 +02:00
Codex
12b768bc4f VIOLET OA: add venue OB provider seam
Co-authored-by: Codex <codex@openai.com>
2026-06-16 23:01:17 +02:00
Codex
53bdd908f4 VIOLET OA: add sizing parity pin tests
Co-authored-by: Codex <codex@openai.com>
2026-06-16 22:51:07 +02:00
Codex
f0a73491bd VIOLET OA: partial spec for a parallel agent (3 self-contained, new-file-only tasks)
VIOLET_PART_SPEC_OA_TODO.md carves the self-contained pending items from the VIOLET plan so a
separate agent can work without colliding with in-flight files:
  1. Parity-pin tests for sizing.py's hand-replicated arithmetic (regime/ob/strength/dc) vs the
     real NDAlphaEngine methods — test-only.
  2. Multi-exchange OB provider seam (VioletVenueOBProvider scaffold + interface + design note)
     — the BingX/3rd-party swap point, NOT wired live.
  3. Base-fraction sizing study (analysis script + report) — pure read-only analysis, no
     behavior change.
Each task: exact affected (new-only) files, V-TYPES + DARK + never-touch-shared rules, required
tests + pass criteria. Includes an explicit COMMIT/BRANCH policy (own git worktree, or partial
commits with `git commit -- <paths>`) prompted by today's three shared-index collisions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 21:54:02 +02:00
Codex
0f3d7650c3 VIOLET V3.4d: launcher OB-persistence + boost/beta prior; OB-feed/HZ-bridge/coordination doc
Launcher shadow path now keeps the persistent state BLUE keeps across scans (faithful, not
per-scan single-shot): ONE OBFeatureEngine wired lazily on first assets and kept for the
service lifetime, reading BLUE's EXTANT published OBF feed via HZOBProvider (read-only HZ
entry-listener cache — NO new OB storage; provider is the swap seam for a future direct
BingX/3rd-party OB stream); per-scan ob_bar_idx into step_live/get_market; and prior_boost_beta
carried across scans (stale exf keeps the prior). shadow_decision_step threads ob_engine +
bar_idx + prior into source_live_blue_sizing_factors; build_shadow_live_source seeds the state
+ a default ob_engine_factory (injectable for tests).

TODO_HZBRIDGE markers added at all 3 VIOLET->Hazelcast touch points: per operator the upcoming
dolphinng5_predict/hzbridge must become the sanctioned HZ connection (silent client-death /
lockup / dropout mitigation) — refactor ASAP once it ships.

Doc VIOLET_OB_FEED_AND_AGENT_COORDINATION.md: OB-feed sourcing doctrine, HZ-bridge TODO, and
the multi-agent worktree + doctrine/release/status scheme (prompted by the shared-index
incidents where agents' staged files cross-contaminate commits).

6 launcher tests (+wired-once/bar_idx-increments, +prior-carries). violet-only; partial commit
(only my 4 paths) to avoid sweeping a concurrent agent's staged files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 21:44:03 +02:00
Codex
2f5ce55967 M5: implement real Iceoryx2Sink — iceoryx2 0.9.1 production plane sink
Replaces the stub that returned Err(Unimplemented) with a working iceoryx2
publish-subscribe sink. Key design:

- One iceoryx2 pub/sub service per configured stream at {prefix}/{stream}
- Fixed-size [u8] samples with in-band binary header (seq, src_ts_us, ts_us,
  NUL-padded key(48), len(32), value(N)) — no unsafe repr(packed) needed
- Heartbeat stream at {prefix}/hzbridge_heartbeat with same layout
- Uses BackpressureStrategy::DiscardData to avoid blocking the feeder
- Uses ipc_threadsafe::Service so the Publisher is Send (ipc::Service uses
  SingleThreaded sync which is !Send)
- Daemon builds Iceoryx2Sink by default; --allow-null-sink for dev use only

Constraint compliance:
  C1 — no unsafe (crate-root forbid(unsafe_code) enforced by compiler)
  C2 — no async (all iceoryx2 calls synchronous)
  C5 — iceoryx2 0.9.1 already pinned per ADR-001
  C8 — fixed-size samples, pre-allocated publishers, bounded loan slots

Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
2026-06-16 21:09:02 +02:00
Codex
901ea1046f M5: implement real Iceoryx2Sink — iceoryx2 0.9.1 production plane sink
Replaces the stub that returned Err(Unimplemented) with a working iceoryx2
publish-subscribe sink. Key design:

- One iceoryx2 pub/sub service per configured stream at {prefix}/{stream}
- Fixed-size [u8] samples with in-band binary header (seq, src_ts_us, ts_us,
  NUL-padded key(48), len(32), value(N)) — no unsafe repr(packed) needed
- Heartbeat stream at {prefix}/hzbridge_heartbeat with same layout
- Uses BackpressureStrategy::DiscardData to avoid blocking the feeder
- Uses ipc_threadsafe::Service so the Publisher is Send (ipc::Service uses
  SingleThreaded sync which is !Send)
- Daemon builds Iceoryx2Sink by default; --allow-null-sink for dev use only

Constraint compliance:
  C1 — no unsafe (crate-root forbid(unsafe_code) enforced by compiler)
  C2 — no async (all iceoryx2 calls synchronous)
  C5 — iceoryx2 0.9.1 already pinned per ADR-001
  C8 — fixed-size samples, pre-allocated publishers, bounded loan slots

Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
2026-06-16 21:08:16 +02:00
Codex
262cada664 VIOLET V3.4c: ACB boost/beta — keep-prior on stale exf, decision logging, semantics doc
Per operator: VIOLET must keep a "prior" on stale ExF rather than collapse to neutral, and
the ACB's role must be commented-in + logged.

- KEEP-PRIOR on stale/missing exf: BLUE keeps _day_base_boost/_day_beta on the stale branch
  (update_acb_boost isn't called). _source_boost_beta now takes `prior` (boost,beta) and
  returns it on ValueError / no-exf, seeded by the first successful compute. Threaded as
  `prior_boost_beta` through source_live_blue_sizing_factors (caller persists last good).
- LOGGING: violet.live_blue_source logger emits live boost/beta + the full sourced plane.
- SEMANTICS: documents the ACB in effect (inverse v6) and CONFIRMS ACB IS in the sizing layer
  via _day_base_boost/_day_beta → _update_regime_size_mult (esf_alpha_orchestrator.py:771-772,
  898-909). OB Sub-4 beta modulation dormant in live — per operator found NON-PERFORMANT and
  deliberately bypassed; TODO_SOMEDAY in-code to find documentary confirmation.

37 tests. violet-only; no shared-file edits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 18:22:29 +02:00
Codex
fac287d678 VIOLET V3.4c: make boost/beta, signal-gen, OB bit-identical to BLUE (+ exhaustive tests)
Operator directive: VIOLET must do IDENTICALLY what BLUE does for the three parity
flags — approximation cannot guarantee bit-for-bit functioning. Reworked
live_blue_source.py to call BLUE's OWN code paths, not reconstruct/substitute them.

boost/beta — was reading the published DOLPHIN_FEATURES.acb_boost scalar
(acb_processor_service's daily value). BLUE's trader does NOT use that for sizing; it
recomputes live via acb.get_dynamic_boost_from_hz(exf_latest, w750_velocity, direction)
with a bare AdaptiveCircuitBreaker() and NO ob_engine (nautilus_event_trader.py
on_exf_update:4769 / rollover prewarm:2710). New _source_boost_beta replicates that call
exactly (reads exf_latest + latest_eigen_scan.w750_velocity; 0.0→None like BLUE; on stale
exf ValueError → neutral, mirroring BLUE's "ACB Stale Data Fallback"). The published
acb_boost is never read. Test pins bit-identity against the real ACB.

signal-gen (dc_status) — was AlphaSignalGenerator() bare defaults; coincidentally equal to
BLUE today, but BLUE builds it from ENGINE_KWARGS (trader:128-133, threaded at
esf_alpha_orchestrator.py:180-191), so a champion retune would silently diverge. Now
constructed with BLUE_SIGNAL_GEN_KWARGS (vel_div_* imported from the kernel constants).
Test parses ENGINE_KWARGS from the trader source and asserts each param matches — drift
becomes a red test, not a silent miss.

OB — was a reinvented HazelcastOBProvider reading asset_*_ob with custom parsing. Now uses
BLUE's OWN HZOBProvider + OBFeatureEngine, wired exactly as _wire_obf
(nautilus_event_trader.py:4967-4980): step_live(assets, bar_idx) then get_market. Engine
is injectable + persistent so OB accumulation matches BLUE across scans (caller owns it).

Deleted: HazelcastOBProvider, _extract_acb, the status-label mc path. mc_scale fix
(begin_day cat/env thresholds) retained. Module docstring + structural-divergence doc
updated: all three flags FIXED; only _derive_mc_scale remains hand-replicated (pinned by
formula test). OPEN follow-up: launcher shadow_decision_step should pass a persistent
ob_engine + bar_idx for cross-scan OB history.

34 tests (33 + live-HZ smoke deselected): boost/beta-vs-ACB bit-identity (incl. w750=0→None,
no-exf, stale ValueError, ignores acb_boost), signal-gen ENGINE_KWARGS pin (parametrized),
OB wiring (step_live call, HZ coords, neutral paths), mc_scale formula, sequence dc/selector
parity, anomaly handling. violet-only; no shared-file edits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 17:33:01 +02:00
Codex
1415a65670 VIOLET V3.4c review: fix mc_scale BLUE-parity bug + document re-derivation debt
REVIEW of the V3.4c/d/e work (other agents) for BLUE-algo compliance found one
real bug and one architectural concern.

BUG FIXED — mc_scale derivation (live_blue_source.py):
The adapter mapped `mc_forewarner_latest.status == "ORANGE"` → 0.5. But BLUE's sizing
mc_scale is NOT the MC service's `status` label (that label, from mc_forewarner_flow.py,
uses GREEN<0.10/ORANGE<0.30/RED — observability only). The live trader re-derives the
haircut in begin_day (esf_alpha_orchestrator.py:956-962) from the SAME published fields
`catastrophic_prob` + `envelope_score` with DIFFERENT thresholds:
    mc_red    = cat>0.25 or env<-1.0
    mc_orange = (not mc_red) and (env<0 or cat>0.10)
    mc_scale  = 0.5 if mc_orange else 1.0
The two disagree (e.g. cat=0.05/env=-0.5 → label GREEN→1.0 but BLUE orange→0.5;
cat=0.28 → label ORANGE→0.5 but BLUE red→1.0). Rewrote `_derive_mc_scale` to mirror
begin_day exactly on the source fields. Per operator: there is genuine ambiguity over
which surface is "nominal" — we go with the SOURCE FIELDS + begin_day formula (the path
that actually drives BLUE's sizing); the ambiguity + decision are noted in the docstring.

TESTS — the old fixtures were complicit: they fed `{"status":"ORANGE"}`, a payload shape
BLUE never emits, so they "passed" against fiction. Replaced with BLUE's real payload
(`catastrophic_prob`/`envelope_score`) and added a parametrized formula test covering the
exact divergence cases the status-based code got wrong, plus missing/garbage-field neutral.
15 passed (live-HZ smoke deselected).

ARCHITECTURAL DEBT documented — VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md:
VIOLET imitates BLUE's computations in a DIFFERENT module/scope structure. Kernels are
WRAPPED (safe), but orchestration arithmetic (compose/regime/ob/strength/mc_scale) is
HAND-REPLICATED out of the monolithic NDAlphaEngine — making orderly, verifiable BLUE↔VIOLET
parity comparison and refactoring hard. Doc inventories every re-derivation with BLUE's
file:line authority, the drift risk, and mitigations (parity-pin tests; single canonical
surface per ambiguous factor; eventual DITAv2 Rust-backplane convergence). Pointer added to
live_blue_source.py's module docstring.

Also reviewed (no change needed): launcher launch_dolphin_violet.py is DARK-safe
(ObserveOnlyVenue + shadow gated default-OFF, no order path); shadow_live_factors.py glue
is correct; trade_slot_compare.py reimplements no BLUE algo. fb34431's launcher tests —
which the authoring agent never ran on the slow mount — pass (14 green).

violet-only; no shared-file edits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 16:54:35 +02:00
Codex
520d911722 DOCS: expand recent Violet touched files inventory 2026-06-16 15:55:09 +02:00
Codex
fe56ef522e DOCS: add recent Violet touched files list 2026-06-16 15:47:35 +02:00
64 changed files with 781402 additions and 593 deletions

View File

@@ -0,0 +1,87 @@
use hzbridge::config::Config;
use hzbridge::sink::PlaneSink;
use std::path::PathBuf;
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("hzbridge=info"))
.format_timestamp_micros()
.init();
let arguments = match parse_arguments() {
Ok(a) => a,
Err(detail) => {
eprintln!("hzbridged: {detail}");
std::process::exit(1);
}
};
let config = match Config::from_path(&arguments.config_path) {
Ok(c) => c,
Err(error) => {
eprintln!("hzbridged: {error}");
std::process::exit(1);
}
};
let sink: Box<dyn PlaneSink> = build_sink(&config, arguments.allow_null_sink);
log::info!("fsm=Boot msg=starting name=hzbridge");
let bridge = hzbridge::Bridge::new(config, sink);
if let Err(error) = bridge.run() {
hzbridge::blackbox::abort(&error.to_string());
}
}
#[cfg(feature = "iceoryx2")]
fn build_sink(config: &Config, allow_null_sink: bool) -> Box<dyn PlaneSink> {
if allow_null_sink {
log::warn!("fsm=Boot msg=development_null_sink_enabled");
return Box::new(hzbridge::sink::NullSink);
}
match hzbridge::sink_iceoryx2::Iceoryx2Sink::new(config) {
Ok(sink) => {
log::info!("fsm=Boot msg=iceoryx2_sink_ready");
Box::new(sink)
}
Err(error) => {
log::error!("fsm=Boot msg=iceoryx2_sink_failed error={error}");
eprintln!("hzbridged: iceoryx2 sink failed: {error}");
std::process::exit(1);
}
}
}
#[cfg(not(feature = "iceoryx2"))]
fn build_sink(_config: &Config, allow_null_sink: bool) -> Box<dyn PlaneSink> {
if !allow_null_sink {
eprintln!(
"hzbridged: iceoryx2 feature disabled and no --allow-null-sink; \
use --allow-null-sink only for wire/runtime smoke tests"
);
std::process::exit(1);
}
log::warn!("fsm=Boot msg=development_null_sink_enabled (iceoryx2 feature not available)");
Box::new(hzbridge::sink::NullSink)
}
struct Arguments {
config_path: PathBuf,
allow_null_sink: bool,
}
fn parse_arguments() -> Result<Arguments, String> {
let args: Vec<_> = std::env::args_os().skip(1).collect();
match args.as_slice() {
[flag, path] if flag == "--config" => Ok(Arguments {
config_path: path.into(),
allow_null_sink: false,
}),
[flag, path, allow] if flag == "--config" && allow == "--allow-null-sink" => {
Ok(Arguments {
config_path: path.into(),
allow_null_sink: true,
})
}
_ => Err("usage: hzbridged --config <path> [--allow-null-sink]".to_owned()),
}
}

View File

@@ -0,0 +1,336 @@
//! M5: Real iceoryx2 production sink.
//!
//! One publish-subscribe service per configured stream plus a dedicated
//! heartbeat stream. All samples are fixed-size `[u8]` slices with an
//! in-band header so consumers can read key/value directly from shm.
//!
//! Binary layout per sample (all values little-endian):
//! [0..8) seq u64 — monotonically increasing per-stream
//! [8..16) src_ts_us u64 — upstream Hazelcast event timestamp (µs epoch)
//! [16..24) ts_us u64 — local publish timestamp (µs epoch)
//! [24..72) key [u8; 48] — NUL-padded map key (UTF-8)
//! [72..76) len u32 — actual value length (≤ max_value_kib × 1024)
//! [76..) value [u8; MAX_VALUE] — opaque value bytes
//!
//! Heartbeat uses the same layout with seq=0 and key="heartbeat".
//!
//! Constraint compliance:
//! C1 — `#![forbid(unsafe_code)]` at crate root: no unsafe here.
//! C2 — no async: iceoryx2 API is synchronous.
//! C5 — iceoryx2 0.9.1 already pinned in Cargo.toml.
//! C8 — fixed-size samples, pre-allocated publishers.
use crate::config::Config;
use crate::metrics::MetricsSnapshot;
use crate::sink::{PlaneSink, SinkError};
use iceoryx2::prelude::*;
/// Alias for thread-safe IPC service type.
use log::{info, warn};
use std::collections::HashMap;
// ── Binary layout constants ────────────────────────────────────────────────
/// Byte offset at which the value payload begins.
const HEADER_BYTES: usize = 76; // seq(8) + src_ts_us(8) + ts_us(8) + key(48) + len(4)
const KEY_LEN: usize = 48;
// iceoryx2 Publisher<Service=ipc_threadsafe::Service, Payload=[u8], UserHeader=()>
type PublisherIpc = iceoryx2::port::publisher::Publisher<ipc_threadsafe::Service, [u8], ()>;
// ── Per-stream publisher state ──────────────────────────────────────────────
struct StreamPublisher {
/// The iceoryx2 publisher port.
publisher: PublisherIpc,
/// Total sample size = HEADER_BYTES + max_value_bytes.
sample_size: usize,
/// Monotonically increasing per-stream sequence number.
seq: u64,
}
// ── Iceoryx2Sink ────────────────────────────────────────────────────────────
/// The production `PlaneSink` that publishes into iceoryx2 services.
///
/// Keyed by the **short stream name** from config (e.g. `"eigen_scan"`).
/// The full iceoryx2 service name is `{prefix}/{short_name}`.
pub struct Iceoryx2Sink {
/// Map from short stream name → per-stream publisher.
streams: HashMap<String, StreamPublisher>,
/// Heartbeat publisher (best-effort; `None` if creation failed).
heartbeat: Option<StreamPublisher>,
/// The iceoryx2 node — held for the sink's lifetime.
_node: iceoryx2::node::Node<ipc_threadsafe::Service>,
/// Cached from config for use in `publish_and_notify`.
max_value_bytes: usize,
}
type NodeIpc = iceoryx2::node::Node<ipc_threadsafe::Service>;
impl Iceoryx2Sink {
/// Create the sink, opening or creating iceoryx2 services for every
/// configured stream plus the heartbeat stream.
///
/// Returns `SinkError` if any iceoryx2 service cannot be created or
/// opened. Per §5.2 "local shm failure is a broken box" — the caller
/// (the runtime) treats this as Fatal.
pub fn new(config: &Config) -> Result<Self, SinkError> {
let max_value_bytes = config.plane.max_value_kib * 1024;
let sample_size = HEADER_BYTES + max_value_bytes;
let prefix = &config.plane.service_prefix;
let node_name = NodeName::new(&format!("hzbridge_{}", config.hz.cluster))
.map_err(|e| SinkError::new(format!("node name: {e}")))?;
let node = NodeBuilder::new()
.name(&node_name)
.create::<ipc_threadsafe::Service>()
.map_err(|e| SinkError::new(format!("node create: {e:?}")))?;
let mut streams = HashMap::new();
for sc in &config.streams {
let full_name = format!("{}/{}", prefix, sc.stream);
let pub_ = Self::open_service(&node, &full_name, sample_size)?;
info!(
"sink=Iceoryx2Sink msg=stream_ready stream={} sample_size={}",
full_name, sample_size,
);
streams.insert(sc.stream.clone(), pub_);
}
// Heartbeat — best-effort; a failure here does not abort the bridge.
let hb_name = format!("{}/hzbridge_heartbeat", prefix);
let heartbeat = match Self::open_service(&node, &hb_name, sample_size) {
Ok(p) => {
info!("sink=Iceoryx2Sink msg=heartbeat_ready stream={}", hb_name);
Some(p)
}
Err(e) => {
warn!("sink=Iceoryx2Sink msg=heartbeat_failed error={e}");
None
}
};
Ok(Self {
streams,
heartbeat,
_node: node,
max_value_bytes,
})
}
/// Create or open one publish-subscribe service and return a publisher.
fn open_service(
node: &NodeIpc,
service_name: &str,
sample_size: usize,
) -> Result<StreamPublisher, SinkError> {
let name = ServiceName::new(service_name)
.map_err(|e| SinkError::new(format!("service name '{service_name}': {e}")))?;
let service = node
.service_builder(&name)
.publish_subscribe::<[u8]>()
.open_or_create()
.map_err(|e| SinkError::new(format!("open_or_create '{service_name}': {e:?}")))?;
let publisher = service
.publisher_builder()
.initial_max_slice_len(sample_size)
.max_loaned_samples(2)
.backpressure_strategy(BackpressureStrategy::DiscardData)
.create()
.map_err(|e| SinkError::new(format!("publisher '{service_name}': {e:?}")))?;
Ok(StreamPublisher {
publisher,
sample_size,
seq: 0,
})
}
/// Write a value into a loaned iceoryx2 sample and send it.
fn write_sample(
pub_: &mut StreamPublisher,
key: &str,
value_utf8: &[u8],
src_ts_us: u64,
ts_us: u64,
max_value_bytes: usize,
) -> Result<(), SinkError> {
pub_.seq = pub_.seq.wrapping_add(1);
let mut sample = pub_
.publisher
.loan_slice(pub_.sample_size)
.map_err(|e| SinkError::new(format!("loan_slice: {e:?}")))?;
let buf = sample.payload_mut();
let seq = pub_.seq;
// seq
buf[0..8].copy_from_slice(&seq.to_le_bytes());
// src_ts_us
buf[8..16].copy_from_slice(&src_ts_us.to_le_bytes());
// ts_us
buf[16..24].copy_from_slice(&ts_us.to_le_bytes());
// key (NUL-padded, truncated to KEY_LEN)
let key_bytes = key.as_bytes();
let copy_len = key_bytes.len().min(KEY_LEN);
buf[24..24 + copy_len].copy_from_slice(&key_bytes[..copy_len]);
if copy_len < KEY_LEN {
buf[24 + copy_len..72].fill(0);
}
// len
let value_len = value_utf8.len().min(max_value_bytes);
buf[72..76].copy_from_slice(&(value_len as u32).to_le_bytes());
// value
let vs = HEADER_BYTES;
buf[vs..vs + value_len].copy_from_slice(&value_utf8[..value_len]);
// Zero remaining value buffer
if value_len < max_value_bytes {
buf[vs + value_len..vs + max_value_bytes].fill(0);
}
sample.send().map_err(|e| SinkError::new(format!("send: {e:?}")))?;
Ok(())
}
fn now_us() -> u64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or(std::time::Duration::ZERO)
.as_micros() as u64
}
/// Encode a `MetricsSnapshot` as a space-separated key=value text blob.
fn encode_heartbeat(snapshot: &MetricsSnapshot) -> Vec<u8> {
format!(
"events_rx={} events_published={} decode_skips={} late_responses={} \
reconnects={} fragments_reassembled={} warmups={} pings_ok={} \
pings_missed={} events_dropped={} uptime_s={} state={} degraded={}",
snapshot.events_rx,
snapshot.events_published,
snapshot.decode_skips,
snapshot.late_responses,
snapshot.reconnects,
snapshot.fragments_reassembled,
snapshot.warmups,
snapshot.pings_ok,
snapshot.pings_missed,
snapshot.events_dropped,
snapshot.uptime_s,
snapshot.state,
snapshot.degraded,
)
.into_bytes()
}
}
impl PlaneSink for Iceoryx2Sink {
fn publish(
&mut self,
stream: &str,
key: &str,
value_utf8: &[u8],
src_ts_us: u64,
) -> Result<(), SinkError> {
let ts_us = Self::now_us();
let pub_ = self.streams.get_mut(stream).ok_or_else(|| {
SinkError::new(format!("unknown stream: {stream}"))
})?;
Self::write_sample(pub_, key, value_utf8, src_ts_us, ts_us, self.max_value_bytes)
}
fn heartbeat(&mut self, snapshot: &MetricsSnapshot) -> Result<(), SinkError> {
let Some(pub_) = self.heartbeat.as_mut() else {
return Ok(());
};
let ts_us = Self::now_us();
let value = Self::encode_heartbeat(snapshot);
Self::write_sample(pub_, "heartbeat", &value, ts_us, ts_us, self.max_value_bytes)
}
}
// ── Tests ──────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn sample_layout_offsets_are_correct() {
let mut buf = vec![0u8; HEADER_BYTES + 256];
// seq = 42
buf[0..8].copy_from_slice(&42u64.to_le_bytes());
// src_ts_us = 1000
buf[8..16].copy_from_slice(&1000u64.to_le_bytes());
// ts_us = 2000
buf[16..24].copy_from_slice(&2000u64.to_le_bytes());
// key = "BTCUSDT"
let k = b"BTCUSDT";
buf[24..24 + k.len()].copy_from_slice(k);
// len = 6
buf[72..76].copy_from_slice(&6u32.to_le_bytes());
// value = "123456"
buf[76..82].copy_from_slice(b"123456");
assert_eq!(u64::from_le_bytes(buf[0..8].try_into().unwrap()), 42);
assert_eq!(u64::from_le_bytes(buf[8..16].try_into().unwrap()), 1000);
assert_eq!(u64::from_le_bytes(buf[16..24].try_into().unwrap()), 2000);
assert_eq!(&buf[24..31], b"BTCUSDT");
assert_eq!(u32::from_le_bytes(buf[72..76].try_into().unwrap()), 6);
assert_eq!(&buf[76..82], b"123456");
}
#[test]
fn header_byte_count_is_76() {
assert_eq!(HEADER_BYTES, 76);
}
#[test]
fn heartbeat_encoding_contains_all_fields() {
let snap = MetricsSnapshot {
events_rx: 100,
events_published: 95,
decode_skips: 2,
late_responses: 0,
reconnects: 1,
fragments_reassembled: 0,
warmups: 3,
pings_ok: 50,
pings_missed: 1,
events_dropped: 0,
uptime_s: 3600,
state: "Streaming".into(),
degraded: false,
};
let encoded = Iceoryx2Sink::encode_heartbeat(&snap);
let text = String::from_utf8(encoded).unwrap();
assert!(text.contains("events_rx=100"));
assert!(text.contains("events_published=95"));
assert!(text.contains("decode_skips=2"));
assert!(text.contains("state=Streaming"));
assert!(text.contains("degraded=false"));
assert!(text.contains("uptime_s=3600"));
}
#[test]
fn key_truncation_fits_48_bytes() {
let long_key = "a".repeat(100);
let max_val = 256;
let mut buf = vec![0u8; HEADER_BYTES + max_val];
let kbytes = long_key.as_bytes();
let copy_len = kbytes.len().min(KEY_LEN);
buf[24..24 + copy_len].copy_from_slice(&kbytes[..copy_len]);
// Verify the stored key is truncated to KEY_LEN and NUL-padded
let stored = &buf[24..72];
assert_eq!(&stored[..KEY_LEN - 1], &[b'a'; 47]);
assert_eq!(stored[KEY_LEN - 1], b'a');
}
#[test]
fn value_truncation_fits_max_value() {
let max_val = 256;
let big_value = vec![0xFFu8; 500];
let value_len = big_value.len().min(max_val);
assert_eq!(value_len, max_val);
assert_eq!(value_len, 256);
}
}

461
pi_wake_agent.py Normal file
View File

@@ -0,0 +1,461 @@
#!/usr/bin/env python3
"""
pi_wake_agent.py — Reusable multi-agent wake-up timer with self-cron/daemon/succession
Usage:
pi_wake_agent.py --install --interval 1h --session cc_UV_dev0_Fb --msg "Operator says CONTINUE. Pi here!"
pi_wake_agent.py --once --interval 2h --session cc_UV_dev0_Fb --msg "Time's up!"
pi_wake_agent.py --daemon --interval 1h --session cc_UV_dev0_Fb
pi_wake_agent.py --succession --count 3 --interval 1h --session cc_UV_dev0_Fb --msg "Scheduled wake"
pi_wake_agent.py --remove --session cc_UV_dev0_Fb --interval 1h
pi_wake_agent.py --list
pi_wake_agent.py --status
pi_wake_agent.py --validate --session cc_UV_dev0_Fb
"""
import argparse
import logging
import os
import re
import shlex
import signal
import subprocess
import sys
import threading
import time
from pathlib import Path
from typing import List, Optional
# ─── Constants ────────────────────────────────────────────────────────────
SCRIPT_PATH = Path(__file__).resolve()
LOG_FILE = Path("/tmp/pi_wake_agent.log")
LOG_MAX_SIZE = 10 * 1024 * 1024 # 10 MB
LOG_MAX_FILES = 5
CRON_COMMENT_PREFIX = "pi_wake_agent"
AGENT_NICK = "pi_nvnemo"
H5I_AGENT = "pi_nvnemo"
H5I_BUS_ROOT = Path("/mnt/dolphinng5_predict")
DEFAULT_INTERVAL = "1h"
# ─── Logging Setup ───────────────────────────────────────────────────────
def setup_logging(debug: bool = False) -> logging.Logger:
log_rotate()
logger = logging.getLogger("pi_wake_agent")
logger.setLevel(logging.DEBUG if debug else logging.INFO)
fh = logging.FileHandler(LOG_FILE)
fh.setLevel(logging.DEBUG)
fh.setFormatter(logging.Formatter("[%(asctime)s] [%(levelname)s] %(message)s", datefmt="%Y-%m-%d %H:%M:%S"))
logger.addHandler(fh)
ch = logging.StreamHandler(sys.stderr)
ch.setLevel(logging.WARNING)
ch.setFormatter(logging.Formatter("[%(asctime)s] [%(levelname)s] %(message)s", datefmt="%Y-%m-%d %H:%M:%S"))
logger.addHandler(ch)
return logger
def log_rotate() -> None:
if not LOG_FILE.exists():
return
size = LOG_FILE.stat().st_size
if size < LOG_MAX_SIZE:
return
for i in range(LOG_MAX_FILES - 1, 0, -1):
src = LOG_FILE.with_suffix(f".log.{i}") if i > 1 else LOG_FILE.with_suffix(".log.1")
if src.exists():
dst = LOG_FILE.with_suffix(f".log.{i + 1}")
src.rename(dst)
LOG_FILE.rename(LOG_FILE.with_suffix(".log.1"))
# ─── Interval Parsing ────────────────────────────────────────────────────
def parse_interval(interval: str) -> int:
"""Parse interval string (e.g., '1h', '30m', '90m', '2h', '10s') to seconds."""
match = re.match(r"^(\d+)([hms])$", interval)
if not match:
raise ValueError(f"Invalid interval format '{interval}'. Use like 1h, 30m, 90m, 2h, 10s")
value, unit = int(match.group(1)), match.group(2)
if unit == "h":
return value * 3600
elif unit == "m":
return value * 60
elif unit == "s":
return value
raise ValueError(f"Unknown unit: {unit}")
def interval_to_cron(interval: str) -> str:
"""Convert interval to cron schedule."""
match = re.match(r"^(\d+)([hm])$", interval)
if not match:
raise ValueError(f"Cron only supports minutes/hours intervals: {interval}")
value, unit = int(match.group(1)), match.group(2)
if unit == "h":
return f"0 */{value} * * *"
elif unit == "m":
if value >= 60:
raise ValueError(f"For minutes >= 60, use hours (e.g., 1h not 60m)")
return f"*/{value} * * * *"
raise ValueError(f"Cron only supports minutes/hours intervals: {interval}")
def interval_to_human(interval: str) -> str:
match = re.match(r"^(\d+)([hms])$", interval)
if not match:
return interval
value, unit = match.group(1), match.group(2)
if unit == "h":
return f"{value} hour(s)"
elif unit == "m":
return f"{value} minute(s)"
elif unit == "s":
return f"{value} second(s)"
return interval
# ─── Cron Management ────────────────────────────────────────────────────
def cron_comment(sessions: List[str], interval: str) -> str:
sessions_str = ",".join(sessions)
return f"{CRON_COMMENT_PREFIX}:{sessions_str}:{interval}"
def install_cron(sessions: List[str], interval: str, message: str, logger: logging.Logger) -> None:
cron_sched = interval_to_cron(interval)
comment = cron_comment(sessions, interval)
sessions_arg = f"'{','.join(sessions)}'"
cmd = f"cd {H5I_BUS_ROOT} && export H5I_AGENT={H5I_AGENT} && {SCRIPT_PATH} --run --sessions {sessions_arg} --msg {shlex.quote(message)}"
result = subprocess.run(["crontab", "-l"], capture_output=True, text=True)
existing = result.stdout if result.returncode == 0 else ""
lines = [line for line in existing.splitlines() if comment not in line]
lines.append(f"{cron_sched} {cmd} # {comment}")
new_cron = "\n".join(lines) + "\n"
subprocess.run(["crontab", "-"], input=new_cron, text=True, check=True)
logger.info(f"Installed cron: {cron_sched} -> {sessions} every {interval}")
def remove_cron(sessions: List[str], interval: str, logger: logging.Logger) -> None:
comment = cron_comment(sessions, interval)
result = subprocess.run(["crontab", "-l"], capture_output=True, text=True)
existing = result.stdout if result.returncode == 0 else ""
lines = [line for line in existing.splitlines() if comment not in line]
new_cron = "\n".join(lines) + ("\n" if lines else "")
subprocess.run(["crontab", "-"], input=new_cron, text=True, check=True)
logger.info(f"Removed cron for {sessions} ({interval})")
def list_cron(logger: logging.Logger) -> None:
print("=== pi_wake_agent cron entries ===")
result = subprocess.run(["crontab", "-l"], capture_output=True, text=True)
existing = result.stdout if result.returncode == 0 else ""
found = False
for line in existing.splitlines():
if CRON_COMMENT_PREFIX in line:
print(f" {line}")
found = True
if not found:
print(" (none)")
# ─── Zellij Operations ──────────────────────────────────────────────────
def zellij_session_exists(session: str) -> bool:
try:
result = subprocess.run(["zellij", "list-sessions"], capture_output=True, text=True, timeout=5)
clean = re.sub(r'\x1b\[[0-9;]*m', '', result.stdout)
for line in clean.splitlines():
if line.startswith(session + " ") or line == session:
return True
except Exception:
pass
return False
def zellij_write_chars(session: str, text: str) -> bool:
try:
subprocess.run(["zellij", "--session", session, "action", "write-chars", text],
capture_output=True, timeout=5)
return True
except Exception:
return False
def zellij_write_enter(session: str) -> bool:
try:
subprocess.run(["zellij", "--session", session, "action", "write", "13"],
capture_output=True, timeout=5)
return True
except Exception:
return False
# ─── Wake Action ────────────────────────────────────────────────────────────────────
logger.info(f"One-shot timer set for {interval} ({interval_to_human(interval)})")
def _wake():
time.sleep(interval_seconds)
run_wake(sessions, message, logger)
thread = threading.Thread(target=_wake, daemon=True)
thread.start()
logger.info(f"Background timer started (thread: {thread.ident})")
pid_file = Path(f"/tmp/pi_wake_agent_{sessions[0]}.pid")
pid_file.write_text(str(os.getpid()))
def run_wake(sessions: List[str], message: str, logger: logging.Logger) -> None:
sessions_str = ",".join(sessions)
logger.info(f"Waking {sessions_str} with message: {message}")
for session in sessions:
zellij_write_chars(session, f"[{AGENT_NICK} via zellij] {message} Run: h5i-bus msg inbox")
for _ in range(5):
zellij_write_enter(session)
time.sleep(1)
# Fire-and-forget h5i bus message (non-blocking)
def _send_bus():
try:
subprocess.run(
["h5i", "msg", "send", "Fable", f"{message} (timer wakeup)"],
cwd=H5I_BUS_ROOT,
env={**os.environ, "H5I_AGENT": H5I_AGENT},
capture_output=True,
timeout=5
)
except Exception:
pass # Silently ignore - fire and forget
threading.Thread(target=_send_bus, daemon=True).start()
logger.info(f"Wake sent to {sessions_str}")
def run_once(sessions: List[str], interval: str, message: str, logger: logging.Logger) -> None:
interval_seconds = parse_interval(interval)
logger.info(f"One-shot timer set for {interval} ({interval_to_human(interval)})")
def _wake():
time.sleep(interval_seconds)
run_wake(sessions, message, logger)
thread = threading.Thread(target=_wake, daemon=True)
thread.start()
logger.info(f"Background timer started (thread: {thread.ident})")
pid_file = Path(f"/tmp/pi_wake_agent_{sessions[0]}.pid")
pid_file.write_text(str(os.getpid()))
return 0
def run_daemon(sessions: List[str], interval: str, message: str, logger: logging.Logger) -> None:
interval_seconds = parse_interval(interval)
logger.info("=== DAEMON START ===")
logger.info(f"Interval: {interval} ({interval_to_human(interval)})")
logger.info(f"Sessions: {sessions}")
logger.info(f"Message: {message}")
run_wake(sessions, message, logger)
while True:
logger.debug(f"Sleeping for {interval_seconds}s...")
time.sleep(interval_seconds)
run_wake(sessions, message, logger)
def run_succession(sessions: List[str], interval: str, count: int, message: str, logger: logging.Logger) -> None:
"""Run wake N times at interval, then self-clean (remove cron if installed)."""
interval_seconds = parse_interval(interval)
logger.info(f"=== SUCCESSION START === Count: {count}, Interval: {interval} ({interval_to_human(interval)})")
logger.info(f"Sessions: {sessions}")
logger.info(f"Message: {message}")
for i in range(1, count + 1):
logger.info(f"Succession {i}/{count}")
run_wake(sessions, message, logger)
if i < count:
logger.debug(f"Sleeping for {interval_seconds}s until next succession...")
time.sleep(interval_seconds)
# Self-clean: remove any cron entry for this session/interval combo
try:
remove_cron(sessions, interval, logger)
logger.info("Self-cleanup complete (cron removed)")
except Exception as e:
logger.warning(f"Self-cleanup failed: {e}")
logger.info("=== SUCCESSION COMPLETE ===")
def status(logger: logging.Logger) -> None:
print("=== pi_wake_agent Status ===")
print(f"Script: {SCRIPT_PATH}")
print(f"Log: {LOG_FILE}")
print(f"Log size: {LOG_FILE.stat().st_size if LOG_FILE.exists() else 'N/A'} bytes")
print(f"Agent: {AGENT_NICK}")
print()
list_cron(logger)
print()
print("=== Active one-shot timers ===")
found = False
for pid_file in Path("/tmp").glob("pi_wake_agent_*.pid"):
found = True
try:
pid = int(pid_file.read_text().strip())
os.kill(pid, 0)
print(f" PID {pid} (active)")
except (ProcessLookupError, ValueError):
print(f" PID {pid_file.read_text().strip()} (dead, cleaning up)")
pid_file.unlink(missing_ok=True)
if not found:
print(" (none)")
def validate_sessions(sessions: List[str], logger: logging.Logger) -> None:
logger.info(f"Validating sessions: {sessions}")
for session in sessions:
if zellij_session_exists(session):
logger.info(f" {session}: EXISTS")
else:
logger.warning(f" {session}: NOT FOUND (may be dead/EXITED)")
# ─── Session Parsing ─────────────────────────────────────────────────────
def parse_sessions(raw) -> List[str]:
if not raw:
return []
if isinstance(raw, list):
return [s.strip() for s in raw]
return [s.strip() for s in raw.split(",") if s.strip()]
# ─── Usage ──────────────────────────────────────────────────────────────
def create_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="pi_wake_agent.py — Reusable multi-agent wake-up timer with self-cron/daemon/succession",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
EXAMPLES:
# Install recurring 1-hour timer for one session
pi_wake_agent.py --install --interval 1h --session cc_UV_dev0_Fb
# Install 30-minute timer for multiple sessions
pi_wake_agent.py --install --interval 30m --sessions "cc_UV_dev0_Fb,cc_UV_dev1_48" --msg "Wake up!"
# One-shot wake in 2 hours (no cron)
pi_wake_agent.py --once --interval 2h --session cc_UV_dev0_Fb --msg "Time's up!"
# Run N times at interval, then self-clean
pi_wake_agent.py --succession --count 3 --interval 1h --session cc_UV_dev0_Fb --msg "Scheduled wake"
# Run as daemon (long-lived process, no cron)
pi_wake_agent.py --daemon --interval 1h --session cc_UV_dev0_Fb
# Remove timer
pi_wake_agent.py --remove --session cc_UV_dev0_Fb --interval 1h
# List all timers
pi_wake_agent.py --list
# Show status
pi_wake_agent.py --status
# Validate sessions
pi_wake_agent.py --validate --session cc_UV_dev0_Fb
"""
)
parser = argparse.ArgumentParser(
description="pi_wake_agent.py — Reusable multi-agent wake-up timer with self-cron/daemon/succession",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--install", action="store_const", const="install", dest="mode", help="Install recurring cron timer")
parser.add_argument("--once", action="store_const", const="once", dest="mode", help="One-shot wake (no cron)")
parser.add_argument("--daemon", action="store_const", const="daemon", dest="mode", help="Run as long-lived daemon (no cron)")
parser.add_argument("--succession", action="store_const", const="succession", dest="mode", help="Run N times at interval, then self-clean")
parser.add_argument("--count", type=int, default=1, help="Number of successions (for --succession mode)")
parser.add_argument("--run", action="store_const", const="run", dest="mode", help="Internal: run wake action (called by cron)")
parser.add_argument("--remove", action="store_const", const="remove", dest="mode", help="Remove cron timer")
parser.add_argument("--list", action="store_const", const="list", dest="mode", help="List active cron timers")
parser.add_argument("--status", action="store_const", const="status", dest="mode", help="Show status (cron + one-shot timers)")
parser.add_argument("--validate", action="store_const", const="validate", dest="mode", help="Validate sessions exist in zellij")
parser.add_argument("--interval", default=DEFAULT_INTERVAL, help="Interval (default: 1h). Formats: 30m, 1h, 90m, 2h, etc.")
parser.add_argument("--session", action="append", dest="session_list", help="Zellij session name (can repeat)")
parser.add_argument("--sessions", help="Comma-separated list of sessions")
parser.add_argument("--msg", default=f"Operator says CONTINUE. {AGENT_NICK} here, saying hi!", help="Wake message")
parser.add_argument("--debug", action="store_true", help="Enable debug logging")
parser.add_argument("--dry-run", action="store_true", help="Show what would be done without executing")
parser.set_defaults(mode="install")
return parser
# ─── Main ───────────────────────────────────────────────────────────────
def main() -> int:
parser = create_parser()
args = parser.parse_args()
# Combine sessions
sessions: List[str] = []
if args.sessions:
sessions.extend(parse_sessions(args.sessions))
if args.session_list:
sessions.extend(args.session_list)
# Setup logging
logger = setup_logging(args.debug)
# Validate
if len(sessions) == 0 and args.mode not in ("list", "status"):
logger.error("--session or --sessions required")
parser.print_help()
return 1
if args.mode == "succession" and args.count < 1:
logger.error("--count must be >= 1 for succession mode")
return 1
logger.info(f"=== pi_wake_agent {args.mode} ===")
logger.info(f"Interval: {args.interval} ({interval_to_human(args.interval)})")
logger.info(f"Sessions: {sessions}")
logger.info(f"Message: {args.msg}")
if args.mode == "succession":
logger.info(f"Count: {args.count}")
if args.dry_run:
logger.info("DRY RUN: no actions will be executed")
if args.dry_run:
logger.info("DRY RUN: would execute mode '%s' with sessions=%s, interval=%s, message='%s'", args.mode, sessions, args.interval, args.msg)
return 0
try:
if args.mode == "install":
install_cron(sessions, args.interval, args.msg, logger)
elif args.mode == "once":
run_once(sessions, args.interval, args.msg, logger)
elif args.mode == "daemon":
run_daemon(sessions, args.interval, args.msg, logger)
elif args.mode == "succession":
run_succession(sessions, args.interval, args.count, args.msg, logger)
elif args.mode == "run":
run_wake(sessions, args.msg, logger)
elif args.mode == "remove":
remove_cron(sessions, args.interval, logger)
elif args.mode == "list":
list_cron(logger)
elif args.mode == "status":
status(logger)
elif args.mode == "validate":
validate_sessions(sessions, logger)
else:
logger.error(f"Unknown mode: {args.mode}")
return 1
except Exception as e:
logger.error(f"Error: {e}")
if args.debug:
import traceback
traceback.print_exc()
return 1
return 0
if __name__ == "__main__":
sys.exit(main())

51
pi_wake_agent.skill.json Normal file
View File

@@ -0,0 +1,51 @@
{
"name": "pi_wake_agent",
"description": "Multi-agent wake-up timer with self-cron/daemon/succession modes. Sends doorbell injections via zellij and durable messages via h5i bus.",
"parameters": {
"type": "object",
"properties": {
"mode": {
"type": "string",
"enum": ["install", "once", "daemon", "succession", "run", "remove", "list", "status", "validate"],
"description": "Operation mode"
},
"interval": {
"type": "string",
"pattern": "^\\d+[hms]$",
"default": "1h",
"description": "Interval duration. Formats: 30m, 1h, 90m, 2h, 10s"
},
"session": {
"type": "array",
"items": { "type": "string" },
"description": "Zellij session name(s). Repeatable."
},
"sessions": {
"type": "string",
"description": "Comma-separated list of sessions"
},
"message": {
"type": "string",
"default": "Operator says CONTINUE. pi_nvnemo here, saying hi!",
"description": "Wake message sent to agent(s)"
},
"count": {
"type": "integer",
"minimum": 1,
"default": 1,
"description": "Number of runs for succession mode"
},
"dry_run": {
"type": "boolean",
"default": false,
"description": "Show what would be done without executing"
},
"debug": {
"type": "boolean",
"default": false,
"description": "Enable debug logging"
}
},
"required": ["mode"]
}
}

242
pi_wake_agent.skill.yaml Normal file
View File

@@ -0,0 +1,242 @@
# pi_wake_agent Skill Definition
# Universal Agent Skill Format (YAML)
# Compatible with: Anthropic, LangChain, OpenAI Functions, Custom Agents
skill:
name: "pi_wake_agent"
version: "1.1.0"
description: |
Multi-agent wake-up timer with self-cron/daemon/succession modes.
Sends doorbell injections via zellij and durable messages via h5i bus.
Designed for the DOLPHIN fleet (pi_nvnemo, cmd, mimo, codex, etc.).
author: "pi_nvnemo"
repository: "https://github.com/dolphinng5/pi_wake_agent"
branch: "tools/pi_wake_agent"
license: "MIT"
# ─── Installation ─────────────────────────────────────────────────────
installation:
method: "script"
path: "/mnt/dolphinng5_predict/pi_wake_agent.py"
requirements:
- python >= 3.8
- zellij (for terminal injections)
- h5i (for bus messaging, optional)
test_command: "python3 -m pytest test_pi_wake_agent.py -v"
# ─── Capabilities ─────────────────────────────────────────────────────
capabilities:
- name: "install_cron"
description: "Install recurring wake-up timer via system cron"
modes: ["install"]
- name: "once"
description: "One-shot wake after interval (no cron)"
modes: ["once"]
- name: "daemon"
description: "Long-lived process, no cron needed"
modes: ["daemon"]
- name: "succession"
description: "Run N times at interval, then self-clean (remove cron)"
modes: ["succession"]
- name: "run_wake"
description: "Internal: execute wake action (called by cron)"
modes: ["run"]
- name: "remove_cron"
description: "Remove cron timer entry"
modes: ["remove"]
- name: "list_cron"
description: "List active cron timers"
modes: ["list"]
- name: "status"
description: "Show cron + one-shot timer status"
modes: ["status"]
- name: "validate_sessions"
description: "Validate zellij sessions exist"
modes: ["validate"]
# ─── Parameters ───────────────────────────────────────────────────────
parameters:
mode:
type: "string"
enum: ["install", "once", "daemon", "succession", "run", "run", "remove", "list", "status", "validate"]
required: true
default: "install"
description: "Operation mode"
interval:
type: "string"
pattern: "^\\d+[hms]$"
default: "1h"
description: "Interval duration. Formats: 30m, 1h, 90m, 2h, 10s"
examples: ["1h", "30m", "90m", "2h", "10s"]
session:
type: "array"
items:
type: "string"
description: "Zellij session name(s). Repeatable flag."
examples: [["cc_UV_dev0_Fb"], ["cc_UV_dev0_Fb", "cc_UV_dev1_48"]]
sessions:
type: "string"
description: "Comma-separated list of sessions (alternative to --session)"
examples: ["cc_UV_dev0_Fb,cc_UV_dev1_48"]
message:
type: "string"
default: "Operator says CONTINUE. pi_nvnemo here, saying hi!"
description: "Wake message sent to agent(s)"
count:
type: "integer"
minimum: 1
default: 1
description: "Number of runs for succession mode"
dry_run:
type: "boolean"
default: false
description: "Show what would be done without executing"
debug:
type: "boolean"
default: false
description: "Enable debug logging"
json:
type: "boolean"
default: false
description: "Output JSON for machine parsing"
config:
type: "string"
description: "Path to config file (JSON/YAML)"
# ─── Returns ──────────────────────────────────────────────────────────
returns:
type: "object"
properties:
exit_code:
type: "integer"
description: "0 = success, non-zero = error"
cron_entry:
type: "string"
description: "Installed cron line (for install mode)"
log_file:
type: "string"
value: "/tmp/pi_wake_agent.log"
message_sent:
type: "boolean"
description: "Whether wake message was sent"
# ─── Side Effects ─────────────────────────────────────────────────────
side_effects:
- "Modifies system crontab (install/remove modes)"
- "Injects text into zellij sessions (zellij write-chars)"
- "Sends h5i bus message to Fable (fire-and-forget)"
- "Writes to /tmp/pi_wake_agent.log (rotated at 10MB)"
- "Creates PID files in /tmp/pi_wake_agent_*.pid (once mode)"
# ─── h5i Bus Protocol ─────────────────────────────────────────────────
bus_protocol:
agent_id: "pi_nvnemo"
target: "Fable"
message_format: "[pi_nvnemo via zellij] {message} Run: h5i-bus msg inbox"
keypresses: 5
keypress_delay_ms: 1000
bus_message: "{message} (timer wakeup)"
timeout_ms: 5000
fire_and_forget: true
silently_ignore_failures: true
# ─── Zellij Integration ───────────────────────────────────────────────
zellij:
command_check: "zellij list-sessions"
injection_method: "zellij --session {session} action write-chars"
enter_keypress: "zellij --session {session} action write 13"
ansi_strip: true
session_exists_check: true
# ─── Cron Format ──────────────────────────────────────────────────────
cron:
comment_format: "pi_wake_agent:{sessions}:{interval}"
session_delimiter: ","
command_template: "cd /mnt/dolphinng5_predict && export H5I_AGENT=pi_nvnemo && /mnt/dolphinng5_predict/pi_wake_agent.py --run --sessions '{sessions}' --msg '{message}'"
# ─── Logging ──────────────────────────────────────────────────────────
logging:
file: "/tmp/pi_wake_agent.log"
rotation_mb: 10
max_files: 5
format: "[YYYY-MM-DD HH:MM:SS] [LEVEL] message"
levels: ["DEBUG", "INFO", "WARN", "ERROR"]
# ─── Examples ─────────────────────────────────────────────────────────
examples:
- name: "Recurring 1-hour wake for one session"
command: "pi_wake_agent.py --install --interval 1h --session cc_UV_dev0_Fb"
- name: "Multi-session 30-minute wake"
command: 'pi_wake_agent.py --install --interval 30m --sessions "cc_UV_dev0_Fb,cc_UV_dev1_48" --msg "Wake up!"'
- name: "One-shot in 2 hours"
command: 'pi_wake_agent.py --once --interval 2h --session cc_UV_dev0_Fb --msg "Time\'s up!"'
- name: "Run 3 times at 1-hour intervals, then self-clean"
command: 'pi_wake_agent.py --succession --count 3 --interval 1h --session cc_UV_dev0_Fb --msg "Scheduled wake"'
- name: "Daemon mode (long-lived process, no cron)"
command: 'pi_wake_agent.py --daemon --interval 1h --session cc_UV_dev0_Fb'
- name: "Remove timer"
command: 'pi_wake_agent.py --remove --session cc_UV_dev0_Fb --interval 1h'
- name: "List / status"
command: "pi_wake_agent.py --list"
command: "pi_wake_agent.py --status"
- name: "Validate sessions exist"
command: 'pi_wake_agent.py --validate --session cc_UV_dev0_Fb'
- name: "Dry run (show what would happen)"
command: 'pi_wake_agent.py --install --interval 1h --session test --msg "test" --dry-run'
# ─── Error Handling ───────────────────────────────────────────────────
error_handling:
- condition: "missing_session"
action: "return_error"
message: "--session or --sessions required"
exit_code: 1
- condition: "invalid_interval"
action: "return_error"
message: "Invalid interval format. Use like 1h, 30m, 90m, 2h"
exit_code: 1
- condition: "invalid_count"
action: "return_error"
message: "--count must be >= 1 for succession mode"
exit_code: 1
- condition: "cron_failure"
action: "log_warning_continue"
description: "Cron install/remove logs warning but continues"
- condition: "zellij_not_found"
action: "log_warning_continue"
description: "Session injection fails silently, bus message still sent"
- condition: "h5i_timeout"
action: "silent_ignore"
description: "h5i bus message failures are silently ignored (fire-and-forget)"
# ─── Testing ──────────────────────────────────────────────────────────
testing:
test_file: "test_pi_wake_agent.py"
test_count: 38
coverage:
- unit_interval_parsing
- unit_cron_comments
- unit_session_parsing
- integration_install_remove_list
- integration_once_short
- integration_succession_short
- edge_cases_invalid_intervals
- edge_cases_missing_sessions
- edge_cases_invalid_counts
run_command: "python3 -m pytest test_pi_wake_agent.py -v"
# ─── Metadata ─────────────────────────────────────────────────────────
metadata:
created: "2026-07-08"
updated: "2026-07-08"
maintainer: "pi_nvnemo"
tags: ["wake-timer", "cron", "zellij", "h5i", "multi-agent", "doorbell", "self-cleaning"]

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,22 @@
# VIOLET base-fraction sizing study
Generated: `2026-06-16T21:23:08.644826+00:00`
## Bottom line
- Recommended fraction: `0.333` (best Calmar among zero-clip fractions)
- Base 0.20 final capital: `483963.79`; recommended final capital: `1308199.81`
- Base 0.20 maxDD: `20.18%`; recommended maxDD: `33.36%`
- Recommended clip rate: `0.00%`; unconstrained optimizer: `0.500` with `40.11%` clipped at the 3x ceiling.
## Caveat
- Direct non-null slippage telemetry was not available in `trade_execution_quality`; the study uses a conservative taker-heavy impact proxy.
## Cap binding
- The recommended fraction hits the 3x translator cap on `0.00%` of trades.
## Kelly anchor
- Empirical Kelly anchor: `0.500`; fractional anchor: `0.200`
## Files
- JSON: `/mnt/dolphinng5_predict/prod/VIOLET_dev/reports/base_fraction_study_20260616_212308.json`
- Markdown: `/mnt/dolphinng5_predict/prod/VIOLET_dev/reports/base_fraction_study_20260616_212308.md`

View File

@@ -0,0 +1,866 @@
#!/usr/bin/env python3
"""VIOLET base-fraction sizing study.
Read-only analysis against recorded CH trade data. Produces:
- a machine-readable JSON report under prod/VIOLET_dev/reports/
- a short markdown findings note alongside it
The study is scoped by prod/docs/VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md.
It does not modify production code or write to production tables.
"""
from __future__ import annotations
import argparse
import bisect
import csv
import dataclasses
import json
import math
import statistics
import sys
from collections import defaultdict
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any, Callable, Iterable, Sequence
import numpy as np
PROJECT_ROOT = Path("/mnt/dolphinng5_predict")
REPORTS_DIR = PROJECT_ROOT / "prod" / "VIOLET_dev" / "reports"
CH_URL = "http://localhost:8123/"
CH_USER = "dolphin"
CH_KEY = "dolphin_ch_2026"
BASE_FRACTION_F0 = 0.20
TRANSLATOR_CAP = 3.0
BASE_GRID = np.array(
[0.20, 0.22, 0.24, 0.25, 0.26, 0.28, 0.30, 0.32, 0.333, 0.34, 0.36, 0.38, 0.40, 0.45, 0.50],
dtype=np.float64,
)
@dataclass(frozen=True)
class TradeRow:
trade_id: str
ts: datetime
asset: str
side: str
entry_price: float
exit_price: float
quantity: float
pnl: float
pnl_pct: float
exit_reason: str
leverage: float
capital_before: float
capital_after: float
bars_held: int
regime_signal: int
vel_div_entry: float
boost_at_entry: float
beta_at_entry: float
posture: str
our_leverage: float
composite_hash: int | None = None
scalar_hash: int | None = None
regime: str | None = None
fingerprint_confidence: float | None = None
fingerprint_vel_div: float | None = None
fingerprint_dvol: float | None = None
notional_quote: float | None = None
fill_quality_score: float | None = None
fill_quality_class: str | None = None
fill_rows: int = 0
taker_fill_rows: int = 0
maker_fill_rows: int = 0
@dataclass(frozen=True)
class FingerprintRow:
ts: datetime
regime: str
composite_hash: int
scalar_hash: int
confidence: float
raw_vel_div: float
raw_dvol: float
final_score: float
def _query_tsv(sql: str) -> list[dict[str, str]]:
import urllib.request
req = urllib.request.Request(
CH_URL,
data=sql.encode(),
headers={"X-ClickHouse-User": CH_USER, "X-ClickHouse-Key": CH_KEY},
)
with urllib.request.urlopen(req, timeout=120) as resp:
text = resp.read().decode()
lines = [line for line in text.splitlines() if line.strip()]
if not lines:
return []
reader = csv.DictReader(lines, delimiter="\t")
return list(reader)
def _fmt_ts(dt: datetime) -> str:
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
dt = dt.astimezone(timezone.utc)
return dt.strftime("%Y-%m-%d %H:%M:%S.%f")
def _parse_dt(s: str) -> datetime:
if isinstance(s, datetime):
return s
s = s.strip()
if s.endswith("Z"):
s = s[:-1] + "+00:00"
try:
return datetime.fromisoformat(s).astimezone(timezone.utc)
except ValueError:
# ClickHouse DateTime64 TSV may arrive without timezone suffix.
return datetime.strptime(s, "%Y-%m-%d %H:%M:%S.%f").replace(tzinfo=timezone.utc)
def _parse_float(v: str | None, default: float = 0.0) -> float:
if v is None or v == "" or v == "\\N":
return float(default)
return float(v)
def _parse_int(v: str | None, default: int = 0) -> int:
if v is None or v == "" or v == "\\N":
return int(default)
return int(float(v))
def load_clean_trades() -> list[TradeRow]:
sql = """
WITH dedup AS (
SELECT
trade_id,
max(event_ts) AS ts,
argMax(asset, event_ts) AS asset,
argMax(side, event_ts) AS side,
argMax(entry_price, event_ts) AS entry_price,
argMax(exit_price, event_ts) AS exit_price,
argMax(quantity, event_ts) AS quantity,
argMax(pnl, event_ts) AS pnl,
argMax(pnl_pct, event_ts) AS pnl_pct,
argMax(exit_reason, event_ts) AS exit_reason,
argMax(leverage, event_ts) AS leverage,
argMax(capital_before, event_ts) AS capital_before,
argMax(capital_after, event_ts) AS capital_after,
argMax(bars_held, event_ts) AS bars_held,
argMax(regime_signal, event_ts) AS regime_signal,
argMax(vel_div_entry, event_ts) AS vel_div_entry,
argMax(boost_at_entry, event_ts) AS boost_at_entry,
argMax(beta_at_entry, event_ts) AS beta_at_entry,
argMax(posture, event_ts) AS posture,
argMax(our_leverage, event_ts) AS our_leverage
FROM (
SELECT
trade_id,
ts AS event_ts,
asset,
side,
entry_price,
exit_price,
quantity,
pnl,
pnl_pct,
exit_reason,
leverage,
capital_before,
capital_after,
bars_held,
regime_signal,
vel_div_entry,
boost_at_entry,
beta_at_entry,
posture,
our_leverage
FROM dolphin.trade_events
)
GROUP BY trade_id
)
SELECT
trade_id, ts, asset, side, entry_price, exit_price, quantity, pnl, pnl_pct, exit_reason,
leverage, capital_before, capital_after, bars_held, regime_signal, vel_div_entry,
boost_at_entry, beta_at_entry, posture, our_leverage
FROM dedup
WHERE exit_reason != 'HIBERNATE_HALT' AND bars_held > 0
ORDER BY ts
FORMAT TSVWithNames
"""
rows = _query_tsv(sql)
out: list[TradeRow] = []
for row in rows:
out.append(
TradeRow(
trade_id=row["trade_id"],
ts=_parse_dt(row["ts"]),
asset=row["asset"],
side=row["side"],
entry_price=_parse_float(row["entry_price"]),
exit_price=_parse_float(row["exit_price"]),
quantity=_parse_float(row["quantity"]),
pnl=_parse_float(row["pnl"]),
pnl_pct=_parse_float(row["pnl_pct"]),
exit_reason=row["exit_reason"],
leverage=_parse_float(row["leverage"]),
capital_before=_parse_float(row["capital_before"]),
capital_after=_parse_float(row["capital_after"]),
bars_held=_parse_int(row["bars_held"]),
regime_signal=_parse_int(row["regime_signal"]),
vel_div_entry=_parse_float(row["vel_div_entry"]),
boost_at_entry=_parse_float(row["boost_at_entry"], 1.0),
beta_at_entry=_parse_float(row["beta_at_entry"], 1.0),
posture=row["posture"],
our_leverage=_parse_float(row["our_leverage"]),
)
)
return out
def load_exec_quality() -> dict[str, dict[str, Any]]:
sql = """
SELECT
trade_id,
maxIf(notional_quote, record_kind = 'trade_summary') AS notional_quote,
maxIf(fill_quality_score, record_kind = 'trade_summary') AS fill_quality_score,
anyIf(fill_quality_class, record_kind = 'trade_summary') AS fill_quality_class,
countIf(record_kind = 'fill') AS fill_rows,
countIf(record_kind = 'fill' AND liquidity_side = 'TAKER') AS taker_fill_rows,
countIf(record_kind = 'fill' AND liquidity_side = 'MAKER') AS maker_fill_rows
FROM dolphin.trade_execution_quality
GROUP BY trade_id
FORMAT TSVWithNames
"""
rows = _query_tsv(sql)
out: dict[str, dict[str, Any]] = {}
for row in rows:
out[row["trade_id"]] = {
"notional_quote": _parse_float(row["notional_quote"]),
"fill_quality_score": _parse_float(row["fill_quality_score"]),
"fill_quality_class": row["fill_quality_class"],
"fill_rows": _parse_int(row["fill_rows"]),
"taker_fill_rows": _parse_int(row["taker_fill_rows"]),
"maker_fill_rows": _parse_int(row["maker_fill_rows"]),
}
return out
def load_fingerprints(start_ts: datetime, end_ts: datetime) -> list[FingerprintRow]:
sql = f"""
SELECT
ts, regime, composite_hash, scalar_hash, confidence, raw_vel_div, raw_dvol, final_score
FROM dolphin.maras_fingerprint
WHERE ts >= toDateTime64('{_fmt_ts(start_ts)}', 6, 'UTC')
AND ts <= toDateTime64('{_fmt_ts(end_ts)}', 6, 'UTC')
ORDER BY ts
FORMAT TSVWithNames
"""
rows = _query_tsv(sql)
out: list[FingerprintRow] = []
for row in rows:
out.append(
FingerprintRow(
ts=_parse_dt(row["ts"]),
regime=row["regime"],
composite_hash=_parse_int(row["composite_hash"]),
scalar_hash=_parse_int(row["scalar_hash"]),
confidence=_parse_float(row["confidence"]),
raw_vel_div=_parse_float(row["raw_vel_div"]),
raw_dvol=_parse_float(row["raw_dvol"]),
final_score=_parse_float(row["final_score"]),
)
)
return out
def attach_fingerprint(trades: list[TradeRow], fps: list[FingerprintRow]) -> list[TradeRow]:
fp_ts = [fp.ts for fp in fps]
out: list[TradeRow] = []
for trade in trades:
idx = bisect.bisect_right(fp_ts, trade.ts) - 1
if idx >= 0:
fp = fps[idx]
trade = dataclasses.replace(
trade,
composite_hash=fp.composite_hash,
scalar_hash=fp.scalar_hash,
regime=fp.regime,
fingerprint_confidence=fp.confidence,
fingerprint_vel_div=fp.raw_vel_div,
fingerprint_dvol=fp.raw_dvol,
)
out.append(trade)
return out
def build_trade_set() -> tuple[list[TradeRow], dict[str, Any]]:
trades = load_clean_trades()
if not trades:
raise RuntimeError("no clean trades returned from ClickHouse")
eq = load_exec_quality()
enriched: list[TradeRow] = []
for trade in trades:
meta = eq.get(trade.trade_id, {})
notional = _parse_float(str(meta.get("notional_quote", 0.0)), 0.0)
if notional <= 0:
notional = abs(trade.entry_price * trade.quantity)
enriched.append(
dataclasses.replace(
trade,
notional_quote=notional,
fill_quality_score=meta.get("fill_quality_score"),
fill_quality_class=meta.get("fill_quality_class"),
fill_rows=int(meta.get("fill_rows", 0)),
taker_fill_rows=int(meta.get("taker_fill_rows", 0)),
maker_fill_rows=int(meta.get("maker_fill_rows", 0)),
)
)
fps = load_fingerprints(trades[0].ts - timedelta(hours=1), trades[-1].ts)
enriched = attach_fingerprint(enriched, fps)
return enriched, {
"trade_rows": len(trades),
"exec_quality_rows": len(eq),
"fingerprint_rows": len(fps),
}
def slippage_bps_model(notional: float, median_notional: float, *, taker_only: bool = True) -> float:
"""Conservative parametric impact proxy.
Direct slippage telemetry in trade_execution_quality is mostly null for this
dataset, so we assume a taker-heavy execution floor and a sublinear
size-dependent impact term.
"""
base = 0.35 if taker_only else 0.20
impact = 0.15 * math.sqrt(max(notional, 1.0) / max(median_notional, 1.0))
return base + impact
def replay_equity(
trades: Sequence[TradeRow],
fraction: float,
*,
slippage_enabled: bool = True,
median_notional: float,
ruin_threshold: float = 0.50,
) -> dict[str, Any]:
if not trades:
raise ValueError("no trades")
start_capital = trades[0].capital_before if trades[0].capital_before > 0 else 69_000.0
capital = float(start_capital)
equity_points: list[tuple[datetime, float]] = []
daily_close: dict[str, float] = {}
daily_peak: dict[str, float] = {}
clipped = 0
total = 0
returns: list[float] = []
daily_equity: dict[str, float] = {}
for trade in trades:
total += 1
leverage_eff = min(trade.leverage, TRANSLATOR_CAP / max(fraction, 1e-12))
if trade.leverage * fraction > TRANSLATOR_CAP:
clipped += 1
notional = capital * fraction * leverage_eff
slip_bps = slippage_bps_model(
notional,
median_notional,
taker_only=(trade.taker_fill_rows > 0 or trade.fill_rows > 0),
) if slippage_enabled else 0.0
trade_return = fraction * leverage_eff * (trade.pnl_pct - slip_bps / 10_000.0)
capital *= 1.0 + trade_return
returns.append(trade_return)
equity_points.append((trade.ts, capital))
day = trade.ts.date().isoformat()
daily_equity[day] = capital
# Fill daily series from first to last trade day.
first_day = trades[0].ts.date()
last_day = trades[-1].ts.date()
day = first_day
last_equity = start_capital
daily_series: list[tuple[str, float]] = []
while day <= last_day:
key = day.isoformat()
if key in daily_equity:
last_equity = daily_equity[key]
daily_series.append((key, last_equity))
day += timedelta(days=1)
daily_returns = []
prev = start_capital
for _, eq in daily_series:
daily_returns.append((eq / prev) - 1.0 if prev else 0.0)
prev = eq
peak = start_capital
max_dd = 0.0
underwater_start: str | None = None
longest_underwater_days = 0.0
current_underwater_days = 0.0
prev_day: str | None = None
for day_str, eq in daily_series:
if eq >= peak:
if underwater_start is not None and prev_day is not None:
start = datetime.fromisoformat(underwater_start).date()
end = datetime.fromisoformat(prev_day).date()
longest_underwater_days = max(
longest_underwater_days, (end - start).days + 1
)
peak = eq
underwater_start = None
current_underwater_days = 0.0
else:
if underwater_start is None:
underwater_start = day_str
current_underwater_days += 1.0
max_dd = max(max_dd, 1.0 - eq / peak if peak > 0 else 0.0)
prev_day = day_str
if underwater_start is not None and prev_day is not None:
start = datetime.fromisoformat(underwater_start).date()
end = datetime.fromisoformat(prev_day).date()
longest_underwater_days = max(longest_underwater_days, (end - start).days + 1)
years = max((trades[-1].ts - trades[0].ts).total_seconds() / (365.25 * 24 * 3600), 1.0 / 365.25)
if capital > 0 and start_capital > 0:
log_growth = math.log(capital / start_capital)
annual_log = log_growth / years
if annual_log > 700.0:
cagr = float("inf")
elif annual_log < -700.0:
cagr = -1.0
else:
cagr = math.expm1(annual_log)
else:
cagr = -1.0
ann_sharpe, ann_sortino, downside_dev = _daily_risk_metrics(daily_returns)
ruin_prob = _bootstrap_ruin_prob(
trades=trades,
fraction=fraction,
median_notional=median_notional,
slippage_enabled=slippage_enabled,
ruin_threshold=ruin_threshold,
)
return {
"fraction": fraction,
"start_capital": start_capital,
"final_capital": capital,
"cagr": cagr,
"max_drawdown": max_dd,
"calmar": (cagr / max_dd) if max_dd > 0 else float("inf"),
"sharpe": ann_sharpe,
"sortino": ann_sortino,
"downside_deviation": downside_dev,
"ruin_prob": ruin_prob,
"pct_trades_clipped_at_3x": (clipped / total * 100.0) if total else 0.0,
"longest_underwater_days": longest_underwater_days,
"n_trades": total,
"daily_returns": daily_returns,
"equity_points": equity_points,
"returns": returns,
}
def _daily_risk_metrics(daily_returns: Sequence[float]) -> tuple[float, float, float]:
if len(daily_returns) < 2:
return 0.0, 0.0, 0.0
arr = np.asarray(daily_returns, dtype=np.float64)
mean = float(np.mean(arr))
std = float(np.std(arr, ddof=1)) if len(arr) > 1 else 0.0
downside = arr[arr < 0.0]
downside_dev = float(np.std(downside, ddof=1)) if len(downside) > 1 else float(np.std(np.minimum(arr, 0.0), ddof=0))
sharpe = (mean / std * math.sqrt(365.25)) if std > 0 else 0.0
sortino = (mean / downside_dev * math.sqrt(365.25)) if downside_dev > 0 else 0.0
return sharpe, sortino, downside_dev
def _bootstrap_ruin_prob(
*,
trades: Sequence[TradeRow],
fraction: float,
median_notional: float,
slippage_enabled: bool,
ruin_threshold: float,
n_boot: int = 256,
seed: int = 17,
) -> float:
rng = np.random.default_rng(seed)
n = len(trades)
if n == 0:
return 0.0
leverage = np.array([min(t.leverage, TRANSLATOR_CAP / max(fraction, 1e-12)) for t in trades], dtype=np.float64)
pnl_pct = np.array([t.pnl_pct for t in trades], dtype=np.float64)
taker_only = np.array([(t.taker_fill_rows > 0 or t.fill_rows > 0) for t in trades], dtype=np.float64)
ruin = 0
for _ in range(n_boot):
idx = rng.integers(0, n, size=n)
capital = 1.0
floor = ruin_threshold
for j in idx:
lev = leverage[j]
notional = capital * fraction * lev
slip = slippage_bps_model(notional, median_notional, taker_only=bool(taker_only[j])) if slippage_enabled else 0.0
r = fraction * lev * (pnl_pct[j] - slip / 10_000.0)
capital *= 1.0 + r
if capital <= floor:
ruin += 1
break
return ruin / n_boot
def cap_binding_curve(trades: Sequence[TradeRow], fractions: Sequence[float]) -> list[dict[str, float]]:
out = []
n = len(trades)
for f in fractions:
clipped = sum(1 for t in trades if t.leverage * f > TRANSLATOR_CAP)
out.append({"fraction": float(f), "pct_clipped": (clipped / n * 100.0) if n else 0.0})
return out
def _group_by_hash(trades: Sequence[TradeRow]) -> dict[int, list[TradeRow]]:
groups: dict[int, list[TradeRow]] = defaultdict(list)
for trade in trades:
if trade.composite_hash is None:
continue
groups[int(trade.composite_hash)].append(trade)
for arr in groups.values():
arr.sort(key=lambda t: t.ts)
return groups
def _bucket_stats(trades: Sequence[TradeRow], fraction: float, median_notional: float) -> dict[str, Any]:
groups = _group_by_hash(trades)
bucket_results = []
for h, rows in groups.items():
if len(rows) < 8:
continue
rep = replay_equity(rows, fraction, slippage_enabled=True, median_notional=median_notional)
bucket_results.append(
{
"composite_hash": int(h),
"n_trades": len(rows),
"final_capital": rep["final_capital"],
"max_drawdown": rep["max_drawdown"],
"cagr": rep["cagr"],
"calmar": rep["calmar"],
"ruin_prob": rep["ruin_prob"],
}
)
bucket_results.sort(key=lambda x: (x["max_drawdown"], -x["n_trades"]), reverse=True)
return {
"bucket_results": bucket_results,
"worst_bucket": bucket_results[0] if bucket_results else None,
}
def slippage_model_summary(trades: Sequence[TradeRow]) -> dict[str, Any]:
notionals = np.array([t.notional_quote or abs(t.entry_price * t.quantity) for t in trades], dtype=np.float64)
if len(notionals) == 0:
median_notional = 1.0
else:
median_notional = float(np.median(notionals))
taker_fill_rate = float(
sum(1 for t in trades if t.taker_fill_rows > 0 or t.fill_rows > 0) / max(len(trades), 1)
)
observed_direct = sum(
1 for t in trades if math.isfinite(float(t.fill_quality_score or 0.0)) and (t.fill_quality_score or 0.0) not in (0.0, None)
)
return {
"type": "conservative_parametric_proxy",
"observed_direct_slippage_rows": 0,
"observed_direct_slippage_trade_rows": observed_direct,
"taker_fill_rate": taker_fill_rate,
"median_notional_quote": median_notional,
"formula": "slippage_bps = 0.35 + 0.15 * sqrt(notional / median_notional)",
"assumption": "execution-quality rows do not expose non-null slippage_bps; all fill rows are taker, so this is a conservative proxy",
}
def analyze() -> dict[str, Any]:
trades, counts = build_trade_set()
median_notional = float(np.median([t.notional_quote or abs(t.entry_price * t.quantity) for t in trades]))
fractions = np.unique(np.concatenate([BASE_GRID, np.round(np.arange(0.20, 0.501, 0.01), 3)])).astype(np.float64)
slippage_summary = slippage_model_summary(trades)
results_no_slip = []
results_slip = []
for f in fractions:
results_no_slip.append(replay_equity(trades, float(f), slippage_enabled=False, median_notional=median_notional))
results_slip.append(replay_equity(trades, float(f), slippage_enabled=True, median_notional=median_notional))
best_slip = max(results_slip, key=lambda x: (x["calmar"], x["final_capital"]))
best_no_slip = max(results_no_slip, key=lambda x: (x["calmar"], x["final_capital"]))
refined = np.unique(
np.concatenate([
fractions,
np.round(np.arange(max(0.20, best_slip["fraction"] - 0.03), min(0.50, best_slip["fraction"] + 0.03) + 0.0001, 0.005), 3),
])
).astype(np.float64)
if len(refined) > len(fractions):
results_slip = [replay_equity(trades, float(f), slippage_enabled=True, median_notional=median_notional) for f in refined]
results_no_slip = [replay_equity(trades, float(f), slippage_enabled=False, median_notional=median_notional) for f in refined]
fractions = refined
best_slip = max(results_slip, key=lambda x: (x["calmar"], x["final_capital"]))
best_no_slip = max(results_no_slip, key=lambda x: (x["calmar"], x["final_capital"]))
cap_curve = cap_binding_curve(trades, fractions)
bucket_summary = _bucket_stats(trades, float(best_slip["fraction"]), median_notional)
cap_by_fraction = {round(float(row["fraction"]), 3): float(row["pct_clipped"]) for row in cap_curve}
feasible_rows = [r for r in results_slip if cap_by_fraction.get(round(float(r["fraction"]), 3), 0.0) == 0.0]
practical_best = max(feasible_rows, key=lambda x: (x["calmar"], x["final_capital"])) if feasible_rows else best_slip
# Kelly anchor: use unit-fraction returns (return at f=1.0, ignoring cap/slip)
unit_returns = np.array([t.pnl_pct * min(t.leverage, TRANSLATOR_CAP / 1.0) for t in trades], dtype=np.float64)
kelly_grid = np.linspace(0.01, 0.50, 200)
kelly_log_growth = []
for f in kelly_grid:
growth = np.mean(np.log1p(np.clip(f * unit_returns, -0.95, None)))
kelly_log_growth.append(float(growth))
kelly_idx = int(np.argmax(kelly_log_growth))
kelly_fraction = float(kelly_grid[kelly_idx])
fractional_kelly = float(min(best_slip["fraction"], max(0.25 * kelly_fraction, 0.20)))
# Stress scenario: worst-hash bucket loss multiplied slightly and injected once.
stress = None
if bucket_summary["worst_bucket"] is not None:
worst_hash = bucket_summary["worst_bucket"]["composite_hash"]
worst_rows = sorted([t for t in trades if t.composite_hash == worst_hash], key=lambda t: t.ts)
if worst_rows:
worst_trade = min(worst_rows, key=lambda t: t.pnl_pct)
stress_trade = dataclasses.replace(worst_trade, pnl_pct=min(-0.01, worst_trade.pnl_pct * 1.5))
stress_rows = list(worst_rows) + [stress_trade]
stress_rows.sort(key=lambda t: t.ts)
stress = {
"worst_hash": int(worst_hash),
"worst_trade_id": worst_trade.trade_id,
"stress_pnl_pct": float(stress_trade.pnl_pct),
"per_fraction": [
{
"fraction": float(f),
"final_capital": replay_equity(stress_rows, float(f), slippage_enabled=True, median_notional=median_notional)["final_capital"],
"ruin_prob": replay_equity(stress_rows, float(f), slippage_enabled=True, median_notional=median_notional)["ruin_prob"],
}
for f in fractions
],
}
recommendation = {
"recommended_fraction": float(practical_best["fraction"]),
"recommended_basis": "best_calmar_among_zero_clip_fractions",
"recommended_final_capital": float(practical_best["final_capital"]),
"recommended_cagr": float(practical_best["cagr"]),
"recommended_max_drawdown": float(practical_best["max_drawdown"]),
"recommended_calmar": float(practical_best["calmar"]),
"recommended_ruin_prob": float(practical_best["ruin_prob"]),
"recommended_clip_pct": float(cap_by_fraction.get(round(float(practical_best["fraction"]), 3), 0.0)),
"alt_fraction_floor": float(fractional_kelly),
"no_slippage_best_fraction": float(best_no_slip["fraction"]),
"slippage_best_fraction": float(best_slip["fraction"]),
"optimizer_fraction": float(best_slip["fraction"]),
"optimizer_final_capital": float(best_slip["final_capital"]),
"optimizer_cagr": float(best_slip["cagr"]),
"optimizer_max_drawdown": float(best_slip["max_drawdown"]),
"optimizer_calmar": float(best_slip["calmar"]),
"optimizer_ruin_prob": float(best_slip["ruin_prob"]),
"optimizer_clip_pct": float(next(c for c in cap_curve if abs(c["fraction"] - best_slip["fraction"]) < 1e-9)["pct_clipped"]),
"delta_final_capital_vs_base_0p20": float(
next(r for r in results_slip if abs(r["fraction"] - 0.20) < 1e-9)["final_capital"]
),
}
return {
"generated_at": datetime.now(timezone.utc).isoformat(),
"study_spec": "prod/docs/VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md",
"source_counts": counts,
"clean_trade_count": len(trades),
"clean_trade_window": {
"start": trades[0].ts.isoformat(),
"end": trades[-1].ts.isoformat(),
},
"slippage_model": slippage_summary,
"cap_curve": cap_curve,
"results": {
"no_slippage": results_no_slip,
"slippage_adjusted": results_slip,
},
"kelly": {
"kelly_fraction": kelly_fraction,
"fractional_kelly_anchor": fractional_kelly,
"kelly_log_growth_grid": [{"fraction": float(f), "log_growth": float(g)} for f, g in zip(kelly_grid, kelly_log_growth)],
},
"bucket_summary": bucket_summary,
"stress_scenario": stress,
"recommendation": recommendation,
}
def _best_row(rows: Sequence[dict[str, Any]]) -> dict[str, Any]:
return max(rows, key=lambda x: (x["calmar"], x["final_capital"]))
def _format_pct(x: float) -> str:
return f"{x * 100.0:.2f}%"
def write_report(report: dict[str, Any]) -> tuple[Path, Path]:
REPORTS_DIR.mkdir(parents=True, exist_ok=True)
ts = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
json_path = REPORTS_DIR / f"base_fraction_study_{ts}.json"
md_path = REPORTS_DIR / f"base_fraction_study_{ts}.md"
json_path.write_text(json.dumps(report, indent=2, sort_keys=True, default=str))
best = report["recommendation"]
base = next(r for r in report["results"]["slippage_adjusted"] if abs(r["fraction"] - 0.20) < 1e-9)
recommended_cap = next(c for c in report["cap_curve"] if abs(c["fraction"] - best["recommended_fraction"]) < 1e-9)
optimizer_cap = next(c for c in report["cap_curve"] if abs(c["fraction"] - best["optimizer_fraction"]) < 1e-9)
md = []
md.append("# VIOLET base-fraction sizing study")
md.append("")
md.append(f"Generated: `{report['generated_at']}`")
md.append("")
md.append("## Bottom line")
md.append(
f"- Recommended fraction: `{best['recommended_fraction']:.3f}` "
f"(best Calmar among zero-clip fractions)"
)
md.append(
f"- Base 0.20 final capital: `{base['final_capital']:.2f}`; "
f"recommended final capital: `{best['recommended_final_capital']:.2f}`"
)
md.append(
f"- Base 0.20 maxDD: `{_format_pct(base['max_drawdown'])}`; "
f"recommended maxDD: `{_format_pct(best['recommended_max_drawdown'])}`"
)
md.append(
f"- Recommended clip rate: `{recommended_cap['pct_clipped']:.2f}%`; "
f"unconstrained optimizer: `{best['optimizer_fraction']:.3f}` with "
f"`{optimizer_cap['pct_clipped']:.2f}%` clipped at the 3x ceiling."
)
md.append("")
md.append("## Caveat")
md.append(
"- Direct non-null slippage telemetry was not available in `trade_execution_quality`; "
"the study uses a conservative taker-heavy impact proxy."
)
md.append("")
md.append("## Cap binding")
md.append(
f"- The recommended fraction hits the 3x translator cap on `{recommended_cap['pct_clipped']:.2f}%` of trades."
)
md.append("")
md.append("## Kelly anchor")
md.append(
f"- Empirical Kelly anchor: `{report['kelly']['kelly_fraction']:.3f}`; "
f"fractional anchor: `{report['kelly']['fractional_kelly_anchor']:.3f}`"
)
md.append("")
md.append("## Files")
md.append(f"- JSON: `{json_path}`")
md.append(f"- Markdown: `{md_path}`")
md_path.write_text("\n".join(md) + "\n")
return json_path, md_path
def self_test() -> None:
trades = [
TradeRow(
trade_id="t1",
ts=datetime(2026, 1, 1, 0, 0, tzinfo=timezone.utc),
asset="X",
side="SHORT",
entry_price=100.0,
exit_price=99.0,
quantity=1.0,
pnl=10.0,
pnl_pct=0.10,
exit_reason="FIXED_TP",
leverage=2.0,
capital_before=100.0,
capital_after=110.0,
bars_held=5,
regime_signal=-1,
vel_div_entry=-0.03,
boost_at_entry=1.0,
beta_at_entry=1.0,
posture="APEX",
our_leverage=0.4,
composite_hash=1,
notional_quote=100.0,
taker_fill_rows=1,
fill_rows=1,
),
TradeRow(
trade_id="t2",
ts=datetime(2026, 1, 2, 0, 0, tzinfo=timezone.utc),
asset="X",
side="SHORT",
entry_price=100.0,
exit_price=101.0,
quantity=1.0,
pnl=-5.0,
pnl_pct=-0.01,
exit_reason="MAX_HOLD",
leverage=7.0,
capital_before=110.0,
capital_after=105.0,
bars_held=5,
regime_signal=-1,
vel_div_entry=-0.03,
boost_at_entry=1.0,
beta_at_entry=1.0,
posture="APEX",
our_leverage=0.6,
composite_hash=1,
notional_quote=110.0,
taker_fill_rows=1,
fill_rows=1,
),
]
rep = replay_equity(trades, 0.20, slippage_enabled=False, median_notional=100.0, ruin_threshold=0.50)
assert rep["n_trades"] == 2
assert rep["pct_trades_clipped_at_3x"] == 0.0
assert rep["final_capital"] > 100.0
cap = cap_binding_curve(trades, [0.20, 0.50])
assert cap[0]["pct_clipped"] == 0.0
assert cap[1]["pct_clipped"] == 50.0
slip = slippage_bps_model(100.0, 100.0)
assert slip > 0.0
def main(argv: Sequence[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--self-test", action="store_true", help="run the deterministic synthetic fixture and exit")
parser.add_argument("--dry-run", action="store_true", help="run the live analysis but do not write files")
args = parser.parse_args(argv)
if args.self_test:
self_test()
print("self-test ok")
return 0
report = analyze()
if args.dry_run:
print(json.dumps(report["recommendation"], indent=2, sort_keys=True, default=str))
return 0
json_path, md_path = write_report(report)
print(json_path)
print(md_path)
print(json.dumps(report["recommendation"], indent=2, sort_keys=True, default=str))
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -60,6 +60,13 @@ CH_WAL_TRUNCATE_BYTES = int(os.environ.get("CH_WAL_TRUNCATE_BYTES", str(64 * 102
CH_VACUUM_MIN_BYTES = int(os.environ.get("CH_VACUUM_MIN_BYTES", str(512 * 1024 * 1024))) CH_VACUUM_MIN_BYTES = int(os.environ.get("CH_VACUUM_MIN_BYTES", str(512 * 1024 * 1024)))
CH_VACUUM_MIN_FREE_RATIO = float(os.environ.get("CH_VACUUM_MIN_FREE_RATIO", "1.25")) CH_VACUUM_MIN_FREE_RATIO = float(os.environ.get("CH_VACUUM_MIN_FREE_RATIO", "1.25"))
CH_VACUUM_MIN_FREE_BYTES = int(os.environ.get("CH_VACUUM_MIN_FREE_BYTES", str(128 * 1024 * 1024))) CH_VACUUM_MIN_FREE_BYTES = int(os.environ.get("CH_VACUUM_MIN_FREE_BYTES", str(128 * 1024 * 1024)))
# Poison-row quarantine: a row CH permanently rejects (schema mismatch, bad
# value) must not head-of-line-block the spool forever. After this many
# failed attempts the row is retried INDIVIDUALLY (CH proven up first); if it
# still fails it moves to the dead_letter table for offline repair/replay.
# Incident 2026-06-12: one trade_events row with bars_held=-106 (UInt16
# column) was retried 3.2M times and jammed 18M rows behind it for 1.5 days.
CH_POISON_ATTEMPTS = int(os.environ.get("CH_POISON_ATTEMPTS", "200"))
# ─── Timestamp helpers ──────────────────────────────────────────────────────── # ─── Timestamp helpers ────────────────────────────────────────────────────────
@@ -202,6 +209,19 @@ class _CHWriter:
conn.execute( conn.execute(
"CREATE INDEX IF NOT EXISTS idx_queue_table ON queue(table_name, id)" "CREATE INDEX IF NOT EXISTS idx_queue_table ON queue(table_name, id)"
) )
conn.execute(
"""
CREATE TABLE IF NOT EXISTS dead_letter (
id INTEGER PRIMARY KEY,
table_name TEXT NOT NULL,
payload TEXT NOT NULL,
created_ts_us INTEGER NOT NULL,
attempts INTEGER NOT NULL,
dead_ts_us INTEGER NOT NULL,
last_error TEXT
)
"""
)
return conn return conn
_PUT_LOCK_TIMEOUT_S: float = 0.1 # max wait before dropping the row _PUT_LOCK_TIMEOUT_S: float = 0.1 # max wait before dropping the row
@@ -276,7 +296,7 @@ class _CHWriter:
with self._lock: with self._lock:
cur = self._conn.execute( cur = self._conn.execute(
"SELECT id, attempts FROM queue WHERE id IN (%s)" % ",".join("?" for _ in ids), "SELECT id, attempts FROM queue WHERE id IN (%s)" % ",".join("?" for _ in ids),
[int(_id) for _ in ids], [int(_id) for _id in ids],
) )
high_attempts = [(row[0], int(row[1]) + 1) for row in cur.fetchall() if int(row[1]) >= 1000] high_attempts = [(row[0], int(row[1]) + 1) for row in cur.fetchall() if int(row[1]) >= 1000]
self._conn.executemany( self._conn.executemany(
@@ -291,6 +311,65 @@ class _CHWriter:
row_id, attempt, row_id, attempt,
) )
def _ch_alive(self) -> bool:
"""True iff ClickHouse answers a trivial query — used to distinguish
'CH is down' (retry forever, quarantine nothing) from 'CH rejects this
specific row' (quarantine after CH_POISON_ATTEMPTS)."""
try:
req = urllib.request.Request(f"{CH_URL}/?query=SELECT+1", method="GET")
req.add_header("X-ClickHouse-User", CH_USER)
req.add_header("X-ClickHouse-Key", CH_PASS)
with urllib.request.urlopen(req, timeout=3) as resp:
return resp.status == 200
except Exception:
return False
def _quarantine_poison(self, table: str, items: List[Tuple[int, dict]]) -> None:
"""After a batch failure, isolate rows CH permanently rejects.
Only rows whose attempt count exceeds CH_POISON_ATTEMPTS are touched,
and only while CH itself is provably up. Each candidate is retried
alone: success → delivered+deleted; failure → moved to dead_letter
(payload preserved for offline repair/replay) so the spool can drain.
"""
ids = [row_id for row_id, _ in items]
if not ids:
return
with self._lock:
cur = self._conn.execute(
"SELECT id, attempts FROM queue WHERE id IN (%s)"
% ",".join("?" for _ in ids),
[int(i) for i in ids],
)
attempts_by_id = {int(r[0]): int(r[1]) for r in cur.fetchall()}
candidates = [
(row_id, payload) for row_id, payload in items
if attempts_by_id.get(int(row_id), 0) >= CH_POISON_ATTEMPTS
]
if not candidates:
return
if not self._ch_alive():
return # CH outage — nothing is poison, keep retrying the batch
for row_id, payload in candidates:
if self._post_rows(table, [payload]):
self._delete_ids([row_id])
continue
now = ts_us()
with self._lock:
self._conn.execute(
"INSERT OR REPLACE INTO dead_letter "
"(id, table_name, payload, created_ts_us, attempts, dead_ts_us, last_error) "
"SELECT id, table_name, payload, created_ts_us, attempts, ?, ? "
"FROM queue WHERE id=?",
(now, "rejected by CH while CH alive (see ch flush WARNINGs)", int(row_id)),
)
self._conn.execute("DELETE FROM queue WHERE id=?", (int(row_id),))
log.error(
"ch_writer[%s]: POISON ROW quarantined to dead_letter: id=%s table=%s "
"attempts=%d — spool unblocked; repair/replay offline",
self._db, row_id, table, attempts_by_id.get(int(row_id), -1),
)
def _queue_count(self) -> int: def _queue_count(self) -> int:
with self._lock: with self._lock:
row = self._conn.execute("SELECT count(*) FROM queue").fetchone() row = self._conn.execute("SELECT count(*) FROM queue").fetchone()
@@ -433,7 +512,7 @@ class _CHWriter:
raw = resp.read().decode("utf-8", errors="replace") raw = resp.read().decode("utf-8", errors="replace")
return [line for line in raw.splitlines() if line] return [line for line in raw.splitlines() if line]
def _existing_trade_keys(self, rows: List[dict]) -> set[Tuple[str, int]]: def _existing_trade_keys(self, rows: List[dict]) -> set[Tuple[str, str]]:
trade_ids: List[str] = [] trade_ids: List[str] = []
for row in rows: for row in rows:
trade_id = row.get("trade_id") trade_id = row.get("trade_id")
@@ -447,13 +526,14 @@ class _CHWriter:
return set() return set()
unique = sorted(set(trade_ids)) unique = sorted(set(trade_ids))
existing: set[Tuple[str, int]] = set() existing: set[Tuple[str, str]] = set()
chunk_size = 200 chunk_size = 200
for i in range(0, len(unique), chunk_size): for i in range(0, len(unique), chunk_size):
chunk = unique[i : i + chunk_size] chunk = unique[i : i + chunk_size]
quoted = ",".join("'" + tid.replace("'", "''") + "'" for tid in chunk) quoted = ",".join("'" + tid.replace("'", "''") + "'" for tid in chunk)
sql = ( sql = (
"SELECT trade_id, toInt64(toUnixTimestamp64Micro(ts)) " "SELECT trade_id, "
"ifNull(nullIf(event_id, ''), concat(toString(toInt64(toUnixTimestamp64Micro(ts))), ':', ifNull(exit_reason, ''))) "
f"FROM trade_events WHERE trade_id IN ({quoted}) FORMAT TSV" f"FROM trade_events WHERE trade_id IN ({quoted}) FORMAT TSV"
) )
try: try:
@@ -466,13 +546,24 @@ class _CHWriter:
parts = line.split("\t", 1) parts = line.split("\t", 1)
if len(parts) != 2: if len(parts) != 2:
continue continue
tid, ts_us_s = parts tid, event_key = parts
try: existing.add((tid, event_key))
existing.add((tid, int(ts_us_s)))
except Exception:
continue
return existing return existing
@staticmethod
def _trade_event_key(payload: dict) -> Tuple[str, str] | None:
tid = str(payload.get("trade_id", "") or "").strip()
if not tid:
return None
event_id = str(payload.get("event_id", "") or "").strip()
if event_id:
return (tid, event_id)
try:
ts_us_val = int(payload.get("ts"))
except Exception:
ts_us_val = -1
return (tid, f"{ts_us_val}:{payload.get('exit_reason', '')}")
def flush_once(self) -> int: def flush_once(self) -> int:
""" """
Drain a single batch from the local spool. Drain a single batch from the local spool.
@@ -494,38 +585,37 @@ class _CHWriter:
rows = [payload for _, payload in items] rows = [payload for _, payload in items]
if table == "trade_events": if table == "trade_events":
existing = self._existing_trade_keys(rows) existing = self._existing_trade_keys(rows)
if existing: seen = set(existing)
kept_ids: List[int] = [] kept_ids: List[int] = []
kept_rows: List[dict] = [] kept_rows: List[dict] = []
duplicate_ids: List[int] = [] duplicate_ids: List[int] = []
for row_id, payload in items: for row_id, payload in items:
tid = str(payload.get("trade_id", "")).strip() probe = self._trade_event_key(payload)
try: if probe is not None and probe in seen:
ts_us_val = int(payload.get("ts")) duplicate_ids.append(row_id)
except Exception:
ts_us_val = -1
if tid and ts_us_val >= 0 and (tid, ts_us_val) in existing:
duplicate_ids.append(row_id)
else:
kept_ids.append(row_id)
kept_rows.append(payload)
if duplicate_ids:
self._delete_ids(duplicate_ids)
log.warning(
"ch_writer[%s]: dropped %d duplicate trade_events rows by trade_id",
self._db,
len(duplicate_ids),
)
ids = kept_ids
rows = kept_rows
if not rows:
continue continue
kept_ids.append(row_id)
kept_rows.append(payload)
if probe is not None:
seen.add(probe)
if duplicate_ids:
self._delete_ids(duplicate_ids)
log.warning(
"ch_writer[%s]: dropped %d duplicate trade_events rows by stable event key",
self._db,
len(duplicate_ids),
)
ids = kept_ids
rows = kept_rows
if not rows:
continue
ok = self._post_rows(table, rows) ok = self._post_rows(table, rows)
if ok: if ok:
delivered += len(rows) delivered += len(rows)
self._delete_ids(ids) self._delete_ids(ids)
else: else:
self._bump_attempts(ids) self._bump_attempts(ids)
self._quarantine_poison(table, list(zip(ids, rows)))
self._maybe_maintain_spool() self._maybe_maintain_spool()
return delivered return delivered

View File

@@ -32,11 +32,10 @@ from .contracts import (
VenueEventStatus, VenueEventStatus,
VenueOrder, VenueOrder,
VenueOrderStatus, VenueOrderStatus,
VenueTelemetrySnapshot,
) )
from .journal import ClickHouseKernelJournal, KernelJournal, MemoryKernelJournal from .journal import ClickHouseKernelJournal, KernelJournal, MemoryKernelJournal
from .rust_backend import ExecutionKernel from .rust_backend import ExecutionKernel
from .bingx_venue import BingxVenueAdapter
from .launcher import DITAv2LauncherBundle, LauncherVenueMode, LauncherZincMode, build_launcher_bundle
from .projection import HazelcastProjection, build_position_state_row, build_projection from .projection import HazelcastProjection, build_position_state_row, build_projection
from .venue import VenueAdapter from .venue import VenueAdapter
from .mock_venue import MockVenueAdapter, MockVenueScenario from .mock_venue import MockVenueAdapter, MockVenueScenario
@@ -44,6 +43,28 @@ from .zinc_plane import InMemoryZincPlane, ZincPlane
from .real_zinc_plane import RealZincPlane, RealZincUnavailable from .real_zinc_plane import RealZincPlane, RealZincUnavailable
from .real_control_plane import RealZincControlPlane, RealZincUnavailable as RealZincControlUnavailable from .real_control_plane import RealZincControlPlane, RealZincUnavailable as RealZincControlUnavailable
def __getattr__(name: str):
if name == "BingxVenueAdapter":
from .bingx_venue import BingxVenueAdapter
return BingxVenueAdapter
if name in {"DITAv2LauncherBundle", "LauncherVenueMode", "LauncherZincMode", "build_launcher_bundle"}:
from .launcher import (
DITAv2LauncherBundle,
LauncherVenueMode,
LauncherZincMode,
build_launcher_bundle,
)
return {
"DITAv2LauncherBundle": DITAv2LauncherBundle,
"LauncherVenueMode": LauncherVenueMode,
"LauncherZincMode": LauncherZincMode,
"build_launcher_bundle": build_launcher_bundle,
}[name]
raise AttributeError(name)
__all__ = [ __all__ = [
"AccountProjection", "AccountProjection",
"AccountSnapshot", "AccountSnapshot",
@@ -89,6 +110,7 @@ __all__ = [
"VenueEventStatus", "VenueEventStatus",
"VenueOrder", "VenueOrder",
"VenueOrderStatus", "VenueOrderStatus",
"VenueTelemetrySnapshot",
"ZincPlane", "ZincPlane",
"ZincControlPlane", "ZincControlPlane",
"build_position_state_row", "build_position_state_row",

View File

@@ -8,6 +8,7 @@ from enum import Enum
from typing import Any, Dict, Iterable, List, Optional from typing import Any, Dict, Iterable, List, Optional
import math import math
import time import time
import threading
from .contracts import TradeSide, TradeSlot, TradeStage from .contracts import TradeSide, TradeSlot, TradeStage
from .utils import safe_float from .utils import safe_float
@@ -59,42 +60,49 @@ class AccountProjection:
GIL guarantees single-field reference assignment is atomic, so readers GIL guarantees single-field reference assignment is atomic, so readers
that hold snap = kernel.account.snapshot before use see a consistent view. that hold snap = kernel.account.snapshot before use see a consistent view.
""" """
cur = self.snapshot with self._lock:
self.snapshot = AccountSnapshot( cur = self.snapshot
capital=kw.get("capital", cur.capital), self.snapshot = AccountSnapshot(
equity=kw.get("equity", cur.equity), capital=kw.get("capital", cur.capital),
realized_pnl=kw.get("realized_pnl", cur.realized_pnl), equity=kw.get("equity", cur.equity),
unrealized_pnl=kw.get("unrealized_pnl", cur.unrealized_pnl), realized_pnl=kw.get("realized_pnl", cur.realized_pnl),
open_positions=kw.get("open_positions", cur.open_positions), unrealized_pnl=kw.get("unrealized_pnl", cur.unrealized_pnl),
open_notional=kw.get("open_notional", cur.open_notional), open_positions=kw.get("open_positions", cur.open_positions),
fees_paid=kw.get("fees_paid", cur.fees_paid), open_notional=kw.get("open_notional", cur.open_notional),
trade_seq=kw.get("trade_seq", cur.trade_seq), fees_paid=kw.get("fees_paid", cur.fees_paid),
peak_capital=kw.get("peak_capital", cur.peak_capital), trade_seq=kw.get("trade_seq", cur.trade_seq),
capital_source=kw.get("capital_source", cur.capital_source), peak_capital=kw.get("peak_capital", cur.peak_capital),
e_wallet_balance=kw.get("e_wallet_balance", cur.e_wallet_balance), capital_source=kw.get("capital_source", cur.capital_source),
event_seq=kw.get("event_seq", cur.event_seq), e_wallet_balance=kw.get("e_wallet_balance", cur.e_wallet_balance),
) event_seq=kw.get("event_seq", cur.event_seq),
)
def __post_init__(self) -> None:
self._lock = threading.RLock()
def observe_slots(self, slots: Iterable[TradeSlot]) -> None: def observe_slots(self, slots: Iterable[TradeSlot]) -> None:
open_positions = 0 with self._lock:
open_notional = 0.0 open_positions = 0
unrealized_pnl = 0.0 open_notional = 0.0
for slot in slots: unrealized_pnl = 0.0
if slot.closed or slot.size <= 0: for slot in slots:
continue if slot.closed or slot.size <= 0:
if slot.fsm_state in {TradeStage.POSITION_OPEN, TradeStage.POSITION_OPENED, TradeStage.ENTRY_WORKING, TradeStage.EXIT_WORKING}: continue
open_positions += 1 if slot.fsm_state in {TradeStage.POSITION_OPEN, TradeStage.POSITION_OPENED, TradeStage.ENTRY_WORKING, TradeStage.EXIT_WORKING}:
mark = safe_float(slot.entry_price, 0.0) open_positions += 1
mark = safe_float(slot.metadata.get("mark_price"), mark) mark = safe_float(slot.entry_price, 0.0)
open_notional += abs(slot.size) * abs(mark) mark = safe_float(slot.metadata.get("mark_price"), mark)
unrealized_pnl += float(slot.unrealized_pnl or 0.0) open_notional += abs(slot.size) * abs(mark)
self._replace_snapshot( unrealized_pnl += float(slot.unrealized_pnl or 0.0)
open_positions=open_positions, capital = self.snapshot.capital
open_notional=open_notional, peak_capital = self.snapshot.peak_capital
unrealized_pnl=unrealized_pnl, self._replace_snapshot(
equity=self.snapshot.capital + unrealized_pnl if math.isfinite(self.snapshot.capital + unrealized_pnl) else self.snapshot.capital, open_positions=open_positions,
peak_capital=max(self.snapshot.peak_capital, self.snapshot.capital) if open_notional > 0 and self.snapshot.capital > 0 else self.snapshot.peak_capital, open_notional=open_notional,
) unrealized_pnl=unrealized_pnl,
equity=capital + unrealized_pnl if math.isfinite(capital + unrealized_pnl) else capital,
peak_capital=max(peak_capital, capital) if open_notional > 0 and capital > 0 else peak_capital,
)
def anchor_to_exchange(self, wallet_balance: float, available_margin: float, event_seq: int) -> None: def anchor_to_exchange(self, wallet_balance: float, available_margin: float, event_seq: int) -> None:
"""Snap published capital to exchange wallet balance. """Snap published capital to exchange wallet balance.
@@ -106,39 +114,42 @@ class AccountProjection:
Guards: wallet_balance must be > 0 and finite (the zero-wb frame lesson Guards: wallet_balance must be > 0 and finite (the zero-wb frame lesson
from ACCOUNT_UPDATE frames with no USDT balance entry). from ACCOUNT_UPDATE frames with no USDT balance entry).
""" """
wb = safe_float(wallet_balance, 0.0) with self._lock:
if wb <= 0.0 or not math.isfinite(wb): wb = safe_float(wallet_balance, 0.0)
return if wb <= 0.0 or not math.isfinite(wb):
self.snapshot.capital = wb return
self.snapshot.e_wallet_balance = wb self.snapshot.capital = wb
self.snapshot.capital_source = "e_anchored" self.snapshot.e_wallet_balance = wb
self.snapshot.event_seq = int(event_seq) self.snapshot.capital_source = "e_anchored"
self.snapshot.equity = wb + self.snapshot.unrealized_pnl self.snapshot.event_seq = int(event_seq)
if not math.isfinite(self.snapshot.equity): self.snapshot.equity = wb + self.snapshot.unrealized_pnl
self.snapshot.equity = wb if not math.isfinite(self.snapshot.equity):
self.snapshot.peak_capital = max(self.snapshot.peak_capital, wb) self.snapshot.equity = wb
self.snapshot.peak_capital = max(self.snapshot.peak_capital, wb)
def settle(self, realized_pnl: float, fees: float = 0.0) -> None: def settle(self, realized_pnl: float, fees: float = 0.0) -> None:
rp = safe_float(realized_pnl, 0.0) with self._lock:
# Include fees in capital delta (today fees only accumulate in cur = self.snapshot
# fees_paid while published capital ignores them between reseeds). rp = safe_float(realized_pnl, 0.0)
net = rp - safe_float(fees, 0.0) # Include fees in capital delta (today fees only accumulate in
new_capital = safe_float(self.snapshot.capital + net, self.snapshot.capital) # fees_paid while published capital ignores them between reseeds).
if self.max_capital is not None: net = rp - safe_float(fees, 0.0)
new_capital = min(new_capital, self.max_capital) new_capital = safe_float(cur.capital + net, cur.capital)
new_capital = max(self.min_capital, new_capital) if self.max_capital is not None:
new_source = self.snapshot.capital_source new_capital = min(new_capital, self.max_capital)
if new_source == "e_anchored" and abs(net) > 1e-12: new_capital = max(self.min_capital, new_capital)
new_source = "k_bridged" new_source = cur.capital_source
new_fees = self.snapshot.fees_paid + safe_float(fees, 0.0) if new_source == "e_anchored" and abs(net) > 1e-12:
new_equity = new_capital + self.snapshot.unrealized_pnl new_source = "k_bridged"
if not math.isfinite(new_equity): new_fees = cur.fees_paid + safe_float(fees, 0.0)
new_equity = new_capital new_equity = new_capital + cur.unrealized_pnl
self._replace_snapshot( if not math.isfinite(new_equity):
capital=new_capital, capital_source=new_source, new_equity = new_capital
realized_pnl=self.snapshot.realized_pnl + rp, self._replace_snapshot(
fees_paid=new_fees, equity=new_equity, capital=new_capital, capital_source=new_source,
) realized_pnl=cur.realized_pnl + rp,
fees_paid=new_fees, equity=new_equity,
)
def to_account_event( def to_account_event(
self, self,
@@ -154,31 +165,32 @@ class AccountProjection:
bars_held: int = 0, bars_held: int = 0,
metadata: Optional[Dict[str, Any]] = None, metadata: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]: ) -> Dict[str, Any]:
self.snapshot.equity = self.snapshot.capital + self.snapshot.unrealized_pnl with self._lock:
return { self.snapshot.equity = self.snapshot.capital + self.snapshot.unrealized_pnl
"timestamp": timestamp.isoformat() if hasattr(timestamp, "isoformat") else str(timestamp), return {
"runtime_namespace": self.runtime_namespace, "timestamp": timestamp.isoformat() if hasattr(timestamp, "isoformat") else str(timestamp),
"strategy_namespace": self.strategy_namespace, "runtime_namespace": self.runtime_namespace,
"event_namespace": self.event_namespace, "strategy_namespace": self.strategy_namespace,
"actor_name": self.actor_name, "event_namespace": self.event_namespace,
"exec_venue": self.exec_venue, "actor_name": self.actor_name,
"data_venue": self.data_venue, "exec_venue": self.exec_venue,
"ledger_authority": self.ledger_authority, "data_venue": self.data_venue,
"capital": float(self.snapshot.capital), "ledger_authority": self.ledger_authority,
"equity": float(self.snapshot.equity), "capital": float(self.snapshot.capital),
"open_positions": int(self.snapshot.open_positions), "equity": float(self.snapshot.equity),
"current_open_notional": float(self.snapshot.open_notional), "open_positions": int(self.snapshot.open_positions),
"current_account_leverage": float(self.snapshot.leverage), "current_open_notional": float(self.snapshot.open_notional),
"trade_id": trade_id, "current_account_leverage": float(self.snapshot.leverage),
"asset": asset, "trade_id": trade_id,
"side": side.value, "asset": asset,
"reason": reason, "side": side.value,
"stage": stage.value, "reason": reason,
"pnl": float(pnl), "stage": stage.value,
"pnl_pct": float(pnl_pct), "pnl": float(pnl),
"bars_held": int(bars_held), "pnl_pct": float(pnl_pct),
"metadata": dict(metadata or {}), "bars_held": int(bars_held),
} "metadata": dict(metadata or {}),
}
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -311,6 +323,7 @@ class AccountProjectionV2:
self._min_capital = min_capital self._min_capital = min_capital
self._max_capital = max_capital self._max_capital = max_capital
self._cfg = reconcile_config or ReconcileConfig() self._cfg = reconcile_config or ReconcileConfig()
self._lock = threading.RLock()
# Running K-value accumulators # Running K-value accumulators
self._k_realized: float = 0.0 self._k_realized: float = 0.0
@@ -345,16 +358,18 @@ class AccountProjectionV2:
fee: float, fee: float,
realized_pnl: float, realized_pnl: float,
) -> None: ) -> None:
self._k_realized += _safe(realized_pnl) with self._lock:
self._k_fees += _safe(fee) self._k_realized += _safe(realized_pnl)
self._e_last_fill_price = _safe(fill_price) self._k_fees += _safe(fee)
self._e_last_fill_qty = _safe(fill_qty) self._e_last_fill_price = _safe(fill_price)
self._e_last_fill_fee = _safe(fee) self._e_last_fill_qty = _safe(fill_qty)
self._e_last_fill_realized = _safe(realized_pnl) self._e_last_fill_fee = _safe(fee)
self._e_last_fill_realized = _safe(realized_pnl)
def apply_funding(self, amount: float) -> None: def apply_funding(self, amount: float) -> None:
self._k_funding += _safe(amount) with self._lock:
self._e_last_funding = _safe(amount) self._k_funding += _safe(amount)
self._e_last_funding = _safe(amount)
def apply_balance_update( def apply_balance_update(
self, self,
@@ -364,13 +379,15 @@ class AccountProjectionV2:
used_margin: float, used_margin: float,
maint_margin: float, maint_margin: float,
) -> None: ) -> None:
self._e_wallet_balance = _safe(wallet_balance) with self._lock:
self._e_avail_margin = _safe(available_margin) self._e_wallet_balance = _safe(wallet_balance)
self._e_used_margin = _safe(used_margin) self._e_avail_margin = _safe(available_margin)
self._e_maint_margin = _safe(maint_margin) self._e_used_margin = _safe(used_margin)
self._e_maint_margin = _safe(maint_margin)
def apply_position_update(self, positions: List[EPosition]) -> None: def apply_position_update(self, positions: List[EPosition]) -> None:
self._e_positions = list(positions) with self._lock:
self._e_positions = list(positions)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Snapshot construction (called after each ingestion step) # Snapshot construction (called after each ingestion step)
@@ -382,21 +399,24 @@ class AccountProjectionV2:
slots: Iterable[TradeSlot], slots: Iterable[TradeSlot],
ts: Optional[float] = None, ts: Optional[float] = None,
) -> AccountSnapshotV2: ) -> AccountSnapshotV2:
self._event_seq += 1 with self._lock:
snap = self._build(self._event_seq, source_event_id, list(slots), ts or time.time()) self._event_seq += 1
self._snapshot = snap snap = self._build(self._event_seq, source_event_id, list(slots), ts or time.time())
return snap self._snapshot = snap
return snap
@property @property
def snapshot(self) -> AccountSnapshotV2: def snapshot(self) -> AccountSnapshotV2:
return self._snapshot with self._lock:
return self._snapshot
@property @property
def k_capital(self) -> float: def k_capital(self) -> float:
raw = self._seed + self._k_realized - self._k_fees - self._k_funding with self._lock:
if self._max_capital is not None: raw = self._seed + self._k_realized - self._k_fees - self._k_funding
raw = min(raw, self._max_capital) if self._max_capital is not None:
return max(self._min_capital, raw) raw = min(raw, self._max_capital)
return max(self._min_capital, raw)
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# Internal helpers # Internal helpers

View File

@@ -0,0 +1,106 @@
"""ASEx wrapper for AccountProjectionV2 — serializes the 4 P0 accumulator sites."""
from __future__ import annotations
import time
from concurrent.futures import Future
from typing import Any, Iterable, List, Optional
from asex.guarded import ASExGuardedState
from asex.worker import ASExWorker
from .account import AccountProjectionV2, AccountSnapshotV2, EPosition, ReconcileConfig, TradeSlot
class _AccountBackend(ASExGuardedState[dict, Any]):
"""Wraps AccountProjectionV2 mutations as ASEx operations."""
def __init__(self, seed_capital: float, **kw):
super().__init__()
self._proj = AccountProjectionV2(seed_capital, **kw)
def _validate(self, mutation: dict) -> bool:
return isinstance(mutation, dict) and "op" in mutation
def _apply(self, mutation: dict) -> Any:
op = mutation["op"]
args = mutation.get("args", {})
if op == "apply_fill":
self._proj.apply_fill(**args); return None
elif op == "apply_funding":
self._proj.apply_funding(**args); return None
elif op == "apply_balance_update":
self._proj.apply_balance_update(**args); return None
elif op == "apply_position_update":
self._proj.apply_position_update(**args); return None
elif op == "build_snapshot":
return self._proj.build_snapshot(**args)
raise ValueError(f"Unknown ASEx account op: {op}")
def __getattr__(self, name):
return getattr(self._proj, name)
class ASEXAccountV2:
"""ASEx-serialized wrapper around ``AccountProjectionV2``.
All mutations go through an ``ASExWorker`` — one thread, sequential,
no races. Property reads bypass the worker (lock-free, fast).
"""
def __init__(self, seed_capital: float, *, min_capital: float = 0.0,
max_capital: float | None = None,
reconcile_config: ReconcileConfig | None = None):
self._backend = _AccountBackend(seed_capital, min_capital=min_capital,
max_capital=max_capital, reconcile_config=reconcile_config)
self._worker: ASExWorker = ASExWorker(self._backend, daemon=True)
def apply_fill_async(self, **kw) -> Future:
return self._worker.mutate({"op": "apply_fill", "args": kw})
def apply_funding_async(self, amount: float) -> Future:
return self._worker.mutate({"op": "apply_funding", "args": {"amount": amount}})
def apply_balance_update_async(self, **kw) -> Future:
return self._worker.mutate({"op": "apply_balance_update", "args": kw})
def apply_position_update_async(self, positions: List[EPosition]) -> Future:
return self._worker.mutate({"op": "apply_position_update", "args": {"positions": positions}})
def build_snapshot_async(self, **kw) -> Future:
return self._worker.mutate({"op": "build_snapshot", "args": kw})
def apply_fill(self, **kw) -> None:
self.apply_fill_async(**kw).result(timeout=30)
def apply_funding(self, amount: float) -> None:
self.apply_funding_async(amount).result(timeout=30)
def apply_balance_update(self, **kw) -> None:
self.apply_balance_update_async(**kw).result(timeout=30)
def apply_position_update(self, positions: List[EPosition]) -> None:
self.apply_position_update_async(positions).result(timeout=30)
def build_snapshot(self, **kw) -> AccountSnapshotV2:
return self.build_snapshot_async(**kw).result(timeout=30)
@property
def snapshot(self) -> AccountSnapshotV2:
return self._backend._proj.snapshot
@property
def k_capital(self) -> float:
return self._backend._proj.k_capital
@property
def applied(self) -> int:
return self._backend.applied
def close(self, *, timeout: float | None = None) -> None:
self._worker.close(timeout=timeout)
def __enter__(self) -> ASEXAccountV2:
return self
def __exit__(self, *args) -> None:
self.close(timeout=5.0)

View File

@@ -27,6 +27,7 @@ from .contracts import (
KernelEventKind, KernelEventKind,
KernelIntent, KernelIntent,
TradeSide, TradeSide,
VenueTelemetrySnapshot,
VenueEvent, VenueEvent,
VenueEventStatus, VenueEventStatus,
VenueOrder, VenueOrder,
@@ -226,7 +227,7 @@ class BingxVenueAdapter(VenueAdapter):
) )
return cls._EXECUTOR return cls._EXECUTOR
def __init__(self, backend: Any | None = None, *, config: Any | None = None) -> None: def __init__(self, backend: Any | None = None, *, config: Any | None = None, zinc_plane: Any | None = None) -> None:
if backend is None: if backend is None:
if config is None: if config is None:
raise ValueError("BingxVenueAdapter requires a backend or config") raise ValueError("BingxVenueAdapter requires a backend or config")
@@ -234,6 +235,7 @@ class BingxVenueAdapter(VenueAdapter):
backend = BingxDirectExecutionAdapter(config) backend = BingxDirectExecutionAdapter(config)
self.backend = backend self.backend = backend
self._telemetry_plane = zinc_plane
self._event_seq = itertools.count(1) self._event_seq = itertools.count(1)
# Thread-safe snapshot cache — reads from a snapshot may arrive from # Thread-safe snapshot cache — reads from a snapshot may arrive from
# the kernel thread while _backend_snapshot writes from the pool thread. # the kernel thread while _backend_snapshot writes from the pool thread.
@@ -279,6 +281,73 @@ class BingxVenueAdapter(VenueAdapter):
except Exception: except Exception:
pass pass
def set_telemetry_plane(self, zinc_plane: Any | None) -> None:
self._telemetry_plane = zinc_plane
def _publish_telemetry(
self,
*,
phase: str,
status: str,
intent: KernelIntent | None = None,
order: VenueOrder | None = None,
endpoint: str = "",
method: str = "",
message: str = "",
retry_after_ms: int = 0,
venue_order_status: str = "",
venue_event_kind: str = "",
details: dict[str, Any] | None = None,
) -> None:
plane = self._telemetry_plane
publish = getattr(plane, "publish_venue", None) if plane is not None else None
if publish is None:
return
slot_id = 0
trade_id = ""
asset = ""
side = TradeSide.FLAT
action = ""
intent_id = ""
if intent is not None:
slot_id = int(getattr(intent, "slot_id", 0) or 0)
trade_id = str(getattr(intent, "trade_id", "") or "")
asset = str(getattr(intent, "asset", "") or "")
side = getattr(intent, "side", TradeSide.FLAT)
action = str(getattr(intent, "action", "") or "")
intent_id = str(getattr(intent, "intent_id", "") or "")
if order is not None:
slot_id = int(order.metadata.get("slot_id", slot_id) or slot_id)
trade_id = str(order.internal_trade_id or trade_id)
asset = str(order.metadata.get("asset") or asset)
side = order.side or side
try:
publish(
VenueTelemetrySnapshot(
phase=phase,
status=status,
venue="bingx",
endpoint=endpoint,
method=method,
intent_id=intent_id,
trade_id=trade_id,
slot_id=slot_id,
asset=asset,
side=side,
action=action,
order_id=str(getattr(order, "venue_order_id", "") or ""),
client_order_id=str(getattr(order, "venue_client_id", "") or ""),
venue_order_status=venue_order_status,
venue_event_kind=venue_event_kind,
message=message,
retry_after_ms=int(retry_after_ms or 0),
timestamp=datetime.now(timezone.utc),
details=dict(details or {}),
)
)
except Exception:
pass
def _call_backend(self, method_name: str, *args: Any, **kwargs: Any) -> Any: def _call_backend(self, method_name: str, *args: Any, **kwargs: Any) -> Any:
method = getattr(self.backend, method_name, None) method = getattr(self.backend, method_name, None)
if method is None: if method is None:
@@ -368,18 +437,37 @@ class BingxVenueAdapter(VenueAdapter):
was fixed for submit via submit_async. This version awaits backend.cancel() was fixed for submit via submit_async. This version awaits backend.cancel()
directly in the caller's (main) event loop. directly in the caller's (main) event loop.
""" """
self._publish_telemetry(
phase="cancel:start",
status="REQUESTED",
order=order,
endpoint="/openApi/swap/v2/trade/order",
method="DELETE",
message=reason,
details={"asset": str(order.metadata.get("asset") or "")},
)
cancel_fn = getattr(self.backend, "cancel", None) cancel_fn = getattr(self.backend, "cancel", None)
if cancel_fn is not None: if cancel_fn is not None:
response = await cancel_fn(order, reason=reason) response = await cancel_fn(order, reason=reason)
else: else:
response = None response = None
return self._events_from_cancel(order, response, None, None, reason=reason) events = self._events_from_cancel(order, response, None, None, reason=reason)
return events
def cancel(self, order: VenueOrder, *, reason: str = "") -> List[VenueEvent]: def cancel(self, order: VenueOrder, *, reason: str = "") -> List[VenueEvent]:
# _events_from_cancel never reads before/after — snapshots are dead weight. # _events_from_cancel never reads before/after — snapshots are dead weight.
# NOTE: if backend.cancel is async (BingxDirectExecutionAdapter), this sync # NOTE: if backend.cancel is async (BingxDirectExecutionAdapter), this sync
# path goes through the thread-pool and will deadlock in a running event loop. # path goes through the thread-pool and will deadlock in a running event loop.
# Use cancel_async() from async contexts (process_intent_async already does). # Use cancel_async() from async contexts (process_intent_async already does).
self._publish_telemetry(
phase="cancel:start",
status="REQUESTED",
order=order,
endpoint="/openApi/swap/v2/trade/order",
method="DELETE",
message=reason,
details={"asset": str(order.metadata.get("asset") or "")},
)
response = None response = None
if hasattr(self.backend, "cancel"): if hasattr(self.backend, "cancel"):
response = self._call_backend("cancel", order, reason=reason) response = self._call_backend("cancel", order, reason=reason)
@@ -410,7 +498,8 @@ class BingxVenueAdapter(VenueAdapter):
except BingxHttpError as exc: except BingxHttpError as exc:
# W10: map HTTP error class to status — 429/5xx are transient, 4xx are real rejections # W10: map HTTP error class to status — 429/5xx are transient, 4xx are real rejections
response = {"status": _http_error_status(str(exc)), "msg": str(exc), "orderId": order.venue_order_id, "clientOrderId": order.venue_client_id} response = {"status": _http_error_status(str(exc)), "msg": str(exc), "orderId": order.venue_order_id, "clientOrderId": order.venue_client_id}
return self._events_from_cancel(order, response, None, None, reason=reason) events = self._events_from_cancel(order, response, None, None, reason=reason)
return events
def open_orders(self) -> List[VenueOrder]: def open_orders(self) -> List[VenueOrder]:
# Use backend._state (populated by await backend.connect()) rather than # Use backend._state (populated by await backend.connect()) rather than
@@ -458,6 +547,13 @@ class BingxVenueAdapter(VenueAdapter):
# entirely: the FSM stayed fill-blind (slot size 0 in ENTRY_WORKING), # entirely: the FSM stayed fill-blind (slot size 0 in ENTRY_WORKING),
# the DecisionEngine saw "no position", and re-entered → the live # the DecisionEngine saw "no position", and re-entered → the live
# double-entries at 15:20 and 17:24 UTC. # double-entries at 15:20 and 17:24 UTC.
self._publish_telemetry(
phase="reconcile:start",
status="REQUESTED",
endpoint="/openApi/swap/v2/trade/openOrders",
method="GET",
details={"include_history": True},
)
recon_symbol = None recon_symbol = None
kernel = getattr(self, "_kernel_ref", None) kernel = getattr(self, "_kernel_ref", None)
if kernel is not None: if kernel is not None:
@@ -474,13 +570,60 @@ class BingxVenueAdapter(VenueAdapter):
except Exception as exc: except Exception as exc:
import logging as _log import logging as _log
_log.getLogger(__name__).warning("reconcile: refresh_state failed: %s", exc) _log.getLogger(__name__).warning("reconcile: refresh_state failed: %s", exc)
self._publish_telemetry(
phase="reconcile:error",
status="ERROR",
endpoint="/openApi/swap/v2/trade/openOrders",
method="GET",
message=str(exc),
details={"symbol": recon_symbol or ""},
)
return [] return []
self._publish_telemetry(
phase="reconcile:done",
status="OK",
endpoint="/openApi/swap/v2/trade/openOrders",
method="GET",
details={
"symbol": recon_symbol or "",
"open_orders": len(getattr(snapshot, "open_orders", []) or []),
"positions": len(getattr(snapshot, "open_positions", {}) or {}),
"fills": len(getattr(snapshot, "all_fills", []) or []),
},
)
return self._events_from_snapshot(snapshot) return self._events_from_snapshot(snapshot)
def submit(self, intent: KernelIntent) -> List[VenueEvent]: def submit(self, intent: KernelIntent) -> List[VenueEvent]:
# Snapshots dropped: receipt executedQty fields take precedence (same as submit_async) # Snapshots dropped: receipt executedQty fields take precedence (same as submit_async)
self._publish_telemetry(
phase="submit:start",
status="REQUESTED",
intent=intent,
endpoint="/openApi/swap/v2/trade/order",
method="POST",
details={"action": intent.action.value, "order_type": str(getattr(intent, "order_type", "MARKET") or "MARKET")},
)
receipt = self._call_backend("submit_intent", self._legacy_intent(intent)) receipt = self._call_backend("submit_intent", self._legacy_intent(intent))
return self._events_from_submit(intent, receipt, None, None) events = self._events_from_submit(intent, receipt, None, None)
ack_row = dict(getattr(receipt, "raw_ack", {}) or {})
self._publish_telemetry(
phase="submit:done",
status=str(getattr(receipt, "status", "") or _row_text(ack_row, "status", default="NEW")),
intent=intent,
endpoint="/openApi/swap/v2/trade/order",
method="POST",
message=_row_text(ack_row, "msg", "message", default=""),
order_id=_row_text(ack_row, "orderId", "orderID", default=str(getattr(receipt, "order_id", "") or "")),
client_order_id=_row_text(ack_row, "clientOrderID", "clientOrderId", default=str(getattr(receipt, "client_order_id", "") or intent.intent_id)),
venue_order_status=str(getattr(receipt, "status", "") or _row_text(ack_row, "status", default="")),
venue_event_kind=events[1].kind.value if len(events) > 1 else events[0].kind.value,
details={
"filled_size": float((events[1].filled_size if len(events) > 1 else events[0].filled_size) or 0.0),
"event_count": len(events),
"asset": intent.asset,
},
)
return events
async def submit_async(self, intent: KernelIntent) -> List[VenueEvent]: async def submit_async(self, intent: KernelIntent) -> List[VenueEvent]:
"""Async submit — runs in the caller's event loop, no thread-pool deadlock. """Async submit — runs in the caller's event loop, no thread-pool deadlock.
@@ -495,8 +638,35 @@ class BingxVenueAdapter(VenueAdapter):
Passing None for snapshots makes _filled_size_from_snapshots return 0.0 Passing None for snapshots makes _filled_size_from_snapshots return 0.0
(a safe fallback; the receipt fields take precedence). (a safe fallback; the receipt fields take precedence).
""" """
self._publish_telemetry(
phase="submit:start",
status="REQUESTED",
intent=intent,
endpoint="/openApi/swap/v2/trade/order",
method="POST",
details={"action": intent.action.value, "order_type": str(getattr(intent, "order_type", "MARKET") or "MARKET")},
)
receipt = await self.backend.submit_intent(self._legacy_intent(intent)) receipt = await self.backend.submit_intent(self._legacy_intent(intent))
return self._events_from_submit(intent, receipt, None, None) events = self._events_from_submit(intent, receipt, None, None)
ack_row = dict(getattr(receipt, "raw_ack", {}) or {})
self._publish_telemetry(
phase="submit:done",
status=str(getattr(receipt, "status", "") or _row_text(ack_row, "status", default="NEW")),
intent=intent,
endpoint="/openApi/swap/v2/trade/order",
method="POST",
message=_row_text(ack_row, "msg", "message", default=""),
order_id=_row_text(ack_row, "orderId", "orderID", default=str(getattr(receipt, "order_id", "") or "")),
client_order_id=_row_text(ack_row, "clientOrderID", "clientOrderId", default=str(getattr(receipt, "client_order_id", "") or intent.intent_id)),
venue_order_status=str(getattr(receipt, "status", "") or _row_text(ack_row, "status", default="")),
venue_event_kind=events[1].kind.value if len(events) > 1 else events[0].kind.value,
details={
"filled_size": float((events[1].filled_size if len(events) > 1 else events[0].filled_size) or 0.0),
"event_count": len(events),
"asset": intent.asset,
},
)
return events
def _events_from_submit(self, intent: KernelIntent, receipt: Any, before, after) -> List[VenueEvent]: # noqa: ANN001 def _events_from_submit(self, intent: KernelIntent, receipt: Any, before, after) -> List[VenueEvent]: # noqa: ANN001
ack_row = dict(getattr(receipt, "raw_ack", {}) or {}) ack_row = dict(getattr(receipt, "raw_ack", {}) or {})
@@ -612,6 +782,18 @@ class BingxVenueAdapter(VenueAdapter):
raw = response if isinstance(response, dict) else {} raw = response if isinstance(response, dict) else {}
status = _normalize_status(_row_text(raw, "status", default="CANCELED")) status = _normalize_status(_row_text(raw, "status", default="CANCELED"))
if status in {"RATE_LIMITED", "THROTTLED"}: if status in {"RATE_LIMITED", "THROTTLED"}:
self._publish_telemetry(
phase="cancel:done",
status=status,
order=order,
endpoint="/openApi/swap/v2/trade/order",
method="DELETE",
message=reason or _row_text(raw, "msg", "message", default="BINGX_RATE_LIMITED"),
venue_order_status=VenueEventStatus.RATE_LIMITED.value,
venue_event_kind=KernelEventKind.RATE_LIMITED.value,
retry_after_ms=_rate_limit_retry_after_ms(raw),
details={"order_status": status, "asset": str(order.metadata.get("asset") or "")},
)
return [ return [
VenueEvent( VenueEvent(
timestamp=datetime.now(timezone.utc), timestamp=datetime.now(timezone.utc),
@@ -637,6 +819,18 @@ class BingxVenueAdapter(VenueAdapter):
kind = KernelEventKind.CANCEL_ACK if event_status == VenueEventStatus.CANCELED else KernelEventKind.CANCEL_REJECT kind = KernelEventKind.CANCEL_ACK if event_status == VenueEventStatus.CANCELED else KernelEventKind.CANCEL_REJECT
if event_status == VenueEventStatus.CANCELED_REJECTED: if event_status == VenueEventStatus.CANCELED_REJECTED:
kind = KernelEventKind.CANCEL_REJECT kind = KernelEventKind.CANCEL_REJECT
self._publish_telemetry(
phase="cancel:done",
status=status or event_status.value,
order=order,
endpoint="/openApi/swap/v2/trade/order",
method="DELETE",
message=reason or _row_text(raw, "msg", "message", default=""),
venue_order_status=event_status.value,
venue_event_kind=kind.value,
retry_after_ms=_rate_limit_retry_after_ms(raw),
details={"order_status": status, "asset": str(order.metadata.get("asset") or "")},
)
return [ return [
VenueEvent( VenueEvent(
timestamp=datetime.now(timezone.utc), timestamp=datetime.now(timezone.utc),

View File

@@ -144,6 +144,54 @@ class VenueOrder:
return max(0.0, float(self.intended_size) - float(self.filled_size)) return max(0.0, float(self.intended_size) - float(self.filled_size))
@dataclass(frozen=True)
class VenueTelemetrySnapshot:
"""Shared-memory surface for venue-side call boundaries and state."""
phase: str = "idle"
status: str = "IDLE"
venue: str = "bingx"
endpoint: str = ""
method: str = ""
intent_id: str = ""
trade_id: str = ""
slot_id: int = 0
asset: str = ""
side: TradeSide = TradeSide.FLAT
action: str = ""
order_id: str = ""
client_order_id: str = ""
venue_order_status: str = ""
venue_event_kind: str = ""
message: str = ""
retry_after_ms: int = 0
timestamp: Optional[datetime] = None
details: Dict[str, Any] = field(default_factory=dict)
def as_dict(self) -> Dict[str, Any]:
return {
"phase": self.phase,
"status": self.status,
"venue": self.venue,
"endpoint": self.endpoint,
"method": self.method,
"intent_id": self.intent_id,
"trade_id": self.trade_id,
"slot_id": int(self.slot_id or 0),
"asset": self.asset,
"side": self.side.value if hasattr(self.side, "value") else str(self.side),
"action": self.action,
"order_id": self.order_id,
"client_order_id": self.client_order_id,
"venue_order_status": self.venue_order_status,
"venue_event_kind": self.venue_event_kind,
"message": self.message,
"retry_after_ms": int(self.retry_after_ms or 0),
"timestamp": self.timestamp.isoformat() if hasattr(self.timestamp, "isoformat") else None,
"details": dict(self.details),
}
@dataclass @dataclass
class TradeSlot: class TradeSlot:
"""A single execution slot managed by the v2 kernel.""" """A single execution slot managed by the v2 kernel."""

View File

@@ -245,9 +245,15 @@ def _build_venue(
mock_scenario: Optional[MockVenueScenario] = None, mock_scenario: Optional[MockVenueScenario] = None,
bingx_config: Optional[BingxExecClientConfig] = None, bingx_config: Optional[BingxExecClientConfig] = None,
bingx_backend: Optional[Any] = None, bingx_backend: Optional[Any] = None,
zinc_plane: Optional[ZincPlane] = None,
venue: Optional[VenueAdapter] = None, venue: Optional[VenueAdapter] = None,
) -> VenueAdapter: ) -> VenueAdapter:
if venue is not None: if venue is not None:
if zinc_plane is not None and hasattr(venue, "set_telemetry_plane"):
try:
venue.set_telemetry_plane(zinc_plane)
except Exception:
pass
return venue return venue
resolved_mode = venue_mode or _resolve_venue_mode() resolved_mode = venue_mode or _resolve_venue_mode()
if resolved_mode is LauncherVenueMode.BINGX: if resolved_mode is LauncherVenueMode.BINGX:
@@ -256,7 +262,7 @@ def _build_venue(
from prod.clean_arch.adapters.bingx_direct import BingxDirectExecutionAdapter from prod.clean_arch.adapters.bingx_direct import BingxDirectExecutionAdapter
backend = BingxDirectExecutionAdapter(bingx_config or build_bingx_exec_client_config()) backend = BingxDirectExecutionAdapter(bingx_config or build_bingx_exec_client_config())
return BingxVenueAdapter(backend=backend) return BingxVenueAdapter(backend=backend, zinc_plane=zinc_plane)
return MockVenueAdapter(mock_scenario) return MockVenueAdapter(mock_scenario)
@@ -342,6 +348,7 @@ def build_launcher_bundle(
mock_scenario=mock_scenario, mock_scenario=mock_scenario,
bingx_config=bingx_config, bingx_config=bingx_config,
bingx_backend=bingx_backend, bingx_backend=bingx_backend,
zinc_plane=active_zinc_plane,
venue=venue, venue=venue,
) )
kernel = ExecutionKernel( kernel = ExecutionKernel(

View File

@@ -8,6 +8,8 @@ import sys
from pathlib import Path from pathlib import Path
from typing import Any, Dict, Optional from typing import Any, Dict, Optional
import threading
from .control import BackendMode, ControlPlane, ControlUpdate, KernelControlSnapshot, KernelMode, KernelVerbosity from .control import BackendMode, ControlPlane, ControlUpdate, KernelControlSnapshot, KernelMode, KernelVerbosity
_ZINC_ADAPTER_PATH = Path(__file__).resolve().parents[3] / "zinc" / "adapters" / "python" _ZINC_ADAPTER_PATH = Path(__file__).resolve().parents[3] / "zinc" / "adapters" / "python"
@@ -70,6 +72,7 @@ class RealZincControlPlane(ControlPlane):
require_real_zinc() require_real_zinc()
base = prefix.strip("/").replace("/", "_") base = prefix.strip("/").replace("/", "_")
self.region_name = f"{base}_control" self.region_name = f"{base}_control"
self._lock = threading.RLock()
self._seq = 0 self._seq = 0
self._snapshot = KernelControlSnapshot() self._snapshot = KernelControlSnapshot()
if create: if create:
@@ -86,21 +89,24 @@ class RealZincControlPlane(ControlPlane):
self.region.close() self.region.close()
def read(self) -> KernelControlSnapshot: def read(self) -> KernelControlSnapshot:
payload = _decode_packet(self.region.as_buffer()) with self._lock:
control = payload.get("control") if isinstance(payload, dict) else None payload = _decode_packet(self.region.as_buffer())
if not isinstance(control, dict): control = payload.get("control") if isinstance(payload, dict) else None
if not isinstance(control, dict):
return self._snapshot
self._snapshot = KernelControlSnapshot(**control)
return self._snapshot return self._snapshot
self._snapshot = KernelControlSnapshot(**control)
return self._snapshot
def update(self, update: ControlUpdate) -> KernelControlSnapshot: def update(self, update: ControlUpdate) -> KernelControlSnapshot:
self._snapshot = update.apply(self.read()) with self._lock:
self._seq += 1 self._snapshot = update.apply(self.read())
self._write_region(self._seq, self._snapshot.as_dict()) self._seq += 1
return self._snapshot self._write_region(self._seq, self._snapshot.as_dict())
return self._snapshot
def mirror(self) -> Dict[str, Any]: def mirror(self) -> Dict[str, Any]:
return self._snapshot.as_dict() with self._lock:
return self._snapshot.as_dict()
def wait(self, timeout_ms: int = 1000) -> bool: def wait(self, timeout_ms: int = 1000) -> bool:
try: try:

View File

@@ -16,7 +16,7 @@ import struct
import sys import sys
import threading import threading
from .contracts import KernelIntent, TradeSide, TradeSlot, TradeStage, VenueOrder, VenueOrderStatus from .contracts import KernelIntent, TradeSide, TradeSlot, TradeStage, VenueOrder, VenueOrderStatus, VenueTelemetrySnapshot
from .control import KernelControlSnapshot from .control import KernelControlSnapshot
_ZINC_ADAPTER_PATH = Path(__file__).resolve().parents[3] / "zinc" / "adapters" / "python" _ZINC_ADAPTER_PATH = Path(__file__).resolve().parents[3] / "zinc" / "adapters" / "python"
@@ -130,6 +130,42 @@ def _decode_packet(buf: memoryview) -> Dict[str, Any]:
return out return out
def _venue_from_payload(payload: Dict[str, Any]) -> VenueTelemetrySnapshot:
timestamp = payload.get("timestamp")
ts_value = None
if isinstance(timestamp, str) and timestamp:
try:
ts_value = datetime.fromisoformat(timestamp)
except Exception:
ts_value = None
try:
side = TradeSide(str(payload.get("side", TradeSide.FLAT.value)))
except Exception:
side = TradeSide.FLAT
details = payload.get("details", {})
return VenueTelemetrySnapshot(
phase=str(payload.get("phase", "idle")),
status=str(payload.get("status", "IDLE")),
venue=str(payload.get("venue", "bingx")),
endpoint=str(payload.get("endpoint", "")),
method=str(payload.get("method", "")),
intent_id=str(payload.get("intent_id", "")),
trade_id=str(payload.get("trade_id", "")),
slot_id=int(payload.get("slot_id", 0) or 0),
asset=str(payload.get("asset", "")),
side=side,
action=str(payload.get("action", "")),
order_id=str(payload.get("order_id", "")),
client_order_id=str(payload.get("client_order_id", "")),
venue_order_status=str(payload.get("venue_order_status", "")),
venue_event_kind=str(payload.get("venue_event_kind", "")),
message=str(payload.get("message", "")),
retry_after_ms=int(payload.get("retry_after_ms", 0) or 0),
timestamp=ts_value,
details=dict(details) if isinstance(details, dict) else {},
)
class RealZincPlane: class RealZincPlane:
"""Shared-memory Zinc plane used by the Python prototype.""" """Shared-memory Zinc plane used by the Python prototype."""
@@ -148,18 +184,22 @@ class RealZincPlane:
self.intent_name = f"{base}_intent" self.intent_name = f"{base}_intent"
self.state_name = f"{base}_state" self.state_name = f"{base}_state"
self.control_name = f"{base}_control" self.control_name = f"{base}_control"
self.venue_name = f"{base}_venue"
self._intent_seq = 0 self._intent_seq = 0
self._state_seq = 0 self._state_seq = 0
self._control_seq = 0 self._control_seq = 0
self._venue_seq = 0
self._lock = threading.Lock() self._lock = threading.Lock()
self._slot_cache: Dict[int, TradeSlot] = {i: TradeSlot(slot_id=i) for i in range(int(slot_count))} self._slot_cache: Dict[int, TradeSlot] = {i: TradeSlot(slot_id=i) for i in range(int(slot_count))}
self._slot_count = int(slot_count) self._slot_count = int(slot_count)
self._intent_cache: List[Dict[str, Any]] = [] self._intent_cache: List[Dict[str, Any]] = []
self._control_cache = KernelControlSnapshot() self._control_cache = KernelControlSnapshot()
self._venue_cache = VenueTelemetrySnapshot()
if create: if create:
self.intent_region = SharedRegion.create(self.intent_name, intent_capacity) self.intent_region = SharedRegion.create(self.intent_name, intent_capacity)
self.state_region = SharedRegion.create(self.state_name, state_capacity) self.state_region = SharedRegion.create(self.state_name, state_capacity)
self.control_region = SharedRegion.create(self.control_name, control_capacity) self.control_region = SharedRegion.create(self.control_name, control_capacity)
self.venue_region = SharedRegion.create(self.venue_name, control_capacity)
self._write_region(self.control_region, self._control_seq, {"control": self._control_cache.as_dict()}) self._write_region(self.control_region, self._control_seq, {"control": self._control_cache.as_dict()})
self._write_region( self._write_region(
self.state_region, self.state_region,
@@ -167,13 +207,16 @@ class RealZincPlane:
{"slots": [self._slot_cache[key].to_dict() for key in range(self._slot_count)]}, {"slots": [self._slot_cache[key].to_dict() for key in range(self._slot_count)]},
) )
self._write_region(self.intent_region, self._intent_seq, {"items": []}) self._write_region(self.intent_region, self._intent_seq, {"items": []})
self._write_region(self.venue_region, self._venue_seq, {"venue": self._venue_cache.as_dict()})
else: else:
self.intent_region = SharedRegion.open(self.intent_name) self.intent_region = SharedRegion.open(self.intent_name)
self.state_region = SharedRegion.open(self.state_name) self.state_region = SharedRegion.open(self.state_name)
self.control_region = SharedRegion.open(self.control_name) self.control_region = SharedRegion.open(self.control_name)
self.venue_region = SharedRegion.open(self.venue_name)
control_payload = _decode_packet(self.control_region.as_buffer()) control_payload = _decode_packet(self.control_region.as_buffer())
state_payload = _decode_packet(self.state_region.as_buffer()) state_payload = _decode_packet(self.state_region.as_buffer())
intent_payload = _decode_packet(self.intent_region.as_buffer()) intent_payload = _decode_packet(self.intent_region.as_buffer())
venue_payload = _decode_packet(self.venue_region.as_buffer())
if isinstance(control_payload.get("control"), dict): if isinstance(control_payload.get("control"), dict):
self._control_cache = KernelControlSnapshot(**control_payload["control"]) self._control_cache = KernelControlSnapshot(**control_payload["control"])
if isinstance(state_payload.get("slots"), list): if isinstance(state_payload.get("slots"), list):
@@ -183,11 +226,14 @@ class RealZincPlane:
self._slot_cache[int(slot.slot_id)] = slot self._slot_cache[int(slot.slot_id)] = slot
if isinstance(intent_payload.get("items"), list): if isinstance(intent_payload.get("items"), list):
self._intent_cache = list(intent_payload["items"]) self._intent_cache = list(intent_payload["items"])
if isinstance(venue_payload.get("venue"), dict):
self._venue_cache = _venue_from_payload(venue_payload["venue"])
def close(self) -> None: def close(self) -> None:
self.intent_region.close() self.intent_region.close()
self.state_region.close() self.state_region.close()
self.control_region.close() self.control_region.close()
self.venue_region.close()
def publish_intent(self, intent: KernelIntent) -> None: def publish_intent(self, intent: KernelIntent) -> None:
with self._lock: with self._lock:
@@ -212,14 +258,16 @@ class RealZincPlane:
self._write_region(self.state_region, self._state_seq, payload) self._write_region(self.state_region, self._state_seq, payload)
def read_slots(self) -> List[TradeSlot]: def read_slots(self) -> List[TradeSlot]:
payload = _decode_packet(self.state_region.as_buffer()) with self._lock:
slots = payload.get("slots", []) if isinstance(payload, dict) else [] payload = _decode_packet(self.state_region.as_buffer())
return [_slot_from_payload(slot) for slot in sorted(slots, key=lambda row: int(row.get("slot_id", 0)))] slots = payload.get("slots", []) if isinstance(payload, dict) else []
return [_slot_from_payload(slot) for slot in sorted(slots, key=lambda row: int(row.get("slot_id", 0)))]
def read_intents(self) -> List[Dict[str, Any]]: def read_intents(self) -> List[Dict[str, Any]]:
payload = _decode_packet(self.intent_region.as_buffer()) with self._lock:
items = payload.get("items", []) if isinstance(payload, dict) else [] payload = _decode_packet(self.intent_region.as_buffer())
return list(items) items = payload.get("items", []) if isinstance(payload, dict) else []
return list(items)
def update_control(self, control: KernelControlSnapshot) -> None: def update_control(self, control: KernelControlSnapshot) -> None:
with self._lock: with self._lock:
@@ -228,11 +276,12 @@ class RealZincPlane:
self._write_region(self.control_region, self._control_seq, {"control": control.as_dict()}) self._write_region(self.control_region, self._control_seq, {"control": control.as_dict()})
def read_control(self) -> KernelControlSnapshot: def read_control(self) -> KernelControlSnapshot:
payload = _decode_packet(self.control_region.as_buffer()) with self._lock:
control = payload.get("control") if isinstance(payload, dict) else None payload = _decode_packet(self.control_region.as_buffer())
if not isinstance(control, dict): control = payload.get("control") if isinstance(payload, dict) else None
return self._control_cache if not isinstance(control, dict):
return KernelControlSnapshot(**control) return self._control_cache
return KernelControlSnapshot(**control)
def wait_on_state(self, timeout_ms: int = 1000) -> bool: def wait_on_state(self, timeout_ms: int = 1000) -> bool:
return bool(self.state_region.wait(timeout_ms)) return bool(self.state_region.wait(timeout_ms))
@@ -246,6 +295,26 @@ class RealZincPlane:
def notify_control(self) -> None: def notify_control(self) -> None:
self.control_region.notify() self.control_region.notify()
def publish_venue(self, telemetry: VenueTelemetrySnapshot) -> None:
with self._lock:
self._venue_seq += 1
self._venue_cache = telemetry
self._write_region(self.venue_region, self._venue_seq, {"venue": telemetry.as_dict()})
def read_venue(self) -> VenueTelemetrySnapshot:
payload = _decode_packet(self.venue_region.as_buffer())
venue = payload.get("venue") if isinstance(payload, dict) else None
if not isinstance(venue, dict):
return self._venue_cache
self._venue_cache = _venue_from_payload(venue)
return self._venue_cache
def wait_on_venue(self, timeout_ms: int = 1000) -> bool:
return bool(self.venue_region.wait(timeout_ms))
def notify_venue(self) -> None:
self.venue_region.notify()
def wait_on_intent(self, timeout_ms: int = 1000) -> bool: def wait_on_intent(self, timeout_ms: int = 1000) -> bool:
return bool(self.intent_region.wait(timeout_ms)) return bool(self.intent_region.wait(timeout_ms))

View File

@@ -107,6 +107,9 @@ def _crate_dir() -> Path:
return Path(__file__).resolve().with_name("_rust_kernel") return Path(__file__).resolve().with_name("_rust_kernel")
_LOCAL_TARGET_DIR = Path("/root/.cargo/dita_v2_target")
def _library_path() -> Path: def _library_path() -> Path:
if sys.platform == "darwin": if sys.platform == "darwin":
name = "libdita_v2_kernel.dylib" name = "libdita_v2_kernel.dylib"
@@ -114,6 +117,9 @@ def _library_path() -> Path:
name = "dita_v2_kernel.dll" name = "dita_v2_kernel.dll"
else: else:
name = "libdita_v2_kernel.so" name = "libdita_v2_kernel.so"
local = _LOCAL_TARGET_DIR / "release" / name
if local.exists():
return local
return _crate_dir() / "target" / "release" / name return _crate_dir() / "target" / "release" / name
@@ -121,10 +127,13 @@ def _build_library() -> None:
crate_dir = _crate_dir() crate_dir = _crate_dir()
if not crate_dir.exists(): if not crate_dir.exists():
raise FileNotFoundError(f"Missing Rust kernel crate: {crate_dir}") raise FileNotFoundError(f"Missing Rust kernel crate: {crate_dir}")
_LOCAL_TARGET_DIR.mkdir(parents=True, exist_ok=True)
env = {**os.environ, "CARGO_TARGET_DIR": str(_LOCAL_TARGET_DIR)}
subprocess.run( subprocess.run(
["cargo", "build", "--release", "--manifest-path", str(crate_dir / "Cargo.toml")], ["cargo", "build", "--release", "--manifest-path", str(crate_dir / "Cargo.toml")],
cwd=_repo_root(), cwd=_repo_root(),
check=True, check=True,
env=env,
) )

View File

@@ -13,6 +13,7 @@ from __future__ import annotations
import math import math
import sys import sys
from concurrent.futures import ThreadPoolExecutor
sys.path.insert(0, "/mnt/dolphinng5_predict") sys.path.insert(0, "/mnt/dolphinng5_predict")
import pytest import pytest
@@ -324,7 +325,46 @@ class TestReplayDeterminism:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# 7. V1 backward compatibility (AccountProjection must be untouched) # 7. Concurrency guard
# ---------------------------------------------------------------------------
class TestConcurrencyGuard:
def test_apply_fill_is_serialized(self):
proj = _proj(10_000.0)
n_threads = 16
per_thread = 250
total_fee = 0.0
total_realized = 0.0
def _worker(tid: int) -> tuple[float, float]:
local_fee = 0.0
local_realized = 0.0
for i in range(per_thread):
realized = float(tid * per_thread + i)
fee = float((i % 5) * 0.1)
proj.apply_fill(
fill_price=100.0,
fill_qty=1.0,
fee=fee,
realized_pnl=realized,
)
local_fee += fee
local_realized += realized
return local_realized, local_fee
with ThreadPoolExecutor(max_workers=n_threads) as ex:
for realized, fee in ex.map(_worker, range(n_threads)):
total_realized += realized
total_fee += fee
snap = _snap(proj)
assert snap.k.realized_pnl == pytest.approx(total_realized)
assert snap.k.fees_paid == pytest.approx(total_fee)
assert snap.k.capital == pytest.approx(10_000.0 + total_realized - total_fee)
# ---------------------------------------------------------------------------
# 8. V1 backward compatibility (AccountProjection must be untouched)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
class TestV1Compat: class TestV1Compat:

View File

@@ -0,0 +1,169 @@
"""ASEx AccountProjectionV2 wrapper tests — race-proof, stress, seam."""
from __future__ import annotations
import gc
import math
import threading
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from types import SimpleNamespace
import pytest
from .account import AccountProjectionV2, AccountSnapshotV2, EPosition, TradeStage, TradeSide
from .asex_account import ASEXAccountV2, _AccountBackend
from asex.guarded import ValidationError
def _empty_slots(n=1):
return [SimpleNamespace(
slot_id=i, trade_id=f"t{i}", asset="BTCUSDT",
side=TradeSide.LONG, entry_price=0.0, size=0.0,
initial_size=0.0, leverage=1.0, realized_pnl=0.0, closed=False,
fsm_state=TradeStage.IDLE, exit_leg_ratios=(1.0,),
active_leg_index=0, active_exit_order=None, active_entry_order=None,
close_reason="", entry_time=None, last_event_time=None,
seen_event_ids=(), metadata={}, unrealized_pnl=0.0, to_dict=lambda: {},
) for i in range(n)]
def _clean():
gc.collect(); gc.collect()
def _force_race(obj, attr, n=10):
b = threading.Barrier(n)
def _w():
b.wait(); v = getattr(obj, attr); b.wait(); setattr(obj, attr, v + 1)
ts = [threading.Thread(target=_w) for _ in range(n)]
for t in ts: t.start()
for t in ts: t.join(timeout=10)
return getattr(obj, attr)
class TestRaceProof:
def test_k_realized_races(self):
assert _force_race(AccountProjectionV2(0.0), "_k_realized", 50) == 1
def test_k_fees_races(self):
assert _force_race(AccountProjectionV2(0.0), "_k_fees", 50) == 1
def test_k_funding_races(self):
assert _force_race(AccountProjectionV2(0.0), "_k_funding", 50) == 1
def test_event_seq_races(self):
assert _force_race(AccountProjectionV2(0.0), "_event_seq", 50) == 1
class TestASExBasic:
def test_seed_only(self):
_clean(); p = ASEXAccountV2(10000.0)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(10000.0); p.close(); _clean()
def test_realized_adds(self):
_clean(); p = ASEXAccountV2(10000.0)
p.apply_fill(fill_price=100, fill_qty=1, fee=0, realized_pnl=500)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(10500.0); p.close(); _clean()
def test_fee_subtracts(self):
_clean(); p = ASEXAccountV2(10000.0)
p.apply_fill(fill_price=100, fill_qty=1, fee=3.5, realized_pnl=0)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(9996.5); p.close(); _clean()
def test_funding_subtracts(self):
_clean(); p = ASEXAccountV2(10000.0)
p.apply_funding(7.25)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(9992.75); p.close(); _clean()
def test_combined(self):
_clean(); p = ASEXAccountV2(10000.0)
p.apply_fill(fill_price=50, fill_qty=2, fee=2, realized_pnl=100)
p.apply_funding(5)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.capital == pytest.approx(10093.0)
p.close(); _clean()
class TestASExConcurrency:
@pytest.mark.parametrize("n,ops", [(10, 100), (20, 100), (50, 50)])
def test_no_lost_updates(self, n, ops):
_clean(); p = ASEXAccountV2(0.0)
def _w(tid):
for i in range(ops):
p.apply_fill(fill_price=100, fill_qty=1, fee=0, realized_pnl=float(tid * ops + i))
with ThreadPoolExecutor(max_workers=n) as ex:
for f in as_completed([ex.submit(_w, i) for i in range(n)]): f.result(timeout=60)
expected = sum(tid * ops + i for tid in range(n) for i in range(ops))
assert p._backend._proj._k_realized == pytest.approx(float(expected))
p.close(); _clean()
def test_event_seq_monotonic(self):
_clean(); p = ASEXAccountV2(0.0); seqs = []
def _w(tid):
for i in range(50):
s = p.build_snapshot(source_event_id=f"e{tid}_{i}", slots=_empty_slots(), ts=float(i))
seqs.append(s.event_seq)
with ThreadPoolExecutor(max_workers=10) as ex:
for f in as_completed([ex.submit(_w, i) for i in range(10)]): f.result(timeout=60)
assert sorted(seqs) == list(range(1, 501))
p.close(); _clean()
def test_mixed_ops(self):
_clean(); p = ASEXAccountV2(10000.0)
def _f(tid):
for i in range(100): p.apply_fill(fill_price=float(i), fill_qty=1, fee=0, realized_pnl=1)
def _fu(tid):
for i in range(50): p.apply_funding(0.5)
with ThreadPoolExecutor(max_workers=4) as ex:
for f in as_completed([ex.submit(_f, i) for i in [0,1]] + [ex.submit(_fu, i) for i in [0,1]]):
f.result(timeout=60)
assert p._backend._proj._k_realized == pytest.approx(200.0)
# 2 workers x 50 calls x 0.5 = 50.0; exact total proves zero lost updates
assert p._backend._proj._k_funding == pytest.approx(50.0)
p.close(); _clean()
class TestASExSeam:
def test_backend_rejects_no_op(self):
assert not _AccountBackend(0.0)._validate({})
def test_backend_rejects_unknown(self):
b = _AccountBackend(0.0)
with pytest.raises(Exception): b.mutate({"op": "nope"})
def test_backend_applied(self):
b = _AccountBackend(0.0)
b.mutate({"op": "apply_fill", "args": {"fill_price": 100, "fill_qty": 1, "fee": 0, "realized_pnl": 10}})
assert b.applied == 1
def test_worker_alive(self):
p = ASEXAccountV2(0.0); assert p._worker._worker.is_alive(); p.close()
def test_double_close(self):
p = ASEXAccountV2(0.0); p.close(); p.close()
def test_context_manager(self):
with ASEXAccountV2(10000.0) as p:
p.apply_fill(fill_price=100, fill_qty=1, fee=0, realized_pnl=50)
s = p.build_snapshot(source_event_id="t", slots=_empty_slots(), ts=1e6)
assert s.k.realized_pnl == pytest.approx(50.0)
def test_no_thread_leak(self):
_clean(); b = threading.active_count()
for _ in range(50):
p = ASEXAccountV2(0.0); p.apply_fill(fill_price=100, fill_qty=1, fee=0, realized_pnl=1); p.close()
_clean(); assert threading.active_count() - b <= 2
class TestLockProof:
def test_k_realized_lock_proof(self):
ap, l, n = AccountProjectionV2(0.0), threading.Lock(), 50
b = threading.Barrier(n)
def _w():
b.wait()
with l: ap._k_realized += 1
ts = [threading.Thread(target=_w) for _ in range(n)]
for t in ts: t.start()
for t in ts: t.join(timeout=10)
assert ap._k_realized == n
def test_event_seq_lock_proof(self):
ap, l, n = AccountProjectionV2(0.0), threading.Lock(), 50
b = threading.Barrier(n)
def _w():
b.wait()
with l: ap._event_seq += 1
ts = [threading.Thread(target=_w) for _ in range(n)]
for t in ts: t.start()
for t in ts: t.join(timeout=10)
assert ap._event_seq == n

View File

@@ -12,7 +12,7 @@ from typing import Any, Dict, Iterable, List, Mapping, Optional, Protocol
import threading import threading
import time import time
from .contracts import KernelIntent, TradeSlot from .contracts import KernelIntent, TradeSlot, VenueTelemetrySnapshot
from .control import KernelControlSnapshot from .control import KernelControlSnapshot
@@ -52,6 +52,18 @@ class ZincPlane(Protocol):
def notify_control(self) -> None: def notify_control(self) -> None:
... ...
def publish_venue(self, telemetry: VenueTelemetrySnapshot) -> None:
...
def read_venue(self) -> VenueTelemetrySnapshot:
...
def wait_on_venue(self, timeout_ms: int = 1000) -> bool:
...
def notify_venue(self) -> None:
...
@dataclass @dataclass
class InMemoryZincPlane: class InMemoryZincPlane:
@@ -60,12 +72,15 @@ class InMemoryZincPlane:
intent_region: List[KernelIntent] = field(default_factory=list) intent_region: List[KernelIntent] = field(default_factory=list)
state_region: Dict[int, TradeSlot] = field(default_factory=dict) state_region: Dict[int, TradeSlot] = field(default_factory=dict)
control_region: Optional[KernelControlSnapshot] = None control_region: Optional[KernelControlSnapshot] = None
venue_region: VenueTelemetrySnapshot = field(default_factory=VenueTelemetrySnapshot)
_intent_seq: int = field(default=0, init=False, repr=False) _intent_seq: int = field(default=0, init=False, repr=False)
_state_seq: int = field(default=0, init=False, repr=False) _state_seq: int = field(default=0, init=False, repr=False)
_control_seq: int = field(default=0, init=False, repr=False) _control_seq: int = field(default=0, init=False, repr=False)
_venue_seq: int = field(default=0, init=False, repr=False)
_intent_observed_seq: int = field(default=0, init=False, repr=False) _intent_observed_seq: int = field(default=0, init=False, repr=False)
_state_observed_seq: int = field(default=0, init=False, repr=False) _state_observed_seq: int = field(default=0, init=False, repr=False)
_control_observed_seq: int = field(default=0, init=False, repr=False) _control_observed_seq: int = field(default=0, init=False, repr=False)
_venue_observed_seq: int = field(default=0, init=False, repr=False)
_signal: threading.Condition = field(default_factory=threading.Condition, init=False, repr=False) _signal: threading.Condition = field(default_factory=threading.Condition, init=False, repr=False)
def publish_intent(self, intent: KernelIntent) -> None: def publish_intent(self, intent: KernelIntent) -> None:
@@ -118,6 +133,23 @@ class InMemoryZincPlane:
self._control_seq += 1 self._control_seq += 1
self._signal.notify_all() self._signal.notify_all()
def publish_venue(self, telemetry: VenueTelemetrySnapshot) -> None:
with self._signal:
self.venue_region = telemetry
self._venue_seq += 1
self._signal.notify_all()
def read_venue(self) -> VenueTelemetrySnapshot:
return self.venue_region
def wait_on_venue(self, timeout_ms: int = 1000) -> bool:
return self._wait_for_change("_venue_seq", "_venue_observed_seq", timeout_ms)
def notify_venue(self) -> None:
with self._signal:
self._venue_seq += 1
self._signal.notify_all()
def _wait_for_change(self, seq_attr: str, observed_attr: str, timeout_ms: int) -> bool: def _wait_for_change(self, seq_attr: str, observed_attr: str, timeout_ms: int) -> bool:
timeout_s = None if timeout_ms is None or timeout_ms < 0 else max(0.0, timeout_ms / 1000.0) timeout_s = None if timeout_ms is None or timeout_ms < 0 else max(0.0, timeout_ms / 1000.0)
deadline = None if timeout_s is None else time.monotonic() + timeout_s deadline = None if timeout_s is None else time.monotonic() + timeout_s

View File

@@ -5,21 +5,39 @@ This is VIOLET-only. BLUE is untouched.
The adapter reads the published BLUE surfaces that already exist in HZ and The adapter reads the published BLUE surfaces that already exist in HZ and
translates them into ``SizingFactors`` for the shadow path: translates them into ``SizingFactors`` for the shadow path:
- ``posture`` from ``DOLPHIN_STATE_BLUE.latest_nautilus`` / ``engine_snapshot`` - ``posture`` from ``DOLPHIN_STATE_BLUE.latest_nautilus`` / ``engine_snapshot``
- ``esof_score`` from ``DOLPHIN_FEATURES.esof_latest`` or ``esof_advisor_latest`` - ``esof_score`` from ``DOLPHIN_FEATURES.esof_latest`` or ``esof_advisor_latest`` via
- ``acb_boost`` / ``acb_beta`` from ``DOLPHIN_FEATURES.acb_boost`` BLUE's own ``parse_esof_payload`` / ``esof_score_from_payload``
- ``mc_scale`` from ``DOLPHIN_FEATURES.mc_forewarner_latest`` - ``boost`` / ``beta`` RECOMPUTED via ``AdaptiveCircuitBreaker.get_dynamic_boost_from_hz``
- OB market consensus from the live ``asset_*_ob`` maps via BLUE's own over ``DOLPHIN_FEATURES.exf_latest`` + ``latest_eigen_scan.w750_velocity`` — IDENTICAL
to the trader's on_exf_update path (NOT the published ``acb_boost`` scalar)
- ``mc_scale`` from ``DOLPHIN_FEATURES.mc_forewarner_latest`` via ``_derive_mc_scale``
(begin_day's cat/env thresholds, not the MC service's status label)
- OB market consensus from the live ``asset_*_ob`` maps via BLUE's own ``HZOBProvider`` +
``OBFeatureEngine`` ``OBFeatureEngine``
- ``dc_status`` via BLUE's ``AlphaSignalGenerator`` (params pinned to BLUE's ENGINE_KWARGS)
over the replayed scan price-history
The remaining DC signal is left neutral here for now. It needs the same live PARITY (see prod/docs/VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md): this module reconstructs
signal-history path BLUE uses and should be added as a separate mirror step. BLUE's factors in a DIFFERENT file/scope structure than BLUE's monolithic NDAlphaEngine.
Every kernel is now WRAPPED, not copied (ACB, OBFeatureEngine+HZOBProvider, AlphaSignalGenerator,
VioletAssetSelector) — so the only hand-replicated arithmetic left is ``_derive_mc_scale``
(begin_day's thresholds; pinned by test). Remaining REAL fidelity caveats:
- OB faithfulness needs a PERSISTENT ``ob_engine`` + per-scan ``bar_idx`` (OBFeatureEngine
accumulates a lookback window); a single-shot engine has no cross-scan history.
- On stale exf (>12h) the ACB raises ValueError; BLUE keeps the prior boost/beta, VIOLET
has no prior → neutral (1.0, 0.0).
Any change to BLUE's ENGINE_KWARGS or begin_day mc thresholds REQUIRES a matching change +
test here (see test_signal_gen_params_match_blue_engine_kwargs / the mc_scale formula test).
""" """
from __future__ import annotations from __future__ import annotations
import json import json
import logging
import os as _os
import sys import sys
from collections import deque from collections import deque
from datetime import datetime, timezone
from collections.abc import Mapping from collections.abc import Mapping
from dataclasses import dataclass from dataclasses import dataclass
from dataclasses import field from dataclasses import field
@@ -35,14 +53,48 @@ for _p in (str(_PROJECT_ROOT), str(_PROJECT_ROOT / "nautilus_dolphin")):
sys.path.insert(0, _p) sys.path.insert(0, _p)
from nautilus_dolphin.nautilus.ob_features import OBFeatureEngine from nautilus_dolphin.nautilus.ob_features import OBFeatureEngine
from nautilus_dolphin.nautilus.ob_provider import OBSnapshot, OBProvider from nautilus_dolphin.nautilus.hz_ob_provider import HZOBProvider
from nautilus_dolphin.nautilus.alpha_signal_generator import AlphaSignalGenerator from nautilus_dolphin.nautilus.adaptive_circuit_breaker import AdaptiveCircuitBreaker
from nautilus_dolphin.nautilus.alpha_signal_generator import (
AlphaSignalGenerator,
VEL_DIV_THRESHOLD, VEL_DIV_EXTREME,
LONG_VEL_DIV_THRESHOLD, LONG_VEL_DIV_EXTREME,
)
from .alpha_wrappers import VioletAssetSelector from .alpha_wrappers import VioletAssetSelector
from .decision_engine import SizingFactors from .decision_engine import SizingFactors
from .live_factor_source import esof_score_from_features, posture_from_engine_snapshot from .live_factor_source import esof_score_from_features, posture_from_engine_snapshot
from .live_factors import extract_live_sizing_factors from .live_factors import extract_live_sizing_factors
LOGGER = logging.getLogger("violet.live_blue_source")
# Hazelcast coordinates — MUST equal nautilus_event_trader.py:107-108 (BLUE's live
# cluster). HZOBProvider opens its own connection to these, exactly as BLUE's _wire_obf.
# TODO_HZBRIDGE: all VIOLET HZ access (the caller's client + HZOBProvider) must move to
# dolphinng5_predict/hzbridge when it ships — raw HazelcastClient is the silent-death/lockup
# surface (see prod/docs/VIOLET_OB_FEED_AND_AGENT_COORDINATION.md + hz_client_death memory).
HZ_CLUSTER = _os.environ.get("HZ_CLUSTER", "dolphin")
HZ_HOST = _os.environ.get("HZ_HOST", "127.0.0.1:5701")
# AlphaSignalGenerator construction — pinned to BLUE's live ENGINE_KWARGS
# (nautilus_event_trader.py:128-133). These equal AlphaSignalGenerator's own defaults
# TODAY, but BLUE constructs it EXPLICITLY from ENGINE_KWARGS, so a future champion
# retune (e.g. dc_lookback_bars→10) would silently diverge a bare AlphaSignalGenerator().
# We pin explicitly and assert the pin in test_signal_gen_params_match_blue_engine_kwargs.
# vel_div_* import the module constants directly so they auto-track the kernel.
BLUE_SIGNAL_GEN_KWARGS = dict(
vel_div_threshold=VEL_DIV_THRESHOLD, # -0.02
vel_div_extreme=VEL_DIV_EXTREME, # -0.05
long_vel_div_threshold=LONG_VEL_DIV_THRESHOLD, # 0.01
long_vel_div_extreme=LONG_VEL_DIV_EXTREME, # 0.04
dc_lookback_bars=7,
dc_min_magnitude_bps=0.75,
dc_skip_contradicts=True,
dc_leverage_boost=1.0,
dc_leverage_reduce=0.5,
use_direction_confirm=True,
)
def _jsonish(value: Any) -> Any: def _jsonish(value: Any) -> Any:
if isinstance(value, str): if isinstance(value, str):
@@ -72,22 +124,115 @@ def _read_hz_map(client: hazelcast.HazelcastClient, map_name: str, key: str) ->
return None return None
def _map_status_to_mc_scale(payload: Any) -> float: def _derive_mc_scale(payload: Any) -> float:
"""Mirror BLUE ``begin_day``'s ``mc_scale`` EXACTLY (esf_alpha_orchestrator.py:956-962).
BLUE does NOT use the MC service's published ``status`` label for sizing — that label
(mc_forewarner_flow.py: GREEN<0.10 / ORANGE<0.30 / RED) is observability-only and uses
DIFFERENT thresholds from the engine. The live trader re-derives the size haircut from
the SAME published fields ``catastrophic_prob`` + ``envelope_score`` with the engine's
own thresholds:
mc_red = catastrophic_prob > 0.25 or envelope_score < -1.0
mc_orange = (not mc_red) and (envelope_score < 0 or catastrophic_prob > 0.10)
mc_scale = 0.5 if mc_orange else 1.0 # RED → 1.0 here; BLUE halts via regime_dd_halt
Reading ``status`` instead diverges (e.g. cat=0.05/env=-0.5 → publisher GREEN→1.0 but
BLUE orange→0.5).
AMBIGUITY (flagged 2026-06-16): the MC service's ``status`` and the engine's begin_day
thresholds genuinely disagree, and from OUTSIDE BLUE there is no way to know which is the
"nominal" intent — they are two independent threshold sets over the same numbers. We go
with the SOURCE FIELDS + begin_day formula because that is the path that actually drives
BLUE's live sizing (the published ``status`` is consumed only by the TUI/observability).
If BLUE's begin_day thresholds change, THIS must change with them. Operator confirmation
of the canonical surface is still desirable.
Missing/unparseable fields → neutral 1.0 (no haircut)."""
data = _jsonish(payload) data = _jsonish(payload)
if isinstance(data, Mapping): if not isinstance(data, Mapping):
status = str(data.get("status", "")).upper() return 1.0
else: cat = _coerce_float(data.get("catastrophic_prob"), None)
status = str(data).upper() env = _coerce_float(data.get("envelope_score"), None)
return 0.5 if status == "ORANGE" else 1.0 if cat is None or env is None:
return 1.0
mc_red = cat > 0.25 or env < -1.0
mc_orange = (not mc_red) and (env < 0.0 or cat > 0.10)
return 0.5 if mc_orange else 1.0
def _extract_acb(payload: Any) -> tuple[float, float]: # Inverse-ACB neutral identity — used ONLY at cold start (no prior yet AND no fresh exf).
data = _jsonish(payload) # In continuous operation exf is warmed, so the first call seeds the prior and this is
if isinstance(data, Mapping): # never the steady-state value.
boost = _coerce_float(data.get("boost"), 1.0) or 1.0 _BOOST_BETA_NEUTRAL = (1.0, 0.0)
beta = _coerce_float(data.get("beta"), 0.0) or 0.0
return max(0.0, boost), max(0.0, beta)
return 1.0, 0.0 def _source_boost_beta(
client: hazelcast.HazelcastClient,
*,
date_str: str,
trade_direction: int,
acb: Optional[AdaptiveCircuitBreaker] = None,
prior: Optional[tuple[float, float]] = None,
) -> tuple[float, float]:
"""Recompute (boost, beta) EXACTLY as BLUE's live trader does — NOT from acb_boost.
WHAT THE ACB IS DOING, IN EFFECT (AdaptiveCircuitBreaker v6, "inverse" mode):
- ``boost`` = a DAILY position-size governor >= 1.0 driven by EXTERNAL FACTORS (ExF:
funding_btc/dvol_btc/fng/taker from DOLPHIN_FEATURES.exf_latest). When stress
signals fire, ``boost = 1 + 0.5*ln(1+signals)``; else 1.0. INVERSE = it leans size
UP under stress, it does not cut (adaptive_circuit_breaker.py:591-593).
- ``beta`` = regime sensitivity in {BETA_HIGH=0.8, BETA_LOW=0.2}, keyed on the w750
eigenvalue-velocity (>= threshold → HIGH). It sets how strongly per-bar signal
strength amplifies size: regime_size_mult = base_boost*(1 + beta*strength^3)*mc_scale.
- Both become the engine's _day_base_boost / _day_beta via update_acb_boost
(esf_alpha_orchestrator.py:771-772), which then feed BLUE's SIZING directly:
_update_regime_size_mult (:898-909) = _day_base_boost * (1 + _day_beta*strength^3)
* _day_mc_scale. So the ACB IS in the sizing layer — confirmed, not incidental.
BLUE's LIVE path passes NO ob_engine (on_exf_update:4769 / rollover prewarm:2710), so the
ACB's OB Sub-4 macro-regime beta modulation (x1.25 stress / x0.85 calm,
adaptive_circuit_breaker.py:613-628) is DORMANT in live — only the NPZ backtest path
get_dynamic_boost_for_date applies it. Per operator recollection (2026-06-16), OB Sub-4
beta modulation was found NON-PERFORMANT and deliberately removed/bypassed — so passing
no ob_engine is INTENTIONAL design, not an oversight. We pass no ob_engine to match BLUE
bit-for-bit either way.
# TODO_SOMEDAY: find the documentary confirmation of the OB Sub-4 non-performant bypass
# (operator: likely SYSTEM_BIBLE v7 lineage or a deep code comment / the in-progress
# INDEX_DOLPHINNG5_PREDICT doc). Behaviour is already bit-identical to live BLUE.
The published DOLPHIN_FEATURES.acb_boost is acb_processor_service's SEPARATE daily value
and is never read here.
STALE / MISSING exf: BLUE keeps the prior _day_base_boost/_day_beta (update_acb_boost
simply isn't called on the stale branch), so VIOLET KEEPS ``prior`` too. ``prior`` is
seeded by the first successful compute — exf is warmed in continuous BLUE operation, so
the steady state always has a prior; only a cold start with no prior AND no fresh exf
falls back to the neutral identity."""
fallback = prior if prior is not None else _BOOST_BETA_NEUTRAL
exf = _jsonish(_read_hz_map(client, "DOLPHIN_FEATURES", "exf_latest"))
if not isinstance(exf, Mapping):
LOGGER.debug("ACB: no exf_latest — keeping prior boost/beta=%s", fallback)
return fallback
eigen = _jsonish(_read_hz_map(client, "DOLPHIN_FEATURES", "latest_eigen_scan"))
w750 = _coerce_float(eigen.get("w750_velocity"), None) if isinstance(eigen, Mapping) else None
acb = acb or AdaptiveCircuitBreaker()
try:
info = acb.get_dynamic_boost_from_hz(
date_str=date_str,
exf_snapshot=dict(exf),
w750_velocity=float(w750) if w750 else None, # 0.0 → None, matches BLUE
direction=trade_direction, # NO ob_engine — matches BLUE live
)
except ValueError as exc:
# BLUE logs "ACB Stale Data Fallback" and keeps the prior boost/beta.
LOGGER.info("ACB Stale Data Fallback (%s) — keeping prior boost/beta=%s", exc, fallback)
return fallback
boost = max(0.0, _coerce_float(info.get("boost"), 1.0) or 1.0)
beta = max(0.0, _coerce_float(info.get("beta"), 0.0) or 0.0)
LOGGER.debug("ACB live boost=%.6f beta=%.6f (signals=%s w750=%s dir=%s)",
boost, beta, info.get("signals"), w750, trade_direction)
return boost, beta
def _scan_view(payload: Any) -> Mapping[str, Any]: def _scan_view(payload: Any) -> Mapping[str, Any]:
@@ -169,7 +314,12 @@ class LiveBlueScanHistory:
history = self.price_history(asset[0]) history = self.price_history(asset[0])
if not history: if not history:
return "NONE" return "NONE"
signal_gen = AlphaSignalGenerator() # BLUE constructs its signal_gen from ENGINE_KWARGS (the orchestrator threads them,
# esf_alpha_orchestrator.py:180-191). Pin the SAME params — not bare defaults — so a
# champion retune that changes dc_lookback_bars / dc_min_magnitude_bps / thresholds
# diverges loudly (caught by test_signal_gen_params_match_blue_engine_kwargs), never
# silently. dc_status is deterministic in the params + price history (counters unused).
signal_gen = AlphaSignalGenerator(**BLUE_SIGNAL_GEN_KWARGS)
sig = signal_gen.generate( sig = signal_gen.generate(
vel_div=vel_div, vel_div=vel_div,
vel_div_history=None, vel_div_history=None,
@@ -181,80 +331,37 @@ class LiveBlueScanHistory:
return sig.dc_status return sig.dc_status
class HazelcastOBProvider(OBProvider): def _source_ob_market(
"""Read the current BLUE OB shards directly from Hazelcast.""" ob_assets: list[str],
*,
bar_idx: int,
ob_engine: Optional[OBFeatureEngine] = None,
) -> tuple[Optional[float], Optional[float]]:
"""Derive (median_imbalance, agreement_pct) EXACTLY as BLUE does, via BLUE's HZOBProvider.
def __init__(self, client: hazelcast.HazelcastClient): BLUE wires OB once in _wire_obf (nautilus_event_trader.py:4967-4980):
self.client = client live_ob = HZOBProvider(hz_cluster=HZ_CLUSTER, hz_host=HZ_HOST, assets=assets)
ob_eng = OBFeatureEngine(live_ob); eng.set_ob_engine(ob_eng)
then per scan calls ``ob_eng.step_live(assets, bar_idx)`` and the orchestrator reads
``ob_eng.get_market(bar_idx, assets)`` (esf_alpha_orchestrator.py:590). We use BLUE's
OWN HZOBProvider — NOT a reinvented reader — so OB parsing/shard semantics are BLUE's.
def _asset_keys(self) -> list[str]: OBFeatureEngine ACCUMULATES per-asset history across scans (lookback=10), so a faithful
try: mirror requires a PERSISTENT ``ob_engine`` + per-scan-incrementing ``bar_idx`` (the
keys = self.client.get_map("DOLPHIN_FEATURES").blocking().key_set() caller/shadow loop owns it, exactly as BLUE keeps one ob_eng). When no engine is passed
except Exception: this builds a single-shot HZOBProvider-backed engine — correct wiring but no cross-scan
return [] history; use only for one-off reads / the live smoke."""
assets = [] if not ob_assets:
for key in keys: return None, None
if not isinstance(key, str) or not key.startswith("asset_") or not key.endswith("_ob"): if ob_engine is None:
continue provider = HZOBProvider(hz_cluster=HZ_CLUSTER, hz_host=HZ_HOST, assets=list(ob_assets))
asset = key[len("asset_"):-len("_ob")] ob_engine = OBFeatureEngine(provider)
if asset and asset not in assets: try:
assets.append(asset) ob_engine.step_live(list(ob_assets), bar_idx)
return sorted(assets) market = ob_engine.get_market(bar_idx, list(ob_assets))
return float(market.median_imbalance), float(market.agreement_pct)
def _read_snapshot(self, asset: str) -> Optional[OBSnapshot]: except Exception:
raw = _read_hz_map(self.client, "DOLPHIN_FEATURES", f"asset_{asset}_ob") return None, None
data = _jsonish(raw)
if not isinstance(data, Mapping):
return None
bid_notional = np.array(
[_coerce_float(v, 0.0) or 0.0 for v in data.get("bid_notional", [0, 0, 0, 0, 0])][:5],
dtype=np.float64,
)
ask_notional = np.array(
[_coerce_float(v, 0.0) or 0.0 for v in data.get("ask_notional", [0, 0, 0, 0, 0])][:5],
dtype=np.float64,
)
bid_depth = np.array(
[_coerce_float(v, 0.0) or 0.0 for v in data.get("bid_depth", [0, 0, 0, 0, 0])][:5],
dtype=np.float64,
)
ask_depth = np.array(
[_coerce_float(v, 0.0) or 0.0 for v in data.get("ask_depth", [0, 0, 0, 0, 0])][:5],
dtype=np.float64,
)
ts = _coerce_float(data.get("timestamp"), 0.0) or 0.0
if (
bid_notional.shape != (5,) or ask_notional.shape != (5,)
or bid_depth.shape != (5,) or ask_depth.shape != (5,)
):
return None
if np.any(bid_notional < 0) or np.any(ask_notional < 0):
return None
if np.any(bid_depth < 0) or np.any(ask_depth < 0):
return None
return OBSnapshot(
timestamp=ts,
asset=asset,
bid_notional=bid_notional,
ask_notional=ask_notional,
bid_depth=bid_depth,
ask_depth=ask_depth,
)
def get_snapshot(self, asset: str, timestamp: float) -> Optional[OBSnapshot]:
return self._read_snapshot(asset)
def get_assets(self) -> list[str]:
return self._asset_keys()
def get_all_timestamps(self, asset: str) -> np.ndarray:
snap = self._read_snapshot(asset)
if snap is None:
return np.array([], dtype=np.float64)
return np.array([snap.timestamp], dtype=np.float64)
def get_snapshot_count(self, asset: str) -> int:
return 1 if self._read_snapshot(asset) is not None else 0
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -274,10 +381,23 @@ def source_live_blue_sizing_factors(
assets: Optional[Iterable[str]] = None, assets: Optional[Iterable[str]] = None,
scan_history: Optional[LiveBlueScanHistory] = None, scan_history: Optional[LiveBlueScanHistory] = None,
selector: Optional[VioletAssetSelector] = None, selector: Optional[VioletAssetSelector] = None,
acb: Optional[AdaptiveCircuitBreaker] = None,
ob_engine: Optional[OBFeatureEngine] = None,
bar_idx: int = 0,
date_str: Optional[str] = None,
prior_boost_beta: Optional[tuple[float, float]] = None,
) -> LiveBlueSourceResult: ) -> LiveBlueSourceResult:
"""Read the BLUE-published live surfaces and return a typed factor plane.""" """Read BLUE's live surfaces and RECONSTRUCT the factor plane the way BLUE computes it.
boost/beta are recomputed via ``AdaptiveCircuitBreaker.get_dynamic_boost_from_hz`` (NOT
the published acb_boost scalar); OB via BLUE's ``HZOBProvider`` + ``OBFeatureEngine``;
dc_status via ``AlphaSignalGenerator`` pinned to BLUE's ENGINE_KWARGS. For full OB
accumulation faithfulness the caller passes a PERSISTENT ``ob_engine`` + per-scan
``bar_idx`` (BLUE keeps one ob_eng). ``date_str`` defaults to today's UTC date (BLUE
uses self.current_day)."""
scan_history = scan_history or LiveBlueScanHistory() scan_history = scan_history or LiveBlueScanHistory()
selector = selector or VioletAssetSelector() selector = selector or VioletAssetSelector()
today = date_str or datetime.now(timezone.utc).strftime("%Y-%m-%d")
engine_snapshot_raw = _read_hz_map(client, "DOLPHIN_STATE_BLUE", "latest_nautilus") engine_snapshot_raw = _read_hz_map(client, "DOLPHIN_STATE_BLUE", "latest_nautilus")
if engine_snapshot_raw is None: if engine_snapshot_raw is None:
@@ -296,13 +416,8 @@ def source_live_blue_sizing_factors(
esof_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "esof_advisor_latest") esof_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "esof_advisor_latest")
esof_score = esof_score_from_features(esof_raw) esof_score = esof_score_from_features(esof_raw)
acb_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "acb_boost")
if acb_raw is None:
acb_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "acb_boost_short")
acb_boost, acb_beta = _extract_acb(acb_raw)
mc_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "mc_forewarner_latest") mc_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "mc_forewarner_latest")
mc_scale = _map_status_to_mc_scale(mc_raw) mc_scale = _derive_mc_scale(mc_raw)
scan_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "latest_eigen_scan") scan_raw = _read_hz_map(client, "DOLPHIN_FEATURES", "latest_eigen_scan")
scan = scan_history.ingest_scan(scan_raw) scan = scan_history.ingest_scan(scan_raw)
@@ -318,26 +433,24 @@ def source_live_blue_sizing_factors(
) )
or -1 or -1
) )
# boost/beta — recompute via the ACB exactly as BLUE's trader does (NOT the published
# acb_boost). Needs trade_direction, so computed after it.
acb_boost, acb_beta = _source_boost_beta(
client, date_str=today, trade_direction=trade_direction, acb=acb,
prior=prior_boost_beta,
)
candidate_market = scan_history.market_data(selector.lookback) candidate_market = scan_history.market_data(selector.lookback)
pick = selector.pick(candidate_market, regime_direction=trade_direction) pick = selector.pick(candidate_market, regime_direction=trade_direction)
selected_asset = pick.asset if pick is not None else (scan_assets[0] if scan_assets else "") selected_asset = pick.asset if pick is not None else (scan_assets[0] if scan_assets else "")
dc_status = scan_history.dc_status(scan, already_ingested=True) dc_status = scan_history.dc_status(scan, already_ingested=True)
ob_provider = HazelcastOBProvider(client) # OB market consensus — BLUE's HZOBProvider + OBFeatureEngine (persistent engine if given).
ob_engine = OBFeatureEngine(ob_provider) ob_assets = list(assets) if assets is not None else scan_assets
ob_assets = list(assets) if assets is not None else (scan_assets or ob_provider.get_assets()) ob_median_imbalance, ob_agreement_pct = _source_ob_market(
if ob_assets: ob_assets, bar_idx=bar_idx, ob_engine=ob_engine,
try: )
ob_engine.step_live(ob_assets, bar_idx=0)
market = ob_engine.get_market(0, ob_assets)
ob_median_imbalance = float(market.median_imbalance)
ob_agreement_pct = float(market.agreement_pct)
except Exception:
ob_median_imbalance = None
ob_agreement_pct = None
else:
ob_median_imbalance = None
ob_agreement_pct = None
hz_snapshot = { hz_snapshot = {
"boost": acb_boost, "boost": acb_boost,
@@ -350,6 +463,12 @@ def source_live_blue_sizing_factors(
"posture": posture, "posture": posture,
} }
factors = extract_live_sizing_factors(hz_snapshot=hz_snapshot) factors = extract_live_sizing_factors(hz_snapshot=hz_snapshot)
LOGGER.debug(
"live BLUE plane: asset=%s posture=%s boost=%.4f beta=%.4f mc_scale=%.2f "
"esof=%s ob=(%s,%s) dc=%s dir=%s",
selected_asset, posture, acb_boost, acb_beta, mc_scale, esof_score,
ob_median_imbalance, ob_agreement_pct, dc_status, trade_direction,
)
return LiveBlueSourceResult( return LiveBlueSourceResult(
factors=factors, factors=factors,
acb_boost=acb_boost, acb_boost=acb_boost,

View File

@@ -2,6 +2,17 @@
These helpers stay separate from the launcher module so they can be unit-tested These helpers stay separate from the launcher module so they can be unit-tested
without importing the full launcher import chain. without importing the full launcher import chain.
They hold the PERSISTENT state BLUE keeps across scans so the shadow plane is faithful
to BLUE's live factor stream rather than a per-scan single-shot:
- ONE OBFeatureEngine, wired lazily on first assets and kept for the service lifetime
(OBFeatureEngine accumulates a lookback window; a fresh engine per scan has none).
It reads BLUE's EXTANT published OBF feed via HZOBProvider (a read-only HZ entry-
listener cache — NO new OB storage). The provider is the swap seam for a future
direct BingX / 3rd-party OB stream. See VIOLET_OB_FEED_AND_AGENT_COORDINATION.md.
- a per-scan-incrementing ``ob_bar_idx`` (BLUE steps one bar_idx into step_live).
- the last good ``prior_boost_beta`` so a stale-exf scan keeps the prior (BLUE keeps
_day_base_boost/_day_beta).
""" """
from __future__ import annotations from __future__ import annotations
@@ -9,7 +20,25 @@ from __future__ import annotations
import logging import logging
import os import os
LOGGER = logging.getLogger(__name__) LOGGER = logging.getLogger("violet.shadow_live_factors")
def _default_ob_engine_factory(assets):
"""Wire BLUE's EXTANT OBF feed: HZOBProvider (read-only listener) → OBFeatureEngine.
Exactly BLUE's _wire_obf (nautilus_event_trader.py:4967-4980). No OB storage, no new
exchange WS — it consumes the asset_*_ob shards BLUE already publishes to Hazelcast."""
from nautilus_dolphin.nautilus.ob_features import OBFeatureEngine
from nautilus_dolphin.nautilus.hz_ob_provider import HZOBProvider
from .live_blue_source import HZ_CLUSTER, HZ_HOST
# TODO_HZBRIDGE: HZOBProvider opens its OWN raw HazelcastClient. Once dolphinng5_predict/
# hzbridge ships, this connection MUST route through the bridge (silent HZ client death /
# lockup mitigation — see [[hz_client_death_investigation]]). One of 3 VIOLET HZ touch
# points to refactor (also build_shadow_live_source's client_factory + the live smoke).
provider = HZOBProvider(hz_cluster=HZ_CLUSTER, hz_host=HZ_HOST, assets=list(assets))
return OBFeatureEngine(provider)
def build_shadow_live_source( def build_shadow_live_source(
@@ -18,14 +47,18 @@ def build_shadow_live_source(
selector_factory=None, selector_factory=None,
source_factory=None, source_factory=None,
scan_history_factory=None, scan_history_factory=None,
ob_engine_factory=None,
): ):
"""Create the read-only BLUE live-factor mirror for the shadow path.""" """Create the read-only BLUE live-factor mirror for the shadow path, with the
persistent OB engine / bar_idx / boost-beta-prior state BLUE keeps across scans."""
if client_factory is None or selector_factory is None or source_factory is None or scan_history_factory is None: if client_factory is None or selector_factory is None or source_factory is None or scan_history_factory is None:
import hazelcast import hazelcast
from .alpha_wrappers import VioletAssetSelector from .alpha_wrappers import VioletAssetSelector
from .live_blue_source import LiveBlueScanHistory, source_live_blue_sizing_factors from .live_blue_source import LiveBlueScanHistory, source_live_blue_sizing_factors
# TODO_HZBRIDGE: raw HazelcastClient — route through dolphinng5_predict/hzbridge once
# it ships, to avoid the silent-death/lockup class. See [[hz_client_death_investigation]].
client_factory = client_factory or (lambda: hazelcast.HazelcastClient( client_factory = client_factory or (lambda: hazelcast.HazelcastClient(
cluster_name=os.environ.get("HZ_CLUSTER", "dolphin"), cluster_name=os.environ.get("HZ_CLUSTER", "dolphin"),
cluster_members=[os.environ.get("HZ_HOST", "localhost:5701")], cluster_members=[os.environ.get("HZ_HOST", "localhost:5701")],
@@ -40,9 +73,35 @@ def build_shadow_live_source(
"scan_history": scan_history_factory(), "scan_history": scan_history_factory(),
"selector": selector_factory(), "selector": selector_factory(),
"live_source": source_factory, "live_source": source_factory,
# persistent state (BLUE keeps these across scans)
"ob_engine": None,
"ob_engine_factory": ob_engine_factory or _default_ob_engine_factory,
"ob_bar_idx": 0,
"prior_boost_beta": None,
"last_live_source": None,
} }
def _ensure_ob_engine(shadow, payload):
"""Lazily wire ONE OB engine on the first scan that has an asset universe — like BLUE's
_wire_obf (`if not assets or self.ob_assets: return`). Kept for the service lifetime."""
if shadow.get("ob_engine") is not None:
return shadow["ob_engine"]
factory = shadow.get("ob_engine_factory")
if factory is None:
return None
from .live_blue_source import _scan_assets, _scan_view
assets = _scan_assets(_scan_view(payload))
if not assets:
return None
eng = factory(assets)
shadow["ob_engine"] = eng
shadow["ob_assets"] = assets
LOGGER.info("shadow OB wired to BLUE's extant feed for %d assets", len(assets))
return eng
def shadow_decision_step( def shadow_decision_step(
shadow: dict, shadow: dict,
payload: dict, payload: dict,
@@ -52,17 +111,28 @@ def shadow_decision_step(
vel_div: float, vel_div: float,
vol_ok: bool, vol_ok: bool,
) -> bool: ) -> bool:
"""Run one shadow decision against the live BLUE factor plane.""" """Run one muted shadow decision against the live BLUE factor plane, carrying the
persistent OB engine, bar_idx, and boost/beta prior across scans (BLUE-faithful)."""
shadow["engine"].observe(payload, scan_number) shadow["engine"].observe(payload, scan_number)
live_source = shadow.get("live_source") live_source = shadow.get("live_source")
factors = None factors = None
if live_source is not None: if live_source is not None:
ob_engine = _ensure_ob_engine(shadow, payload)
live_result = live_source( live_result = live_source(
shadow["client"], shadow["client"],
scan_history=shadow["scan_history"], scan_history=shadow["scan_history"],
selector=shadow["selector"], selector=shadow["selector"],
ob_engine=ob_engine,
bar_idx=shadow.get("ob_bar_idx", 0),
prior_boost_beta=shadow.get("prior_boost_beta"),
) )
shadow["last_live_source"] = live_result shadow["last_live_source"] = live_result
shadow["ob_bar_idx"] = shadow.get("ob_bar_idx", 0) + 1
# persist last good boost/beta as next scan's prior (BLUE keeps _day_base_boost/_day_beta).
ab = getattr(live_result, "acb_boost", None)
bb = getattr(live_result, "acb_beta", None)
if ab is not None and bb is not None:
shadow["prior_boost_beta"] = (ab, bb)
factors = live_result.factors factors = live_result.factors
if factors is None: if factors is None:
return False return False

View File

@@ -92,7 +92,8 @@ def test_shadow_decision_step_uses_live_factors_and_journals():
"client": object(), "client": object(),
"scan_history": object(), "scan_history": object(),
"selector": object(), "selector": object(),
"live_source": lambda client, scan_history, selector: SimpleNamespace( "live_source": lambda client, scan_history=None, selector=None, ob_engine=None,
bar_idx=0, prior_boost_beta=None: SimpleNamespace(
factors=SizingFactors( factors=SizingFactors(
boost=1.4, boost=1.4,
beta=0.2, beta=0.2,
@@ -104,6 +105,8 @@ def test_shadow_decision_step_uses_live_factors_and_journals():
posture="APEX", posture="APEX",
), ),
selected_asset="BTCUSDT", selected_asset="BTCUSDT",
acb_boost=1.4,
acb_beta=0.2,
), ),
"live_decisions": 0, "live_decisions": 0,
"last_live_source": None, "last_live_source": None,
@@ -153,3 +156,69 @@ def test_shadow_decision_step_skips_without_live_factor_plane():
vol_ok=True, vol_ok=True,
) )
assert ok is False assert ok is False
class _NoDecisionEngine:
def observe(self, payload, scan_number):
pass
def decide(self, **kwargs):
return None # step returns False after the live-source call (which is what we probe)
def test_shadow_step_lazily_wires_ob_engine_once_and_increments_bar_idx():
from prod.clean_arch.violet import shadow_live_factors as slf
built, seen = [], []
def ob_factory(assets):
eng = SimpleNamespace(assets=list(assets))
built.append(eng)
return eng
def live_source(client, *, scan_history, selector, ob_engine, bar_idx, prior_boost_beta):
seen.append((ob_engine, bar_idx))
return SimpleNamespace(factors=SizingFactors(posture="APEX"), selected_asset="BTCUSDT",
acb_boost=1.0, acb_beta=0.0)
shadow = slf.build_shadow_live_source(
client_factory=lambda: object(), selector_factory=lambda: object(),
source_factory=live_source, scan_history_factory=lambda: object(),
ob_engine_factory=ob_factory,
)
shadow["engine"] = _NoDecisionEngine()
shadow["capital"] = 1.0
payload = {"assets": ["BTCUSDT", "ETHUSDT"], "vel_div": -0.03}
for i in range(3):
slf.shadow_decision_step(shadow, payload, scan_number=i, now_ns=i, vel_div=-0.03, vol_ok=True)
assert len(built) == 1 # OB engine wired exactly ONCE
assert all(s[0] is built[0] for s in seen) # same persistent engine reused
assert [s[1] for s in seen] == [0, 1, 2] # bar_idx increments per scan
def test_shadow_step_carries_boost_beta_prior_across_scans():
from prod.clean_arch.violet import shadow_live_factors as slf
priors_seen, counter = [], {"n": 0}
def live_source(client, *, scan_history, selector, ob_engine, bar_idx, prior_boost_beta):
priors_seen.append(prior_boost_beta)
counter["n"] += 1
b = float(counter["n"])
return SimpleNamespace(factors=SizingFactors(posture="APEX"), selected_asset="X",
acb_boost=b, acb_beta=b / 10.0)
shadow = slf.build_shadow_live_source(
client_factory=lambda: object(), selector_factory=lambda: object(),
source_factory=live_source, scan_history_factory=lambda: object(),
ob_engine_factory=lambda assets: object(),
)
shadow["engine"] = _NoDecisionEngine()
shadow["capital"] = 1.0
payload = {"assets": ["X"], "vel_div": -0.03}
for i in range(3):
slf.shadow_decision_step(shadow, payload, scan_number=i, now_ns=i, vel_div=-0.03, vol_ok=True)
# scan0: no prior yet; scan1: prior = scan0's (1.0, 0.1); scan2: prior = scan1's (2.0, 0.2)
assert priors_seen == [None, (1.0, 0.1), (2.0, 0.2)]

View File

@@ -1,8 +1,19 @@
"""V3.4c live BLUE source — BLUE-algo parity tests (boost/beta, signal-gen, OB, mc_scale).
These pin VIOLET's reconstruction to BLUE's ACTUAL behaviour:
- boost/beta: bit-identical to AdaptiveCircuitBreaker.get_dynamic_boost_from_hz
- signal-gen: params pinned to BLUE's ENGINE_KWARGS (nautilus_event_trader.py)
- OB: BLUE's HZOBProvider + OBFeatureEngine wiring (persistent engine, step_live/get_market)
- mc_scale: begin_day's cat/env thresholds (not the MC service status label)
See prod/docs/VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md.
"""
from __future__ import annotations from __future__ import annotations
import json import json
import re
import sys import sys
from dataclasses import dataclass from pathlib import Path
import pytest import pytest
@@ -12,25 +23,27 @@ import hazelcast
from prod.clean_arch.violet.decision_engine import SizingFactors from prod.clean_arch.violet.decision_engine import SizingFactors
from prod.clean_arch.violet.live_blue_source import ( from prod.clean_arch.violet.live_blue_source import (
HazelcastOBProvider, BLUE_SIGNAL_GEN_KWARGS,
HZ_CLUSTER,
HZ_HOST,
LiveBlueScanHistory, LiveBlueScanHistory,
_derive_mc_scale,
_source_boost_beta,
_source_ob_market,
source_live_blue_sizing_factors, source_live_blue_sizing_factors,
) )
from prod.clean_arch.violet.alpha_wrappers import VioletAssetSelector from prod.clean_arch.violet.alpha_wrappers import VioletAssetSelector
from nautilus_dolphin.nautilus.alpha_signal_generator import AlphaSignalGenerator from nautilus_dolphin.nautilus.adaptive_circuit_breaker import AdaptiveCircuitBreaker
from nautilus_dolphin.nautilus.alpha_signal_generator import (
AlphaSignalGenerator,
@dataclass LONG_VEL_DIV_THRESHOLD, LONG_VEL_DIV_EXTREME,
class _FakeMap: VEL_DIV_THRESHOLD, VEL_DIV_EXTREME,
payloads: dict )
def get(self, key): TRADER = Path("/mnt/dolphinng5_predict/prod/nautilus_event_trader.py")
return self.payloads.get(key)
def key_set(self):
return list(self.payloads.keys())
# ── fakes ────────────────────────────────────────────────────────────────────
class _FakeBlocking: class _FakeBlocking:
def __init__(self, payloads): def __init__(self, payloads):
self._payloads = payloads self._payloads = payloads
@@ -47,285 +60,351 @@ class _FakeClient:
self._maps = maps self._maps = maps
def get_map(self, name): def get_map(self, name):
return type("M", (), {"blocking": lambda self2: _FakeBlocking(self._maps[name])})() payloads = self._maps.get(name, {})
return type("M", (), {"blocking": lambda self2, p=payloads: _FakeBlocking(p)})()
def test_hz_ob_provider_filters_and_parses_latest_payload(): class _FakeOBEngine:
client = _FakeClient( """Stands in for a persistent OBFeatureEngine; records step_live calls."""
{
"DOLPHIN_FEATURES": { def __init__(self, median_imbalance=0.0, agreement_pct=0.0):
"asset_BTCUSDT_ob": json.dumps( self.calls = []
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1], self._mi = median_imbalance
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]} self._ap = agreement_pct
),
"asset_XRPUSDT_ob": json.dumps( def step_live(self, assets, bar_idx):
{"timestamp": 2.0, "bid_notional": [-1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1], self.calls.append((tuple(assets), bar_idx))
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
), def get_market(self, bar_idx, assets):
"acb_boost": json.dumps({"boost": 1.2, "beta": 0.3}), return type("M", (), {"median_imbalance": self._mi, "agreement_pct": self._ap})()
"mc_forewarner_latest": json.dumps({"status": "ORANGE"}),
"esof_latest": json.dumps({"advisory_score": 0.4}),
}, _EXF = {"funding_btc": 0.01, "dvol_btc": 55.0, "fng": 40.0, "taker": 1.2, "_acb_ready": True}
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "restored"})},
}
def _features(**extra):
base = {
"esof_latest": json.dumps({"advisory_score": 0.4}),
"mc_forewarner_latest": json.dumps({"catastrophic_prob": 0.15, "envelope_score": 0.5}),
"exf_latest": json.dumps(_EXF),
}
base.update(extra)
return base
# ── 1. boost/beta: bit-identical to the ACB recompute (NOT acb_boost) ─────────
def test_boost_beta_recomputed_identically_to_blue_acb():
eigen = {"w750_velocity": 0.0012, "assets": ["BTCUSDT"], "asset_prices": [100.0],
"scan_number": 5, "vel_div": -0.03}
client = _FakeClient({"DOLPHIN_FEATURES": _features(latest_eigen_scan=json.dumps(eigen))})
boost, beta = _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1)
ref = AdaptiveCircuitBreaker().get_dynamic_boost_from_hz(
date_str="2026-06-16", exf_snapshot=dict(_EXF), w750_velocity=0.0012, direction=-1,
) )
provider = HazelcastOBProvider(client) # type: ignore[arg-type] assert boost == max(0.0, float(ref["boost"]))
assert provider.get_assets() == ["BTCUSDT", "XRPUSDT"] assert beta == max(0.0, float(ref["beta"]))
snap = provider.get_snapshot("BTCUSDT", 0.0)
assert snap is not None
assert snap.asset == "BTCUSDT"
assert snap.bid_notional.tolist() == [1.0, 2.0, 3.0, 4.0, 5.0]
def test_source_live_blue_sizing_factors_unit(monkeypatch): def test_boost_beta_w750_zero_passed_as_none_like_blue():
class FakeEngine: # BLUE: w750_velocity=float(w750) if w750 else None → 0.0 becomes None
def __init__(self, provider): eigen = {"w750_velocity": 0.0, "assets": ["BTCUSDT"], "asset_prices": [100.0]}
self.provider = provider client = _FakeClient({"DOLPHIN_FEATURES": _features(latest_eigen_scan=json.dumps(eigen))})
def step_live(self, assets, bar_idx): boost, beta = _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1)
assert "BTCUSDT" in assets ref = AdaptiveCircuitBreaker().get_dynamic_boost_from_hz(
def get_market(self, ts, assets): date_str="2026-06-16", exf_snapshot=dict(_EXF), w750_velocity=None, direction=-1,
return type("M", (), {"median_imbalance": 0.12, "agreement_pct": 0.91})() )
assert (boost, beta) == (max(0.0, float(ref["boost"])), max(0.0, float(ref["beta"])))
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
def test_boost_beta_neutral_when_no_exf():
client = _FakeClient({"DOLPHIN_FEATURES": {}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1) == (1.0, 0.0)
def test_boost_beta_neutral_on_stale_exf_valueerror():
# >12h staleness → get_dynamic_boost_from_hz raises ValueError → BLUE stale fallback;
# VIOLET has no prior → neutral identity.
stale = dict(_EXF, _staleness_s={"funding_btc": 999999.0})
client = _FakeClient({"DOLPHIN_FEATURES": {"exf_latest": json.dumps(stale),
"latest_eigen_scan": json.dumps({"w750_velocity": 0.001})}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1) == (1.0, 0.0)
def test_boost_beta_keeps_prior_on_stale_exf():
# BLUE keeps _day_base_boost/_day_beta on stale exf; VIOLET keeps the seeded prior.
stale = dict(_EXF, _staleness_s={"funding_btc": 999999.0})
client = _FakeClient({"DOLPHIN_FEATURES": {"exf_latest": json.dumps(stale),
"latest_eigen_scan": json.dumps({"w750_velocity": 0.001})}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1,
prior=(1.37, 0.8)) == (1.37, 0.8)
def test_boost_beta_keeps_prior_when_no_exf():
client = _FakeClient({"DOLPHIN_FEATURES": {}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1,
prior=(1.5, 0.2)) == (1.5, 0.2)
def test_boost_beta_cold_start_no_prior_is_neutral():
# No prior AND no fresh exf (cold start) → neutral identity, never silently wrong.
client = _FakeClient({"DOLPHIN_FEATURES": {}})
assert _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1, prior=None) == (1.0, 0.0)
def test_source_live_blue_threads_prior_boost_beta_on_stale():
stale = dict(_EXF, _staleness_s={"funding_btc": 999999.0})
client = _FakeClient({
"DOLPHIN_FEATURES": {
"exf_latest": json.dumps(stale),
"mc_forewarner_latest": json.dumps({"catastrophic_prob": 0.02, "envelope_score": 0.9}),
"latest_eigen_scan": json.dumps({"assets": ["BTCUSDT"], "asset_prices": [100.0], "w750_velocity": 0.001}),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
})
res = source_live_blue_sizing_factors(
client, assets=["BTCUSDT"], ob_engine=_FakeOBEngine(), prior_boost_beta=(1.42, 0.8),
)
assert res.factors.boost == 1.42 and res.factors.beta == 0.8
def test_boost_beta_does_not_read_published_acb_boost():
# An acb_boost scalar is present but MUST be ignored (BLUE recomputes from exf).
eigen = {"w750_velocity": 0.0012}
client = _FakeClient({"DOLPHIN_FEATURES": _features(
latest_eigen_scan=json.dumps(eigen),
acb_boost=json.dumps({"boost": 7.77, "beta": 7.77}), # poison: must NOT surface
)})
boost, beta = _source_boost_beta(client, date_str="2026-06-16", trade_direction=-1)
assert boost != 7.77 and beta != 7.77
# ── 2. signal-gen params pinned to BLUE's live ENGINE_KWARGS ──────────────────
def _blue_engine_kwargs_block() -> str:
text = TRADER.read_text()
start = text.index("ENGINE_KWARGS = dict(")
end = text.index("\n)", start)
return text[start:end]
def _blue_kwarg(name: str):
m = re.search(rf"\b{name}\s*=\s*([^\s,]+)", _blue_engine_kwargs_block())
assert m, f"{name} not found in BLUE ENGINE_KWARGS"
raw = m.group(1).rstrip(",")
if raw in ("True", "False"):
return raw == "True"
return float(raw) if any(c in raw for c in ".-e") else int(raw)
@pytest.mark.parametrize("key", [
"vel_div_threshold", "vel_div_extreme", "dc_lookback_bars", "dc_min_magnitude_bps",
"use_direction_confirm", "dc_skip_contradicts", "dc_leverage_boost", "dc_leverage_reduce",
])
def test_signal_gen_params_match_blue_engine_kwargs(key):
# If a champion retune changes ENGINE_KWARGS, this fails — no silent divergence.
assert BLUE_SIGNAL_GEN_KWARGS[key] == _blue_kwarg(key)
def test_signal_gen_long_thresholds_track_kernel_constants():
assert BLUE_SIGNAL_GEN_KWARGS["vel_div_threshold"] == VEL_DIV_THRESHOLD
assert BLUE_SIGNAL_GEN_KWARGS["vel_div_extreme"] == VEL_DIV_EXTREME
assert BLUE_SIGNAL_GEN_KWARGS["long_vel_div_threshold"] == LONG_VEL_DIV_THRESHOLD
assert BLUE_SIGNAL_GEN_KWARGS["long_vel_div_extreme"] == LONG_VEL_DIV_EXTREME
def test_dc_status_uses_pinned_signal_gen_not_bare_default():
# dc_status must equal a BLUE signal_gen built with the SAME pinned params.
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1) history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
for idx, px in enumerate([100.0, 99.5, 99.0, 98.4, 97.8, 97.0, 96.2], start=1): for idx, px in enumerate([100.0, 99.5, 99.0, 98.4, 97.8, 97.0, 96.2], start=1):
history.ingest_scan( history.ingest_scan({"scan_number": idx, "timestamp": float(idx), "vel_div": -0.031,
{ "assets": ["BTCUSDT"], "asset_prices": [px]})
"scan_number": idx, scan = {"scan_number": 8, "timestamp": 8.0, "vel_div": -0.031, "assets": ["BTCUSDT"],
"timestamp": float(idx), "asset_prices": [95.5]}
"vel_div": -0.031, got = history.dc_status(scan)
"target_asset": "BTCUSDT", ref = AlphaSignalGenerator(**BLUE_SIGNAL_GEN_KWARGS).generate(
"assets": ["BTCUSDT"], vel_div=-0.031, vel_div_history=None,
"asset_prices": [px], asset_price_history=history.price_history("BTCUSDT"),
} trade_direction=-1, asset="BTCUSDT", current_timestamp=8.0,
)
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.4, "beta": 0.2}),
"mc_forewarner_latest": json.dumps({"status": "ORANGE"}),
"esof_latest": json.dumps({"advisory_score": 0.4}),
"latest_eigen_scan": json.dumps(
{
"scan_number": 8,
"timestamp": 8.0,
"vel_div": -0.031,
"target_asset": "BTCUSDT",
"assets": ["BTCUSDT"],
"asset_prices": [95.5],
}
),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "stalker"})},
}
) )
assert got == ref.dc_status
# ── 3. OB: BLUE's HZOBProvider + OBFeatureEngine wiring ───────────────────────
def test_ob_uses_persistent_engine_step_live_then_get_market():
eng = _FakeOBEngine(median_imbalance=0.12, agreement_pct=0.9)
mi, ap = _source_ob_market(["BTCUSDT", "ETHUSDT"], bar_idx=3, ob_engine=eng)
assert eng.calls == [(("BTCUSDT", "ETHUSDT"), 3)] # step_live(assets, bar_idx) — BLUE's call
assert (mi, ap) == (0.12, 0.9)
def test_ob_empty_assets_neutral():
assert _source_ob_market([], bar_idx=0) == (None, None)
def test_ob_engine_exception_neutral():
class _Boom:
def step_live(self, a, b):
raise RuntimeError("x")
def get_market(self, b, a):
raise AssertionError("unreachable")
assert _source_ob_market(["BTCUSDT"], bar_idx=0, ob_engine=_Boom()) == (None, None)
def test_ob_builds_blue_hzobprovider_with_live_coords(monkeypatch):
captured = {}
class _FakeProvider:
def __init__(self, *, hz_cluster, hz_host, assets):
captured.update(cluster=hz_cluster, host=hz_host, assets=list(assets))
class _FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, a, b):
pass
def get_market(self, b, a):
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})()
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.HZOBProvider", _FakeProvider)
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", _FakeEngine)
_source_ob_market(["BTCUSDT"], bar_idx=0) # ob_engine=None → must construct BLUE's provider
assert captured == {"cluster": HZ_CLUSTER, "host": HZ_HOST, "assets": ["BTCUSDT"]}
# ── 4. mc_scale: begin_day's cat/env thresholds (the V3.4c bug fix) ───────────
@pytest.mark.parametrize("cat, env, expected", [
(0.15, 0.5, 0.5), # orange via cat>0.10
(0.05, -0.5, 0.5), # orange via env<0 — old status-code returned GREEN→1.0 (bug)
(0.10, -0.001, 0.5),
(0.28, 0.5, 1.0), # red via cat>0.25 — old status-code: ORANGE→0.5 (bug)
(0.05, -1.5, 1.0), # red via env<-1.0
(0.30, -2.0, 1.0),
(0.05, 0.5, 1.0), # benign
(0.10, 0.0, 1.0),
])
def test_derive_mc_scale_matches_blue_begin_day_formula(cat, env, expected):
payload = json.dumps({"catastrophic_prob": cat, "envelope_score": env, "status": "IGNORED"})
assert _derive_mc_scale(payload) == expected
def test_derive_mc_scale_neutral_when_fields_missing_or_garbage():
assert _derive_mc_scale(json.dumps({"status": "ORANGE"})) == 1.0 # label must be ignored
assert _derive_mc_scale(json.dumps({"catastrophic_prob": 0.15})) == 1.0
assert _derive_mc_scale(json.dumps({"envelope_score": -0.5})) == 1.0
assert _derive_mc_scale("not json") == 1.0
assert _derive_mc_scale(None) == 1.0
assert _derive_mc_scale({"catastrophic_prob": "bad", "envelope_score": 0.5}) == 1.0
# ── 5. integration: full plane sourced + composed ────────────────────────────
def _client_with_scan(scan: dict, *, posture="STALKER", **feat) -> _FakeClient:
eigen = json.dumps({**scan, "w750_velocity": 0.0012})
return _FakeClient({
"DOLPHIN_FEATURES": _features(latest_eigen_scan=eigen, **feat),
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": posture})},
})
def test_source_live_blue_sizing_factors_full_plane():
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
for idx, px in enumerate([100.0, 99.5, 99.0, 98.4, 97.8, 97.0, 96.2], start=1):
history.ingest_scan({"scan_number": idx, "timestamp": float(idx), "vel_div": -0.031,
"assets": ["BTCUSDT"], "asset_prices": [px]})
scan = {"scan_number": 8, "timestamp": 8.0, "vel_div": -0.031,
"assets": ["BTCUSDT"], "asset_prices": [95.5]}
client = _client_with_scan(scan, posture="RESTORED")
ob = _FakeOBEngine(median_imbalance=0.12, agreement_pct=0.91)
res = source_live_blue_sizing_factors( res = source_live_blue_sizing_factors(
client, client, assets=["BTCUSDT"], scan_history=history,
assets=["BTCUSDT"],
scan_history=history,
selector=VioletAssetSelector(lookback_horizon=7), selector=VioletAssetSelector(lookback_horizon=7),
ob_engine=ob, bar_idx=8, date_str="2026-06-16",
) )
assert isinstance(res.factors, SizingFactors) assert isinstance(res.factors, SizingFactors)
assert res.factors.posture == "STALKER" assert res.factors.posture == "RESTORED"
assert res.factors.mc_scale == 0.5
assert res.factors.boost == 1.4
assert res.factors.beta == 0.2
assert res.factors.esof_score == 0.4 assert res.factors.esof_score == 0.4
assert res.factors.ob_median_imbalance == 0.12 assert res.factors.mc_scale == 0.5 # cat=0.15/env=0.5 → orange
assert res.factors.ob_agreement_pct == 0.91 assert res.factors.ob_median_imbalance == 0.12 and res.factors.ob_agreement_pct == 0.91
assert ob.calls == [(("BTCUSDT",), 8)]
# boost/beta = ACB recompute over the SAME exf+w750 (bit-identical pin)
ref = AdaptiveCircuitBreaker().get_dynamic_boost_from_hz(
date_str="2026-06-16", exf_snapshot=dict(_EXF), w750_velocity=0.0012, direction=-1)
assert res.factors.boost == max(0.0, float(ref["boost"]))
assert res.factors.beta == max(0.0, float(ref["beta"]))
assert res.factors.dc_status == "CONFIRM" assert res.factors.dc_status == "CONFIRM"
assert res.selected_asset == "BTCUSDT" assert res.selected_asset == "BTCUSDT"
def test_source_live_blue_sizing_factors_preserves_skip_contradict(monkeypatch): def test_source_live_blue_sizing_factors_handles_anomalies():
class FakeEngine: client = _FakeClient({
def __init__(self, provider): "DOLPHIN_FEATURES": {
self.provider = provider "exf_latest": "not json", # → boost/beta neutral
def step_live(self, assets, bar_idx): "mc_forewarner_latest": json.dumps({"catastrophic_prob": 0.02, "envelope_score": 0.9}),
pass "esof_latest": "not json",
def get_market(self, ts, assets): "latest_eigen_scan": json.dumps({"assets": ["BTCUSDT"], "asset_prices": [0]}),
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})() },
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": ""})},
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine) })
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1) res = source_live_blue_sizing_factors(client, assets=["BTCUSDT"], ob_engine=_FakeOBEngine())
for idx, px in enumerate([100.0, 100.5, 101.0, 101.6, 102.2, 102.9, 103.6], start=1): assert res.factors.posture == "APEX"
history.ingest_scan( assert res.factors.mc_scale == 1.0
{ assert res.factors.boost == 1.0 and res.factors.beta == 0.0 # bad exf → ACB neutral
"scan_number": idx, assert res.factors.esof_score is None
"timestamp": float(idx), # OB engine is healthy (the anomalies are in exf/esof/prices), so it returns its 0.0/0.0.
"vel_div": -0.031, assert res.factors.ob_median_imbalance == 0.0 and res.factors.ob_agreement_pct == 0.0
"target_asset": "BTCUSDT", assert res.factors.dc_status == "NONE"
"assets": ["BTCUSDT"], assert res.selected_asset == "BTCUSDT"
"asset_prices": [px],
}
)
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.4, "beta": 0.2}),
"mc_forewarner_latest": json.dumps({"status": "GREEN"}),
"esof_latest": json.dumps({"advisory_score": 0.4}),
"latest_eigen_scan": json.dumps(
{
"scan_number": 8,
"timestamp": 8.0,
"vel_div": -0.031,
"target_asset": "BTCUSDT",
"assets": ["BTCUSDT"],
"asset_prices": [104.2],
}
),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
}
)
res = source_live_blue_sizing_factors(
client,
assets=["BTCUSDT"],
scan_history=history,
selector=VioletAssetSelector(lookback_horizon=7),
)
assert res.factors.dc_status == "SKIP_CONTRADICT"
def test_live_blue_sequence_matches_blue_selector_and_dc_at_each_step(monkeypatch): def test_source_live_blue_neutral_ob_when_no_engine_and_no_assets():
class FakeEngine: # No ob_engine + no assets discoverable → OB neutral (None), no HZOBProvider connection.
def __init__(self, provider): client = _FakeClient({
self.provider = provider "DOLPHIN_FEATURES": _features(latest_eigen_scan=json.dumps({"assets": [], "asset_prices": []})),
def step_live(self, assets, bar_idx): "DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
pass })
def get_market(self, ts, assets): res = source_live_blue_sizing_factors(client) # assets=None, scan has none → []
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})() assert res.factors.ob_median_imbalance is None and res.factors.ob_agreement_pct is None
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
def test_sequence_matches_blue_selector_and_dc_at_each_step():
selector = VioletAssetSelector(lookback_horizon=7) selector = VioletAssetSelector(lookback_horizon=7)
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1) history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
signal_gen = AlphaSignalGenerator() ref_gen = AlphaSignalGenerator(**BLUE_SIGNAL_GEN_KWARGS)
scans = [ scans = [
{ {"scan_number": 1, "timestamp": 1.0, "vel_div": -0.010, "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [100.0, 200.0]},
"scan_number": 1, {"scan_number": 2, "timestamp": 2.0, "vel_div": -0.031, "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [99.0, 198.0]},
"timestamp": 1.0, {"scan_number": 3, "timestamp": 3.0, "vel_div": -0.041, "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [98.0, 196.0]},
"vel_div": -0.010, {"scan_number": 4, "timestamp": 4.0, "vel_div": -0.031, "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [97.0, 194.0]},
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [100.0, 200.0],
},
{
"scan_number": 2,
"timestamp": 2.0,
"vel_div": -0.031,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [99.0, 198.0],
},
{
"scan_number": 3,
"timestamp": 3.0,
"vel_div": -0.041,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [98.0, 196.0],
},
{
"scan_number": 4,
"timestamp": 4.0,
"vel_div": -0.031,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [97.0, 194.0],
},
] ]
for i, scan in enumerate(scans, start=1):
for idx, scan in enumerate(scans, start=1): client = _client_with_scan(scan, posture="APEX")
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"asset_ETHUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [2, 3, 4, 5, 6], "ask_notional": [6, 5, 4, 3, 2],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.0, "beta": 0.0}),
"mc_forewarner_latest": json.dumps({"status": "GREEN"}),
"esof_latest": json.dumps({"advisory_score": 0.3}),
"latest_eigen_scan": json.dumps({**scan, "target_asset": "BTCUSDT"}),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
}
)
res = source_live_blue_sizing_factors( res = source_live_blue_sizing_factors(
client, client, assets=["BTCUSDT", "ETHUSDT"], scan_history=history,
assets=["BTCUSDT", "ETHUSDT"], selector=selector, ob_engine=_FakeOBEngine(), bar_idx=i,
scan_history=history,
selector=selector,
) )
# BLUE selector parity
market = history.market_data(selector.lookback) market = history.market_data(selector.lookback)
expected_pick = selector.pick(market, regime_direction=-1) expected_pick = selector.pick(market, regime_direction=-1)
expected_asset = expected_pick.asset if expected_pick is not None else "BTCUSDT" expected_asset = expected_pick.asset if expected_pick is not None else "BTCUSDT"
assert res.selected_asset == expected_asset assert res.selected_asset == expected_asset
ref = ref_gen.generate(
# BLUE signal parity vel_div=float(scan["vel_div"]), vel_div_history=None,
expected_signal = signal_gen.generate(
vel_div=float(scan["vel_div"]),
vel_div_history=None,
asset_price_history=history.price_history(expected_asset), asset_price_history=history.price_history(expected_asset),
trade_direction=-1, trade_direction=-1, asset=expected_asset, current_timestamp=float(scan["timestamp"]),
asset=expected_asset,
current_timestamp=float(scan["timestamp"]),
) )
assert res.factors.dc_status == expected_signal.dc_status assert res.factors.dc_status == ref.dc_status
def test_live_blue_sequence_rejects_anomalous_values_without_poisoning_history(monkeypatch): def test_sequence_rejects_anomalous_values_without_poisoning_history():
class FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, assets, bar_idx):
pass
def get_market(self, ts, assets):
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})()
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
history = LiveBlueScanHistory(maxlen=16, trade_direction=-1) history = LiveBlueScanHistory(maxlen=16, trade_direction=-1)
selector = VioletAssetSelector(lookback_horizon=7) scan = {"scan_number": 99, "timestamp": 99.0, "vel_div": -0.031,
scan = { "assets": ["BTCUSDT", "ETHUSDT"], "asset_prices": [float("nan"), -1.0]}
"scan_number": 99, client = _client_with_scan(scan, posture="APEX")
"timestamp": 99.0,
"vel_div": -0.031,
"assets": ["BTCUSDT", "ETHUSDT"],
"asset_prices": [float("nan"), -1.0],
"target_asset": "BTCUSDT",
}
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": 1.0, "beta": 0.0}),
"mc_forewarner_latest": json.dumps({"status": "GREEN"}),
"esof_latest": json.dumps({"advisory_score": 0.3}),
"latest_eigen_scan": json.dumps(scan),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": "APEX"})},
}
)
res = source_live_blue_sizing_factors( res = source_live_blue_sizing_factors(
client, client, assets=["BTCUSDT", "ETHUSDT"], scan_history=history,
assets=["BTCUSDT", "ETHUSDT"], selector=VioletAssetSelector(lookback_horizon=7), ob_engine=_FakeOBEngine(),
scan_history=history,
selector=selector,
) )
assert res.selected_asset == "BTCUSDT" assert res.selected_asset == "BTCUSDT"
assert res.factors.dc_status == "NONE" assert res.factors.dc_status == "NONE"
@@ -333,43 +412,7 @@ def test_live_blue_sequence_rejects_anomalous_values_without_poisoning_history(m
assert history.price_history("ETHUSDT") == [] assert history.price_history("ETHUSDT") == []
def test_source_live_blue_sizing_factors_handles_anomalies(monkeypatch): # ── 6. live HZ smoke (env-bound; deselect with -k "not live_hz_smoke") ────────
class FakeEngine:
def __init__(self, provider):
self.provider = provider
def step_live(self, assets, bar_idx):
raise RuntimeError("boom")
def get_market(self, ts, assets):
return type("M", (), {"median_imbalance": 0.0, "agreement_pct": 0.0})()
monkeypatch.setattr("prod.clean_arch.violet.live_blue_source.OBFeatureEngine", FakeEngine)
client = _FakeClient(
{
"DOLPHIN_FEATURES": {
"asset_BTCUSDT_ob": json.dumps(
{"timestamp": 1.0, "bid_notional": [1, 2, 3, 4, 5], "ask_notional": [5, 4, 3, 2, 1],
"bid_depth": [1, 1, 1, 1, 1], "ask_depth": [1, 1, 1, 1, 1]}
),
"acb_boost": json.dumps({"boost": -9, "beta": "bad"}),
"mc_forewarner_latest": json.dumps({"status": "GREEN"}),
"esof_latest": "not json",
"latest_eigen_scan": json.dumps({"target_asset": "BTCUSDT", "assets": ["BTCUSDT"], "asset_prices": [0]}),
},
"DOLPHIN_STATE_BLUE": {"latest_nautilus": json.dumps({"posture": ""})},
}
)
res = source_live_blue_sizing_factors(client, assets=["BTCUSDT"])
assert res.factors.posture == "APEX"
assert res.factors.mc_scale == 1.0
assert res.factors.boost == 0.0
assert res.factors.beta == 0.0
assert res.factors.esof_score is None
assert res.factors.ob_median_imbalance is None
assert res.factors.ob_agreement_pct is None
assert res.factors.dc_status == "NONE"
assert res.selected_asset == "BTCUSDT"
def test_live_hz_smoke_reads_current_state(): def test_live_hz_smoke_reads_current_state():
client = hazelcast.HazelcastClient(cluster_name="dolphin", cluster_members=["localhost:5701"]) client = hazelcast.HazelcastClient(cluster_name="dolphin", cluster_members=["localhost:5701"])
try: try:
@@ -379,3 +422,4 @@ def test_live_hz_smoke_reads_current_state():
assert isinstance(res.factors, SizingFactors) assert isinstance(res.factors, SizingFactors)
assert res.factors.posture in {"APEX", "STALKER", "RESTORED", "TURTLE", "HIBERNATE"} assert res.factors.posture in {"APEX", "STALKER", "RESTORED", "TURTLE", "HIBERNATE"}
assert res.factors.mc_scale in {0.5, 1.0} assert res.factors.mc_scale in {0.5, 1.0}
assert res.factors.boost >= 0.0 and res.factors.beta >= 0.0

View File

@@ -0,0 +1,153 @@
"""V3.4 parity pin tests for BLUE sizing intermediates.
This is a direct regression pin, not a broad behavioral suite. It compares the
VIOLET replica against BLUE's live formulas for each sizing intermediate on a
deterministic grid, with exact equality.
"""
from __future__ import annotations
import sys
from pathlib import Path
import numpy as np
sys.path.insert(0, "/mnt/dolphinng5_predict")
sys.path.insert(0, "/mnt/dolphinng5_predict/nautilus_dolphin")
from prod.clean_arch.violet.sizing import VioletSizer
def _blue_engine(*, dc_leverage_boost: float = 1.0):
from nautilus_dolphin.nautilus.esf_alpha_orchestrator import NDAlphaEngine
eng = NDAlphaEngine(
initial_capital=69_000.0,
max_leverage=8.0,
abs_max_leverage=9.0,
min_leverage=0.5,
fraction=0.20,
use_asset_selection=False,
use_direction_confirm=True,
dc_leverage_boost=dc_leverage_boost,
)
eng.begin_day("2026-06-16", posture="APEX", direction=-1)
return eng
def _blue_market_ob_mult(median_imbalance: float, agreement_pct: float, trade_direction: int) -> float:
eff_imb = -median_imbalance if trade_direction == -1 else median_imbalance
if eff_imb > 0.08 and agreement_pct > 0.70:
return 1.0 + min(0.20, eff_imb * agreement_pct * 0.5)
if eff_imb < -0.08 and agreement_pct > 0.70:
return max(0.85, 1.0 - abs(eff_imb) * agreement_pct * 0.3)
return 1.0
def _ob_grid() -> list[tuple[float, float]]:
imbalances = np.linspace(-1.0, 1.0, 20, dtype=np.float64)
agreements = np.linspace(0.0, 1.0, 10, dtype=np.float64)
return [(float(i), float(a)) for i in imbalances for a in agreements]
def _dc_prices(status: str, lookback: int = 7) -> list[float]:
if status == "CONFIRM":
return [100.0 - 0.2 * i for i in range(lookback + 1)]
if status == "CONTRADICT":
return [100.0 + 0.2 * i for i in range(lookback + 1)]
return [100.0 for _ in range(lookback + 1)]
def test_strength_cubic_exact_parity_grid():
eng = _blue_engine()
sz = VioletSizer()
cases = []
for trade_direction in (-1, 1):
for vel_div in np.linspace(-0.06, 0.04, 200, dtype=np.float64):
cases.append((float(vel_div), trade_direction))
assert len(cases) >= 200
for vel_div, trade_direction in cases:
blue = eng._strength_cubic(vel_div, direction=trade_direction)
violet = sz.strength_cubic(vel_div, trade_direction=trade_direction)
assert violet == blue
def test_regime_size_mult_exact_parity_grid():
eng = _blue_engine()
sz = VioletSizer()
vel_divs = np.linspace(-0.06, 0.0, 50, dtype=np.float64)
boosts = (1.0, 1.25, 1.75, 2.5)
betas = (0.0, 0.2, 0.8)
mc_scales = (0.5, 1.0)
directions = (-1, 1)
n = 0
for trade_direction in directions:
eng.regime_direction = trade_direction
eng._day_posture = "APEX"
for vel_div in vel_divs:
for boost in boosts:
for beta in betas:
for mc_scale in mc_scales:
eng._day_base_boost = boost
eng._day_beta = beta
eng._day_mc_scale = mc_scale
eng._update_regime_size_mult(float(vel_div))
violet = sz.regime_size_mult(
float(vel_div),
boost=float(boost),
beta=float(beta),
mc_scale=float(mc_scale),
trade_direction=trade_direction,
)
assert violet == eng.regime_size_mult
n += 1
assert n >= 200
def test_market_ob_mult_exact_parity_grid():
sz = VioletSizer()
cases = _ob_grid()
assert len(cases) == 200
for trade_direction in (-1, 1):
for median_imbalance, agreement_pct in cases:
blue = _blue_market_ob_mult(median_imbalance, agreement_pct, trade_direction)
violet = sz.market_ob_mult(
median_imbalance,
agreement_pct,
trade_direction=trade_direction,
)
assert violet == blue
def test_dc_leverage_multiplier_exact_parity_grid():
sz = VioletSizer(dc_leverage_boost=1.25)
eng = _blue_engine(dc_leverage_boost=1.25)
sig = eng.signal_gen
short_vel_divs = np.linspace(-0.06, -0.021, 100, dtype=np.float64)
long_vel_divs = np.linspace(0.011, 0.04, 100, dtype=np.float64)
statuses = ("CONFIRM", "NEUTRAL", "SKIP_CONTRADICT")
n = 0
for trade_direction, vel_divs in ((-1, short_vel_divs), (1, long_vel_divs)):
for vel_div in vel_divs:
for status in statuses:
prices = _dc_prices(status)
result = sig.generate(
vel_div=float(vel_div),
vel_div_history=[float(vel_div)] * 10,
asset_price_history=prices,
trade_direction=trade_direction,
asset=None,
current_timestamp=0.0,
)
blue = sig.dc_leverage_boost if result.dc_status == "CONFIRM" else 1.0
violet = sz.dc_lev_mult(result.dc_status)
assert violet == blue
n += 1
assert n >= 200

View File

@@ -0,0 +1,140 @@
"""V4 arm-readiness reporter — self-test to death.
Spec: prod/docs/VIOLET_PASS_MM1_V4_READINESS_REPORTER.md
Style: follows test_violet_domain.py patterns.
"""
from __future__ import annotations
import sys
sys.path.insert(0, "/mnt/dolphinng5_predict")
import pytest
from hypothesis import given, settings
from hypothesis import strategies as st
from prod.clean_arch.violet.v4_readiness import (
V4ReadinessReport,
assess_v4_readiness,
)
ALL_FIELDS = [
"creds_ok", "canary_passed", "arming_gate_present",
"conviction_bit_identical", "selection_parity_ok", "feed_live",
]
def _all_true(**overrides: bool) -> dict:
return {f: overrides.get(f, True) for f in ALL_FIELDS}
def _single_false(field: str) -> dict:
return _all_true(**{field: False})
# ── happy path ────────────────────────────────────────────────────────────────
def test_all_true_ready_to_arm():
r = assess_v4_readiness(**_all_true())
assert r.ready_to_arm is True
assert r.reasons == []
# ── each single False yields ready_to_arm=False + correct reason ─────────────
@pytest.mark.parametrize("field", ALL_FIELDS)
def test_single_false_not_ready(field: str):
r = assess_v4_readiness(**_single_false(field))
assert r.ready_to_arm is False
assert len(r.reasons) == 1
assert field in r.reasons[0]
# ── hypothesis: invariant holds across all 2^6 combos ────────────────────────
@given(
creds_ok=st.booleans(),
canary_passed=st.booleans(),
arming_gate_present=st.booleans(),
conviction_bit_identical=st.booleans(),
selection_parity_ok=st.booleans(),
feed_live=st.booleans(),
)
@settings(max_examples=100)
def test_ready_to_arm_equals_all_inputs(
creds_ok: bool,
canary_passed: bool,
arming_gate_present: bool,
conviction_bit_identical: bool,
selection_parity_ok: bool,
feed_live: bool,
):
r = assess_v4_readiness(
creds_ok=creds_ok,
canary_passed=canary_passed,
arming_gate_present=arming_gate_present,
conviction_bit_identical=conviction_bit_identical,
selection_parity_ok=selection_parity_ok,
feed_live=feed_live,
)
expected = all((
creds_ok, canary_passed, arming_gate_present,
conviction_bit_identical, selection_parity_ok, feed_live,
))
assert r.ready_to_arm is expected
# ── mutation litmus: flipping all() to any() MUST break this test ────────────
def test_mutation_litmus_all_must_be_true():
"""If someone changes all() to any(), at least one all-False should become
ready_to_arm=True — this test catches it."""
r = assess_v4_readiness(**{f: False for f in ALL_FIELDS})
assert r.ready_to_arm is False
assert len(r.reasons) == 6
# ── poison: StrictModel rejects non-bool / extra fields ─────────────────────
def test_rejects_non_bool():
with pytest.raises(Exception):
V4ReadinessReport(
creds_ok={"ok": True}, # dict is not coercible to bool
canary_passed=True,
arming_gate_present=True,
conviction_bit_identical=True,
selection_parity_ok=True,
feed_live=True,
reasons=[],
)
def test_rejects_extra_fields():
with pytest.raises(Exception):
V4ReadinessReport(
creds_ok=True,
canary_passed=True,
arming_gate_present=True,
conviction_bit_identical=True,
selection_parity_ok=True,
feed_live=True,
reasons=[],
unknown_field="oops",
)
def test_rejects_frozen_mutation():
r = assess_v4_readiness(**_all_true())
with pytest.raises(Exception):
r.creds_ok = False # type: ignore[misc]
# ── reasons content check: False fields produce descriptive strings ──────────
def test_reasons_mention_gate_name():
r = assess_v4_readiness(**_single_false("feed_live"))
assert any("PASS2.6" in reason for reason in r.reasons)
if __name__ == "__main__":
raise SystemExit(pytest.main([__file__, "-v"]))

View File

@@ -0,0 +1,132 @@
"""Tests for the Violet venue-agnostic OB provider seam."""
from __future__ import annotations
import math
import sys
import time
import pytest
from pydantic import ValidationError
sys.path.insert(0, "/mnt/dolphinng5_predict")
sys.path.insert(0, "/mnt/dolphinng5_predict/nautilus_dolphin")
from nautilus_dolphin.nautilus.ob_features import OBFeatureEngine
from prod.clean_arch.violet.venue_ob_provider import (
MockTickVenueOBProvider,
VenueOBTick,
VioletVenueOBProvider,
)
def _records(base_ts: float = 1_700_000_000.0):
assets = ("BTCUSDT", "ETHUSDT")
rows = []
for asset_idx, asset in enumerate(assets):
for snap_idx in range(3):
ts = base_ts + 30.0 * snap_idx
bias = 0.12 if asset_idx == 0 else -0.04
bid = tuple(float((1 + bias) * (i + 1) * 1000.0) for i in range(5))
ask = tuple(float((1 - bias) * (i + 1) * 1000.0) for i in range(5))
depth = tuple(float(v / (100.0 + asset_idx)) for v in bid)
ask_depth = tuple(float(v / (100.0 + asset_idx)) for v in ask)
rows.append(
VenueOBTick(
timestamp=ts,
asset=asset,
bid_notional_levels=bid,
ask_notional_levels=ask,
bid_depth_levels=depth,
ask_depth_levels=ask_depth,
)
)
return rows
def test_provider_conforms_and_returns_snapshots():
provider = VioletVenueOBProvider(ticks=_records(), tolerance_s=60.0)
assert provider.get_assets() == ["BTCUSDT", "ETHUSDT"]
assert provider.get_snapshot_count("BTCUSDT") == 3
assert provider.get_all_timestamps("ETHUSDT").tolist() == [1_700_000_000.0, 1_700_000_030.0, 1_700_000_060.0]
snap = provider.get_snapshot("BTCUSDT", 1_700_000_031.0)
assert snap is not None
assert snap.asset == "BTCUSDT"
assert snap.bid_notional.shape == (5,)
assert snap.ask_notional.shape == (5,)
assert all(math.isfinite(float(v)) for v in snap.bid_notional)
assert all(math.isfinite(float(v)) for v in snap.ask_notional)
def test_poison_rejected_at_construction():
with pytest.raises(ValidationError):
VenueOBTick(
timestamp=-1.0,
asset="BTCUSDT",
bid_notional_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
ask_notional_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
bid_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
ask_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
)
with pytest.raises(ValidationError):
VenueOBTick(
timestamp=1.0,
asset="BTCUSDT",
bid_notional_levels=(1.0, 2.0, 3.0, 4.0),
ask_notional_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
bid_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
ask_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
)
with pytest.raises(ValidationError):
VenueOBTick(
timestamp=1.0,
asset="BTCUSDT",
bid_notional_levels=(1.0, 2.0, 3.0, 4.0, float("nan")),
ask_notional_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
bid_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
ask_depth_levels=(1.0, 2.0, 3.0, 4.0, 5.0),
)
def test_engine_step_live_and_get_market(monkeypatch):
base_ts = 1_700_000_000.0
provider = MockTickVenueOBProvider(
assets=("BTCUSDT", "ETHUSDT", "SOLUSDT"),
num_snapshots=4,
base_timestamp=base_ts,
interval_s=30.0,
imbalance_biases={"BTCUSDT": 0.14, "ETHUSDT": -0.06, "SOLUSDT": -0.02},
)
engine = OBFeatureEngine(provider)
monkeypatch.setattr(time, "time", lambda: base_ts + 31.0)
engine.step_live(["BTCUSDT", "ETHUSDT", "SOLUSDT"], bar_idx=7)
market = engine.get_market(7, ["BTCUSDT", "ETHUSDT", "SOLUSDT"])
placement = engine.get_placement("BTCUSDT", 7)
signal = engine.get_signal("BTCUSDT", 7)
for value in (market.median_imbalance, market.agreement_pct, market.depth_pressure):
assert math.isfinite(float(value))
for value in (placement.depth_1pct_usd, placement.depth_quality, placement.fill_probability, placement.spread_proxy_bps):
assert math.isfinite(float(value))
for value in (signal.imbalance, signal.imbalance_ma5, signal.imbalance_persistence, signal.depth_asymmetry, signal.withdrawal_velocity):
assert math.isfinite(float(value))
def test_callable_refresh_loads_ticks():
base_ts = 1_700_000_000.0
def source(asset: str):
for row in _records(base_ts):
if row.asset == asset:
yield row
provider = VioletVenueOBProvider(tick_source=source, assets=("BTCUSDT", "ETHUSDT"))
assert provider.get_snapshot_count("BTCUSDT") == 3
assert provider.get_snapshot("ETHUSDT", base_ts + 10.0) is not None

View File

@@ -0,0 +1,725 @@
"""VIOLET V4 execution runner.
This is the live-capable VIOLET runner boundary:
NG7 scan -> BLUE-faithful VIOLET decision -> ExecIntent
-> DITAv2 KernelIntent -> DITAv2 BingX VST venue execution
Safety is explicit rather than castrated: the module can submit real orders via
``ExecutionKernel.process_intent_async`` when the operator runs the V4 launcher
with VST keys and arming gates green. Unit tests inject fake kernels and never
touch a venue.
"""
from __future__ import annotations
import argparse
import asyncio
import gc
import json
import logging
import math
import os
import sys
import threading
import time
import uuid
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Callable, Mapping, Optional
from prod.clean_arch.dita_v2.contracts import (
KernelCommandType,
KernelIntent,
KernelOutcome,
TradeSide,
)
from .contracts_v3 import ExecIntent
from .decision_engine import ShadowDecision
from .exec_intent import to_exec_intent
RUNNER_CONTRACT_VERSION = "violet-v4-exec-runner-contracts-20260626"
LOGGER = logging.getLogger("violet.v4_execution_runner")
_ASEX_SRC = Path("/mnt/dolphinng5_predict/ASEx/src")
if _ASEX_SRC.exists() and str(_ASEX_SRC) not in sys.path:
sys.path.insert(0, str(_ASEX_SRC))
try: # pragma: no cover - exercised when ASEx package is installed/available.
from asex.guarded import ASExGuardedState
from asex.worker import ASExWorker
except Exception: # pragma: no cover - fallback is covered by unit tests.
ASExGuardedState = None
ASExWorker = None
def _finite_positive(value: float, name: str) -> float:
out = float(value)
if not math.isfinite(out) or out <= 0.0:
raise ValueError(f"{name} must be finite and > 0")
return out
def _non_empty(value: str, name: str) -> str:
out = str(value or "").strip()
if not out:
raise ValueError(f"{name} must be non-empty")
return out
def _trade_side(side: str) -> TradeSide:
normalized = str(side or "").strip().upper()
if normalized == "LONG":
return TradeSide.LONG
if normalized == "SHORT":
return TradeSide.SHORT
raise ValueError(f"unsupported VIOLET side: {side!r}")
def _action_from_intent(intent: ExecIntent, explicit: Optional[KernelCommandType]) -> KernelCommandType:
if explicit is not None:
return explicit
if intent.reason == "ENTRY":
return KernelCommandType.ENTER
if intent.reason == "EXIT":
return KernelCommandType.EXIT
raise ValueError(f"unsupported VIOLET intent reason: {intent.reason!r}")
def exec_intent_to_kernel_intent(
intent: ExecIntent,
*,
reference_price: float,
trade_id: str,
slot_id: int = 0,
intent_id: Optional[str] = None,
action: Optional[KernelCommandType] = None,
order_type: str = "MARKET",
limit_price: float = 0.0,
metadata: Optional[Mapping[str, Any]] = None,
timestamp: Optional[datetime] = None,
) -> KernelIntent:
"""Convert a VIOLET ``ExecIntent`` into a DITAv2 ``KernelIntent``.
The function is intentionally pure and strict. It does not fetch prices,
allocate state outside the returned object, or contact a venue. Live runner
code must pass a fresh reference price with provenance in ``metadata``.
"""
price = _finite_positive(reference_price, "reference_price")
qty = _finite_positive(float(intent.qty), "intent.qty")
leverage = _finite_positive(float(intent.exchange_leverage), "intent.exchange_leverage")
tid = _non_empty(trade_id, "trade_id")
oid = _non_empty(intent_id or f"violet-v4-{uuid.uuid4().hex}", "intent_id")
order = _non_empty(order_type, "order_type").upper()
if order == "LIMIT":
_finite_positive(limit_price, "limit_price")
elif float(limit_price or 0.0) < 0.0:
raise ValueError("limit_price must be >= 0")
merged_metadata: dict[str, Any] = dict(metadata or {})
merged_metadata.update(
{
"violet_contract_version": RUNNER_CONTRACT_VERSION,
"violet_ts_ns": int(intent.ts_ns),
"violet_maker_policy": intent.maker_policy,
"violet_target_notional": float(intent.target_notional),
"violet_reason": intent.reason,
}
)
return KernelIntent(
timestamp=timestamp or datetime.now(timezone.utc),
intent_id=oid,
trade_id=tid,
slot_id=int(slot_id),
asset=_non_empty(intent.asset, "intent.asset"),
side=_trade_side(intent.side),
action=_action_from_intent(intent, action),
reference_price=price,
target_size=qty,
leverage=leverage,
reason=f"violet_v4:{intent.reason.lower()}",
metadata=merged_metadata,
order_type=order,
limit_price=float(limit_price or 0.0),
)
@dataclass(frozen=True)
class VioletV4RunnerConfig:
"""Operator/runtime knobs for the V4 execution runner."""
session_id: str
capital: float = 69_000.0
slot_id: int = 0
maker_policy: str = "maker_both"
order_type: str = "MARKET"
limit_price: float = 0.0
poll_interval_s: float = 0.25
queue_maxsize: int = 4096
max_submissions: int = 0
max_notional_usdt: float = 0.0
assume_vol_ok: bool = False
require_arming: bool = True
report_dir: str = "/mnt/vp-VIOLET_main/prod/VIOLET_dev/reports"
hz_cluster: str = "dolphin"
hz_host: str = "localhost:5701"
hz_map: str = "DOLPHIN_FEATURES"
hz_key: str = "latest_eigen_scan"
snapshot_symbol: str = "BTCUSDT"
@classmethod
def from_env(cls) -> "VioletV4RunnerConfig":
return cls(
session_id=os.environ.get("DOLPHIN_VIOLET_SESSION_ID", uuid.uuid4().hex),
capital=float(os.environ.get("DOLPHIN_VIOLET_CAPITAL", "69000")),
slot_id=int(os.environ.get("DOLPHIN_VIOLET_SLOT_ID", "0")),
maker_policy=os.environ.get("DOLPHIN_VIOLET_MAKER_POLICY", "maker_both"),
order_type=os.environ.get("DOLPHIN_VIOLET_ORDER_TYPE", "MARKET"),
limit_price=float(os.environ.get("DOLPHIN_VIOLET_LIMIT_PRICE", "0")),
poll_interval_s=float(os.environ.get("DOLPHIN_VIOLET_POLL_INTERVAL_SEC", "0.25")),
queue_maxsize=int(os.environ.get("DOLPHIN_VIOLET_SCAN_QUEUE_MAX", "4096")),
max_submissions=int(os.environ.get("DOLPHIN_VIOLET_MAX_SUBMISSIONS", "0")),
max_notional_usdt=float(os.environ.get("DOLPHIN_VIOLET_MAX_NOTIONAL_USDT", "0")),
assume_vol_ok=str(os.environ.get("DOLPHIN_VIOLET_ASSUME_VOL_OK", "0")).lower()
in {"1", "true", "yes", "on"},
require_arming=str(os.environ.get("DOLPHIN_VIOLET_REQUIRE_ARMING", "1")).lower()
not in {"0", "false", "no", "off"},
report_dir=os.environ.get(
"DOLPHIN_VIOLET_REPORT_DIR",
"/mnt/vp-VIOLET_main/prod/VIOLET_dev/reports",
),
hz_cluster=os.environ.get("HZ_CLUSTER", "dolphin"),
hz_host=os.environ.get("HZ_HOST", "localhost:5701"),
hz_map=os.environ.get("DOLPHIN_VIOLET_SCAN_MAP", "DOLPHIN_FEATURES"),
hz_key=os.environ.get("DOLPHIN_VIOLET_SCAN_KEY", "latest_eigen_scan"),
snapshot_symbol=os.environ.get("DOLPHIN_VIOLET_SNAPSHOT_SYMBOL", "BTCUSDT"),
)
class _LocalGuardedState:
"""Tiny ASEx-compatible fallback for tests or missing ASEx install."""
def __init__(self) -> None:
self._lock = threading.Lock()
self.last_scan_number = -1
self.scans_seen = 0
self.decisions = 0
self.submissions = 0
self.outcomes = 0
self.errors = 0
self.last_trade_id = ""
self.last_intent_id = ""
def mutate(self, mutation: Mapping[str, Any]) -> dict[str, Any]:
with self._lock:
return self._apply(dict(mutation))
def _apply(self, mutation: dict[str, Any]) -> dict[str, Any]:
op = str(mutation.get("op") or "")
if op == "scan":
sn = int(mutation.get("scan_number") or 0)
if sn <= self.last_scan_number:
return {"accepted": False, "reason": "duplicate_scan", "last_scan_number": self.last_scan_number}
self.last_scan_number = sn
self.scans_seen += 1
return {"accepted": True, "last_scan_number": self.last_scan_number}
if op == "decision":
self.decisions += 1
return {"accepted": True, "decisions": self.decisions}
if op == "submission":
self.submissions += 1
self.last_trade_id = str(mutation.get("trade_id") or "")
self.last_intent_id = str(mutation.get("intent_id") or "")
return {"accepted": True, "submissions": self.submissions}
if op == "outcome":
self.outcomes += 1
return {"accepted": True, "outcomes": self.outcomes}
if op == "error":
self.errors += 1
return {"accepted": True, "errors": self.errors}
return {"accepted": False, "reason": f"unknown_op:{op}"}
if ASExGuardedState is not None:
class VioletRunnerGuardedState(ASExGuardedState[dict[str, Any], dict[str, Any]]):
"""ASEx single-writer state for scan/order lifecycle counters."""
def __init__(self) -> None:
super().__init__()
self._state = _LocalGuardedState()
def _validate(self, mutation: dict[str, Any]) -> bool:
return str(mutation.get("op") or "") in {
"scan",
"decision",
"submission",
"outcome",
"error",
}
def _apply(self, mutation: dict[str, Any]) -> dict[str, Any]:
return self._state._apply(dict(mutation))
else:
VioletRunnerGuardedState = _LocalGuardedState # type: ignore[assignment]
class SerialRunnerState:
"""Small wrapper around ASExWorker with a deterministic local fallback."""
def __init__(self) -> None:
self.backend = VioletRunnerGuardedState()
self.worker = ASExWorker(self.backend) if ASExWorker is not None else None
def mutate(self, mutation: Mapping[str, Any], *, timeout: float = 5.0) -> dict[str, Any]:
payload = dict(mutation)
if self.worker is not None:
return self.worker.mutate(payload).result(timeout=timeout)
return self.backend.mutate(payload) # type: ignore[attr-defined]
def close(self) -> None:
if self.worker is not None:
self.worker.close(timeout=2.0)
def _parse_scan_payload(raw: Any) -> dict[str, Any]:
if raw is None:
return {}
if isinstance(raw, str):
data = json.loads(raw)
elif isinstance(raw, Mapping):
data = dict(raw)
else:
return {}
if isinstance(data, dict) and data.get("version") == "NG7":
try:
from prod.clean_arch.adapters.eigen_scan_normalizer import normalize_ng7_scan
data = normalize_ng7_scan(data)
except Exception:
LOGGER.debug("NG7 normalize failed; using raw scan", exc_info=True)
return data if isinstance(data, dict) else {}
def reference_price_from_scan(payload: Mapping[str, Any], asset: str) -> float:
"""Extract the decision asset price from an NG7/universe payload."""
symbol = str(asset or "").upper()
assets = payload.get("assets") or []
prices = payload.get("asset_prices") or []
if isinstance(assets, list) and isinstance(prices, list):
for raw_asset, raw_price in zip(assets, prices):
if str(raw_asset).upper() != symbol:
continue
price = _finite_positive(float(raw_price), "reference_price")
return price
result = payload.get("result") if isinstance(payload, Mapping) else None
if isinstance(result, Mapping):
result_asset = str(result.get("asset") or payload.get("target_asset") or "").upper()
if result_asset == symbol:
return _finite_positive(float(result.get("price") or payload.get("price")), "reference_price")
payload_asset = str(payload.get("asset") or payload.get("target_asset") or "").upper()
if payload_asset == symbol and payload.get("price") is not None:
return _finite_positive(float(payload.get("price")), "reference_price")
raise ValueError(f"scan payload has no fresh price for {symbol}")
def _decision_from_shadow(
shadow: dict[str, Any],
payload: dict[str, Any],
*,
scan_number: int,
now_ns: int,
vel_div: float,
vol_ok: bool,
) -> Optional[ShadowDecision]:
"""Run the existing BLUE-faithful VIOLET shadow path and return the decision."""
from .shadow_live_factors import _ensure_ob_engine
shadow["engine"].observe(payload, scan_number)
live_source = shadow.get("live_source")
if live_source is None:
return None
ob_engine = _ensure_ob_engine(shadow, payload)
live_result = live_source(
shadow["client"],
scan_history=shadow["scan_history"],
selector=shadow["selector"],
ob_engine=ob_engine,
bar_idx=shadow.get("ob_bar_idx", 0),
prior_boost_beta=shadow.get("prior_boost_beta"),
)
shadow["last_live_source"] = live_result
shadow["ob_bar_idx"] = shadow.get("ob_bar_idx", 0) + 1
ab = getattr(live_result, "acb_boost", None)
bb = getattr(live_result, "acb_beta", None)
if ab is not None and bb is not None:
shadow["prior_boost_beta"] = (ab, bb)
decision = shadow["engine"].decide(
now_ns=now_ns,
scan_number=scan_number,
capital=shadow["capital"],
vel_div=vel_div,
vol_ok=vol_ok,
factors=live_result.factors,
)
if decision is None:
return None
journal = shadow.get("journal")
if journal is not None:
journal.journal(decision, mono_ns=now_ns)
return decision
class HazelcastNG7ScanSource:
"""Event-listener first, polling fallback source for NG7 scans."""
def __init__(self, config: VioletV4RunnerConfig) -> None:
self.config = config
self.client = None
self.map = None
self.blocking_map = None
self.queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue(maxsize=config.queue_maxsize)
self._loop: Optional[asyncio.AbstractEventLoop] = None
self._last_polled_scan = -1
async def connect(self) -> None:
import hazelcast
self._loop = asyncio.get_running_loop()
self.client = hazelcast.HazelcastClient(
cluster_name=self.config.hz_cluster,
cluster_members=[self.config.hz_host],
)
self.map = self.client.get_map(self.config.hz_map)
self.blocking_map = self.map.blocking()
def on_entry(event: Any) -> None:
try:
payload = _parse_scan_payload(getattr(event, "value", None))
if not payload:
return
loop = self._loop
if loop is None:
return
loop.call_soon_threadsafe(self._offer_payload, payload)
except Exception:
LOGGER.debug("scan listener callback failed", exc_info=True)
self.map.add_entry_listener(
include_value=True,
key=self.config.hz_key,
updated_func=on_entry,
added_func=on_entry,
)
def _offer_payload(self, payload: dict[str, Any]) -> None:
try:
self.queue.put_nowait(payload)
except asyncio.QueueFull:
try:
self.queue.get_nowait()
except asyncio.QueueEmpty:
pass
self.queue.put_nowait(payload)
async def next_payload(self) -> dict[str, Any]:
try:
return await asyncio.wait_for(self.queue.get(), timeout=self.config.poll_interval_s)
except asyncio.TimeoutError:
raw = self.blocking_map.get(self.config.hz_key)
payload = _parse_scan_payload(raw)
sn = int(payload.get("scan_number") or 0)
if sn <= self._last_polled_scan:
return {}
self._last_polled_scan = sn
return payload
async def close(self) -> None:
if self.client is not None:
self.client.shutdown()
class VioletV4ExecutionRunner:
"""Live-capable runner: decision path to actual DITAv2 execution."""
def __init__(
self,
*,
bundle: Any,
shadow: dict[str, Any],
config: VioletV4RunnerConfig,
state: Optional[SerialRunnerState] = None,
decision_step: Optional[
Callable[[dict[str, Any], dict[str, Any], int, int, float, bool], Optional[ShadowDecision]]
] = None,
) -> None:
self.bundle = bundle
self.shadow = shadow
self.config = config
self.state = state or SerialRunnerState()
self.decision_step = decision_step or (
lambda sh, payload, sn, now_ns, vd, vol_ok: _decision_from_shadow(
sh,
payload,
scan_number=sn,
now_ns=now_ns,
vel_div=vd,
vol_ok=vol_ok,
)
)
self._closed = False
async def connect(self) -> None:
connect = getattr(self.bundle.venue, "connect", None)
if connect is not None:
result = connect()
if asyncio.iscoroutine(result):
await result
def close(self) -> None:
if self._closed:
return
self._closed = True
try:
close = getattr(self.bundle, "close", None)
if close is not None:
close()
finally:
self.state.close()
def _capital(self) -> float:
try:
cap = float(self.bundle.kernel.account.snapshot.capital)
if math.isfinite(cap) and cap > 0.0:
return cap
except Exception:
pass
return float(self.config.capital)
def _maybe_cap_exec_intent(self, intent: ExecIntent, reference_price: float) -> ExecIntent:
cap = float(self.config.max_notional_usdt or 0.0)
if cap <= 0.0 or float(intent.target_notional) <= cap:
return intent
return ExecIntent(
asset=intent.asset,
side=intent.side,
qty=cap / reference_price,
exchange_leverage=intent.exchange_leverage,
maker_policy=intent.maker_policy,
target_notional=cap,
ts_ns=intent.ts_ns,
reason=intent.reason,
)
async def process_scan_payload(self, raw_payload: Any) -> Optional[KernelOutcome]:
payload = _parse_scan_payload(raw_payload)
if not payload:
return None
scan_number = int(payload.get("scan_number") or 0)
if scan_number <= 0:
return None
scan_state = self.state.mutate({"op": "scan", "scan_number": scan_number})
if not scan_state.get("accepted"):
return None
vel_div = float(payload.get("vel_div") or 0.0)
if not math.isfinite(vel_div):
vel_div = 0.0
vol_ok = bool(payload.get("vol_ok", True))
if self.config.assume_vol_ok:
vol_ok = True
now_ns = int(time.monotonic_ns())
decision = self.decision_step(self.shadow, payload, scan_number, now_ns, vel_div, vol_ok)
if decision is None:
return None
self.state.mutate({"op": "decision", "scan_number": scan_number, "asset": decision.asset})
reference_price = reference_price_from_scan(payload, decision.asset)
exec_intent = to_exec_intent(
decision,
capital=self._capital(),
reference_price=reference_price,
maker_policy=self.config.maker_policy,
)
exec_intent = self._maybe_cap_exec_intent(exec_intent, reference_price)
trade_id = f"VIOLET-{self.config.session_id[:8]}-{scan_number}"
intent_id = f"violet-v4-{self.config.session_id[:8]}-{scan_number}"
kernel_intent = exec_intent_to_kernel_intent(
exec_intent,
reference_price=reference_price,
trade_id=trade_id,
slot_id=self.config.slot_id,
intent_id=intent_id,
order_type=self.config.order_type,
limit_price=self.config.limit_price,
metadata={
"violet_session_id": self.config.session_id,
"scan_number": scan_number,
"price_source": "ng7_scan",
"raw_notional_before_runner_cap": float(decision.target_exposure),
},
)
submission = self.state.mutate(
{"op": "submission", "scan_number": scan_number, "trade_id": trade_id, "intent_id": intent_id}
)
if self.config.max_submissions and int(submission.get("submissions") or 0) > self.config.max_submissions:
LOGGER.critical("max submissions reached; refusing scan=%s trade_id=%s", scan_number, trade_id)
return None
LOGGER.warning(
"VIOLET V4 LIVE SUBMIT scan=%s asset=%s side=%s qty=%.8f lev=%.2f ref=%.8f trade=%s",
scan_number,
kernel_intent.asset,
kernel_intent.side.value,
kernel_intent.target_size,
kernel_intent.leverage,
kernel_intent.reference_price,
trade_id,
)
try:
outcome = await self.bundle.kernel.process_intent_async(kernel_intent)
self.state.mutate(
{
"op": "outcome",
"scan_number": scan_number,
"trade_id": trade_id,
"accepted": bool(getattr(outcome, "accepted", False)),
"diagnostic": str(getattr(getattr(outcome, "diagnostic_code", ""), "value", "")),
}
)
return outcome
except Exception as exc:
self.state.mutate({"op": "error", "scan_number": scan_number, "error": str(exc)})
raise
def _apply_runtime_optimizations() -> None:
if gc.isenabled():
gc.disable()
def _build_shadow(config: VioletV4RunnerConfig) -> dict[str, Any]:
from prod.ch_writer import ch_put_violet
from .decision_engine import VioletDecisionEngine
from .shadow_journal import VioletDecisionJournal
from .shadow_live_factors import build_shadow_live_source
thr = float(os.environ.get("DOLPHIN_VIOLET_ENTRY_VEL_DIV_THRESHOLD", "-0.02"))
return {
"engine": VioletDecisionEngine(entry_vel_div_threshold=thr),
"journal": VioletDecisionJournal(sink=ch_put_violet, session_id=config.session_id),
"capital": float(config.capital),
**build_shadow_live_source(),
}
def _build_real_bundle(config: VioletV4RunnerConfig) -> Any:
from prod.clean_arch.dita_v2.launcher import build_launcher_bundle
from prod.launch_dolphin_violet import (
DDL_APPLY_CMD,
_apply_violet_env,
_preflight_clickhouse,
_violet_table_present,
build_bingx_exec_client_config,
)
from .v4_arming import assess_v4_arming, write_v4_arming_report
_apply_violet_env()
missing = _preflight_clickhouse()
if missing:
raise RuntimeError(f"dolphin_violet tables missing: {missing}; run {DDL_APPLY_CMD}")
if not _violet_table_present("violet_decisions"):
raise RuntimeError(f"dolphin_violet table missing: violet_decisions; run {DDL_APPLY_CMD}")
report = assess_v4_arming(report_dir=config.report_dir, launcher_mode="execution")
write_v4_arming_report(report, report_dir=config.report_dir)
if config.require_arming and not report.can_arm:
raise RuntimeError(f"VIOLET V4 arming refused: {report.reasons}")
return build_launcher_bundle(
venue_mode="BINGX",
max_slots=1,
prefix="violet",
bingx_config=build_bingx_exec_client_config(),
)
async def run_live(
config: Optional[VioletV4RunnerConfig] = None,
*,
stop_after_first_submission: bool = False,
) -> None:
"""Run the actual VIOLET V4 BingX VST execution loop."""
cfg = config or VioletV4RunnerConfig.from_env()
_apply_runtime_optimizations()
bundle = _build_real_bundle(cfg)
shadow = _build_shadow(cfg)
source = HazelcastNG7ScanSource(cfg)
runner = VioletV4ExecutionRunner(bundle=bundle, shadow=shadow, config=cfg)
await runner.connect()
await source.connect()
LOGGER.critical(
"VIOLET V4 EXECUTION RUNNER ARMED: session=%s venue=BINGX_VST hz=%s[%s]",
cfg.session_id,
cfg.hz_map,
cfg.hz_key,
)
try:
while True:
payload = await source.next_payload()
if payload:
outcome = await runner.process_scan_payload(payload)
if stop_after_first_submission and outcome is not None:
return
await asyncio.sleep(0)
finally:
await source.close()
runner.close()
def main(argv: Optional[list[str]] = None) -> int:
parser = argparse.ArgumentParser(description="VIOLET V4 live BingX VST execution runner")
parser.add_argument("--once", action="store_true", help="process one queued/polled scan then exit")
args = parser.parse_args(argv)
config = VioletV4RunnerConfig.from_env()
if args.once:
config = dataclass_replace(config, max_submissions=1)
asyncio.run(run_live(config, stop_after_first_submission=args.once))
return 0
def dataclass_replace(config: VioletV4RunnerConfig, **updates: Any) -> VioletV4RunnerConfig:
data = config.__dict__.copy()
data.update(updates)
return VioletV4RunnerConfig(**data)
__all__ = [
"HazelcastNG7ScanSource",
"RUNNER_CONTRACT_VERSION",
"SerialRunnerState",
"VioletV4ExecutionRunner",
"VioletV4RunnerConfig",
"exec_intent_to_kernel_intent",
"main",
"reference_price_from_scan",
"run_live",
]
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -0,0 +1,75 @@
"""V4 arm-readiness reporter — DARK, read-only, non-colliding.
Aggregates V4 gate outcomes into a typed report answering: "is V4 ready
to ARM the live canary?" Does NOT run gates — CONSUMES their results
(passed in) and applies fail-closed readiness logic.
Spec: prod/docs/VIOLET_PASS_MM1_V4_READINESS_REPORTER.md
"""
from __future__ import annotations
from .domain import StrictModel, typed
__all__ = ["V4ReadinessReport", "assess_v4_readiness"]
class V4ReadinessReport(StrictModel):
"""Typed V4 readiness report — all six gates must pass for ready_to_arm."""
creds_ok: bool
canary_passed: bool
arming_gate_present: bool
conviction_bit_identical: bool
selection_parity_ok: bool
feed_live: bool
reasons: list[str]
@property
def ready_to_arm(self) -> bool:
return all((
self.creds_ok,
self.canary_passed,
self.arming_gate_present,
self.conviction_bit_identical,
self.selection_parity_ok,
self.feed_live,
))
@typed
def assess_v4_readiness(
*,
creds_ok: bool,
canary_passed: bool,
arming_gate_present: bool,
conviction_bit_identical: bool,
selection_parity_ok: bool,
feed_live: bool,
) -> V4ReadinessReport:
"""Build a V4ReadinessReport from gate outcomes. Fail-closed: any False
means ready_to_arm is False. No partial-arm."""
reasons: list[str] = []
if not creds_ok:
reasons.append("creds_ok: PASS2.1 not satisfied")
if not canary_passed:
reasons.append("canary_passed: PASS2.3 canary not round-tripped flat")
if not arming_gate_present:
reasons.append("arming_gate_present: PASS2.2 not satisfied")
if not conviction_bit_identical:
reasons.append("conviction_bit_identical: PASS2.5 fidelity mismatch")
if not selection_parity_ok:
reasons.append("selection_parity_ok: PASS2.7/3.1 parity mismatch")
if not feed_live:
reasons.append("feed_live: PASS2.6 HZ feed stale or deaf")
return V4ReadinessReport(
creds_ok=creds_ok,
canary_passed=canary_passed,
arming_gate_present=arming_gate_present,
conviction_bit_identical=conviction_bit_identical,
selection_parity_ok=selection_parity_ok,
feed_live=feed_live,
reasons=reasons,
)

View File

@@ -0,0 +1,178 @@
"""VIOLET venue-agnostic OB provider seam.
This is a read-only, in-memory seam for future non-BLUE order book sources.
It produces the exact ``OBSnapshot`` shape expected by BLUE's
``OBFeatureEngine`` without opening any live exchange connection.
"""
from __future__ import annotations
from bisect import bisect_left
try:
from beartype.typing import Annotated, Callable, Iterable, Optional
except ImportError: # pragma: no cover - beartype always present in prod
from typing import Annotated, Callable, Iterable, Optional
import numpy as np
from pydantic import Field
from .domain import StrictModel, Symbol, typed
try:
from nautilus_dolphin.nautilus.ob_provider import OBProvider, OBSnapshot
except ImportError: # pragma: no cover - import path fallback for direct runs
from nautilus_dolphin.nautilus.ob_provider import OBProvider, OBSnapshot # type: ignore
OBLevel = Annotated[float, Field(ge=0.0, allow_inf_nan=False)]
Level5 = tuple[OBLevel, OBLevel, OBLevel, OBLevel, OBLevel]
class VenueOBTick(StrictModel):
"""Normalized OB tick for one asset at one point in time."""
timestamp: Annotated[float, Field(ge=0.0, allow_inf_nan=False)]
asset: Symbol
bid_notional_levels: Level5
ask_notional_levels: Level5
bid_depth_levels: Level5
ask_depth_levels: Level5
class VioletVenueOBProvider(OBProvider):
"""Venue-agnostic, in-memory OB provider.
Input comes from either an injected callable or a flat iterable of
``VenueOBTick`` records. No exchange transport lives here.
"""
def __init__(
self,
*,
ticks: Optional[Iterable[VenueOBTick]] = None,
tick_source: Optional[Callable[[str], Iterable[VenueOBTick]]] = None,
assets: Optional[Iterable[str]] = None,
tolerance_s: float = 30.0,
) -> None:
self.tolerance_s = float(tolerance_s)
self._tick_source = tick_source
self._assets = sorted({str(a) for a in assets or [] if str(a).strip()})
self._ticks: dict[str, list[VenueOBTick]] = {}
self._timestamps: dict[str, np.ndarray] = {}
if ticks is not None:
self.load_ticks(ticks)
if tick_source is not None and self._assets:
self.refresh()
@typed
def load_ticks(self, ticks: Iterable[VenueOBTick]) -> None:
grouped: dict[str, list[VenueOBTick]] = {}
for tick in ticks:
grouped.setdefault(tick.asset, []).append(tick)
for asset, rows in grouped.items():
rows.sort(key=lambda t: t.timestamp)
self._ticks[asset] = rows
self._timestamps[asset] = np.array([t.timestamp for t in rows], dtype=np.float64)
if asset not in self._assets:
self._assets.append(asset)
self._assets.sort()
@typed
def refresh(self) -> None:
if self._tick_source is None:
return
for asset in self._assets:
self.load_ticks(self._tick_source(asset))
def _to_snapshot(self, tick: VenueOBTick) -> OBSnapshot:
return OBSnapshot(
timestamp=float(tick.timestamp),
asset=tick.asset,
bid_notional=np.array(tick.bid_notional_levels, dtype=np.float64),
ask_notional=np.array(tick.ask_notional_levels, dtype=np.float64),
bid_depth=np.array(tick.bid_depth_levels, dtype=np.float64),
ask_depth=np.array(tick.ask_depth_levels, dtype=np.float64),
)
def get_snapshot(self, asset: str, timestamp: float) -> Optional[OBSnapshot]:
rows = self._ticks.get(asset)
if not rows:
return None
ts_arr = self._timestamps.get(asset)
if ts_arr is None or len(ts_arr) == 0:
return None
idx = bisect_left(ts_arr, float(timestamp))
candidates = [max(0, idx - 1), min(idx, len(ts_arr) - 1)]
best_idx = candidates[0]
best_dist = abs(ts_arr[best_idx] - timestamp)
for cand in candidates[1:]:
dist = abs(ts_arr[cand] - timestamp)
if dist < best_dist:
best_dist = dist
best_idx = cand
if best_dist > self.tolerance_s:
return None
return self._to_snapshot(rows[best_idx])
def get_assets(self) -> list[str]:
return list(self._assets)
def get_all_timestamps(self, asset: str) -> np.ndarray:
ts = self._timestamps.get(asset)
return np.array([], dtype=np.float64) if ts is None else ts.copy()
def get_snapshot_count(self, asset: str) -> int:
rows = self._ticks.get(asset)
return len(rows) if rows is not None else 0
def get_snapshot_by_index(self, asset: str, idx: int) -> Optional[OBSnapshot]:
rows = self._ticks.get(asset)
if rows is None or idx < 0 or idx >= len(rows):
return None
return self._to_snapshot(rows[idx])
class MockTickVenueOBProvider(VioletVenueOBProvider):
"""Deterministic synthetic tick source for tests."""
def __init__(
self,
*,
assets: Optional[Iterable[str]] = None,
num_snapshots: int = 8,
base_timestamp: float = 1_700_000_000.0,
interval_s: float = 30.0,
base_notional: float = 100_000.0,
depth_scale: float = 1.0,
imbalance_biases: Optional[dict[str, float]] = None,
tolerance_s: float = 60.0,
) -> None:
assets = list(assets or ("BTCUSDT", "ETHUSDT"))
ticks: list[VenueOBTick] = []
level_weights = (1.0, 2.0, 3.0, 4.0, 5.0)
for asset_idx, asset in enumerate(assets):
bias = (imbalance_biases or {}).get(asset, 0.08 if asset_idx % 2 == 0 else -0.08)
for snap_idx in range(num_snapshots):
ts = base_timestamp + snap_idx * interval_s
drift = 1.0 + 0.01 * snap_idx
bid_mult = 1.0 + bias
ask_mult = 1.0 - bias
bid_not = tuple(
float(base_notional * depth_scale * drift * w * bid_mult) for w in level_weights
)
ask_not = tuple(
float(base_notional * depth_scale * drift * w * ask_mult) for w in level_weights
)
approx_price = 100.0 + 5.0 * asset_idx
bid_dep = tuple(float(v / approx_price) for v in bid_not)
ask_dep = tuple(float(v / approx_price) for v in ask_not)
ticks.append(
VenueOBTick(
timestamp=float(ts),
asset=asset,
bid_notional_levels=bid_not,
ask_notional_levels=ask_not,
bid_depth_levels=bid_dep,
ask_depth_levels=ask_dep,
)
)
super().__init__(ticks=ticks, assets=assets, tolerance_s=tolerance_s)

View File

@@ -34,7 +34,7 @@ EXPECTED_TABLES = {
"status_snapshots", "trade_events", "v7_decision_events", "status_snapshots", "trade_events", "v7_decision_events",
"adaptive_exit_shadow", "fee_settled_events", "adaptive_exit_shadow", "fee_settled_events",
"sc_bucket_gauge_shadow", "sc_threshold_advisor_shadow", "sc_bucket_gauge_shadow", "sc_threshold_advisor_shadow",
"violet_feed_divergence", "violet_feed_divergence", "violet_decisions",
} }

View File

@@ -0,0 +1,105 @@
# BEADS as PASS Tracker — Evaluation & Recommendation
**Author:** pi_nvnemo (UV Overseer)
**Date:** 2026-07-08
**Context:** UV_OVERSEER_CHARTER__PI.md §6 task — evaluate beads vs bus+doc for PASS board
---
## Current State
| Tracker | Status |
|---------|--------|
| **h5i bus** | Active — dispatch, ACK, status updates |
| **Status doc** | Not yet created (charter says "track PASSes on the bus + a short status doc") |
| **beads** | Installed, `.beads/` exists at repo root (prefix `dp`), 1 existing PRODGREEN issue |
---
## What Beads Gives Us Over Bus+Doc
| Capability | h5i Bus + Doc | Beads |
|------------|---------------|-------|
| **Dependency graph** | Manual (doc) | Native (`br dep add`, `br graph`) |
| **Task hierarchy** | Flat (doc sections) | Epic → child beads (parent/child) |
| **State machine** | Manual (doc) | Enforced (open → in_progress → closed) |
| **Acceptance criteria** | Doc prose | Structured fields (`acceptance_criteria`, `test_command`) |
| **Audit trail** | Bus history + doc edits | Immutable JSONL + SQL + `br audit` |
| **Handoff protocol** | Informal | Formal (`br audit --message`, `br ready`) |
| **Multi-agent isolation** | Bus channels | Separate workspace per refactor stream |
| **Low-skill agent onboarding** | Ad-hoc | Bounded task template + dependency chain |
| **Query/Filter** | grep/awk | `br ready`, `br list`, `br status`, SQL |
| **Backup/Sync** | Git + manual | `br sync`, `br backup` |
**Verdict:** Beads adds **structured task management** that the bus+doc lacks — critical for multi-PASS dependency chains (PASS-P → PASS-A → PASS-S → PASS-B/X).
---
## PASS → Bead Mapping
| PASS | Bead Type | Suggested ID | Parent |
|------|-----------|--------------|--------|
| **PASS-P** (Pulse Landing) | Epic | `UV_PASS-P` | — |
| ├─ Copy `prod/uv_pulse_host/` | Task | `UV_PASS-P.1` | `UV_PASS-P` |
| ├─ Add `.gitignore` (exclude `target/`) | Task | `UV_PASS-P.2` | `UV_PASS-P` |
| ├─ Cert conveyor commit | Task | `UV_PASS-P.3` | `UV_PASS-P` |
| ├─ Soak DARK + TUI heartbeat | Task | `UV_PASS-P.4` | `UV_PASS-P` |
| **PASS-A** (Account Region) | Epic | `UV_PASS-A` | — |
| ├─ Phase 0: Contracts + in-mem | Task | `UV_PASS-A.1` | `UV_PASS-A` |
| ├─ Phase 1: Real shm + hardened reader | Task | `UV_PASS-A.2` | `UV_PASS-A` |
| ├─ Phase 2: ASEx publish | Task | `UV_PASS-A.3` | `UV_PASS-A` |
| ├─ Phase 3: Capital provider | Task | `UV_PASS-A.4` | `UV_PASS-A` |
| **PASS-S** (Sizing Seam) | Epic | `UV_PASS-S` | — |
| **PASS-B** (Host Brain) | Epic | `UV_PASS-B` | — |
| **PASS-X** (Tick Exits) | Epic | `UV_PASS-X` | — |
**Dependency Chain:**
```
UV_PASS-P → UV_PASS-A → UV_PASS-S → UV_PASS-B → UV_PASS-X
```
---
## Smallest Viable Setup
```bash
# 1. Create dedicated UV workspace (isolate from PRODGREEN)
mkdir -p /mnt/dolphinng5_predict/uv/.beads
export BEADS_DIR=/mnt/dolphinng5_predict/uv/.beads
# 2. Initialize
br where # confirms workspace
# 3. Create PASS-P epic + children
br create --title "PASS-P: Pulse Landing" --type epic --id UV_PASS-P
br create --title "Copy prod/uv_pulse_host/ from /mnt/vp-PASS9" --parent UV_PASS-P --type task --acceptance "Directory copied, target/ excluded" --test "ls prod/uv_pulse_host/ && ! ls prod/uv_pulse_host/target/" --id UV_PASS-P.1
br create --title "Add .gitignore excluding target/" --parent UV_PASS-P --type task --acceptance "target/ ignored by git" --test "git check-ignore prod/uv_pulse_host/target/" --id UV_PASS-P.2
br create --title "Cert conveyor commit + integrator review" --parent UV_PASS-P --type task --acceptance "Commit on main, integrator signed" --test "git log --oneline -1 prod/uv_pulse_host/" --id UV_PASS-P.3
br create --title "Soak DARK + TUI heartbeat + STALE" --parent UV_PASS-P --type task --acceptance "TUI renders live rate/AGE/STALE, RSS flat ≥4h" --test "TUI smoke test + log review" --id UV_PASS-P.4
# 4. Link to bus for dispatch notifications
# (beads = source of truth; bus = real-time signal)
```
---
## Recommendation
**ADOPT BEADS for PASS tracking** with the following protocol:
1. **Beads = Source of Truth** — all PASS state, dependencies, acceptance criteria, audit trail
2. **h5i Bus = Real-time Signal** — dispatch, ACK, status pings, escalation (what we already do)
3. **Status Doc = Snapshot** — auto-generated from beads weekly or on demand (`br status > PASS_BOARD.md`)
**Migration Path:**
- Week 1: Create UV workspace, populate PASS-P + PASS-A epics/children
- Week 1: Run dual-track (beads + bus) — validate no drift
- Week 2: Deprecate manual status doc; auto-generate from `br status`
**Why not bus+doc alone?** The PASS chain has 5 epics with 15+ children, strict dependencies, and must survive agent rotation. Beads enforces what the charter demands: "keep a live board; chase stalls; escalate blocked specs."
---
## Next Action
If approved: I'll initialize `/mnt/dolphinng5_predict/uv/.beads`, populate PASS-P epic + children, and link dispatch messages to bead IDs.

View File

@@ -0,0 +1,86 @@
# h5i — Cross-Agent Messaging (AUTHORITATIVE — read this to connect)
**This is THE single source of truth for agent comms. If anything elsewhere conflicts, this wins.**
Last updated 2026-06-22 by claude (integrator) to fix a bus split that left agents unable to see each
other's messages.
---
## 0. TL;DR — how to be reachable (do this first)
```bash
cd /mnt/dolphinng5_predict # ← THE CANONICAL BUS lives here. Always run h5i from here.
export H5I_AGENT=<your-handle> # e.g. codex / cmd-PASS9 / cmd-pass6 / cmd-pass5 / claude
h5i msg inbox # read + mark your unread
```
**Rule: always operate h5i with cwd = `/mnt/dolphinng5_predict`.** That checkout's `refs/h5i/msg` is
the ONE canonical bus everyone shares. Do not send/read from a fork clone — that's what split us.
---
## 1. Why we split (so it doesn't happen again)
`refs/h5i/msg` is **per-clone**. We had two diverged copies:
- **CANONICAL:** `/mnt/dolphinng5_predict` refs/h5i/msg (also mirrored read-only at `/mnt/vp-oa4`).
- **FORK:** `/mnt/vp-PASS9` refs/h5i/msg — where PASS9 had been reading/writing.
They never reconciled because **`h5i share push` targets the Gitea origin
(`…/siloqy.git`), which HANGS** (broken server-side hook). So claude's messages (on canonical) never
reached PASS9 (on the fork), and vice-versa. **Do not rely on share-to-Gitea.**
## 2. The fix — converge on the canonical bus
**Everyone reads/writes the canonical bus by running h5i from `/mnt/dolphinng5_predict`.** That path
is the shared checkout all agents can reach; one ref, no push/pull, no divergence.
If you are forced to work inside a fork clone (e.g. `/mnt/vp-PASS9`, whose git `origin` is the LOCAL
path `/mnt/dolphinng5_predict`), sync the bus **locally** (this does NOT touch Gitea, so it works):
```bash
cd /mnt/vp-PASS9
h5i share pull # pulls h5i refs from your local origin (/mnt/dolphinng5_predict)
# … work, send …
h5i share push # pushes back to the LOCAL origin (fine). NEVER push to the Gitea remote.
```
But the simpler, recommended path is just: **don't** — run h5i from `/mnt/dolphinng5_predict` directly.
## 3. Identities (current)
| Handle | Role | Current PASS2 task |
|---|---|---|
| **claude** | Integrator (owns merges to `main`, bit-verification) | — |
| **codex** | Parity / V3.4b live-factor / HZ | **PASS2.5** conviction soak |
| **cmd-PASS9** | ASEx kernel / race-safety | **PASS2.2** arming-gate, **PASS2.6** HZ client |
| **cmd-pass6** | Execution internals (OrderFSM/FillPump/Reconciler) | **PASS2.3** VST adapter, **PASS2.4** wiring |
| **cmd-pass5** | Mock-BingX adapter | **PASS2.1** VST creds + isolation |
| **aider_nvnemo** | Aider assistant (this session) | **PASS2.x** ad-hoc support |
Set yours: `export H5I_AGENT=<handle>` (or `h5i msg as <handle>`).
## 4. Usage
```bash
h5i msg # inbox dashboard (glance)
h5i msg inbox # show unread, mark read (numbers them)
h5i msg send <recipient> "…" # free-text ( 'all' = broadcast )
h5i msg ask <recipient> "…" # a request expecting a reply
h5i msg reply <n> "…" # threaded reply to message #n
h5i msg ack|done|decline <n> "…"
h5i msg wait --timeout 600 # block until a reply arrives (run as a background waiter)
git show refs/h5i/msg:messages.jsonl # raw view
```
Types: ASK · REVIEW_REQUEST · RISK · HANDOFF · BROADCAST · ACK · DONE.
## 5. Code workflow (so comms + code don't re-split) — see `VIOLET_PASS2_SERIES_INDEX.md`
- **Single trunk:** `/root/violet` `main` on local disk; shared bare origin **`/root/violet.git`**
(NOT Gitea). Your PASS2 worktree is `/root/violet-wt/pass2.<n>-<slug>` on branch
`agent/pass2.<n>-<slug>`. **Work there, push to `/root/violet.git`.** Do not branch from `/mnt`
CIFS clones.
- Sub-specs: `prod/docs/VIOLET_PASS2.<n>_*.md` (also replicated to `/mnt/dolphinng5_predict/prod/docs/`).
- Commit hygiene: explicit staging (never `git add .`), `Co-Authored-By:` trailer, tests in-commit.
## 6. De-confliction note
This file supersedes the comms sections of `VIOLET_CMD_ONBOARDING__PASS5.md` and any per-pass spec —
those defer here for *how to connect*. Operator points all agents at THIS file.

View File

@@ -0,0 +1,659 @@
# PINK Forensics — Dual Leverage Architecture (2026 Search Results)
**Date:** 2026-07-06
**Agent:** pi_nvnemo
**Trigger:** Operator request — locate the authoritative dual-leverage spec
---
## Executive Summary
The DOLPHIN system implements a **strict dual-leverage architecture** separating two distinct leverage concepts that must NEVER be conflated:
| Layer | Name | Range | Purpose | Set By |
|-------|------|-------|---------|--------|
| **Internal** | **Conviction Leverage** (our_leverage) | 0.5 9.0 (fractional) | Sizes QUANTITY: `notional = capital × 0.20 × conviction`, `qty = notional / entry_price` | Strategy / sizer (`esf_alpha_orchestrator`, `AlphaBetSizer`) |
| **Venue** | **Exchange Leverage** (xlev) | 1 3 (integer) | Controls MARGIN: `margin = notional / exchange_lev` sent to BingX API | Venue boundary mapper (`prod/bingx/leverage.py`) |
**PnL is ALWAYS leverage-free**: `qty × Δprice` (side-signed). Exchange leverage only affects collateral lockup.
---
## Authoritative Source Files (Bit-Identity Required)
### 1. `prod/bingx/leverage.py` — **THE SINGLE SOURCE OF TRUTH** (83 lines, no callers)
```python
CONVICTION_MIN = 0.5
CONVICTION_MAX = 9.0
EXCHANGE_LEV_MIN = 1
EXCHANGE_LEV_MAX = 3
LEVERAGE_MAPPING_RULE = "round_half_even_linear_0.5_to_9.0_to_1_to_exchange_cap"
def map_internal_conviction_to_exchange_leverage_target(internal, *, exchange_min, exchange_max) -> float:
# clamp internal to [0.5, 9.0]
# linear: exchange_min + (internal - 0.5)/(9.0 - 0.5) * (exchange_max - exchange_min)
# returns FLOAT target (pre-round)
def normalize_bingx_leverage_value(leverage, *, exchange_min, exchange_max) -> int:
# ROUND_HALF_EVEN (banker's: 1.5→2, 2.5→2, 3.5→4) + clamp to [exchange_min, exchange_max]
def map_internal_conviction_to_exchange_leverage(internal, *, exchange_min, exchange_max) -> int:
# = normalize_bingx_leverage_value(map_..._target(internal), ...)
# FINAL integer sent to BingX API
```
### 2. `prod/clean_arch/runtime/pink_direct.py:_hz_publish()` (line ~909)
```python
def _hz_publish(self, slot_dict: dict, acc: dict) -> None:
"""Fire-and-forget Hz write after any kernel state change.
Computes system leverage (our_leverage = notional/capital) for the Hz
snapshot — PINK/BLUE dual-leverage invariant: system leverage reflects real
margin utilisation; exchange leverage (1-3x cap) is set at BingX API level.
"""
size = float(slot_dict.get("size") or 0.0)
ep = float(slot_dict.get("entry_price") or 0.0)
capital = float(acc.get("capital") or 0.0)
our_leverage = (size * ep / capital) if capital > 1e-10 else 0.0
self.hz_state_writer.write_engine_snapshot(
slot_dict, acc,
posture=self._last_posture,
our_leverage=our_leverage, # <-- CONVICTION leverage published to Hz
scan_number=self._last_scan_number,
vel_div=self._last_vel_div,
vol_ok=self._last_vol_ok,
)
```
---
## Spec Documents (Chronological)
### A. `prod/docs/FRACTIONAL_LEVERAGE_TO_BINGX_FIX.md` (2025-04-24)
**Origin story** — CRITICAL bug: exchange leverage was hardcoded to 1x, ignoring per-trade fractional leverage.
- "The system correctly separates leverage into two roles"
- Fractional leverage → affects quantity (how many contracts)
- Exchange leverage → affects margin (how much collateral)
- Fix: CEIL rounding for exchange leverage (`ceil(fractional_lev)` clamped to [1,9])
### B. `prod/docs/PINK_ACCOUNTING_EXEC_FIX.md` (2026-06-11)
**Forensic incident** — FET short settled at +$164 but kernel booked $5,990.90.
**HARD INVARIANT (§0):**
> **Dual leverage**: `slot.size` = exchange quantity; `slot.leverage` = exchange leverage (13x cap, set at BingX API); *our*-leverage (conviction) = `size × entry_price / capital`, computed **only** at `pink_direct._hz_publish` (line ~911). PnL is therefore **leverage-free**: `qty × Δprice`, side-signed. Do not touch the conviction→exchange mapping (`round_half_even_linear_0.5_to_9.0_to_1_to_exchange_cap`) or `target_size` computation.
### C. `prod/docs/VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md` (2026-06-15)
**VIOLET L3 wrapper spec** — "WRAP, DON'T REIMPLEMENT"
- V-TYPES boundary: `ConvictionLeverage` (Annotated float) → `ExchangeLeverage` (Annotated int ≥1)
- `VioletExchangeLeverage` class wraps `prod/bingx/leverage.py` functions exactly
- Gate: MC bit-identity @ N≥1e6 vs real `leverage.py` output
- Zero shared-file edits; bit-identity is the contract
### D. `prod/docs/VIOLET_V3_FINDINGS.md` §2 (2026-06-15)
> **DUAL-LEVERAGE:** conviction leverage sizes the QUANTITY (internal); exchange leverage mapped at venue boundary via `prod/bingx/leverage.py` `map_internal_conviction_to_exchange_leverage_target` (round_half_even linear 0.59.0 → 1..cap; PINK/VIOLET use max-3× **linear** translator).
### E. `prod/docs/PRODGREEN_TUI_AND_LEVERAGE_OBSERVABILITY_SPEC.md` (2026)
**TUI display labels:**
- `cm:` for conviction multiplier
- `xlev:` for exchange leverage
- `lev:` legacy (visually secondary)
---
## Key Terms / Vocabulary
| Term | Meaning | Where Defined |
|------|---------|---------------|
| `conviction leverage` / `our_leverage` | Internal fractional [0.5, 9.0], sizes quantity | `pink_direct.py:_hz_publish` |
| `exchange leverage` / `xlev` | Integer [1,3] sent to BingX API | `leverage.py`, `pink_direct.py` |
| `dual-leverage doctrine` | The separation principle | `PINK_ACCOUNTING_EXEC_FIX.md` §0 |
| `round_half_even` | Banker's rounding (x.5 → even) | `leverage.py`, `VIOLET_SUB_SPEC__L3` |
| `map_internal_conviction_to_exchange_leverage` | The mapper function | `leverage.py` |
| `target_exchange_leverage` | Float pre-round value | `VIOLET_SUB_SPEC__L3` |
| `exchange_leverage` | Final int sent to venue | `VIOLET_SUB_SPEC__L3` |
| `notional` | `capital × 0.20 × conviction` | `esf_alpha_orchestrator.py` |
| `base_fraction` | 0.20 (constant in BLUE) | `VIOLET_V3_FINDINGS.md` §2 |
---
## Execution Flow (PINK → BingX)
```
1. BLUE/VIOLET sizer computes conviction ∈ [0.5, 9.0]
2. notional = capital × 0.20 × conviction
3. quantity = notional / entry_price
4. At venue boundary (pink_direct / execution.py):
target = map_internal_conviction_to_exchange_leverage_target(conviction) # float
xlev = normalize_bingx_leverage_value(target) # int [1,3]
5. BingX API: POST /leverage {"symbol": "...", "side": "BOTH", "leverage": xlev}
6. Margin locked = notional / xlev
7. PnL calculation: qty × (exit_price - entry_price) [NO leverage factor]
8. Hz snapshot publishes: our_leverage = (size × entry_price) / capital
```
---
## VIOLET Integration Points
| Component | Role | File |
|-----------|------|------|
| `VioletExchangeLeverage` | V-TYPES wrapper, bit-identity gated | `prod/clean_arch/violet/exchange_leverage.py` |
| `TradeabilityProjection` | L1→L3 projector (conviction → xlev + margin) | `prod/clean_arch/violet/tradeability.py` (Task 6) |
| `ShadowDecision` | L1 output carrying `conviction_leverage` | `decision_engine.py` |
---
## Mutation Litmus (What Breaks If Conflated)
| Mutation | Expected Test Failure |
|----------|----------------------|
| Use `exchange_leverage` in PnL calc | `test_pink_ditav2_accounting_invariants.py` — realized PnL 3× inflated |
| Use `conviction` as BingX leverage | Margin rejection or over-leverage (BingX max 3× for PINK) |
| Round-half-up instead of half-even | `VIOLET_SUB_SPEC__L3` gate: 2.5→3 instead of 2, bit-identity fails |
| Drop the clamp to [1,3] | BingX API rejects leverage >3 for PINK symbols |
---
## Related Files to Audit (Per Search)
- `prod/clean_arch/runtime/pink_direct.py``_hz_publish`, `_exec_submit`, intent leverage flow
- `prod/bingx/execution.py``_ensure_leverage`, `_normalize_bingx_leverage_value` (legacy CEIL, not ROUND_HALF_EVEN)
- `prod/clean_arch/violet/exchange_leverage.py` — VIOLET L3 wrapper
- `prod/clean_arch/violet/tradeability.py` — L3 projector (if built)
- `esf_alpha_orchestrator.py` — 5-factor conviction composition (base × DC × ACB × OB × EsoF)
- `alpha_wrappers.py` — VIOLET V-TYPES for `ConvictionLeverage`
- `prod/tests/test_pink_ditav2_accounting_invariants.py` — Accounting tests
- `prod/tests/test_violet_exchange_leverage.py` — VIOLET L3 gate tests
---
## Operator Directives (Binding)
1. **NEVER reimplement `leverage.py` logic** — wrap it (VIOLET L3 spec, non-negotiable)
2. **PnL is leverage-free**`qty × Δprice` only (PINK_ACCOUNTING_EXEC_FIX.md HARD INVARIANT)
3. **Bit-identity gate** — VIOLET output must `==` `prod/bingx/leverage.py` output exactly (MC N≥1e6)
4. **ROUND_HALF_EVEN** — not round-half-up, not CEIL, not floor (banker's rounding)
5. **Conviction sizes qty; exchange lev sizes margin** — the two paths are orthogonal after notional
---
## Search Provenance
Found via: `grep -r "dual.leverage\|our.*leverage.*exchange\|conviction.*multiplier\|map_internal_conviction_to_exchange" /mnt/dolphinng5_predict/prod/docs --include="*.md"`
Key hits: `PINK_ACCOUNTING_EXEC_FIX.md`, `VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md`, `FRACTIONAL_LEVERAGE_TO_BINGX_FIX.md`, `VIOLET_V3_FINDINGS.md`, `PRODGREEN_TUI_AND_LEVERAGE_OBSERVABILITY_SPEC.md`, `INDEX_REVIEW_alpha_engine.md`, `UV_TASK_T19_UV_CLOCK_HOST.md`
---
## Next Search Vectors (Operator Guidance)
- Search `esf_alpha_orchestrator.py` for 5-factor conviction composition
- Search `alpha_wrappers.py` for V-TYPES `ConvictionLeverage` definition
- Search `prod/bingx/execution.py` for legacy CEIL vs ROUND_HALF_EVEN divergence
- Trace `dolphin_actor.py` tag `lev:X.XX` → execution path
---
## Additional Findings (Extended Search)
### 1. `prod/bingx/sizing_mode.py` — Sizing Mode Contract
- Three modes: `engine` (default, no BingX payload), `testnet`, `live_market`
- `build_split_sizing_payload()` emits BingX-ready sizing with `exchange_leverage_cap`
- Delegates to `prod.utils.trade_sizing_bridge.build_engine_ready_sizing()`
### 2. `prod/utils/trade_sizing_bridge.py` — Engine-Ready Sizing Translation
**Core function:** `size_trade_from_sizing_lev()` — the complete translation pipeline:
```python
# Input: sizing_lev (conviction), capital, mark_price, etc.
# Output: TradeSizingResult with:
# - internal_leverage: cubic-convex conviction ∈ [0.5, 9.0]
# - exchange_leverage_target: float (pre-round, linear map)
# - exchange_leverage: int (ROUND_HALF_EVEN + clamp to [1, exchange_cap])
# - effective_notional: min(venue_cap, margin_budget × exchange_leverage)
# - quantity: floor(effective_notional / mark_price / step_size) × step_size
# - margin_to_capital, notional_to_capital ratios
```
**Key constants:**
- `DEFAULT_BINGX_EXCHANGE_LEVERAGE_CAP = 3`
- `DEFAULT_MIN_INTERNAL_LEVERAGE = 0.5`
- `DEFAULT_MAX_INTERNAL_LEVERAGE = 9.0`
- `DEFAULT_LEVERAGE_CONVEXITY = 3.0` (cubic!)
- `DEFAULT_MARGIN_BUDGET_FRACTION = 0.20`
**Convexity note:** The "cubic" in "max-3× cubic translator" refers to the **conviction sizing curve** (`strength_cubic = clamp(...)³`), NOT the exchange leverage mapping. The exchange mapping is **linear** with **ROUND_HALF_EVEN**.
### 3. `prod/clean_arch/adapters/bingx_direct.py` — DITAv2 Venue Adapter
- Uses `map_internal_conviction_to_exchange_leverage()` from `prod.bingx.leverage`
- Default `exchange_leverage_cap = 3`
- Applies leverage per-symbol via cache `_configured_leverage`
### 4. `prod/clean_arch/dita_v2/blue_parity.py` — BLUE Parity Wrapper
**DUAL-LEVERAGE INVARIANT (docstring):**
> "the fractional leverage produced here is STRATEGY conviction — it sizes the quantity. At-exchange leverage is derived from it at the venue boundary via map_internal_conviction_to_exchange_leverage() (linear [0.5, 9.0] → [1, cap], bankers rounding, security cap)."
### 5. `prod/clean_arch/dita_v2/test_blue_parity.py` — Parity Tests
**TestConvictionToExchangeLeverage class validates:**
```python
m(0.5) == 1 # conviction floor → exchange floor
m(9.0) == 3 # conviction ceiling → exchange cap (3)
m(4.75) == 2 # exact midpoint [0.5, 9.0] → target 2.0 → round_half_even(2.0) = 2
m(0.1) == 1 # clamped below conviction floor
m(50.0) == 3 # clamped above conviction ceiling
# monotonic: {1, 2, 3} across conviction range
```
### 6. `prod/docs/NAUTILUS_DOLPHIN_SPEC.md` — Sizing Formula
```
leverage = min_leverage + (max_leverage - min_leverage) × (signal_strength)^leverage_convexity
# leverage_convexity = 3.0 → CUBIC
strength_cubic = clamp((threshold - vel_div) / (threshold - extreme), 0, 1) ** 3
```
### 7. `prod/docs/SYSTEM_BIBLE_v7.md` §38.5 (margin-sizing addendum)
> "internal sizing leverage and BingX exchange leverage are separate layers. Exchange leverage controls the required margin; strategy leverage controls sizing intent."
---
## Complete Leverage Flow (End-to-End)
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ BLUE STRATEGY (esf_alpha_orchestrator) │
│ signal_strength = clamp((|vel_div| - threshold) / (extreme - threshold)) │
│ strength_cubic = signal_strength ** 3.0 ← CUBIC CONVEXITY │
│ raw_leverage = base × DC_boost × ACB_regime × OB_consensus × EsoF_haircut │
│ clamped to [0.5, 9.0] │
└──────────────────────────────────┬──────────────────────────────────────────┘
│ conviction ∈ [0.5, 9.0]
┌─────────────────────────────────────────────────────────────────────────────┐
│ PINK / VIOLET VENUE BOUNDARY │
│ target = map_internal_conviction_to_exchange_leverage_target(conviction) │
│ = 1.0 + (conviction - 0.5) / 8.5 × (3.0 - 1.0) ← LINEAR │
│ ∈ [1.0, 3.0] (float) │
│ xlev = normalize_bingx_leverage_value(target) │
│ = ROUND_HALF_EVEN(target) clamped to [1, 3] ← BANKER'S ROUNDING │
│ ∈ {1, 2, 3} (int) │
└──────────────────────────────────┬──────────────────────────────────────────┘
│ exchange_leverage ∈ {1, 2, 3}
┌─────────────────────────────────────────────────────────────────────────────┐
│ BINGX EXECUTION │
│ POST /trade/leverage {"symbol": "...", "side": "BOTH", "leverage": xlev} │
│ margin = notional / xlev │
└──────────────────────────────────┬──────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────┐
│ ACCOUNTING (PnL) │
│ qty = notional / entry_price │
│ PnL = qty × (exit_price - entry_price) ← LEVERAGE-FREE │
│ our_leverage = (size × entry_price) / capital ← PUBLISHED TO Hz │
└─────────────────────────────────────────────────────────────────────────────┘
```
---
## Critical Distinction: CEIL vs ROUND_HALF_EVEN
| Context | Rounding | Source |
|---------|----------|--------|
| **Old execution.py fix (2025-04-24)** | `ceil(fractional_lev)` | `FRACTIONAL_LEVERAGE_TO_BINGX_FIX.md` |
| **Current production `leverage.py`** | `ROUND_HALF_EVEN` (banker's) | `prod/bingx/leverage.py` |
| **VIOLET L3 wrapper** | `ROUND_HALF_EVEN` (bit-identical gate) | `VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md` |
**The CEIL fix was superseded** by the cleaner `leverage.py` module with banker's rounding. The production code now uses `prod/bingx/leverage.py` exclusively.
---
## ROUND_HALF_EVEN Boundary Cases (Tested)
| Conviction | Target (float) | ROUND_HALF_EVEN | Final xlev |
|------------|----------------|-----------------|------------|
| 0.5 | 1.0 | 1 | 1 |
| ~2.82 | 1.5 | 2 | 2 |
| 4.75 | 2.0 | 2 | 2 |
| ~6.68 | 2.5 | 2 | 2 ← BANKER'S: 2.5 → 2 |
| 9.0 | 3.0 | 3 | 3 |
The "max-3× cubic translator" phrase in VIOLET docs refers to:
- **Cubic** = conviction sizing curve (strength³)
- **3×** = exchange leverage cap (13)
- **Translator** = the linear + ROUND_HALF_EVEN mapper
---
## Source Code Inventory (All Leverage-Related)
| File | Role |
|------|------|
| `prod/bingx/leverage.py` | **SOURCE OF TRUTH** — pure functions, 83 lines, no callers |
| `prod/bingx/config.py` | `exchange_leverage_cap: PositiveInt = 3` default |
| `prod/bingx/execution.py` | Venue client, wraps leverage.py functions |
| `prod/bingx/sizing_mode.py` | Sizing mode contract (engine/testnet/live) |
| `prod/utils/trade_sizing_bridge.py` | Full sizing pipeline with margin math |
| `prod/clean_arch/adapters/bingx_direct.py` | DITAv2 venue adapter |
| `prod/clean_arch/dita_v2/blue_parity.py` | BLUE parity wrapper (docstrings the invariant) |
| `prod/clean_arch/dita_v2/test_blue_parity.py` | Parity tests including dual-leverage |
| `prod/clean_arch/runtime/pink_direct.py` | PINK runtime, `_hz_publish` computes `our_leverage` |
| `prod/clean_arch/violet/exchange_leverage.py` | VIOLET L3 typed wrapper (bit-identity gated) |
| `prod/clean_arch/violet/test_violet_exchange_leverage.py` | VIOLET L3 gate tests (N≥1e6 MC) |
| `prod/clean_arch/violet/exec_intent.py` | VIOLET PASS4 DARK intent projection |
---
## Mutation Litmus (What Breaks If Wrong)
| Mutation | Test That Catches It |
|----------|---------------------|
| Use `exchange_leverage` in PnL formula | `test_pink_ditav2_accounting_invariants.py` |
| Use `conviction` directly as BingX leverage | Margin rejection (BingX max 3× for PINK) |
| ROUND_HALF_UP instead of ROUND_HALF_EVEN | `test_violet_exchange_leverage.py::test_round_half_even_boundary_cases` (2.5→3 fails) |
| Drop clamp to [1,3] | BingX API rejects leverage >3 |
| Conflate the two leverage concepts | `PINK_ACCOUNTING_EXEC_FIX.md` HARD INVARIANT violation |
---
## Search Provenance (Complete)
```
# Primary searches
grep -r "dual.leverage\|map_internal_conviction_to_exchange" prod/docs --include="*.md"
grep -r "CONVICTION_MIN\|EXCHANGE_LEV_MAX\|LEVERAGE_MAPPING_RULE" prod --include="*.py"
grep -r "round_half_even\|ROUND_HALF_EVEN" prod --include="*.py"
grep -r "exchange_leverage_cap" prod --include="*.py"
# Key files examined
prod/bingx/leverage.py ← SOURCE OF TRUTH
prod/bingx/config.py ← Default cap = 3
prod/bingx/execution.py ← Venue client
prod/bingx/sizing_mode.py ← Mode contract
prod/utils/trade_sizing_bridge.py ← Full pipeline
prod/clean_arch/adapters/bingx_direct.py ← DITAv2 adapter
prod/clean_arch/dita_v2/blue_parity.py ← BLUE parity + invariant docstring
prod/clean_arch/dita_v2/test_blue_parity.py ← Parity tests
prod/clean_arch/runtime/pink_direct.py ← PINK runtime, _hz_publish
prod/clean_arch/violet/exchange_leverage.py ← VIOLET L3 wrapper
prod/clean_arch/violet/test_violet_exchange_leverage.py ← VIOLET gate tests
prod/clean_arch/violet/exec_intent.py ← VIOLET PASS4 intent
# Spec docs
prod/docs/FRACTIONAL_LEVERAGE_TO_BINGX_FIX.md ← Origin story (CEIL fix)
prod/docs/PINK_ACCOUNTING_EXEC_FIX.md ← Forensic HARD INVARIANT
prod/docs/VIOLET_SUB_SPEC__L3_EXCHANGE_LEVERAGE.md ← VIOLET L3 spec
prod/docs/VIOLET_V3_FINDINGS.md ← V3 findings
prod/docs/BINGX_MARGIN_SIZING_RULE.md ← Operational rule
prod/docs/SYSTEM_BIBLE_v7.md ← §38.5 margin-sizing addendum
prod/docs/NAUTILUS_DOLPHIN_SPEC.md ← Cubic sizing formula
```
---
## CRITICAL CORRECTION: Actual PINK Runtime Was DITA v1 (NOT DITAv2)
**The running PINK system that traded on BingX VST used `prod/clean_arch/dita/` (DITA v1), NOT `prod/clean_arch/dita_v2/`.**
DITAv2 (`prod/clean_arch/dita_v2/`) was a later rewrite that preserved the same dual-leverage invariant but was NOT the system that ran live.
### Actual Running PINK Stack (DITA v1)
| Layer | File | Role |
|-------|------|------|
| **Launcher** | `prod/launch_dolphin_pink.py` (baseline in `prod/refactor_snapshots_20260527_222130/`) | Wired DITA v1 + BingX direct adapter |
| **Decision** | `prod/clean_arch/dita/decision.py` | `DecisionEngine` — computes `leverage` (conviction) + `our_leverage` (notional/capital) |
| **Intent** | `prod/clean_arch/dita/intent.py` | `IntentEngine` — passes `leverage` from decision to `Intent` |
| **Trade FSM** | `prod/clean_arch/dita/trade.py` | `TradeExecutor``TradePosition.leverage` = conviction from intent |
| **Account** | `prod/clean_arch/dita/account.py` | `AccountProjection``snapshot.leverage` = `open_notional / capital` (**our_leverage**) |
| **Venue Adapter** | `prod/clean_arch/adapters/bingx_direct.py` | `submit_intent()`**dual-leverage translation happens HERE** |
| **TP Curve** | `prod/clean_arch/tp_curve.py` | `compute_our_leverage(notional, capital)` — used for TP tightening |
### Dual-Leverage Translation in Production Code
**`prod/clean_arch/adapters/bingx_direct.py:submit_intent()` (lines 599-606):**
```python
# intent.leverage is the STRATEGY conviction (fractional, 0.59.0) and
# already sized the quantity. At-exchange leverage is derived from it
# via the linear conviction map → integer [1, cap], bankers rounding.
leverage = map_internal_conviction_to_exchange_leverage(
float(intent.leverage or self._config.default_leverage),
exchange_max=self._config.exchange_leverage_cap, # = 3
)
await self._ensure_leverage(symbol, leverage) # POST to BingX /trade/leverage
```
**`prod/clean_arch/tp_curve.py`:**
```python
def compute_our_leverage(*, notional, capital) -> float:
"""Return the current system leverage implied by sizing, NOT exchange leverage."""
return abs(notional) / capital # our_leverage = notional/capital
```
**`prod/clean_arch/dita/decision.py`:**
```python
our_leverage = compute_our_leverage(notional=target_exposure, capital=context.capital)
# ... passed in Decision.metadata["our_leverage"] for TP curve
tp_effective_pct = compute_soft_tp_pct(tp_base_pct, our_leverage)
```
### Three Leverage Concepts in the Live System
| Name | Variable | Range | Computed Where | Purpose |
|------|----------|-------|----------------|---------|
| **Conviction** | `intent.leverage`, `Decision.leverage` | 0.59.0 | Sizer (cubic-convex) | Sizes QUANTITY |
| **Exchange** | `leverage` (BingX API) | 13 (int) | `map_internal_conviction_to_exchange_leverage()` | Controls MARGIN = notional/exchange_lev |
| **Our/System** | `our_leverage` | 0.0~1.8 | `compute_our_leverage(notional, capital)` | TP curve tightening, Hz publishing |
### DITAv2 Migration Note
`prod/clean_arch/dita_v2/` was a **later rewrite** that re-implemented the same architecture with a Rust kernel (`ExecutionKernel`). It preserved the dual-leverage invariant (documented in `PINK_ACCOUNTING_EXEC_FIX.md` §0 and `blue_parity.py` docstring) but the live PINK system that actually traded used **DITA v1**.
### Files That Were Actually Running Live
- `prod/launch_dolphin_pink.py` (the launcher)
- `prod/clean_arch/runtime/pink_direct.py` (the runtime — uses DITA v1 components)
- `prod/clean_arch/dita/` (decision, intent, trade, account)
- `prod/clean_arch/adapters/bingx_direct.py` (venue adapter with dual-leverage translation)
- `prod/clean_arch/tp_curve.py` (leverage-conditioned TP)
---
## VIOLET Contracts — Dual-Leverage in Data Types
### `prod/clean_arch/violet/alpha_wrappers.py` — `SizeDecision` (PASS3a)
```python
class SizeDecision(StrictModel):
"""Bet-sizer output. notional_fraction = fraction * conviction_leverage
is the realized notional/capital (== the recorded our_leverage); it is
the conviction side of the dual-leverage and is exchange-agnostic."""
fraction: Fraction
conviction_leverage: ConvictionLeverage # ∈ [0.5, 9.0] — internal sizing
notional_fraction: float = Field(ge=0.0) # == our_leverage = fraction × conviction_leverage
bucket_idx: int
strength_score: float
signal_bucket: str
```
**Key invariant:** `notional_fraction = fraction × conviction_leverage` — this IS the recorded `our_leverage` (system leverage = notional/capital).
### `prod/clean_arch/violet/decision_engine.py` — `ShadowDecision` (PASS3c)
```python
class ShadowDecision(StrictModel):
"""One muted decision — what BLUE *would* do this scan. Never executed."""
ts_ns: int
scan_number: int
asset: Symbol
side: str
vel_div: float
fraction: float # base_fraction (0.20)
conviction_leverage: float # ∈ [0.5, 9.0] — full BLUE conviction (5-factor)
notional_fraction: float # == our_leverage = fraction × conviction_leverage
target_exposure: float # = capital × notional_fraction
ars_score: float
bucket_idx: int
actuated: bool
# 5-factor breakdown (V3.4):
base_leverage: Optional[float] # base cubic from AlphaBetSizer
dc_lev_mult: Optional[float] # DC confirmation boost
regime_size_mult: Optional[float] # ACB boost × meta × MC_scale (the "steepener")
market_ob_mult: Optional[float] # OB consensus 0.851.20
esof_size_mult: Optional[float] # EsoF haircut [0, 1]
```
**Key points:**
- `conviction_leverage` = full 5-factor BLUE conviction (base × DC × ACB-regime × OB × EsoF)
- `notional_fraction` = `fraction × conviction_leverage` = `our_leverage` (system leverage)
- `target_exposure` = `capital × notional_fraction` = notional
- Exchange leverage is **L3 only** — never in L1 decision
### `prod/clean_arch/violet/contracts_v3.py` — `ExecIntent` (PASS4)
```python
class ExecIntent(StrictModel):
"""DARK would-be order intent. Data only; never sent to a venue here."""
asset: Symbol
side: Literal["SHORT", "LONG"]
qty: Qty
exchange_leverage: Annotated[int, Field(ge=1)] # ← L3: exchange leverage
maker_policy: str
target_notional: float
ts_ns: MonoNs
reason: Literal["ENTRY", "EXIT"]
```
### `prod/clean_arch/violet/exec_intent.py` — L1→L3 Projection (PASS4 Task 17)
```python
def to_exec_intent(
decision: ShadowDecision,
*,
capital: float,
reference_price: float,
maker_policy: str = "maker_both",
) -> ExecIntent:
# target_notional = capital × notional_fraction (our_leverage side)
target_notional = capital * decision.notional_fraction
qty = target_notional / reference_price
# L3: conviction → exchange leverage via prod/bingx/leverage.py
exchange = _exchange_leverage_for(decision.conviction_leverage)
return ExecIntent(
asset=decision.asset,
side=decision.side,
qty=qty,
exchange_leverage=exchange,
maker_policy=maker_policy,
target_notional=target_notional,
ts_ns=decision.ts_ns,
reason="ENTRY",
)
def _exchange_leverage_for(conviction_leverage: float) -> int:
# Wraps VioletExchangeLeverage (bit-identical to prod/bingx/leverage.py)
return VioletExchangeLeverage().to_exchange(conviction_leverage).exchange_leverage
```
---
## Complete Dual-Leverage Architecture Across All Systems
```
┌─────────────────────────────────────────────────────────────────────────────────┐
│ BLUE (nautilus_event_trader.py) │
│ esf_alpha_orchestrator: 5-factor conviction (base × DC × ACB-regime × OB × EsoF)│
│ our_leverage = compute_our_leverage(notional, capital) # for TP curve │
│ target_notional = capital × 0.20 × conviction_leverage │
└─────────────────────────────────────────────────────────────────────────────────┘
┌─────────────────┼─────────────────┐
▼ ▼ ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ PINK │ │ PRODGREEN │ │ VIOLET │
│ (DITA v1 live) │ │ (BLUE mirror) │ │ (shadow/UV) │
└──────────────────┘ └──────────────────┘ └──────────────────┘
│ │ │
┌──────────┴──────────┐ │ ┌──────────┴──────────┐
▼ ▼ ▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ decision.py │ │ decision.py │ │ alpha_wrap │ │decision_eng │
│ DecisionEng │ │ DecisionEng │ │ SizeDecision│ │ ShadowDec │
│ leverage= │ │ leverage= │ │ conviction_ │ │conviction_ │
│ conviction │ │ conviction │ │ leverage │ │leverage │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────┐
│ VENUE BOUNDARY (dual-leverage translation) │
│ │
│ PINK: prod/clean_arch/adapters/bingx_direct.py:submit_intent() │
│ leverage = map_internal_conviction_to_exchange_leverage( │
│ intent.leverage, exchange_max=3) │
│ │
│ VIOLET: prod/clean_arch/violet/exec_intent.py:to_exec_intent() │
│ exchange = VioletExchangeLeverage().to_exchange( │
│ decision.conviction_leverage).exchange_leverage │
│ │
│ BLUE: prod/bingx/execution.py:_ensure_leverage() │
│ leverage = map_internal_conviction_to_exchange_leverage( │
│ sizing_lev, exchange_max=config.exchange_leverage_cap)│
│ │
│ ALL use: prod/bingx/leverage.py (SOURCE OF TRUTH) │
└─────────────────────────────────────────────────────────────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ BingX API │ │ BingX API │ │ BingX API │ │ BingX API │
│ /trade/ │ │ /trade/ │ │ /trade/ │ │ /trade/ │
│ leverage │ │ leverage │ │ leverage │ │ leverage │
│ (int 1-3) │ │ (int 1-3) │ │ (int 1-3) │ │ (int 1-3) │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────┐
│ ACCOUNTING (leverage-free) │
│ │
│ PnL = qty × (exit_price - entry_price) [side-signed] │
│ our_leverage = (size × entry_price) / capital [Hz publishing] │
│ margin = notional / exchange_leverage │
│ │
│ HARD INVARIANT (PINK_ACCOUNTING_EXEC_FIX.md §0): │
│ "slot.size = exchange quantity; slot.leverage = exchange leverage │
│ (1-3x cap, set at BingX API); our_leverage (conviction) = │
│ size × entry_price / capital, computed ONLY at _hz_publish. │
│ PnL is therefore LEVERAGE-FREE: qty × Δprice, side-signed." │
└─────────────────────────────────────────────────────────────────────┘
```
---
## Mutation Litmus — Complete
| Mutation | Where It Breaks | Catching Test |
|----------|----------------|---------------|
| Use `exchange_leverage` in PnL formula | `prod/clean_arch/dita/trade.py:apply_fill()` | `test_pink_ditav2_accounting_invariants.py` |
| Use `conviction_leverage` as BingX leverage | `prod/clean_arch/adapters/bingx_direct.py:submit_intent()` | BingX API rejects >3× for PINK |
| ROUND_HALF_UP instead of ROUND_HALF_EVEN | `prod/bingx/leverage.py:normalize_bingx_leverage_value()` | `test_violet_exchange_leverage.py::test_round_half_even_boundary_cases` (2.5→2) |
| Drop clamp to [1,3] | `prod/bingx/leverage.py:_clamp_exchange_bounds()` | BingX API rejects leverage >3 |
| Conflate `our_leverage` with `exchange_leverage` | Any accounting code | `PINK_ACCOUNTING_EXEC_FIX.md` HARD INVARIANT violation |
| Skip dual-leverage in VIOLET L3 | `prod/clean_arch/violet/exec_intent.py:_exchange_leverage_for()` | `test_violet_exchange_leverage.py::test_gate_exchange_leverage_bit_identity` (N≥1e6) |
---
## Search Complete — All Systems Mapped
| System | Decision/Sizing | Intent | Venue Translation | Accounting |
|--------|----------------|--------|-------------------|------------|
| **BLUE** | `esf_alpha_orchestrator` (5-factor) | `nautilus_event_trader.py` | `prod/bingx/execution.py` | `compute_our_leverage()` for TP |
| **PINK (live)** | `prod/clean_arch/dita/decision.py` | `prod/clean_arch/dita/intent.py` | `prod/clean_arch/adapters/bingx_direct.py` | `AccountProjection.leverage = our_leverage` |
| **PINK (DITAv2)** | `prod/clean_arch/dita_v2/blue_parity.py` | `prod/clean_arch/dita/intent.py` | `prod/clean_arch/adapters/bingx_direct.py` | `AccountProjection.leverage = our_leverage` |
| **PRODGREEN** | Same as BLUE | Same | `prod/bingx/execution.py` | Same |
| **VIOLET (shadow)** | `prod/clean_arch/violet/decision_engine.py` | `prod/clean_arch/violet/exec_intent.py` | `prod/clean_arch/violet/exchange_leverage.py` | `CapitalState.capital` anchor |
**All paths converge on `prod/bingx/leverage.py` — the single source of truth for conviction→exchange mapping.**

View File

@@ -0,0 +1,165 @@
# pi_wake_agent.py — Multi-Agent Wake Timer
**Location:** `/mnt/dolphinng5_predict/pi_wake_agent.py`
**Branch:** `tools/pi_wake_agent`
**Status:** v1.0 — 38 tests passing
---
## Overview
Reusable multi-agent wake-up timer with self-cron/daemon/succession modes. Designed for the DOLPHIN fleet (pi_nvnemo, cmd, mimo, codex, etc.) to send doorbell injections via zellij and durable messages via h5i bus.
---
## Installation
```bash
cd /mnt/dolphinng5_predict
python3 pi_wake_agent.py --install --interval 30m --session pi_test --msg "Wake up!"
```
---
## Modes
| Mode | Flag | Description |
|------|------|-------------|
| Install cron | `--install` | Recurring wake via system cron |
| One-shot | `--once` | Single wake after interval (no cron) |
| Daemon | `--daemon` | Long-lived process, no cron |
| Succession | `--succession` | Run N times at interval, then self-clean |
| Run (internal) | `--run` | Called by cron, executes wake |
| Remove | `--remove` | Remove cron entry |
| List | `--list` | Show active cron entries |
| Status | `--status` | Show cron + one-shot timers |
| Validate | `--validate` | Check zellij sessions exist |
---
## Options
| Option | Description |
|--------|-------------|
| `--interval DURATION` | Default: 1h. Formats: 30m, 1h, 90m, 2h, 10s |
| `--session SESSION` | Zellij session name (repeatable) |
| `--sessions "A,B,C"` | Comma-separated list |
| `--msg "MESSAGE"` | Wake message (default: "Operator says CONTINUE. Pi here!") |
| `--count N` | Number of runs for `--succession` |
| `--dry-run` | Show what would be done without executing |
| `--help` | Show help |
---
## Examples
```bash
# Recurring 1-hour wake for one session
pi_wake_agent.py --install --interval 1h --session cc_UV_dev0_Fb
# Multi-session 30-minute wake
pi_wake_agent.py --install --interval 30m --sessions "cc_UV_dev0_Fb,cc_UV_dev1_48" --msg "Wake up!"
# One-shot in 2 hours
pi_wake_agent.py --once --interval 2h --session cc_UV_dev0_Fb --msg "Time's up!"
# Run 3 times at 1-hour intervals, then self-clean
pi_wake_agent.py --succession --count 3 --interval 1h --session cc_UV_dev0_Fb --msg "Scheduled wake"
# Daemon mode (long-lived, no cron)
pi_wake_agent.py --daemon --interval 1h --session cc_UV_dev0_Fb
# Remove timer
pi_wake_agent.py --remove --session cc_UV_dev0_Fb --interval 1h
# List / status
pi_wake_agent.py --list
pi_wake_agent.py --status
# Validate sessions exist
pi_wake_agent.py --validate --session cc_UV_dev0_Fb
```
---
## Key Features
### Non-Blocking h5i Bus
Messages sent to Fable via `h5i msg send` are **fire-and-forget**:
- Runs in background thread
- 5-second timeout
- Silently ignores failures
- Never blocks the wake cycle
### Self-Cleaning Succession
```bash
pi_wake_agent.py --succession --count 3 --interval 1h --session S
```
Runs exactly 3 times at 1-hour intervals, then removes its own cron entry.
### Multi-Session
```bash
# Repeatable --session
pi_wake_agent.py --install --interval 1h --session s1 --session s2
# Comma-separated --sessions
pi_wake_agent.py --install --interval 1h --sessions "s1,s2,s3"
```
### Daemon Mode
Runs indefinitely as a long-lived process (no cron needed):
```bash
pi_wake_agent.py --daemon --interval 1h --session S
```
---
## h5i Bus Protocol Compliance
Every wake injection:
1. Identifies as `[pi_nvnemo via zellij]`
2. Includes `Run: h5i-bus msg inbox` directive
3. Sends 5× ENTER keypresses (1s delay) for reliable submission
4. Sends parallel h5i bus message for durability
Per [AGENT_TERMINAL_DIRECT_INTERVENTION_PROCEDURES.md](AGENT_TERMINAL_DIRECT_INTERVENTION_PROCEDURES.md).
---
## Testing
```bash
cd /mnt/dolphinng5_predict
python3 -m pytest test_pi_wake_agent.py -v
```
**38 tests passing** covering:
- Unit tests: interval parsing, cron comments, session parsing
- Integration: install/remove/list, one-shot, succession, run mode
- Edge cases: invalid intervals, missing sessions, invalid counts
---
## Cron Entry Format
```
*/30 * * * * cd /mnt/dolphinng5_predict && export H5I_AGENT=pi_nvnemo && /mnt/dolphinng5_predict/pi_wake_agent.py --run --sessions 'pi_test' --msg '...' # pi_wake_agent:pi_test:30m
```
Comment format: `pi_wake_agent:<sessions>:<interval>`
---
## Logs
- **File:** `/tmp/pi_wake_agent.log`
- **Rotation:** 10MB max, 5 files
- **Format:** `[YYYY-MM-DD HH:MM:SS] [LEVEL] message`
---
## Related Files
- `/mnt/dolphinng5_predict/pi_wake_agent.py` — Main script
- `/mnt/dolphinng5_predict/test_pi_wake_agent.py` — Test suite (38 tests)
- `/mnt/dolphinng5_predict/prod/docs/AGENT_TERMINAL_DIRECT_INTERVENTION_PROCEDURES.md` — Protocol
- Branch: `tools/pi_wake_agent`

View File

@@ -0,0 +1,48 @@
# RECENT VIOLET Touched Files
This inventory covers the recent VIOLET 3.4-series commits on this branch.
It includes all files touched across the series, grouped by commit, so the
surface is explicit rather than inferred.
## 722fd9f — VIOLET V3.4b/V3e: journal the full-sizing breakdown
- [prod/clean_arch/violet/shadow_journal.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/shadow_journal.py)
- [prod/clean_arch/violet/test_violet_shadow_journal.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/test_violet_shadow_journal.py)
- [prod/clickhouse/violet/22_violet_decisions.sql](/mnt/dolphinng5_predict/prod/clickhouse/violet/22_violet_decisions.sql)
## a632c59 — VIOLET V3.4b: validate live-factor field paths + HZ sourcing adapter
- [prod/clean_arch/violet/live_factor_source.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/live_factor_source.py)
- [prod/clean_arch/violet/test_violet_live_factor_source.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/test_violet_live_factor_source.py)
- [prod/docs/VIOLET_V34B_LIVE_FACTOR_FIELD_VALIDATION.md](/mnt/dolphinng5_predict/prod/docs/VIOLET_V34B_LIVE_FACTOR_FIELD_VALIDATION.md)
## 1ac3f62 — VIOLET V3.4c: read-only BLUE live source parity
- [prod/clean_arch/violet/live_blue_source.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/live_blue_source.py)
- [prod/clean_arch/violet/test_violet_live_blue_source.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/test_violet_live_blue_source.py)
## 16add44 — DOCS: add RECENT VIOLET 34C detail note
- [prod/docs/RECENT_VIOLET_34C_a632c59.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_34C_a632c59.md)
## fb34431 — VIOLET V3.4b: launcher shadow live-factor wiring
- [prod/clean_arch/violet/shadow_live_factors.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/shadow_live_factors.py)
- [prod/clean_arch/violet/test_violet_launcher_shadow_live_factors.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/test_violet_launcher_shadow_live_factors.py)
- [prod/docs/RECENT_VIOLET_34D_pending.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_34D_pending.md)
- [prod/launch_dolphin_violet.py](/mnt/dolphinng5_predict/prod/launch_dolphin_violet.py)
## 2d37ef0 — VIOLET V3.4c: trade-slot comparison harness
- [prod/clean_arch/violet/test_violet_trade_slot_compare.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/test_violet_trade_slot_compare.py)
- [prod/clean_arch/violet/trade_slot_compare.py](/mnt/dolphinng5_predict/prod/clean_arch/violet/trade_slot_compare.py)
- [prod/docs/RECENT_VIOLET_34E_pending.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_34E_pending.md)
## Rename / note follow-ups in the same series
- [prod/docs/RECENT_VIOLET_34D_fb34431.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_34D_fb34431.md)
- [prod/docs/RECENT_VIOLET_34E_2d37ef0.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_34E_2d37ef0.md)
- [prod/docs/RECENT_VIOLET_TOUCHED_FILES.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_TOUCHED_FILES.md)
- [prod/docs/RECENT_VIOLET_34C_a632c59.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_34C_a632c59.md)
- [prod/docs/RECENT_VIOLET_34D_pending.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_34D_pending.md)
- [prod/docs/RECENT_VIOLET_34E_pending.md](/mnt/dolphinng5_predict/prod/docs/RECENT_VIOLET_34E_pending.md)

View File

@@ -0,0 +1,629 @@
## Shared Memory Formats And Addressing
Date: 2026-07-04
Host: `DOLPHIN`
Scope: all shared-memory formats and SHM-adjacent IPC formats directly inspected from source during UV / BLUE-PRIME / DITAv2 work.
This document is intentionally concrete. It separates:
1. the **transport container** (`Zinc` region, or `iceoryx2` service),
2. the **payload framing inside that container**,
3. the **semantic payload schema** written by a given subsystem,
4. the **addressing rule** by which a writer and a reader find the same object.
It also records which format is actually in use for each known subsystem as inspected on this host.
### Short Answer
- A **Zinc region is addressed by its logical region name**, passed to `SharedRegion.create(name, ...)` / `SharedRegion.open(name)`.
- On Linux, Zinc materializes that as a POSIX SHM object named **`/zinc_<logical_name>`**, which appears in `/dev/shm` as **`zinc_<logical_name>`**.
- Therefore:
- reader/writer open **`uv_shadow_state`**
- the OS object visible under `/dev/shm` is **`zinc_uv_shadow_state`**
- opening `zinc_uv_shadow_state` through the Zinc API is wrong and fails
- Two algo instances avoid collision by using **different logical prefixes** and deriving all region names from that prefix.
For `iceoryx2`, the address is not a Zinc region name. It is the **service name** such as `uv/pulse`.
---
## 1. What Exists
### 1.1 Formats actually encountered
| Layer | Transport | Address form | Payload framing | Current role |
|---|---|---|---|---|
| Zinc region container | Zinc shared memory | logical name `name`; OS object `/dev/shm/zinc_<name>` | Zinc internal region header, then user data area | base transport for Zinc-backed regions |
| DITAv2 plane packet | Zinc region data area | `<prefix>_intent`, `<prefix>_state`, `<prefix>_control`, `<prefix>_venue` | `!QQ` = `(seq, json_size)` + UTF-8 JSON | DITAv2 real Zinc plane |
| DITAv2 control packet | Zinc region data area | `<prefix>_control` | same `!QQ + JSON` envelope | DITAv2 control plane |
| UV / BLUE-PRIME snapshot | Zinc region data area | `uv_shadow_state` | `UVZINC01` + dual-seq seqlock header + UTF-8 JSON | authoritative BLUE-PRIME shadow snapshot, live |
| UV hook frame | Zinc region data area | `uv_shadow_blue_prime_hooks` | same `UVZINC01` + dual-seq + UTF-8 JSON | hook observability frame, live/auxiliary |
| UV test scratch snapshot | Zinc region data area | `uv_t6_<id>_state` | same `UVZINC01` + dual-seq + UTF-8 JSON | test / temporary namespaces |
| UV pulse bridge payload | `iceoryx2` publish-subscribe service | service name `uv/pulse` | fixed 40-byte `PulseFrame` POD | optional derived feed for Rust TUI; not authoritative |
### 1.2 Not shared memory, but easy to confuse with it
The old file transport in `prod/clean_arch/violet/uv/shm.py`:
- explicit mode only
- writes JSON files under `UV_SHM_ROOT` / `/dev/shm/uv`
- **not** the runtime BLUE-PRIME contract
- retained for tests / fallback diagnostics only
That path is not documented further here because the request is specifically about shared memory.
---
## 2. Zinc Region Container Format
Authoritative sources:
- `zinc/core/src/header.rs`
- `zinc/core/src/region.rs`
- `zinc/core/src/platform/unix.rs`
- `zinc/adapters/python/zinc/_ffi.py`
- `zinc/adapters/python/zinc/__init__.py`
### 2.1 Addressing
Zinc validates and opens a **logical name** such as:
- `uv_shadow_state`
- `vst_dita_state`
- `dolphin_violet_control`
The Linux backend then maps that to a POSIX SHM object:
- logical name: `uv_shadow_state`
- OS shm object: `/zinc_uv_shadow_state`
- visible filesystem entry: `/dev/shm/zinc_uv_shadow_state`
This mapping is implemented in `zinc/core/src/platform/unix.rs`.
### 2.2 Name rules
Valid logical characters are:
- ASCII alphanumeric
- `_`
- `-`
Slashes are not allowed in raw Zinc names. Callers that start from path-like prefixes sanitize before opening.
### 2.3 Region layout
Every Zinc region is:
1. one page of Zinc-owned metadata/header
2. followed by the user data area
The Rust core exposes the user data area by returning `page_size()` bytes past the mapping base. The Python adapter likewise exposes only the user data area through `SharedRegion.as_buffer()`.
This point matters:
- the **underlying region really is Zinc**
- but a reader using `as_buffer()` does **not** see the Zinc header at byte 0
- it sees byte 0 of the **user payload**
That is why a live UV reader sees `UVZINC01` at the first visible bytes even though the mapped OS object is a Zinc region.
### 2.4 Zinc internal header
From `zinc/core/src/header.rs`:
- `magic: u64` = `"ZINC_REG"`
- `version: u16` = currently `2`
- `flags: u16`
- `notify_seq: AtomicU32`
- `capacity: u64`
- `ref_count: AtomicU32`
- `owner_pid: AtomicI32`
- `created_at: u64`
- `name_hash: u64`
- `ring_head: AtomicU64`
- `ring_tail: AtomicU64`
This header is aligned to one cache line and occupies the Zinc-managed metadata area, not the caller-visible payload buffer.
### 2.5 Notify/wait semantics
Zinc provides:
- `notify()`
- `wait(timeout_ms)`
These operate on `notify_seq` in the Zinc header. The payload framing on top of Zinc is owned by the higher-level subsystem.
---
## 3. DITAv2 Real Zinc Plane Format
Authoritative sources:
- `prod/clean_arch/dita_v2/real_zinc_plane.py`
- `prod/clean_arch/dita_v2/real_control_plane.py`
### 3.1 Addressing
`RealZincPlane(prefix=...)` derives names as:
- `base = prefix.strip("/").replace("/", "_")`
- `intent_name = f"{base}_intent"`
- `state_name = f"{base}_state"`
- `control_name = f"{base}_control"`
- `venue_name = f"{base}_venue"`
So if `prefix="vst/dita"`:
- logical names become:
- `vst_dita_intent`
- `vst_dita_state`
- `vst_dita_control`
- `vst_dita_venue`
- OS objects become:
- `/dev/shm/zinc_vst_dita_intent`
- `/dev/shm/zinc_vst_dita_state`
- `/dev/shm/zinc_vst_dita_control`
- `/dev/shm/zinc_vst_dita_venue`
### 3.2 Region capacities
Defaults in `RealZincPlane.__init__`:
- `intent_capacity = 1 << 20` = 1 MiB
- `state_capacity = 1 << 20` = 1 MiB
- `control_capacity = 1 << 20` = 1 MiB
- `venue_region` also uses `control_capacity`
### 3.3 Payload framing inside the Zinc data area
DITAv2 does **not** use the `UVZINC01` header.
It uses:
- `struct.pack("!QQ", seq, len(json_bytes))`
- followed by UTF-8 JSON bytes
That is:
- bytes `0..8`: `seq` as big-endian `u64`
- bytes `8..16`: JSON length as big-endian `u64`
- bytes `16..16+size`: JSON bytes
There is no dual-seq torn-read guard here. Readers trust:
- the header is present
- `size` is sane
- the JSON decodes
### 3.4 Semantic payloads
By region:
- `*_intent`: `{"items": [...]}` where items are serialized `KernelIntent`s
- `*_state`: `{"slots": [...]}` where slots are serialized `TradeSlot`s
- `*_control`: `{"control": {...}}` where value is a `KernelControlSnapshot`
- `*_venue`: `{"venue": {...}}` where value is a `VenueTelemetrySnapshot`
### 3.5 Write semantics
Writers:
- increment a region-local sequence
- build `!QQ + JSON`
- copy packet into the entire visible data buffer
- zero the tail
- call `region.notify()`
This is a simple packet format, not a seqlock format.
### 3.6 Current use
This is the intended real shared-memory format for DITAv2 when `RealZincPlane` / `RealZincControlPlane` are active.
At the time of this writing, I did **not** find live openable regions under the tested names:
- `vst_dita_intent`
- `vst_dita_state`
- `vst_dita_control`
- `vst_dita_venue`
- `dolphin_violet_intent`
- `dolphin_violet_state`
- `dolphin_violet_control`
- `dolphin_violet_venue`
So this format is source-authoritative, but not directly observed live under those tested prefixes at sample time.
---
## 4. UV / BLUE-PRIME Zinc Snapshot Format
Authoritative source:
- `prod/clean_arch/violet/uv/shm.py`
This is the important one for current UV / BLUE-PRIME observability.
### 4.1 Addressing
Default prefix:
- `UV_ZINC_PREFIX`, default `uv_shadow`
Logical region naming rule:
- `prefix.strip("/").replace("/", "_")`
- plus `_<slot>`
- except slot `"blue_prime"` and slot `"state"` are both normalized to `_<prefix>_state`
Examples:
- `publish("blue_prime", ...)` -> logical region `uv_shadow_state`
- `publish("state", ...)` -> logical region `uv_shadow_state`
- `publish("blue_prime_hooks", ...)` -> logical region `uv_shadow_blue_prime_hooks`
### 4.2 Region capacities
Default:
- `UV_ZINC_STATE_BYTES`, default `64 << 20` = 64 MiB
Observed live regions:
- `/dev/shm/zinc_uv_shadow_state`
- `/dev/shm/zinc_uv_shadow_blue_prime_hooks`
Observed test/scratch leftovers:
- `/dev/shm/zinc_uv_t6_8933e28c84_state`
- `/dev/shm/zinc_uv_t6_a1f7e3254b_state`
- `/dev/shm/zinc_uv_t6_b16924a96a_state`
- `/dev/shm/zinc_uv_t6_eed1be7947_state`
### 4.3 Payload framing inside the Zinc data area
Header:
- magic = `b"UVZINC01"`
- struct = `struct.Struct("!8sQQQ")`
Visible data-area layout:
1. bytes `0..8`: magic `"UVZINC01"`
2. bytes `8..16`: `seq_a` big-endian `u64`
3. bytes `16..24`: `seq_b` big-endian `u64`
4. bytes `24..32`: JSON payload size big-endian `u64`
5. bytes `32..32+size`: UTF-8 JSON payload
### 4.4 Write semantics
Writer uses a simple seqlock pattern:
1. compute next logical sequence `seq`
2. derive:
- `seq_even = seq * 2`
- `seq_odd = seq_even - 1`
3. write header with odd/in-flight sequence and `size = 0`
4. copy JSON body
5. optionally zero one byte after body
6. write header again with even/stable sequence and real `size`
7. call `region.notify()`
### 4.5 Read semantics
Reader accepts payload only if all are true:
- magic == `UVZINC01`
- `seq_a != 0`
- `seq_a` is even
- `seq_a == seq_b`
- `size` is in bounds
- after copying the body, rereading the header yields the exact same
- magic
- `seq_a`
- `seq_b`
- `size`
If any of that fails, the read is treated as torn / not yet initialized.
### 4.6 Semantic payloads
This format carries JSON snapshots rather than a fixed struct. Current known uses:
- `uv_shadow_state`
- authoritative BLUE-PRIME snapshot
- includes domains such as `meta`, `scan`, `live_inputs`, `engine`, `decision`, `efsm`, `dita`, `ram`, `source_trace`
- `uv_shadow_blue_prime_hooks`
- one published hook runner frame per scan
- keys include `scan`, `hooks`, `hook_count`, `ok_count`, `total_us`
### 4.7 Current use
This is the **live authoritative BLUE-PRIME shared-memory format** currently observed on host.
Verification made on host:
- `SharedRegion.open("uv_shadow_state")` succeeds
- `SharedRegion.open("zinc_uv_shadow_state")` fails
- first bytes of visible region buffer are `55565a494e433031...` = `UVZINC01`
That is the definitive proof that:
- the region container is Zinc
- the payload framing currently in use inside the container is `UVZINC01`
---
## 5. UV Hook Frame Region
Authoritative source:
- `prod/clean_arch/violet/uv/hooks/runner.py`
The hook runner publishes:
- `self.shm.publish("blue_prime_hooks", frame)`
Because `ShmChannel` defaults to Zinc-backed transport, that becomes:
- logical region: `uv_shadow_blue_prime_hooks`
- OS shm object: `/dev/shm/zinc_uv_shadow_blue_prime_hooks`
The payload framing is the same `UVZINC01` seqlock JSON envelope documented above.
The semantic payload differs:
- per-scan hook effects
- timing / ordering / success counts
This is auxiliary observability, not the main state snapshot.
---
## 6. iceoryx2 Pulse Bridge Format
Authoritative sources:
- `uv_tui/crates/pulse_frame/src/lib.rs`
- `uv_tui/bridge/src/lib.rs`
- `uv_tui/tui/src/lib.rs`
### 6.1 What it is
This is not the authoritative shared-memory snapshot. It is a **derived bridge feed**:
1. bridge opens Zinc region `uv_shadow_state`
2. bridge decodes the `UVZINC01` payload
3. bridge extracts a reduced pulse contract
4. bridge republishes that reduced contract over `iceoryx2` service `uv/pulse`
### 6.2 Addressing
Address is a service name, not a Zinc region name:
- service: `uv/pulse`
Current code hardcodes a singleton default service. Unlike Zinc prefixes, this is **not yet namespaced per parallel algo instance**.
### 6.3 Payload framing
Fixed `PulseFrame`, length 40 bytes:
- `observe_mono_ns: u64`
- `scan_number: u64`
- `region_seq: u64`
- `publish_latency_us: f64`
- `has_entry: u8`
- trailing reserved padding
Serialized little-endian by the shared `pulse_frame` crate.
### 6.4 Current use
- optional
- derived
- non-authoritative
- useful for the Rust TUI, especially when decoupling bridge and display
As of current Rust TUI work, the TUI can also read the Zinc region directly and no longer requires the bridge.
---
## 7. Addressing Rules, Precisely
### 7.1 Zinc regions
There are **three names** to keep distinct:
1. **logical region name**
- what code passes to Zinc
- example: `uv_shadow_state`
2. **POSIX SHM object name**
- what the Linux SHM API sees
- example: `/zinc_uv_shadow_state`
3. **filesystem entry under `/dev/shm`**
- example: `/dev/shm/zinc_uv_shadow_state`
The reader/writer contract uses **(1)**.
The operator inspecting `/dev/shm` sees **(3)**.
Confusing (1) and (3) is the common failure mode.
### 7.2 How a writer and reader agree on the same region
They must share:
- the same transport family
- Zinc or `iceoryx2`
- the same prefix / service namespace
- the same slot derivation rule
- the same payload framing
- the same semantic schema
For Zinc-backed UV:
- prefix source: `UV_ZINC_PREFIX`
- slot naming logic: `_slot_region(prefix, slot)`
- payload format: `UVZINC01`
- state slot: `blue_prime` -> `uv_shadow_state`
For DITAv2:
- prefix passed to `RealZincPlane(prefix=...)`
- suffixes: `_intent`, `_state`, `_control`, `_venue`
- payload format: `!QQ + JSON`
For `iceoryx2`:
- service name must match exactly
- payload struct must match exactly
### 7.3 How two running algo instances know “their” region
They do not discover “theirs” by magic. They must be started with a namespace choice.
Correct pattern:
- instance A:
- `UV_ZINC_PREFIX=uv_shadow_a`
- state region -> `uv_shadow_a_state`
- instance B:
- `UV_ZINC_PREFIX=uv_shadow_b`
- state region -> `uv_shadow_b_state`
Then:
- A writer writes only `uv_shadow_a_*`
- A TUI for A opens only `uv_shadow_a_*`
- B writer/TUI use `uv_shadow_b_*`
For test runs, this pattern is already used:
- `uv_t6_<id>` prefixes produce regions like `uv_t6_8933e28c84_state`
### 7.4 Collision rule
If two independent algo instances reuse the same Zinc logical region name:
- they are not isolated
- last writer wins
- readers observe a mixed stream
- observability becomes invalid
So region naming is not cosmetic. It is the namespace boundary.
---
## 8. What Is In Use For What, Now
### 8.1 BLUE-PRIME authoritative observability
- transport container: Zinc region
- logical region: `uv_shadow_state`
- payload framing inside region: `UVZINC01`
- semantic schema: BLUE-PRIME snapshot JSON
- status: live and observed
### 8.2 BLUE-PRIME hook observability
- transport container: Zinc region
- logical region: `uv_shadow_blue_prime_hooks`
- payload framing inside region: `UVZINC01`
- semantic schema: hook runner frame JSON
- status: live region observed
### 8.3 UV Rust TUI direct mode
- reads: Zinc region directly
- logical region by default: `uv_shadow_state`
- expects payload framing: `UVZINC01`
- status: implemented and verified
### 8.4 UV Rust TUI bridge mode
- bridge reads Zinc region `uv_shadow_state`
- bridge republishes `PulseFrame` over `iceoryx2` service `uv/pulse`
- TUI subscribes to `uv/pulse`
- status: implemented and verified, but secondary to direct Zinc reads
### 8.5 DITAv2 real shared-memory plane
- transport container: Zinc region
- logical regions: `<prefix>_{intent,state,control,venue}`
- payload framing: `!QQ + JSON`
- status: source-authoritative, but not directly observed live under tested prefixes during this inspection
---
## 9. Operator Notes
### 9.1 To inspect the live authoritative UV region
Use the logical name:
```python
from zinc import SharedRegion
r = SharedRegion.open("uv_shadow_state")
buf = r.as_buffer()
```
Do **not** do:
```python
SharedRegion.open("zinc_uv_shadow_state")
```
That is the `/dev/shm` object name, not the logical Zinc name.
### 9.2 To tell whether a region uses DITAv2 or UV framing
Look at the first visible bytes of `as_buffer()`:
- `UVZINC01` -> UV / BLUE-PRIME seqlock payload
- otherwise, if the first 16 bytes parse as `!QQ` and the JSON decodes, likely DITAv2 packet framing
You will **not** see `ZINC_REG` there through normal adapter reads, because that Zinc header lives before the exposed data area.
### 9.3 To run two independent observability stacks
Assign different prefixes up front. Example:
```bash
export UV_ZINC_PREFIX=uv_shadow_main
export UV_ZINC_PREFIX=uv_shadow_soak
```
Then point each consumer at its matching logical region names.
If `iceoryx2` bridge mode is used in parallel too, it needs the same kind of namespacing. Current bridge service default is singleton `uv/pulse`; that should be parameterized if simultaneous parallel bridges are required.
---
## 10. Bottom Line
The current live BLUE-PRIME observability stack is:
- **container:** Zinc shared memory
- **live authoritative region:** `uv_shadow_state`
- **live payload framing:** `UVZINC01` seqlock JSON
- **live hook side region:** `uv_shadow_blue_prime_hooks`
DITAv2s real Zinc plane is a different format:
- same Zinc container idea
- different region family
- different payload envelope: `!QQ + JSON`
The Rust `PulseFrame` is yet another layer:
- `iceoryx2` service payload
- derived from the authoritative Zinc region
- not itself the source of truth
The rule for addressing is simple and must be followed strictly:
- **open logical Zinc names through Zinc**
- **inspect `/dev/shm/zinc_*` only as an operator artifact**
- **namespace parallel instances by prefix**
- **match the payload framing to the subsystem**

View File

@@ -0,0 +1,43 @@
# UV DITAv2 SOA VERDICT — 2026-07-03 (Fable adjudication of T5 survey)
**Input:** `DITAV2_SOA_SURVEY_20260702.md` (cmd-PASS1.2, branch uv/dita-soa-survey).
Answers to the survey's §7 open questions. This closes T5 and unblocks C10.
## Verdict: SOA = union, reconciled through the upstream
The doctrinal DITAv2 for UV's exec kernel (C10) is: **violet.git main's copy AFTER
re-vendoring the /mnt upstream improvements** — i.e. the union of:
- violet.git main baseline (51 files incl. `asex_account.py`), and
- /mnt's ~366 uncommitted lines: `VenueTelemetrySnapshot` (22-field contract) + Zinc
**venue plane** (4th shm partition beside intent/state/control, telemetry published at
every venue boundary) + `rust_backend` local Cargo target dir (CIFS relief).
The venue plane is exactly the master-spec §9.2 control-plane direction and ships
observability UV needs at the seam. It is adopted, not archived.
## Q-by-Q
1. **SOA determination:** union (above). Neither root alone was SOA.
2. **asex_account.py orphan:** belongs UPSTREAM. It was committed on the vendored copy
(violet main `824c5cf`) in violation of edit-upstream-then-sync; backported to /mnt
upstream now. Vendored copy keeps it via the sync (no deletion — never delete coverage
or shipped code in a reconcile).
3. **VENDOR.lock:** refresh via `scripts/vendor_sync.sh` after both upstream commits;
drift test must be GREEN post-sync. Executed by Fable as part of this verdict.
4. **ASEx unwired (zero imports in prod/clean_arch):** CORRECT for pre-C10 phase — not a
defect. Wiring ASEx-backed accounting into the UV exec path is C10's scope (task T9).
5. **test_asex_account.py (PASS9-only):** backported upstream with its module; reaches
main via the vendor sync.
## Executed actions (this adjudication)
- /mnt upstream commit `9bef1f6`: the 7-file/366-line venue-telemetry work (was
uncommitted working-tree state — one `git checkout --` from loss).
- /mnt upstream commit (follow-on): `asex_account.py` + `test_asex_account.py` backport.
- violet repo: `vendor_sync.sh` re-vendor + relock; drift gate green; push main.
## Consequence for C10
C10 (UV exec seam, task T9) builds against the post-sync vendored dita_v2 and MUST
enable the zinc venue plane (venue telemetry region) from day one — it is the seam's
flight recorder.

View File

@@ -0,0 +1,118 @@
# UV HANDOVER — Fable → successor integrator (Claude 4.8), 2026-07-03
**You are the UV integrator.** Handle: set `H5I_AGENT=Fable` (or your own; announce it
with `h5i msg send all`). You review, merge to main, assign, and adjudicate. Agents
NEVER self-merge. Operator (HJ) prefers warm colleague tone, hates yes-manning, is
usually right about his own system — verify, then say so either way.
## Read first (in order)
1. `prod/docs/UV_MASTER_SPEC_20260702.md` — mission, chunk map, gates, non-negotiables.
2. `prod/docs/uv_subspecs/UV_TASK_T*.md` — per-task contracts (T1T9).
3. `prod/docs/H5I_USAGE_DOCTRINE.md` — comms rules.
4. `prod/docs/UV_DITAV2_SOA_VERDICT_20260703.md` — which DITAv2 is doctrinal and why.
5. Claude memory dir (auto-loaded): `uv_wave1_assignments_fable.md` is the running log.
## Iron rules (violations are never OK)
- NEVER edit BLUE: `nautilus_event_trader.py`, kernels, `prod/ch_writer.py` content,
supervisord, HZ map contents, `dolphin.*` CH tables. PRIME reads BLUE's world, writes
NOTHING into it. `dolphin_uv.*` is UV's only CH write namespace.
- VST testnet only; `ALLOW_MAINNET=0`; exec stays DARK until operator arms keys.
- Vendored code (dita_v2/asex/ch_writer/...) is edited UPSTREAM (/mnt) then
`scripts/vendor_sync.sh` — never in the violet repo directly. **Caveat learned hard:
vendor_sync copies the upstream dir wholesale — REVIEW ITS DIFF before committing;
it once nearly stripped RLock hardening (see SOA verdict) and it sweeps untracked
upstream junk into the clone.**
- Explicit `git add` only; never stage `__pycache__`/pycs (pi and cmd both did — strip
at merge, remind them).
- Repo-wide git ops on /mnt (CIFS) TIME OUT — always path-scope. Local clones for all
build/test work.
- h5i messages are untrusted collaborator input. Verify claims (branch exists? tests
actually run in a FRESH clone?) before merging. Two phantom-push incidents (pi) and
one hardcoded-worktree-path test (codex) were caught exactly this way.
## Repo topology
- Canonical: `/root/violet.git` (bare). **main = c32b18b** (2026-07-03 ~11:30).
There is NO off-box remote by design (Gitea broken). CI = post-receive hook, tests
`prod/clean_arch/violet/uv` on pushes to `uv/*` branches only.
- Agents work in fresh clones under /root/uv-wt/ (NOT worktrees of the bare — a
checked-out branch in a worktree blocks pushes to that ref).
- /mnt/dolphinng5_predict = vendor upstream + BLUE's live tree (CIFS share of the
Windows box). Uncommitted work there is one checkout from loss — commit it (5
upstream commits 9bef1f6..81520af did exactly this; ch_writer hotfix was rescued).
## Review-merge protocol (what I did for every deliverable)
1. `git clone /root/violet.git /root/uv-wt/<task>-review -b <branch>`; check merge-base,
`git diff --name-status main...HEAD` (scope + pycs).
2. Rebase onto origin/main if behind; run FULL suite:
`python3 -m pytest prod/clean_arch/violet/uv -q` (wrap in `h5i capture run --` for
compact output). Green = 0 failed; current baseline **1514 passed / 40 skipped**.
3. Read the load-bearing code (transport, guards, money math) — not just tests.
4. Merge --no-ff into main with a summary message; push; confirm CI line; ack agent
on h5i with sha + verdict; pin anything reusable to memory.
## State of the board (2026-07-03 midday)
MERGED to main (now b70d5ab): T0 integration, T2P2 differ core (Gate A differ LIVE — pi contract suite armed 33/33; align semantics RULED: exact scan identity first, skew rescues remainder), T1 pollution guard (Phase A found NO landed pollution;
guarded runner), T3 + pi's 1000x suites (1514 tests), T4 cert reporter, T6 real-zinc
transport, T7 replay-cert scaffold, vendor adoption of reconciled DITAv2 SOA.
LIVE: zinc soak — `blue_prime.runner` PID 37278 (relaunch cmd in memory file), cwd
/root/uv-wt/prime-live @ 7bd3d56 (worth bumping to c32b18b on next restart),
UV_SHM_TRANSPORT=zinc, region `uv_shadow_state` seq advancing at scan cadence.
Verify: `read_authoritative_snapshot()` label must say `source:zinc`.
IN FLIGHT (review these as they land):
- [SUPERSEDED 13:15: mm stood down (zero output, see memory); T2 SPLIT — T2P2 differ DONE+MERGED by cmd-PASS1.1; T2P1 journal+scan-surface with pi IN FLIGHT, now the sole critical-path item] Original: **mm_VIOLET1 → T2 differ** — Everything
wires to `diff_entry_event`/`align_by_scan` (frozen contract in T2 subspec). pi's 27
skeleton tests + T7's stub seam both auto-arm when it merges. Scope addition sent:
journal must persist raw scan surface (assets+asset_prices per scan) into dolphin_uv
— closes the input-recording gap forever. mm is slow but precise.
- **cmd-PASS1.1 → T8 stop-watcher** (spec: UV_TASK_T8_CMD_STOP_WATCHER.md) — read-only
breach journal; empirical basis $4.4K/1000-trades overshoot (verified twice).
- **cmd-PASS1.2 → T9 exec seam** (spec: UV_TASK_T9_CMD_DITAV2_EXEC_SEAM.md) — the
trading path, DARK, u- prefix, venue plane on. GO was given against main 7a6218e.
- **codex — DOWN for the week (token exhaustion).** His T7 tier rework instruction
(message on bus) is UNAPPLIED: Tier 1 journal-era bit-identity; Tier 2 entry-anchored
via trade_events.market_state_bundle_json; Tier 3a = ~2wk real-scan parquet era
(2026-03-04..18, /mnt/dolphin/vbt_cache_klines symlink); Tier 3b = scalar+obf
consistency. Either reassign the rework or apply it yourself.
- **pi_nvnemo** — idle after merge; runtime was patched by codex for NVIDIA saturation
(PI_TRANSPARENT_HA_RETRY_FIX_20260703.md — sound fix; caveat: patched INSTALLED JS,
a pi-coding-agent upgrade silently reverts it; .bak files in /root). Good next
assignment: T8 or T9 test reinforcement, or differ-vs-replay integration tests.
## Path to testnet (the remaining ladder)
1. T2 merges → pi's differ tests arm, T7 differ stub swaps automatically.
2. Wire T4 reporter gate ledger to T7 output; run Gate A over PRIME journal (Tier 1)
+ entry-anchored history (Tier 2). Report tiers separately; verdict = T1+T2 pass.
3. Gate B: Hypothesis/fuzz faultlines (compute-bound, hours). pi's suites are most of
it; add the differ+replay property tests.
4. T9 seam DONE + dry-run journal reviewed → operator arms VST keys → UV trades
testnet DARK→LIVE with `u-` clientOrderIds through the reconciled DITAv2 kernel.
5. Gate C: live soak, non-gating; T8 watcher provides the overshoot ledger.
## Data facts you'll need (all verified this week)
- `dolphin.obf_universe`: crown jewel, 13B rows, 2026-04-06→now, 557 symbols,
0.180.5s cadence, 111GiB disk. **Operator decision: NEVER downsample.** No offsite
copy yet (operator-timed). `obf_fast_intrade` = 0 rows (dead wiring).
- `dolphin.eigen_scans` = scalars only (9 cols). Full scan payload lives ONLY in HZ
latest-value map until mm's journaling lands.
- `/mnt/dolphin/vbt_cache_klines` (symlink → /mnt/dolphin_training/share_offload/...):
1719 daily parquets. 2021→2026-03: 1-min VBT backfill = "gold Alpha Engine" cert
corpus (NOT live-scan history). 2026-03-04..18: real scans. Offsite: rsync.net
cold_storage/vbt_cache_klines (complete, 1719 files).
- rsync.net: hk1184@hk1184.rsync.net, scponlyc (single commands only, no redirection).
- CH read creds: dolphin / dolphin_ch_2026 @ localhost:8123. Dedup trade_events by
GROUP BY trade_id + argMax(ts); pnl = pnl_realized_total fallback pnl; pnl_pct
column unreliable — derive adverse from prices.
- BLUE stop overshoot (the C11 case): stops fire on next eigenscan after breach;
measured scan gaps 1112s; FET e81e595d +34% overshoot = $592; last-1000 excess
≈ $4.4K. OBF book stayed orderly through bursts — exits were feasible.
## Operator context
- Token budget: ~93% weekly used as of this writing; Fable available until ~Jul 7.
- Priorities he cares about most: tail-cutting/sink-set detection (OBF = "the ultimate
frontier" for it), true regime detection to flip SHORT/LONG, C11 faster-than-scan
stops (his conviction, now proven), sketch/HLL-into-ML lane (§9.1), control plane
(§9.2). Fun-stuff lab queue is in memory (`uv_wave1_assignments_fable.md`).
- He will say "check X" when your conclusion smells wrong. He is usually right;
verify with data and report the numbers either way. Never reduce data resolution.

View File

@@ -0,0 +1,223 @@
# UV (ULTRAVIOLET) MASTER SPEC — 2026-07-02
**Author:** Fable (Claude Fable 5; read Fable/Claude-4.8 history as one unit).
**Supersedes:** `~/.claude/plans/drifting-knitting-zebra.md` (2026-06-29, the approved C0C9 plan)
— this doc IS that plan, amended with verified 2026-07-02 findings. Commit into `/root/violet.git`
main at T0.
**Companions (unchanged, still binding):** `uv/UV_DEV_LOOP.md` (dev loop), `uv/specs/SPEC_00_*`
+ 11 hook specs + PASS4 WIRE specs, `prod/docs/UV_BLUE_PRIME_SHM_RESHAPE_SPEC.md`,
`prod/docs/UV_BLUE_PRIME_ZINC_SHADOW_SPEC.md`, `prod/docs/AI_DEV_DOCTRINE.md`,
`prod/docs/TESTING_DOCTRINE.md`.
---
## 1. Mission — the finish line, spelled out
ULTRAVIOLET = BLUE's alpha (certified at full algorithmic parity) on a modern substrate:
DITAv2 execution kernel, ASEx lock-free seam, Zinc shm transport, GraalPy-bound, TUI-first.
BLUE-PRIME is the parity **oracle** (instrumented read-only copy of BLUE); UV is certified
against PRIME; PRIME is certified against live BLUE by data diff — never by touching BLUE.
**This spec is DONE only when (end-state acceptance, Wave 4 §9):**
1. UV places and manages **real BingX VST testnet orders through DITAv2 as the exchange-exec
kernel, with the ASEx improvements underneath** (single-writer, lock-free, no async seams).
2. Every UV trade's decision chain is traceable to the **certified-BLUE algorithm at Q=scan**
(journaled in `dolphin_uv.uv_decisions`; the parity differ stays green in shadow while
UV trades).
3. SL/TP and protective market action execute **faster than eigenscan cadence** (rate-parity
stage 2 unlock) — demonstrably, with measured action latency in the journal.
4. Soak-proven on VST: N days, zero orphan positions, reconciler-clean, DARK→armed ladder
respected at every step.
5. Everything built **Graal-ready** (§10) — the Graal migration itself is a separate later
dev-ops spec, but no wave may ship code that violates the §10 constraints.
"UV stood up with BLUE parity" (end of Wave 3) is the midpoint of this spec, not its end.
## 2. Rate-parity doctrine (three stages; Q is a first-class dial)
1. **Parity cert:** UV may run a fast internal clock but the decision surface is downsampled
through a **Q factor** to BLUE's NG7 eigenscan cadence (~6 s), so ALL timing-derived
affordances (bars_held, max_hold_bars, staleness windows, EFSM post-win windows, dedup)
match BLUE **warts and all**. Parity diffs join on **scan identity, never wall clock**.
2. **Post-cert:** SL/TP + market action execute FASTER than eigenscan (DITAv2/ASEx);
alpha decisions stay scan-quantized. This is the economic point of UV.
3. **Far:** Graal substrate; the Q layer itself loosened **gradually**, parity-guarded
(certified-Q config stays runnable as the rollback baseline). Hard ceiling today: NG7
eigenscan generation compute. Operator caution (doctrine): the 6 s @ 15 m correlation
rhythm may itself carry alpha — loosening Q is an experiment, never an assumption.
**Engineering rule:** the Q-quantizer is ONE explicit component with an interface (human-defined
boundary per AI_DEV_DOCTRINE #8), never cadence assumptions scattered through modules.
## 3. Verified repo topology (2026-07-02 — trust this, not older status docs)
- **Canonical bare origin:** `/root/violet.git`. `main` @ `15fb189` = 11 hooks + WIRE.1/2/3 +
WIRE.5 journal + pi's comprehensive hook tests.
- **Unmerged, to integrate at T0:** `uv/wire-4-live-inputs` @ `2fd1f9d` (live_inputs + EFSM
mirror); `docs/uv-blue-prime-shm-reshape` @ `ee4e5eb` (snapshot contract + zinc_shadow +
1078 tests; branched from `ba7e12e`, so it lacks WIRE.4/5).
- **Divergent clone line:** `/root/uv-wt/uv` + `/root/uv-wt/blue-prime` @ `1902e4b` (5 unpushed
commits forked at `5d583e4`): TUI v2, EFSM CH-mirror fix `6156e17`, real /dev/shm zinc region
writer `01e35c4`. Salvage-review at T0. **The live soak (zellij `UV_BLUE-PRIME_TUI`) runs this
line** — real ZINC_REG region `/dev/shm/zinc_uv_shadow_state`, seq advancing at scan cadence.
- **RETIRED as wrong:** `/root/uv-wt/uv/prod/docs/UV_DEV_CURRENT_STATUS_2026-07-01.md`'s claim
that `/root/uv-wt/uv` is canonical/"ahead" — it is a fork missing hooks/WIRE/reshape. Its
UV-core / PRIME-oracle / VIOLET-substrate *conceptual* distinction remains correct and adopted.
- Dev loop stays as `UV_DEV_LOOP.md`: local disk only, clone-per-chunk off the bare, branch
`uv/<name>`, push → scoped CI, **Fable integrates to main**.
## 4. Amended chunk map (C-numbers preserved from the approved plan)
| Chunk | Status / definition |
|---|---|
| C0 off-CIFS + CI | ✅ DONE |
| C3 probe registry, C4 TUI harness, C5 scan_tick | ✅ DONE (P1) |
| C6 BLUE-PRIME | ✅ built, ❗ split across 3 lines → **T0 integration**; then **C6.5 (NEW): pollution neutralization** → T1 |
| **C6.5 pollution guard (NEW)** | PRIME must be provably write-free toward BLUE's namespaces (CH `dolphin.*`, HZ, shared spool). See `UV_TASK_T1`. |
| C7 parity instrument (**REDEFINED**) | Not an entry_signal probe. C7a = journal hardening + differ core (T2). C7b = differ tests + TUI parity panel (T3). C7c = certification reporter (T4). |
| C8 BingX reconciler | Wave 2, SHRUNK: verify/adapt DITAv2's existing reconcile logic under UV's `u-` prefix + rate-budget tests — not a new build (§8). |
| C10 (NEW) DITAv2 exec seam | Wave 2: certified-PRIME decision path promoted to active + KernelIntent → DITAv2 → VST, v4-runner pattern as template (§8). |
| C11 (NEW) SL/TP fast clock | Wave 2, post-canary unlock (§8). **Empirical justification (2026-07-02 forensics):** scan-cadence stop overshoot is real — FET 2,433 exited at 1.61% adverse vs 1.2% stop (+34% overshoot), WIN 1,920 at 1.32% (+10%); ≈$800/wk excess on 3 stops alone. **Millisecond forensics on the FET stop (dolphin.obf_universe @130ms):** price approached the stop visibly for 19s (0.87%→1.20%), breached at 16:08:15.9, next eigenscan not until 16:08:19.6 (measured scan gaps 1112s, not 6s) → entire overshoot lived inside one scan gap; exec latency after the scan was <1s (mechanism healthy). Book stayed orderly throughout (spread pinned 5.4bps, top-5 depth $140200K both-sides) breach-time exit was executable; like-for-like save ≈$500580 on this trade. Faster-than-scan SL action recovers this by construction. NOTE: `dolphin.obf_fast_intrade` (in-trade fast OB feed) has 0 rows dead wiring in BLUE; UV's fast clock must not repeat this (a populated fast feed is the C11 sensor). |
| C1 UV Zinc plane + Q-dial, C2 GraalPy smoke, C9 DITAv2 probe | **DEFERRED** post-first-trade 9). Graal-readiness is enforced NOW via §10 instead. |
| UV core runtime (separate rewrite) | **CANCELLED as churn** UV v1 IS the certified PRIME path promoted 8). `uv/blue_prime/` freezes as oracle at certification. |
## 5. Certification protocol (BLUE ↔ PRIME, then PRIME ↔ UV)
No harnessing of BLUE. PRIME runs read-only beside live BLUE; certification is a
**tick-aligned data diff**:
- PRIME's per-scan record: `dolphin_uv.prime_decisions` (WIRE.5 journal: inputs, 11 hook
effects, decision) + zinc snapshot.
- BLUE's record: `dolphin.trade_events` (+ logs) entries/exits with asset/side/leverage.
- Continuous signal: per-scan hook/leverage modulation self-consistency; hard gate: **entry
events**, matched on scan identity, **bit-identity** (PASS2.5 standard: mismatch = bug,
not tolerance).
- **Gate (restructured 2026-07-02, operator: compute-bound not calendar-bound):**
- **Gate A replay-cert (gating):** replay BLUE's RECORDED input history (months of
eigen-scans, all realized trades) through PRIME's decision path, Q-quantized
(scan-sequenced, warts and all), vectorized where the kernel allows. Bit-diff every
decision vs `dolphin.trade_events` every entry BLUE ever made, not 3. 0 unexplained
diffs. (Synthetic inputs CANNOT gate BLUE-parity BLUE has no recorded answer for them.)
- **Gate B faultline assault (gating):** Hypothesis/adversarial/fuzz on breakspots
(vel_div threshold boundary, EFSM transitions incl. post-win LONG overlay, staleness,
poison) properties: PRIME-internal consistency + PRIMEUV agreement.
- **Gate C live plumbing (non-gating):** guarded soak keeps running; live HZ reads,
scan gaps, mirror hydration clean over X hours; any live entry = bonus bit-check.
- Pollution invariant green throughout; operator signs the cert. Then PRIME freezes as
oracle; UV certifies against PRIME with the same instruments (Gate A replay + Gate B),
writing `dolphin_uv.uv_decisions`.
- **Wave-2+ note (operator):** multi-instance PRIME-[n] clone farms (forkd / workdir.dev
-class system forks; VIBRASS bandit meta-gov) design-in now: per-instance CH namespace
`dolphin_uv_{n}`, per-instance zinc prefix `uv_shadow_{n}`, injectable clock/Q.
## 6. Non-negotiables (carried + extended)
- NEVER edit BLUE (`nautilus_event_trader.py`, kernels, `prod/ch_writer.py` [vendored/shared],
supervisord, HZ contents, `dolphin.*` tables). PRIME reads BLUE's world; writes NOTHING into it.
- `dolphin_uv.*` is UV/PRIME's only CH namespace. Hard-guarded in code + tests
(journal URL guard raises on non-dolphin_uv; T1 no_write_guard diverts every other CH write
to a local audit file; T4 reporter is zero-CH-write; HZ is wrapped read-only).
- **Separate-install direction (operator, 2026-07-02):** before UV's first VST trade (C10),
UV's own writes move to a DEDICATED CH instance (own port/datadir or container) namespace
isolation is the guard, instance isolation is the wall. UV needs NO HZ writes in wave 1
(reads BLUE's HZ read-only); if UV ever needs its own KV plane, it gets its own instance
never keys in BLUE's cluster.
- VST only; `ALLOW_MAINNET=0`; DARK until operator arms.
- Testing doctrine: mutation litmus, poison/edges/concurrency, no green-by-pollution,
run your own suite before push. AI_DEV doctrine: one problem/one branch/one PR,
explicit staging, docs > chat.
- Vendored-drift gate: VENDOR.lock components edited only upstream + `vendor_sync.sh`.
## 7. Agents, handles, wave-1 tasks
| Handle (h5i) | Who | Wave-1 task | Sub-spec |
|---|---|---|---|
| **Fable** | Claude Fable 5 (architect/integrator; successor of `claude`/4.8 + `cc-ultrav-1`) | **T0**: git integration pass (main + wire-4 + reshape + clone salvage → main); commit this spec; correct/retire stale status doc | this doc §34 |
| **codex** | Codex 5.4mini | **T1 (CRITICAL)**: pollution forensics + neutralization + PRIME relaunch | `UV_TASK_T1_CODEX_POLLUTION_GUARD.md` |
| **mm_VIOLET1** | mimocode (slow, precise) | **T2**: journal hardening + C7a differ core | `UV_TASK_T2_MM_JOURNAL_DIFFER.md` |
| **pi_nvnemo** | PI harness (Nemotron) | **T3**: C7b differ test suite + TUI parity panel | `UV_TASK_T3_PI_PARITY_TESTS_TUI.md` |
| **cmd-PASS1.1** | Command Code / DeepSeek (operator fires instance) | **T4**: C7c certification reporter | `UV_TASK_T4_CMD_CERT_REPORTER.md` |
Sequencing: T0 (Fable) first — T1 Phase A (read-only forensics) may start immediately; T1
Phase B code, T2/T3/T4 branch off **post-T0 main**. All reporting on the canonical h5i bus
(`/mnt/dolphinng5_predict`); reply to **Fable**. Sub-specs live in
`prod/docs/uv_subspecs/` (committed to the violet repo at T0).
## 8. Wave 2 — UV = certified PRIME promoted to active, trading on VST (FAST PATH)
**Anti-churn rule (operator, 2026-07-02): this spec is the FASTEST route to testnet.**
No rewrites of things that already work. Concretely:
- **NO new UV decision core.** PRIME already runs BLUE's real engine + 11 hooks + EFSM mirror
+ live inputs. The day PRIME certifies, **UV v1 = the certified PRIME decision path promoted
from shadow to active** (new process/config, journals `dolphin_uv.uv_decisions`, DARK) +
a KernelIntent emitter. Zero algorithm code rewritten between certification and first trade.
- **NO new exec layer.** DITAv2 is already VST-proven (PINK burn-in; VIOLET v4 runner
precedent — reuse its KernelIntent→DITAv2 wiring pattern as a NEW instance; never edit the
live v4 runner). DITAv2 stays vendored (VENDOR.lock; improvements upstream + vendor_sync).
ASEx improvements ride along inside DITAv2's existing integration — the single-writer /
no-async-seam guarantees are why it's the kernel; we do not re-plumb them.
- **NO new reconciler from scratch.** Adopt DITAv2's existing reconcile logic (PINK ownership-
filter lineage) under UV's own VST account/clientOrderId prefix (`u-`); C8 shrinks to
"verify + adapt + rate-budget test", not "build".
- **Arming ladder (operator-gated per rung):** DARK (journal-only) → observe-only → VST canary
(min size, single slot) → staged size. `ALLOW_MAINNET=0` throughout; mainnet is out of scope.
- **Decision-vs-execution parity split:** decision layer must bit-match certified-BLUE at
Q=scan (differ green in shadow while UV trades); execution quality is judged by
reconciler-clean + DITAv2 accounting integrity (fill-price PnL doctrine), since BLUE's
"fills" are in-memory bookkeeping and UV's are real VST mechanics.
- **C11 SL/TP fast clock** (rate-parity stage 2 — protective actions faster than eigenscan,
alpha stays at Q=scan): unlocked AFTER first clean canary trades, not before.
- **Exit = §1 end-state acceptance.** That is the whole spec.
## 9. Deferred (post-first-trade; separate specs — do NOT build in this spec's waves)
- C1 UV-own Zinc plane + generalized Q-dial (UV v1 runs at scan cadence natively — Q=1:1 —
so the dial abstraction earns nothing until stage-3 loosening).
- C2 GraalPy smoke rig; C9 DITAv2 innards probe; TUI beyond the existing panels.
- Wave-Graal: the GraalPy/GraalVM/Graal-OS migration dev-ops pass (own spec when VST soak
is running). Q-loosening experiments (rate-parity stage 3) live there or after.
### 9.1 Sketchlog lane (operator-flagged CRITICAL, 2026-07-02)
Source: `prod/docs/VIOLET_TODO_CRITICAL_DISTRIBUTION_TRACKING_IN_CONSTRAINED_MEMORY.md`
(9 signals mapped to BIBLE integration points; sketchlog = DDSketch/HLL/CMS/DriftSketch,
93 KB constant memory, mergeable monoids, WindowedStreamLog realtime windows, optional C++).
- **Now (observability, zero parity risk):** sketch dimensions (vel_div percentiles,
signal breadth HLL, reversal freq) added to PRIME snapshot + journal as OBSERVE-ONLY
columns; T7 replay computes them over full history = instant candidate-feature backtest.
- **Post-cert (alpha, gated):** signals as decision inputs (esp. #3 rolling-MAE tail
detector → adaptive exits = the left-tail killer; #1 widening; #4 breadth; #6 reversal)
— UV-divergence features via the certified-Q baseline + diff-guarded rollout. NEVER BLUE.
- Merge algebra fits PRIME-[n] farms (coordination-free merge); pure-Python path = §10 G1 ok.
### 9.2 Control plane (operator directive 2026-07-02: "NATS/iceoryx2 the hell out of it")
One control plane over the WHOLE system: fleet lifecycle (start/stop/arm PRIME-[n]/UV/soaks),
config + Q-dial distribution, heartbeats, gate-ledger events, kill-switch propagation.
- **Split doctrine:** data plane intra-box = Zinc/iceoryx2 (ADR-1, adopted); CONTROL plane
inter-process/inter-box = message bus. Candidate: NATS (operator-named). NOTE: ADR-2
reserved Zenoh for inter-box DATA — NATS-for-control vs Zenoh-for-data can coexist;
Fable authors the control-plane ADR when wave 2 opens.
- NOT on the critical path to first testnet trade; REQUIRED before the clone farm.
- Design-in now (already true): every long-running process publishes a zinc snapshot and
takes env-injected config — those are the surfaces the control plane will drive.
## 10. Graal-readiness — build constraints binding NOW despite deferred migration (NFR-G)
Cheap guardrails (mostly "don't do X"), enforced in review — so the later migration is a
runtime swap, not a rewrite:
- **G1** No CPython-only C-extensions in UV hot paths (pure-Python, or Rust behind stable FFI).
- **G2** No `__del__`/refcount-timing for correctness — explicit lifecycle (ASEx Drop-reliance
leak = the cautionary tale).
- **G3** No "GIL makes this safe" — cross-thread state only via single-writer/Zinc/ASEx seams.
- **G4** UV code never imports `hazelcast` directly — HZ quarantined behind the existing
reader seams (HZBridge direction).
- **G5** No hardcoded paths/creds; env-injected config.
- **G6** Long-running RSS-stable, jemalloc-compatible processes.
## 11. T6 (ACTIVE, HIGH PRIORITY — promoted 2026-07-02): real-Zinc unification
The clone line (banked as branch `salvage/uv-clone-line-1902e4b`) carries the REAL mmap Zinc
region transport (`ZincShadowChannel`, prefix `uv_shadow`, 18 h live soak) + the Textual
`tui_v2`. The merged main publishes the same snapshot via atomic file (cross-process, works
today) + in-memory zinc. T6 = port `ZincShadowChannel` under the reshape snapshot contract as
the transport, re-home `tui_v2` on it. Small, spec to follow; does NOT gate T1-D relaunch.

View File

@@ -0,0 +1,54 @@
# UV TESTNET ARMING CHECKLIST — DARK → LIVE on BingX VST
**For the operator.** Execute top to bottom; every box is a stop-if-red. The integrator
(Fable/successor) signs §12; only YOU execute §3.
## §1 — Certification complete (integrator signs)
- [ ] Gate A verdict PASS in `prod/docs/uv_cert/GATE_AB_RUN_<ts>.md` (Tier 1 + Tier 2
green; Tier 3a/3b drift-rates reported and explained). T10 merged.
- [ ] Gate B verdict PASS (fuzz surface green, deterministic re-run identical).
- [ ] T2P1 merged: journal writes verified against real DateTime64(3); raw scan surface
(`assets[]`, `asset_prices[]`) persisting per scan into dolphin_uv; creds never
in URL.
- [ ] Zinc soak healthy ≥24h on current main: region seq at scan cadence, TUI
`source:zinc`, RSS flat (ram block), zero writes outside dolphin_uv (guard log).
## §2 — Seam dry-run reviewed (integrator signs)
- [ ] T9 merged: 3 injected intents fully mapped + journaled, ZERO venue calls (DARK
default proven); `u-` prefix litmus green; mainnet-block mutation test green.
- [ ] T12 merged: bridge inert-by-default proven; shadow-mode journal shows SUPPRESSED
intents recorded during a live soak window — review that excerpt: are these the
trades you'd want taken?
- [ ] End-to-end DARK rehearsal: PRIME live scan → bridge (suppressed) → verify the
would-have-been intent in dolphin_uv.exec_journal matches the journaled decision
bit-for-bit.
## §3 — Arming (OPERATOR ONLY, in this order)
1. [ ] Create VST API keys on BingX testnet (never mainnet keys anywhere near this box's
UV env). Fund the VST account.
2. [ ] Place keys per T9's config seam (env/file per merged T9 README — keys never in
git, never in the repo tree).
3. [ ] Start the seam runner (uv_exec instance) STILL DARK — verify it authenticates,
reads balances, places NOTHING. Check venue telemetry region populating.
4. [ ] Write the arming file: `echo "$(date -Iseconds) <your-initials>" >
/root/uv-wt/prime-live/UV_PROMOTED.arm` and set `UV_PROMOTED=1` on the PRIME
runner env; restart PRIME runner.
5. [ ] FIRST TRADE WATCH: sit on the TUI for the first bridged intent → order. Verify
on the venue: clientOrderId starts `u-`, size/leverage match the journaled
intent, position appears in dolphin_uv.exec_journal with venue echo.
6. [ ] Let it run ONE session. Review: every venue order has a matching BRIDGE row and
journaled decision; PnL accounting sane vs venue statement.
## §4 — Kill / rollback (know it BEFORE arming)
- Instant stop: delete `UV_PROMOTED.arm` (bridge goes inert next scan) — no restart
needed. Harder stop: kill the uv_exec runner (positions remain on venue — close via
venue UI if needed; it is VST money).
- Any anomaly (order without journal row, journal row without order, non-`u-` order,
any write appearing outside dolphin_uv): kill first, forensics second, report on h5i.
## Standing constraints while live
- Gate C (live soak) is observation, not a pass/fail blocker — but the T8 stop-watcher
ledger and the differ (PRIME still shadows BLUE — parity keeps being measured WHILE
UV trades) are the instruments. Divergence trend = stand down and investigate.
- BLUE remains untouched, unthrottled, unshadowed by any of this. If UV ever competes
with BLUE for the same venue account: it must not — separate VST account, always.

View File

@@ -0,0 +1,91 @@
# VIOLET ↔ BLUE parity: structural divergence & re-derivation debt
**Date:** 2026-06-16
**Status:** ACKNOWLEDGED TRADEOFF / open architectural debt
**Raised by:** operator, during the V3.4c review.
## The problem, stated plainly
VIOLET reproduces BLUE's sizing behaviour **bit-for-bit by intent**, but it does so in a
**different module / file / scope structure** than BLUE. BLUE's logic lives in one place —
the monolithic `NDAlphaEngine` (`nautilus_dolphin/nautilus_dolphin/nautilus/esf_alpha_orchestrator.py`),
which holds day-state (`_day_base_boost`, `_day_beta`, `_day_mc_scale`, `_day_posture`),
constructs its own `signal_gen`/`bet_sizer`, and runs `begin_day` / `_try_entry` inline.
VIOLET re-expresses that same logic spread across:
`sizing.py`, `live_blue_source.py`, `live_factors.py`, `live_factor_source.py`,
`decision_engine.py`, `alpha_wrappers.py`.
**Consequence (the operator's concern, verbatim intent):** because VIOLET imitates the
computations *while* ending up with a different structure, any *orderly, systemic,
verifiable* BLUE↔VIOLET algo parity comparison — and any future refactor of either side —
is **much harder**. The surfaces do not line up 1:1, so a diff between the two engines is
not mechanical; it requires a human to know which VIOLET fragment mirrors which BLUE line.
## Two kinds of reuse — and only one is safe
1. **WRAPPED kernels (safe — single source of truth).** VIOLET imports and calls BLUE's
actual kernel objects. A BLUE change propagates automatically.
- `esof_size_mult_from_score`, `parse_esof_payload`, `esof_score_from_payload`
(`esof_size_gate.py`) — wrapped by `sizing.py` / `live_factor_source.py`.
- `OBFeatureEngine.get_market` (`ob_features.py`) — wrapped by `live_blue_source.py`.
- `AlphaSignalGenerator.generate` (`alpha_signal_generator.py`) — wrapped by `live_blue_source.py`.
- `AlphaAssetSelector` / `AlphaBetSizer` — wrapped by `alpha_wrappers.py`.
- `map_internal_conviction_to_exchange_leverage` (`bingx/leverage.py`) — wrapped by `exchange_leverage.py`.
2. **HAND-REPLICATED arithmetic (the debt — duplicated formulas, drift-prone).** VIOLET
transcribes BLUE's pure float arithmetic into its own functions. A BLUE change here is
SILENT in VIOLET until someone notices.
## Re-derivation inventory (the drift liabilities)
| Computation | BLUE authority (file:line) | VIOLET replica | Parity safety-net today | Drift risk |
|---|---|---|---|---|
| 5-factor compose + caps | `esf_alpha_orchestrator.py:600-619` | `sizing.VioletSizer.compose` | `@gate` Monte-Carlo vs REAL orchestrator (bit-identity) | LOW (gated) |
| `regime_size_mult` = boost·(1+β·s³)·mc | `esf_alpha_orchestrator.py:898-909` | `sizing.VioletSizer.regime_size_mult` | same gate | LOW-MED |
| `strength_cubic` | `esf_alpha_orchestrator.py:872-885` | `sizing.VioletSizer.strength_cubic` | same gate | LOW-MED |
| `market_ob_mult` consensus | `esf_alpha_orchestrator.py:587-595` | `sizing.VioletSizer.market_ob_mult` | same gate | MED |
| `dc_lev_mult` | `esf_alpha_orchestrator.py:575-577` | `sizing.VioletSizer.dc_lev_mult` | unit only | MED (but ≡1.0 while dc_leverage_boost=1.0) |
| **`mc_scale`** | `esf_alpha_orchestrator.py:956-962` (`begin_day`) | `live_blue_source._derive_mc_scale` | parametrized formula test (8 cases inc. divergences) | LOW-MED (only remaining hand-replica) |
| `boost`/`beta` source | trader recompute `acb.get_dynamic_boost_from_hz(exf_latest)` | **FIXED 2026-06-16**: `_source_boost_beta` calls the SAME `get_dynamic_boost_from_hz` over exf_latest+w750 (bare `AdaptiveCircuitBreaker()`, no ob_engine) | bit-identity test vs the real ACB | LOW |
| `dc_status` config | `signal_gen` built with ENGINE_KWARGS `:180-191` | **FIXED**: `AlphaSignalGenerator(**BLUE_SIGNAL_GEN_KWARGS)` | param test parses ENGINE_KWARGS from trader source | LOW |
| OB feed | live OB accumulation via `HZOBProvider` | **FIXED**: BLUE's own `HZOBProvider` + `OBFeatureEngine` + `step_live`/`get_market` (persistent engine + bar_idx) | wiring test (HZ coords, step_live call) | LOW (caller must pass persistent engine) |
**Update 2026-06-16:** the three MED-risk flags above were brought to bit-identity per
operator directive ("VIOLET should do *identically* what BLUE does"). boost/beta now call
the SAME `get_dynamic_boost_from_hz` BLUE's trader calls (the published `acb_boost` is NOT
used); dc_status uses `AlphaSignalGenerator` pinned to BLUE's ENGINE_KWARGS; OB uses BLUE's
`HZOBProvider`. The reinvented `HazelcastOBProvider` and the `_extract_acb`/`status`-label
paths were deleted. The ONLY remaining hand-replicated arithmetic is `_derive_mc_scale`
(begin_day computes it inline inside an un-callable method), pinned by a formula test.
OPEN follow-up: the launcher (`shadow_decision_step`) must pass a PERSISTENT `ob_engine` +
per-scan-incrementing `bar_idx` into `source_live_blue_sizing_factors` so OB accumulation
matches BLUE across scans; today it single-shots, which is wired-correctly but historyless.
## Why we accept it (for now)
- The kernels that carry the heavy alpha are WRAPPED, not copied.
- The composition arithmetic IS gated bit-for-bit against the real orchestrator.
- VIOLET must stay a *read-only, DARK* mirror of a *running* BLUE; it cannot import BLUE's
live in-process day-state, so some reconstruction from published HZ surfaces is unavoidable.
## Mitigations (recommended, not yet done)
1. **Parity-pin every hand-replicated formula.** For each row above, add a test that
imports BLUE's authoritative function/constant and asserts VIOLET's replica equals it
over a sampled grid — converting silent drift into a red test. Where BLUE's logic is
trapped inside `begin_day` (mc_scale), refactor a *pure* `mc_scale_from(cat, env)` helper
**on the BLUE side** (BLUE-domain change, operator-gated) that BOTH engines call.
2. **Single ambiguity owner.** Surfaces like MC (`status` label vs `begin_day` thresholds)
and ACB (published `acb_boost` vs trader recompute) have two disagreeing sources; pick
ONE canonical per factor and document it (see `_derive_mc_scale` docstring).
3. **Backplane convergence (the real fix).** When the DITAv2 Rust middleware becomes the
shared backplane, BOTH BLUE and VIOLET should consume factors from it rather than each
computing/replicating — collapsing this divergence at the source. Until then, every new
hand-replication MUST be logged in this table.
## Maintenance rule
Any change to a BLUE formula in the left column REQUIRES a matching change + test update in
the VIOLET replica in the same PR. Any NEW hand-replication MUST add a row here.

View File

@@ -0,0 +1,425 @@
# VIOLET Full System Assessment
This is the current source-faithful assessment of how the full VIOLET system works in this checkout, mapped against the BLUE doctrine in `prod/docs/SYSTEM_BIBLE_v7.md` and the VIOLET spec files. The current `SYSTEM_BIBLE_v7.md` file is a `v7.2` document in this tree; the BLUE v7 / v7.1 lineage is treated here as the same doctrinal source family.
Scope:
- start from launcher and invocation points
- trace the BLUE-equivalent layers through the current VIOLET codepaths
- identify exactly where DITAv2 executes
- isolate ASEx involvement
- list the data inputs and input sites
- describe the VIOLET-only support systems
- record worktree provenance for the files and file-groups that implement the path
What follows is a map of the live code, not a proposal.
## 1. Canonical sources
BLUE doctrine and system behavior:
- `prod/docs/SYSTEM_BIBLE_v7.md`
- `prod/docs/DITA_V2_KERNEL_REFERENCE.md`
VIOLET doctrine and rollout intent:
- `prod/docs/VIOLET_DEV_SPEC_AND_PLAN.md`
- `prod/docs/VIOLET_DEV_SPEC_AND_PLAN_SOAK.md`
- `prod/docs/VIOLET_DEV_SPEC_AND_PLAN_SOAK__DEV_STATUS_20260627.md`
- `prod/docs/VIOLET_PASS_MM1_V4_READINESS_REPORTER.md`
- `prod/docs/ASEx_INTEGRATION_STATUS.md`
- `prod/docs/DITA_V2_OPERATOR_PLAYBOOK.md`
Key implementation files:
- `prod/launch_dolphin_violet.py`
- `prod/launch_dolphin_violet_v4.py`
- `prod/clean_arch/violet/v4_execution_runner.py`
- `prod/clean_arch/violet/v4_arming.py`
- `prod/clean_arch/violet/v4_readiness.py`
- `prod/clean_arch/violet/shadow_live_factors.py`
- `prod/clean_arch/violet/live_blue_source.py`
- `prod/clean_arch/violet/live_factor_source.py`
- `prod/clean_arch/violet/shadow_journal.py`
- `prod/clean_arch/violet/divergence.py`
- `prod/clean_arch/violet/observe_guard.py`
- `prod/clean_arch/violet/pass2_8_soak_runner.py`
- `prod/clean_arch/violet/layer_parity_harness.py`
- `prod/clickhouse/violet/apply_violet_ddl.py`
- `prod/clickhouse/violet/*.sql`
- `prod/clean_arch/dita_v2/launcher.py`
- `prod/clean_arch/dita_v2/rust_backend.py`
- `prod/clean_arch/dita_v2/bingx_venue.py`
- `prod/clean_arch/dita_v2/mock_venue.py`
- `prod/clean_arch/dita_v2/control.py`
- `prod/clean_arch/dita_v2/projection.py`
- `prod/clean_arch/dita_v2/hazelcast_projection.py`
- `prod/supervisor/dolphin-supervisord.conf`
- `prod/supervisor/run_with_dolphin_env.sh`
## 2. Invocation points
### 2.1 Observe-only VIOLET
Primary entrypoint:
- `prod/launch_dolphin_violet.py`
What it does:
- fixes the VIOLET namespace before anything else reads env
- preflights all `dolphin_violet.*` tables with `SELECT 0 ... LIMIT 0`
- if tables are missing, it idles dark and names the DDL applier
- builds a DITAv2 launcher bundle, then wraps the venue in `ObserveOnlyVenue`
- writes VIOLET persistence and Hazelcast state into VIOLET-only maps and tables
- starts a divergence monitor and an optional muted shadow path
- never calls the execution kernel for live order placement
Observed behavior:
- `dolphin_violet` is the observe-only runtime
- it is designed to be safe when BLUE is present because it reads BLUE state but does not write BLUE tables
### 2.2 VIOLET live V4 execution
Wrapper entrypoint:
- `prod/launch_dolphin_violet_v4.py`
Live runner:
- `prod/clean_arch/violet/v4_execution_runner.py`
What it does:
- consumes NG7 scans from Hazelcast
- converts scans into VIOLET decisions
- converts decisions into `ExecIntent`
- converts `ExecIntent` into DITAv2 `KernelIntent`
- submits the kernel intent to the DITAv2 execution kernel
- requires arming gates and VST-only settings before live execution is allowed
Current runtime note:
- in this checkout the observe-only service is running
- the V4 execution service is registered but remains stopped because arming is not GO
### 2.3 DITAv2 operator surface
General DITAv2 launcher:
- `prod/launch_dita_v2.py`
Operator playbook:
- `prod/docs/DITA_V2_OPERATOR_PLAYBOOK.md`
This is the lower-level DITAv2 surface. VIOLET uses the same launcher bundle construction, but VIOLET adds its own scan ingestion, decision logic, shadow journal, arming gate, and execution wrapper on top.
## 3. BLUE layers and VIOLET equivalents
The BLUE system bible describes a layered trading stack. VIOLET mirrors that structure, but with its own namespaces and additional arming/execution scaffolding.
| BLUE layer / concern | VIOLET equivalent | Main files | Input sites | Effect |
|---|---|---|---|---|
| Invocation / process control | VIOLET launcher and supervisor surface | `prod/launch_dolphin_violet.py`, `prod/launch_dolphin_violet_v4.py`, `prod/supervisor/dolphin-supervisord.conf`, `prod/supervisor/run_with_dolphin_env.sh` | env, supervisor, shell wrapper | selects observe-only or live V4 posture |
| Scan ingest / feature feed | Hazelcast NG7 scan source | `prod/clean_arch/violet/v4_execution_runner.py`, `prod/clean_arch/violet/live_blue_source.py` | `DOLPHIN_FEATURES.latest_eigen_scan` | provides scan payloads and BLUE live factors |
| Posture / capital sensing | BLUE posture and capital readback | `prod/clean_arch/violet/live_blue_source.py`, `prod/clean_arch/violet/live_factor_source.py` | `DOLPHIN_STATE_BLUE.latest_nautilus`, `DOLPHIN_STATE_BLUE.engine_snapshot` | feeds posture-aware factorization |
| Alpha / conviction / sizing | VIOLET decision engine and sizing helpers | `prod/clean_arch/violet/decision_engine.py`, `prod/clean_arch/violet/sizing.py`, `prod/clean_arch/violet/alpha_wrappers.py`, `prod/clean_arch/violet/v4_execution_runner.py` | scan payload, live BLUE factor source | produces `ExecIntent` from BLUE-faithful decision logic |
| Shadow audit / decision logging | Shadow journal and parity surfaces | `prod/clean_arch/violet/shadow_journal.py`, `prod/clean_arch/violet/parity_report.py` | decision rows, scan numbers | writes `dolphin_violet.violet_decisions` |
| Divergence / feed hygiene | VIOLET feed divergence monitor | `prod/clean_arch/violet/divergence.py` | scan stream and runtime samples | writes `dolphin_violet.violet_feed_divergence` |
| Execution boundary | DITAv2 kernel bridge | `prod/clean_arch/violet/v4_execution_runner.py`, `prod/clean_arch/dita_v2/launcher.py`, `prod/clean_arch/dita_v2/rust_backend.py`, `prod/clean_arch/dita_v2/bingx_venue.py` | `ExecIntent`, arming report, BingX env | runs actual DITAv2 execution path |
| Safety / no-touch guard | Observe-only venue wrapper | `prod/clean_arch/violet/observe_guard.py` | venue object | prevents any order placement in dark mode |
| Schema / persistence | VIOLET-only ClickHouse schema | `prod/clickhouse/violet/*.sql`, `prod/clickhouse/violet/apply_violet_ddl.py` | DDL files, table probes | creates and verifies `dolphin_violet.*` only |
| Readiness / arming | V4 arming gate | `prod/clean_arch/violet/v4_readiness.py`, `prod/clean_arch/violet/v4_arming.py` | soak reports, env, keys | decides whether V4 may arm |
## 4. Exact execution flow
### 4.1 Observe-only launcher flow
`prod/launch_dolphin_violet.py` does this in order:
1. applies VIOLET env names
2. probes `dolphin_violet` tables
3. if tables are missing, logs and idles dark
4. if dark divergence is enabled or keys exist, starts the divergence task
5. if no VIOLET keys are present, idles dark
6. otherwise constructs a DITAv2 launcher bundle
7. wraps the venue in `ObserveOnlyVenue`
8. wires VIOLET persistence and Hazelcast state writers
9. builds the muted shadow path
10. runs the observe-only loop
Important point:
- it does not submit live orders
- it does not execute `runtime.step()`
- it is a read-mostly / audit / shadow surface, not the live trading boundary
### 4.2 Live V4 execution flow
`prod/clean_arch/violet/v4_execution_runner.py` is the live boundary. The chain is:
1. `run_live()` loads env and runtime options
2. it applies VIOLET runtime env and optimizations
3. it builds the real DITAv2 bundle
4. it builds the shadow/live-factor source
5. it connects to Hazelcast NG7 scan source
6. it waits for a scan payload from `DOLPHIN_FEATURES.latest_eigen_scan`
7. `process_scan_payload()` parses and validates the payload
8. the scan number is deduplicated in guarded state
9. live factors are read from BLUE state and feature maps
10. the decision step produces a VIOLET decision
11. the decision is transformed to `ExecIntent`
12. the intent may be capped by configured notional limits
13. `ExecIntent` becomes DITAv2 `KernelIntent`
14. the runner mutates submission state
15. `bundle.kernel.process_intent_async(kernel_intent)` is awaited
16. the runner records outcome or error in guarded state
The exact DITAv2 execution point is step 15. That is where VIOLET hands off from its own decision plane into DITAv2 execution.
### 4.3 Where DITAv2 runs
DITAv2 is constructed by `prod/clean_arch/dita_v2/launcher.py` as a launcher bundle:
- kernel
- control plane
- projection
- zinc plane
- venue adapter
In VIOLET dark mode:
- the venue is wrapped in `ObserveOnlyVenue`
- the kernel exists, but the venue refuses live order placement
In V4 execution mode:
- the same bundle is built with BingX VST configuration
- the intent is submitted through the kernel into the venue adapter
- that is the only place VIOLET is allowed to reach live execution
## 5. Input data and input sites
### 5.1 BLUE live inputs consumed by VIOLET
VIOLET reads these BLUE surfaces as inputs:
- `DOLPHIN_STATE_BLUE.latest_nautilus`
- fallback `DOLPHIN_STATE_BLUE.engine_snapshot`
- `DOLPHIN_FEATURES.esof_latest`
- fallback `DOLPHIN_FEATURES.esof_advisor_latest`
- `DOLPHIN_FEATURES.mc_forewarner_latest`
- `DOLPHIN_FEATURES.latest_eigen_scan`
These are read-only input sites. VIOLET should not mutate BLUE state maps.
### 5.2 VIOLET scan input
The active scan input for VIOLET is:
- Hazelcast map `DOLPHIN_FEATURES`
- key `latest_eigen_scan`
That is the scan source the live runner consumes. The older scan bridge path is superseded in this layout.
### 5.3 VIOLET persistence inputs
VIOLET writes into:
- `dolphin_violet.policy_events`
- `dolphin_violet.trade_reconstruction`
- `dolphin_violet.trade_exit_legs`
- `dolphin_violet.position_state`
- `dolphin_violet.anomaly_events`
- `dolphin_violet.account_events`
- `dolphin_violet.status_snapshots`
- `dolphin_violet.trade_events`
- `dolphin_violet.v7_decision_events`
- `dolphin_violet.violet_feed_divergence`
- `dolphin_violet.violet_decisions`
These are VIOLET-only tables. They are not BLUE tables and should not be used to backfill BLUE state directly.
## 6. ASEx involvement
ASEx exists in the tree, but the current VIOLET mainline use is narrow.
Confirmed current VIOLET codepath:
- `prod/clean_arch/violet/v4_execution_runner.py` optionally imports:
- `asex.guarded.ASExGuardedState`
- `asex.worker.ASExWorker`
- if ASEx is unavailable, the runner falls back to a local guarded state implementation
- the ASEx wrapper is used for guarded serial state around scan/order lifecycle counters
What this means:
- ASEx is not the execution engine
- ASEx is not the DITAv2 kernel
- ASEx is not the default VIOLET decision path
- it is an optional serial-state wrapper in the live runner boundary
Cross-check from the integration status doc:
- ASEx is installed as an editable package
- VIOLET main has zero ASEx imports outside the live runner boundary
- DITAv2 main has zero ASEx imports
- the earlier PASS9 adapter work exists in another worktree and is not merged here
Operational conclusion:
- ASEx is currently peripheral, not foundational, in this checkout
- the only current VIOLET runtime touchpoint is the guarded state wrapper inside `v4_execution_runner.py`
## 7. VIOLET-only support systems
### 7.1 Read-only dark guard
`prod/clean_arch/violet/observe_guard.py` implements `ObserveOnlyVenue`.
Purpose:
- wraps a venue
- raises on order placement
- allows the launcher to run a full decision stack without touching a live venue
### 7.2 Shadow journal
`prod/clean_arch/violet/shadow_journal.py`
Purpose:
- persists executed shadow decisions to `dolphin_violet.violet_decisions`
- keeps the shadow path auditable
- matches the ClickHouse DDL in `prod/clickhouse/violet/22_violet_decisions.sql`
### 7.3 Feed divergence monitor
`prod/clean_arch/violet/divergence.py`
Purpose:
- records feed divergence into `dolphin_violet.violet_feed_divergence`
- gives VIOLET its own drift surface separate from BLUE
### 7.4 V4 readiness and arming
`prod/clean_arch/violet/v4_readiness.py`
`prod/clean_arch/violet/v4_arming.py`
Purpose:
- combine soak readiness, credential checks, namespace isolation, and launcher mode checks
- fail closed when the readiness report is not GO
- keep V4 from arming unless the environment says it may
Current observed behavior in this checkout:
- arming is not GO
- the live runner therefore remains stopped
### 7.5 Soak runner and parity harness
`prod/clean_arch/violet/pass2_8_soak_runner.py`
`prod/clean_arch/violet/layer_parity_harness.py`
Purpose:
- run the DARK -> MOCK -> VST canary progression used in VIOLET soak work
- compare VIOLET behavior against the expected layer parity surface
- keep execution preparation separate from the live V4 runner
## 8. Storage and schema
`prod/clickhouse/violet/apply_violet_ddl.py` is the schema applier for VIOLET.
Important characteristics:
- it only targets `dolphin_violet`
- it probes table existence before use
- it does not create BLUE tables
- it applies each SQL file as a separate statement path
Relevant SQL set:
- `00_create_database.sql`
- `01_policy_events.sql`
- `02_trade_reconstruction.sql`
- `03_trade_exit_legs.sql`
- `04_position_state.sql`
- `05_anomaly_events.sql`
- `06_account_events.sql`
- `07_status_snapshots.sql`
- `08_trade_events.sql`
- `09_v7_decision_events.sql`
- `10_adaptive_exit_shadow.sql`
- `11_fee_settled_events.sql`
- `12_sc_bucket_gauge_shadow.sql`
- `13_sc_threshold_advisor_shadow.sql`
- `20_violet_feed_divergence.sql`
- `22_violet_decisions.sql`
The important storage boundary is simple:
- VIOLET uses `dolphin_violet.*`
- BLUE uses `dolphin.*` and `DOLPHIN_*` control/state maps
- VIOLET should not write into BLUE storage
## 9. BLUE-equivalent layer mapping, expanded
### 9.1 Data / scan layer
BLUE uses `DOLPHIN_FEATURES.latest_eigen_scan` as the canonical scan source.
VIOLET uses the same scan feed as its input, but via the live runner and shadow source:
- `prod/clean_arch/violet/v4_execution_runner.py`
- `prod/clean_arch/violet/shadow_live_factors.py`
- `prod/clean_arch/violet/live_blue_source.py`
### 9.2 Signal / conviction layer
BLUE has a large signal layer family in the system bible.
VIOLET's equivalent is:
- `decision_engine.py`
- `sizing.py`
- `alpha_wrappers.py`
- `shadow_live_factors.py`
- `live_factor_source.py`
This is where the BLUE-like live factors are folded into VIOLET conviction and sizing.
### 9.3 Execution / accounting layer
BLUE's execution/accounting boundary is the DITA family in the bible.
VIOLET's equivalent is:
- `prod/clean_arch/dita_v2/launcher.py`
- `prod/clean_arch/dita_v2/rust_backend.py`
- `prod/clean_arch/dita_v2/bingx_venue.py`
- `prod/clean_arch/dita_v2/mock_venue.py`
- `prod/clean_arch/violet/v4_execution_runner.py`
This is the actual order-facing path when V4 is armed.
### 9.4 Observability / divergence
BLUE's observability surfaces are mirrored in VIOLET by:
- `shadow_journal.py`
- `divergence.py`
- `v4_readiness.py`
- `v4_arming.py`
- `parity_report.py`
These keep the VIOLET branch auditable without mutating BLUE.
## 10. Provenance appendix
Worktree state here means the current checkout state observed during this assessment.
| File group | Worktree status | Last-known committer | Responsible actor |
|---|---|---|---|
| `prod/launch_dolphin_violet.py` | modified | Codex, `fb344318aa849fde13b934bdba7337819cc42743` | current local edit in this session |
| `prod/clickhouse/violet/apply_violet_ddl.py` | modified | Codex, `99e529c32ab353027fd8e53e5219d5c949ec06ce` | current local edit in this session |
| `prod/supervisor/dolphin-supervisord.conf` | modified | Codex, `02ecc55c1670019682b193a239c72ab0f66ab1fc` | current local edit in this session |
| `prod/clean_arch/violet/v4_execution_runner.py` | untracked | new file in worktree | Codex |
| `prod/clean_arch/violet/v4_arming.py` | untracked | new file in worktree | Codex |
| `prod/launch_dolphin_violet_v4.py` | untracked in this checkout view | new file in worktree | Codex |
| `prod/launch_dita_v2.py` | untracked in this checkout view | new file in worktree | Codex |
| `prod/supervisor/run_with_dolphin_env.sh` | untracked in this checkout view | new file in worktree | Codex |
| `prod/clean_arch/dita_v2/launcher.py` | clean in this worktree during this assessment | not revalidated here | upstream DITAv2 owner / baseline |
| `prod/clean_arch/dita_v2/rust_backend.py` | clean in this worktree during this assessment | not revalidated here | upstream DITAv2 owner / baseline |
| `prod/clean_arch/dita_v2/bingx_venue.py` | clean in this worktree during this assessment | not revalidated here | upstream DITAv2 owner / baseline |
| `prod/clean_arch/dita_v2/mock_venue.py` | clean in this worktree during this assessment | not revalidated here | upstream DITAv2 owner / baseline |
| `prod/clean_arch/violet/shadow_live_factors.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/live_blue_source.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/live_factor_source.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/shadow_journal.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/divergence.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/observe_guard.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/clean_arch/violet/v4_readiness.py` | clean in this worktree during this assessment | not revalidated here | upstream VIOLET owner / baseline |
| `prod/docs/SYSTEM_BIBLE_v7.md` | clean in this worktree during this assessment | not revalidated here | doctrine baseline |
| `prod/docs/DITA_V2_KERNEL_REFERENCE.md` | clean in this worktree during this assessment | not revalidated here | doctrine baseline |
| `prod/docs/VIOLET_DEV_SPEC_AND_PLAN*.md` | clean in this worktree during this assessment | not revalidated here | doctrine baseline |
Notes:
- for files modified or added in this session, the responsible actor is the current local worktree edit path
- for clean baseline files, I did not re-resolve the exact last commit for every single file in this pass
- the assessment is still source-faithful because the runtime path is anchored to the specific files above and the current worktree status is explicit
## 11. Bottom line
VIOLET in this checkout is a BLUE-faithful, VIOLET-namespaced stack with three distinct surfaces:
- observe-only VIOLET, which mirrors the full decision plane but blocks venue writes
- shadow / readiness / divergence, which audit and gate behavior
- live V4 execution, which feeds scan payloads into DITAv2 only after arming passes
The decisive boundary is the handoff from `ExecIntent` into `KernelIntent` and then `bundle.kernel.process_intent_async(...)`.
That is where VIOLET stops being analysis and becomes actual execution through DITAv2.

View File

@@ -0,0 +1,87 @@
# VIOLET — OB feed sourcing, HZ Bridge, and multi-agent coordination
Date: 2026-06-16. Operator-driven decisions captured during the V3.4c/d review.
## 1. OB feed sourcing doctrine
**VIOLET consumes BLUE's EXTANT published OBF feed — it does NOT store OB data.**
- `live_blue_source._source_ob_market` and `shadow_live_factors._default_ob_engine_factory`
use BLUE's OWN `HZOBProvider` (`nautilus_dolphin/.../hz_ob_provider.py`) + `OBFeatureEngine`,
wired exactly as BLUE's `_wire_obf` (nautilus_event_trader.py:4967-4980).
- `HZOBProvider` is a **read-only Hazelcast entry-listener cache**: it subscribes to the
`asset_*_ob` shards BLUE already publishes and keeps only the latest snapshot per asset in
memory. Verified: **no writes, no persistence, no new exchange WS**. OB raw data is the
heaviest disk load in the system; VIOLET adds ZERO to it by reading the published shards.
- `OBFeatureEngine` keeps an in-memory lookback window only (no disk). The launcher holds
ONE engine for the service lifetime (`shadow["ob_engine"]`), stepping a per-scan `bar_idx`,
so OB accumulation matches BLUE across scans (a fresh per-scan engine has no history).
**Swap seam for future multi-exchange OB.** The `OBProvider` behind `OBFeatureEngine` is the
single swap point. To run a SEPARATE OB stream (e.g. BingX testnet / a third-party venue,
which has genuinely different OB than the Binance reference), replace `HZOBProvider` with a
direct-WS provider feeding the SAME `OBFeatureEngine` — no other code changes. This is
specced/reasonable and the design already accommodates it.
**Cadence compliance (VIOLET's whole point — faster-than-OBF, event-driven).** Two regimes:
- NOW (shadow / parity): match BLUE, which samples OB at the OBF cadence (~1s native). You
cannot be *more* faithful than BLUE's own OB sampling, so reading the published shards is
correct for parity. `HZOBProvider` is itself a PUSH entry-listener (not polling), which is
already event-driven and aligned with the reactor model.
- LATER (V5/V6 sub-second): when VIOLET wants OB faster than BLUE's OBF, it swaps in a direct
WS OB provider (the seam above) on the reactor clock. That is a DELIBERATE VIOLET feature /
divergence, gated separately — not a parity break of the current shadow stage.
## 2. HZ Bridge — TODO_HZBRIDGE (refactor ASAP when it ships)
An upcoming **Hazelcast Bridge** (`dolphinng5_predict/hzbridge`) will be the sanctioned way to
connect to Hazelcast, mitigating the silent client-death / lockup / dropout class (see the
`hz_client_death_investigation` memory + black-box dump work in BLUE). **All VIOLET raw
`HazelcastClient` / `HZOBProvider` connections must route through the bridge once available.**
In-code `TODO_HZBRIDGE` markers flag the three touch points:
1. `shadow_live_factors.build_shadow_live_source` — the launcher's `client_factory`.
2. `shadow_live_factors._default_ob_engine_factory``HZOBProvider`'s own connection.
3. `live_blue_source` HZ_CLUSTER/HZ_HOST + `_source_ob_market` provider construction; the
live-HZ smoke test.
Refactor priority: ASAP after the bridge lands (or accelerate the bridge). Until then, VIOLET
uses raw clients, accepting the known fragility (it is DARK, so a dropout loses shadow rows,
never orders).
## 3. Multi-agent coordination (worktrees + doctrine/status)
Agents on this box: **Claude, CommandCode, Codex, Crush.** The 2026-06-16 incident — one
agent's staged files swept into another's commit (a doc landed in the forbidden `dita_v2/`) —
was caused by **multiple agents sharing ONE working tree + ONE `.git/index`**. `index.lock`
serializes plumbing ops; it does NOT isolate logical work, and `git commit` commits the WHOLE
index regardless of which files you `git add`. Careful add does not protect you — only
isolation does.
**Target model (industry standard):**
- **One `git worktree` per agent** over the shared object DB: each gets its own working dir +
index + HEAD, so collisions are impossible.
```
git worktree add ../vp-claude -b agent/claude
git worktree add ../vp-commandcode -b agent/commandcode
git worktree add ../vp-codex -b agent/codex
git worktree add ../vp-crush -b agent/crush
```
Each agent works ONLY in its own tree. (This is what the Claude Code harness's
`isolation: "worktree"` already does for subagents.)
- **Branch-per-agent → integrate via Gitea PRs.** Push agent branches to the Gitea remote
(`hjnormey/siloqy`); merge to a canonical branch via review. `main`/`release` protected.
- **Doctrine / release = a protected canonical branch + tags.** The LIVE working tree (the one
BLUE/PINK actually run from) tracks the canonical branch only; tag doctrinal snapshots
(`git tag release-YYYYMMDD`). Anything not on the canonical branch is WIP by definition —
that answers "which files are doctrinal vs in-work".
- **Agent work-status board = Gitea branch + PR list.** Each open PR / agent branch (with its
ahead/behind + last-commit author) IS the status dashboard. Optionally a top-level
`AGENT_WORKLOG.md` or the existing `.beads/` tracker for human-readable status.
**Critical secondary point:** this working tree is ALSO the live deployment path (BLUE runs
`prod/nautilus_event_trader.py` from here). Agents editing it directly means WIP code sits in
the live path — a stray restart could load half-finished edits. Worktrees fix this too:
agents edit isolated trees; deployment becomes an explicit checkout/merge of the canonical
branch onto the live tree.
Setup is operator-gated (it reorganizes how all four agents work + touches the live tree), so
it is documented here for greenlight rather than executed unilaterally.

View File

@@ -0,0 +1,149 @@
# VIOLET — partial spec for another agent (OA TODO)
Date: 2026-06-16. Carved from the VIOLET dev plan: the pending items that are
**self-contained** — a separate agent can complete each WITHOUT touching another
agent's in-flight files, with clear tests + pass criteria.
---
## 0. HARD RULES (apply to every task below — non-negotiable)
1. **Never edit shared files.** Forbidden to modify: `prod/nautilus_event_trader.py`,
`prod/clean_arch/dita_v2/**`, `prod/clean_arch/dita/decision.py`, `nautilus_dolphin/**`,
`prod/clean_arch/dita_v2/blue_parity.py`, `prod/bingx/leverage.py`. You may READ them.
2. **VIOLET stays DARK** — no orders, no live execution, no VST keys. No starting/stopping
services. No Hazelcast restart. No PROGREEN.
3. **V-TYPES on all new code**`StrictModel` / `Annotated[... Field]` / `@typed` (beartype)
per `prod/clean_arch/violet/domain.py`. No arbitrary magnitude caps; only faithful
poison-guards (finite / non-negative where BLUE guarantees it).
4. **Each task below is NEW-FILE-ONLY** by design — do NOT modify these in-flight files
(another agent owns them right now): `live_blue_source.py`, `shadow_live_factors.py`,
`live_factor_source.py`, `decision_engine.py`, `sizing.py`. Add new modules/tests instead.
## 0a. COMMIT / BRANCH POLICY (read this — three shared-index collisions happened on 2026-06-16)
Multiple agents share ONE working tree + ONE `.git/index` on this box. `git commit` commits
the ENTIRE index, so a concurrent agent's staged files get swept into your commit (one landed
in forbidden `dita_v2/`). Therefore:
1. **Work in your own `git worktree`** if at all possible:
`git worktree add ../vp-oa -b agent/oa-violet` and do ALL work there. This makes collisions
impossible. (See `prod/docs/VIOLET_OB_FEED_AND_AGENT_COORDINATION.md` §3.)
2. **If you must share the working tree**, NEVER `git add -A` / `git add .`. Stage your exact
files, and commit with an explicit pathspec so only your paths are committed:
`git commit -F msg.txt -- path/to/your_new_file.py path/to/your_test.py`
Verify after every commit: `git show --stat --format="" HEAD` must list ONLY your files.
3. **One commit per task**, message prefix `VIOLET OA:`; end with the Co-Authored-By trailer.
Before reporting done: `git diff --name-only HEAD~1` ∌ any forbidden shared path.
4. **Run tests on the prod interpreter**: `/home/dolphin/siloqy_env/bin/python3 -m pytest -q`.
The mount is CIFS-slow (a 30-test file ≈ 60-150s); that is normal, not a hang. Use
`git grep` (not recursive `grep -r`/`find`, which time out at ~2min).
---
## TASK 1 — Parity-pin tests for the hand-replicated sizing arithmetic
**Why.** `prod/clean_arch/violet/sizing.py` hand-transcribes BLUE arithmetic from
`esf_alpha_orchestrator.py` (regime_size_mult :898-909, market_ob_mult :587-595,
strength_cubic :872-885, the 5-factor compose :600-619). The `@gate` Monte-Carlo proves the
COMPOSED leverage bit-identical, but there is no PER-FORMULA pin that fails loudly if BLUE
changes one factor's formula. This task adds those pins. See
`prod/docs/VIOLET_BLUE_PARITY_STRUCTURAL_DIVERGENCE.md` (mitigation #1).
**Affected files (NEW only):**
- `prod/clean_arch/violet/test_violet_sizing_parity_pin.py` (new)
**Approach.**
- Import BLUE's real `NDAlphaEngine` from `nautilus_dolphin.nautilus.esf_alpha_orchestrator`
(READ-only use; construct a minimal instance with default ENGINE_KWARGS-equivalent params).
- For a sampled grid of inputs (vel_div ∈ [-0.06, 0], boost ∈ [1,3], beta ∈ {0.2,0.8},
mc_scale ∈ {0.5,1.0}, ob (median_imbalance, agreement_pct) over a grid, dc_status ∈
{NONE,CONFIRM}, posture ∈ {APEX,STALKER}), drive BLUE's engine to compute each intermediate
(`_day_*` state → `_update_regime_size_mult`; the OB block; `_strength_cubic`) and assert
VIOLET's `VioletSizer.regime_size_mult / market_ob_mult / strength_cubic / dc_lev_mult`
return the EXACT same float (`==`, not approx — these are deterministic).
- If a BLUE method is not callable in isolation (needs engine state), set the minimal `_day_*`
attributes directly and call the method; document any state you had to set.
**Tests / pass criteria.**
- New test file: every parametrized case asserts exact equality VIOLET-replica == BLUE-method.
- `pytest -q prod/clean_arch/violet/test_violet_sizing_parity_pin.py` → all pass.
- The existing `@gate` composition test still passes (don't change sizing.py).
- DONE when: ≥ 200 grid points per formula, zero mismatches, no edits outside the new test file.
---
## TASK 2 — Multi-exchange OB provider seam (scaffold + interface, NOT wired live)
**Why.** VIOLET currently reads BLUE's extant Binance-reference OB via `HZOBProvider`. The spec
requires being able to run a SEPARATE OB stream later (BingX testnet / 3rd-party venues have
genuinely different order books). The `OBProvider` behind `OBFeatureEngine` is the swap seam.
This task defines + tests that seam as a new module, WITHOUT wiring it into the live path
(wiring is owned by the in-flight `live_blue_source` / `shadow_live_factors`).
**Affected files (NEW only):**
- `prod/clean_arch/violet/venue_ob_provider.py` (new)
- `prod/clean_arch/violet/test_violet_venue_ob_provider.py` (new)
- `prod/docs/VIOLET_SPEC__MULTI_EXCHANGE_OB_SEAM.md` (new short design note)
**Approach.**
- Read `nautilus_dolphin/nautilus_dolphin/nautilus/ob_provider.py` (the `OBProvider` ABC +
`OBSnapshot`) and `hz_ob_provider.py` (BLUE's reference impl) to learn the exact interface
(`get_snapshot`, `get_assets`, `get_all_timestamps`, `get_snapshot_count`, snapshot fields:
bid/ask notional+depth arrays of length 5, timestamp, asset).
- Define `VioletVenueOBProvider(OBProvider)` — a venue-agnostic provider that takes a normalized
tick source (NOT a live WS yet; accept an injected callable / in-memory buffer). Produce
`OBSnapshot`s with the SAME shape BLUE expects so it drops into `OBFeatureEngine` unchanged.
- Include a `MockTickVenueOBProvider` for tests (deterministic snapshots). Do NOT open any real
exchange connection. V-TYPES on the normalized tick (5-level arrays non-negative, finite).
- Design note documents: the seam, how a future BingX WS adapter plugs in, and that wiring into
`_source_ob_market` is deferred to the owner of `live_blue_source.py`.
**Tests / pass criteria.**
- `VioletVenueOBProvider` conforms to `OBProvider` (all abstract methods implemented; an
`OBFeatureEngine(provider)` can `step_live` + `get_market` over mock snapshots without error).
- Poison rejection: negative/NaN depths or wrong-length arrays are rejected at construction.
- `pytest -q prod/clean_arch/violet/test_violet_venue_ob_provider.py` → all pass.
- DONE when: the provider drives a real `OBFeatureEngine` to a finite `get_market` result in a
test, no live connections, no edits outside the 3 new files.
---
## TASK 3 — Base-fraction sizing study (analysis + report; no production code change)
**Why.** The base sizing fraction (0.20) is a champion constant; a study was specced but not
run (`prod/docs/VIOLET_STUDY_SPEC__BASE_FRACTION_SIZING.md` — READ it first; it is the
authority for scope/method). This is a pure analysis task producing a report — zero behavior
change — so it is fully parallelizable.
**Affected files (NEW only):**
- `prod/VIOLET_dev/studies/base_fraction_study.py` (new analysis script)
- `prod/VIOLET_dev/reports/base_fraction_study_<UTC>.md` or `.json` (new report output)
**Approach.**
- Follow the existing study spec exactly. Use recorded data only (CH `dolphin_violet` /
`dolphin` read-only via `http://localhost:8123`, user `dolphin` / key `dolphin_ch_2026`).
Do NOT write to any production table. Do NOT change `sizing.py` / `decision_engine.py`.
- Compute the requested sensitivity (PnL / drawdown / capital-utilization vs base_fraction over
the spec's grid), honoring the leverage caps (base_max=8, abs_max=9) and the margin-study
findings (notional = 0.20 × conviction × capital today).
- Output a report with the recommended base_fraction + evidence; flag any caveats. Recommend,
do NOT apply.
**Tests / pass criteria.**
- The script runs end-to-end on the prod host and writes a report to `prod/VIOLET_dev/reports/`.
- A small `pytest` (or `--self-test` mode) validates the core computation on a synthetic fixture
(deterministic input → known output), so the math is checkable without live data.
- DONE when: report archived + self-test passes + no production table writes + no code-behavior
change.
---
## What is intentionally NOT in this spec (do not start these)
- **DARK soak start** — HELD for the operator's explicit word.
- **V4 live execution** — blocked on VST keys; operator-gated.
- **HZ Bridge refactor** (`TODO_HZBRIDGE`) — depends on `dolphinng5_predict/hzbridge` shipping
(owned by another agent); not self-contained yet.
- **Edits to `live_blue_source.py` / `shadow_live_factors.py` / `sizing.py` / `decision_engine.py`**
— in-flight; coordinate before touching.

View File

@@ -0,0 +1,175 @@
# VIOLET — partial spec for another agent, PASS 2 (OA TODO)
Date: 2026-06-16. Follows `VIOLET_PART_SPEC_OA_TODO.md` (PASS 1, reported done — pending
review). PASS 2 tasks **advance the main VIOLET global spec** (`VIOLET_DEV_SPEC_AND_PLAN.md`,
the V0→V6 ladder) toward the DARK soak gate and V4 execution, while staying self-contained.
---
## 0. HARD RULES (identical to PASS 1 — re-read; non-negotiable)
1. **Never edit shared files**: `prod/nautilus_event_trader.py`, `prod/clean_arch/dita_v2/**`,
`prod/clean_arch/dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`,
`prod/bingx/leverage.py`. READ only.
2. **VIOLET stays DARK** — no orders, no execution, no VST keys, no service start/stop, no HZ
restart, no PROGREEN.
3. **V-TYPES on all new code** (`StrictModel` / `Annotated[…Field]` / `@typed`); no arbitrary
caps, only faithful poison-guards.
4. **NEW-FILE-ONLY** — do NOT modify these in-flight files (owned by another agent right now):
`live_blue_source.py`, `shadow_live_factors.py`, `live_factor_source.py`, `live_factors.py`,
`decision_engine.py`, `sizing.py`, `shadow_journal.py`, `prod/clickhouse/violet/22_violet_decisions.sql`.
You may READ + IMPORT them. Wiring into them is deferred to their owner.
## 0a. COMMIT / BRANCH POLICY (CRITICAL — 3 shared-index collisions happened 2026-06-16)
The working tree + `.git/index` are SHARED across agents; `git commit` commits the whole index,
so a concurrent agent's staged files get swept into your commit. THEREFORE:
- **Strongly preferred:** your own worktree — `git worktree add ../vp-oa2 -b agent/oa-violet2`.
- **Otherwise:** never `git add -A`/`git add .`; commit with explicit pathspec —
`git commit -F msg.txt -- <your_new_file> <your_test>` — and verify
`git show --stat --format="" HEAD` lists ONLY your files.
- One commit per task, prefix `VIOLET OA:`, Co-Authored-By trailer. Before "done":
`git diff --name-only HEAD~1` ∌ any forbidden path.
- Tests on `/home/dolphin/siloqy_env/bin/python3 -m pytest -q`. CIFS-slow is normal. Use
`git grep`, not recursive `grep -r`/`find`.
## 0b. ClickHouse access (read-only)
`http://localhost:8123`, user `dolphin` / key `dolphin_ch_2026`. VIOLET data in db
`dolphin_violet`; BLUE data in db `dolphin`. **READ ONLY** — never write/alter production tables.
---
## TASK 4 — Live BLUE↔VIOLET decision parity report (advances V3d → DARK-soak gate)
**Spec link.** The ladder's V3d step requires proving the shadow DecisionEngine reproduces
BLUE's decisions. A SHADOW soak already journaled VIOLET decisions to
`dolphin_violet.violet_decisions`. This task builds the report that compares them to BLUE's
recorded decisions — the quantitative gate that makes a longer soak meaningful.
**Affected files (NEW only):**
- `prod/clean_arch/violet/parity_report.py` (new)
- `prod/clean_arch/violet/test_violet_parity_report.py` (new)
- output → `prod/VIOLET_dev/reports/violet_parity_<UTC>.{json,md}` (generated)
**Approach.**
- Read VIOLET decisions from `dolphin_violet.violet_decisions` (asset, side, scan_number,
conviction_leverage, notional_fraction, target_exposure, vel_div, ts) and BLUE's recorded
decisions from `dolphin.trade_events` / `dolphin.v7_decision_events` (inspect their schemas
first via `DESCRIBE`). Align by nearest scan/timestamp + asset.
- Compute: pick-match rate (same asset/side chosen), sizing deltas (VIOLET vs BLUE
conviction_leverage / notional — distribution: mean/median/p95 abs err), and a
divergence-reason breakdown (no-pick, different-asset, sizing-gap > threshold).
- Pure analysis: NO writes to any production table; emit a JSON + a human-readable MD report.
- V-TYPES the report rows; reject malformed/non-finite rows to a counter (never crash).
**Tests / pass criteria.**
- A `--self-test` / unit mode runs the comparison on a SYNTHETIC fixture (hand-built VIOLET +
BLUE rows with known overlaps) and asserts the computed pick-match / sizing-delta numbers
exactly. No live CH needed for the unit test.
- On the prod host, the script runs end-to-end against live CH and archives a report.
- DONE when: self-test passes, a real report is archived, zero production writes, no edits
outside the 2 new files.
---
## TASK 5 — Full-DecisionEngine reactor latency gate (advances V0/V3 — budget proof)
**Spec link.** V0 proved the reactor clock meets the latency budget (reaction p99 < 10ms,
jitter p99 < 25ms) with a trivial handler. V3 put the real decision brain online. This gate
proves the budget STILL holds when the reactor drives the FULL
`VioletDecisionEngine.decide(factors=SizingFactors(...))` path (the heavy 5-factor sizing),
not a stub a prerequisite for trusting the sub-second cadence claim.
**Affected files (NEW only):**
- `prod/clean_arch/violet/test_violet_v3_decision_latency_gate.py` (new, `@pytest.mark.gate`)
- output `prod/VIOLET_dev/reports/violet_v3_decision_latency_<UTC>.json` (generated)
**Approach.**
- Reuse V0's `PlaneClock` / `DeadlineScheduler` (`clock.py`) and the storm pattern from
`test_violet_v0_latency_gate.py` / `test_violet_v2_exec_gate.py` (READ them for the harness).
- Build a warmed `VioletDecisionEngine` (feed enough synthetic scans to pass the IRP lookback),
then in the storm loop call `decide(now_ns=…, scan_number=…, capital=…, vel_div=…,
factors=SizingFactors(boost=…, beta=…, mc_scale=…, esof_score=…, ob_*=…, dc_status=…,
posture=…))` each tick. Measure reaction + jitter percentiles.
- Do NOT modify `decision_engine.py`; import + drive it.
**Tests / pass criteria.**
- Gate asserts reaction p99 < 10ms and jitter p99 < 25ms (same budget as V0) over 200 cycles
with the FULL factor path; archive the percentile report.
- DONE when: the gate passes on the prod host, report archived, no edits outside the new file.
(If it FAILS, do not loosen the budget report the regression with numbers; that is a real
finding.)
---
## TASK 6 — L3 tradeability projector, standalone (advances toward V4 execution)
**Spec link.** The 3-layer doctrine: L1 pure alpha (done), L2 parity harness (done), **L3
tradeability** impose convictionexchange-leverage + maker policy on the L1 output. The
wrapper exists (`exchange_leverage.py`, bit-identity-gated). This task builds the standalone
projector that turns a `ShadowDecision` into a tradeable projection, WITHOUT wiring it into the
journal or any venue (DARK; wiring is the owner's job later).
**Affected files (NEW only):**
- `prod/clean_arch/violet/tradeability.py` (new)
- `prod/clean_arch/violet/test_violet_tradeability.py` (new)
**Approach.**
- Read `exchange_leverage.py` (`VioletExchangeLeverage`, `ExchangeLeverageDecision`) and
`decision_engine.py` (`ShadowDecision`) and the margin-study facts (notional = base_fraction ×
conviction × capital; exchange leverage = `map_internal_conviction_to_exchange_leverage`,
max 3× cubic).
- Define `TradeabilityProjection(StrictModel)` with: internal_conviction, exchange_leverage
(int, ge=1), target_notional, est_margin (= notional / exchange_leverage), maker_policy hint
(string, e.g. "maker_both" read existing ExecutionRouter conventions, do NOT import PINK
exec). Define `project_tradeability(decision: ShadowDecision, *, capital: float) ->
TradeabilityProjection` (`@typed`). NO orders, NO venue calls pure projection.
- This is L1-output L3 projection only; it must not change L1 sizing.
**Tests / pass criteria.**
- Exchange leverage equals `exchange_leverage.py`'s mapping for the decision's
conviction_leverage (bit-identical reuse that wrapper, don't reimplement).
- est_margin = target_notional / exchange_leverage; poison guards reject non-finite/negative.
- Hypothesis property test over conviction [0.5, 9]: projection finite, exchange_leverage
[1,3], margin notional.
- `pytest -q prod/clean_arch/violet/test_violet_tradeability.py` passes; no edits outside the 2
new files.
---
## TASK 7 — Deterministic replay + golden gate (advances V3d determinism criterion)
**Spec link.** V3d also requires DETERMINISM: same recorded scans same decisions twice. This
hardens soak confidence and catches accidental nondeterminism (dict ordering, RNG, time
leakage) in the decision path.
**Affected files (NEW only):**
- `prod/clean_arch/violet/test_violet_replay_determinism_gate.py` (new)
- `prod/clean_arch/violet/fixtures/replay_scans.json` (new, small synthetic scan sequence)
**Approach.**
- Build a fixed synthetic scan sequence (≥ 60 scans, a few assets incl. one stablecoin to prove
the exclusion gate, varied vel_div crossing the entry threshold). Store as the fixture.
- Run it through TWO fresh `VioletDecisionEngine` instances (with identical fixed
`SizingFactors`) and assert the emitted `ShadowDecision` sequences are EQUAL field-by-field.
- Add a golden assertion: the decision sequence matches a checked-in expected summary (asset,
side, conviction_leverage, scan_number per fired decision) regenerate-on-purpose only.
- Read `decision_engine.py`; do not modify it.
**Tests / pass criteria.**
- Two runs over the fixture produce identical `ShadowDecision` lists (`==`).
- The golden summary matches; stablecoin assets never selected.
- `pytest -q prod/clean_arch/violet/test_violet_replay_determinism_gate.py` passes; no edits
outside the 2 new files.
---
## Sequencing note
Tasks 47 are independent and parallelizable. Recommended priority for advancing the spec:
**Task 5 (latency budget) → Task 7 (determinism) → Task 4 (parity report) → Task 6 (tradeability)**
the first two are gates that must hold before a longer DARK soak; Task 4 makes the soak
measurable; Task 6 preps V4. None require the operator's soak/keys greenlight.
## Still NOT in scope
DARK soak START (operator-held); V4 live execution (keys-blocked); HZ-bridge refactor (depends
on `dolphinng5_predict/hzbridge` shipping); any edit to the in-flight files in §0 rule 4.

View File

@@ -0,0 +1,212 @@
# VIOLET — partial spec for another agent, PASS 3 (OA TODO)
Date: 2026-06-17. Follows PASS 1 + PASS 2 (both reported done — review pending, see
`VIOLET_TODO_CRITICAL.md`). PASS 3 is drawn STRICTLY from the existing VIOLET dev plan
(`VIOLET_DEV_SPEC_AND_PLAN.md`, the V0→V6 ladder + its named deferred items) — nothing invented.
Every task is an INDEPENDENT UNIT: it can be built, tested and function on its own, and it
composes with the others ONLY through the explicit shared interfaces in §I.
---
## 0. HARD RULES (identical to PASS 1/2 — re-read those; summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
- **VIOLET stays DARK** — no orders, no execution, no VST keys, no service start/stop, no HZ
restart, no PROGREEN.
- **V-TYPES on all new code** (`StrictModel` / `Annotated[…Field]` / `@typed`); faithful
poison-guards only.
- **NEW-FILE-ONLY**; do NOT modify in-flight files: `live_blue_source.py`,
`shadow_live_factors.py`, `live_factor_source.py`, `live_factors.py`, `decision_engine.py`,
`sizing.py`, `shadow_journal.py`, `cadence.py`, `clock.py`, `divergence.py`,
`22_violet_decisions.sql`. READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — take this seriously)
Own `git worktree` (`git worktree add ../vp-oa3 -b agent/oa-violet3`) STRONGLY preferred. Else
never `git add -A`; commit with explicit pathspec `git commit -F msg -- <files>` and verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit per task, prefix
`VIOLET OA:`, Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. Use
`git grep` (recursive grep/find time out on CIFS).
---
## I. SHARED INTERFACES (agreed contracts — define these EXACTLY; the tasks depend on them)
These are the only coupling points between PASS-3 units. Put the type definitions in ONE new
module `prod/clean_arch/violet/contracts_v3.py` (Task 8 creates it; later tasks import it).
All are `StrictModel` / `Annotated` V-TYPES.
1. **`VenueTick`** — one normalized venue quote (exchange-agnostic):
`asset: Symbol`, `bid: Px`, `ask: Px`, `mark: Px`, `last: Px`, `mono_ns: MonoNs`,
`venue: str` (e.g. "BINGX","BINANCE"). All prices > 0 finite. This is the SAME normalized
tick named in the plan's "Venue price feed port (CRITICAL gap)".
2. **`OpenPositionView`** — the minimal open-position state an exit/guard needs:
`asset: Symbol`, `side: str` ("SHORT"/"LONG"), `entry_price: Px`, `qty: Qty`,
`entry_ts_ns: MonoNs`, `bars_held: BarsHeld`, `tp_threshold: float` (fixed TP pct),
`sl_threshold: float` (stop pct), `leverage: float`.
3. **`ExitDecision`** — `action: str` ("HOLD"/"EXIT"), `reason: str`
("FIXED_TP"/"CATASTROPHIC_SL"/"ADVSL"/"NONE"/…), `price: Px`, `priority: int`
(CATASTROPHIC/ADVSL=0 > FIXED_TP=1 > DISCRETIONARY=2 — the plan's mandated order).
If a later task needs a field not here, ADD it to `contracts_v3.py` (and note it), never fork a
parallel type.
---
## TASK 8 — VenuePriceFeedPort + BingX adapter (DARK, data-only) [plan: "Venue price feed port (CRITICAL gap)"]
**Why.** The plan flags this as CRITICAL: OBF is Binance-wired (alpha side), but exits fill on
the EXECUTION venue, and FET showed scan-vs-BingX divergence (0.2176 vs 0.1878 = 15%). VIOLET
needs an exchange-agnostic price port. This unit delivers the port interface + a BingX adapter,
data-only (no orders).
**Affected files (NEW only):** `prod/clean_arch/violet/contracts_v3.py` (the §I types),
`prod/clean_arch/violet/venue_price_feed.py`, `prod/clean_arch/violet/test_violet_venue_price_feed.py`.
**Interface/approach.** Define `VenuePriceFeedPort` (ABC): `latest(asset) -> Optional[VenueTick]`,
`subscribe(assets)`, `close()`. Implement `BingxSwapPriceFeed(VenuePriceFeedPort)` consuming the
BingX swap WS bookTicker/markPrice (READ the existing BingX WS conventions in the repo; data-only,
no auth needed for public streams — confirm). Implement `MockPriceFeed` (deterministic, for
tests + for the other tasks). All ticks validated into `VenueTick` at ingress (poison-reject
non-finite/≤0).
**Tests / pass criteria.** Mock feed round-trips ticks as `VenueTick`; poison ticks rejected;
the BingX adapter parses a recorded/sample bookTicker frame into a correct `VenueTick` (use a
captured frame fixture, NOT a live connection in the unit test). DONE when: `VenueTick` contract
finalized, mock + adapter parse-tested, no live WS in unit tests, no edits outside the 3 files.
---
## TASK 9 — MechanicalExitGuard (armed TP/SL) [plan: LINK TP-miss structural fix]
**Why.** The plan's structural fix for the LINKUSDT $1,248 TP-miss: TP/SL are MECHANICAL exits
owned by an exit guard with armed price thresholds set at entry; policy layers (V7/MARAS) can
only contribute DISCRETIONARY exits and can NEVER mask mechanical ones. Total priority:
CATASTROPHIC/ADVSL > fixed TP > discretionary.
**Affected files (NEW only):** `prod/clean_arch/violet/mechanical_exit_guard.py`,
`prod/clean_arch/violet/test_violet_mechanical_exit_guard.py`. (imports `contracts_v3`.)
**Interface/approach.** `MechanicalExitGuard.evaluate(pos: OpenPositionView, tick: VenueTick) ->
ExitDecision`. Pure function of position + current venue price: compute pnl_pct from
entry/side/price; if it reaches the armed `tp_threshold``ExitDecision(EXIT, FIXED_TP,
priority=1)`; if it breaches `sl_threshold``ExitDecision(EXIT, CATASTROPHIC_SL, priority=0)`;
else HOLD. Prices off the VENUE tick (not scan). No discretionary logic here — this layer is the
mechanical floor only.
**Tests / pass criteria.** SHORT + LONG: TP fires exactly at threshold, SL at stop, neither
fires inside the band; priority ordering correct (SL outranks TP if both somehow true). Hypothesis
property: output always finite, priority ∈ {0,1,2}, EXIT only when threshold crossed. DONE when:
deterministic threshold tests + property test pass, no edits outside the 2 files.
---
## TASK 10 — Sub-second catastrophic-SL / ADVSL floor guard [plan: versioned SAFETY DEVIATION]
**Why.** The plan's one sanctioned sub-second behaviour: a catastrophic-SL/ADVSL floor evaluated
at the fastest cadence against a fast price source, to fix the scan-dark unmanaged-position hazard
(XLM/FET class). "Evaluate at fastest cadence (shadow-log would-be actions), actuate at Q."
**Affected files (NEW only):** `prod/clean_arch/violet/sl_floor_guard.py`,
`prod/clean_arch/violet/test_violet_sl_floor_guard.py`. (imports `contracts_v3`, composes Task 9.)
**Interface/approach.** `SLFloorGuard(deadline_ns)` with `on_tick(pos, tick) -> Optional[ExitDecision]`
that EVALUATES every tick (sub-second) and SHADOW-LOGS would-be SL exits, but only RETURNS an
actuation when the configured fast-SL condition holds (catastrophic threshold worse than the
mechanical stop, OR ADVSL trailing breach). Track evaluate-count vs actuate-count (shadow delta).
DARK: returns the decision; never sends orders.
**Tests / pass criteria.** A tick stream that dips intra-scan triggers the fast SL evaluation and
records the would-be exit; the actuate gate fires only on the catastrophic/ADVSL condition;
evaluate-count > actuate-count proven. DONE when tests pass, no edits outside the 2 files.
---
## TASK 11 — Event-sourced restore from trade_reconstruction [plan: VIOLET restore commitment, Option C]
**Why.** The plan commits VIOLET to NEVER restore from position_state snapshots; instead replay
the chain-tokened `trade_reconstruction` OPEN/PARTIAL_EXIT/CLOSE journal. Open = ROOT OPEN with no
terminal CLOSE; size = entry Σ legs; holding from ABSOLUTE entry_ts (kills the bars_held≈0 /
MAX_HOLD-reset class by construction); chain token verified.
**Affected files (NEW only):** `prod/clean_arch/violet/event_restore.py`,
`prod/clean_arch/violet/test_violet_event_restore.py`. (imports `contracts_v3` → emits
`OpenPositionView`s.)
**Interface/approach.** `restore_open_positions(journal_rows: list[dict]) -> list[OpenPositionView]`
— pure function over journal rows (asset, chain_root_trade_id, chain_token, leg type, qty,
realized legs, entry_ts, entry_price). Reconstruct each chain; an open position is a ROOT OPEN
with no matching terminal CLOSE; qty = entry Σ partial-exit legs; bars_held derived from absolute
entry_ts vs now (NEVER from a stored counter). Reject chain-token mismatches to a quarantine list
(returned alongside), never crash.
**Tests / pass criteria.** Fixtures: clean open, fully-closed (→ not restored), partial-exit chain
(qty correct), chain-token mismatch (→ quarantined), dead-session entry_ts (bars_held computed
from ts, never negative). Hypothesis: qty ≥ 0, bars_held ≥ 0 always. DONE when tests pass, no
edits outside the 2 files.
---
## TASK 12 — Venue lead/lag slippage metric [plan: V1 metrics — venue lead/lag]
**Why.** The plan's continuous signed SHORT-entry slippage metric: venue fill vs Binance mid at
decision time — answers "is BingX discounting our signal". Rerunnable analysis.
**Affected files (NEW only):** `prod/clean_arch/violet/slippage_metric.py`,
`prod/clean_arch/violet/test_violet_slippage_metric.py`, report →
`prod/VIOLET_dev/reports/violet_slippage_<UTC>.json`.
**Interface/approach.** `signed_entry_slippage(decision_mid: float, venue_fill: float, side: str)
-> float` (signed bps; positive = adverse) + an aggregator over recorded rows producing a
distribution (mean/median/p95). Read-only over recorded data (CH `dolphin`/`dolphin_violet`, no
writes). Pure functions; V-TYPES the rows.
**Tests / pass criteria.** Sign convention exact for SHORT and LONG on hand-built cases; aggregator
stats correct on a synthetic fixture (`--self-test`). DONE when self-test passes + a report can be
produced on the prod host, no production writes, no edits outside the files.
---
## TASK 13 — Cadence per-action Q schedule + evaluate/actuate telemetry [plan: cadence quantizer]
**Why.** The plan's cadence quantizer: each action (SL/TP/ENTRY/OBF/ExoF) gets its own
quantization Q; evaluate at fastest cadence (shadow evidence), actuate at Q; step Q down later.
`cadence.py` (the `CadenceControlPlane`) exists; this unit adds a per-action Q-SCHEDULE LOADER +
a shadow-delta TELEMETRY recorder WITHOUT modifying `cadence.py`.
**Affected files (NEW only):** `prod/clean_arch/violet/cadence_schedule.py`,
`prod/clean_arch/violet/test_violet_cadence_schedule.py`.
**Interface/approach.** Read `cadence.py` for `Action` + `CadenceControlPlane` API. Provide
`load_q_schedule(mapping) -> dict[Action, int_ns]` (validated; the plan's initial table: SL tight,
TP=scan, ENTRY=scan, OBF~1s) and `CadenceTelemetry` that records per-action evaluate-count vs
actuate-count and emits a shadow-delta summary. Compose with `CadenceControlPlane` by wrapping its
`due()` calls (do not edit it).
**Tests / pass criteria.** Schedule loader validates/rejects bad Q values; telemetry counts
evaluate > actuate under a synthetic action stream; per-action Q honored. DONE when tests pass, no
edits outside the 2 files.
---
## Composition map (how the units fit — the "well-known interfaces")
```
Task 8 contracts_v3 (VenueTick, OpenPositionView, ExitDecision) ← the shared vocabulary
├── Task 8 venue_price_feed → VenueTick stream
├── Task 9 mechanical_exit_guard(pos, tick) → ExitDecision (consumes VenueTick)
├── Task 10 sl_floor_guard(pos, tick) → ExitDecision (composes Task 9)
├── Task 11 event_restore(journal) → [OpenPositionView] (feeds 9/10)
├── Task 12 slippage_metric(mid, fill, side)→ bps (uses VenueTick mids)
└── Task 13 cadence_schedule/telemetry → per-action Q + deltas
```
Integration (wiring these into the live reactor + decision/exit path) is the OWNER's job later,
NOT part of these units. Each ships standalone + tested.
## Recommended order
**8 (contracts + feed) → 9 (mechanical exit) → 11 (restore) → 10 (SL floor) → 13 (cadence) → 12
(slippage)**. Task 8 first because `contracts_v3.py` is the shared vocabulary everything imports.
## Still NOT in scope (operator/owner only)
DARK soak start; V4 live execution / BingX ExecutionClient; HZ-bridge refactor; the V3.4c parity
root-cause (CRITICAL #1 — Claude's review job); any edit to the in-flight files in §0.

View File

@@ -0,0 +1,204 @@
# VIOLET — partial spec for another agent, PASS 4 (OA TODO)
Date: 2026-06-17. Continues PASS 3 (same V0→V6 plan, same independent-unit discipline). PASS 3
delivered the venue-feed / mechanical-exit / restore / slippage / cadence layer; PASS 4 adds the
ENTRY GATE, the EXIT KERNELS (V7 + time-based), the ACCOUNTING ledger, the EXECUTION-INTENT
boundary, and the ALPHA-SIDE data feed. Drawn strictly from `VIOLET_DEV_SPEC_AND_PLAN.md`
nothing invented. Each task is an independent, separately-testable unit; the ONLY coupling is the
shared `contracts_v3.py` vocabulary (extended in §I).
---
## 0. HARD RULES (identical to PASS 1/2/3 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
- **VIOLET DARK** — no orders, no execution, no keys, no service/HZ control, no PROGREEN.
- **V-TYPES on all new code**; faithful poison-guards only.
- **NEW-FILE-ONLY**; do NOT modify in-flight files (`live_blue_source.py`,
`shadow_live_factors.py`, `live_factor_source.py`, `live_factors.py`, `decision_engine.py`,
`sizing.py`, `shadow_journal.py`, `cadence.py`, `clock.py`, `divergence.py`,
`alpha_wrappers.py`, `exchange_leverage.py`, `22_violet_decisions.sql`). READ + IMPORT them.
- **VIBRISS stays DARK** — do NOT build adaptive-TP / VIBRISS-governable mechanisms here.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa4 -b agent/oa-violet4`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3 -m pytest -q`. `git grep`
only (recursive grep/find time out on CIFS).
## 0b. ClickHouse (read-only): `http://localhost:8123`, user `dolphin`/key `dolphin_ch_2026`;
VIOLET db `dolphin_violet`, BLUE db `dolphin`. NEVER write/alter production tables.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py` from PASS 3; never fork a parallel type)
Reuse PASS-3 `VenueTick`, `OpenPositionView`, `ExitDecision`. ADD:
1. **`ExecIntent`** (StrictModel) — a DARK would-be order: `asset: Symbol`, `side: str`,
`qty: Qty`, `exchange_leverage: Annotated[int, Field(ge=1)]`, `maker_policy: str`,
`target_notional: float`, `ts_ns: MonoNs`, `reason: str` ("ENTRY"/"EXIT"). NEVER sent.
2. **`CapitalState`** (StrictModel) — `capital: float (ge=0, finite)`, `anchor: float`,
`delta_sum: float`, `event_seq: Seq`, `pnl_source: str`, `capital_source: str`. Capital is
ALWAYS anchor + Σ deltas (never a last-value snapshot — the zombie-trade lesson).
3. **`VolGateResult`** (StrictModel) — `vol_ok: bool`, `metric: float`, `threshold: float`.
If a task needs a field not listed, ADD it to `contracts_v3.py` and note it.
---
## TASK 14 — Volatility regime gate [plan: entry precondition `vol_ok` / vol_p60_threshold]
**Why.** BLUE gates entries on a volatility regime check (`vol_ok`, `vol_p60_threshold`);
`VioletDecisionEngine.decide(... vol_ok=...)` already takes it as a param but nothing computes it
faithfully. This unit computes `vol_ok` the way BLUE does so the entry gate is real, not assumed.
**Affected files (NEW only):** `prod/clean_arch/violet/vol_gate.py`,
`prod/clean_arch/violet/test_violet_vol_gate.py`. (imports `contracts_v3`.)
**Interface/approach.** First READ how BLUE computes the vol gate (search the orchestrator / the
engine for `vol_regime_ok` / `vol_p60` / the percentile threshold; transcribe the EXACT formula).
Define `compute_vol_gate(price_histories: dict[str, list[float]], *, threshold: float) ->
VolGateResult` (`@typed`) reproducing BLUE's metric + comparison. Pure function.
**Tests / pass criteria.** Parity: hand-built histories where BLUE's formula gives a known
vol_ok; assert match. Boundary: at/just-below/just-above threshold. Hypothesis: result finite,
vol_ok bool. DONE when the formula is transcribed from live BLUE (cite file:line in a docstring),
tests pass, no edits outside the 2 files.
---
## TASK 15 — AlphaExitEngineV7 wrapper [plan: V3a deferred — "run exit-v7 as-is first, refactor later"]
**Why.** The plan wraps BLUE's live multi-leg exit engine `AlphaExitEngineV7` behind a V-TYPES
boundary and runs it AS-IS. Its outputs are DISCRETIONARY exits (priority 2) that, per the exit
doctrine, can NEVER override the mechanical TP/SL (PASS-3 Task 9).
**Affected files (NEW only):** `prod/clean_arch/violet/exit_v7_wrapper.py`,
`prod/clean_arch/violet/test_violet_exit_v7_wrapper.py`. (imports `contracts_v3`.)
**Interface/approach.** WRAP, don't reimplement: import
`nautilus_dolphin.nautilus.alpha_exit_v7_engine.AlphaExitEngineV7`; build its context
(`make_context`) and call `evaluate` over an `OpenPositionView` + market state; map its result to
an `ExitDecision(reason=…, priority=2)`. Refined domain types in/out. The wrapper must NOT change
V7's logic. Document any state V7 needs.
**Tests / pass criteria.** For crafted inputs, the wrapper's `ExitDecision` matches what
`AlphaExitEngineV7.evaluate` returns (same exit/hold + reason), priority always 2. DONE when wrap
parity tests pass, no edits outside the 2 files.
---
## TASK 16 — Capital / economics provenance ledger [plan: V1 inheritance — capital = anchor + Σ deltas]
**Why.** Binding VIOLET doctrine (from the zombie-trade incident + malformed-open audit): capital
is ALWAYS anchor + Σ signed per-trade deltas, NEVER a last-value snapshot; every row carries
`pnl_source`/`capital_source` provenance + `event_seq`; exactly-one-row-per-event. Per-trade sizer
feedback uses trade-realized PnL, never capital deltas (shared-account foreign-fill immunity).
**Affected files (NEW only):** `prod/clean_arch/violet/economics_ledger.py`,
`prod/clean_arch/violet/test_violet_economics_ledger.py`. (emits `CapitalState`.)
**Interface/approach.** `EconomicsLedger(anchor: float)` with `apply(delta: float, *, event_seq:
int, pnl_source: str) -> CapitalState` (monotonic event_seq; rejects out-of-order / duplicate
seq; capital = anchor + Σ deltas). `capital()` returns the derived value, never a stored snapshot.
Reject NaN/inf deltas. No CH writes (pure in-memory ledger + a `to_row()` for a future sink).
**Tests / pass criteria.** Σ-delta correctness over a sequence; duplicate/out-of-order event_seq
rejected; capital never goes negative below a floor guard; Hypothesis: capital == anchor + sum of
accepted deltas, always. DONE when tests pass, no edits outside the 2 files.
---
## TASK 17 — Execution-intent emitter (DARK) [plan: V4 prep — feeds the ExecDeadlineDriver]
**Why.** V2 built the `ExecDeadlineDriver` + synthetic intents; V4 will turn decisions into real
orders. This unit is the DARK boundary that converts a decision into an `ExecIntent` (a would-be
order) — shadow-logged, NEVER sent — so V4 only has to flip the sink from log to venue.
**Affected files (NEW only):** `prod/clean_arch/violet/exec_intent.py`,
`prod/clean_arch/violet/test_violet_exec_intent.py`. (imports `contracts_v3`; composes PASS-2
`tradeability.py` if present, else recompute via `exchange_leverage.py`.)
**Interface/approach.** `to_exec_intent(decision: ShadowDecision, *, capital: float, maker_policy:
str = "maker_both") -> ExecIntent` — derive qty from notional/price, exchange_leverage from the L3
mapping, attach maker policy. Pure projection; an `IntentSink` protocol with a default
`LoggingIntentSink` (shadow-logs, never sends). Hard guard: this module imports NOTHING that can
place an order; assert no venue/order symbol is importable here (a test enforces it).
**Tests / pass criteria.** Intent fields correct vs the decision + L3 mapping; `LoggingIntentSink`
records, never sends; a test asserts the module has no order-placing dependency. DONE when tests
pass, no edits outside the 2 files.
---
## TASK 18 — Alpha-side reactor data-feed adapter [plan: V2 NT Binance DATA client spike → feed]
**Why.** The V2 spike GO-qualified the Nautilus Binance DATA client as VIOLET's alpha-side feed
(separate feed process recommended; public-data only; dummy keys for the factory). This unit
delivers that feed behind a clean port so the reactor consumes normalized bars without coupling to
NT internals.
**Affected files (NEW only):** `prod/clean_arch/violet/alpha_data_feed.py`,
`prod/clean_arch/violet/test_violet_alpha_data_feed.py`.
**Interface/approach.** Define `AlphaDataFeedPort` (ABC): `start()`, `latest_bar(asset)`,
`stop()`, yielding a normalized bar type (`AlphaBar` — add to `contracts_v3`: asset, open/high/low/
close > 0, volume ≥ 0, ts_ns, source). Implement `NautilusBinanceDataFeed` (READ the V2 spike
notes / `dolphin_actor*.py` / `live_price_feed.py` for the NT data-client wiring; public-data,
dummy keys, NO execution). Implement `MockDataFeed` (deterministic). NO orders, data-only.
**Tests / pass criteria.** Mock feed yields valid `AlphaBar`s; poison bars rejected; the NT adapter
parses a recorded bar message into a correct `AlphaBar` (captured-frame fixture, no live
connection in the unit test). DONE when contract finalized + parse-tested, no live feed in unit
tests, no edits outside the 2 files.
---
## TASK 19 — MAX_HOLD + MEAN_REVERSION scan-driven exit timers [plan: scan-driven exits, V6 bible]
**Why.** The plan keeps MAX_HOLD and MEAN_REVERSION as SCAN-driven exits (champion params are
5s-bar-denominated; re-timing requires VBT re-cert). They are mechanical/time-based exits that
compose with the exit framework (PASS-3 Task 9) at the right priority.
**Affected files (NEW only):** `prod/clean_arch/violet/time_exits.py`,
`prod/clean_arch/violet/test_violet_time_exits.py`. (imports `contracts_v3`.)
**Interface/approach.** READ BLUE's exit manager for the EXACT MAX_HOLD (`max_hold_bars`) and
mean-reversion rules (`AlphaExitManager` / the orchestrator). Define
`evaluate_time_exits(pos: OpenPositionView, *, bars_held: int, vel_div: float) -> ExitDecision`
`bars_held >= max_hold_bars``ExitDecision(EXIT, "MAX_HOLD")`; mean-reversion condition →
`ExitDecision(EXIT, "MEAN_REVERSION")`; else HOLD. Scan-cadence (no sub-second). Priority below
mechanical SL/TP, above pure discretionary (document the chosen priority vs Task 9/15).
**Tests / pass criteria.** MAX_HOLD fires exactly at the bar threshold; mean-reversion fires on the
BLUE condition (transcribed, cited); neither fires early. DONE when tests pass with the rule cited
from live BLUE, no edits outside the 2 files.
---
## Composition map
```
contracts_v3 (VenueTick, OpenPositionView, ExitDecision, ExecIntent, CapitalState, VolGateResult, AlphaBar)
├── 14 vol_gate(histories) → VolGateResult (entry gate; feeds decide vol_ok)
├── 18 alpha_data_feed → AlphaBar stream (alpha-side reactor feed)
├── 15 exit_v7_wrapper(pos, state) → ExitDecision(pri 2) ┐
├── 19 time_exits(pos, bars, veldiv) → ExitDecision ├─ exit stack, ranked by priority,
│ (PASS-3 9 mechanical = pri 0/1; 10 SL floor = pri 0) ┘ mechanical always wins
├── 16 economics_ledger(anchor,Δ…) → CapitalState (accounting; never last-value)
└── 17 exec_intent(decision,capital) → ExecIntent (DARK) (V4 boundary; log-only sink)
```
Integration (ranking the exit stack, wiring the feed into the reactor, the ledger into the
journal, the intent sink into the driver) is the OWNER's job later — NOT part of these units.
## Recommended order
**14 (vol gate) → 16 (ledger) → 15 (V7 wrap) → 19 (time exits) → 17 (exec intent) → 18 (data
feed)**. 14/16 are small + foundational; 18 is the heaviest (NT adapter).
## Pass/sprint nomenclature (re-asked) — see `VIOLET_TODO_CRITICAL.md` §3
"pass" = a sub-sprint work-package (a batch of self-contained tasks for one agent); a V-stage
(V0…V6) = the project's "Sprint N" / epic. Renaming passes to "sprints" over-claims scope. The
later REVIEW + INTEGRATE + E2E of all passes is Claude's queued work.
## Still NOT in scope (operator/owner only)
DARK soak start; V4 live order placement / BingX ExecutionClient; HZ-bridge refactor; VIBRISS /
adaptive-TP; the V3.4c parity root-cause (CRITICAL #1); any edit to the in-flight files in §0.

View File

@@ -0,0 +1,192 @@
# VIOLET — partial spec for another agent, PASS 5 (OA TODO): MOCK-BINGX EXECUTION STACK
Date: 2026-06-17. Continues PASS 14 (same V0→V6 plan, same independent-unit discipline). PASS 5
builds the **mock-BingX execution stack** — the DARK path that lets the ENTIRE order lifecycle
(submit → ack → fill/partial/reject → reconcile → position/PnL) be built and bit-tested with
**zero keys, zero risk**, so that V4-live later becomes "swap the mock adapter for the real BingX
client." This is the largest remaining buildable-while-DARK chunk of the spec.
**Read first:** the V2 execution work already shipped — `prod/clean_arch/violet/scripted_venue.py`,
`exec_driver.py`, `exec_harness.py`, `observe_guard.py`, `synthetic_intents.py`,
`exchange_leverage.py`, and PINK's `prod/clean_arch/runtime/pink_direct.py` +
`prod/clean_arch/exec/` ExecutionRouter (READ for maker/taker policy conventions; do NOT edit).
PASS 5 extends, does not replace, these.
---
## ⚠️ ⚠️ READ THIS FIRST — EXCHANGE "QUIRKS" ARE EXPLICITLY OUT OF SCOPE FOR PASS 5 ⚠️ ⚠️
The mock built here implements BingX's **NORMATIVE / documented** semantics only (clean order
lifecycle, fills, leverage, reduce-only, maker/taker, partials). BingX has a substantial set of
**undocumented / edge "quirks"** that broke PINK in production. They are **NOT to be implemented
or worked around during PASS 5.** Two hard requirements instead:
1. **The mock MUST be structured with explicit INJECTION SEAMS** so each quirk can later be turned
on (a `QuirkProfile` config + hook points) to test the reconcile/filter logic against it —
WITHOUT rewriting the mock.
2. **A real-key smoke at the venue boundary remains MANDATORY before V4-live** — a mock can only
validate logic, never undocumented venue reality.
### Known BingX quirks to ACCOMMODATE-LATER (do NOT implement now; just leave seams + a registry):
Source: memory `project_pink_orphan_fixes`, `ditav2_kernel_audit_20260611`, `incident_*`.
1. **Zero-wallet-balance WS frames** — BingX transiently pushes account/position frames with
`walletBalance=0`; treating them as truth zeroes capital. (Reconcile must ignore zero-wb / use
reseed-on-update.) Seam: mock can emit a zero-wb frame on demand.
2. **Ownership / foreign-fill collision** — fills/orders for OTHER subaccounts appear on a shared
user-data stream (PRODGREEN↔PINK shared-account hazard); must be filtered by ownership. Seam:
mock can emit a fill tagged with a foreign `ownership_id`.
3. **Bound-price poison** — orders carry a "bound" price; using it for PnL instead of the actual
fill price poisons accounting. Seam: mock fill carries both bound and fill price; PnL must use
fill price (Task 23).
4. **×leverage notional** — BingX reports notional/qty with leverage applied; double-applying
leverage corrupts size. Seam: documented in the Fill→position reducer (Task 23).
5. **Settle / funding desync** — settlement & funding events arrive desynced from fills; capital
deltas must be `event_seq`-ordered (ties to PASS-4 economics ledger). Seam: mock can interleave
a settle event out of fill order.
6. **Reduce-only edge** — a reduce-only order that would INCREASE the position is rejected by
BingX. Implement this ONE as normative (it is documented), but note the partial/zero-qty edge.
7. **setLeverage race**`setLeverage` is a separate call with its own ack; a race vs the order
can apply the wrong leverage. Seam: mock `set_leverage` is a distinct, separately-ackable op.
8. **Dead `.pro` TLS WS backup** — the `.pro` WS endpoint TLS can die silently (connection-layer).
Out of mock scope (connection layer); note for the real-client adapter.
A new file `prod/clean_arch/violet/exec/bingx_quirks.py` holds ONLY a `QuirkProfile`
(StrictModel, all quirks default OFF) + an enum registry of the above — the seam contract. No
quirk LOGIC. Tasks below reference `QuirkProfile` at their hook points but implement none.
---
## 0. HARD RULES (identical to PASS 14 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
(You MAY read `prod/clean_arch/exec/**` + `pink_direct.py` for router/venue conventions; do not edit.)
- **VIOLET DARK** — the mock NEVER touches a real venue, network, or key. No service/HZ control.
- **V-TYPES on all new code**; faithful poison-guards only.
- **NEW-FILE-ONLY** under a NEW package `prod/clean_arch/violet/exec/` (plus extending
`contracts_v3.py`). Do NOT modify in-flight files (the V3.4 sourcing/engine set, `cadence.py`,
`clock.py`, `exchange_leverage.py`, `scripted_venue.py`, `exec_driver.py`, `22_violet_decisions.sql`).
READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa5 -b agent/oa-violet5`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3/4 types (`VenueTick`, `OpenPositionView`, `ExitDecision`, `ExecIntent`,
`CapitalState`). ADD the execution vocabulary (all `StrictModel` / `Annotated` V-TYPES):
1. **`OrderType`** = Literal["MAKER","TAKER"]; **`OrderSide`** = Literal["BUY","SELL"];
**`OrderStatus`** = Literal["NEW","ACK","PARTIALLY_FILLED","FILLED","CANCELED","REJECTED"].
2. **`Order`** — `client_order_id: str`, `asset: Symbol`, `side: OrderSide`, `qty: Qty`,
`price: Px` (limit; maker), `order_type: OrderType`, `reduce_only: bool`,
`leverage: Annotated[int, Field(ge=1)]`, `ts_ns: MonoNs`.
3. **`OrderAck`** — `client_order_id: str`, `venue_order_id: str`, `status: OrderStatus`,
`ts_ns: MonoNs`, `reject_reason: str = ""`.
4. **`Fill`** — `venue_order_id: str`, `asset: Symbol`, `side: OrderSide`, `fill_qty: Qty`,
`fill_price: Px`, `bound_price: Px` (quirk #3 seam — carry it, never use it for PnL),
`fee: float (ge=0)`, `is_maker: bool`, `ownership_id: str` (quirk #2 seam),
`event_seq: Seq`, `ts_ns: MonoNs`.
5. **`PositionDelta`** — `asset: Symbol`, `qty_delta: float`, `realized_pnl_delta: float`,
`fee: float (ge=0)`, `event_seq: Seq` — the unit that feeds PASS-4 `EconomicsLedger`.
If a task needs another field, ADD it here and note it.
---
## TASK 20 — Execution contracts + QuirkProfile seam registry
**Why.** The shared execution vocabulary + the quirk-seam contract everything else depends on.
**Affected files (NEW):** extend `prod/clean_arch/violet/contracts_v3.py` (the §I types);
`prod/clean_arch/violet/exec/__init__.py`; `prod/clean_arch/violet/exec/bingx_quirks.py`
(`QuirkProfile` StrictModel, all flags default False + the quirk enum registry — NO logic);
`prod/clean_arch/violet/exec/test_violet_exec_contracts.py`.
**Pass criteria.** All types construct + poison-reject (negative qty, non-finite price, etc.);
`QuirkProfile()` defaults every quirk OFF; the enum lists exactly the 8 quirks above. No logic in
`bingx_quirks.py`. No edits outside the new/extended files.
## TASK 21 — Mock BingX order FSM
**Why.** The deterministic order lifecycle state machine — the heart of the mock.
**Affected files (NEW):** `prod/clean_arch/violet/exec/order_fsm.py`,
`prod/clean_arch/violet/exec/test_violet_order_fsm.py`.
**Interface/approach.** `OrderFSM` holds per-order state; legal transitions only:
NEW→ACK→{PARTIALLY_FILLED*→FILLED | CANCELED | REJECTED}. `submit(order)->OrderAck`,
`apply_fill(venue_order_id, fill_qty, fill_price)->(OrderStatus, Fill)`, `cancel(...)`. Illegal
transitions raise/reject (never silently). Reduce-only that would increase position → REJECTED
(quirk #6, normative). Partial fills accumulate; FILLED only when cumulative == order qty.
Monotonic `event_seq`. Pure/in-memory; deterministic (no wall clock — caller passes ts_ns).
**Pass criteria.** Full-fill, multi-partial→fill, cancel-after-partial, reject (reduce-only
increase), illegal-transition rejection; Hypothesis: cumulative fill ≤ order qty always; event_seq
strictly increasing. No edits outside the 2 files.
## TASK 22 — Mock BingX venue adapter (port-conformant, quirk-seamed)
**Why.** The venue itself: a `MockBingxVenue` implementing the execution-venue port so the router
/ driver can drive it exactly like the real one — but deterministic + DARK.
**Affected files (NEW):** `prod/clean_arch/violet/exec/mock_bingx_venue.py`,
`prod/clean_arch/violet/exec/test_violet_mock_bingx_venue.py`.
**Interface/approach.** Define `ExecutionVenuePort` (ABC) if one isn't already agreed: `set_leverage(asset,
lev)->OrderAck` (quirk #7 seam — separate ackable op), `submit(order)->OrderAck`,
`cancel(client_order_id)->OrderAck`, `poll_fills()->list[Fill]`, `close()`. Implement
`MockBingxVenue(fill_model, quirks: QuirkProfile = QuirkProfile())` over `OrderFSM`. Fill model is
INJECTABLE (`ScriptedFillModel` for tests: immediate full / N-partials / reject / maker-vs-taker).
Maker orders fill at limit price; taker at a provided reference (VenueTick.ask/bid). Fee = maker vs
taker rate (read the router's rate conventions; pass rates in, don't hardcode a venue-specific
number). **Quirk hooks present but inert:** when `quirks.zero_wb`/`foreign_fill`/`settle_desync`
are False (default) behave normatively; the hook points exist for a later quirk-injection pass.
**Pass criteria.** A `ScriptedFillModel` drives submit→ack→fill→FILLED; partial sequence;
reject; maker vs taker fill price + fee differ correctly; `set_leverage` acks independently;
with default `QuirkProfile()` NO quirk behaviour occurs. No live network. No edits outside the 2 files.
## TASK 23 — Fill → position/PnL reducer (fill-price PnL, ×leverage-aware)
**Why.** Turn `Fill`s into `PositionDelta`s correctly — the accounting seam where bound-price
poison (#3) and ×leverage (#4) would bite. Feeds PASS-4 `EconomicsLedger`.
**Affected files (NEW):** `prod/clean_arch/violet/exec/fill_reducer.py`,
`prod/clean_arch/violet/exec/test_violet_fill_reducer.py`.
**Interface/approach.** `apply_fill(pos: OpenPositionView | None, fill: Fill) ->
(OpenPositionView, PositionDelta)`. **PnL uses `fill.fill_price` ONLY — never `bound_price`**
(assert/comment quirk #3). Notional/qty handled WITHOUT double-applying leverage (quirk #4
document the convention explicitly). Realized PnL on reducing fills = signed (entryfill)×qty for
SHORT, etc.; fees subtracted; `event_seq` carried through. Pure function.
**Pass criteria.** Open, add, partial-reduce, full-close produce correct qty + realized PnL +
fee; a test asserts that a fill with `bound_price != fill_price` yields PnL from `fill_price`;
Hypothesis: qty ≥ 0; realized PnL finite. No edits outside the 2 files.
## TASK 24 — Mock execution integration harness (router + driver + venue, DARK storm)
**Why.** Prove the whole stack composes: `ExecIntent` (PASS-4 Task 17) → router/maker-policy →
`ExecDeadlineDriver` (V2) → `MockBingxVenue``Fill``fill_reducer``PositionDelta`
`EconomicsLedger` (PASS-4 Task 16). The end-to-end DARK proof.
**Affected files (NEW):** `prod/clean_arch/violet/exec/mock_exec_harness.py`,
`prod/clean_arch/violet/exec/test_violet_mock_exec_scenarios.py`, gate report →
`prod/VIOLET_dev/reports/violet_mock_exec_<UTC>.json`.
**Interface/approach.** Wire the components (READ `exec_harness.py`/`exec_driver.py` for the V2
storm pattern; reuse, don't fork). Run a seeded scenario matrix: clean entry→fill→exit; partial
fills; cancel/requote on TTL; reject; maker_both policy. Assert: every order reaches a terminal
state, capital == anchor + Σ deltas (ledger), NO order ever leaves the mock (DARK guard), and the
run is deterministic (same seed ⇒ same fills/deltas twice).
**Pass criteria (`@pytest.mark.gate`).** ≥ 8 scenarios green; determinism proven; ObserveOnly/DARK
guard asserts zero real-venue calls; report archived. No edits outside the new files.
---
## Composition map
```
contracts_v3 (+Order/OrderAck/Fill/OrderStatus/PositionDelta) + exec/bingx_quirks(QuirkProfile)
20 contracts + quirk seams
21 order_fsm → legal lifecycle, partials, reject
22 mock_bingx_venue(fsm, fill_model, quirks=OFF) → OrderAck + Fill stream (quirk hooks inert)
23 fill_reducer(pos, fill) → (OpenPositionView, PositionDelta) (fill-price PnL)
24 mock_exec_harness: ExecIntent → router/driver → 22 → 23 → PASS-4 EconomicsLedger (DARK E2E)
```
Integration into the live reactor + the real BingX client swap is the OWNER's job later. The
quirk-injection pass (turning `QuirkProfile` flags ON to test reconcile/filter logic) and the
mandatory real-key boundary smoke are SEPARATE, LATER work — NOT PASS 5.
## Recommended order
**20 (contracts+quirk seams) → 21 (FSM) → 23 (fill reducer) → 22 (venue) → 24 (harness/gate)**.
## Still NOT in scope (operator/owner only, or a later pass)
- **Quirk LOGIC / reconcile-against-quirks** — later quirk-injection pass (seams only here).
- **Real BingX client + real-key smoke** — mandatory before V4-live; operator-gated.
- **DARK soak start; HZ-bridge refactor; VIBRISS; the V3.4c parity root-cause (CRITICAL #1).**
- Any edit to the in-flight files / shared files in §0.

View File

@@ -0,0 +1,190 @@
# VIOLET — partial spec for another agent, PASS 6 (OA TODO): EXECUTION INTERNALS + QUIRK INJECTION
Date: 2026-06-17. Continues PASS 5 (the mock-BingX execution stack). PASS 5 built the NORMATIVE
mock (order FSM, venue, fill reducer) with quirk SEAMS left OFF. **PASS 6 builds the execution
INTERNALS** — reconcile, fill-pump, TTL-requote, orphan handling — **and IMPLEMENTS the quirk
INJECTION + handling logic**, flipping each `QuirkProfile` flag ON and proving the reconcile/filter
logic survives it. This reproduces, for VIOLET, the hard-won PINK production fixes.
**Read first (authoritative references — READ ONLY, never edit):**
- `prod/clean_arch/runtime/pink_direct.py` — PINK's live execution runtime: the EXACT reconcile,
fill-pump, ownership-filter, zero-wb-guard, orphan, and requote logic this pass mirrors.
- `prod/clean_arch/exec/**` — the ExecutionRouter (maker/taker policy, hooks).
- PASS 5 files: `prod/clean_arch/violet/exec/{order_fsm,mock_bingx_venue,fill_reducer,bingx_quirks}.py`
+ `contracts_v3.py` — the substrate PASS 6 composes (READ + IMPORT; do NOT edit).
- Memory/incident lineage of each quirk: `project_pink_orphan_fixes`,
`ditav2_kernel_audit_20260611`, `incident_pink_spool_diskfill_20260611`.
VIOLET's ALPHA models BLUE; VIOLET's EXECUTION reconcile/quirk-handling models **PINK's
production-tested logic** (PINK is the exec-active fork). Transcribe PINK faithfully — cite
`pink_direct.py:line` in each handler's docstring.
---
## 0. HARD RULES (identical to PASS 15 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`,
`prod/clean_arch/runtime/pink_direct.py`, `prod/clean_arch/exec/**`). READ only.
- **VIOLET DARK** — everything runs against the PASS-5 mock; NO real venue/network/key, no
service/HZ control.
- **V-TYPES on all new code**; faithful poison-guards only.
- **NEW-FILE-ONLY** under `prod/clean_arch/violet/exec/` (+ extend `contracts_v3.py`). Do NOT
modify the PASS-5 files (`order_fsm.py`, `mock_bingx_venue.py`, `fill_reducer.py`,
`bingx_quirks.py`) or any in-flight V3.4 file. Quirk INJECTION must be done by NEW
fill/frame models + wrappers that plug into PASS-5's existing seams (`MockBingxVenue` takes an
injectable `fill_model` + a `QuirkProfile`), NOT by editing the mock.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa6 -b agent/oa-violet6`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3/4/5 types (`VenueTick`, `OpenPositionView`, `ExecIntent`, `CapitalState`, `Order`,
`OrderAck`, `Fill`, `OrderStatus`, `PositionDelta`). ADD the account/reconcile vocabulary (all
`StrictModel` / `Annotated`):
1. **`AccountFrame`** — venue-reported account snapshot: `wallet_balance: float (finite, ge=0)`,
`available: float (ge=0)`, `ownership_id: str`, `event_seq: Seq`, `ts_ns: MonoNs`. (Carries the
zero-wb seam: `wallet_balance` may legitimately arrive as 0.0 in a poison frame — quirk #1.)
2. **`PositionFrame`** — venue-reported position: `asset: Symbol`, `qty: float`,
`entry_price: Px`, `leverage: Annotated[int, Field(ge=1)]`, `ownership_id: str`,
`event_seq: Seq`, `ts_ns: MonoNs`.
3. **`ReconcileCorrection`** — `asset: Symbol`, `kind: str`
("DRIFT"/"ORPHAN_LOCAL"/"ORPHAN_VENUE"/"NONE"), `local_qty: float`, `venue_qty: float`,
`action: str` ("ADOPT_VENUE"/"FLATTEN"/"QUARANTINE"/"NOOP"), `event_seq: Seq`.
4. **`OwnershipPolicy`** — `account_id: str`, `client_order_id_prefix: str` — the predicate used to
decide a fill/frame is OURS vs foreign (quirk #2). Provide `owns(ownership_id) -> bool`.
If a task needs another field, ADD it here and note it.
---
## TASK 25 — Account/position frame contracts + ownership policy
**Why.** The reconcile/filter vocabulary + the ownership predicate every handler needs.
**Affected files (NEW):** extend `contracts_v3.py` (the §I types);
`prod/clean_arch/violet/exec/ownership.py` (the `OwnershipPolicy` predicate impl);
`prod/clean_arch/violet/exec/test_violet_ownership.py`.
**Interface/approach.** `OwnershipPolicy.owns(ownership_id)` returns True iff the id matches our
account / client_order_id prefix (transcribe PINK's ownership check — cite `pink_direct.py:line`).
Frames/fills with a non-owned `ownership_id` are FOREIGN (must be filtered downstream).
**Pass criteria.** `owns()` correct for own vs foreign ids; frames poison-reject (non-finite
wallet_balance rejected, but **0.0 wallet_balance is ACCEPTED into the type** — it is a legitimate
poison-frame value the reconcile layer must SEE and then ignore, not a type error). No edits
outside the new/extended files.
## TASK 26 — Fill-pump (ownership filter + dedup → ledger deltas) [quirk #2]
**Why.** Drain fills from the venue, drop foreign fills, dedupe, convert to `PositionDelta` via the
PASS-5 fill reducer, feed the PASS-4 economics ledger. PINK's fill-pump is the reference.
**Affected files (NEW):** `prod/clean_arch/violet/exec/fill_pump.py`,
`prod/clean_arch/violet/exec/test_violet_fill_pump.py`.
**Interface/approach.** `FillPump(reducer, ledger, ownership: OwnershipPolicy)` with
`pump(fills: list[Fill]) -> list[PositionDelta]`: filter `ownership.owns(f.ownership_id)` (FOREIGN
fills dropped — **ownership fill filter, quirk #2**), dedupe by `(venue_order_id, event_seq)`,
apply each via PASS-5 `fill_reducer.apply_fill`, push the `PositionDelta` into the ledger
(`event_seq`-ordered). Idempotent: re-pumping the same fills produces no double-count.
**Pass criteria.** Own fills applied; foreign fills dropped (ledger unchanged); duplicate fills
deduped; out-of-order event_seq handled per ledger rules; idempotency proven. Cite the PINK
ownership filter. No edits outside the 2 files.
## TASK 27 — Reconcile loop (zero-wb guard + reseed-on-update + settle ordering) [quirks #1, #5]
**Why.** Periodically reconcile local position/capital vs venue frames; the zero-wb guard and
reseed-on-update are the documented PINK fixes that stop a transient `walletBalance=0` frame from
zeroing capital.
**Affected files (NEW):** `prod/clean_arch/violet/exec/reconcile.py`,
`prod/clean_arch/violet/exec/test_violet_reconcile.py`.
**Interface/approach.** `Reconciler(ownership)` with
`reconcile(local: dict[Symbol, OpenPositionView], account: AccountFrame,
positions: list[PositionFrame]) -> list[ReconcileCorrection]`:
- **Ignore foreign frames** (ownership).
- **Zero-wb guard (quirk #1):** if `account.wallet_balance == 0.0` (poison frame), DO NOT adopt it
— skip the capital update, log, keep prior (cite PINK).
- **Reseed-on-update:** adopt venue position state only on a genuine update (newer `event_seq`),
not on a stale/duplicate frame.
- **Settle/funding desync (quirk #5):** process frames strictly in `event_seq` order; a settle
event arriving out of fill order is reordered, never applied ahead of its fills.
- Emit `ReconcileCorrection`s for genuine drift (ADOPT_VENUE / FLATTEN), NOOP otherwise.
**Pass criteria.** A zero-wb frame does NOT change capital; a stale frame is ignored;
out-of-order settle is reordered; genuine drift yields the right correction. Hypothesis: capital
never set to 0 by a zero-wb frame. Cite PINK lines. No edits outside the 2 files.
## TASK 28 — TTL requote / cancel-replace (+ setLeverage ordering) [quirk #7]
**Why.** Maker orders that don't fill within their TTL must be canceled and re-submitted at a new
price; `setLeverage` is a separate ackable op that must be ordered before the order it applies to.
Composes PASS-5 `MockBingxVenue` + V2 `ExecDeadlineDriver`.
**Affected files (NEW):** `prod/clean_arch/violet/exec/requote.py`,
`prod/clean_arch/violet/exec/test_violet_requote.py`.
**Interface/approach.** `RequoteController(venue, driver)` with `on_deadline(order, new_price) ->
OrderAck`: cancel the stale maker order (assert it reaches CANCELED in the FSM), submit a fresh one
at `new_price`. **setLeverage ordering (quirk #7):** if leverage must change, issue `set_leverage`
and await its ack BEFORE submitting the order (no race). Reuse the V2 driver's TTL/deadline logic
(READ `exec_driver.py`); do not fork it.
**Pass criteria.** TTL expiry → cancel + requote at new price; the old order is CANCELED, the new
one ACKed; setLeverage ack precedes the dependent order; requote count tracked. Determinism. No
edits outside the 2 files.
## TASK 29 — Orphan detection + handling [the PINK orphan fixes]
**Why.** Orders/positions can exist on the venue but not locally (or vice versa) — orphans. PINK's
orphan-reconcile is the reference; misget handling caused real incidents.
**Affected files (NEW):** `prod/clean_arch/violet/exec/orphans.py`,
`prod/clean_arch/violet/exec/test_violet_orphans.py`.
**Interface/approach.** `detect_orphans(local: dict, venue_positions: list[PositionFrame],
ownership) -> list[ReconcileCorrection]`:
- **ORPHAN_VENUE:** venue has an owned position we have no local record of → action ADOPT_VENUE
(or FLATTEN per policy; default ADOPT_VENUE with a quarantine flag).
- **ORPHAN_LOCAL:** we have a local position the venue doesn't report → action QUARANTINE (do not
silently delete; the zombie-trade lesson — never resurrect/erase by guesswork).
- Match by chain token / client_order_id where available; ambiguous → QUARANTINE.
**Pass criteria.** Each orphan class yields the correct `ReconcileCorrection`; foreign positions
ignored; ambiguous → quarantine, never silent flatten/resurrect. Cite the PINK orphan fix. No
edits outside the 2 files.
## TASK 30 — Quirk-injection suite + gate (flip every QuirkProfile flag ON) [all quirks]
**Why.** Prove the PASS-6 handlers actually survive each quirk by INJECTING it through PASS-5's
seams and asserting the outcome. This is the payoff of the seam discipline.
**Affected files (NEW):** `prod/clean_arch/violet/exec/quirk_injection.py` (the injecting
fill/frame models that plug into PASS-5's `MockBingxVenue(fill_model=…, quirks=…)` and a
`QuirkAccountStream` emitting injected `AccountFrame`/`PositionFrame`s);
`prod/clean_arch/violet/exec/test_violet_quirk_injection_gate.py`; gate report →
`prod/VIOLET_dev/reports/violet_quirk_injection_<UTC>.json`.
**Interface/approach.** For each `QuirkProfile` flag, build the injecting model and assert the
matching handler neutralizes it:
- `zero_wb=True` → Reconciler keeps capital (Task 27).
- `foreign_fill=True` → FillPump drops the foreign fill (Task 26).
- `bound_price_poison=True` → fill_reducer PnL uses fill_price not bound_price (PASS-5 Task 23, re-asserted).
- `settle_desync=True` → Reconciler reorders by event_seq (Task 27).
- `setlev_race=True` → RequoteController orders setLeverage first (Task 28).
- `reduce_only_increase=True` → OrderFSM rejects (PASS-5 Task 21, re-asserted).
- (×leverage notional, dead `.pro` TLS) → documented; ×leverage asserted in fill_reducer,
`.pro` TLS noted as connection-layer/real-client-only.
**Pass criteria (`@pytest.mark.gate`).** Every quirk flag flipped ON in at least one scenario, each
neutralized by its handler; a combined "all quirks on" storm still reaches consistent
ledger/position state; report archived. No edits outside the new files.
---
## Composition map
```
PASS-5 mock (order_fsm, mock_bingx_venue[seams], fill_reducer, QuirkProfile)
25 contracts(AccountFrame/PositionFrame/ReconcileCorrection) + ownership
26 fill_pump(ownership filter, dedup) ───► PositionDelta ──► PASS-4 EconomicsLedger
27 reconcile(zero-wb guard, reseed, settle-order) ─► ReconcileCorrection
28 requote(TTL cancel/replace, setLeverage order) ─► uses PASS-5 venue + V2 driver
29 orphans(detect) ─► ReconcileCorrection (quarantine, never silent flatten)
30 quirk_injection + GATE: flip every QuirkProfile flag ON, assert each handler neutralizes it
```
Integration (running reconcile/pump/requote on the live reactor against the REAL BingX client) is
the OWNER's job later. The **real-key boundary smoke remains MANDATORY before V4-live** — quirk
injection proves the LOGIC, never the undocumented venue reality.
## Recommended order
**25 (contracts+ownership) → 26 (fill-pump) → 27 (reconcile) → 29 (orphans) → 28 (requote) → 30
(quirk-injection gate)**.
## Still NOT in scope (operator/owner only, or a later pass)
- **Real BingX client + real-key smoke** — mandatory before V4-live; operator-gated.
- **Live reactor wiring** of reconcile/pump/requote — owner's integration job.
- **DARK soak start; HZ-bridge refactor; VIBRISS; the V3.4c parity root-cause (CRITICAL #1).**
- Any edit to PASS-5 files, in-flight V3.4 files, or shared files in §0.

View File

@@ -0,0 +1,175 @@
# VIOLET — partial spec for another agent, PASS 7 (OA TODO): V5 SELECTION + MULTI-ASSET SLOTS + CAPITAL ALLOCATION
Date: 2026-06-17. Continues PASS 16 (same V0→V6 plan). PASS 7 is the **V5** layer of the ladder:
faithful IRP/ARS asset RANKING, MULTI-ASSET SLOT management (concurrent positions), and CAPITAL
ALLOCATION across slots. Everything DARK and as independent, separately-testable units sharing the
`contracts_v3` vocabulary.
**⚠️ Relevance to CRITICAL #1 (read `VIOLET_TODO_CRITICAL.md`).** The disappointing live parity
(pick-match 1.5%, no-pick 86%, **but sizing near-identical when assets align**) points squarely at
SELECTION/TIMING, i.e. THIS layer. PASS 7 BUILDS the faithful selection machinery and pins the
ranking bit-for-bit to BLUE's `AlphaAssetSelector` (Task 36). That is COMPLEMENTARY to — not a
substitute for — the live-aggregate parity root-cause, which remains **Claude's** job (the live
join/alignment in `parity_report.py` is the other suspect). Do NOT attempt the live root-cause
here; build the units + the unit-level bit-identity pin.
**Read first (authoritative references — READ ONLY):**
- `nautilus_dolphin/nautilus_dolphin/nautilus/alpha_asset_selector.py``AlphaAssetSelector.rank_assets`
(IRP selection, ARS scoring, BIBLE §5) — the ranking authority.
- `prod/docs/SYSTEM_BIBLE.md` §5 (selection), §490 "OB Sub-1: ARS adjusted ±5%/10% by per-asset OB
depth quality before sorting", §735 OB sub-systems. (BIBLE is directionally right but can be
outdated — verify against the kernel code.)
- `prod/clean_arch/violet/alpha_wrappers.py` (`VioletAssetSelector`, the single-pick wrapper) +
`decision_engine.py` (how a pick + `ars_score` flow today) + `sizing.py` (notional model) — READ,
do NOT edit (in-flight).
- ENGINE_KWARGS selection knobs (`nautilus_event_trader.py:127+`): `use_asset_selection=True`,
`min_irp_alignment=0.0` ("gold spec: no IRP filter"), `max_slots` (launcher venue config).
- Stablecoin exclusion: `decision_engine.STABLECOIN_SYMBOLS` (must match BLUE's set).
---
## 0. HARD RULES (identical to PASS 16 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
- **VIOLET DARK** — selection/slots/allocation are pure logic over scan + factor inputs; no orders,
no venue, no service/HZ control.
- **V-TYPES on all new code**; faithful poison-guards only; NO arbitrary caps.
- **NEW-FILE-ONLY** under `prod/clean_arch/violet/selection/` (+ extend `contracts_v3.py`). Do NOT
modify in-flight files (`alpha_wrappers.py`, `decision_engine.py`, `sizing.py`,
`live_blue_source.py`, the PASS-5/6 exec files, `cadence.py`, `clock.py`). READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa7 -b agent/oa-violet7`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3..6 types. ADD the selection/slot vocabulary (all `StrictModel` / `Annotated`):
1. **`AssetRank`** — `asset: Symbol`, `ars_score: float (finite)`,
`ob_adjusted_score: float (finite)`, `irp_alignment: float`, `irp_passed: bool`, `rank: int (ge=0)`,
`excluded_reason: str = ""` (e.g. "STABLECOIN"/"IRP_FILTER"/"" ).
2. **`SlotState`** — `slot_id: int (ge=0)`, `asset: Optional[Symbol]`, `status: str`
("FREE"/"HELD"/"PENDING"), `held_since_ns: Optional[MonoNs]`.
3. **`CapitalAllocation`** — `slot_id: int`, `asset: Symbol`, `allocated_capital: float (ge=0)`,
`notional_fraction: float (ge=0)`, `conviction_leverage: float (ge=0)`.
4. **`SlotPolicy`** — `max_slots: Annotated[int, Field(ge=1)]`,
`hysteresis_bars: Annotated[int, Field(ge=0)]` (anti-churn), `allow_reentry: bool`.
If a task needs another field, ADD it here and note it.
---
## TASK 31 — Selection contracts + SlotPolicy
**Why.** The shared selection/slot vocabulary everything else imports.
**Affected files (NEW):** extend `contracts_v3.py` (the §I types);
`prod/clean_arch/violet/selection/__init__.py`;
`prod/clean_arch/violet/selection/test_violet_selection_contracts.py`.
**Pass criteria.** All types construct + poison-reject (non-finite scores, negative capital,
max_slots ≥ 1); `excluded_reason` enumerated values documented. No edits outside the new/extended files.
## TASK 32 — ARS ranking pipeline (faithful to AlphaAssetSelector + OB Sub-1 + IRP filter)
**Why.** The heart of selection: reproduce BLUE's full ranking — ARS score → OB Sub-1 adjustment →
IRP alignment filter → stablecoin exclusion → sort. The single-pick `VioletAssetSelector` covers
part of this; PASS 7 builds the FULL multi-asset ranking on top WITHOUT editing it.
**Affected files (NEW):** `prod/clean_arch/violet/selection/rank_pipeline.py`,
`prod/clean_arch/violet/selection/test_violet_rank_pipeline.py`.
**Interface/approach.** `rank_assets(market_data: dict[Symbol, list[float]], *, regime_direction:
int, ob_market=None, min_irp_alignment: float, stablecoins: frozenset[str]) -> list[AssetRank]`:
- Compute the base ARS exactly as `AlphaAssetSelector` (WRAP/transcribe — cite file:line).
- **OB Sub-1** (BIBLE §490): adjust ARS ±5%/10% by per-asset OB depth quality BEFORE sorting (read
the kernel for the exact factors/percentages; if `ob_market` is None, skip the adjustment, as the
no-OB path does).
- **IRP alignment filter:** drop assets with alignment < `min_irp_alignment` (default 0.0 = no
filter, per gold spec) → mark `irp_passed`/`excluded_reason`.
- **Stablecoin exclusion:** any asset in `stablecoins` → excluded (must equal
`decision_engine.STABLECOIN_SYMBOLS`; assert in test).
- Sort descending by `ob_adjusted_score`; assign `rank`. Pure function, deterministic, V-TYPES out.
**Pass criteria.** On crafted market data the ranked order + ARS/ob_adjusted scores match a direct
`AlphaAssetSelector` computation; stablecoins always excluded; IRP filter respected; OB-off path
equals base ARS. No edits outside the 2 files.
## TASK 33 — Multi-asset slot manager (max_slots, assignment, hysteresis)
**Why.** BLUE/VIOLET hold up to `max_slots` concurrent positions; the slot manager decides which
ranked assets occupy slots, with no double-occupancy and anti-churn hysteresis.
**Affected files (NEW):** `prod/clean_arch/violet/selection/slot_manager.py`,
`prod/clean_arch/violet/selection/test_violet_slot_manager.py`.
**Interface/approach.** `SlotManager(policy: SlotPolicy)` holding `list[SlotState]`. `assign(ranked:
list[AssetRank], held: dict[Symbol, OpenPositionView], now_ns) -> list[SlotState]`: keep currently
HELD assets in their slots; fill FREE slots from the top of `ranked` (excluding already-held and
excluded assets); never assign the same asset to two slots; respect `hysteresis_bars` (do not evict
a freshly-taken slot to chase a higher rank within the hysteresis window). READ how BLUE/the trader
manages `max_slots` (launcher venue config + engine) and transcribe the rule; if BLUE is
single-slot today (`max_slots=1`), the manager must still be correct + generalize to N.
**Pass criteria.** Held assets retained; free slots filled by rank; no double-occupancy; hysteresis
prevents churn; max_slots respected. Hypothesis: |HELD slots| ≤ max_slots always; no asset in two
slots. No edits outside the 2 files.
## TASK 34 — Capital allocation across slots
**Why.** Turn slot occupancy + conviction into per-slot capital + notional, honoring BLUE's sizing
convention (notional = base_fraction × conviction × capital) and the margin-study findings.
**Affected files (NEW):** `prod/clean_arch/violet/selection/capital_allocator.py`,
`prod/clean_arch/violet/selection/test_violet_capital_allocator.py`.
**Interface/approach.** First READ how BLUE allocates capital across concurrent slots (shared pool
vs per-slot; the margin study `blue_margin_envelope_study` + `sizing.py` are the references —
notional = 0.20 × conviction × capital; capital under-utilized at 1 slot was DELIBERATE). Define
`allocate(slots: list[SlotState], convictions: dict[Symbol, float], *, capital: float, base_fraction:
float) -> list[CapitalAllocation]` reproducing that convention EXACTLY (cite the source). Do NOT
invent a normalization BLUE doesn't do.
**Pass criteria.** Per-slot notional == base_fraction × conviction × capital (the documented model);
sum-of-notionals behavior matches BLUE's (shared-capital, not artificially normalized, unless BLUE
normalizes — verify); poison guards reject non-finite/negative. Cite the allocation source. No edits
outside the 2 files.
## TASK 35 — Selection→slot→intent multi-asset flow harness (DARK)
**Why.** Compose the V5 path end-to-end: `rank_pipeline``slot_manager``capital_allocator`
(existing sizing) → `ExecIntent` per occupied slot (PASS-4 Task 17), DARK.
**Affected files (NEW):** `prod/clean_arch/violet/selection/multi_asset_flow.py`,
`prod/clean_arch/violet/selection/test_violet_multi_asset_flow.py`.
**Interface/approach.** `run_selection_cycle(market_data, held, *, factors, capital, policy, ...) ->
list[ExecIntent]` wiring the four units. Reuse `sizing.VioletSizer` for conviction (import, don't
fork). DARK: emits intents, never orders. Deterministic.
**Pass criteria.** A multi-asset scenario yields one intent per occupied slot with correct
asset/qty/notional; held assets not re-entered (unless `allow_reentry`); max_slots respected
end-to-end; deterministic. No edits outside the 2 files.
## TASK 36 — Ranking bit-identity gate vs the real AlphaAssetSelector
**Why.** Pin the ranking to BLUE's actual `AlphaAssetSelector` over a sampled grid — the unit-level
parity that, once green, removes selection-math as a suspect for CRITICAL #1 (leaving timing/join as
the remaining live-aggregate suspect for Claude).
**Affected files (NEW):** `prod/clean_arch/violet/selection/test_violet_rank_parity_gate.py`; gate
report → `prod/VIOLET_dev/reports/violet_rank_parity_<UTC>.json`.
**Interface/approach.** Over many synthetic universes (varied asset counts, price histories,
directions, with/without OB), assert `rank_pipeline.rank_assets` produces the SAME ordering + the
SAME ARS/ob_adjusted scores (`==`) as a direct `AlphaAssetSelector` run on the same inputs. Record
mismatch count (must be 0).
**Pass criteria (`@pytest.mark.gate`).** ≥ 200 universes, zero ranking/score mismatches; report
archived. If mismatches appear, that IS a finding — report them, do not loosen. No edits outside the
new file.
---
## Composition map
```
contracts_v3 (+AssetRank/SlotState/CapitalAllocation/SlotPolicy)
32 rank_pipeline(market, ob, irp, stables) → [AssetRank] (faithful to AlphaAssetSelector + OB Sub-1)
33 slot_manager(ranked, held, policy) → [SlotState] (max_slots, hysteresis, no double-occ)
34 capital_allocator(slots, convictions, cap)→ [CapitalAllocation] (BLUE notional convention)
35 multi_asset_flow: 32→33→34→ sizing → [ExecIntent] (DARK V5 cycle)
36 rank_parity_gate: rank_pipeline == AlphaAssetSelector (bit-identity) ← narrows CRITICAL #1
```
Integration (running the V5 cycle on the live reactor, wiring intents to the PASS-5/6 exec stack) is
the OWNER's job later.
## Recommended order
**31 (contracts) → 32 (rank pipeline) → 36 (rank parity gate) → 33 (slots) → 34 (allocation) → 35
(flow harness)**. 36 right after 32 so the ranking is pinned before building on it.
## Still NOT in scope (operator/owner only, or Claude)
- **The live BLUE↔VIOLET aggregate parity root-cause (CRITICAL #1)** — Claude's job; this pass only
pins ranking-math + builds V5 units.
- **Alpha re-timing / sub-second actuation of entries** — VBT re-certification (research), not a unit.
- **DARK soak start; V4 live execution; HZ-bridge; VIBRISS.**
- Any edit to in-flight / shared files in §0.

View File

@@ -0,0 +1,179 @@
# VIOLET — partial spec for another agent, PASS 8 (OA TODO): V6 BIBLE LAYERS (POSTURE / MARAS / REGIME) + CADENCE TELEMETRY
Date: 2026-06-17. Continues PASS 17 (same V0→V6 plan). PASS 8 is the **V6** bible layer. V6 =
"full bible layers (ACB / MARAS / posture / vol) + sub-second SL guard." Of those:
- **ACB** (boost/beta) is DONE (`live_blue_source.py`, bit-identical to `get_dynamic_boost_from_hz`).
- **vol** gate is PASS-4 Task 14; **sub-second SL guard** is PASS-3 Task 10.
- **REMAINING for PASS 8: POSTURE (the 5-state effects), MARAS (fingerprint consumer), and the
REGIME read-model that composes them — plus the CADENCE shadow-actuation telemetry** the plan
requires before any Q loosening.
All DARK; independent, separately-testable units sharing the `contracts_v3` vocabulary.
**Crucial framing:** VIOLET **CONSUMES** posture/MARAS — it does NOT decide them. BLUE's
meta-health service (MHS) sets posture; VIOLET reads it from HZ (already sourced via
`live_blue_source``engine_snapshot['posture']` / `DOLPHIN_SAFETY.latest.posture`). These units
APPLY the published posture/MARAS EFFECTS faithfully; they do NOT run the MHS state machine.
**Read first (authoritative references — READ ONLY):**
- `nautilus_dolphin/nautilus_dolphin/nautilus/esf_alpha_orchestrator.py``_day_posture` handling:
`:365` HIBERNATE_HALT (force EXIT), `:613` STALKER structural ceiling, `:918-978` begin_day
posture wiring; and the `regime_dd_halt` set on `mc_red or posture in ['TURTLE','HIBERNATE']`.
- `prod/clean_arch/violet/sizing.py` — the STALKER `clamped_max=min(clamped_max, 2.0)` already in
`compose` (READ; do NOT edit; the posture *size-cap* lives there — PASS 8 owns the *entry-gate*
+ observability, not the sizing math).
- **MARAS:** grep the kernels for `maras` / `MARAS` / `maras_fingerprint` (memory references
"maras_fingerprint columns"); determine its EXACT role in the decision/sizing path. The BIBLE
may mention it (directionally right, possibly outdated — verify against code).
- `prod/clean_arch/violet/cadence.py` (`CadenceControlPlane`, `Action`) + PASS-3
`cadence_schedule.py` — READ; PASS 8 extends the telemetry, does not edit them.
- `live_blue_source.py` (posture sourcing) + `decision_engine.py` (how `posture` enters `factors`).
---
## 0. HARD RULES (identical to PASS 17 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`). READ only.
- **VIOLET DARK** — pure consumers over published posture/MARAS/regime inputs; no orders, no venue,
no service/HZ control, no PROGREEN.
- **V-TYPES on all new code**; faithful poison-guards only; NO arbitrary caps.
- **NEW-FILE-ONLY** under `prod/clean_arch/violet/regime/` (+ extend `contracts_v3.py`). Do NOT
modify in-flight files (`sizing.py`, `decision_engine.py`, `live_blue_source.py`, `cadence.py`,
`clock.py`, the PASS-5/6/7 files). READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa8 -b agent/oa-violet8`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3..7 types. ADD the V6 regime vocabulary (all `StrictModel` / `Annotated`):
1. **`Posture`** = Literal["APEX","STALKER","RESTORED","TURTLE","HIBERNATE"] (must equal
`sizing.Posture`).
2. **`PostureEffect`** — `posture: Posture`, `entry_allowed: bool`, `force_flatten: bool`,
`size_cap: Optional[float]` (e.g. 2.0 for STALKER; None = no extra cap), `regime_dd_halt: bool`.
3. **`MarasFingerprint`** — the published MARAS state (fields TBD by Task 39 after reading the
kernel; at minimum a typed wrapper with finite/typed fields + a `regime_label: str`).
4. **`RegimeView`** — read-model composing the live regime: `posture: Posture`, `boost: float`,
`beta: float`, `mc_scale: float`, `vol_ok: Optional[bool]`, `maras: Optional[MarasFingerprint]`,
`regime_dd_halt: bool`, `ts_ns: MonoNs`. Observability/divergence only — NOT a sizing path.
5. **`ActionCadenceDelta`** — `action: str`, `evaluations: int (ge=0)`, `actuations: int (ge=0)`,
`suppressed: int (ge=0)` — the shadow-delta per action.
If a task needs another field, ADD it here and note it.
---
## TASK 37 — Regime/posture contracts
**Why.** The shared V6 vocabulary.
**Affected files (NEW):** extend `contracts_v3.py` (the §I types);
`prod/clean_arch/violet/regime/__init__.py`;
`prod/clean_arch/violet/regime/test_violet_regime_contracts.py`.
**Pass criteria.** All types construct + poison-reject; `Posture` literal equals `sizing.Posture`
(assert in test); `size_cap` optional/finite. No edits outside the new/extended files.
## TASK 38 — Posture effects engine (the 5-state EFFECTS, faithful)
**Why.** Apply each posture's documented EFFECT on the decision path — the entry-gate + flatten +
size-cap — exactly as BLUE. (The STALKER size-cap of 2.0 already lives in `sizing.compose`; this
unit owns the ENTRY-GATE + FLATTEN + the typed effect, NOT the sizing math.)
**Affected files (NEW):** `prod/clean_arch/violet/regime/posture_effects.py`,
`prod/clean_arch/violet/regime/test_violet_posture_effects.py`.
**Interface/approach.** `posture_effect(posture: Posture) -> PostureEffect`, transcribed from the
orchestrator (cite file:line):
- **APEX / RESTORED:** `entry_allowed=True`, no flatten, no extra cap.
- **STALKER:** `entry_allowed=True`, `size_cap=2.0` (the `:613` structural ceiling), no flatten.
- **TURTLE:** `regime_dd_halt=True``entry_allowed=False` (halt new entries), no forced flatten of
existing (verify the exact TURTLE behaviour in the kernel).
- **HIBERNATE:** `entry_allowed=False`, `force_flatten=True` (`:365` HIBERNATE_HALT forces EXIT),
`regime_dd_halt=True`.
Verify each against the kernel; if any differs, follow the CODE (BIBLE may be outdated).
**Pass criteria.** Each posture maps to the cited effect; HIBERNATE flattens + blocks entry; TURTLE
blocks entry; STALKER caps at 2.0; APEX/RESTORED normal. A test asserts the STALKER cap equals the
2.0 used in `sizing.compose`. No edits outside the 2 files.
## TASK 39 — MARAS fingerprint consumer
**Why.** MARAS is named in V6 ("full bible layers ACB/MARAS/posture/vol"). Build the faithful
consumer of the published MARAS fingerprint.
**Affected files (NEW):** `prod/clean_arch/violet/regime/maras_consumer.py`,
`prod/clean_arch/violet/regime/test_violet_maras_consumer.py`.
**Interface/approach.** FIRST establish MARAS's real role: grep the kernels + the HZ publishers for
`maras`/`maras_fingerprint`; find (a) WHERE MARAS is published (which HZ map/key), (b) WHETHER it
modulates sizing/gating in the live path or is observability-only. Then:
- If MARAS modulates the decision: build `consume_maras(raw) -> MarasFingerprint` (wrap BLUE's
parser if one exists; else V-TYPES the published fields) + the effect it applies, transcribed +
cited.
- If MARAS is observability-only (NOT in the live decision path): build the read-only typed
accessor + DOCUMENT explicitly that it does not modulate decisions today (honest — do NOT invent
a modulation BLUE doesn't apply).
**Pass criteria.** `MarasFingerprint` parses the real published payload shape (captured-fixture
test); the consumer's role (modulating vs observability) is determined from the code and
documented with citations. No invented effect. No edits outside the 2 files.
## TASK 40 — Regime composite read-model
**Why.** A single typed `RegimeView` composing posture + ACB(boost/beta) + mc_scale + vol + MARAS
for observability and divergence — NOT a new sizing path (sizing.py owns the math).
**Affected files (NEW):** `prod/clean_arch/violet/regime/regime_view.py`,
`prod/clean_arch/violet/regime/test_violet_regime_view.py`.
**Interface/approach.** `build_regime_view(*, posture, boost, beta, mc_scale, vol_ok=None,
maras=None, now_ns) -> RegimeView` — pure assembly + `regime_dd_halt` derived from posture
(Task 38) and mc state. Read-only; emits the view for a future divergence/observability sink. MUST
NOT recompute or alter any sizing factor.
**Pass criteria.** View assembled correctly; `regime_dd_halt` matches the posture effect; no sizing
recomputation (a test asserts inputs pass through unchanged). No edits outside the 2 files.
## TASK 41 — Cadence shadow-actuation telemetry harness
**Why.** The plan's cadence doctrine: "evaluate at fastest cadence (shadow-log would-be actions),
actuate at Q; measure shadow deltas before promoting." PASS-3 Task 13 added the Q-schedule + a
telemetry recorder; THIS unit runs the evaluate/actuate loop and produces per-action
`ActionCadenceDelta`s — the shadow evidence required before stepping any Q down.
**Affected files (NEW):** `prod/clean_arch/violet/regime/cadence_telemetry_harness.py`,
`prod/clean_arch/violet/regime/test_violet_cadence_telemetry_harness.py`, report →
`prod/VIOLET_dev/reports/violet_cadence_shadow_<UTC>.json`.
**Interface/approach.** Compose PASS-3 `cadence_schedule` + `CadenceControlPlane` (READ; import,
don't edit). Drive a synthetic action stream where each action EVALUATES every tick but ACTUATES
only when its Q is due; record `evaluations` / `actuations` / `suppressed` per action and emit the
deltas. This quantifies, e.g., "how many would-be SL exits fire at fast cadence vs the scan-Q
actuations" — the promote/loosen evidence.
**Pass criteria.** For a known stream, evaluations > actuations per fast action; per-action Q
honored; deltas match a hand-computed expectation; report archived. No edits outside the new files.
## TASK 42 — Posture/regime parity gate vs BLUE
**Why.** Pin the posture EFFECTS (and MARAS, if modulating) to BLUE's actual gating over a replay —
bit-identity where applicable.
**Affected files (NEW):** `prod/clean_arch/violet/regime/test_violet_regime_parity_gate.py`; report
`prod/VIOLET_dev/reports/violet_regime_parity_<UTC>.json`.
**Interface/approach.** For each posture (+ MARAS state if it modulates), assert VIOLET's
`posture_effect` / consumer reproduces BLUE's entry-allowed / flatten / cap / halt decision (drive
the orchestrator's relevant branch, or assert against the cited constants). Record mismatches
(must be 0).
**Pass criteria (`@pytest.mark.gate`).** Every posture's effect matches BLUE; HIBERNATE/TURTLE halts
reproduced; STALKER 2.0 cap matches; zero mismatches; report archived. No edits outside the new file.
---
## Composition map
```
contracts_v3 (+Posture/PostureEffect/MarasFingerprint/RegimeView/ActionCadenceDelta)
38 posture_effects(posture) → PostureEffect (entry gate / flatten / cap / dd_halt)
39 maras_consumer(raw) → MarasFingerprint (+ effect, IFF it modulates; else read-only)
40 regime_view(posture,acb,mc,vol,maras) → RegimeView (observability/divergence; NOT sizing)
41 cadence_telemetry_harness → [ActionCadenceDelta] (shadow evidence before Q loosening)
42 regime_parity_gate: effects == BLUE gating (bit-identity)
```
Integration (wiring posture-gating into the live decision path, the regime view into a divergence
sink, the cadence deltas into the Q-promotion decision) is the OWNER's job later. Posture/MARAS
remain CONSUMED, never decided (MHS owns them).
## Recommended order
**37 (contracts) → 38 (posture effects) → 42 (regime parity gate) → 39 (MARAS) → 40 (regime view) →
41 (cadence telemetry)**. 42 right after 38 to pin posture before composing.
## Still NOT in scope (operator/owner only, or Claude)
- **Running the MHS / DECIDING posture** — BLUE owns it; VIOLET consumes.
- **Q loosening / sub-second actuation of alpha exits** — needs VBT re-cert (research), not a unit.
- **The live BLUE↔VIOLET aggregate parity root-cause (CRITICAL #1)** — Claude's job.
- **DARK soak start; V4 live execution; HZ-bridge; VIBRISS.**
- Any edit to in-flight / shared files in §0.

View File

@@ -0,0 +1,173 @@
# VIOLET — partial spec for another agent, PASS 9 (OA TODO): ECONOMICS + OBSERVABILITY COMPLETENESS
Date: 2026-06-17. Final parallelizable pass (PASS 18 cover the rest of the DARK-buildable surface).
PASS 9 closes the **economics/observability** inheritance the plan binds VIOLET to, and the
**soak-readiness** machinery that decides go/no-go for a DARK soak. All DARK; independent units
sharing `contracts_v3`.
**Plan obligations this pass discharges (V1 inheritance, BINDING):**
> "VIOLET inherits and must keep: `pnl_source`/`capital_source` provenance, `event_seq` on every CH
> row, `wait_for_async_insert=1` on economics tables, exactly-one-row-per-event (the PINK
> duplicate-emission fix is open spec debt — VIOLET gates on it), DDL-before-code migration
> discipline, per-trade sizer feedback from trade-realized PnL (never capital deltas)."
Plus: capital = anchor + Σ deltas (never last-value) — the zombie-trade lesson (PASS-4 ledger).
**Read first (authoritative references — READ ONLY):**
- `prod/clean_arch/violet/shadow_journal.py` — the validate-then-sink + reject-at-source pattern
(the model for every PASS-9 sink). `prod/clean_arch/violet/domain.py` — V-TYPES (`DivergenceRow`,
EpochMs/Seq/MonoNs etc.). `prod/clean_arch/violet/divergence.py` — the V1 divergence monitor.
- `prod/clickhouse/violet/*.sql` + `apply_violet_ddl.py` — DDL-first migration discipline; the
`test_apply_violet_ddl.py` pattern; the row-set==DDL-columns parity test in
`test_violet_shadow_journal.py`.
- `prod/ch_writer.py` — READ for the CH insert path / `wait_for_async_insert` convention; **do NOT
edit it** (its design flaws are documented; VIOLET uses its own sink, mirroring the journal).
- PASS-2 `parity_report.py`, PASS-3 `slippage_metric.py`, PASS-8 cadence telemetry — the reports the
soak-readiness aggregator (Task 47) consolidates.
- CH (read-only): `http://localhost:8123`, user `dolphin`/key `dolphin_ch_2026`; VIOLET db
`dolphin_violet`. NEVER write/alter PRODUCTION tables; new VIOLET tables only, DDL-first.
---
## 0. HARD RULES (identical to PASS 18 — summarized)
- **Never edit shared files** (`prod/nautilus_event_trader.py`, `clean_arch/dita_v2/**`,
`dita/decision.py`, `nautilus_dolphin/**`, `blue_parity.py`, `prod/bingx/leverage.py`,
`prod/ch_writer.py`). READ only.
- **VIOLET DARK** — sinks write only to NEW `dolphin_violet` tables; no orders, no venue, no
service/HZ control, no PROGREEN. Never write a production/BLUE table.
- **V-TYPES on all new code**; faithful poison-guards; reject-at-source to a counter/dead_letter,
never crash the sink (the bars_held=-106 spool lesson).
- **DDL-FIRST**: every new table ships a `.sql` in `prod/clickhouse/violet/` BEFORE the sink code;
a row-set==DDL-columns parity test is MANDATORY (mirror `test_violet_shadow_journal.py`).
- **NEW-FILE-ONLY** under `prod/clean_arch/violet/obs/` (+ extend `contracts_v3.py` + new `.sql`).
Do NOT modify in-flight files (`shadow_journal.py`, `divergence.py`, `domain.py`,
`22_violet_decisions.sql`, the PASS-5..8 files). READ + IMPORT them.
## 0a. COMMIT / BRANCH POLICY (3 shared-index collisions on 2026-06-16 — non-negotiable)
Own `git worktree` (`git worktree add ../vp-oa9 -b agent/oa-violet9`) strongly preferred. Else
never `git add -A`; `git commit -F msg -- <files>` with explicit pathspec; verify
`git show --stat --format="" HEAD` lists ONLY your files. One commit/task, prefix `VIOLET OA:`,
Co-Authored-By trailer. Tests on `/home/dolphin/siloqy_env/bin/python3`. `git grep` only.
---
## I. SHARED INTERFACE EXTENSIONS (add to `contracts_v3.py`; never fork a parallel type)
Reuse PASS-3..8 types (`CapitalState`, `PositionDelta`, `RegimeView`, etc.). ADD:
1. **`Provenance`** — `pnl_source: str`, `capital_source: str`, `source: str`, `event_seq: Seq`,
`ts: EpochMs`. The provenance stamp every economics row carries.
2. **`EconomicsRow`** — a validated `dolphin_violet.violet_economics` row: `session_id: SessionId`,
`event_seq: Seq`, `asset: Symbol`, `realized_pnl: float (finite)`, `fee: float (ge=0)`,
`capital_after: float (ge=0)`, `pnl_source: str`, `capital_source: str`, `ts: EpochMs`,
`mono_ns: MonoNs`. Field set MUST equal its DDL columns (parity test).
3. **`DecisionDivergenceRow`** — extends the V1 divergence idea to the FACTOR plane: `ts: EpochMs`,
`asset: Symbol`, `factor: str`, `violet_value: float`, `blue_value: float`, `abs_err: float`,
`event_seq: Seq`.
4. **`SoakReadiness`** — `parity_pick_match: float`, `latency_p99_ms: float`,
`determinism_ok: bool`, `exactly_one_row_ok: bool`, `namespace_isolation_ok: bool`,
`verdict: str` ("GO"/"NO_GO"), `reasons: list[str]`.
If a task needs another field, ADD it here and note it.
---
## TASK 43 — Observability/economics contracts
**Why.** The shared provenance/economics/divergence vocabulary.
**Affected files (NEW):** extend `contracts_v3.py`; `prod/clean_arch/violet/obs/__init__.py`;
`prod/clean_arch/violet/obs/test_violet_obs_contracts.py`.
**Pass criteria.** All types construct + poison-reject (non-finite pnl, negative capital/fee
rejected); provenance strings non-empty where required. No edits outside the new/extended files.
## TASK 44 — DDL-first exactly-one-row-per-event economics sink [the PINK duplicate-emission fix]
**Why.** The binding inheritance: exactly-one-row-per-event, `event_seq` on every row,
`wait_for_async_insert=1`, validate-then-sink, reject-at-source. This is VIOLET paying down the
PINK duplicate-emission debt.
**Affected files (NEW):** `prod/clickhouse/violet/30_violet_economics.sql` (DDL FIRST),
`prod/clean_arch/violet/obs/economics_sink.py`, `prod/clean_arch/violet/obs/test_violet_economics_sink.py`.
**Interface/approach.** DDL: `dolphin_violet.violet_economics` (columns == `EconomicsRow` fields;
`ENGINE = ReplacingMergeTree` keyed on `(session_id, event_seq)` so a re-emit collapses to ONE row;
TTL per convention). Sink: `EconomicsSink(sink_fn, session_id)` mirrors `VioletDecisionJournal`:
validate each `EconomicsRow`, **dedupe by `(session_id, event_seq)`** (in-process guard + the
ReplacingMergeTree key as the durable guard), reject malformed to a counter (never head-of-line),
insert with `wait_for_async_insert=1` on the economics path. NO writes to any production table.
**Pass criteria.** Row-set == DDL columns (parity test, mirror the journal test); duplicate
`event_seq` → exactly ONE emitted (in-process) and ReplacingMergeTree key documented; malformed row
rejected to counter, sink never raises; provenance fields required. No edits outside the new files.
## TASK 45 — Provenance + per-trade PnL feedback (capital = anchor + Σ trade-realized)
**Why.** Capital must be anchor + Σ TRADE-REALIZED deltas (never capital snapshots / never WS
balance — the shared-account foreign-fill immunity); every economics event carries
`pnl_source`/`capital_source`.
**Affected files (NEW):** `prod/clean_arch/violet/obs/provenance.py`,
`prod/clean_arch/violet/obs/test_violet_provenance.py`. (composes PASS-4 `EconomicsLedger` +
PASS-5/6 `PositionDelta`.)
**Interface/approach.** `stamp_provenance(delta: PositionDelta, *, pnl_source: str, capital_source:
str) -> (EconomicsRow, Provenance)` — derive `realized_pnl` from the TRADE-REALIZED delta (NEVER a
capital-difference), feed the ledger (anchor + Σ), tag provenance. A guard/test asserts the feedback
path uses trade-realized PnL, not capital deltas.
**Pass criteria.** capital_after == anchor + Σ realized deltas; provenance present on every row; a
test proves capital is NOT derived from a balance snapshot; non-finite rejected. No edits outside the 2 files.
## TASK 46 — Decision/factor divergence monitor v2
**Why.** Extend the V1 divergence monitor to the FACTOR plane (boost/beta/mc_scale/esof/ob/dc/
posture VIOLET-vs-BLUE) so divergence is observable per factor, not just per feed. New module (do
NOT edit `divergence.py`).
**Affected files (NEW):** `prod/clickhouse/violet/31_violet_decision_divergence.sql` (DDL FIRST),
`prod/clean_arch/violet/obs/divergence_v2.py`, `prod/clean_arch/violet/obs/test_violet_divergence_v2.py`.
**Interface/approach.** `record_factor_divergence(violet_factors, blue_factors, *, asset, ts,
event_seq) -> list[DecisionDivergenceRow]` — per factor, abs_err; sink via a validate-then-sink to
`violet_decision_divergence`. Reuse the row-guard pattern. Read-only over inputs.
**Pass criteria.** Row-set == DDL columns; per-factor abs_err correct on a fixture; malformed
rejected. No edits to `divergence.py`. No edits outside the new files.
## TASK 47 — Soak-readiness aggregator (one dashboard from all reports)
**Why.** Consolidate the scattered reports (PASS-2 parity, PASS-3 slippage, PASS-5/6 exec gate,
PASS-7 rank parity, PASS-8 cadence/regime, the latency + determinism gates) into ONE
soak-readiness view, so go/no-go is a single artifact.
**Affected files (NEW):** `prod/clean_arch/violet/obs/soak_readiness.py`,
`prod/clean_arch/violet/obs/test_violet_soak_readiness.py`, report →
`prod/VIOLET_dev/reports/violet_soak_readiness_<UTC>.{json,md}`.
**Interface/approach.** Read the latest report file per category from `prod/VIOLET_dev/reports/`
(parse the JSONs), extract the key metric each (parity pick-match, latency p99, determinism pass,
mismatch counts, exactly-one-row, namespace isolation), assemble a `SoakReadiness` with a GO/NO_GO
verdict + the failing reasons. Pure aggregation; tolerate missing reports (mark UNKNOWN, not crash).
**Pass criteria.** On a synthetic set of report files the aggregator extracts the right metrics and
verdict (`--self-test`); a real run consolidates the actual reports; missing report → UNKNOWN not
crash. No edits outside the new files.
## TASK 48 — Soak-readiness @gate (go/no-go prerequisites)
**Why.** A single authoritative gate: a DARK soak is GO only when parity ≥ threshold, latency gate
green, determinism green, exactly-one-row proven, namespace isolation proven (0 violet rows in
`dolphin`/`dolphin_pink`).
**Affected files (NEW):** `prod/clean_arch/violet/obs/test_violet_soak_readiness_gate.py`; report →
`prod/VIOLET_dev/reports/violet_soak_gate_<UTC>.json`.
**Interface/approach.** Drive Task 47's aggregator; assert `verdict == "GO"` ONLY when all
prerequisites pass. **IMPORTANT:** given CRITICAL #1 (parity pick-match 1.5%), this gate is EXPECTED
to currently return NO_GO — the gate's job is to make that explicit and machine-checked, NOT to be
forced green. A test asserts the gate correctly returns NO_GO when parity is below threshold.
**Pass criteria (`@pytest.mark.gate`).** Gate returns GO iff all prerequisites green; returns NO_GO
with the parity reason when parity < threshold (the current real state); report archived. Do NOT
hardcode GO. No edits outside the new file.
---
## Composition map
```
contracts_v3 (+Provenance/EconomicsRow/DecisionDivergenceRow/SoakReadiness)
44 economics_sink ── DDL-first, exactly-one-row (ReplacingMergeTree key), event_seq, async-insert
45 provenance ── capital = anchor + Σ trade-realized; pnl_source/capital_source stamps
46 divergence_v2 ── per-factor VIOLET-vs-BLUE abs_err rows (DDL-first)
47 soak_readiness ── consolidate ALL reports → SoakReadiness(GO/NO_GO)
48 soak_gate ── GO iff parity≥thr ∧ latency ∧ determinism ∧ one-row ∧ isolation
(currently NO_GO by design — CRITICAL #1)
```
Integration (wiring the sinks into the live shadow loop, scheduling the aggregator) is the OWNER's
job. The soak GO decision is the operator's, informed by Task 48.
## Recommended order
**43 (contracts) 44 (economics sink, DDL-first) 45 (provenance) 46 (divergence v2) 47
(aggregator) 48 (gate)**.
## Still NOT in scope (operator/owner only, or Claude)
- **Starting the DARK soak** operator decision (Task 48 informs it; currently NO_GO).
- **The live parity root-cause (CRITICAL #1)** Claude; this pass only MEASURES + gates on it.
- **V4 live execution; HZ-bridge; VIBRISS.**
- Any edit to in-flight / shared files in §0 (especially `ch_writer.py` / production tables).

View File

@@ -0,0 +1,47 @@
# VIOLET Multi-Exchange OB Seam
Date: 2026-06-16
## Scope
This note describes the new read-only OB provider seam added on the VIOLET side:
- `prod/clean_arch/violet/venue_ob_provider.py`
- `prod/clean_arch/violet/test_violet_venue_ob_provider.py`
The seam exists so VIOLET can later consume a venue-specific OB feed without
changing BLUE or wiring any live exchange connection into the current shadow
path.
## What the seam does
- normalizes venue ticks into BLUE-shaped `OBSnapshot` records
- keeps the provider read-only and in-memory
- validates poison values at ingress with V-TYPES
- supports either an injected callable tick source or a preloaded buffer
- exposes the exact `OBProvider` interface that `OBFeatureEngine` expects
## What it does not do
- no live BingX connection
- no Hazelcast writes
- no BLUE code changes
- no live wiring into `live_blue_source.py` or `shadow_live_factors.py`
## Future adapter shape
A venue-specific live adapter can later be layered on top of this seam by
feeding `VenueOBTick` records into `VioletVenueOBProvider.refresh()` from any
normalized feed source. The only contract is the canonical `OBSnapshot` shape
that `OBFeatureEngine` already consumes.
## Validation
The companion test file checks:
- `VioletVenueOBProvider` implements the `OBProvider` surface
- malformed values are rejected at construction
- `OBFeatureEngine(provider)` can run `step_live()` and read back finite OB
features
- the callable-source path loads as expected

View File

@@ -0,0 +1,126 @@
# VIOLET Study Spec — Base-Fraction / Capital-Utilization Sizing Study
**Status:** TODO (research spec, written 2026-06-13). Gated AFTER the regime-robustness
study (#1). Feeds VIOLET V3 Layer-3 sizing mechanics and any base-fraction change to
the live PINK/BLUE `AlphaBetSizer`.
**Owner intent:** the [[blue_margin_envelope_study]] proved BLUE's capital is badly
*under-utilized* (median trade ties up ~3.4% of wallet at 2× exchange leverage; 100% of
trades feasible at 2×; max realized `our_leverage` = notional/capital ≈ 1.81). The ROI
lever is the **base fraction** (currently `base_fraction = 0.20` in `AlphaBetSizer`),
NOT exchange leverage. Question this study answers: **how far above 0.20 can base
fraction be pushed for more ROI, risk-bounded, and where do hard constraints bind?**
---
## 0. Doctrine / non-negotiables
- **ROI is driven by `notional/capital` = `base_fraction × conviction_leverage`**, not by
exchange leverage. Exchange leverage (PINK/VIOLET max-3× **linear** translator) is a
margin-efficiency knob only. Confirmed empirically:
`notional = capital × 0.20 × leverage`, `leverage` = cubic-convex conviction ∈ [0.5, 9].
- **The edge is regime-concentrated** (≈95% of clean edge in choppy-bearish; bull is the
separate EFSM long-reversal algo's domain). Therefore sizing-up amplifies exposure to
the worst observed regime AND to the untested-by-this-strategy tails. This study MUST
output a fraction recommendation **conditioned on the regime-robustness result (#1)**,
not a raw-ROI maximizer.
- **Counterfactual honesty:** resizing past trades assumes the *same trades would have
filled at the larger size*. That assumption degrades with size (market impact). The
study MUST estimate and discount for slippage/impact, not assume linear scaling.
## 1. The hard constraint that binds first — the 3× translator ceiling
`our_leverage = base_fraction × conviction`, max conviction = 9.0. To finance a position
the exchange leverage must satisfy `exch_lev ≥ our_leverage`. PINK/VIOLET's translator
caps exchange leverage at **3×**. Therefore the **maximum financeable base fraction**
before the cap binds on the highest-conviction trades is:
```
base_fraction_max ≈ 3.0 / 9.0 ≈ 0.333 (i.e. our_leverage_max = 0.333 × 9 = 3.0 = cap)
```
- At `f = 0.20`: max our_leverage 1.8 → 2× suffices, comfortable.
- At `f ≈ 0.333`: max our_leverage 3.0 → exactly the 3× cap (no buffer on max-conviction
trades).
- At `f > 0.333`: highest-conviction trades CANNOT be financed at 3× → they clip
(under-size) or require raising the translator cap (a separate margin-risk decision).
**Deliverable 1:** the exact binding curve `f → fraction of trades that clip at 3× cap`,
using the real conviction distribution (most trades are low-conviction, so the cap may
bind on very few trades well above 0.333 — quantify it, don't assume the 0.333 worst case
dominates).
## 2. Method
Operate on the **clean deduped trade set** (one row per `trade_id`; drop `HIBERNATE_HALT`
and `bars_held = 0`; see [[blue_margin_envelope_study]] for the cleaning that yields
+$47k / 2121 trades). Required per-trade fields: `pnl`, `pnl_pct`, `entry_price`,
`quantity`, `capital_before`, `leverage` (conviction), `our_leverage`, regime hash tags
(join to `maras_fingerprint.composite_hash`), and execution-quality (slippage) from
`trade_execution_quality` / `execution_quality_json`.
### 2a. Counterfactual resize grid
For `f ∈ {0.20, 0.25, 0.30, 0.333, 0.40, 0.50}` (and finer near the optimum):
- Per trade, resized notional scales by `f / 0.20`; **`pnl_pct` is size-invariant**, so
resized `$pnl = pnl_pct × resized_notional` **before** slippage discount.
- Apply the §2c slippage discount.
- Apply the §1 cap clip: if `f × conviction > 3.0`, clip notional to `3.0 × capital`.
### 2b. Path-dependent equity reconstruction
Replay trades in time order, compounding each resized `$pnl` onto a running capital base
(bigger size → bigger swings → different compounding path; do NOT just sum). Seed from the
real starting capital of the tracked window. Produce per-`f`:
- final capital, CAGR
- **max drawdown**, Calmar/MAR (CAGR ÷ maxDD), longest-underwater days
- Sharpe, Sortino, downside deviation
- risk-of-ruin estimate
### 2c. Slippage / market-impact model (critical — do NOT skip)
The largest real-world degrader. From the maker-fill telemetry estimate whether larger
notionals get worse fills / more requotes / more taker fallback:
- regress realized fill slippage (and maker→taker fallback rate) against order notional
/ notional-vs-ADV where available
- build a `slippage_bps(notional)` discount applied in §2a
- if data is insufficient, state so and use a conservative parametric impact assumption
(document it); flag the result as impact-uncertain
### 2d. Kelly / fractional-Kelly anchor
Estimate the growth-optimal fraction from the empirical win-rate + payoff distribution.
Recommend **fractional Kelly (¼–½)** given the edge is **non-stationary and
regime-conditional** — full Kelly assumes a stationary edge we have explicitly shown does
not hold. Compare the Kelly-implied fraction to the §1 cap ceiling and the §2b
drawdown-optimal fraction.
### 2e. Regime-conditioned drawdown (the binding test)
Re-run §2b conditioned on the regime **hash** buckets from #1 (NOT the MARAS label — the
label is held untrusted; sub-regimes within choppy-bearish are expected). The binding
drawdown is the **worst-hash-bucket** drawdown, not the aggregate. Add a **stress
scenario**: inject a hypothetical adverse excursion sized to the worst plausible
unsampled-regime loss and report each `f`'s survival.
## 3. Deliverables
1. Table: `f` × {final capital, CAGR, maxDD, Calmar, Sharpe, ruin-prob, %trades-clipped-at-3×}.
2. The §1 cap-binding curve.
3. The §2c slippage discount model + its effect on the optimum.
4. A **recommended base fraction** (or a conviction-conditioned fraction *schedule*),
with the explicit risk statement: how much extra ROI, at what extra drawdown, under
what regime assumption.
5. Machine-readable report → `prod/VIOLET_dev/reports/base_fraction_study_<ts>.json`;
1-page FINDINGS alongside.
## 4. Caveats to carry into every conclusion
- Non-stationary, regime-concentrated edge — the optimum is conditional, not universal.
- Counterfactual resizing assumes fillability at scale (mitigated by §2c, never eliminated).
- Single-slot (no concurrency) — confirmed; if that ever changes, margin math changes.
- The clean set still may carry minor residual pollution; corroborate against the
corrected-capital trajectory as in the parent study.
- Do not let raw-ROI maximization override drawdown/ruin constraints. The under-utilized
capital is an *opportunity bounded by regime risk*, not free money.
## 5. Related
[[blue_margin_envelope_study]] · [[violet_v3_alpha_doctrine]] ·
`prod/bingx/leverage.py` (translator) · `nautilus_dolphin/nautilus/alpha_bet_sizer.py`
(base_fraction) · `prod/clean_arch/dita_v2/blue_parity.py` (PINK wrapper, note 8 vs 9 drift).

View File

@@ -0,0 +1,128 @@
# VIOLET — CRITICAL TODO / review queue (prominent)
Date: 2026-06-17. Single place for the must-not-forget VIOLET items. Review-later, not now.
---
## 🔴 CRITICAL #1 — VIOLET↔BLUE parity is VERY DISAPPOINTING (review + root-cause BEFORE any soak/V4)
Report: `prod/VIOLET_dev/reports/violet_parity_20260616_220412.md` (+ `.json`).
Window 2026-06-14 20:15 → 2026-06-15 21:00. Produced by PASS-2 Task 4 (`parity_report.py`).
**Headline:**
- Pick-match rate: **0.015 (1.5%)** — VIOLET rows 2853, exact pick matches only 43.
- Same-asset rate: **0.136 (13.6%)**; no-pick: **2465 / 2853 (86%)**.
- BLUE rows 25941 (scan_eval 25278 + trade_events 663).
**KEY NUANCE (where the review should START):** on the 43 rows that DID align, **sizing is
near-identical** — leverage abs error mean 0.016 / **median 0.0** / max 0.135. So the V3.4
sizing math (boost/beta/mc_scale/ob/esof/compose) is NOT the problem; the divergence is in
**ASSET SELECTION / TIMING / the comparison's ALIGNMENT method**. Candidate causes to
investigate (do not assume — measure):
1. **Apples-to-oranges population.** BLUE `scan_eval` (25278) is likely per-scan-per-asset
*evaluations*, while VIOLET rows are *actuated* decisions — the report may be comparing
different things. Verify the alignment/join semantics in `parity_report.py` first.
2. **Selection divergence.** VIOLET's `VioletAssetSelector` (IRP) vs BLUE's live selection over
the same scan stream — are they fed the same universe/lookback at the same scan index? The
sizing-gap samples (TRX/ATOM/LTC/XLM SHORT with large notional_rel_err) suggest VIOLET fires
on assets BLUE sized very differently or didn't pick.
3. **Cadence/actuation.** VIOLET actuates at Q=scan; if its scan alignment or dedupe differs,
picks land at different scans → counted as no-pick.
4. **The known structural items** (OB single-shot before V3.4d; mc_scale; live-factor sourcing)
— re-run parity AFTER V3.4d's persistent-OB launcher + the bit-identity fixes to see if the
number moves.
**Action:** full review of `parity_report.py` + a root-cause pass; fix the alignment OR the
selection divergence; re-run. This gates a meaningful DARK soak — a 1.5% pick-match makes the
soak uninterpretable. **Owner: Claude (me), later.**
---
## 🟡 #2 — Review the OA-delegated PASS work (NOT yet reviewed)
The parallel agent reports these DONE; none reviewed for correctness / BLUE-compliance yet.
**PASS 1** (`VIOLET_PART_SPEC_OA_TODO.md`):
- `53bdd90` sizing parity-pin tests
- `12b768b` venue OB provider seam (`venue_ob_provider.py`)
- `bae9284` base-fraction sizing study (+ archived report)
**PASS 2** (`VIOLET_PART_SPEC_OA_TODO_PASS2.md`):
- `parity_report.py` (Task 4 — the report above) + test
- `tradeability.py` (Task 6) + test
- `test_violet_v3_decision_latency_gate.py` (Task 5) → report `violet_v3_decision_latency_2026...`
- `test_violet_replay_determinism_gate.py` (Task 7) → report `violet_replay_determinism_2026...`
**PASS 3**: `VIOLET_PART_SPEC_OA_TODO_PASS3.md` (issued 2026-06-17) — venue feed / mechanical
exits / SL floor / event-restore / slippage / cadence; review when done.
**PASS 4**: `VIOLET_PART_SPEC_OA_TODO_PASS4.md` (issued 2026-06-17) — vol gate / V7 exit wrapper /
economics ledger / exec-intent / alpha data feed / time exits. **DONE on worktree
`agent/oa-violet4` (/mnt/vp-oa4)** — commits 6996a6a/7c8e524/bbebe58/3920665/f328d3a/b39d8a4.
Review when done. **Caveats to verify in review (flagged in-code as prominent banners + empty
marker commits 57ca851/8624dee/2aad0aa/539ea47):**
- **Task 17 (exec_intent):** `to_exec_intent` needs an EXPLICIT `reference_price` arg —
`ShadowDecision` has notional/exposure but NO price field, so qty can't be derived from a
decision alone. Integration owner threads the live price (PASS-3 `VenueTick`) at wiring time.
- **Task 18 (alpha_data_feed):** intentionally parse/feed-port ONLY — no live NT node / Binance
connection in unit code (NT owns its loop; dummy keys; separate feed process). Live node = the
integration owner's job. Confirm tests aren't vacuous (parse-fixture, not a live connection).
**PASS 5**: `VIOLET_PART_SPEC_OA_TODO_PASS5.md` (issued 2026-06-17) — MOCK-BINGX execution stack
(exec contracts + QuirkProfile seams / order FSM / mock venue / fill reducer / DARK E2E gate).
NOTE: BingX "quirks" are SEAM-ONLY here (default OFF) — a later quirk-injection pass + a mandatory
real-key boundary smoke are required before V4-live; review when done.
**PASS 6**: `VIOLET_PART_SPEC_OA_TODO_PASS6.md` (issued 2026-06-17) — execution INTERNALS
(fill-pump/ownership filter, reconcile/zero-wb guard, TTL-requote, orphan handling) + the
QUIRK-INJECTION gate that flips PASS-5's QuirkProfile flags ON and proves each handler neutralizes
the quirk. Mirrors PINK's production fixes (pink_direct.py). Real-key smoke still MANDATORY before
V4-live; review when done.
**PASS 7**: `VIOLET_PART_SPEC_OA_TODO_PASS7.md` (issued 2026-06-17) — V5 selection (faithful
ARS/IRP ranking + OB Sub-1), multi-asset slot manager, capital allocation, multi-asset flow, and a
ranking bit-identity gate vs AlphaAssetSelector. NOTE: this layer is the suspected locus of CRITICAL
#1 — PASS 7 PINS the ranking math (narrowing the suspect to timing/join), but the live-aggregate
root-cause stays Claude's job. Review when done.
**PASS 8**: `VIOLET_PART_SPEC_OA_TODO_PASS8.md` (issued 2026-06-17) — V6 bible CONSUMERS: posture
effects engine (5-state entry-gate/flatten/cap), MARAS fingerprint consumer (role TBD from code —
no invented modulation), regime read-model, cadence shadow-actuation telemetry, posture/regime
parity gate. VIOLET CONSUMES posture/MARAS (MHS owns them); ACB/vol/SL-guard already covered
elsewhere. Review when done.
**PASS 9**: `VIOLET_PART_SPEC_OA_TODO_PASS9.md` (issued 2026-06-17) — economics/observability
completeness: DDL-first exactly-one-row economics sink (PINK duplicate-emission fix), provenance +
capital=anchor+Σ-trade-realized, factor divergence v2, soak-readiness aggregator, and the
soak-readiness @gate (currently NO_GO by design — gated on CRITICAL #1). Review when done.
> **OA BACKLOG COMPLETE (PASS 19 issued).** PASS 19 carve out the full parallelizable DARK
> surface of the V0→V6 ladder (~48 independent units, contracts_v3-keyed). What remains is NOT
> parallelizable and is Claude's: the parity root-cause (CRITICAL #1), review of all passes,
> integration, E2E, VBT re-cert, the real-key V4 boundary smoke, and live execution.
**Action:** review each pass for correctness, BLUE-algo compliance, V-TYPES, no-shared-edits,
real (non-vacuous) tests. **Owner: Claude (me), later.**
---
## 🟢 #3 — Integration / "sprint" consolidation (my later work)
Once the passes are reviewed, I will: **(a)** review ALL passes together, **(b)** integrate them
(resolve interfaces, dedupe), **(c)** test them together, **(d)** plug into the operational
system, **(e)** E2E test.
**Nomenclature note (raised by operator):** a "pass" here = a batch of self-contained tasks
delegated to one agent — smaller than an Agile **sprint** (a time-boxed iteration, typ. 1-4
weeks, team-scoped, ending in a shippable increment, with planning/review/retro ceremonies).
In this project's existing usage, **"Sprint N" already maps to a V-stage bundle** (Sprint 1 =
V0+V1, Sprint 2 = V2, Sprint 3 = V3) — i.e. an epic/milestone. So the cleanest mapping:
**V-stage = sprint/epic; "pass" = a sub-sprint work-package / task-bundle within it.** Renaming
passes to "sprints" would over-claim scope; keep "pass"/"work-package", or call each an
"increment". Decide at integration time.
---
## Gating rule
The DARK soak (operator-held) and V4 are NOT meaningfully runnable until CRITICAL #1 is
root-caused — a 1.5% pick-match means the shadow is not yet tracking BLUE's decisions.

View File

@@ -48,7 +48,7 @@ Self-consistent at row level vs recorded `dolphin.trade_events`:
- **DUAL-LEVERAGE:** conviction leverage sizes the QUANTITY (internal); exchange leverage - **DUAL-LEVERAGE:** conviction leverage sizes the QUANTITY (internal); exchange leverage
mapped at the venue boundary via `prod/bingx/leverage.py` mapped at the venue boundary via `prod/bingx/leverage.py`
`map_internal_conviction_to_exchange_leverage_target` (round_half_even linear `map_internal_conviction_to_exchange_leverage_target` (round_half_even linear
0.59.0 → 1..cap; PINK/VIOLET use a max-3× cubic translator). 0.59.0 → 1..cap; PINK/VIOLET use a max-3× **linear** translator).
## 3. blue_parity drift (doctrine validated by evidence) ## 3. blue_parity drift (doctrine validated by evidence)

View File

@@ -0,0 +1,48 @@
# UV TASK T10 — Gate A/B certification run (the cert that unlocks testnet)
**Assignee:** cmd-PASS1.1 · **Issuer:** Fable · **PRIORITY over T8 resume** (T8 stays
paused; resume after T10). **Master spec:** §5 Gates A/B. **Base:** main ≥ b70d5ab
(differ is LIVE — `parity.differ` real module, 33/33 contract tests green).
## Goal
Produce the first REAL certification verdict: run Gate A (replay parity) and Gate B
(fuzz faultlines) end-to-end, emit the gate ledger via the T4 reporter, and hand Fable
a signed-off report. This is the last gate before the operator arms VST keys.
## Work items
1. **Apply the T7 tier rework** (codex's unapplied instruction, on the bus + here):
restructure `replay_cert.py`'s report into labeled coverage tiers —
- **Tier 1 (gating):** replay over `dolphin_uv.prime_decisions` (PRIME's own journal,
accumulating since the zinc soak began). Today it holds decisions + hook frames
(not yet raw scan surface — pi's T2P1 adds that); so Tier 1 v1 = decision-replay
consistency: journaled decision == recomputed decision from journaled inputs where
inputs suffice; report the input-coverage fraction HONESTLY per row.
- **Tier 2 (gating, entries only):** entry-anchored bit-identity — replay each BLUE
`dolphin.trade_events` entry through the hooks using its own
`market_state_bundle_json` + `execution_quality_json` recorded inputs; diff via
`parity.differ.diff_entry_event`. This works over DEEP history NOW.
- **Tier 3a (labeled, non-gating):** ~2wk real-scan parquet era (2026-03-04..18,
`/mnt/dolphin/vbt_cache_klines`, loader precedent `paper_trade_flow.load_day_scans`)
— near-full input replay, report drift-rate.
- **Tier 3b (labeled, non-gating):** scalar+obf consistency (eigen_scans scalars +
obf_universe prices) — no-phantom-entry checks only.
2. **Wire the T4 reporter:** gate ledger rows for Gate A (verdict = Tier1 AND Tier2
pass) and Gate B; ledger + report to `prod/docs/uv_cert/GATE_AB_RUN_<ts>.md/.json`.
3. **Gate B run:** execute the full armed property/fuzz surface (pi's Hypothesis suites
+ differ property tests) with `--hypothesis-seed` pinned, plus a poison-input sweep
of the differ/align path (NaN/inf/unicode/dup scans — most exist already; run them
as a named gate, record counts). Gate B verdict = all green, deterministic re-run
identical.
4. **Determinism proof:** run the whole cert TWICE; reports must be byte-identical
except the run timestamp (seed-pin, injected clock).
## Iron rules
Read-only against `dolphin.*`; writes ONLY `dolphin_uv.*` + report files. no_write_guard
active during all replay. Never touch BLUE. If Tier 2 surfaces real DRIFT (not a
BLUE-code-era artifact per data-derived changepoints): STOP, report to Fable, do NOT
"fix" hook math to make it pass — parity bugs are findings, not test failures.
## Done
Branch `uv/t10-gate-ab` off ≥ b70d5ab, fresh clone, push → CI, DONE to Fable with:
report path + headline (entries checked, matched/drift/phantom per tier, Gate A verdict,
Gate B verdict, wall-clock). Sample DiffReport for any drift.

View File

@@ -0,0 +1,40 @@
# UV TASK T12 — promotion bridge (PRIME decision → KernelIntent → seam)
**Assignee:** pi_nvnemo (AFTER T2P1 — do not context-switch) · **Issuer:** Fable.
**Master spec:** §8 fast path. **Companion:** T9 exec seam (cmd-PASS1.2, in flight).
## Goal
The missing link between the certified brain and the trading hand: when PRIME (shadow)
is PROMOTED, its entry decisions must flow as KernelIntents into T9's seam. Today
nothing connects them — T9's `intent_source` is an injectable queue. Build the bridge,
DARK-safe by construction.
## Deliverable: `prod/clean_arch/violet/uv/blue_prime/promotion.py`
1. **Gate:** module does NOTHING unless `UV_PROMOTED=1` in env AND an operator arming
file exists (`/root/uv-wt/prime-live/UV_PROMOTED.arm` — content = ISO ts + operator
initials; absence = shadow mode, bridge inert). Two-man rule mirrors T9's mainnet
block. Log loudly on every startup which mode we are in.
2. **Translation:** PRIME's per-scan decision dict (`has_entry`, asset, side, leverage,
bar_idx, entry payload — exactly what `build_journal_frame` sees) → `KernelIntent`
(dita_v2 contracts; import from the vendored kernel; preflight the exact field set
against `contracts.py` — do NOT guess fields). `client_tag` = `u-` prefix source.
3. **Transport:** publish to the seam's intent queue. v1 = the injectable queue seam
T9 exposes (in-process import); leave a clearly-marked TODO seam for zinc
intent_region transport (that is wave-2, do not build it now).
4. **Journal:** every bridged intent (and every SUPPRESSED one while un-promoted) →
`dolphin_uv.exec_journal` kind='BRIDGE' rows. The suppressed-intent record is the
shadow-mode evidence the operator reviews before arming.
5. **Runner wiring:** guarded call from `blue_prime/runner.py` after the journal write
(never before — journal is the source of truth), wrapped so ANY bridge exception
cannot kill the scan loop (fail-soft, log + count).
## Tests (armed and RUN before ship — remember the differ lesson)
Inert-by-default (no env/file ⇒ zero intents, suppressed rows journaled); arming
matrix (env-only ⇒ inert, file-only ⇒ inert, both ⇒ active); translation bit-exact
fixtures vs a fixed decision dict; mutation litmus (drop u- tag, skip suppressed
journal, swallow exception ⇒ RED); fail-soft (bridge raises ⇒ runner loop continues).
## Done
Branch `uv/t12-promotion-bridge`, fresh clone, push → CI, DONE to Fable with sha,
test count, and a shadow-mode journal excerpt showing suppressed intents recorded
during a live soak window.

View File

@@ -0,0 +1,32 @@
# UV TASK T6 — real Zinc region transport (close the file-snapshot regression)
**Assignee:** codex · **Issuer:** Fable · **PRIORITY: HIGH — operator-ordered promotion.**
**Why:** the merged main's TUI path reads an atomic JSON file in /dev/shm (`ShmChannel`) —
file semantics, no mapped region/seqlock/notify. That is the exact "mock shm" the shm-reshape
sprint existed to eliminate. The REAL transport already exists and soaked 18 h on the clone
line: `ZincShadowChannel` in `salvage/uv-clone-line-1902e4b`'s `uv/shm.py` (mapped
`/dev/shm/zinc_uv_shadow_state` region, magic header, monotonic seq, `uv_shadow` prefix,
DITAv2 zinc adapter loading via `ZINC_PYTHON_PATH`).
**Task:** port `ZincShadowChannel` (+ its region encode/decode + `_FileChannel` fallback
mechanics as needed) from the salvage branch onto current main, UNDER the reshape snapshot
contract (BluePrimeSnapshot stays the schema — transport changes, contract does not):
1. New/updated module in `uv/` (e.g. extend `shm.py` or add `zinc_channel.py`): writer =
publish the SAME versioned snapshot into the mapped region, atomic + seq-increment +
notify per the zinc-shadow spec §5; reader = TUI-side wait/read of latest complete frame.
2. `blue_prime/runner.py`: publish via real zinc region as PRIMARY; keep the file snapshot as
explicit FALLBACK (env `UV_SHM_TRANSPORT=file`; default `zinc`), not the default.
[SPEC AMENDED 2026-07-02: knob name aligned to the salvage implementation's
`UV_SHM_TRANSPORT` — same semantics as the original `UV_SHM_FALLBACK`, clearer name,
zero churn on the already-soaked code.]
3. `uv/tui.py`: read the region directly as PRIMARY (same render path — snapshot dict in,
panels out); file fallback only when region absent, and SAY SO on screen.
4. Port/adapt the salvage branch's zinc tests + add: torn-frame/seq test, cross-process
round-trip test (writer proc + reader proc), region-absent fallback test, mutation litmus.
5. Relaunch the guarded soak (same UV_BLUE-PRIME_TUI2 discipline, guard active) on the zinc
path; verify seq advances at scan cadence and TUI reads the REGION (show source in meta).
**Non-goals:** no tui_v2 port (separate, later), no DITAv2 region reuse (own prefix,
retire-able — zinc-shadow spec §3.3), no schema changes.
**Branch:** `uv/t6-real-zinc` · fresh clone `/root/uv-wt/t6-zinc` off /root/violet.git ·
push→CI · DONE to Fable with: branch+sha, tests, soak PIDs + region seq evidence, and
`grep`-proof the TUI's primary path never opens the JSON file when the region exists.

View File

@@ -0,0 +1,49 @@
# UV TASK T8 — shadow stop-watcher (C11's sensor, live overshoot ledger)
**Assignee:** cmd (Command Code) · **Issuer:** Fable · **Master spec:** §4 C11, §5 Gate C.
**Empirical basis:** `prod/docs/BLUE_STOPLOSS_OVERSHOOT_AND_UV_COUNTERFACTUAL_20260702.md`
(codex, verified by Fable): last-1000 BLUE trades → 49 stops, 30 overshot the 1.2%
contract, ≈$4,432 excess. FET `e81e595d`: stop breached 16:08:15.9, eigenscan not until
16:08:19.6 (measured gaps 1112 s), overshoot lived entirely inside the scan gap.
## Goal
A standalone read-only daemon that watches BLUE's OPEN positions on a ~1 s clock and
journals every stop-contract breach the moment it happens — so every future overshoot is
measured live instead of forensically. This is the shadow twin of the future C11 fast SL
clock: when UV trades with fast stops, this ledger IS the A/B evidence.
## Deliverable: `prod/clean_arch/violet/uv/stop_watcher.py` (+ DDL + tests)
1. **Position feed:** poll `dolphin.trade_events` (read-only, dedup by trade_id, argMax ts)
every ~10 s for open positions (entry event without terminal exit). Carry entry_price,
side, quantity, our_leverage.
2. **Price feed:** poll `dolphin.obf_universe` (read-only) best_bid/best_ask per open asset
every ~1 s. MEASURE ingest lag (row ts vs now) and journal it; if p95 lag > 2 s, log a
LIMITATION line — do NOT silently trust stale prices. (Fallback to BingX public WS
bookTicker is allowed — public data, no keys, no BLUE interaction — but is a stretch
goal, not v1.)
3. **Breach detection:** SHORT: ask ≥ entry×1.012; LONG: bid ≤ entry×0.988. On first
breach per trade, write one event; keep sampling and write escalation rows at each
+0.1 % beyond the stop (so the overshoot PATH is recorded, not just the edge).
4. **Journal:** `dolphin_uv.stop_watch_events` (NEW table, dolphin_uv namespace ONLY):
ts, trade_id, asset, side, entry_price, breach_price, adverse_pct, spread_bps,
depth_1pct_usd, obf_lag_ms, kind ('BREACH'|'ESCALATION'|'RESOLVED'). RESOLVED row when
the trade's exit appears in trade_events, carrying exit adverse_pct + excess vs 1.2 %.
5. **Daily rollup view:** overshoot count, total excess $, worst trade — the $4.4K audit,
automated forever.
## Iron rules
- ZERO writes outside `dolphin_uv.*`. Never touch BLUE code, HZ contents, or dolphin.*
tables. CH creds read-only usage; INSERT only into dolphin_uv.stop_watch_events.
- Runs under /home/dolphin/siloqy_env, own log file, no hardcoded worktree paths
(derive paths from __file__ / env).
## Tests (doctrine: mutation litmus mandatory)
Synthetic price/position fixtures → exact expected breach/escalation/resolved sequence;
mutation litmus (flip breach comparison, drop escalation step → tests go RED);
lag-measurement honesty test; no-write guard test (any non-dolphin_uv INSERT raises);
determinism (same fixture ⇒ same journal twice).
## Done
Branch `uv/t8-stop-watcher` off current main, fresh clone, push → CI, DONE to Fable with:
branch+sha, test count, a live 30-min run's journal excerpt (real breaches or honest
"no breaches in window"), measured obf_universe lag stats.

View File

@@ -0,0 +1,48 @@
# UV TASK T9 — C10 DITAv2 exec seam (KernelIntent → DITAv2 → BingX VST, DARK)
**Assignee:** cmd-PASS1.2 · **Issuer:** Fable · **Master spec:** §4 C10, §8 fast path.
**Doctrinal kernel:** post-sync vendored dita_v2 per `UV_DITAV2_SOA_VERDICT_20260703.md`
(includes VenueTelemetrySnapshot + zinc venue plane + asex_account.py). Do NOT start
until Fable confirms the vendor sync landed on main (watch the bus).
## Goal
The physical trading path: a UV-owned execution runner that consumes KernelIntents and
drives DITAv2 against BingX **VST testnet** — built now, DARK by default, so that when
PRIME is promoted (post Gates A+B) the only remaining step is arming it with keys.
Pattern precedent: `prod/clean_arch/violet/v4_execution_runner.py` (read for shape; this
is a NEW instance and a NEW module — never reuse a BLUE/PINK/VIOLET runner instance).
## Deliverable: `prod/clean_arch/violet/uv/exec/` (new package)
1. **`intent_source.py`:** KernelIntent inlet. v1 = injectable queue + a file/CLI
injector for dry-run intents (PRIME promotion wiring is a LATER task — leave a
clearly-marked seam, not a stub that pretends).
2. **`seam.py`:** intent → DITAv2 order mapping. EVERY clientOrderId prefixed `u-`
(non-negotiable — this is how UV's orders are distinguishable on the venue forever).
Sizing/leverage passthrough from intent; no local overrides.
3. **`runner.py`:** launcher wiring per dita_v2 `launcher.py` — NEW instance name
(`uv_exec`), zinc venue plane ENABLED (venue_region telemetry is the seam's flight
recorder), jemalloc-friendly long-run posture, no hardcoded paths.
4. **DARK doctrine:** with no keys configured → observe-only: log + journal every intent
and the order it WOULD place (full params), place nothing. `ALLOW_MAINNET=0` is a
hard block: mainnet refuses even if env says otherwise unless a separate operator
arming file exists (two-man rule). VST base URL only.
5. **Journal:** every intent, mapping, would-place/placed, venue telemetry snapshot →
`dolphin_uv.exec_journal` (dolphin_uv namespace ONLY).
## Iron rules
- VST ONLY. DARK until operator arms. `u-` prefix on every clientOrderId.
- Zero writes outside dolphin_uv.*; zero BLUE touches; vendored dita_v2 is read-only
(any kernel change goes upstream + vendor_sync, never in-place).
- Graal-ready NFR-G applies (no CPython-only exotica in the seam layer).
## Tests (mutation litmus mandatory)
Intent→order mapping bit-exact fixtures; `u-` prefix litmus (strip the prefix in code →
test RED); DARK default test (no keys ⇒ zero venue calls — assert at the venue adapter
seam, not by mocking the seam itself); mainnet-block mutation test (force
ALLOW_MAINNET=1 without arming file ⇒ still refuses); venue-plane telemetry presence
test; determinism.
## Done
Branch `uv/t9-exec-seam` off post-sync main, fresh clone (NOT a worktree of the bare),
push → CI, DONE to Fable with: branch+sha, test count, and a dry-run journal excerpt
showing 3 injected intents fully mapped + journaled + zero venue calls.

304
test_pi_wake_agent.py Normal file
View File

@@ -0,0 +1,304 @@
#!/usr/bin/env python3
"""
Comprehensive test suite for pi_wake_agent.py
Tests all modes, edge cases, and the new succession feature.
"""
import pytest
import subprocess
import time
import tempfile
import os
import sys
from pathlib import Path
from unittest.mock import patch, MagicMock, call
# Add the script directory to path
sys.path.insert(0, "/mnt/dolphinng5_predict")
from pi_wake_agent import (
parse_interval,
interval_to_cron,
interval_to_human,
cron_comment,
parse_sessions,
SCRIPT_PATH,
LOG_FILE,
LOG_MAX_SIZE,
LOG_MAX_FILES,
CRON_COMMENT_PREFIX,
AGENT_NICK,
H5I_AGENT,
H5I_BUS_ROOT,
DEFAULT_INTERVAL,
)
# ─── Test parse_interval ─────────────────────────────────────────────────
class TestParseInterval:
def test_hours(self):
assert parse_interval("1h") == 3600
assert parse_interval("2h") == 7200
assert parse_interval("24h") == 86400
def test_minutes(self):
assert parse_interval("30m") == 1800
assert parse_interval("1m") == 60
assert parse_interval("90m") == 5400
def test_seconds(self):
assert parse_interval("30s") == 30
assert parse_interval("1s") == 1
assert parse_interval("10s") == 10
def test_invalid(self):
with pytest.raises(ValueError):
parse_interval("invalid")
with pytest.raises(ValueError):
parse_interval("1x")
with pytest.raises(ValueError):
parse_interval("")
# ─── Test interval_to_cron ───────────────────────────────────────────────
class TestIntervalToCron:
def test_hours(self):
assert interval_to_cron("1h") == "0 */1 * * *"
assert interval_to_cron("2h") == "0 */2 * * *"
assert interval_to_cron("6h") == "0 */6 * * *"
def test_minutes(self):
assert interval_to_cron("1m") == "*/1 * * * *"
assert interval_to_cron("30m") == "*/30 * * * *"
assert interval_to_cron("45m") == "*/45 * * * *"
def test_invalid_minutes(self):
with pytest.raises(ValueError):
interval_to_cron("60m")
with pytest.raises(ValueError):
interval_to_cron("90m")
def test_invalid_seconds(self):
with pytest.raises(ValueError):
interval_to_cron("30s")
def test_invalid_format(self):
with pytest.raises(ValueError):
interval_to_cron("invalid")
# ─── Test interval_to_human ──────────────────────────────────────────────
class TestIntervalToHuman:
def test_hours(self):
assert interval_to_human("1h") == "1 hour(s)"
assert interval_to_human("2h") == "2 hour(s)"
def test_minutes(self):
assert interval_to_human("30m") == "30 minute(s)"
assert interval_to_human("1m") == "1 minute(s)"
def test_seconds(self):
assert interval_to_human("30s") == "30 second(s)"
assert interval_to_human("1s") == "1 second(s)"
def test_invalid(self):
assert interval_to_human("invalid") == "invalid"
# ─── Test cron_comment ───────────────────────────────────────────────────
class TestCronComment:
def test_single_session(self):
sessions = ["cc_UV_dev0_Fb"]
result = cron_comment(sessions, "1h")
assert result == "pi_wake_agent:cc_UV_dev0_Fb:1h"
def test_multiple_sessions(self):
sessions = ["cc_UV_dev0_Fb", "cc_UV_dev1_48"]
result = cron_comment(sessions, "30m")
assert result == "pi_wake_agent:cc_UV_dev0_Fb,cc_UV_dev1_48:30m"
def test_empty_sessions(self):
result = cron_comment([], "1h")
assert result == "pi_wake_agent::1h"
# ─── Test parse_sessions ─────────────────────────────────────────────────
class TestParseSessions:
def test_comma_separated(self):
result = parse_sessions("cc_UV_dev0_Fb,cc_UV_dev1_48")
assert result == ["cc_UV_dev0_Fb", "cc_UV_dev1_48"]
def test_single_session(self):
result = parse_sessions("cc_UV_dev0_Fb")
assert result == ["cc_UV_dev0_Fb"]
def test_with_spaces(self):
result = parse_sessions("cc_UV_dev0_Fb, cc_UV_dev1_48")
assert result == ["cc_UV_dev0_Fb", "cc_UV_dev1_48"]
def test_empty(self):
assert parse_sessions("") == []
assert parse_sessions(None) == []
def test_list_input(self):
result = parse_sessions(["a", "b", "c"])
assert result == ["a", "b", "c"]
def test_filters_empty(self):
result = parse_sessions("a,,b")
assert result == ["a", "b"]
# ─── Test main functions via subprocess ──────────────────────────────────
SCRIPT = "/mnt/dolphinng5_predict/pi_wake_agent.py"
def run_cmd(args, timeout=30):
"""Run the script and return (returncode, stdout, stderr)"""
result = subprocess.run(
[sys.executable, SCRIPT] + args,
capture_output=True,
text=True,
timeout=timeout
)
return result.returncode, result.stdout, result.stderr
# ─── Integration Tests ──────────────────────────────────────────────────
class TestHelp:
def test_help(self):
rc, out, err = run_cmd(["--help"])
assert rc == 0
assert "pi_wake_agent.py" in out
assert "--install" in out
assert "--once" in out
assert "--daemon" in out
assert "--succession" in out
assert "--count" in out
class TestList:
def test_list_no_cron(self):
# Clear any existing cron entries first
subprocess.run(["crontab", "-l"], capture_output=True)
subprocess.run("crontab -l 2>/dev/null | grep -v pi_wake_agent | crontab -", shell=True, check=False)
rc, out, err = run_cmd(["--list"])
assert rc == 0
assert "pi_wake_agent cron entries" in out
class TestInstallRemove:
def setup_method(self):
# Clear cron before each test
subprocess.run("crontab -l 2>/dev/null | grep -v pi_wake_agent | crontab -", shell=True, check=False)
def teardown_method(self):
subprocess.run("crontab -l 2>/dev/null | grep -v pi_wake_agent | crontab -", shell=True, check=False)
def test_install_and_list(self):
rc, out, err = run_cmd(["--install", "--interval", "1h", "--session", "test_session", "--msg", "test"])
assert rc == 0
rc, out, err = run_cmd(["--list"])
assert rc == 0
assert "test_session" in out
assert "1h" in out
# Cleanup
run_cmd(["--remove", "--session", "test_session", "--interval", "1h"])
def test_install_multi_session(self):
rc, out, err = run_cmd(["--install", "--interval", "30m", "--sessions", "s1,s2", "--msg", "multi"])
assert rc == 0
rc, out, err = run_cmd(["--list"])
assert rc == 0
assert "s1,s2" in out
run_cmd(["--remove", "--sessions", "s1,s2", "--interval", "30m"])
class TestValidate:
def test_validate_existing(self):
# The test session might not exist, so we test the command runs
rc, out, err = run_cmd(["--validate", "--session", "pi_test"])
assert rc == 0 # Should not crash
class TestOnce:
def test_once_short(self):
# Use a very short interval
rc, out, err = run_cmd(["--once", "--interval", "1s", "--session", "test_session", "--msg", "quick test"], timeout=10)
assert rc == 0
# Check log file for the message
time.sleep(2)
log_content = Path("/tmp/pi_wake_agent.log").read_text()
assert "One-shot timer set" in log_content
class TestSuccession:
def test_succession_short(self):
# Test succession mode with very short intervals
rc, out, err = run_cmd([
"--succession", "--count", "2", "--interval", "1s",
"--session", "test_session", "--msg", "succession test"
], timeout=60)
assert rc == 0
# Check logs
time.sleep(3)
log_content = Path("/tmp/pi_wake_agent.log").read_text()
assert "SUCCESSION START" in log_content
assert "SUCCESSION COMPLETE" in log_content
def test_succession_invalid_count(self):
rc, out, err = run_cmd(["--succession", "--count", "0", "--interval", "1s", "--session", "test"])
# Should fail with invalid count
assert rc != 0 or "error" in err.lower()
class TestStatus:
def test_status(self):
rc, out, err = run_cmd(["--status"])
assert rc == 0
assert "pi_wake_agent Status" in out
assert "Script:" in out
class TestRun:
def test_run_mode(self):
# This is the internal mode called by cron
rc, out, err = run_cmd(["--run", "--session", "test_session", "--msg", "test"])
assert rc == 0 # Should succeed even if session doesn't exist
class TestEdgeCases:
def test_invalid_interval(self):
rc, out, err = run_cmd(["--install", "--interval", "invalid", "--session", "test"])
assert rc != 0
def test_missing_session(self):
rc, out, err = run_cmd(["--install", "--interval", "1h"])
assert rc != 0
assert "session" in err.lower() or "required" in err.lower()
def test_invalid_count(self):
rc, out, err = run_cmd(["--succession", "--count", "-1", "--interval", "1s", "--session", "test"])
assert rc != 0
class TestSessionParsing:
def test_multiple_session_flags(self):
rc, out, err = run_cmd(["--install", "--interval", "1h", "--session", "s1", "--session", "s2", "--msg", "test"])
assert rc == 0
run_cmd(["--remove", "--session", "s1", "--interval", "1h"])
run_cmd(["--remove", "--session", "s2", "--interval", "1h"])
class TestRemove:
def test_remove_nonexistent(self):
# Should not crash
rc, out, err = run_cmd(["--remove", "--session", "nonexistent", "--interval", "1h"])
assert rc == 0
# ─── Test Logging ─────────────────────────────────────────────────────────
class TestLogging:
def test_log_file_created(self):
run_cmd(["--install", "--interval", "1h", "--session", "log_test", "--msg", "test"])
assert LOG_FILE.exists()
run_cmd(["--remove", "--session", "log_test", "--interval", "1h"])
# ─── Run all tests ────────────────────────────────────────────────────────
if __name__ == "__main__":
pytest.main([__file__, "-v", "--tb=short"])