dita_v2: unknown != flat (VenuePostAckError) + lossless telemetry lane

BUG CLASS (doc: BUGCLASS_INDETERMINATE_OUTCOME_20260713.md): an operation with an external
side effect has THREE outcomes — NOT_ATTEMPTED, ATTEMPTED_REFUSED, ATTEMPTED_INDETERMINATE
— and rollback is sound only for the first two. Collapsing the third into 'failed' is what
orphaned 6 live SHORTs: a post-ack TypeError reached rust_backend's 'except Exception ->
synthetic REJECTED -> FSM rollback', which asserted 'no order exists' about an order that
was already filled. Telemetry never had a veto; it hijacked the failure channel.

FIXES (no new seams, no re-architecture):
- venue.py: VenuePostAckError — typed channel meaning THE EFFECT EXISTS. Carries receipt.
- bingx_venue submit/submit_async: point-of-no-return fence. Post-ack bookkeeping failures
  raise VenuePostAckError instead of a bare exception.
- rust_backend (BOTH submit paths): catch VenuePostAckError FIRST -> no synthetic REJECT,
  no rollback. Slot stays working; E-feed FULL_FILL / reconcile settles the truth.

LOSSLESS TELEMETRY (HJ: 'DITAv2 exists precisely because seams dropped 40% of inputs'):
drop-oldest is data loss and is GONE. Exec path appends O(1) to an unbounded queue and
returns. A SEPARATE spiller thread (which never touches the plane, so a wedged plane cannot
starve it) parks the backlog above HWM into a durable append-only spool; the publisher
replays the spool when the plane recovers.
Proven: wedged-forever plane + 200k records -> 0 dropped, 195903 durable on disk, 4096 in
memory, 7.4 us/call on the exec path. Lossless AND memory-bounded. Healthy plane: 2000/2000.

STILL BROKEN, flagged to codex: the pre-ack branch rolls back on TIMEOUT — but a timeout is
the definition of INDETERMINATE (the order may have filled). Same bug class, older, live.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Codex
2026-07-13 10:31:39 +02:00
parent b13b54e769
commit 3d75e4b59d
4 changed files with 405 additions and 40 deletions

View File

@@ -43,6 +43,7 @@ from .projection import HazelcastProjection
from .projection import build_projection
from .utils import json_safe
from .venue import VenueAdapter
from .venue import VenuePostAckError
from .zinc_plane import InMemoryZincPlane, ZincPlane
@@ -848,12 +849,23 @@ class ExecutionKernel:
if outcome.accepted and intent.action in {KernelCommandType.ENTER, KernelCommandType.EXIT}:
try:
emitted_events = self.venue.submit(intent)
except VenuePostAckError as _post_ack_exc:
# Order is LIVE at the venue: unknown != flat. No rollback, no
# synthetic REJECT — the E-feed FILL / reconcile settles the slot.
# (See the async path below and venue.VenuePostAckError.)
import logging as _log
_log.getLogger(__name__).critical(
"FIX(rust_backend): POST-ACK venue failure (%s) slot=%d action=%s — "
"ORDER IS LIVE. NOT rolling back; awaiting E-feed FILL / reconcile.",
_post_ack_exc, intent.slot_id, intent.action.value,
)
emitted_events = list(getattr(_post_ack_exc, "events", []) or [])
except Exception as _submit_exc:
# venue.submit() failed (e.g. BingX timeout). The Rust FSM already
# advanced to ORDER_REQUESTED / ENTRY_WORKING with no corresponding
# exchange order. Feed a synthetic REJECTED event so the FSM rolls
# back to IDLE — otherwise the slot is stranded and every subsequent
# ENTER with a different trade_id hits SLOT_BUSY forever.
# PRE-ACK: venue.submit() failed (e.g. BingX timeout). The Rust FSM
# already advanced to ORDER_REQUESTED / ENTRY_WORKING with no
# corresponding exchange order. Feed a synthetic REJECTED event so the
# FSM rolls back to IDLE — otherwise the slot is stranded and every
# subsequent ENTER with a different trade_id hits SLOT_BUSY forever.
import logging as _log
_log.getLogger(__name__).error(
"venue.submit failed (%s) — feeding synthetic REJECTED to roll back FSM slot=%d action=%s",
@@ -1004,7 +1016,28 @@ class ExecutionKernel:
emitted_events = await submit_async(intent)
else:
emitted_events = self.venue.submit(intent) # fallback: mock/test venue
except VenuePostAckError as _post_ack_exc:
# THE ORDER IS LIVE AT THE VENUE. This is UNKNOWN, not failure —
# and unknown is NEVER flat. Rolling the FSM back here is what
# orphaned 6 live SHORTs on 2026-07-13: the venue kept the
# positions while the kernel believed it was flat, so no SL/TP/
# ADVSL could ever fire on them.
#
# Emit NO synthetic event: leave the slot in its working state and
# let execution truth settle it — the E-feed delivers FULL_FILL
# from the account stream independently of this call path.
import logging as _log
_log.getLogger(__name__).critical(
"FIX(rust_backend): POST-ACK venue failure (%s) slot=%d action=%s — "
"ORDER IS LIVE. NOT rolling back (unknown != flat); awaiting E-feed "
"FILL / reconcile to settle the slot.",
_post_ack_exc, intent.slot_id, intent.action.value,
)
emitted_events = list(getattr(_post_ack_exc, "events", []) or [])
except Exception as _submit_exc:
# PRE-ACK failure (timeout, connection refused, rejected request):
# the venue never took the order, so the FSM must roll back or the
# slot is stranded in ORDER_REQUESTED forever.
import logging as _log
_log.getLogger(__name__).error(
"venue.submit_async failed (%s) — synthetic REJECTED, FSM rollback slot=%d action=%s",