dita_v2: unknown != flat (VenuePostAckError) + lossless telemetry lane
BUG CLASS (doc: BUGCLASS_INDETERMINATE_OUTCOME_20260713.md): an operation with an external side effect has THREE outcomes — NOT_ATTEMPTED, ATTEMPTED_REFUSED, ATTEMPTED_INDETERMINATE — and rollback is sound only for the first two. Collapsing the third into 'failed' is what orphaned 6 live SHORTs: a post-ack TypeError reached rust_backend's 'except Exception -> synthetic REJECTED -> FSM rollback', which asserted 'no order exists' about an order that was already filled. Telemetry never had a veto; it hijacked the failure channel. FIXES (no new seams, no re-architecture): - venue.py: VenuePostAckError — typed channel meaning THE EFFECT EXISTS. Carries receipt. - bingx_venue submit/submit_async: point-of-no-return fence. Post-ack bookkeeping failures raise VenuePostAckError instead of a bare exception. - rust_backend (BOTH submit paths): catch VenuePostAckError FIRST -> no synthetic REJECT, no rollback. Slot stays working; E-feed FULL_FILL / reconcile settles the truth. LOSSLESS TELEMETRY (HJ: 'DITAv2 exists precisely because seams dropped 40% of inputs'): drop-oldest is data loss and is GONE. Exec path appends O(1) to an unbounded queue and returns. A SEPARATE spiller thread (which never touches the plane, so a wedged plane cannot starve it) parks the backlog above HWM into a durable append-only spool; the publisher replays the spool when the plane recovers. Proven: wedged-forever plane + 200k records -> 0 dropped, 195903 durable on disk, 4096 in memory, 7.4 us/call on the exec path. Lossless AND memory-bounded. Healthy plane: 2000/2000. STILL BROKEN, flagged to codex: the pre-ack branch rolls back on TIMEOUT — but a timeout is the definition of INDETERMINATE (the order may have filled). Same bug class, older, live. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -43,6 +43,7 @@ from .projection import HazelcastProjection
|
||||
from .projection import build_projection
|
||||
from .utils import json_safe
|
||||
from .venue import VenueAdapter
|
||||
from .venue import VenuePostAckError
|
||||
from .zinc_plane import InMemoryZincPlane, ZincPlane
|
||||
|
||||
|
||||
@@ -848,12 +849,23 @@ class ExecutionKernel:
|
||||
if outcome.accepted and intent.action in {KernelCommandType.ENTER, KernelCommandType.EXIT}:
|
||||
try:
|
||||
emitted_events = self.venue.submit(intent)
|
||||
except VenuePostAckError as _post_ack_exc:
|
||||
# Order is LIVE at the venue: unknown != flat. No rollback, no
|
||||
# synthetic REJECT — the E-feed FILL / reconcile settles the slot.
|
||||
# (See the async path below and venue.VenuePostAckError.)
|
||||
import logging as _log
|
||||
_log.getLogger(__name__).critical(
|
||||
"FIX(rust_backend): POST-ACK venue failure (%s) slot=%d action=%s — "
|
||||
"ORDER IS LIVE. NOT rolling back; awaiting E-feed FILL / reconcile.",
|
||||
_post_ack_exc, intent.slot_id, intent.action.value,
|
||||
)
|
||||
emitted_events = list(getattr(_post_ack_exc, "events", []) or [])
|
||||
except Exception as _submit_exc:
|
||||
# venue.submit() failed (e.g. BingX timeout). The Rust FSM already
|
||||
# advanced to ORDER_REQUESTED / ENTRY_WORKING with no corresponding
|
||||
# exchange order. Feed a synthetic REJECTED event so the FSM rolls
|
||||
# back to IDLE — otherwise the slot is stranded and every subsequent
|
||||
# ENTER with a different trade_id hits SLOT_BUSY forever.
|
||||
# PRE-ACK: venue.submit() failed (e.g. BingX timeout). The Rust FSM
|
||||
# already advanced to ORDER_REQUESTED / ENTRY_WORKING with no
|
||||
# corresponding exchange order. Feed a synthetic REJECTED event so the
|
||||
# FSM rolls back to IDLE — otherwise the slot is stranded and every
|
||||
# subsequent ENTER with a different trade_id hits SLOT_BUSY forever.
|
||||
import logging as _log
|
||||
_log.getLogger(__name__).error(
|
||||
"venue.submit failed (%s) — feeding synthetic REJECTED to roll back FSM slot=%d action=%s",
|
||||
@@ -1004,7 +1016,28 @@ class ExecutionKernel:
|
||||
emitted_events = await submit_async(intent)
|
||||
else:
|
||||
emitted_events = self.venue.submit(intent) # fallback: mock/test venue
|
||||
except VenuePostAckError as _post_ack_exc:
|
||||
# THE ORDER IS LIVE AT THE VENUE. This is UNKNOWN, not failure —
|
||||
# and unknown is NEVER flat. Rolling the FSM back here is what
|
||||
# orphaned 6 live SHORTs on 2026-07-13: the venue kept the
|
||||
# positions while the kernel believed it was flat, so no SL/TP/
|
||||
# ADVSL could ever fire on them.
|
||||
#
|
||||
# Emit NO synthetic event: leave the slot in its working state and
|
||||
# let execution truth settle it — the E-feed delivers FULL_FILL
|
||||
# from the account stream independently of this call path.
|
||||
import logging as _log
|
||||
_log.getLogger(__name__).critical(
|
||||
"FIX(rust_backend): POST-ACK venue failure (%s) slot=%d action=%s — "
|
||||
"ORDER IS LIVE. NOT rolling back (unknown != flat); awaiting E-feed "
|
||||
"FILL / reconcile to settle the slot.",
|
||||
_post_ack_exc, intent.slot_id, intent.action.value,
|
||||
)
|
||||
emitted_events = list(getattr(_post_ack_exc, "events", []) or [])
|
||||
except Exception as _submit_exc:
|
||||
# PRE-ACK failure (timeout, connection refused, rejected request):
|
||||
# the venue never took the order, so the FSM must roll back or the
|
||||
# slot is stranded in ORDER_REQUESTED forever.
|
||||
import logging as _log
|
||||
_log.getLogger(__name__).error(
|
||||
"venue.submit_async failed (%s) — synthetic REJECTED, FSM rollback slot=%d action=%s",
|
||||
|
||||
Reference in New Issue
Block a user