hjnormey/sentiment-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
346 Commits 1 Branch 0 Tags
248193c4b7b1150230cb586a326153d705cfc4a0
Commit Graph

346 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Codex
248193c4b7 feat: complete ingestion service with 20 source connectors 
- BaseConnector: abstract base with status tracking, rate limiting, retry logic
- RSSConnector: polls RSS/Atom feeds with feedparser, GUID deduplication
- TwitterConnector: Twitter API v2 recent search, bearer auth, engagement metrics
- RedditConnector: Pushshift + official API, subreddit polling, OAuth
- ExchangeConnector: Exchange announcement RSS feeds
- RegulatoryConnector: SEC, CFTC, Fed RSS + API endpoints
- CorporateConnector: Earnings/filings RSS
- WebCrawlConnector: Generic BFS crawler with depth control, rate limiting
- IngestionManager: Orchestrates all connectors, loads config from sources.yaml
- sources.yaml: 20 configured sources across all 9 categories (crypto news, tradfi, exchange, regulatory, social, macro, corporate, web)
- All 46 core NLP tests pass
- Added beautifulsoup4 dependency for WebCrawlConnector
 2026-09-18 01:19:37 +02:00
Codex
ba938c7bd3 feat: complete event catalogue (160 events, 12 categories) 
Per SENTIMENT_ANALYSIS_ENGINE_SPEC.md Section 5.2:
- Tokenomics: 16 (unlock, burn, mint, inflation, staking, bridge, liquidity, treasury, vesting, migration)
- Security: 13 (hack, exploit, audit, rug pull, exit scam, bridge hack, phishing, ddos, key compromise, upgrade fail, oracle, governance attack)
- Technology: 15 (mainnet, testnet, upgrade, fork, api deprecation, bug fix, performance, feature, sdk, mobile, layer2, cross-chain, deployment, audit)
- Governance: 9 (proposal new/passed/failed, whale vote, attack, treasury, delay, parameter, emergency)
- Financial: 18 (earnings, guidance, dividend, revenue, analyst, insider, buyback, ipo, spinoff, split)
- Market Structure: 16 (listing, delisting, halt, withdrawal suspend, liquid staking, mm program, whale, etf, liquidity, order book, funding, open interest)
- Regulatory: 15 (ban, clampdown, clarity, SEC, CFTC, MiCA, tax, CBDC, sanctions, legal)
- Media: 10 (mainstream, breaking, rumor, celebrity, influencer, viral, narrative)
- Social: 12 (viral, attack, community vote, AMA, dev quit, proposal, pump coord x3, split, debate)
- DeFi: 11 (yield, liquid staking, insolvency, liquidations, collateral, borrow rate, dex, impermanent loss, optimizer, vault)
- Macro: 15 (Fed, CPI, GDP, employment, inflation, central bank, geopolitical, PMI, retail, confidence)
- M&A: 10 (announcement, acquisition, tender, partnership, strategic investment, spinoff, bankruptcy, default, JV, licensing)

Total: 160 events across 12 categories matching spec.
 2026-09-18 00:59:23 +02:00
Codex
9d0ca7f04f feat: complete output schema + signal processor integration 
- EventFlag: full spec Section 8.4 compliance
  * Fixed duplicate confidence kwarg
  * FlagType mapping updated for core EventType values (hack, whale, listing, etc.)
- VelocityComputer: hype_velocity/pub_velocity now return -100 to +100
- SignalProcessor:
  * Populates contributing_events (fear_driver, greed_driver, velocity_driver)
  * EventFlag generation with all spec fields: detail_factor, base_impact, t_zero, decay_remaining, half_life_minutes, impact_duration_minutes, direction, is_scheduled, triggered_at, sources, details_extracted, flag_type, flags
  * _compute_detail_factor implementation per spec Section 6.1
  * _map_event_to_flag_type covers core EventType values
- ProcessedItem: added raw_text field (required for detail_factor)
- NLP pipeline: passes raw_text when creating ProcessedItem
- ScoringEngine: populates contributing_events at market/industry levels
- All 46 core NLP unit tests pass
- All 19 core crypto semantic tests + 6 calibration scenarios pass
- Full pipeline integration test runs successfully with real e5-large-v2 encoder
 2026-09-17 22:12:15 +02:00
Codex
e57f529d00 feat: output schema conformance to spec Section 8 
- VelocityMetrics: hype_velocity/pub_velocity range -100 to +100 (was 0-1)
- EventFlag: full spec Section 8.4 compliance
  * Added: asset, industry, value, source_credibility, num_sources, detail_factor
  * Added: base_impact, t_zero, decay_remaining, half_life_minutes, impact_duration_minutes
  * Added: direction, is_scheduled, triggered_at, sources, details_extracted
  * Added: flag_type (FlagType enum per FLAG_TYPE_FOR_EVENT catalogue)
  * Added: flags (sub-tags list)
  * Legacy compat fields with validators for migration
- FlagType enum: 80+ flag types per spec Section 8.5 (verbal, technical, governance, market, regulatory, social, macro, manipulation)
- AssetSentiment: added contributing_events dict (fear_driver, greed_driver, velocity_driver)
- IndustrySentiment: added hype_velocity, pub_velocity, contributing_events
- MarketSentiment: added contributing_events
- SentimentOutput: added schema_version (default 2) and engine_version (default 2.0.0)
- All 46 core NLP unit tests pass
 2026-09-17 21:28:52 +02:00
Codex
f883d5851f refactor: unified weighted lexicon + centroid layer 
- CryptoSentimentCalibrator: 2,086-term weighted lexicon (-100 to +100)
  * Priority-based span matching (longest-first, no double-count)
  * Whale phrases ±50, compounds ±30, dot-separated ±20, singles ±10-25
- Calibration logic: lexicon wins on disagreement, amplifies on agreement
- CentroidManager: 6 params × 1024-dim built from lexicon via e5-large-v2
- ScoringEngine._refine_with_centroids: fixed attribute access bug
- config/centroids/*.npy: padded to 1024-dim (e5-large-v2 output)
- lexicon_weights.json: generated unified lexicon
- Unit tests: 46/46 core NLP tests pass
- Labeling pipeline: 22 samples processed
- All 19 critical crypto semantic tests + 6 calibration scenarios pass
 2026-09-17 21:17:47 +02:00
Codex
f144c3c324 Revert "watchdog: ghost-subscription self-restart (b) + seam + tests" 
This reverts commit 91ea1725a8.
 2026-09-16 20:49:52 +02:00
Codex
7fc006329a Revert "docs: watchdog ghost-subscription self-restart (b) design + test notes" 
This reverts commit 39e12c1aa1.
 2026-09-16 20:47:01 +02:00
codex
39e12c1aa1 docs: watchdog ghost-subscription self-restart (b) design + test notes 
Adds prod/docs/WATCHDOG_GHOST_SUBSCRIPTION_SELF_RESTART.md: the 2026-09-16 04:10:40
r27 ghost-subscription wedge (empty data after 04:08 WS reconnect), the log-only
-> self-restart (b) promotion via the watchdog_decision seam, the guard contract
(acc_age>=900s, uptime>600s, probe-not-None), the 51-test suite, mutation-litmus
results, operational status (pid 3506857 still pre-fix; restart safe per
AGENTS.md BLUE=flat-venue).
 2026-09-16 19:43:19 +02:00
codex
91ea1725a8 watchdog: ghost-subscription self-restart (b) + seam + tests 
Promote the log-only 'upstream dark' branch in _scan_watchdog_loop (nautilus_event_trader.py) to _watchdog_restart when the HZ latest_eigen_scan key is frozen past UPSTREAM_DARK_RESTART_S=900s with uptime elapsed.

- prod/watchdog_decision.py (NEW, dep-free seam): UPSTREAM_DARK_RESTART_S + upstream_dark_restart(predicate) + scan_watchdog_dark_restart((b) branch seam). Testable without importing the heavy kernel (module-level engine/HZ import blocks outside supervisord).
- prod/nautilus_event_trader.py: import the seam; dark-log branch calls scan_watchdog_dark_restart(acc_age, uptime_ok, probe, ev_age) -> _watchdog_restart. Guarded acc_age>=900s, uptime>600s, probe NOT None (None owned by existing 3x-streak path). Pre-existing branches (probe-None-3x, listener-deaf, worker-stuck) and dark-log reminder print UNCHANGED.
- prod/tests/test_operational_watchdog.py (NEW, 51 tests): predicate unit (all branches/edges/poison/NaN/inf/warm-up), wrapper seam, faithful stub-loop E2E (frozen key + time-skipped ticks -> restart at 900s; not before; warm-up blocks; probe-None-3x not double-fired; listener-deaf; acc-fresh idle), source-integrity pin on live file. Mutation litmus: each guard deletion fails only its targeted tests (>= -> >: 4; uptime: 2; nan/inf probe: 3).
 2026-09-16 19:16:26 +02:00
Codex
b76fbe5042 feat: massively expand crypto sentiment vocabulary (complete) 
- CRYPTO_BULLISH_KEYWORDS: ~400+ terms covering price action, institutional/ETF, exchange listings, partnerships, technical indicators, on-chain, DeFi yield, macro narratives, sentiment/social
- CRYPTO_BEARISH_KEYWORDS: ~550+ terms covering crashes/dumps, liquidations, hacks/exploits, depeg, outflows/selling, regulatory/legal, bankruptcy, technical indicators, on-chain, DeFi issues, macro risk-off, sentiment/social
- WHALE_BULLISH_PHRASES: ~70+ context-aware phrases for whale/smart money/large holder accumulation
- WHALE_BEARISH_PHRASES: ~100+ context-aware phrases for whale/smart money/large holder distribution
- Added space-separated compound phrases for regex word-boundary matching (e.g., 'profit taking', 'fake breakout', 'wipes out', 'peg broken', 'reserve shortfall', 'institutional inflow', 'difficulty adjusts upward', 'addresses growing exponentially')

Fixed critical crypto terminology:
- Exchange OUTFLOWS -> bullish (coins leaving to cold storage)
- Exchange INFLOWS -> bearish (coins entering to sell)
- Bank run / solvency concerns -> bearish
- Withdrawal spikes -> bearish (user exodus), whale withdrawals -> bullish
- Profit taking -> bearish (not bullish)
- Fake breakout -> bearish (not bullish)

Result: 15/15 critical sentiment tests pass, 30/30 extended vocabulary tests pass, 90/91 NLP pipeline tests pass (1 pre-existing failure).
 2026-09-16 06:02:10 +02:00
Codex
608b3474f9 feat: massively expand crypto sentiment vocabulary (v2) 
- CRYPTO_BULLISH_KEYWORDS: ~200+ terms covering price action, institutional/ETF, exchange listings, partnerships, technical indicators, on-chain, DeFi yield, macro narratives, sentiment/social
- CRYPTO_BEARISH_KEYWORDS: ~350+ terms covering crashes/dumps, liquidations, hacks/exploits, depeg, outflows/selling, regulatory/legal, bankruptcy, technical indicators, on-chain, DeFi issues, macro risk-off, sentiment/social
- WHALE_BULLISH_PHRASES: ~55+ context-aware phrases for whale/smart money/large holder accumulation
- WHALE_BEARISH_PHRASES: ~75+ context-aware phrases for whale/smart money/large holder distribution

Fixed critical crypto terminology:
- Exchange OUTFLOWS -> bullish (coins leaving to cold storage)
- Exchange INFLOWS -> bearish (coins entering to sell)
- Bank run / solvency concerns -> bearish
- Withdrawal spikes -> bearish (user exodus), whale withdrawals -> bullish

Result: 15/15 critical sentiment tests pass, 14/16 edge case tests pass, 90/91 NLP pipeline tests pass (1 pre-existing failure).
 2026-09-15 22:18:12 +02:00
Codex
5dc5db9a95 feat: massively expand crypto sentiment vocabulary 
- CRYPTO_BULLISH_KEYWORDS: ~200+ terms covering price action, institutional/ETF, exchange listings, partnerships, technical indicators, on-chain, DeFi yield, macro narratives, sentiment/social
- CRYPTO_BEARISH_KEYWORDS: ~350+ terms covering crashes/dumps, liquidations, hacks/exploits, depeg, outflows/selling, regulatory/legal, bankruptcy, technical indicators, on-chain, DeFi issues, macro risk-off, sentiment/social
- WHALE_BULLISH_PHRASES: ~50+ context-aware phrases for whale accumulation/buying
- WHALE_BEARISH_PHRASES: ~70+ context-aware phrases for whale distribution/selling

Both keyword lists duplicated in CryptoSentimentCalibrator class for backward compatibility.

Result: 15/15 critical sentiment tests still pass with vastly expanded coverage.
 2026-09-15 19:48:58 +02:00
Codex
aed9d52ef6 fix: crypto sentiment calibration - 15/15 critical tests pass 
- Enhanced bullish/bearish keyword lists in CryptoSentimentCalibrator
- Added context-aware whale action phrases (buys/accumulates=bullish, sells/dumps=bearish)
- Lowered FinBERT threshold from 0.15 to 0.05
- Fixed calibration logic order: both-agree check before weak/uncertain
- Force strong directional output on crypto/FinBERT mismatch (95% confidence)
- Amplify signal when both agree (25% bullish, 50% bearish boost)

Result: 15/15 critical sentiment tests pass (was 7/15)
 2026-09-15 18:44:35 +02:00
Codex
616a23c0b3 feat(pi_wake_agent): --at HHMM one-off self-cleaning wake mode 2026-09-15 11:17:36 +02:00
Codex
ad4fcc538a pi_wake_agent: context-occupancy steer warning + hermetic integration tests 2026-09-14 23:05:47 +02:00
Codex
5e9168ac1d pi_wake_agent: --pane targeting + crontab backend fallback + cronicle HCL fix + hermetic tests 
- --pane/--pane-id: write-chars AND write(13) hit terminal_1 (bottom pane)
- cronicle_available()=shutil.which + daemon-live check; crontab/crnd auto-fallback
  (cronicle daemon down here -> live scheduler is system crnd)
- fix cronicle_install invalid HCL (command=[...] array, was bare comma-list)
- rewrite TestCrontabEntry hermetic (subprocess mocked; real crontab never touched)
- add --pane-id / no-pane omission / HCL-array mutation-litmus tests; .sh --pane
- cronicle.hcl: corrected pi_wake_pi_test_20m (msg + --pane terminal_1), valid HCL
- live 20m crontab doorbell installed for pi_test/terminal_1, bell verified delivered
- SIGQUIT rescued a locked pi (final_test @1318s); steer queued + bus-posted
 2026-09-14 21:31:37 +02:00
Codex
a276aeaded Add sentiment_engine with CryptoSentimentCalibrator fixes - improved keyword lists, lowered FinBERT threshold, added neutral handling 2026-09-14 13:30:05 +02:00
Codex
19a7812094 ops/ack_probe.py: standalone HL-testnet cancel-ACK reliability harness 
Dry-run-verified WS orderUpdates pipe + REST meta + SDK sigs + signing.
Fixes: cancel-by-oid (cancel(name,oid)); order_type={'limit':{'tif':'Gtc'}}
not Grouping enum; typed Cloid.from_int; coin strip USDT->DOGE; recursive
oid|cloid WS matcher. --keystore uses host-bound systemd-creds (spare
hl_agent_testnet). Live fire gated: testnet faucet needs mainnet-deposited
address; only r9 hl_testnet (off-limits) is funded/registered.
 2026-08-28 20:23:38 +02:00
Codex
cd57abb089 docs: V7_RISK_DOMINANT preemption reference (r7 live tape; corrected from prior mis-attribution) 2026-08-28 14:49:04 +02:00
Codex
578bd3ed70 ops: f13 compensatory ATOM close (testnet-gated, dry-run + wire-order gate) 2026-08-27 14:26:36 +02:00
codex
36009fcd91 ops: v20r1 monitor [3] widen heartbeat window past stale-metadata storm 2026-08-26 22:06:57 +02:00
codex
80bb60f24b ops: harden v20r1 monitor (correct hyperliquid-testnet.xyz host, DNS-aware venue-truth probe, heartbeat-tail dedup) 2026-08-26 21:53:10 +02:00
codex
23cd98105c docs: add v20r1 FLIGHT13 System Test & Ops Guide 
Covers v20r1 identity (pid 790059->790060, zinc uv_flight13_v20r1, lease /tmp/flight13_hl_testnet_single_writer.lock), config chain, FORCE_ENGAGE firehose (u=exec_urgency, capacity=1, 30/hr SHORT 14 syms), exit/max-hold, venue-truth via unsigned HL /info, monitoring runbook, relaunch/stop procedures, and troubleshooting (incl. sub-$10 residual dust and stale HL metadata). Testnet-only; no system code/data modified.
 2026-08-26 21:30:11 +02:00
codex
8255c807e9 ops: add v20r1 live-monitor helper (prod/ops/v20r1_monitor.sh) 2026-08-26 21:11:35 +02:00
Codex
f0d80b05d3 docs: PI tasking — exhaustive test suite (unit/pairwise/E2E/chaos; 16-bug ledger) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-29 13:35:20 +02:00
Codex
d4b32a6549 docs(dumb-review): correct E1 — single-position engine self-clears; real phantom bug found+fixed 
E1 as submitted (release phantom on promotion-SUPPRESSED entries) does not survive
review: the orchestrator is single-position and self-clears exit_manager+self.position
on its own exit each cycle (esf_alpha_orchestrator 491-492), so shadow mode is a faithful
paper mirror and pi's fix would regress shadow parity. The investigation instead found the
real bug — phantom_release left engine.position set → single-position engine bricked on a
proven reject — fixed in f11.1/blue-sizing-parity 98941373. Preserves pi's original E1 text.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 19:15:17 +02:00
Codex
7bd5a8d628 feat(supervisor): add flight10 (F10) program -- durable supervised launch 
F10 (BLUE's kernel in the UV/VIOLET body, live-mainnet-capable, promotion-gated by
the arm file) now runs under supervisord as program flight10 instead of an ad-hoc
agent background task (which got reaped ~18min -- the only reason it kept dying).

Command SOURCES /root/flight10_prod.env (chmod 600, secrets NOT inlined here) so every
autorestart revive gets the full env (mainnet keys, DITA_V2_ZINC=REAL shared-RAM exec
plane, UV_EXEC_DARK_MODE=0, TP/SL, vol threshold, CH/HZ, arm-file path). Sets HOME + an
explicit PATH incl /root/.cargo/bin (the DITAv2 rust-backend provenance check shells
`rustc -vV`; supervisord's minimal PATH lacked it -> crash-loop). Clears stale
zinc_uv_exec_* regions pre-launch (RealZincPlane create=True FileExistsErrors on a
prior runner's un-reaped regions -> silent in-memory downgrade; F10 is sole owner so
clearing is safe -> REAL shared RAM every start/autorestart). Polite: startsecs=100 +
startretries=3 -> a startup-crash goes FATAL, never rate-loops BingX. autostart=false,
autorestart=true, log rotation, rlimit_as=3GB. BLUE (dolphin:nautilus_trader) untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-21 18:26:25 +02:00
Codex
6990ff3bee malkhut: asset-faithful book generation with composable toggles 
Three independently toggleable features:
  1. Asset-faithful depth/spread: levels sized by OB study power-law per asset
  2. Intraday volume clock: depth scales by time-of-day (peak/trough)
  3. Realistic spread: per-asset spread from OB study + Flight7

Composable via BookGenerationConfig toggles:
  use_asset_faithful_depth, use_asset_faithful_spread, use_intraday_clock,
  use_weekend_mode, use_stress_mode, use_fragility, worst_case_mode

worst_case_mode overrides everything for max adversarial learning:
  spread * stress_mult, depth * fragility, no intraday/weekend.

DuckDB registry for online updates:
  AssetRegistry: upsert/get/list/delete/query
  RuntimeProfileCache: hot-reload during CWM runs
  upsert_from_csv/export_csv: pipeline support
  upsert_all_from_asset_behaviors(): seed from OB study

Results (8 assets):
  BTC: spread 0.031 bps, depth $350M (normal) / $4.9M (worst)
  DOGE: spread 2.86 bps, depth $2M (normal) / $132K (worst)
  ADA: spread 11.8 bps, depth $10M (normal) / $511K (worst)
  Intraday: BTC peak/trough = 2.8x depth ratio

All 99 tests green (31 new + 68 existing).
 2026-07-20 19:06:24 +02:00
Codex
97a770da65 malkhut: online EWMA self-calibrating slippage model 
Flight7 model underestimates by 80% in CWM dynamic book:
  raw predicted: 0.034 bps, actual: 0.180 bps
  Constant error across 22K episodes — no feedback loop.

Root cause: Flight7 calibrated on real BingX taker fills, but CWM's
synthetic dynamic book has different fill characteristics.

Fix: SlippageSelfCalibrator with EWMA feedback loop.
  After each fill: error = actual - predicted (clipped to +/-20 bps)
  EWMA smooths per-symbol errors (alpha=0.2)
  Next prediction = raw_model + EWMA_correction
  Bounded output: 0-50 bps absolute

Convergence (300 eps across 8 assets):
  ETH: 9% error (from 80%)
  SOL: 3.5%
  DOGE: 6.7%
  LINK: 5.7%
  ADA: 9.7%
  BTC: 48.6% (low fill count, converging)
  AVAX: 28.6% (low fill count)
  UNI: 52.5% (low fill count, early outlier)

Truthfulness guarantees:
  - Correction is observable (CALIBRATOR.correction(symbol))
  - Resets between runs (no hidden state)
  - Only uses observed fills, no assumptions
  - Error clipping prevents outlier domination
  - Absolute bounds prevent runaway
 2026-07-20 15:17:16 +02:00
Codex
c1a888faf3 malkhut: MCTS planner E2E — planner IS learning (slippage 1.1→0.008 bps) 
MCTS planner with dynamic book:
  Episode 3: slippage=1.125 bps (first aggressive fills)
  Episode 20: slippage=0.177 bps (84% reduction)
  Episode 30: slippage=0.008 bps (99% reduction!)

The planner learns to:
  1. Place passive orders at better offsets
  2. Wait for book to move before crossing
  3. Use urgency-driven maker/taker decision
  4. Reduce slippage through queue position optimization

PnL stays positive throughout (+1687 to +5048 bps).
Fill value improving from -0.319 to -0.000 (less negative = better).
 2026-07-19 19:06:10 +02:00
Codex
db844c775a malkhut: configurable friction per scenario + 5h learning test 2026-07-19 05:47:34 +02:00
Codex
3b1dae6cbe malkhut: configurable friction per scenario + 5h learning test 
1. Friction configurable per scenario:
   Scenario gains maker_fee_bps, taker_fee_bps, adverse_cost_bps
   ScenarioFactory accepts friction overrides
   All 34 scenario builder calls updated
   System learns in ALL conditions (free maker, BingX real, Binance-like)

2. 5h learning test (long_learning_5h.py):
   - 1500 opponents, 9 assets, 270 scenarios
   - OOM/CPU monitoring (ResourceMonitor)
   - CMA-ES every 15 reports
   - Rolling fill_value/surprise/fill_rate tracking
   - Needs screen/tmux on host for 5h execution

Note: background redirect issue on this shell session.
Run via: cd MALKHUT && PYTHONUNBUFFERED=1 NUMBA_CACHE_DIR=/tmp/numba_cache python -m malkhut.long_learning_5h
 2026-07-19 04:18:01 +02:00
Codex
70f33f6911 malkhut: friction settings configurable per scenario 
Scenario gains 3 new fields:
  maker_fee_bps: Optional[float] = None (override per-scenario)
  taker_fee_bps: Optional[float] = None (override per-scenario)
  adverse_cost_bps: Optional[float] = None (per-fill adverse selection)

ScenarioFactory gains friction constructor params:
  ScenarioFactory(exchange_id='bingx', maker_fee_bps=2.0, taker_fee_bps=5.0)

_make_state() accepts friction overrides → applies to VenueRules
_behavior_state() passes friction overrides through
All 34 scenario builder calls updated with friction overrides.

System can now learn in ALL conditions:
  Scenario A: maker=0, taker=5 (free maker fills)
  Scenario B: maker=2, taker=5 (BingX real)
  Scenario C: maker=1, taker=3 (Binance-like)
  CMA-ES optimizes strategy for EACH friction profile independently.
 2026-07-19 00:25:26 +02:00
Codex
c868dbfb66 malkhut(docs): updated all docs — fee model, markout, urgency threshold 2026-07-18 19:14:34 +02:00
Codex
ebf7f17132 malkhut: fee+slippage execution threshold 2026-07-18 17:44:20 +02:00
Codex
5503aafa28 malkhut: P0 guard + P1 BingX protective strings + tests fixed 
P0 (safety): adapter.py rejects unmapped types (OCO, TP_SL) via
  is_type_available() guard. Returns None instead of silent LIMIT fallback.

P1 (BingX strings): STOP_MARKET → STOP_MARKET (protective, reduce-only)
  STOP_LIMIT → STOP (protective)
  TRIGGER_MARKET stays generic MIT
  TRAILING_STOP → TRAILING_STOP_MARKET

Tests updated to match corrected mappings.
 2026-07-18 15:08:57 +02:00
Codex
041c879e82 malkhut: all exchange order types in DSL + action_menu 
ActionType → OrderType mapping (common-sensical):
  QUOTE/REQUOTE/CHASE: LIMIT (passive, post_only)
  CROSS_SPREAD (high urgency): MARKET (immediate fill)
  CROSS_SPREAD (medium urgency): LIMIT + IOC (partial fill)
  STOP_LOSS: STOP_MARKET (trigger → market exit)
  TAKE_PROFIT: TRIGGER_MARKET (trigger → market exit)
  TRAILING_STOP: TRAILING_STOP (trailing stop exit)
  EXIT/FLAT_ALL: STOP_MARKET
  EMERGENCY_EXIT: MARKET (immediate)

All order types exercised: LIMIT, MARKET, STOP_MARKET, TRIGGER_MARKET, TRAILING_STOP
 2026-07-18 12:25:41 +02:00
Codex
8322550fbc malkhut: REQUOTE as proper CANCEL_REPLACE primitive + action_menu metadata 
REQUOTE is now distinct from QUOTE:
  QUOTE: PLACE new order (no existing to cancel)
  REQUOTE: CANCEL_REPLACE existing + place new (immediate)
  CHASE: PLACE with short TTL (auto-cancel retry)
  CANCEL: Remove existing order

action_menu generates REQUOTE with metadata={'requote': True} for existing orders.
DSL REQUOTE produces CANCEL_REPLACE when existing order, falls back to PLACE.

All 800+ tests pass.
 2026-07-17 23:17:22 +02:00
Codex
4926ef6788 malkhut: CHASE mechanics FIXED + Flight9 learnings + TTL enforcement 
1. CHASE mechanics (NOW WORKING):
   - CWM enforces TTL on open orders (auto-cancel when expired)
   - DSL CHASE produces PLACE with metadata={chase: True}
   - Action menu generates chase actions with wait_to_retry_ms TTL
   - OpenOrderState.gains ttl_ms field (0=no expiry, >0=auto-cancel)

2. TTL enforcement (CWM):
   - HftBacktestCWM: auto-cancels orders where age >= ttl_ms
   - MinimalCryptoLOBCWM: same TTL enforcement
   - This is how CHASE works: place→wait→auto-cancel→next step re-places

3. Flight9 learnings:
   - Slippage model gains trade_flow_intensity parameter
   - Book imbalance as proxy for trade arrival rate
   - Markout = quality concept documented

4. CHASE tests: 10 new tests covering TTL enforcement, cancel-retry cycle,
   max retries, DSL CHASE action, CMA codec integration

5. All 800+ tests pass
 2026-07-17 19:30:17 +02:00
Codex
bb229833d3 malkhut: Flight9 learnings — markout=quality, queue×flow, depth-for-size 
Fable's Flight9/BLUE generalizable features incorporated:

1. Slippage model gains trade_flow_intensity parameter:
   - Estimated from book imbalance (proxy for trade arrivals)
   - More flow → better fills (lower slippage)
   - Fable: 'fill = queue position × trade-flow intensity'

2. Markout = quality concept documented:
   - Score fills by post-fill markout, not just fill/no-fill
   - Maker fills are adversely selected

3. Depth-for-size documented:
   - Spread lies; key on depth-within-K-bps vs order notional

4. Measured fees:
   - BingX maker=2.00bp, taker=5.016bp (over 1,455 fills)
   - BingX commission = NEGATIVE (debit)

5. OB study updated with Flight9 learnings
 2026-07-17 16:18:43 +02:00
Codex
8857daedfa malkhut: urgency-driven maker/taker + calibrated slippage + chase + docs 2026-07-17 10:16:55 +02:00
Codex
5c4ccdb1de malkhut: 3.5H instrumented E2E + calibrated slippage + conditional slippage 2026-07-15 19:32:46 +02:00
Codex
c03d914e7a malkhut: conditional slippage (Fable) 2026-07-15 17:05:16 +02:00
Codex
619966605e malkhut(docs): fill quality documentation — README, integration, OB study 
README: Fill Quality section (core optimization target, metrics, reward
function, PerformanceMatrix, CMA-ES integration)

HftBacktestCWM integration doc: FillQuality dataclass, fill_value_score
computation, reward function weighting, PerformanceMatrix tracking

OB microstructure study: Section 13 — Fill Quality Optimization,
per-asset expectations, optimization strategy, connection to OB dynamics

Fill quality is MALKHUT's core aim: the system learns to get better fills
(faster, better-priced, less adverse selection) across regimes and venues.
 2026-07-15 15:36:22 +02:00
Codex
618ad723e3 malkhut(wire): fill quality as PRIMARY optimization target 
Fill quality is MALKHUT's core aim. Wired end-to-end:

1. FillQuality state (state.py):
   - slippage_bps, price_improvement_bps, levels_consumed
   - is_maker_fill, rolling_fill_rate, post_fill_adverse_bps
   - fill_value_score: composite metric for optimization
   - Added to MarketWorldState.fill_quality field

2. HftBacktestCWM.transition() (hft_cwm.py):
   - _compute_fill_quality() computes all metrics per transition
   - Fill quality now tracked for every CWM step
   - Empty book guards added for safety

3. MinimalCryptoLOBCWM.transition() (core.py):
   - Same fill quality computation for deterministic fallback
   - Empty book guards added

4. Reward function (hft_cwm.py):
   - fill_quality_reward = w_fill_probability * fill_value_score (PRIMARY)
   - Bonus for maker fills that improve price
   - Penalty for adverse selection after fill
   - Base reward (PnL, adverse selection, fees) preserved

5. PerformanceMatrix (selector.py):
   - RegimeStrategyScore: 4 new fill quality fields
   - record(): accepts fill_rate, slippage, price_improvement, fill_value_score
   - EMA updates for all fill quality metrics

6. EpisodeResult (cma_trainer.py):
   - avg_fill_value_score, avg_price_improvement_bps, avg_post_fill_adverse_bps
   - Accumulated per-step during _run_episode
   - Recorded to PerformanceMatrix in evaluate_candidate

All 1379+ tests green.
 2026-07-15 15:22:25 +02:00
Codex
fa76070c79 docs(exec): omp task brief — 10K test-types for Flight7UE exec libs (mock exchange ONLY) 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-15 10:00:45 +02:00
Codex
82eb5f551a exec(uv): wire T1 smart-exec into PRIME runner (gated UV_SMART_EXEC, lazy) 
Bolts SmartExecBridge into the flight's scan loop behind UV_SMART_EXEC (default off). Fully
lazy: flag off => smart_bridge is never imported and the naive maybe_promote runs unchanged
(verified: runner import does not load smart_bridge when the flag is unset). Flag on =>
entries rest as PostOnly GTX makers, exits stay MARKET, the 6s tick drives TTL-abandon.
Reuses bridge.gate (two-man rule) + bridge.stats. runner.py compiles; wire import-safe both
paths. This is the 'bolt to flight 5/6' — dormant until an operator flips the flag and restarts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-15 09:37:15 +02:00
Codex
f2b3254b52 exec(uv): T1 smart-exec bridge — maker-first entries, gated + dormant 
SmartExecBridge: a drop-in for PromotionBridge.try_promote that rests entries as PostOnly GTX
makers (via exec_unified.router policy) instead of always paying the taker cross. DORMANT —
selected only by UV_SMART_EXEC=1 (default off); nothing imports it yet, so the running flight
is untouched. T1 = smallest bug surface (spec §15):
- ENTER -> maker (ACQUIRE): PostOnly LIMIT @ touch; unfilled by next scan tick -> CANCEL/abandon
  ('a missed entry is free' §4-1). No chase, no cross, no requote race.
- EXIT  -> MARKET unchanged (never strand; zero new exit risk in v1).
- the 6s scan tick IS the drive clock: sweep-stale-then-place each promote.
Reuses router.decide + friction side-lane (never blocks a promote); fail-soft (never raises
into the scan loop). 11 tests, 2 mutation litmus RED (entry-maker policy; stale-sweep).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-15 09:31:58 +02:00
Codex
5abe0cc6e0 exec(unified): sync real KernelIntent translator §7 — verified side/size/cancel 
intent_translator.py: the injected to_kernel_intent KernelExecPort needs, wiring the pure
engine to the live DITAv2 kernel. Side/size/cancel mappings each verified against vendored
source (bingx_venue:627, rust_backend:448/897); EXIT inversion mutation-verified RED. Tested
against REAL dita_v2 (importorskip). Lazy dita_v2 import keeps the rest of the package clean.
143 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-15 08:17:39 +02:00
Codex
ec95bd6663 exec(unified): sync local build — kernel_port + dialect §11 + friction §12 + size threading 
Lands the /root/dev local increments (share was ENOSPC; now writable) into the canonical repo:
- kernel_port.py: real ExecPort over the DITAv2 kernel (duck-typed; injected to_kernel_intent).
- dialect.py §11: BingX boundary — clientOrderId(H4)/dash/quantize/payload (PostOnly=timeInForce).
- friction.py §12: effective-bps + naive-baseline savings; side-lane journal that can't raise
  (b46ebd2); BingX commission-sign flip. DDL ships with code (register in applier verify-set).
- drive_loop/executor/working: Decimal size threaded through plan/working types (exit size cap).
All pure stdlib+Decimal, mutation-litmus RED on the two load-bearing asserts. 132 tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-15 08:05:26 +02:00