hjnormey/sentiment-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
359 Commits 1 Branch 0 Tags
c4c8ed7c9f4b1cdf4a508f500a11f66dc3e86b1e
Commit Graph

359 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Codex
c4c8ed7c9f feat(sentiment): register_source upsert fix + connector factory fix for web:telegram: 2026-09-25 11:46:56 +02:00
Codex
5c692cda74 feat(sentiment): Telegram preview scraper + connector factory fix 2026-09-25 11:40:58 +02:00
Codex
ed6a004ef0 feat(sentiment): lexicon expansion with 100+ asset-specific terms + bearish keywords 2026-09-25 11:37:53 +02:00
Codex
a5542e67b7 feat(sentiment): pipeline retune with Telegram preview + lexicon expansion 2026-09-25 11:34:48 +02:00
Codex
41dd1aefdd docs: jev OB-state outsmart study (buildState→SRC-01 mapping + EV gate) 2026-09-23 12:17:46 +02:00
Codex
d69e82200a feat(sentiment): add 50+ social media sources for small-cap coverage 
- Reddit: 15 new subreddits (altcoin, CryptoMoonShots, SatoshiStreetBets, defi, CryptoMarkets + 11 project-specific)
- Twitter: 10 search queries for trade assets (ZIL, ONE, STX, ALGO, DASH, LTC, FET, XTZ, ENJ + combined)
- Telegram: 11 official project channels
- Discord: 11 official project servers
- All configured with appropriate credibility/relevance scores for small-cap coverage
 2026-09-23 10:09:57 +02:00
Codex
c0445e60ea feat(sentiment): add trade asset aliases & known entities; add Glassnode/WSJ RSS sources 
- Added 15 trade assets (ZIL, ONG, ONE, STX, ALGO, DASH, LTC, FET, XTZ, ENJ, XLM, ETC, TRX) to asset_aliases.yaml
- Added same assets with metadata to known_entities.yaml (chain, type, market_cap_rank)
- Added Glassnode Insights (0.85 cred) and WSJ Crypto (0.85 cred) to sources.yaml
- Trade log analysis shows these small-caps had zero news coverage - aliases enable entity extraction
 2026-09-23 10:00:37 +02:00
Codex
0b0ac7a6ef docs: jev-trader exec-flow study + FLIGHT13 DITAv3.00/ASEx comparison 2026-09-22 18:56:34 +02:00
Codex
562d641f9c feat: velocity computation fixes + ProcessedItem metadata field 
- Fixed VelocityComputer._clean_window to handle both dict (asset_windows) and float (source_windows) entries
- Added metadata field to ProcessedItem schema for engagement data
- Fixed VelocityComputer._clean_window to handle both dict and float entries
- All 46 core NLP tests pass
- Full pipeline integration test passes with real e5-large-v2 encoder
 2026-09-18 02:59:18 +02:00
Codex
5ea9a09a89 feat: enhanced credibility scoring per spec Section 4.6 
- CredibilityScorer: composite score with weighted components (source_base 30%, content_quality 25%, engagement_authenticity 20%, cross_source 15%, historical 10%)
- score_source: base credibility from registry
- score_content_quality: length, structure, metadata quality heuristics
- score_engagement_authenticity: bot detection via engagement ratios (like/view, retweet/view, reply/view rates)
- score_cross_source_corroboration: clustering by content similarity (Jaccard n-grams), unique source count in consensus cluster
- _content_hash: MD5 normalization for deduplication
- _text_similarity: Jaccard similarity on word trigrams
- _cluster_by_similarity: clusters items by similarity to query text
- All 46 core NLP tests pass
 2026-09-18 02:35:30 +02:00
Codex
ec217b968e feat: sentence pattern matching per IMPLEMENT_GUIDE 
- 68 regex patterns across 6 parameters (fear_state: 21, greed_state: 19, hype_velocity: 5, pub_velocity: 4, pump_score: 8, dump_score: 11)
- Each pattern has weight (0.8-1.4) for importance scoring
- Patterns cover direct terms, market vernacular, event triggers, technical indicators, on-chain signals, macro, social signals
- Pre-compiled for efficiency with score_text_with_patterns() function
- score_text_with_patterns(text, param) returns weighted match score
- All 46 core NLP tests pass
 2026-09-18 02:02:24 +02:00
Codex
e6ecbfa2a6 feat: cross-source intelligence + bot detection + velocity EMA 
- MultiSourceFusion: cross-source intelligence per spec Section 6.4-6.6
  * 8 source clusters (crypto_native, tradfi, exchange_ann, regulatory, social_twitter, social_reddit, corporate, web_crawl)
  * NUM_SOURCES factor: 1=1.0x, 2=1.5x, 3=2.0x, 4+=2.5x + cross-cluster bonus (+0.25x/cluster)
  * Cross-cluster confirmation bonus (+20% event strength)
  * Source credibility: base × recency decay (3-day half-life) × author trust
- DetailFactorDetector: enhanced per spec Section 6.1
  * Regex patterns for dates, amounts, contract addresses, named individuals, technical terms, official URLs
- Bot Detection per Section 6.6:
  * Echo chamber detection (content replication across ≥3 sources)
  * Coordinated manipulation detection (≥5 similar posts simultaneously)
  * Bot pattern detection: template language, excessive emoji, capitalization
- VelocityComputer: EMA smoothing (α=0.3 per spec)
  * hype_velocity: log-derivative of attention-weighted mentions
  * pub_velocity: log-derivative of publication count
  * EMA smoothing with α=0.3 per spec
- All 46 core NLP tests pass
 2026-09-18 01:51:33 +02:00
Codex
8a182da0d8 feat: output sinks (Hazelcast + ClickHouse) 
- SinkConfig: unified config for both sinks
- HazelcastSink: ExF map writes for market snapshot + per-asset
- ClickHouseSink: exf_data table with flattened schema, buffered inserts
- OutputSinkManager: unified manager for both sinks
- Per-spec Section 13: market_update_interval_seconds=60, asset_update_interval_seconds=5
- clickhouse-connect + hazelcast-python-client dependencies
- All 46 core NLP tests pass
 2026-09-18 01:29:46 +02:00
Codex
248193c4b7 feat: complete ingestion service with 20 source connectors 
- BaseConnector: abstract base with status tracking, rate limiting, retry logic
- RSSConnector: polls RSS/Atom feeds with feedparser, GUID deduplication
- TwitterConnector: Twitter API v2 recent search, bearer auth, engagement metrics
- RedditConnector: Pushshift + official API, subreddit polling, OAuth
- ExchangeConnector: Exchange announcement RSS feeds
- RegulatoryConnector: SEC, CFTC, Fed RSS + API endpoints
- CorporateConnector: Earnings/filings RSS
- WebCrawlConnector: Generic BFS crawler with depth control, rate limiting
- IngestionManager: Orchestrates all connectors, loads config from sources.yaml
- sources.yaml: 20 configured sources across all 9 categories (crypto news, tradfi, exchange, regulatory, social, macro, corporate, web)
- All 46 core NLP tests pass
- Added beautifulsoup4 dependency for WebCrawlConnector
 2026-09-18 01:19:37 +02:00
Codex
ba938c7bd3 feat: complete event catalogue (160 events, 12 categories) 
Per SENTIMENT_ANALYSIS_ENGINE_SPEC.md Section 5.2:
- Tokenomics: 16 (unlock, burn, mint, inflation, staking, bridge, liquidity, treasury, vesting, migration)
- Security: 13 (hack, exploit, audit, rug pull, exit scam, bridge hack, phishing, ddos, key compromise, upgrade fail, oracle, governance attack)
- Technology: 15 (mainnet, testnet, upgrade, fork, api deprecation, bug fix, performance, feature, sdk, mobile, layer2, cross-chain, deployment, audit)
- Governance: 9 (proposal new/passed/failed, whale vote, attack, treasury, delay, parameter, emergency)
- Financial: 18 (earnings, guidance, dividend, revenue, analyst, insider, buyback, ipo, spinoff, split)
- Market Structure: 16 (listing, delisting, halt, withdrawal suspend, liquid staking, mm program, whale, etf, liquidity, order book, funding, open interest)
- Regulatory: 15 (ban, clampdown, clarity, SEC, CFTC, MiCA, tax, CBDC, sanctions, legal)
- Media: 10 (mainstream, breaking, rumor, celebrity, influencer, viral, narrative)
- Social: 12 (viral, attack, community vote, AMA, dev quit, proposal, pump coord x3, split, debate)
- DeFi: 11 (yield, liquid staking, insolvency, liquidations, collateral, borrow rate, dex, impermanent loss, optimizer, vault)
- Macro: 15 (Fed, CPI, GDP, employment, inflation, central bank, geopolitical, PMI, retail, confidence)
- M&A: 10 (announcement, acquisition, tender, partnership, strategic investment, spinoff, bankruptcy, default, JV, licensing)

Total: 160 events across 12 categories matching spec.
 2026-09-18 00:59:23 +02:00
Codex
9d0ca7f04f feat: complete output schema + signal processor integration 
- EventFlag: full spec Section 8.4 compliance
  * Fixed duplicate confidence kwarg
  * FlagType mapping updated for core EventType values (hack, whale, listing, etc.)
- VelocityComputer: hype_velocity/pub_velocity now return -100 to +100
- SignalProcessor:
  * Populates contributing_events (fear_driver, greed_driver, velocity_driver)
  * EventFlag generation with all spec fields: detail_factor, base_impact, t_zero, decay_remaining, half_life_minutes, impact_duration_minutes, direction, is_scheduled, triggered_at, sources, details_extracted, flag_type, flags
  * _compute_detail_factor implementation per spec Section 6.1
  * _map_event_to_flag_type covers core EventType values
- ProcessedItem: added raw_text field (required for detail_factor)
- NLP pipeline: passes raw_text when creating ProcessedItem
- ScoringEngine: populates contributing_events at market/industry levels
- All 46 core NLP unit tests pass
- All 19 core crypto semantic tests + 6 calibration scenarios pass
- Full pipeline integration test runs successfully with real e5-large-v2 encoder
 2026-09-17 22:12:15 +02:00
Codex
e57f529d00 feat: output schema conformance to spec Section 8 
- VelocityMetrics: hype_velocity/pub_velocity range -100 to +100 (was 0-1)
- EventFlag: full spec Section 8.4 compliance
  * Added: asset, industry, value, source_credibility, num_sources, detail_factor
  * Added: base_impact, t_zero, decay_remaining, half_life_minutes, impact_duration_minutes
  * Added: direction, is_scheduled, triggered_at, sources, details_extracted
  * Added: flag_type (FlagType enum per FLAG_TYPE_FOR_EVENT catalogue)
  * Added: flags (sub-tags list)
  * Legacy compat fields with validators for migration
- FlagType enum: 80+ flag types per spec Section 8.5 (verbal, technical, governance, market, regulatory, social, macro, manipulation)
- AssetSentiment: added contributing_events dict (fear_driver, greed_driver, velocity_driver)
- IndustrySentiment: added hype_velocity, pub_velocity, contributing_events
- MarketSentiment: added contributing_events
- SentimentOutput: added schema_version (default 2) and engine_version (default 2.0.0)
- All 46 core NLP unit tests pass
 2026-09-17 21:28:52 +02:00
Codex
f883d5851f refactor: unified weighted lexicon + centroid layer 
- CryptoSentimentCalibrator: 2,086-term weighted lexicon (-100 to +100)
  * Priority-based span matching (longest-first, no double-count)
  * Whale phrases ±50, compounds ±30, dot-separated ±20, singles ±10-25
- Calibration logic: lexicon wins on disagreement, amplifies on agreement
- CentroidManager: 6 params × 1024-dim built from lexicon via e5-large-v2
- ScoringEngine._refine_with_centroids: fixed attribute access bug
- config/centroids/*.npy: padded to 1024-dim (e5-large-v2 output)
- lexicon_weights.json: generated unified lexicon
- Unit tests: 46/46 core NLP tests pass
- Labeling pipeline: 22 samples processed
- All 19 critical crypto semantic tests + 6 calibration scenarios pass
 2026-09-17 21:17:47 +02:00
Codex
f144c3c324 Revert "watchdog: ghost-subscription self-restart (b) + seam + tests" 
This reverts commit 91ea1725a8.
 2026-09-16 20:49:52 +02:00
Codex
7fc006329a Revert "docs: watchdog ghost-subscription self-restart (b) design + test notes" 
This reverts commit 39e12c1aa1.
 2026-09-16 20:47:01 +02:00
codex
39e12c1aa1 docs: watchdog ghost-subscription self-restart (b) design + test notes 
Adds prod/docs/WATCHDOG_GHOST_SUBSCRIPTION_SELF_RESTART.md: the 2026-09-16 04:10:40
r27 ghost-subscription wedge (empty data after 04:08 WS reconnect), the log-only
-> self-restart (b) promotion via the watchdog_decision seam, the guard contract
(acc_age>=900s, uptime>600s, probe-not-None), the 51-test suite, mutation-litmus
results, operational status (pid 3506857 still pre-fix; restart safe per
AGENTS.md BLUE=flat-venue).
 2026-09-16 19:43:19 +02:00
codex
91ea1725a8 watchdog: ghost-subscription self-restart (b) + seam + tests 
Promote the log-only 'upstream dark' branch in _scan_watchdog_loop (nautilus_event_trader.py) to _watchdog_restart when the HZ latest_eigen_scan key is frozen past UPSTREAM_DARK_RESTART_S=900s with uptime elapsed.

- prod/watchdog_decision.py (NEW, dep-free seam): UPSTREAM_DARK_RESTART_S + upstream_dark_restart(predicate) + scan_watchdog_dark_restart((b) branch seam). Testable without importing the heavy kernel (module-level engine/HZ import blocks outside supervisord).
- prod/nautilus_event_trader.py: import the seam; dark-log branch calls scan_watchdog_dark_restart(acc_age, uptime_ok, probe, ev_age) -> _watchdog_restart. Guarded acc_age>=900s, uptime>600s, probe NOT None (None owned by existing 3x-streak path). Pre-existing branches (probe-None-3x, listener-deaf, worker-stuck) and dark-log reminder print UNCHANGED.
- prod/tests/test_operational_watchdog.py (NEW, 51 tests): predicate unit (all branches/edges/poison/NaN/inf/warm-up), wrapper seam, faithful stub-loop E2E (frozen key + time-skipped ticks -> restart at 900s; not before; warm-up blocks; probe-None-3x not double-fired; listener-deaf; acc-fresh idle), source-integrity pin on live file. Mutation litmus: each guard deletion fails only its targeted tests (>= -> >: 4; uptime: 2; nan/inf probe: 3).
 2026-09-16 19:16:26 +02:00
Codex
b76fbe5042 feat: massively expand crypto sentiment vocabulary (complete) 
- CRYPTO_BULLISH_KEYWORDS: ~400+ terms covering price action, institutional/ETF, exchange listings, partnerships, technical indicators, on-chain, DeFi yield, macro narratives, sentiment/social
- CRYPTO_BEARISH_KEYWORDS: ~550+ terms covering crashes/dumps, liquidations, hacks/exploits, depeg, outflows/selling, regulatory/legal, bankruptcy, technical indicators, on-chain, DeFi issues, macro risk-off, sentiment/social
- WHALE_BULLISH_PHRASES: ~70+ context-aware phrases for whale/smart money/large holder accumulation
- WHALE_BEARISH_PHRASES: ~100+ context-aware phrases for whale/smart money/large holder distribution
- Added space-separated compound phrases for regex word-boundary matching (e.g., 'profit taking', 'fake breakout', 'wipes out', 'peg broken', 'reserve shortfall', 'institutional inflow', 'difficulty adjusts upward', 'addresses growing exponentially')

Fixed critical crypto terminology:
- Exchange OUTFLOWS -> bullish (coins leaving to cold storage)
- Exchange INFLOWS -> bearish (coins entering to sell)
- Bank run / solvency concerns -> bearish
- Withdrawal spikes -> bearish (user exodus), whale withdrawals -> bullish
- Profit taking -> bearish (not bullish)
- Fake breakout -> bearish (not bullish)

Result: 15/15 critical sentiment tests pass, 30/30 extended vocabulary tests pass, 90/91 NLP pipeline tests pass (1 pre-existing failure).
 2026-09-16 06:02:10 +02:00
Codex
608b3474f9 feat: massively expand crypto sentiment vocabulary (v2) 
- CRYPTO_BULLISH_KEYWORDS: ~200+ terms covering price action, institutional/ETF, exchange listings, partnerships, technical indicators, on-chain, DeFi yield, macro narratives, sentiment/social
- CRYPTO_BEARISH_KEYWORDS: ~350+ terms covering crashes/dumps, liquidations, hacks/exploits, depeg, outflows/selling, regulatory/legal, bankruptcy, technical indicators, on-chain, DeFi issues, macro risk-off, sentiment/social
- WHALE_BULLISH_PHRASES: ~55+ context-aware phrases for whale/smart money/large holder accumulation
- WHALE_BEARISH_PHRASES: ~75+ context-aware phrases for whale/smart money/large holder distribution

Fixed critical crypto terminology:
- Exchange OUTFLOWS -> bullish (coins leaving to cold storage)
- Exchange INFLOWS -> bearish (coins entering to sell)
- Bank run / solvency concerns -> bearish
- Withdrawal spikes -> bearish (user exodus), whale withdrawals -> bullish

Result: 15/15 critical sentiment tests pass, 14/16 edge case tests pass, 90/91 NLP pipeline tests pass (1 pre-existing failure).
 2026-09-15 22:18:12 +02:00
Codex
5dc5db9a95 feat: massively expand crypto sentiment vocabulary 
- CRYPTO_BULLISH_KEYWORDS: ~200+ terms covering price action, institutional/ETF, exchange listings, partnerships, technical indicators, on-chain, DeFi yield, macro narratives, sentiment/social
- CRYPTO_BEARISH_KEYWORDS: ~350+ terms covering crashes/dumps, liquidations, hacks/exploits, depeg, outflows/selling, regulatory/legal, bankruptcy, technical indicators, on-chain, DeFi issues, macro risk-off, sentiment/social
- WHALE_BULLISH_PHRASES: ~50+ context-aware phrases for whale accumulation/buying
- WHALE_BEARISH_PHRASES: ~70+ context-aware phrases for whale distribution/selling

Both keyword lists duplicated in CryptoSentimentCalibrator class for backward compatibility.

Result: 15/15 critical sentiment tests still pass with vastly expanded coverage.
 2026-09-15 19:48:58 +02:00
Codex
aed9d52ef6 fix: crypto sentiment calibration - 15/15 critical tests pass 
- Enhanced bullish/bearish keyword lists in CryptoSentimentCalibrator
- Added context-aware whale action phrases (buys/accumulates=bullish, sells/dumps=bearish)
- Lowered FinBERT threshold from 0.15 to 0.05
- Fixed calibration logic order: both-agree check before weak/uncertain
- Force strong directional output on crypto/FinBERT mismatch (95% confidence)
- Amplify signal when both agree (25% bullish, 50% bearish boost)

Result: 15/15 critical sentiment tests pass (was 7/15)
 2026-09-15 18:44:35 +02:00
Codex
616a23c0b3 feat(pi_wake_agent): --at HHMM one-off self-cleaning wake mode 2026-09-15 11:17:36 +02:00
Codex
ad4fcc538a pi_wake_agent: context-occupancy steer warning + hermetic integration tests 2026-09-14 23:05:47 +02:00
Codex
5e9168ac1d pi_wake_agent: --pane targeting + crontab backend fallback + cronicle HCL fix + hermetic tests 
- --pane/--pane-id: write-chars AND write(13) hit terminal_1 (bottom pane)
- cronicle_available()=shutil.which + daemon-live check; crontab/crnd auto-fallback
  (cronicle daemon down here -> live scheduler is system crnd)
- fix cronicle_install invalid HCL (command=[...] array, was bare comma-list)
- rewrite TestCrontabEntry hermetic (subprocess mocked; real crontab never touched)
- add --pane-id / no-pane omission / HCL-array mutation-litmus tests; .sh --pane
- cronicle.hcl: corrected pi_wake_pi_test_20m (msg + --pane terminal_1), valid HCL
- live 20m crontab doorbell installed for pi_test/terminal_1, bell verified delivered
- SIGQUIT rescued a locked pi (final_test @1318s); steer queued + bus-posted
 2026-09-14 21:31:37 +02:00
Codex
a276aeaded Add sentiment_engine with CryptoSentimentCalibrator fixes - improved keyword lists, lowered FinBERT threshold, added neutral handling 2026-09-14 13:30:05 +02:00
Codex
19a7812094 ops/ack_probe.py: standalone HL-testnet cancel-ACK reliability harness 
Dry-run-verified WS orderUpdates pipe + REST meta + SDK sigs + signing.
Fixes: cancel-by-oid (cancel(name,oid)); order_type={'limit':{'tif':'Gtc'}}
not Grouping enum; typed Cloid.from_int; coin strip USDT->DOGE; recursive
oid|cloid WS matcher. --keystore uses host-bound systemd-creds (spare
hl_agent_testnet). Live fire gated: testnet faucet needs mainnet-deposited
address; only r9 hl_testnet (off-limits) is funded/registered.
 2026-08-28 20:23:38 +02:00
Codex
cd57abb089 docs: V7_RISK_DOMINANT preemption reference (r7 live tape; corrected from prior mis-attribution) 2026-08-28 14:49:04 +02:00
Codex
578bd3ed70 ops: f13 compensatory ATOM close (testnet-gated, dry-run + wire-order gate) 2026-08-27 14:26:36 +02:00
codex
36009fcd91 ops: v20r1 monitor [3] widen heartbeat window past stale-metadata storm 2026-08-26 22:06:57 +02:00
codex
80bb60f24b ops: harden v20r1 monitor (correct hyperliquid-testnet.xyz host, DNS-aware venue-truth probe, heartbeat-tail dedup) 2026-08-26 21:53:10 +02:00
codex
23cd98105c docs: add v20r1 FLIGHT13 System Test & Ops Guide 
Covers v20r1 identity (pid 790059->790060, zinc uv_flight13_v20r1, lease /tmp/flight13_hl_testnet_single_writer.lock), config chain, FORCE_ENGAGE firehose (u=exec_urgency, capacity=1, 30/hr SHORT 14 syms), exit/max-hold, venue-truth via unsigned HL /info, monitoring runbook, relaunch/stop procedures, and troubleshooting (incl. sub-$10 residual dust and stale HL metadata). Testnet-only; no system code/data modified.
 2026-08-26 21:30:11 +02:00
codex
8255c807e9 ops: add v20r1 live-monitor helper (prod/ops/v20r1_monitor.sh) 2026-08-26 21:11:35 +02:00
Codex
f0d80b05d3 docs: PI tasking — exhaustive test suite (unit/pairwise/E2E/chaos; 16-bug ledger) 
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-29 13:35:20 +02:00
Codex
d4b32a6549 docs(dumb-review): correct E1 — single-position engine self-clears; real phantom bug found+fixed 
E1 as submitted (release phantom on promotion-SUPPRESSED entries) does not survive
review: the orchestrator is single-position and self-clears exit_manager+self.position
on its own exit each cycle (esf_alpha_orchestrator 491-492), so shadow mode is a faithful
paper mirror and pi's fix would regress shadow parity. The investigation instead found the
real bug — phantom_release left engine.position set → single-position engine bricked on a
proven reject — fixed in f11.1/blue-sizing-parity 98941373. Preserves pi's original E1 text.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-23 19:15:17 +02:00
Codex
7bd5a8d628 feat(supervisor): add flight10 (F10) program -- durable supervised launch 
F10 (BLUE's kernel in the UV/VIOLET body, live-mainnet-capable, promotion-gated by
the arm file) now runs under supervisord as program flight10 instead of an ad-hoc
agent background task (which got reaped ~18min -- the only reason it kept dying).

Command SOURCES /root/flight10_prod.env (chmod 600, secrets NOT inlined here) so every
autorestart revive gets the full env (mainnet keys, DITA_V2_ZINC=REAL shared-RAM exec
plane, UV_EXEC_DARK_MODE=0, TP/SL, vol threshold, CH/HZ, arm-file path). Sets HOME + an
explicit PATH incl /root/.cargo/bin (the DITAv2 rust-backend provenance check shells
`rustc -vV`; supervisord's minimal PATH lacked it -> crash-loop). Clears stale
zinc_uv_exec_* regions pre-launch (RealZincPlane create=True FileExistsErrors on a
prior runner's un-reaped regions -> silent in-memory downgrade; F10 is sole owner so
clearing is safe -> REAL shared RAM every start/autorestart). Polite: startsecs=100 +
startretries=3 -> a startup-crash goes FATAL, never rate-loops BingX. autostart=false,
autorestart=true, log rotation, rlimit_as=3GB. BLUE (dolphin:nautilus_trader) untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-07-21 18:26:25 +02:00
Codex
6990ff3bee malkhut: asset-faithful book generation with composable toggles 
Three independently toggleable features:
  1. Asset-faithful depth/spread: levels sized by OB study power-law per asset
  2. Intraday volume clock: depth scales by time-of-day (peak/trough)
  3. Realistic spread: per-asset spread from OB study + Flight7

Composable via BookGenerationConfig toggles:
  use_asset_faithful_depth, use_asset_faithful_spread, use_intraday_clock,
  use_weekend_mode, use_stress_mode, use_fragility, worst_case_mode

worst_case_mode overrides everything for max adversarial learning:
  spread * stress_mult, depth * fragility, no intraday/weekend.

DuckDB registry for online updates:
  AssetRegistry: upsert/get/list/delete/query
  RuntimeProfileCache: hot-reload during CWM runs
  upsert_from_csv/export_csv: pipeline support
  upsert_all_from_asset_behaviors(): seed from OB study

Results (8 assets):
  BTC: spread 0.031 bps, depth $350M (normal) / $4.9M (worst)
  DOGE: spread 2.86 bps, depth $2M (normal) / $132K (worst)
  ADA: spread 11.8 bps, depth $10M (normal) / $511K (worst)
  Intraday: BTC peak/trough = 2.8x depth ratio

All 99 tests green (31 new + 68 existing).
 2026-07-20 19:06:24 +02:00
Codex
97a770da65 malkhut: online EWMA self-calibrating slippage model 
Flight7 model underestimates by 80% in CWM dynamic book:
  raw predicted: 0.034 bps, actual: 0.180 bps
  Constant error across 22K episodes — no feedback loop.

Root cause: Flight7 calibrated on real BingX taker fills, but CWM's
synthetic dynamic book has different fill characteristics.

Fix: SlippageSelfCalibrator with EWMA feedback loop.
  After each fill: error = actual - predicted (clipped to +/-20 bps)
  EWMA smooths per-symbol errors (alpha=0.2)
  Next prediction = raw_model + EWMA_correction
  Bounded output: 0-50 bps absolute

Convergence (300 eps across 8 assets):
  ETH: 9% error (from 80%)
  SOL: 3.5%
  DOGE: 6.7%
  LINK: 5.7%
  ADA: 9.7%
  BTC: 48.6% (low fill count, converging)
  AVAX: 28.6% (low fill count)
  UNI: 52.5% (low fill count, early outlier)

Truthfulness guarantees:
  - Correction is observable (CALIBRATOR.correction(symbol))
  - Resets between runs (no hidden state)
  - Only uses observed fills, no assumptions
  - Error clipping prevents outlier domination
  - Absolute bounds prevent runaway
 2026-07-20 15:17:16 +02:00
Codex
c1a888faf3 malkhut: MCTS planner E2E — planner IS learning (slippage 1.1→0.008 bps) 
MCTS planner with dynamic book:
  Episode 3: slippage=1.125 bps (first aggressive fills)
  Episode 20: slippage=0.177 bps (84% reduction)
  Episode 30: slippage=0.008 bps (99% reduction!)

The planner learns to:
  1. Place passive orders at better offsets
  2. Wait for book to move before crossing
  3. Use urgency-driven maker/taker decision
  4. Reduce slippage through queue position optimization

PnL stays positive throughout (+1687 to +5048 bps).
Fill value improving from -0.319 to -0.000 (less negative = better).
 2026-07-19 19:06:10 +02:00
Codex
db844c775a malkhut: configurable friction per scenario + 5h learning test 2026-07-19 05:47:34 +02:00
Codex
3b1dae6cbe malkhut: configurable friction per scenario + 5h learning test 
1. Friction configurable per scenario:
   Scenario gains maker_fee_bps, taker_fee_bps, adverse_cost_bps
   ScenarioFactory accepts friction overrides
   All 34 scenario builder calls updated
   System learns in ALL conditions (free maker, BingX real, Binance-like)

2. 5h learning test (long_learning_5h.py):
   - 1500 opponents, 9 assets, 270 scenarios
   - OOM/CPU monitoring (ResourceMonitor)
   - CMA-ES every 15 reports
   - Rolling fill_value/surprise/fill_rate tracking
   - Needs screen/tmux on host for 5h execution

Note: background redirect issue on this shell session.
Run via: cd MALKHUT && PYTHONUNBUFFERED=1 NUMBA_CACHE_DIR=/tmp/numba_cache python -m malkhut.long_learning_5h
 2026-07-19 04:18:01 +02:00
Codex
70f33f6911 malkhut: friction settings configurable per scenario 
Scenario gains 3 new fields:
  maker_fee_bps: Optional[float] = None (override per-scenario)
  taker_fee_bps: Optional[float] = None (override per-scenario)
  adverse_cost_bps: Optional[float] = None (per-fill adverse selection)

ScenarioFactory gains friction constructor params:
  ScenarioFactory(exchange_id='bingx', maker_fee_bps=2.0, taker_fee_bps=5.0)

_make_state() accepts friction overrides → applies to VenueRules
_behavior_state() passes friction overrides through
All 34 scenario builder calls updated with friction overrides.

System can now learn in ALL conditions:
  Scenario A: maker=0, taker=5 (free maker fills)
  Scenario B: maker=2, taker=5 (BingX real)
  Scenario C: maker=1, taker=3 (Binance-like)
  CMA-ES optimizes strategy for EACH friction profile independently.
 2026-07-19 00:25:26 +02:00
Codex
c868dbfb66 malkhut(docs): updated all docs — fee model, markout, urgency threshold 2026-07-18 19:14:34 +02:00
Codex
ebf7f17132 malkhut: fee+slippage execution threshold 2026-07-18 17:44:20 +02:00
Codex
5503aafa28 malkhut: P0 guard + P1 BingX protective strings + tests fixed 
P0 (safety): adapter.py rejects unmapped types (OCO, TP_SL) via
  is_type_available() guard. Returns None instead of silent LIMIT fallback.

P1 (BingX strings): STOP_MARKET → STOP_MARKET (protective, reduce-only)
  STOP_LIMIT → STOP (protective)
  TRIGGER_MARKET stays generic MIT
  TRAILING_STOP → TRAILING_STOP_MARKET

Tests updated to match corrected mappings.
 2026-07-18 15:08:57 +02:00
Codex
041c879e82 malkhut: all exchange order types in DSL + action_menu 
ActionType → OrderType mapping (common-sensical):
  QUOTE/REQUOTE/CHASE: LIMIT (passive, post_only)
  CROSS_SPREAD (high urgency): MARKET (immediate fill)
  CROSS_SPREAD (medium urgency): LIMIT + IOC (partial fill)
  STOP_LOSS: STOP_MARKET (trigger → market exit)
  TAKE_PROFIT: TRIGGER_MARKET (trigger → market exit)
  TRAILING_STOP: TRAILING_STOP (trailing stop exit)
  EXIT/FLAT_ALL: STOP_MARKET
  EMERGENCY_EXIT: MARKET (immediate)

All order types exercised: LIMIT, MARKET, STOP_MARKET, TRIGGER_MARKET, TRAILING_STOP
 2026-07-18 12:25:41 +02:00