Files
Codex c32db97d57 feat(sentiment): complete pipeline overhaul with ONNX priority + LoRA retraining
- Added 30 new sources (5 RSS + 25 Telegram) for previously ZERO-coverage assets
- Fixed model loading priority: ONNX > LoRA v2 > PyTorch > Mock
- ONNX FinBERT (pre-trained on 1.2M financial docs) now PRIMARY - best for real-world text
- LoRA v2 models trained on 518 carefully labeled samples (balanced Bearish/Bullish/Neutral)
- Emotion LoRA v2 trained with weighted loss (greed/fear 2x, joy 1.5x)
- 30 new sources: STX, FET, XTZ, ENJ, ETC, TRX, ONG, DASH, LTC, ZIL, NEAR, APT, SUI, ICP
- Early stopping (patience=3) on both LoRA trainings
- Human-in-the-loop verification CLI tool created
- Disk-conscious: save_total_limit=1, adapters 6-8MB each

Pipeline now correctly classifies:
- BTC breaks 100k → +0.54 Bullish ✅
- Major hack → -0.23 Bearish ✅
- HODL → +0.91 Bullish ✅
- Rug pull → -0.30 Bearish ✅
- SEC sues → -0.30 Bearish ✅
- ETF approval → +0.32 Bullish ✅
- Whale accumulation → +0.31 Bullish ✅

Models: ONNX FinBERT (PRIORITY 1) + LoRA v2 adapters (6-8MB each)
Training data: 518 carefully labeled samples (190 real + 328 synthetic)
Early stopping (patience=3) on both FinBERT and DistilRoBERTa LoRA
Emotion LoRA v2: weighted loss (greed/fear 2x, joy 1.5x) + early stopping
2026-09-27 04:34:49 +02:00

3.5 KiB

Sentiment Engine TUI

Textual-based Terminal User Interface for live monitoring of the Sentiment Analysis Engine.

Features

📡 Live Info Fetches

Real-time stream of all incoming payloads from all sources:

  • Timestamp, source ID, source type
  • Extracted assets mentioned
  • Title/preview of content
  • Source credibility score
  • Content length

📊 Live Parameters (Per Asset)

Per-asset sentiment parameters updating in real-time:

  • Fear State (0-100) — color coded (red>70, yellow>40, green<40)
  • Greed State (0-100) — color coded (green>70, yellow>40, red<40)
  • Sentiment Polarity (-100 to +100)
  • Pump Score (0-100) — entry veto threshold at 75
  • Dump Score (0-100) — exit trigger at 70
  • Hype Velocity — sentiment acceleration rate
  • Publication Velocity — source frequency
  • Event Flags — top 3 events with strength
  • Decay Factor — temporal decay applied
  • Contributing Sources — multi-source fusion count

Sorted by pump_score descending for quick risk identification.

🌍 Aggregate Parameters

Market-wide and industry-level aggregates:

  • Market Fear/Greed/Polarity/Hype/Pub Velocity
  • Aggregate Pump/Dump Risk — with color coding
  • Top 5 Pump/Dump Assets
  • Dominant Events — with strength and confidence
  • Industry Breakdown — per-industry fear, greed, polarity, pump/dump risk, asset count

☁️ Word Cloud

Visual word frequency from recent payloads:

  • Top 60 words sized by frequency
  • Color-coded by frequency tier (bright_white/blue > yellow > green > cyan > dim)
  • Asset mentions weighted 3x
  • Stopwords filtered
  • Updates every second from last 100 payloads

🔌 Source Connector Status

Live status of all 8 connector types:

  • Running/Error/Unknown status with color coding
  • Fetch counts (total, successful, errors)
  • Last fetch timestamp
  • Base credibility score

🎯 Live Event Feed

Real-time event detections:

  • Timestamp, asset, event type
  • Strength (0-100) with color coding
  • Confidence percentage
  • Source count
  • Sorted by strength descending

Keyboard Shortcuts

Key Action
q Quit
p Pause/resume updates
r Force refresh
f Focus Info Fetches
a Focus Asset Parameters
m Focus Market Aggregate
w Focus Word Cloud
s Focus Source Status
e Focus Event Feed

Running

# From sentiment_engine directory
pip install -e ".[tui]"

# Run TUI only
python scripts/run_tui.py

# Run engine + TUI together
python scripts/run_engine.py --tui

# Run engine only (headless)
python scripts/run_engine.py --engine-only

Architecture

The TUI runs as a separate asyncio task alongside the main engine. It receives data via direct method calls:

# From ingestion pipeline
tui_app.add_fetch(payload)

# From scoring engine
tui_app.update_assets(asset_signals)
tui_app.update_market(market, industries)

# From connector registry
tui_app.update_source_status(name, stats)

The TUI uses textual (v0.52+) with rich for rendering. All widgets are reactive and update at 1Hz via a timer.

Integration with Engine

In main.py, the SentimentEngine can optionally start the TUI:

engine = SentimentEngine()
await engine.initialize()
await engine.start()

# TUI runs in same process, shares event loop
tui_task = asyncio.create_task(run_tui())
await tui_task

For production deployment, run TUI in a separate terminal/screen session while the engine runs as a systemd service.