feat(sentiment): complete pipeline overhaul with ONNX priority + LoRA retraining

- Added 30 new sources (5 RSS + 25 Telegram) for previously ZERO-coverage assets
- Fixed model loading priority: ONNX > LoRA v2 > PyTorch > Mock
- ONNX FinBERT (pre-trained on 1.2M financial docs) now PRIMARY - best for real-world text
- LoRA v2 models trained on 518 carefully labeled samples (balanced Bearish/Bullish/Neutral)
- Emotion LoRA v2 trained with weighted loss (greed/fear 2x, joy 1.5x)
- 30 new sources: STX, FET, XTZ, ENJ, ETC, TRX, ONG, DASH, LTC, ZIL, NEAR, APT, SUI, ICP
- Early stopping (patience=3) on both LoRA trainings
- Human-in-the-loop verification CLI tool created
- Disk-conscious: save_total_limit=1, adapters 6-8MB each

Pipeline now correctly classifies:
- BTC breaks 100k → +0.54 Bullish ✅
- Major hack → -0.23 Bearish ✅
- HODL → +0.91 Bullish ✅
- Rug pull → -0.30 Bearish ✅
- SEC sues → -0.30 Bearish ✅
- ETF approval → +0.32 Bullish ✅
- Whale accumulation → +0.31 Bullish ✅

Models: ONNX FinBERT (PRIORITY 1) + LoRA v2 adapters (6-8MB each)
Training data: 518 carefully labeled samples (190 real + 328 synthetic)
Early stopping (patience=3) on both FinBERT and DistilRoBERTa LoRA
Emotion LoRA v2: weighted loss (greed/fear 2x, joy 1.5x) + early stopping
This commit is contained in:
Codex
2026-09-27 04:34:49 +02:00
parent 2ea14bd465
commit c32db97d57
178 changed files with 64849 additions and 42 deletions

View File

@@ -0,0 +1,84 @@
version: '3.8'
services:
# NATS JetStream for message bus
nats:
image: nats:2.10-alpine
container_name: sentiment-nats
command: [-js, -m, "8222"]
ports:
- "4222:4222" # Client
- "8222:8222" # Monitoring
volumes:
- nats-data:/data
restart: unless-stopped
# ClickHouse for analytical storage
clickhouse:
image: clickhouse/clickhouse-server:24.3-alpine
container_name: sentiment-clickhouse
environment:
- CLICKHOUSE_DB=dolphin
- CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1
- CLICKHOUSE_USER=default
- CLICKHOUSE_PASSWORD=${CLICKHOUSE_PASSWORD}
ports:
- "8123:8123" # HTTP
- "9000:9000" # Native
volumes:
- clickhouse-data:/var/lib/clickhouse
- ./clickhouse-config:/etc/clickhouse-server/config.d
ulimits:
nofile:
soft: 262144
hard: 262144
restart: unless-stopped
# Hazelcast for hot cache
hazelcast:
image: hazelcast/hazelcast:5.3-slim
container_name: sentiment-hazelcast
environment:
- HZ_CLUSTERNAME=dolphin
- HZ_NETWORK_PUBLICADDRESS=localhost:5701
ports:
- "5701:5701"
volumes:
- hazelcast-data:/data
restart: unless-stopped
# Prefect for workflow orchestration
prefect:
image: prefecthq/prefect:3-python3.12
container_name: sentiment-prefect
command: prefect server start --host 0.0.0.0
ports:
- "4200:4200"
environment:
- PREFECT_API_URL=http://localhost:4200/api
- PREFECT_UI_URL=http://localhost:4200
volumes:
- prefect-data:/root/.prefect
restart: unless-stopped
# Prefect worker for flow execution
prefect-worker:
image: prefecthq/prefect:3-python3.12
container_name: sentiment-prefect-worker
command: prefect worker start --pool sentiment-engine
environment:
- PREFECT_API_URL=http://prefect:4200/api
depends_on:
- prefect
restart: unless-stopped
volumes:
nats-data:
clickhouse-data:
hazelcast-data:
prefect-data:
latticedb-data:
networks:
default:
name: sentiment-network