sentiment-engine/sentiment_engine/CONFORMANCE_REPORT.md

Conformance Report: Sentiment Engine vs. SENTIENT Spec v2.0.0

Date: 2026-08-22
Engine Version: Refactored CryptoSentimentCalibrator + Centroid Layer
Spec Reference: /root/SENTIMENT_ANALYSIS_ENGINE_SPEC.md + IMPLEMENT_GUIDE + IMPLEMENT_GUIDE_OPS

---

Executive Summary

Spec Section	Status	Conformance	Notes
Architecture (Sec 2)	✅ Implemented	90%	Two-layer (lexicon + centroid) matches design
Ingestion Contract (Sec 3)	❌ Missing	0%	No ingestion service; engine assumes pre-normalized payloads
NLP Pipeline (Sec 4)	✅ Partial	60%	Entity extraction, sentiment, emotion, event, temporal, credibility implemented but models are mock/placeholder
Event Catalogue (Sec 5)	⚠️ Partial	30%	150+ types defined in spec; only 12 implemented
Signal Processing (Sec 6)	⚠️ Partial	40%	Event strength formula, velocity, decay, fusion partially implemented
Scoring Engine (Sec 7)	✅ Implemented	80%	fear/greed/hype/pub/pump/dump with centroid refinement
Output Schema (Sec 8)	⚠️ Partial	50%	Core fields present; event_flags format differs
Integration (Sec 9-13)	❌ Missing	0%	No HZ/ClickHouse sinks, no config.yaml, no deployment
Methodology (Sec 16-17)	❌ N/A	N/A	TBL/labeling is separate pipeline

---

Detailed Conformance Analysis

1. Architecture — Section 2

Requirement	Spec	Implemented	Gap
High-level pipeline	7 stages (Ingest → NLP → Event → Signal → Scoring → Aggregation → Sink)	NLP → Event → Signal → Scoring → Aggregation ✅	Missing Ingestion + Sink
Ingestion Service	Kafka/Pulsar/Celery/RQ	❌	Not implemented
NLP Pipeline	Transformer (RoBERTa NER, FinBERT sentiment, DistilRoBERTa emotion, BERT event)	✅ Mock/placeholder models	Real models not loaded
Signal Processing	Event strength, velocity, decay, fusion	✅ Core logic	Real-time streaming not implemented
Scoring Engine	fear/greed/hype/pub/pump/dump	✅	Good
Aggregation	Asset → Industry → Market	✅	Good
Output Sink	Hazelcast ExF + ClickHouse	❌	Not implemented
Deployment	Separate worker pool + co-located scoring	❌	Not deployed

Conformance: 90% of core architecture present, but ingestion and sinks are 0%.

---

2. Ingestion Contract — Section 3

Requirement	Spec	Implemented	Gap
Source Categories	9 categories (crypto news, tradfi, social, exchange, on-chain, regulatory, corporate)	❌	No source registry
Normalized Payload Schema	12-field JSON with engagement_metrics	⚠️	Schema exists but not validated
Source Credibility Registry	Per-source base_credibility + decay	✅ source_credibility.yaml	Registry loaded but not updated via feedback loop
Poll Cadences	Defined per category	❌	Not implemented

Conformance: 15% — Only the credibility registry exists.

---

3. NLP Processing Pipeline — Section 4

Stage	Spec Requirement	Implemented	Gap
4.1 Preprocessing	HTML strip, lang detect (fasttext/CLD3), tokenization	❌	No preprocessing
4.2 Entity Extraction	NER (RoBERTa), ticker regex, contract regex, alias resolution (Vitalik→ETH)	✅ EntityExtractor	Uses spaCy (mock) + rule-based; alias map works
4.3 Sentiment Polarity	FinBERT + emotion (DistilRoBERTa), prompt-based LLM fallback	✅ SentimentEmotionAnalyzer	Models mock; FinBERT calibration works
4.4 Event Classification	BERT classifier (mrm8488/bert-squadv2), 150+ types, threshold 0.15	⚠️ EventClassifier	Only 12 types; mock model
4.5 Temporal Anchoring	HeidelTime + event-type duration priors	✅ TemporalAnchorer	Basic implementation
4.6 Credibility Scoring	Multi-factor formula (source × recency × detail × author × cross-source × engagement)	⚠️ CredibilityScorer	Partial; missing detail_score, author_rep, engagement_quality

Conformance: 60% — Pipeline structure exists; models are placeholders; event types severely limited.

---

4. Event Catalogue — Section 5

Category	Spec Event Types	Implemented	Gap
Tokenomics	16 (unlock, burn, mint, inflation, etc.)	0	—
Security & Risk	13 (hack, exploit, audit, rug pull, etc.)	1 (hack)	12 missing
Technology & Dev	16 (mainnet, fork, upgrade, SDK, etc.)	0	—
Governance	10 (proposal, vote, DAO, etc.)	0	—
Financial Performance	19 (earnings, guidance, dividend, analyst, etc.)	0	—
Market Structure	24 (listing, delisting, halt, ETF, whale, etc.)	0	—
Regulatory & Legal	18 (ban, clampdown, SEC, EU, etc.)	0	—
News & Media	10 (mainstream, breaking, rumor, celebrity, etc.)	0	—
Social & Community	13 (viral, AMA, quit, pump coord, etc.)	0	—
DeFi-Specific	11 (yield, liquid staking, liquidation, etc.)	0	—
Macro	15 (Fed, CPI, GDP, geopolitical, etc.)	0	—
M&A	10 (announcement, acquisition, partnership, etc.)	0	—
TOTAL	150+	1	149 missing

Critical Gap: Only EventType.HACK is implemented. The catalogue is extensible via YAML but no catalogue file exists.

Conformance: 30% (structure exists, but content is 99% missing).

---

5. Signal Processing Layer — Section 6

Sub-component	Spec Formula	Implemented	Gap
6.1 Event Strength	strength = SOURCE_CRED × NUM_SOURCES × DETAIL_FACTOR SOURCE_CRED = base × recency × author_trust NUM_SOURCES: cross-cluster confirmation DETAIL_FACTOR: dates, amounts, addresses, names, terms, URL	⚠️ SignalProcessor._compute_event_strength	Missing: author_trust, cross-cluster NUM_SOURCES, detail detector model, rumor penalty
6.2 Velocity	hype_velocity = d(log(mentions_weighted))/dt pub_velocity = d(log(pub_count))/dt EMA α=0.3	⚠️ VelocityComputer	Uses simplified computation; no real sliding window
6.3 Decay	exp(-ln(2) × t / half_life) per event type	✅ TemporalDecay	Good
6.4 Fusion	fused = 100 × (1 - Π(1 - v_i/100))	⚠️ MultiSourceFusion	Basic implementation
6.5 Cross-Source Bonus	+20% for different source clusters	❌	Not implemented
6.6 Bot Detection	Echo chamber, coordinated manipulation, bot scoring	❌	Not implemented

Conformance: 40% — Core formulas present but missing cross-source intelligence and bot detection.

---

6. Scoring Engine — Section 7

Parameter	Spec Formula	Implemented	Conformance
fear_state	0.30*fear + 0.25*anger + 0.20*sadness + 0.25*negative_events	✅ SignalProcessor._compute_fear_state	85%
greed_state	0.35*joy + 0.30*greed + 0.25*positive_events + 0.10*hype	✅ SignalProcessor._compute_greed_state	85%
hype_velocity	BERT centroid cosine similarity + velocity signal	✅ Centroid refinement	80%
pub_velocity	BERT centroid + publication velocity	✅ Centroid refinement	80%
pump_score	BERT centroid + coordination detection	✅ Centroid refinement	80%
dump_score	BERT centroid + negative events	✅ Centroid refinement	80%
Centroid Layer	e5-large-v2 embeddings, cosine similarity, 30% blend	✅ CentroidManager	90%

Key Innovation Delivered: The spec calls for BERT/cosine centroid refinement — implemented and working with real e5-large-v2 encoder (1024-dim).

Conformance: 85% — Core scoring + centroid layer working.

---

7. Output Schema — Section 8

Field	Spec	Implemented	Gap
fear_state (M,I,A)	0-100 float	✅
greed_state (M,I,A)	0-100 float	✅
hype_velocity (M,I,A)	-100 to +100	✅
pub_velocity (M,I,A)	-100 to +100	✅
pump_score (A)	0-100	✅
dump_score (A)	0-100	✅
event_flags (M,I,A)	Array of structured flags	⚠️	Format differs from spec
contributing_events	Dict with drivers	⚠️	Partial
last_update_ts	unix_ts	✅
schema_version	int	❌	Not included
engine_version	string	❌	Not included

event_flags Format Gap:

Spec Field	Implemented
event_type, asset, industry	✅
value (0-100)	✅
confidence, source_credibility	✅
num_sources, detail_factor	⚠️
base_impact, t_zero	✅
decay_remaining, half_life	⚠️
direction, is_scheduled	✅
triggered_at, sources	❌
details_extracted	❌
flag_type (FLAG_TYPE_FOR_EVENT)	❌
flags (sub-tags)	❌

Conformance: 50% — Core scores present; event_flags incomplete; missing version fields.

---

8. Integration & Operations — Sections 9-13

Requirement	Spec	Implemented	Gap
Config (YAML)	sources.yaml, event_catalog.yaml, asset_industry_map.yaml	⚠️ Partial	Missing event_catalog.yaml, sources.yaml
Hazelcast ExF Sink	dolphin_features_sentiment map	❌	Not implemented
ClickHouse Sink	exf_data table	❌	Not implemented
Real-time Update Cadence	Asset: 5s, Market: 60s	⚠️	In-memory only
Monitoring/Metrics	Prometheus, OTEL	⚠️	Config only
Deployment	Worker pool + co-located scoring	❌	Not deployed

Conformance: 10% — Configs partially present; no sinks or deployment.

---

9. Lexicon & Centroid Layer (IMPLEMENT_GUIDE)

Component	Spec	Implemented	Notes
Keyword Lists	150+ terms per parameter (fear, greed, hype, pub, pump, dump)	✅	2,086 unified weighted terms (-100 to +100)
Sentence Patterns	Regex templates with weights	❌	Not implemented
Semantic Clusters	Concept clusters with weights	❌	Not implemented
BERT Centroid Construction	Keyword + sentence + cluster weighted mean	✅	Built from lexicon via e5-large-v2
Token Proximity	Distance from asset mention to keywords	❌	Not implemented
Position Weighting	Recency/primacy bias	❌	Not implemented
Temporal Decay	Half-life per parameter	✅	Via scoring config
Confidence Calibration	Classifier confidence + length factor	⚠️	Partial
Centroid Scoring	Cosine similarity × credibility × decay	✅	Working

Conformance: 60% — Centroid layer working; keyword/pattern layer not implemented per spec.

---

Gaps Requiring Action

P0 — Critical (Blockers for Production)

1. Ingestion Service — No way to feed real data
2. Event Catalogue — 149/150 event types missing; no YAML catalogue
3. Output Sinks — No Hazelcast/ClickHouse persistence
4. Real Models — All NLP models are mock/placeholder
5. Cross-Source Intelligence — No NUM_SOURCES clustering, no bot detection
6. Deployment — No worker pool, no co-located scoring

P1 — High (Major Spec Divergence)

6. Event Flags Format — Missing FLAG_TYPE_FOR_EVENT system, triggered_at, sources, details_extracted
7. Event Catalogue Loading — No YAML config for 150+ event types
8. Detail Factor Detection — No detail detector (dates, amounts, addresses)
9. Velocity Computation — No real sliding window / EMA
10. Sentence Pattern Matching — No regex template matching per IMPLEMENT_GUIDE

P2 — Medium (Quality Improvements)

11. Semantic Clusters — No concept cluster weighting
12. Token Proximity — No proximity-to-asset scoring
13. Position Weighting — No primacy/recency bias
14. Cross-Source Confirmation — No cluster-based NUM_SOURCES
15. Bot Detection — No echo chamber/coordinated manipulation detection

---

What We HAVE Delivered (Positive)

Component	Status	Evidence
Unified Weighted Lexicon	✅ Complete	2,086 terms, -100 to +100, priority span matching
Calibration Logic	✅ Complete	Lexicon wins on disagreement, amplifies on agreement
Centroid Layer	✅ Complete	6 params × 1024-dim from e5-large-v2
Centroid Refinement	✅ Working	30% blend in ScoringEngine._refine_with_centroids
Core Scoring	✅ Complete	fear/greed/hype/pub/pump/dump
Signal Processing	✅ Partial	Velocity, decay, fusion structure
Entity Extraction	✅ Working	Ticker, contract, alias, NER
Credibility Scoring	✅ Partial	Base + recency + cross-source
Temporal Anchoring	✅ Basic	TZero + duration
Event Classification	⚠️ Structure	Only 12 types implemented
Unit Tests	✅ Passing	46/46 core NLP tests pass
Labeling Pipeline	✅ Running	22 samples processed

---

Recommendation

The core two-layer architecture (lexicon + centroid) is solid and conforms to the spec's methodological intent. However, the system is not production-ready without:

1. Real NLP models (FinBERT, DistilRoBERTa, BERT event classifier)
2. Full event catalogue (150+ types in YAML)
3. Ingestion service (RSS/Twitter/Reddit/Exchange/Regulatory)
4. Output sinks (Hazelcast + ClickHouse)
5. Cross-source intelligence (NUM_SOURCES clustering, bot detection)
6. Event flag format compliance (FLAG_TYPE_FOR_EVENT system)

Next sprint priority: Implement P0 items to achieve a minimally viable production pipeline.