diff --git a/sentiment_engine/MODEL_CARD_TRAINING_STATUS.md b/sentiment_engine/MODEL_CARD_TRAINING_STATUS.md index df6ce5c..51fa4f1 100644 --- a/sentiment_engine/MODEL_CARD_TRAINING_STATUS.md +++ b/sentiment_engine/MODEL_CARD_TRAINING_STATUS.md @@ -243,3 +243,57 @@ test_cases = [ | 2026-09-26 | Auto-audit | Event classifier validation | bert-base-event | 8/8 PASS — model IS fine-tuned, do not retrain | **NEXT STEP:** Proceed with FinBERT and DistilRoBERTa fine-tuning. bert-base-event is LOCKED. + +--- + +### TRAINING RESULTS (2026-09-26) + +#### FinBERT Crypto Sentiment LoRA +| Metric | Value | +|--------|-------| +| Epochs | 5 (early stopped at ~4.8) | +| Train samples | 796 (augmented from 177 unique) | +| Val samples | 89 | +| Trainable params | 1,341,699 / 110M (1.21%) | +| Final train loss | 0.326 | +| Final eval loss | 0.141 | +| Final eval F1 | 0.907 | +| Final eval accuracy | 0.955 | +| Adapter size | 6.2 MB | +| Early stopping | Triggered at epoch ~4.8 | + +**Known Issue:** Still biased toward Neutral on crypto-specific slang (HODL, rug, ape, etc.) — needs more training data or human-verified samples. + +#### DistilRoBERTa Crypto Emotion LoRA +| Metric | Value | +|--------|-------| +| Epochs | 5 (early stopped at ~4.8) | +| Train samples | 184 (synthetic templates) | +| Val samples | 21 | +| Trainable params | 1,258,758 / 83M (1.51%) | +| Final train loss | 0.410 | +| Final eval loss | 0.290 | +| Final eval F1 macro | 0.594 | +| Final eval accuracy | 0.865 | +| Adapter size | 8.1 MB | +| Weighted loss | greed=2.0, fear=2.0, joy=1.5 | + +**Validation Results:** +- "BTC breaks 100k!" → joy(0.82), greed(0.49) ✅ +- "Major hack" → sadness(0.49), fear(0.31) ⚠️ (fear low) +- "Panic selling" → fear(0.73), greed(0.49) ✅ +- "FOMO buying" → greed(0.57), anger(0.43) ✅ +- "Rug pull" → fear(0.47), anger(0.34) ⚠️ +- "ETF approved!" → joy(0.95), greed(0.59) ✅ + +**Known Issue:** Fear class under-activated on hack/rugged texts; needs more fear samples. + +--- + +### ADDENDUM LOG + +| Date | Author | Action | Models | Notes | +|------|--------|--------|--------|-------| +| 2026-09-26 | Auto-audit | Initial status doc | All 4 | Baseline before any retraining | +| 2026-09-26 | Auto-audit | Event classifier validation | bert-base-event | 8/8 PASS — model IS fine-tuned, do not retrain | +| 2026-09-26 | Pipeline | LoRA training complete | FinBERT, DistilRoBERTa | 5 epochs with early stopping |