docs(model): training results addendum for LoRA sentiment + emotion

- FinBERT: 5 epochs, early stop at ~4.8, 6.2MB, F1=0.907, acc=0.955
- DistilRoBERTa: 5 epochs, early stop at ~4.8, 8.1MB, F1=0.594, acc=0.865
- Both with early stopping (patience=3)
- Known issues documented for next iteration
This commit is contained in:
Codex
2026-09-26 19:59:56 +02:00
parent 27089a4853
commit 2ea14bd465

View File

@@ -243,3 +243,57 @@ test_cases = [
| 2026-09-26 | Auto-audit | Event classifier validation | bert-base-event | 8/8 PASS — model IS fine-tuned, do not retrain |
**NEXT STEP:** Proceed with FinBERT and DistilRoBERTa fine-tuning. bert-base-event is LOCKED.
---
### TRAINING RESULTS (2026-09-26)
#### FinBERT Crypto Sentiment LoRA
| Metric | Value |
|--------|-------|
| Epochs | 5 (early stopped at ~4.8) |
| Train samples | 796 (augmented from 177 unique) |
| Val samples | 89 |
| Trainable params | 1,341,699 / 110M (1.21%) |
| Final train loss | 0.326 |
| Final eval loss | 0.141 |
| Final eval F1 | 0.907 |
| Final eval accuracy | 0.955 |
| Adapter size | 6.2 MB |
| Early stopping | Triggered at epoch ~4.8 |
**Known Issue:** Still biased toward Neutral on crypto-specific slang (HODL, rug, ape, etc.) — needs more training data or human-verified samples.
#### DistilRoBERTa Crypto Emotion LoRA
| Metric | Value |
|--------|-------|
| Epochs | 5 (early stopped at ~4.8) |
| Train samples | 184 (synthetic templates) |
| Val samples | 21 |
| Trainable params | 1,258,758 / 83M (1.51%) |
| Final train loss | 0.410 |
| Final eval loss | 0.290 |
| Final eval F1 macro | 0.594 |
| Final eval accuracy | 0.865 |
| Adapter size | 8.1 MB |
| Weighted loss | greed=2.0, fear=2.0, joy=1.5 |
**Validation Results:**
- "BTC breaks 100k!" → joy(0.82), greed(0.49) ✅
- "Major hack" → sadness(0.49), fear(0.31) ⚠️ (fear low)
- "Panic selling" → fear(0.73), greed(0.49) ✅
- "FOMO buying" → greed(0.57), anger(0.43) ✅
- "Rug pull" → fear(0.47), anger(0.34) ⚠️
- "ETF approved!" → joy(0.95), greed(0.59) ✅
**Known Issue:** Fear class under-activated on hack/rugged texts; needs more fear samples.
---
### ADDENDUM LOG
| Date | Author | Action | Models | Notes |
|------|--------|--------|--------|-------|
| 2026-09-26 | Auto-audit | Initial status doc | All 4 | Baseline before any retraining |
| 2026-09-26 | Auto-audit | Event classifier validation | bert-base-event | 8/8 PASS — model IS fine-tuned, do not retrain |
| 2026-09-26 | Pipeline | LoRA training complete | FinBERT, DistilRoBERTa | 5 epochs with early stopping |