docs(model): training results addendum for LoRA sentiment + emotion
- FinBERT: 5 epochs, early stop at ~4.8, 6.2MB, F1=0.907, acc=0.955 - DistilRoBERTa: 5 epochs, early stop at ~4.8, 8.1MB, F1=0.594, acc=0.865 - Both with early stopping (patience=3) - Known issues documented for next iteration
This commit is contained in:
@@ -243,3 +243,57 @@ test_cases = [
|
|||||||
| 2026-09-26 | Auto-audit | Event classifier validation | bert-base-event | 8/8 PASS — model IS fine-tuned, do not retrain |
|
| 2026-09-26 | Auto-audit | Event classifier validation | bert-base-event | 8/8 PASS — model IS fine-tuned, do not retrain |
|
||||||
|
|
||||||
**NEXT STEP:** Proceed with FinBERT and DistilRoBERTa fine-tuning. bert-base-event is LOCKED.
|
**NEXT STEP:** Proceed with FinBERT and DistilRoBERTa fine-tuning. bert-base-event is LOCKED.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### TRAINING RESULTS (2026-09-26)
|
||||||
|
|
||||||
|
#### FinBERT Crypto Sentiment LoRA
|
||||||
|
| Metric | Value |
|
||||||
|
|--------|-------|
|
||||||
|
| Epochs | 5 (early stopped at ~4.8) |
|
||||||
|
| Train samples | 796 (augmented from 177 unique) |
|
||||||
|
| Val samples | 89 |
|
||||||
|
| Trainable params | 1,341,699 / 110M (1.21%) |
|
||||||
|
| Final train loss | 0.326 |
|
||||||
|
| Final eval loss | 0.141 |
|
||||||
|
| Final eval F1 | 0.907 |
|
||||||
|
| Final eval accuracy | 0.955 |
|
||||||
|
| Adapter size | 6.2 MB |
|
||||||
|
| Early stopping | Triggered at epoch ~4.8 |
|
||||||
|
|
||||||
|
**Known Issue:** Still biased toward Neutral on crypto-specific slang (HODL, rug, ape, etc.) — needs more training data or human-verified samples.
|
||||||
|
|
||||||
|
#### DistilRoBERTa Crypto Emotion LoRA
|
||||||
|
| Metric | Value |
|
||||||
|
|--------|-------|
|
||||||
|
| Epochs | 5 (early stopped at ~4.8) |
|
||||||
|
| Train samples | 184 (synthetic templates) |
|
||||||
|
| Val samples | 21 |
|
||||||
|
| Trainable params | 1,258,758 / 83M (1.51%) |
|
||||||
|
| Final train loss | 0.410 |
|
||||||
|
| Final eval loss | 0.290 |
|
||||||
|
| Final eval F1 macro | 0.594 |
|
||||||
|
| Final eval accuracy | 0.865 |
|
||||||
|
| Adapter size | 8.1 MB |
|
||||||
|
| Weighted loss | greed=2.0, fear=2.0, joy=1.5 |
|
||||||
|
|
||||||
|
**Validation Results:**
|
||||||
|
- "BTC breaks 100k!" → joy(0.82), greed(0.49) ✅
|
||||||
|
- "Major hack" → sadness(0.49), fear(0.31) ⚠️ (fear low)
|
||||||
|
- "Panic selling" → fear(0.73), greed(0.49) ✅
|
||||||
|
- "FOMO buying" → greed(0.57), anger(0.43) ✅
|
||||||
|
- "Rug pull" → fear(0.47), anger(0.34) ⚠️
|
||||||
|
- "ETF approved!" → joy(0.95), greed(0.59) ✅
|
||||||
|
|
||||||
|
**Known Issue:** Fear class under-activated on hack/rugged texts; needs more fear samples.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### ADDENDUM LOG
|
||||||
|
|
||||||
|
| Date | Author | Action | Models | Notes |
|
||||||
|
|------|--------|--------|--------|-------|
|
||||||
|
| 2026-09-26 | Auto-audit | Initial status doc | All 4 | Baseline before any retraining |
|
||||||
|
| 2026-09-26 | Auto-audit | Event classifier validation | bert-base-event | 8/8 PASS — model IS fine-tuned, do not retrain |
|
||||||
|
| 2026-09-26 | Pipeline | LoRA training complete | FinBERT, DistilRoBERTa | 5 epochs with early stopping |
|
||||||
|
|||||||
Reference in New Issue
Block a user