bge-m3-edu-scorer-lr3e5-bs32
This model is a fine-tuned version of BAAI/bge-m3 on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2380
- Precision: 0.4817
- Recall: 0.345
- F1 Macro: 0.3403
- Accuracy: 0.3709
Model description
More information needed
Intended uses & limitations
More information needed
Test results
Binary classification accuracy (threshold at label 3) ≈ 77.45%
Test Report:
precision recall f1-score support
0 0.81 0.52 0.63 100
1 0.36 0.47 0.41 100
2 0.28 0.52 0.36 100
3 0.28 0.35 0.31 100
4 0.42 0.15 0.22 100
5 0.75 0.06 0.11 50
accuracy 0.37 550
macro avg 0.48 0.34 0.34 550
weighted avg 0.46 0.37 0.36 550
Confusion Matrix:
[[52 39 8 1 0 0]
[10 47 38 5 0 0]
[ 2 34 52 11 1 0]
[ 0 8 49 35 7 1]
[ 0 3 32 50 15 0]
[ 0 1 10 23 13 3]]
Test metrics
epoch = 20.0
eval_accuracy = 0.3709
eval_f1_macro = 0.3403
eval_loss = 1.238
eval_precision = 0.4817
eval_recall = 0.345
eval_runtime = 0:00:06.03
eval_samples_per_second = 91.183
eval_steps_per_second = 2.984
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 0
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 20
Training results
| Training Loss |
Epoch |
Step |
Validation Loss |
Precision |
Recall |
F1 Macro |
Accuracy |
| No log |
0 |
0 |
3.4165 |
0.0891 |
0.1668 |
0.0874 |
0.3526 |
| 0.9463 |
0.3368 |
1000 |
0.9205 |
0.3220 |
0.2806 |
0.2541 |
0.4234 |
| 0.8023 |
0.6736 |
2000 |
0.7874 |
0.4037 |
0.3211 |
0.3133 |
0.47 |
| 0.7599 |
1.0104 |
3000 |
0.7541 |
0.4143 |
0.3319 |
0.3267 |
0.478 |
| 0.7508 |
1.3473 |
4000 |
0.7518 |
0.4155 |
0.3353 |
0.3289 |
0.4884 |
| 0.7342 |
1.6841 |
5000 |
0.7402 |
0.3963 |
0.3310 |
0.3233 |
0.4746 |
| 0.7122 |
2.0209 |
6000 |
0.7442 |
0.4105 |
0.3344 |
0.3293 |
0.4966 |
| 0.7719 |
2.3577 |
7000 |
0.7282 |
0.4580 |
0.3494 |
0.3499 |
0.4808 |
| 0.7346 |
2.6945 |
8000 |
0.7177 |
0.4993 |
0.3573 |
0.3601 |
0.4962 |
| 0.6964 |
3.0313 |
9000 |
0.7203 |
0.4079 |
0.3454 |
0.3390 |
0.4908 |
| 0.6999 |
3.3681 |
10000 |
0.7195 |
0.4250 |
0.3444 |
0.3422 |
0.5068 |
| 0.7036 |
3.7050 |
11000 |
0.7054 |
0.4842 |
0.3565 |
0.3589 |
0.503 |
| 0.6953 |
4.0418 |
12000 |
0.7028 |
0.4887 |
0.3560 |
0.3592 |
0.4856 |
| 0.6808 |
4.3786 |
13000 |
0.7103 |
0.4289 |
0.3594 |
0.3599 |
0.4778 |
| 0.7021 |
4.7154 |
14000 |
0.6918 |
0.4628 |
0.3604 |
0.3622 |
0.4928 |
| 0.6476 |
5.0522 |
15000 |
0.6884 |
0.4275 |
0.3634 |
0.3631 |
0.5062 |
| 0.6674 |
5.3890 |
16000 |
0.6767 |
0.4543 |
0.3651 |
0.3684 |
0.5084 |
| 0.6683 |
5.7258 |
17000 |
0.6924 |
0.5046 |
0.3600 |
0.3638 |
0.519 |
| 0.6491 |
6.0626 |
18000 |
0.6814 |
0.4656 |
0.3624 |
0.3666 |
0.506 |
| 0.6824 |
6.3995 |
19000 |
0.6724 |
0.4554 |
0.3656 |
0.3697 |
0.5026 |
| 0.6366 |
6.7363 |
20000 |
0.6712 |
0.4314 |
0.3670 |
0.3684 |
0.5062 |
| 0.6594 |
7.0731 |
21000 |
0.6686 |
0.4503 |
0.3679 |
0.3716 |
0.5068 |
| 0.6605 |
7.4099 |
22000 |
0.6669 |
0.4441 |
0.3711 |
0.3720 |
0.506 |
| 0.6448 |
7.7467 |
23000 |
0.6716 |
0.4306 |
0.3676 |
0.3687 |
0.4992 |
| 0.6108 |
8.0835 |
24000 |
0.6646 |
0.4573 |
0.3683 |
0.3728 |
0.5184 |
| 0.6202 |
8.4203 |
25000 |
0.6623 |
0.4336 |
0.3691 |
0.3715 |
0.518 |
| 0.6483 |
8.7572 |
26000 |
0.6623 |
0.4434 |
0.3696 |
0.3735 |
0.5142 |
| 0.667 |
9.0940 |
27000 |
0.6569 |
0.4491 |
0.3724 |
0.3752 |
0.5166 |
| 0.6241 |
9.4308 |
28000 |
0.6572 |
0.4368 |
0.3705 |
0.3735 |
0.516 |
| 0.5817 |
9.7676 |
29000 |
0.6552 |
0.4473 |
0.3739 |
0.3782 |
0.5242 |
| 0.6129 |
10.1044 |
30000 |
0.6603 |
0.4585 |
0.3717 |
0.3764 |
0.5218 |
| 0.6051 |
10.4412 |
31000 |
0.6565 |
0.4783 |
0.3725 |
0.3796 |
0.5272 |
| 0.5914 |
10.7780 |
32000 |
0.6514 |
0.4484 |
0.3741 |
0.3769 |
0.517 |
| 0.6089 |
11.1149 |
33000 |
0.6570 |
0.4449 |
0.3729 |
0.3773 |
0.5054 |
| 0.5907 |
11.4517 |
34000 |
0.6520 |
0.4652 |
0.3696 |
0.3740 |
0.5206 |
| 0.6165 |
11.7885 |
35000 |
0.6501 |
0.4627 |
0.3807 |
0.3867 |
0.5248 |
| 0.5896 |
12.1253 |
36000 |
0.6513 |
0.4351 |
0.3735 |
0.3758 |
0.5116 |
| 0.5681 |
12.4621 |
37000 |
0.6480 |
0.4492 |
0.3749 |
0.3806 |
0.5216 |
| 0.6265 |
12.7989 |
38000 |
0.6498 |
0.4651 |
0.3755 |
0.3816 |
0.5246 |
| 0.5862 |
13.1357 |
39000 |
0.6449 |
0.4551 |
0.3732 |
0.3784 |
0.5256 |
| 0.6074 |
13.4725 |
40000 |
0.6480 |
0.4525 |
0.3765 |
0.3804 |
0.5136 |
| 0.5815 |
13.8094 |
41000 |
0.6495 |
0.4510 |
0.3761 |
0.3808 |
0.5312 |
| 0.5591 |
14.1462 |
42000 |
0.6450 |
0.4608 |
0.3757 |
0.3808 |
0.5176 |
| 0.5616 |
14.4830 |
43000 |
0.6444 |
0.4613 |
0.3802 |
0.3854 |
0.525 |
| 0.5735 |
14.8198 |
44000 |
0.6467 |
0.4756 |
0.3707 |
0.3776 |
0.5266 |
| 0.5891 |
15.1566 |
45000 |
0.6450 |
0.4537 |
0.3764 |
0.3808 |
0.525 |
| 0.571 |
15.4934 |
46000 |
0.6431 |
0.4581 |
0.3783 |
0.3846 |
0.5264 |
| 0.5853 |
15.8302 |
47000 |
0.6412 |
0.4751 |
0.3804 |
0.3858 |
0.5242 |
| 0.5796 |
16.1671 |
48000 |
0.6464 |
0.4628 |
0.3810 |
0.3866 |
0.528 |
| 0.5912 |
16.5039 |
49000 |
0.6443 |
0.4560 |
0.3816 |
0.3869 |
0.5242 |
| 0.5302 |
16.8407 |
50000 |
0.6412 |
0.4610 |
0.3800 |
0.3858 |
0.5292 |
| 0.5752 |
17.1775 |
51000 |
0.6400 |
0.4521 |
0.3793 |
0.3845 |
0.5254 |
| 0.5771 |
17.5143 |
52000 |
0.6412 |
0.4678 |
0.3818 |
0.3873 |
0.535 |
| 0.5383 |
17.8511 |
53000 |
0.6393 |
0.4582 |
0.3778 |
0.3838 |
0.5196 |
| 0.5373 |
18.1879 |
54000 |
0.6393 |
0.4571 |
0.3787 |
0.3845 |
0.5234 |
| 0.565 |
18.5248 |
55000 |
0.6390 |
0.4535 |
0.3766 |
0.3821 |
0.5214 |
| 0.5498 |
18.8616 |
56000 |
0.6394 |
0.4596 |
0.3770 |
0.3823 |
0.5282 |
| 0.5643 |
19.1984 |
57000 |
0.6384 |
0.4572 |
0.3777 |
0.3830 |
0.5238 |
| 0.5697 |
19.5352 |
58000 |
0.6390 |
0.4558 |
0.3766 |
0.3819 |
0.5244 |
| 0.547 |
19.8720 |
59000 |
0.6394 |
0.4573 |
0.3787 |
0.3843 |
0.524 |
Framework versions
- Transformers 4.53.2
- Pytorch 2.7.1+cu126
- Datasets 4.0.0
- Tokenizers 0.21.2
