Hugging Face's logo

faodl
/
20250909_model_g20_multilabel_MiniLM-L12-all-labels-artificial-governance-v03

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Model card Files
xet
Community

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A OneVsRestClassifier instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("faodl/20250909_model_g20_multilabel_MiniLM-L12-all-labels-artificial-governance-v03")
# Run inference
preds = model("The program mainly aims at 
the construction of rural roads, capacity building of local bodies, and 
awareness raising activities.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 1 41.6795 1753

Training Hyperparameters

  • batch_size: (32, 32)
  • num_epochs: (1, 1)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 50
  • body_learning_rate: (2e-05, 2e-05)
  • head_learning_rate: 2e-05
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • l2_weight: 0.01
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0000 1 0.1864 -
0.0020 50 0.1899 -
0.0039 100 0.1866 -
0.0059 150 0.1816 -
0.0078 200 0.1783 -
0.0098 250 0.1743 -
0.0117 300 0.1685 -
0.0137 350 0.1613 -
0.0156 400 0.1533 -
0.0176 450 0.1393 -
0.0196 500 0.1403 -
0.0215 550 0.1276 -
0.0235 600 0.1153 -
0.0254 650 0.1155 -
0.0274 700 0.1074 -
0.0293 750 0.1092 -
0.0313 800 0.1014 -
0.0332 850 0.1005 -
0.0352 900 0.0983 -
0.0372 950 0.0951 -
0.0391 1000 0.0935 -
0.0411 1050 0.0987 -
0.0430 1100 0.0936 -
0.0450 1150 0.092 -
0.0469 1200 0.093 -
0.0489 1250 0.0843 -
0.0508 1300 0.0859 -
0.0528 1350 0.0863 -
0.0001 1 0.0762 -
0.0039 50 0.0869 -
0.0001 1 0.0506 -
0.0039 50 0.084 -
0.0078 100 0.0841 -
0.0117 150 0.0796 -
0.0156 200 0.0821 -
0.0196 250 0.0797 -
0.0235 300 0.0861 -
0.0274 350 0.0827 -
0.0313 400 0.0723 -
0.0352 450 0.0715 -
0.0391 500 0.0762 -
0.0430 550 0.0642 -
0.0469 600 0.07 -
0.0508 650 0.0738 -
0.0548 700 0.0684 -
0.0587 750 0.0679 -
0.0626 800 0.0697 -
0.0665 850 0.0651 -
0.0704 900 0.0668 -
0.0743 950 0.0656 -
0.0782 1000 0.0654 -
0.0821 1050 0.0567 -
0.0860 1100 0.0636 -
0.0899 1150 0.0625 -
0.0939 1200 0.0614 -
0.0978 1250 0.0619 -
0.1017 1300 0.0641 -
0.1056 1350 0.0574 -
0.1095 1400 0.0585 -
0.1134 1450 0.0575 -
0.1173 1500 0.052 -
0.1212 1550 0.0506 -
0.1251 1600 0.0537 -
0.1291 1650 0.0505 -
0.1330 1700 0.0476 -
0.1369 1750 0.0515 -
0.1408 1800 0.0464 -
0.1447 1850 0.0484 -
0.1486 1900 0.0459 -
0.1525 1950 0.0474 -
0.1564 2000 0.0453 -
0.1603 2050 0.0467 -
0.1643 2100 0.0455 -
0.1682 2150 0.0419 -
0.1721 2200 0.0473 -
0.1760 2250 0.0435 -
0.1799 2300 0.0454 -
0.1838 2350 0.0403 -
0.1877 2400 0.04 -
0.1916 2450 0.041 -
0.1955 2500 0.0389 -
0.1995 2550 0.0396 -
0.2034 2600 0.0438 -
0.2073 2650 0.0375 -
0.2112 2700 0.0361 -
0.2151 2750 0.0423 -
0.2190 2800 0.0377 -
0.2229 2850 0.0375 -
0.2268 2900 0.0368 -
0.2307 2950 0.0386 -
0.2346 3000 0.0366 -
0.2386 3050 0.0316 -
0.2425 3100 0.0337 -
0.2464 3150 0.0337 -
0.2503 3200 0.0404 -
0.2542 3250 0.0307 -
0.2581 3300 0.0347 -
0.2620 3350 0.0329 -
0.2659 3400 0.0296 -
0.2698 3450 0.0339 -
0.2738 3500 0.0369 -
0.2777 3550 0.0312 -
0.2816 3600 0.035 -
0.2855 3650 0.0325 -
0.2894 3700 0.0307 -
0.2933 3750 0.0323 -
0.2972 3800 0.0288 -
0.3011 3850 0.0263 -
0.3050 3900 0.0337 -
0.3090 3950 0.0332 -
0.3129 4000 0.0257 -
0.3168 4050 0.0262 -
0.3207 4100 0.0324 -
0.3246 4150 0.0309 -
0.3285 4200 0.0264 -
0.3324 4250 0.0307 -
0.3363 4300 0.0257 -
0.3402 4350 0.0264 -
0.3442 4400 0.0271 -
0.3481 4450 0.0255 -
0.3520 4500 0.0249 -
0.3559 4550 0.0263 -
0.3598 4600 0.0234 -
0.3637 4650 0.0245 -
0.3676 4700 0.0287 -
0.3715 4750 0.0284 -
0.3754 4800 0.0242 -
0.3794 4850 0.0256 -
0.3833 4900 0.025 -
0.3872 4950 0.0209 -
0.3911 5000 0.0245 -
0.3950 5050 0.0271 -
0.3989 5100 0.0274 -
0.4028 5150 0.026 -
0.4067 5200 0.0245 -
0.4106 5250 0.027 -
0.4145 5300 0.0266 -
0.4185 5350 0.0288 -
0.4224 5400 0.0217 -
0.4263 5450 0.0228 -
0.4302 5500 0.0199 -
0.4341 5550 0.0254 -
0.4380 5600 0.0181 -
0.4419 5650 0.0235 -
0.4458 5700 0.0247 -
0.4497 5750 0.024 -
0.4537 5800 0.0239 -
0.4576 5850 0.0259 -
0.4615 5900 0.0209 -
0.4654 5950 0.021 -
0.4693 6000 0.0227 -
0.4732 6050 0.0265 -
0.4771 6100 0.0255 -
0.4810 6150 0.0227 -
0.4849 6200 0.0229 -
0.4889 6250 0.0231 -
0.4928 6300 0.0248 -
0.4967 6350 0.0198 -
0.5006 6400 0.0217 -
0.5045 6450 0.0246 -
0.5084 6500 0.0209 -
0.5123 6550 0.0206 -
0.5162 6600 0.0214 -
0.5201 6650 0.0222 -
0.5241 6700 0.0185 -
0.5280 6750 0.0188 -
0.5319 6800 0.0214 -
0.5358 6850 0.0248 -
0.5397 6900 0.0212 -
0.5436 6950 0.0201 -
0.5475 7000 0.0201 -
0.5514 7050 0.0248 -
0.5553 7100 0.022 -
0.5592 7150 0.0181 -
0.5632 7200 0.0194 -
0.5671 7250 0.0211 -
0.5710 7300 0.0202 -
0.5749 7350 0.022 -
0.5788 7400 0.0238 -
0.5827 7450 0.019 -
0.5866 7500 0.0165 -
0.5905 7550 0.0191 -
0.5944 7600 0.023 -
0.5984 7650 0.0187 -
0.6023 7700 0.0254 -
0.6062 7750 0.0213 -
0.6101 7800 0.0259 -
0.6140 7850 0.0225 -
0.6179 7900 0.0207 -
0.6218 7950 0.0166 -
0.6257 8000 0.0215 -
0.6296 8050 0.0176 -
0.6336 8100 0.02 -
0.6375 8150 0.0208 -
0.6414 8200 0.0186 -
0.6453 8250 0.0179 -
0.6492 8300 0.0173 -
0.6531 8350 0.0216 -
0.6570 8400 0.0212 -
0.6609 8450 0.0213 -
0.6648 8500 0.0191 -
0.6688 8550 0.0212 -
0.6727 8600 0.0184 -
0.6766 8650 0.0202 -
0.6805 8700 0.0215 -
0.6844 8750 0.0163 -
0.6883 8800 0.018 -
0.6922 8850 0.0178 -
0.6961 8900 0.0175 -
0.7000 8950 0.0155 -
0.7039 9000 0.0201 -
0.7079 9050 0.0168 -
0.7118 9100 0.0194 -
0.7157 9150 0.0191 -
0.7196 9200 0.0183 -
0.7235 9250 0.0181 -
0.7274 9300 0.0191 -
0.7313 9350 0.0179 -
0.7352 9400 0.0218 -
0.7391 9450 0.0178 -
0.7431 9500 0.0175 -
0.7470 9550 0.0168 -
0.7509 9600 0.0192 -
0.7548 9650 0.0183 -
0.7587 9700 0.0167 -
0.7626 9750 0.0189 -
0.7665 9800 0.021 -
0.7704 9850 0.0176 -
0.7743 9900 0.0177 -
0.7783 9950 0.0169 -
0.7822 10000 0.0191 -
0.7861 10050 0.0147 -
0.7900 10100 0.0192 -
0.7939 10150 0.0174 -
0.7978 10200 0.017 -
0.8017 10250 0.0155 -
0.8056 10300 0.0179 -
0.8095 10350 0.0192 -
0.8135 10400 0.0153 -
0.8174 10450 0.0195 -
0.8213 10500 0.0196 -
0.8252 10550 0.0192 -
0.8291 10600 0.0148 -
0.8330 10650 0.0175 -
0.8369 10700 0.0146 -
0.8408 10750 0.0178 -
0.8447 10800 0.015 -
0.8487 10850 0.0192 -
0.8526 10900 0.0163 -
0.8565 10950 0.0168 -
0.8604 11000 0.0163 -
0.8643 11050 0.0148 -
0.8682 11100 0.0161 -
0.8721 11150 0.0189 -
0.8760 11200 0.0196 -
0.8799 11250 0.0138 -
0.8838 11300 0.0164 -
0.8878 11350 0.0156 -
0.8917 11400 0.0149 -
0.8956 11450 0.0177 -
0.8995 11500 0.0183 -
0.9034 11550 0.0157 -
0.9073 11600 0.018 -
0.9112 11650 0.0127 -
0.9151 11700 0.0165 -
0.9190 11750 0.0181 -
0.9230 11800 0.0157 -
0.9269 11850 0.0157 -
0.9308 11900 0.0159 -
0.9347 11950 0.0125 -
0.9386 12000 0.0175 -
0.9425 12050 0.018 -
0.9464 12100 0.0181 -
0.9503 12150 0.0173 -
0.9542 12200 0.0182 -
0.9582 12250 0.0189 -
0.9621 12300 0.0124 -
0.9660 12350 0.0175 -
0.9699 12400 0.0139 -
0.9738 12450 0.0161 -
0.9777 12500 0.0168 -
0.9816 12550 0.019 -
0.9855 12600 0.0195 -
0.9894 12650 0.0184 -
0.9934 12700 0.0148 -
0.9973 12750 0.0172 -

Framework Versions

  • Python: 3.12.11
  • SetFit: 1.1.3
  • Sentence Transformers: 5.1.0
  • Transformers: 4.56.1
  • PyTorch: 2.8.0+cu126
  • Datasets: 4.0.0
  • Tokenizers: 0.22.0

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
Downloads last month
20
Safetensors
Model size
118M params
Tensor type
F32
·
Inference Providers NEW
Text Classification
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for faodl/20250909_model_g20_multilabel_MiniLM-L12-all-labels-artificial-governance-v03

Base model

sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Finetuned
(245)
this model