datasets:
- samirmsallem/text_parts_de
language:
- de
base_model:
- deepset/gbert-large
pipeline_tag: text-classification
library_name: transformers
tags:
- science
- german
- text
metrics:
- accuracy
model-index:
- name: checkpoints
results:
- task:
name: Text Classification
type: text-classification
dataset:
name: samirmsallem/text_parts_de
type: samirmsallem/text_parts_de
metrics:
- name: Accuracy
type: accuracy
value: 0.7850140056022409
Sequence classification model for text parts classification in German scientific texts
gbert-large-text_parts is a sequence classification model in the scientific domain in German, finetuned from the model gbert-large. It was trained using a custom annotated dataset of around 4,800 training and 1,400 test examples containing introduction, main part and conclusion text sequences from scientific papers and theses in german.
The two German BERT models — gbert-base and gbert-large — were evaluated on the same text classification task. This repository contains the model and results for gbert-large.
- gbert-base reached a maximum evaluation accuracy of 70.31%, with a minimum evaluation loss of 2.13.
- gbert-large achieved a higher peak accuracy of 78.50% and a lower evaluation loss of 1.67.
Despite its superior peak performance, gbert-large exhibited a consistent degradation in evaluation accuracy and an increase in loss over subsequent epochs. This pattern is characteristic of overfitting, where the model fits the training data too closely and fails to generalize well to unseen data. In contrast, gbert-base maintained more stable performance over time, making it a more robust choice in scenarios where computational resources or dataset size are limited.
| Text Classification Tag | Text Classification Label | Description |
|---|---|---|
| 0 | EINLEITUNG | The text is an introduction (Einleitung). |
| 1 | HAUPTTEIL | The text is a main part (Hauptteil). |
| 2 | SCHLUSS | The text is a conclusion (Schluss). |
Training
Training was conducted on a 7 epoch fine-tuning approach:
| Epoch | Loss | Accuracy |
|---|---|---|
| 1.0 | 1.6681 | 0.7850 |
| 2.0 | 2.0727 | 0.7780 |
| 3.0 | 2.2932 | 0.7703 |
| 4.0 | 2.4115 | 0.7682 |
| 5.0 | 3.8431 | 0.6611 |
| 6.0 | 2.9657 | 0.7129 |
| 7.0 | 4.3850 | 0.6492 |
Training was conducted using a standard Text classification objective. The model achieves an accuracy of approximately 78,5% on the evaluation set.
Here are the overall final metrics on the test dataset after 1 epoch of training:
- Accuracy: 0.7850140056022409
- Loss: 1.668112874031067