CBDC-BERT: Identifying Central Bank Digital Currency Discourse in Policy Speeches

CBDC-BERT is a sentence classification model fine-tuned to detect CBDC-related statements in English-language central bank speeches. It is built on the domain-adapted checkpoint bilalzafar/CentralBank-BERT—a BERT model pre-trained on over 2 million sentences from BIS central bank speeches (1996–2024)—and uses a WordPiece tokenizer with a maximum input length of 128 tokens.

The model performs binary classification:

0 = Non-CBDC
1 = CBDC

Training data: The dataset was sourced from the BIS Central Bank Speeches corpus (1996–2024). A balanced subset of 11,000 sentences was manually labeled for CBDC relevance. The CBDC class contains 5,390 sentences, and the Non-CBDC class contains 5,610 sentences. The data was split 80/20 (8,800 training / 2,200 test) with stratification by label.

Intended use: CBDC-BERT is intended for research on CBDC discourse across time and jurisdictions, for pre-filtering or flagging CBDC-related sentences in large central-bank speech corpora, and as an input to dashboards, indices, or downstream NLP pipelines used in central banking and finance.

Training Details

Base checkpoint: bilalzafar/CentralBank-BERT
Architecture: BertForSequenceClassification (binary head randomly initialized)
Tokenizer: from base checkpoint, max_length=128
Library: Transformers (Trainer)
Epochs: 3
Batch size: 8 (per device)
Optimizer: AdamW (Transformers default)
Learning rate: 5e-5 (Transformers default)
Loss: CrossEntropyLoss
Evaluation: per epoch; best model by F1
Hardware: Google Colab GPU

Performance & Robustness

On the full test split (n=2,200), the model achieves Accuracy = 0.9950 and F1 (binary) = 0.9949. In a separate confusion-matrix run on valid rows (n=2,175), it records TP=1,065, FP=4, FN=1, TN=1,105, yielding Accuracy = 0.9977, Precision (CBDC) = 0.9963, Recall (CBDC) = 0.9991, ROC-AUC = 1.0000, and a Brier score = 0.0024; the class balance is Non-CBDC = 1,109 and CBDC = 1,066. Compared to TF-IDF baselines—Logistic Regression (0.97), Naive Bayes (0.92), Random Forest (0.98), and XGBoost (0.99), CBDC-BERT matches or exceeds these results while delivering near-perfect ROC-AUC with well-calibrated probabilities (low Brier). Robustness checks across edge cases, noise-injected, syntactically altered, and paraphrased (“translated-like”) inputs each show 8/10 correct (80%), and sentence-length bias is low (ρ ≈ 0.1222).

Other CBDC Models

This model is part of the CentralBank-BERT / CBDC model family, a suite of domain-adapted classifiers for analyzing central-bank communication.

Model	Purpose	Intended Use	Link
bilalzafar/CentralBank-BERT	Domain-adaptive masked LM trained on BIS speeches (1996–2024).	Base encoder for CBDC downstream tasks; fill-mask tasks.	CentralBank-BERT
bilalzafar/CBDC-BERT	Binary classifier: CBDC vs. Non-CBDC.	Flagging CBDC-related discourse in large corpora.	CBDC-BERT
bilalzafar/CBDC-Stance	3-class stance model (Pro, Wait-and-See, Anti).	Research on policy stances and discourse monitoring.	CBDC-Stance
bilalzafar/CBDC-Sentiment	3-class sentiment model (Positive, Neutral, Negative).	Tone analysis in central bank communications.	CBDC-Sentiment
bilalzafar/CBDC-Type	Classifies Retail, Wholesale, General CBDC mentions.	Distinguishing policy focus (retail vs wholesale).	CBDC-Type
bilalzafar/CBDC-Discourse	3-class discourse classifier (Feature, Process, Risk-Benefit).	Structured categorization of CBDC communications.	CBDC-Discourse
bilalzafar/CentralBank-NER	Named Entity Recognition (NER) model for central banking discourse.	Identifying institutions, persons, and policy entities in speeches.	CentralBank-NER

Repository and Replication Package

All training pipelines, preprocessing scripts, evaluation notebooks, and result outputs are available in the companion GitHub repository:

🔗 https://github.com/bilalezafar/CentralBank-BERT

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

tokenizer = AutoTokenizer.from_pretrained("bilalzafar/cbdc-bert")
model = AutoModelForSequenceClassification.from_pretrained("bilalzafar/cbdc-bert")

classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)

label_map = {"LABEL_0": "Non-CBDC", "LABEL_1": "CBDC"}

# Example sentence
text = "The central bank is exploring the issuance of a retail digital currency."
result = classifier(text)[0]

print(f"Prediction: {label_map[result['label']]} | Confidence: {result['score']:.4f}")

# Output example:
# [{Prediction: CBDC | Confidence: 0.9993}]

Citation

If you use this model, please cite as:

Zafar, M. B. (2025). CentralBank-BERT: Machine Learning Evidence on Central Bank Digital Currency Discourse. SSRN. https://papers.ssrn.com/abstract=5404456

@article{zafar2025centralbankbert,
  title={CentralBank-BERT: Machine Learning Evidence on Central Bank Digital Currency Discourse},
  author={Zafar, Muhammad Bilal},
  year={2025},
  journal={SSRN Electronic Journal},
  url={https://papers.ssrn.com/abstract=5404456}
}

bilalzafar
/

CBDC-BERT