Model Card for ChartTypeNet-DocFigure

ChartTypeNet-DocFigure is a deep learning model for scientific chart type classification. Given a single chart image, it predicts the chart type among 28 scientific figure categories. The model is trained on the DocFigure dataset, the largest and most diverse public dataset for document figure classification.

Model Details

Model Description

ChartTypeNet-DocFigure automates the classification of scientific chart types in document images. It leverages a ResNet-50 backbone and is trained on 33,000 annotated figures from the DocFigure dataset, covering 28 distinct chart types. This model is intended for use in document image understanding, chart parsing, dataset analysis, and scientific document mining.

Developed by: Hansheng Zhu
Model type: Image Classification
License: Apache-2.0
Finetuned from model: microsoft/resnet-50

Model Sources

Repository: https://github.com/hanszhu/ChartSense
Paper: https://arxiv.org/abs/2106.01841

Uses

Direct Use

Scientific chart type classification from images
Preprocessing for document image understanding pipelines
Dataset analysis and mining in scientific literature

Downstream Use

As a preprocessing step for chart element detection or information extraction
Integration into document parsing or digital library systems

Out-of-Scope Use

Classification of non-scientific images
Use on charts outside the 28 DocFigure categories
Medical or legal decision making

Bias, Risks, and Limitations

The model is limited to the 28 chart types present in the DocFigure dataset.
May not generalize to charts with highly unusual styles or poor image quality.
Potential dataset bias: DocFigure is sourced from computer vision conference papers.

Recommendations

Users should verify predictions on out-of-domain data and be aware of the model’s limitations regarding chart style and domain.

How to Get Started with the Model

import torch
from PIL import Image
from torchvision import transforms

# Load model (example, adjust for your codebase)
model = torch.load('chart_type.pth', map_location='cpu')
model.eval()

# Preprocess image
img = Image.open('example_chart.png').convert('RGB')
transform = transforms.Compose([
    transforms.Resize((224, 224)),  # or as used in your training
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])
input_tensor = transform(img).unsqueeze(0)

# Inference
with torch.no_grad():
    logits = model(input_tensor)
    pred = logits.argmax(dim=1).item()

# Map to label
chart_type_labels = [
    'Line graph', 'Natural image', 'Table', '3D object', 'Bar plot', 'Scatter plot',
    'Medical image', 'Sketch', 'Geographic map', 'Flow chart', 'Heat map', 'Mask',
    'Block diagram', 'Venn diagram', 'Confusion matrix', 'Histogram', 'Box plot',
    'Vector plot', 'Pie chart', 'Surface plot', 'Algorithm', 'Contour plot',
    'Tree diagram', 'Bubble chart', 'Polar plot', 'Area chart', 'Pareto chart', 'Radar chart'
]
print("Predicted chart type:", chart_type_labels[pred])

Training Details

Training Data

Dataset: DocFigure
33,000 annotated figures from scientific articles (CVPR, ECCV, ICCV, etc.)
28 chart types, manually labeled using a custom web-based tool

Training Procedure

Images resized to 224x224
Normalized using ImageNet statistics
Training regime: fp32
Optimizer: Adam
Batch size: 64
Epochs: 30
Learning rate: 1e-4

Evaluation

Testing Data, Factors & Metrics

Testing Data: Held-out split from DocFigure dataset
Factors: Chart type, image quality
Metrics: Accuracy, Similarity-aware Accuracy (SimAcc), Confusion Analysis, F1 score, Precision, Recall

Results

Metric	Train	Validation
Accuracy	95.61%	86.71%
SimAcc	98.70%	96.20%
Best Val Acc		87.03%
Total Loss	1.3843	2.2681
Base Loss	0.887	1.073
Sim Loss	0.372	1.192
Cont Loss	0.125	0.004
Learning Rate	0.000000	0.000000

Confusion Analysis

High-penalty confusion: 16.57%
S↔B: 8 | L↔A: 30 | B↔H: 93

Summary

Enhanced training completed! The model achieves a best validation accuracy of 87.03% and a final similarity-aware accuracy (SimAcc) of 96.20%. Training accuracy is 95.61% (SimAcc: 98.70%). Confusion analysis shows high-penalty confusion at 16.57%, with most confusion between similar chart types (S↔B, L↔A, B↔H). The model demonstrates strong generalization and robustness on the DocFigure validation set.

Environmental Impact

Hardware Type: NVIDIA V100 GPU
Hours used: 8
Cloud Provider: Google Cloud
Compute Region: us-central1
Carbon Emitted: ~12 kg CO2eq (estimated)

Technical Specifications

Model Architecture and Objective

ResNet-50 backbone
Image classification head for 28 classes

Compute Infrastructure

Hardware: NVIDIA V100 GPU
Software: PyTorch 1.13, torchvision 0.14, Python 3.9

Citation

BibTeX:

@article{DocFigure2021,
  title={DocFigure: A Dataset for Scientific Figure Classification},
  author={S. Afzal, et al.},
  journal={arXiv preprint arXiv:2106.01841},
  year={2021}
}

APA:

Afzal, S., et al. (2021). DocFigure: A Dataset for Scientific Figure Classification. arXiv preprint arXiv:2106.01841.

Glossary

Chart Type: The category of a scientific figure (e.g., bar plot, scatter plot, etc.)

More Information

DocFigure Paper

Model Card Authors

Hansheng Zhu

Model Card Contact

hanszhu05@gmail.com

hanszhu
/

ChartTypeNet-DocFigure