Surgical Smoke Type Classifier

This repository contains a PyTorch-based image classifier for identifying and classifying different types of smoke in surgical footage. The model distinguishes between six classes:

debris
haze
none (clear)
opaque
stroke
turbulent

It uses a MobileNetV2 backbone via timm, and is compatible with Hugging Face Transformers' AutoModel and AutoConfig using trust_remote_code=True.

The name "classifier-RARP" refers to the training data source, solely the DaVinci RARP surgical frames.

Training log: gs://noee/mobileNet/RARP_12-03-2025/Run_20h17_lr0.01_pretrained_ReduceLROnPlateau/training.log

Files

classifier.py: Model and config implementation.
config.json: Hugging Face model configuration.
pytorch_model.bin: Model weights.
sample_img.png: Example image for inference.
example_inference.py: Example script for running inference.

Usage

Installation

Install required dependencies:

pip install torch torchvision timm transformers pillow

Model Details

Backbone: MobileNetV2 (mobilenetv2_100)
Classes: debris, haze, none, opaque, stroke, turbulent
Input size: 224x224 RGB images
Normalization: mean=[0.6075, 0.4093, 0.3609], std=[0.2066, 0.2036, 0.1991]

Output : A list of dictionaries with :

{
  "label": <predicted_class>,  # e.g., "haze", "stroke", etc.
  "confidences": {
      "debris": 0.01,
      "haze": 0.97,
      "none": 0.01,
      "opaque": 0.00,
      "stroke": 0.00,
      "turbulent": 0.01
  }
}

Inference Example

You can run the provided script :

# example_inference.py
from transformers import AutoModel
from PIL import Image

# Load model
model = AutoModel.from_pretrained(
    "vopeai/smoke-classifier",
    trust_remote_code=True
)
model.eval()

# Load and preprocess image
img = Image.open("sample_img.png").convert("RGB")

# Run inference
outputs = model(img)

print("Predicted class:", outputs[0]['label'])
print("Confidences:", outputs[0]['confidences'])

Expected output for sample image :

Sample surgical frame

Predicted class: haze
Confidences: {'debris': 0.0, 'haze': 0.926, 'none': 0.0, 'opaque': 0.0, 'stroke': 0.001, 'turbulent': 0.073}

Or use the model in your own code, by loading the model as follows :

from transformers import AutoModel

# Load model
model = AutoModel.from_pretrained("vopeai/classifier-RARP", trust_remote_code=True)

For more details, see the code files in this repository.