ufukuyan/convnext-arcface-product-embeddings-test

ConvNeXt-ArcFace Product Embeddings v1

ConvNeXt-based image embedding model for product unification and visual search on the Trendyol e-commerce catalogue.

Model Details

• Architecture: ConvNeXt-Base (224px) backbone + 512-dim projection head with BatchNorm and ArcFace loss
• Objective: ArcFace with additive angular margin (scale=128, margin=0.25) for improved product similarity learning
• Training Data: Large-scale Trendyol product image dataset covering diverse e-commerce categories
• Hardware: Multi-GPU training with PyTorch Lightning (training epoch: 5, global steps: 108)
• Framework: PyTorch Lightning 1.8.1 with mixed-precision training

Intended Use

• Primary – Generate embeddings for duplicate product detection ("unification"), near-duplicate search, and product similarity ranking in e-commerce applications
• Secondary – Feature extractor for image-based product recommendation systems and visual search
• Downstream Tasks – Product clustering, visual search, duplicate detection, and content-based product recommendation

Usage

Complete example to load the model and generate embeddings:

import torch
import torch.nn as nn
import torch.nn.functional as F
import timm
import json
from safetensors.torch import load_file
from PIL import Image
import torchvision.transforms as transforms
import requests

# 1. Define the model class
class TYArcFaceModel(nn.Module):
    def __init__(self, config):
        super().__init__()
        self.config = config
        self.backbone = timm.create_model(
            config['backbone_name'],
            pretrained=False,
            num_classes=0
        )
        self.bn1 = nn.BatchNorm2d(config['backbone_features'])
        self.fc11 = nn.Linear(
            config['backbone_features'] * config['hidden_size'],
            config['embedding_dim']
        )
        self.bn11 = nn.BatchNorm1d(config['embedding_dim'])

    def forward(self, x):
        features = self.backbone.forward_features(x)
        features = self.bn1(features)
        features = features.flatten(start_dim=1)
        features = self.fc11(features)
        features = self.bn11(features)
        features = F.normalize(features, p=2, dim=1)
        return features

# 2. Load the model
device = "cuda" if torch.cuda.is_available() else "cpu"

# Load configuration and weights
config = json.load(open('config.json'))
model = TYArcFaceModel(config)
state_dict = load_file('model.safetensors')

# Filter to only load compatible weights
model_keys = set(model.state_dict().keys())
filtered_state_dict = {k: v for k, v in state_dict.items() if k in model_keys}

model.load_state_dict(filtered_state_dict, strict=False)
model.to(device)
model.eval()

print(f"✅ Model loaded successfully!")
print(f"📊 Ready to generate {config['embedding_dim']}-dimensional embeddings")

# 3. Define preprocessing transforms
transform = transforms.Compose([
    transforms.Resize((config['input_size'], config['input_size'])),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=config['normalization']['mean'],
        std=config['normalization']['std']
    )
])

# 4. Process an image and generate embeddings
def get_embeddings(image_path_or_url):
    """Get embeddings for a single image"""
    # Load image
    if image_path_or_url.startswith('http'):
        image = Image.open(requests.get(image_path_or_url, stream=True).raw).convert('RGB')
    else:
        image = Image.open(image_path_or_url).convert('RGB')

    # Preprocess
    input_tensor = transform(image).unsqueeze(0).to(device)

    # Generate embeddings
    with torch.no_grad():
        embeddings = model(input_tensor)

    return embeddings

# 5. Example usage
image_url = "https://example.com/product_image.jpg"  # Replace with your image
embeddings = get_embeddings(image_url)
print(f"Embedding shape: {embeddings.shape}")  # torch.Size([1, 512])

# 6. Compute similarity between two products
def compute_similarity(embedding1, embedding2):
    """Compute cosine similarity between two embeddings"""
    return F.cosine_similarity(embedding1, embedding2, dim=1)

# Example: Compare two products
# embedding2 = get_embeddings("path/to/another/image.jpg")
# similarity_score = compute_similarity(embeddings, embedding2)
# print(f"Product similarity: {similarity_score.item():.4f}")

Model Performance

The model has been trained using ArcFace loss which provides several advantages for product similarity tasks:

• Improved Discriminative Power: ArcFace adds angular margin in the feature space, creating better separation between different products
• Normalized Embeddings: All output embeddings are L2-normalized, making cosine similarity computation efficient
• Scale Robustness: The learned representations are robust to scale variations in product images

Training Configuration

• Backbone: ConvNeXt-Base pretrained on ImageNet-22k and fine-tuned on ImageNet-1k
• Embedding Dimension: 512
• ArcFace Scale: 128
• ArcFace Margin: 0.25
• Input Resolution: 224×224
• Normalization: ImageNet statistics
• Training Framework: PyTorch Lightning 1.8.1

Limitations

• Domain Specificity: Optimized for e-commerce product images; may not generalize well to other image domains
• Image Quality: Performance may degrade on low-quality, heavily compressed, or significantly distorted images
• Category Bias: Performance may vary across different product categories based on training data distribution
• Scale Dependency: Input images should be resized to 224×224 for optimal performance

Bias Analysis

• Dataset Bias: The model's embeddings may reflect biases present in the e-commerce training dataset
• Product Category Imbalance: Some product categories may be over-represented in the training data
• Brand and Style Bias: The model may learn to encode brand-specific or style-specific features that could affect similarity judgments

Environmental Impact

• Training Hardware: Multi-GPU setup with PyTorch Lightning
• Training Time: 5 epochs with 108 global steps
• Energy Consumption: Estimated moderate carbon footprint due to relatively short training duration

Ethical Considerations

• Commercial Use: Designed for e-commerce applications; consider potential impacts on market competition
• Privacy: Ensure compliance with data protection regulations when processing product images
• Fairness: Monitor for biased similarity judgments across different product categories or brands

Citation

@misc{trendyol2025convnextarcface,
  title={ConvNeXt-ArcFace Product Embeddings v1: High-Fidelity Image Embeddings for E-commerce Product Unification},
  author={Trendyol Data Science Team},
  year={2025},
  howpublished={\url{https://huggingface.co/trendyol/convnext-arcface-product-embeddings-v1}}
}

Model Card Authors

• Trendyol Data Science Team
• Model trained using the TYArcFace architecture with ConvNeXt backbone

License

This model is released under the Apache 2.0 License. See the LICENSE file for more details.

---

For technical support or questions about this model, please contact the Trendyol Data Science team.