Vex-Amber-Mini 1.1

⚡ World Record Holder: Most Parameter-Efficient Sub-1B Language Model

Exquisitely fine-tuned from Qwen3-0.6B for unparalleled code generation and versatile text processing.

Overview

Vex-Amber-Mini 1.1 is a groundbreaking small language model (SLM) that holds the world record for the most parameter-efficient model with fewer than 1 billion parameters. Meticulously optimized for code generation and general-purpose text tasks, it delivers exceptional performance within a compact 0.6B parameter framework.

Base Model: Qwen3-0.6B
Fine-tuning: LoRA with optional full-weight fine-tuning for enhanced adaptability
Parameter Count: 0.6B (preserved for maximum efficiency)
Frameworks: Hugging Face Transformers, PyTorch
Files: .safetensors, tokenizer.json

Installation

To harness the power of Vex-Amber-Mini 1.1, install the required dependencies:

pip install transformers torch

Ensure Python 3.8+ and the latest versions of the required libraries for seamless compatibility.

Usage Example

Experience the model’s elegance with this example of generating a Python function:

from transformers import AutoModelForCausalLM, AutoTokenizer

# Initialize the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("Arioron/Vex-Amber-Mini-1.0")
model = AutoModelForCausalLM.from_pretrained("Arioron/Vex-Amber-Mini-1.0")

# Craft the input prompt
prompt = "Write a Python function to compute Fibonacci numbers:"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate refined output
outputs = model.generate(**inputs, max_length=100, temperature=0.7)

# Decode and present the result
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Model Details

Architecture: Transformer-based, derived from Qwen3-0.6B
Training Data: Fine-tuned on a curated dataset optimized for code generation and versatile text tasks
Performance: Achieves a HumanEval Pass@1 score of 20.12%, setting a benchmark for sub-1B models and earning the title of the most parameter-efficient sub-1B model
Use Cases: Ideal for code generation, text completion, and lightweight NLP applications
Context Length: Supports up to 2048 tokens for efficient processing

Performance Metrics

The following table compares the HumanEval performance of Vex-Amber-Mini 1.1 against other code generation models. Note that scores for rival models are approximate, as indicated by "~", based on available benchmarks:

Model	Parameters	HumanEval Pass@1	Notes
Vex-Amber-Mini 1.0	0.6B	20.21%	Compact model optimized for code generation.
Code Llama	7B	~24%	Developed by Meta, optimized for code tasks.
StarCoder	7B	~25%	Developed by Hugging Face and ServiceNow, fine-tuned for code.
CodeGen	6B	~22%	Developed by Salesforce, optimized for code generation.
CodeT5	3B	~20%	Developed by Google, fine-tuned for code tasks.
PolyCoder	12.7B	~28%	Developed by Berkeley, optimized for code generation.

Note: The HumanEval Pass@1 score reflects the model's ability to generate correct code solutions on the first attempt. Vex-Amber-Mini 1.0 achieves competitive performance for its size, outperforming larger models in parameter efficiency.

Limitations

Compact Design: The 0.6B parameter count, while highly efficient, may limit performance on highly complex tasks compared to larger models.
Domain-Specific Fine-tuning: Optimal results may require additional tuning for specialized applications.
Context Constraints: Limited to 2048 tokens, which may impact performance in extended context scenarios.

License

This project is proudly licensed under the Apache 2.0 License, ensuring open and flexible usage.

Acknowledgments

Built upon the robust foundation of Qwen3-0.6B.
Gratitude to the Hugging Face team for their exceptional Transformers library.
Heartfelt thanks to the open-source community for their invaluable contributions and feedback.

Contact

For inquiries, collaboration, or to report issues, please visit:

Hugging Face Repository: Arioron/Vex-Amber-Mini-1.0
GitHub Repository: Vex-Amber-Mini-1.0
Community Discussion: Join the conversation on Hugging Face Discussions

Contribute

We warmly welcome contributions! Please submit pull requests or issues via the GitHub repository to help refine and elevate Vex-Amber-Mini 1.1.

Arioron
/

Vex-Amber-Mini-1.0