nvidia
/

Llama-3_3-Nemotron-Super-49B-GenRM-Multilingual

@@ -35,12 +35,41 @@ library_name: transformers
 Llama-3.3-Nemotron-Super-49B-GenRM-Multilingual is a generative reward model that leverages Llama-3.3-Nemotron-Super-49B-v1 as the foundation and is fine-tuned using Reinforcement Learning to predict the quality of LLM generated responses.
 See details on how this model was trained at [https://arxiv.org/abs/2505.11475](https://arxiv.org/abs/2505.11475)
 ## License/Terms of Use:
 GOVERNING TERMS: Use of this model is governed by the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/) . Additional Information: [Llama 3.3 Community License Agreement](https://www.llama.com/llama3_3/license/). Built with Llama.
 ## RM-Bench LeaderBoard
 As of 15 May 2025, our reward models trained with HelpSteer3-Preference are the top performing Bradley-Terry reward models on [RM-Bench](https://arxiv.org/abs/2410.16184), an improved variant of RewardBench for evaluating Reward Models in Chat, Math, Code and Safety. Our GenRMs also outperform the corresponding Bradley-Terry reward models.
@@ -78,34 +107,11 @@ As of 15 May 2025, our reward models trained with HelpSteer3-Preference are the
 *Note that Skywork-Reward-Gemma-2-27B was the best performing reward model reported on JudgeBench and we evaluated all other numbers.*
-## Use Case:
-Llama-3.3-Nemotron-Super-49B-GenRM-Multilingual can be used to judge the quality of one response, or the ranking between two responses given a multilingual conversation history. It will first generate reasoning traces then output an integer score.
-## Release Date:
-05/30/2025
-## Referencess:
-* [HelpSteer3-Preference](https://arxiv.org/abs/2505.11475)
-* [HelpSteer2-Preference](https://arxiv.org/abs/2410.01257)
-* [SteerLM method](https://arxiv.org/abs/2310.05344)
-* [HelpSteer](https://arxiv.org/abs/2311.09528)
-* [HelpSteer2](https://arxiv.org/abs/2406.08673)
-* [Llama-Nemotron: Efficient Reasoning Models](https://arxiv.org/abs/2505.00949)
-* [The future of AI: Built with Llama](https://ai.meta.com/blog/future-of-ai-built-with-llama/)
-* [Meta's Llama 3.3 Webpage](https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_3)
-* [Meta's Llama 3.3 Model Card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)
 ## Model Architecture:
 **Architecture Type:** Transformer <br>
 **Network Architecture:** Llama-3.3-Nemotron-Super-49B-v1 <br>
-We developed this model using Llama-3.3-Nemotron-Super-49B-v1 as its foundation. This model contains 49 billion parameters.
 ## Input:
 **Input Type(s):** Text <br>
@@ -119,19 +125,21 @@ We developed this model using Llama-3.3-Nemotron-Super-49B-v1 as its foundation.
 **Output Parameters:** One-Dimensional (1D) <br>
 **Other Properties Related to Output:**  The output contains a reasoning trace and a final score. <br>
 ## Software Integration:
 **Runtime Engine(s):** <br>
 * vLLM 0.8.3 <br>
 **Supported Hardware Microarchitecture Compatibility:** <br>
 * NVIDIA Ampere <br>
-* NVIDIA Hopper
 **Supported Operating System(s):** Linux <br>
 ## Quick Start
-We recommend serving the model with vLLM. You can use the model with 2 or more 80GB GPUs (NVIDIA Ampere or newer) with at least 100GB of free disk space to accomodate the download.
 ```
 pip install vllm==0.8.3
@@ -221,7 +229,7 @@ Response 2 states "1+2=3", which is accurate and directly addresses the user's q
 ```
 Note that the conversation history should be presented in "user" and "assistant" roles, where the last turn is user turn. The responses to be judged should be in "response_1" (and "response_2") roles.
-### Intepretation of Scores
 When judging one response, the model will generate a helpfulness score from 1 to 5, where higher is better.
 When judging two responses, the model will generate an individual helpfulness score for each response, then a ranking score. The ranking score is a number between 1 and 6, where:
@@ -243,7 +251,7 @@ For details, please see Appendix J in our [paper](https://arxiv.org/abs/2505.114
 ## Model Version:
 v1.0
-# Training and Testing Datasets:
 ## Training Datasets:
@@ -273,6 +281,33 @@ v1.0
 **Properties:** <br>
 * 403 prompts, each with a pair of responses as well as human preferences between the pair of responses.
 # Inference:
 **Engine:** vLLM <br>
@@ -281,6 +316,8 @@ v1.0
 ## Ethical Considerations:
 NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
 Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
 ## Citation

 Llama-3.3-Nemotron-Super-49B-GenRM-Multilingual is a generative reward model that leverages Llama-3.3-Nemotron-Super-49B-v1 as the foundation and is fine-tuned using Reinforcement Learning to predict the quality of LLM generated responses.
+Llama-3.3-Nemotron-Super-49B-GenRM-Multilingual can be used to judge the quality of one response, or the ranking between two responses given a multilingual conversation history. It will first generate reasoning traces then output an integer score. A higher score means the response is of higher quality.
 See details on how this model was trained at [https://arxiv.org/abs/2505.11475](https://arxiv.org/abs/2505.11475)
+This model is ready for commercial/non-commercial use.
 ## License/Terms of Use:
 GOVERNING TERMS: Use of this model is governed by the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/) . Additional Information: [Llama 3.3 Community License Agreement](https://www.llama.com/llama3_3/license/). Built with Llama.
+### Deployment Geography
+Global
+## Use Case:
+Llama-3.3-Nemotron-Super-49B-GenRM-Multilingual can be used to judge the quality of one response, or the ranking between two responses given a multilingual conversation history. It will first generate reasoning traces then output an integer score.
+## Release Date:
+HuggingFace 06/27/2025 via https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-GenRM-Multilingual
+## References:
+* [HelpSteer3-Preference](https://arxiv.org/abs/2505.11475)
+* [HelpSteer2-Preference](https://arxiv.org/abs/2410.01257)
+* [SteerLM method](https://arxiv.org/abs/2310.05344)
+* [HelpSteer](https://arxiv.org/abs/2311.09528)
+* [HelpSteer2](https://arxiv.org/abs/2406.08673)
+* [Llama-Nemotron: Efficient Reasoning Models](https://arxiv.org/abs/2505.00949)
+* [The future of AI: Built with Llama](https://ai.meta.com/blog/future-of-ai-built-with-llama/)
+* [Meta's Llama 3.3 Webpage](https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_3)
+* [Meta's Llama 3.3 Model Card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md)
 ## RM-Bench LeaderBoard
 As of 15 May 2025, our reward models trained with HelpSteer3-Preference are the top performing Bradley-Terry reward models on [RM-Bench](https://arxiv.org/abs/2410.16184), an improved variant of RewardBench for evaluating Reward Models in Chat, Math, Code and Safety. Our GenRMs also outperform the corresponding Bradley-Terry reward models.
 *Note that Skywork-Reward-Gemma-2-27B was the best performing reward model reported on JudgeBench and we evaluated all other numbers.*
 ## Model Architecture:
 **Architecture Type:** Transformer <br>
 **Network Architecture:** Llama-3.3-Nemotron-Super-49B-v1 <br>
+We developed this model using [Llama-3.3-Nemotron-Super-49B-v1](https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1) as its foundation. This model contains 49 billion parameters.
 ## Input:
 **Input Type(s):** Text <br>
 **Output Parameters:** One-Dimensional (1D) <br>
 **Other Properties Related to Output:**  The output contains a reasoning trace and a final score. <br>
+Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. <br>
 ## Software Integration:
 **Runtime Engine(s):** <br>
 * vLLM 0.8.3 <br>
 **Supported Hardware Microarchitecture Compatibility:** <br>
 * NVIDIA Ampere <br>
+* NVIDIA Hopper <br>
 **Supported Operating System(s):** Linux <br>
 ## Quick Start
+We recommend serving the model with vLLM. You can use the model with 2 or more 80GB GPUs (NVIDIA Ampere or newer) with at least 100GB of free disk space to accommodate the download.
 ```
 pip install vllm==0.8.3
 ```
 Note that the conversation history should be presented in "user" and "assistant" roles, where the last turn is user turn. The responses to be judged should be in "response_1" (and "response_2") roles.
+### Interpretation of Scores
 When judging one response, the model will generate a helpfulness score from 1 to 5, where higher is better.
 When judging two responses, the model will generate an individual helpfulness score for each response, then a ranking score. The ranking score is a number between 1 and 6, where:
 ## Model Version:
 v1.0
+# Training, Testing and Evaluation Datasets:
 ## Training Datasets:
 **Properties:** <br>
 * 403 prompts, each with a pair of responses as well as human preferences between the pair of responses.
+## Evaluation Datasets
+**Dataset Name:** RM-Bench <br>
+**Dataset Link:** https://huggingface.co/datasets/THU-KEG/RM-Bench
+**Data Collection Method by dataset** <br>
+* [Hybrid: Human, Synthetic] <br>
+**Labeling Method by dataset** <br>
+* [Hybrid: Human, Synthetic] <br>
+**Properties:** <br>
+* 1,327 prompts, each with three pairs of responses as well as preferences between the pair of responses.
+**Dataset Name:** JudgeBench <br>
+**Dataset Link:** https://huggingface.co/datasets/ScalerLab/JudgeBench
+**Data Collection Method by dataset** <br>
+* [Hybrid: Human, Synthetic] <br>
+**Labeling Method by dataset** <br>
+* [Hybrid: Human, Synthetic] <br>
+**Properties:** <br>
+* 350 prompts, each with a pair of responses as well as preferences between the pair of responses.
 # Inference:
 **Engine:** vLLM <br>
 ## Ethical Considerations:
 NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
+For more detailed information on ethical considerations for this model, please see the Model Card++ [Explainability](explainability.md), [Bias](bias.md), [Safety & Security](safety.md), and [Privacy](privacy.md) Subcards.
 Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
 ## Citation