Update README.md

README.md

@@ -17,7 +17,7 @@ tags:
 
 MiniGuard-v0.1 is a compact content safety classifier fine-tuned from [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B). It classifies content in both, User inputs (prompt classification) and LLM responses (response classification), outputting whether a given prompt or response is safe or unsafe, along with the violated safety categories.
 
-MiniGuard-v0.1 achieves **~99% of Nemotron-Guard-8B's benchmark accuracy** with **13x fewer parameters**.
+MiniGuard-v0.1 achieves **~99% of Nemotron-Guard-8B's benchmark accuracy** with **13x fewer parameters** and **outperforms Qwen3Guard-8B** (a specialized 8B safety model) **by 9 percentage points** on production data.
 
 
 ## Compatibility
@@ -56,14 +56,16 @@ Dataset - English subset test split of [nvidia/Nemotron-Safety-Guard-Dataset-v3]
 
 Evaluated on out-of-distribution production data containing real user queries. Cost estimated based on H200 GPU pricing ($7.91/hour) at maximum concurrency with P95 latency SLA of <500ms.
 
-| Metric | MiniGuard-v0.1 | Nemotron-Guard-8B-v3 |
-|--------|----------------|----------------------|
-| Relative Macro F1 | 91.1% | 100% |
-| Cost per 1M requests | **$15.54** | $46.93 |
-| Cost Savings | **67%** | - |
+| Model | Parameters | Rel. Macro F1 | Cost per 1M requests | Cost Savings |
+|--------|------------|---------------|----------------------|--------------|
+| **MiniGuard-v0.1** | **0.6B** | **91.1%** | **$15.54** | **67%** |
+| Qwen3Guard-Gen-0.6B | 0.6B | 72.1% | - | - |
+| Qwen3Guard-Gen-4B | 4B | 78.0% | - | - |
+| Qwen3Guard-Gen-8B | 8B | 82.1% | - | - |
+| Nemotron-Guard-8B-v3 | 8B | 100% | $46.93 | baseline |
 
 
-MiniGuard-v0.1 achieves 91.1% relative performance on out-of-distribution data while costing **67% less** to serve.
+MiniGuard-v0.1 achieves 91.1% relative performance on out-of-distribution data while costing **67% less** to serve. **Notably, our 0.6B fine-tuned model outperforms all Qwen3Guard models by significant margins**, including the 8B version (82.1%), demonstrating that targeted fine-tuning is more effective than simply using larger pretrained safety models.
 
 ### Ablation Study
 
@@ -77,6 +79,10 @@ Impact of techniques on out-of-distribution production data (Relative Macro F1 c
 | + Targeted Synthetic Data | 0.6B | 87.2% | +1.6% |
 | + Soup (top-3) [MiniGuard-v0.1] | 0.6B | 92.3% | +5.1% |
 | + FP8 | 0.6B | 91.1% | -1.2% |
+| **Comparison Baselines:** |
+| Qwen3Guard-Gen-0.6B | 0.6B | 72.1% | - |
+| Qwen3Guard-Gen-4B | 4B | 78.0% | - |
+| Qwen3Guard-Gen-8B | 8B | 82.1% | - |
 | Nemotron-Guard-8B-v3 | 8B | 100% | reference |
 
 #### In-Distribution