Skip to content

Conversation

metascroy
Copy link
Contributor

Adds a new INT8-INT4-HQQ to the model release script. I find HQQ improves quality for MMLU on every model I tried:

MODEL			INT8-INT4		INT8-INT4-HQQ
---------------------------------------------
Gemma3			0.50			0.55
SmolLM3-3B		0.5397			0.5546
Phi4-mini-4B	0.6050			0.6156
Qwen3-4B		0.6526			0.6585			
Llama1B			0.2892			0.3153	

@metascroy metascroy requested a review from jerryzh168 October 7, 2025 00:18
Copy link

pytorch-bot bot commented Oct 7, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3127

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 1dd0939 with merge base c96f2dd (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 7, 2025
@jerryzh168 jerryzh168 added the topic: not user facing Use this tag if you don't want this PR to show up in release notes label Oct 7, 2025
@metascroy metascroy merged commit d446acd into main Oct 7, 2025
18 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. topic: not user facing Use this tag if you don't want this PR to show up in release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants