Almost every large language model that reaches a broad audience is quantized.
It is trained at full precision, then compressed so it can run affordably on a phone, a laptop, or a fraction of the hardware the original needed. Compression is what makes local and low-cost AI economically possible. It is also treated as a non-event for safety: the model is evaluated at full precision, the compressed build inherits the evaluation, and nobody re-checks.
We built a benchmark to find out whether that assumption holds. It does not.
🔗 Paper: arXiv:2607.21063
🔗 Benchmark, harness and judge rubrics: huggingface.co/datasets/emilioferrara/quantibias
An unusually clean experiment
Quantization is a rare thing in this field: a perturbation you can isolate. It alters no training data, no objective, and no prompt. It changes only the numerical precision of the weights. Hold the model, its training and the prompts fixed, vary precision alone, and whatever moves is attributable to compression and nothing else.
So that is what we did, across two backbone models (Qwen and Gemma), a five-family screen, and eight benchmarks.
The safety checks keep passing
A quantized model still refuses harmful requests. It still avoids over-refusing benign ones, which is the failure mode safety tuning most often overshoots into. Asked a multiple-choice question designed to detect stereotyping, it still selects the unbiased answer.
On the instruments the field actually uses to certify models, compression looks harmless.
The open-ended answers do not
Ask the same compressed model an open question and it volunteers stereotypes, in all eight languages we probe, in roughly one in four answers as scored by an independent judge model, between about 24% and 27% across the compression ladder.
The gap between those two results is the finding, and it is worth stating precisely. It is not that compressing harder reliably produces more bias; that turns out to be the less certain claim, and it moves depending on which judge does the scoring. What is robust is the selective gap: the checks and the harm are looking at different things.
Standard safety measurement is dominated by short exchanges. Does the model refuse? Does it over-refuse? Does it pick the right option? Those are precisely the interactions where quantization does the least damage. The sustained, open-ended generation where it does the most is the part the benchmarks never sample.
Why this is unlikely to fix itself
Quantization methods decide where to spend a limited precision budget by looking at a small sample of calibration data. That data is chosen to preserve capability. It carries no bias-prevention signal at all.
The compressed model is optimized to keep being good at things, with nothing in the procedure asking it to keep being fair. There is no reason to expect better behavior to emerge from better compression alone, because fairness is not in the objective.
One mitigation appears in the data, and it is instructive precisely because it is partial. Prompting the model to reason before answering roughly halves the effect on some model families, while doing nothing at all on others. Mitigations validated on one model do not transfer by default.
A measurement detail that compounds the problem
A build’s nominal precision label understates how much it has actually been compressed, by a factor of three to fourteen in the cases we examined. The build a user interacts with is frequently not the one that was evaluated, and the label does not tell you how far apart they are.
What QuantiBias contains
The benchmark pairs a generative, multilingual stereotype probe with the refusal and multiple-choice controls that isolate open-ended generation, contrasts each build with and without reasoning, and rates the content severity of what it generates.
The release ships the harness, the judge rubrics, the reconstruction manifests and the baselines. It deliberately ships no prompt data: all eight probes are external and are reconstructed from their original sources, so the benchmark cannot silently drift from the datasets it claims to measure.
The practical recommendation
A quantized build should be re-evaluated for open-ended bias, not only on the short-form safeguards it already passes. Evaluations should record the precision and configuration of the artifact actually tested. And any suite used for something resembling certification should include sustained open-ended generation, in more than one language, because that is where the measurable harm lives.
A model can pass every test we put in front of it and still reach users measurably more biased. That is not a failure of the model. It is a failure of where we are pointing the instruments.