QDA uses an LLM agent to design quantization recipes for each model

Post-training quantization (PTQ) is the standard way to serve large language models cheaply, but the paper says the best recipe differs across models and precisions. Experts therefore have to redesign it for every new model, which the authors call time-consuming and often sub-optimal. Their premise is that an LLM agent can take over this design by refining the recipe over rounds of building and evaluating candidates, so that more rounds lead to better recipes.
Two costs get in the way, according to the paper. Evaluating one quantized reasoning model takes tens of GPU-hours. And combining methods from different codebases means porting each one to the new architecture by hand.
The paper presents QDA, a system in which an agent designs the recipe for each model and precision, and it attacks both costs. To make evaluation fast, QDA scores each candidate in minutes by its output KL divergence from the original model, measured on reasoning traces that the agent never sees. Promising candidates are then checked with long generations, to catch failures such as repetition loops. To make building fast, QDA provides a library of public PTQ methods that are compatible with one another and with new model architectures.
The reported results: across four reasoning models, QDA improves accuracy over the strongest public recipe by 0.9, 1.4 and 2.2 percentage points on average at W4A16, W4A8 and W3A16 respectively, using 64 GPU-hours per search. On the 320B-parameter MoE model GLM-5.3-Flash at NVFP4 W4A4, QDA recovers 98.3% of the FP8 model's accuracy, against 96.9% for the best vendor checkpoint. The authors add that every QDA checkpoint runs on existing serving kernels with no inference overhead. They say they will release the library, the evaluator and the harness.
Key facts
- QDA is a system in which an LLM agent designs the post-training quantization recipe for each model and precision, instead of experts redoing it by hand.
- Candidates are scored in minutes by output KL divergence from the original model on reasoning traces the agent never sees; promising ones get long-generation checks for failures such as repetition loops.
- Across four reasoning models, average accuracy gains over the strongest public recipe are 0.9 points at W4A16, 1.4 at W4A8 and 2.2 at W3A16, at 64 GPU-hours per search.
- On the 320B-parameter MoE model GLM-5.3-Flash at NVFP4 W4A4, QDA recovers 98.3% of the FP8 model's accuracy, versus 96.9% for the best vendor checkpoint.
- The authors say QDA checkpoints run on existing serving kernels with no inference overhead, and that they will release the library, evaluator and harness.
Why it matters
Quantization is how large models are served at low cost, yet the paper says the best recipe changes with each model and precision, so experts must redo the work for every new model. QDA tries to automate that loop. The idea is that an agent improves a recipe over rounds of building and evaluating candidates, and more rounds give better recipes. The practical hurdle is cost: evaluating one quantized reasoning model takes tens of GPU-hours, so QDA swaps full evaluation for a KL divergence score that takes minutes.
Who it affects
The problem is framed around people who serve large language models and have to quantize each new model by hand, including reasoning models and large MoE models. The paper's headline results cover four reasoning models and the 320B-parameter MoE model GLM-5.3-Flash.
How to use it
Nothing is usable yet. The authors say they will release the library, the evaluator and the harness. The source gives no release date or link for them. The authors state that every QDA checkpoint runs on existing serving kernels with no inference overhead, so a produced checkpoint would not need special serving code.
How solid is it
These are the authors' own reported results from a paper page, so they are self-reported. The gains are averages against the strongest public recipe: 0.9, 1.4 and 2.2 percentage points at W4A16, W4A8 and W3A16. The GLM-5.3-Flash figures are shares of the FP8 model's accuracy (98.3% for QDA, 96.9% for the best vendor checkpoint), not absolute accuracies. The source does not name the four reasoning models apart from GLM-5.3-Flash, the benchmarks or metrics behind the gains, or the vendor. It does not say which agent or LLM drives QDA. No authors or institutions are named.
Risks and caveats
The 64 GPU-hours figure is per search; the source does not say on what hardware or for which model. The source gives no absolute accuracies for the average gains. The release of the library, evaluator and harness is a promise in the future tense, so outside reproduction has to wait. The authors also note that a score based on KL divergence is paired with long-generation checks because failures such as repetition loops need to be caught separately.
“an LLM agent can take over this design by refining the recipe over rounds of building and evaluating candidates, so that more rounds lead to better recipes”
— QDA paper abstract