Glonce

GPT-5.5 and other LLMs over-edit code fixes, study finds

researchThe Glonce editorsHugging Face🔥69Glonce rating69Impact60Novelty70Relevance95Surprise45Players35Credibility85
GPT-5.5 and other LLMs over-edit code fixes, study finds

Researchers built an evaluation framework to study "over-editing": the tendency of a large language model to rewrite code beyond what is needed to fix a bug. They took 400 problems from the BigCodeBench benchmark and injected controlled corruptions at the AST (abstract syntax tree) level into each reference solution, so every repair task came with a known, minimal correct patch to measure against. Testing frontier LLMs on these tasks, they found over-editing is widespread even in strong models like GPT-5.5: a model can pass the correctness check (high Pass@1) while still making unnecessarily large edits that add cognitive complexity to the code, well beyond what fixing the injected bug required.

Adding a preservation instruction, telling the model to change as little as possible, substantially reduced the behavior. It lowered the average excess Levenshtein distance (a measure of how much of the edit goes beyond the minimal patch) from 0.195 to 0.131, cut the added cognitive complexity by 26.6%, and even raised Pass@1 by 2.3 points. But the researchers found these gains do not simply follow from giving the model a larger reasoning budget or from using a larger model; the instruction itself, not scale, drove the improvement.

The team then tested whether minimal editing could be learned directly during post-training rather than prompted for at inference time. Supervised fine-tuning overfit to the specific corruption patterns it was trained on and did not generalize well. Reinforcement learning produced the best trade-off, giving the strongest edit fidelity on out-of-domain tasks while retaining performance. The authors conclude that edit fidelity, how minimal and faithful a repair is to the original code, is a distinct, measurable axis of code-repair quality separate from correctness, and one that can be explicitly targeted during training.

Key facts

  • The evaluation framework injects controlled AST-level corruptions into 400 BigCodeBench reference solutions, giving each repair task a known minimal patch to measure edits against.
  • Over-editing is widespread even in strong models like GPT-5.5: high Pass@1 correctness can coexist with unnecessarily large edits and added cognitive complexity.
  • A preservation instruction lowers average excess Levenshtein distance from 0.195 to 0.131, cuts added cognitive complexity by 26.6%, and raises Pass@1 by 2.3 points.
  • These gains do not simply follow from a larger reasoning budget or a larger model.
  • Reinforcement learning post-training gives the best out-of-domain edit-fidelity and performance-retention trade-off, while supervised fine-tuning overfits to the corruption patterns it was trained on.

Why it matters

Code-repair models are usually graded only on whether their fix passes tests. This study shows that passing tests and making a minimal, reviewable, faithful edit are separate qualities: a model can score well on correctness while rewriting far more of the file than the bug required, which raises review burden and the risk of introducing new problems in code that did not need to change.

Who it affects

Teams building or relying on LLM-based code-repair and coding-agent tools, and researchers designing code-repair benchmarks that currently score correctness alone without accounting for edit size or complexity.

How to use it

The paper gives a concrete recipe for measuring over-editing: inject controlled corruptions into known-good code so each task has a verifiable minimal patch, then compare a model's edit against it by excess Levenshtein distance and added cognitive complexity. It also identifies two levers that reduce over-editing: a preservation instruction at inference time, and reinforcement learning during post-training, which the authors find works better than either instructing alone or supervised fine-tuning.

How solid is it

The framework is grounded in a controlled setup, 400 BigCodeBench problems with injected AST-level corruptions and known minimal patches, so over-editing is measured against a verifiable optimum rather than judged subjectively. The authors report ablations across preservation instructions, reasoning budget, model size, and training method (SFT versus RL), which is a meaningful methodological signal. The abstract text available here names only GPT-5.5 as an example of the "frontier LLMs" tested and does not give a full list, nor does it name the paper's authors, their institutions, or how the preservation instruction is phrased.

Risks and caveats

The findings come from a synthetic benchmark built by injecting corruptions into reference solutions, which may not capture the full diversity of real-world bug fixes, so how the results generalize to actual production code changes is not addressed in the available text. No publication or submission date, author list, institution, full model roster, or preservation-instruction wording is given in the source material captured here.