Glonce

LeanPRM uses Lean to auto-label proof steps for training verifiers

researchThe Glonce editorsHugging Face🔥58Glonce rating58Impact45Novelty60Relevance95Surprise30Players10Credibility70
LeanPRM uses Lean to auto-label proof steps for training verifiers

AI systems now write long, difficult mathematical proofs, and checking them step by step is becoming a bottleneck. A learned verifier could do the checking cheaply, but training one needs a correctness label on every step, and those labels are hard to get for open-ended proofs. The paper says existing methods either take them from human annotators, which is costly, or derive them from final answers, which proofs do not have. Formalizing each proof in Lean requires expert effort.

The authors introduce LeanPRM, a framework that uses the Lean proof assistant to create step-level labels for natural-language proofs. A model writes a natural-language proof together with a paired formal proof, without feedback from Lean. Lean then labels every step. A second model discards failures caused only by the formal syntax, so that the negative labels mark errors in the mathematics rather than in the formalization.

The resulting dataset, also called LeanPRM, contains 10,634 proofs with 152K labeled steps and no human annotation.

The headline result uses a Qwen3-4B verifier. Trained on LeanPRM, it raises exact step-label accuracy on a held-out set of 200 proofs, called LeanPRM-Bench, from 48.2 to 74.7. The same base model trained on PRM800K reaches 60.3. The source does not state the unit of these figures.

The verifier trained on LeanPRM also selects better proofs in best-of-N selection on IMO-level problems, and it outperforms the PRM800K-trained verifier on three of four external verification benchmarks. Training on LeanPRM improves verifiers at every size from 0.6B to 8B parameters on LeanPRM-Bench, Hard2Verify and IMO-GradingBench.

The authors' argument for the approach: because the labels need no human annotation, LeanPRM can grow with the supply of formal statements. That offers a path to step-level feedback on AI-generated proofs that are hard for humans to check.

Key facts

  • LeanPRM is a framework that uses the Lean proof assistant to label every step of a natural-language proof; a second model removes failures caused only by formal syntax.
  • The dataset has 10,634 proofs and 152K labeled steps, with no human annotation.
  • A Qwen3-4B verifier trained on LeanPRM goes from 48.2 to 74.7 exact step-label accuracy on the 200-proof LeanPRM-Bench; the same model trained on PRM800K reaches 60.3.
  • The LeanPRM-trained verifier beats the PRM800K-trained one on three of four external verification benchmarks and improves verifiers at every size from 0.6B to 8B parameters.
  • It also picks better proofs in best-of-N selection on IMO-level problems, though no numbers are given for that result.

Why it matters

Checking long AI-written proofs is becoming a bottleneck, and a learned verifier needs step-level labels that are expensive from humans and unavailable from final answers. LeanPRM's idea is to let Lean supply those labels automatically, with a model writing the informal proof and a paired formal one. If the labels scale as the authors claim, step-level feedback on proofs that are hard for humans to check becomes more practical.

Who it affects

Researchers building proof verifiers and process reward models for mathematical reasoning, and groups working on best-of-N selection for hard problems such as IMO-level ones. The dataset is relevant to anyone who has relied on PRM800K as the training baseline.

How to use it

The source gives no code, dataset release or license details. What it offers is the recipe: have a model write a natural-language proof with a paired formal proof, let Lean label each step, filter out syntax-only failures with a second model, then train a verifier on the result. The reported experiments use Qwen3-4B, with gains also shown at sizes from 0.6B to 8B parameters.

How solid is it

This is a preprint abstract with concrete numbers. The largest gap, 74.7 against 60.3 for PRM800K, is on LeanPRM-Bench, a 200-proof held-out set named after the framework. On external benchmarks the result is narrower: the LeanPRM-trained verifier wins on three of four. The best-of-N result on IMO-level problems comes without figures. The source names no authors or institutions.

Risks and caveats

The source does not say which external benchmark the LeanPRM-trained verifier fails to beat PRM800K on. The unit of the 48.2, 74.7 and 60.3 figures is not stated. The names of the models that write the proofs and filter syntax-only failures are not given. The claim that LeanPRM can grow with the supply of formal statements is the authors' argument, not a demonstrated scaling result.