Google uses reinforcement learning to keep its Willow quantum chip calibrated mid-computation
Google Research, working with Google DeepMind, published "Reinforcement learning control of quantum error correction" in Nature, describing a system that lets a quantum computer recalibrate itself without halting computation. Quantum computers are analog machines: qubits are controlled by signals whose frequencies, amplitudes and phases drift over time, and until now fixing that drift meant stopping the entire computation to recalibrate. That is a problem because useful quantum algorithms are expected to need to run continuously for days or months.
The team built a reinforcement learning agent that watches the error detection events already produced by Quantum Error Correction (QEC), the technique that bundles many physical qubits into redundant "logical qubits" so errors can be caught without directly measuring (and collapsing) the quantum state. Normally those detection events are only fed to QEC decoders such as Google's neural network decoder AlphaQubit or the algorithmic decoder Tesseract, which locate and correct errors after the fact but do not explain why the errors happened. Google's RL agent instead uses the same stream of detection events as a live learning signal, continuously steering thousands of control parameters to counteract drift as it happens.
The researchers tested the approach on Google's Willow superconducting processor by deliberately injecting artificial drift into the control parameters. The RL steering improved the logical stability of the error-correcting code 3.5 times over, extending how long the processor could reliably function as quantum memory. Even after an exhaustive expert, human-in-the-loop calibration process, adding RL fine tuning on top suppressed the logical error rate by a further 20%. Combining these techniques brought logical errors down to a record low: fewer than one per thousand error correction cycles in the surface code, and one per hundred in the color code.
To check whether the method scales, the team ran numerical simulations with hundreds of qubits and tens of thousands of control parameters. The number of RL training iterations needed stayed independent of system size, because QEC detection events are locally sensitive to errors near where they occur. Google says realizing the full benefit will require speeding up the communication cycle between the agent and the processor and using more advanced machine learning methods.
Key facts
- Google Research and Google DeepMind published the work as "Reinforcement learning control of quantum error correction" in Nature
- An RL agent uses existing QEC error detection events to continuously steer thousands of control parameters during computation, without stopping it
- On the Willow processor, RL steering improved logical stability 3.5 times against injected drift and cut the logical error rate a further 20% beyond expert human calibration
- Combined techniques reached fewer than one logical error per thousand cycles in the surface code and one per hundred in the color code
- Simulations with hundreds of qubits and tens of thousands of control parameters showed the number of required RL training iterations does not grow with system size
Why it matters
Quantum computers are analog devices, and their control signals drift, which has always forced a hard stop to recalibrate before a computation can continue. That decoupling of computation and calibration is a fundamental bottleneck, since practical quantum algorithms are expected to run for days or months at a stretch. By using reinforcement learning to read the error detection events that quantum error correction already produces and steer control parameters on the fly, Google's method lets the machine tune itself while it keeps computing, similar in spirit to how AlphaFold's data driven approach broke through where hand built physical models had stalled on protein folding.
Who it affects
The result is aimed at quantum hardware and quantum error correction researchers, and at Google's own Quantum AI program built around the Willow superconducting processor. It also matters to anyone tracking the practical timeline for fault-tolerant quantum computing, since continuous self calibration is a prerequisite for running algorithms long enough to be useful.
How to use it
This is a research result published in Nature, not a product or a tool with a release date or pricing. The RL control framework was demonstrated on Google's own Willow processor and validated further through numerical simulation; it is not something outside teams can currently deploy on their own hardware.
How solid is it
The central claims come from Google's own peer reviewed paper in Nature and its accompanying blog post, describing hardware experiments on Willow with deliberately injected artificial drift, plus separate numerical simulations at hundreds of qubits and tens of thousands of control parameters. The reported numbers, the 3.5 times stability improvement, the additional 20% error suppression, and the sub-thousand- and sub-hundred-cycle error rates, are Google's measurements from its own experiments and have not yet been independently reproduced by outside teams.
Risks and caveats
The scaling evidence for large systems comes from numerical simulation, not from running the RL framework on a large physical quantum computer, so real hardware may surface effects the simulations do not capture. Google itself notes that realizing the full potential of the approach still requires a faster communication cycle between the RL agent and the processor and more advanced machine learning methods, meaning the current results are a demonstration of feasibility rather than a finished, deployment ready system.
“Our work thus enables a new paradigm: a quantum computer that learns from its errors and doesn't stop computing.”
— Volodymyr Sivak and Paul Klimov, Research Scientists, Google Quantum AI, Google Research