Glonce

OPAL: an agent rewrites the optimizer's code during training

researchThe Glonce editorsHacker News🔥65Glonce rating65Impact45Novelty70Relevance100Surprise55Players15Credibility60
OPAL: an agent rewrites the optimizer's code during training

The post introduces the reactive optimizer, which puts an LLM agent at the heart of the optimizer. The agent watches training dynamics, reacts to them, and rewrites the optimizer's source code while a single training run is under way. The authors name their version OPAL (OPtimizer with an Agent in the Loop). Their headline claim: when no recipe has been tuned for a task, it matches or beats the best recipe one could pick off the shelf, across image and language tasks. They also say the agent spontaneously produces sophisticated schedules and strategies against loss spikes and overfitting, and they point readers to an OPAL package to try on their own runs.

The motivation is that the recipe for training a neural network is normally fixed before training starts: adaptive step sizes, preconditioning, orthogonalized updates, step-tied schedules, learning rates scaled with model size. Even automatically discovered optimizers then run unchanged to the end. The authors ask how an optimizer should respond to a loss spike or stall, or to one module's gradients oscillating while others improve, and whether a new architecture that trains poorly with Adam or Muon is worse or simply has not met a suited optimizer. Their scenario is a PhD student with one GPU for one day and a modest LLM token allowance, who can afford a single serious run. A reactive optimizer spends LLM tokens in place of GPU and human hours. The approach is entirely zero-shot: nothing is learned or retained from repeated runs, and the optimizer's whole algorithm is open to rewriting.

The mechanism is two files. The agent rewrites updater.py, which is used in every optimization step, so code is the whole interface. It also writes analysis.py, a set of probes that run throughout training and extract whatever the agent wants to know. Raw weights, activations and gradients would swamp the agent's context, so the agent picks its own statistics instead. The post lists examples such as per-group gradient norms, the size of an update relative to the weights, the cosine between consecutive gradients, and which tensors show unusually large gradient spikes. Training reaches the agent through analysis.py, and the agent's decisions reach every step through updater.py.

Results. The language tests start from the optimization track (Track 3) of modded-nanogpt, which trains a 162M GPT on FineWeb with recipes tuned for that one setting. The authors built two variants: the same model on an alternate language dataset (Chinese, Thai and Kyrgyz), and the original setup with a quarter of the batch size. Baselines were five leaderboard recipes adopted with natural adjustments. In all the new settings the reactive optimizer outperforms or matches the best baseline.

On Goldfish, a collection of 350 languages with one small GPT per dataset trained on a standard recipe, the Amharic official recipe ends at 83.48 bits per sentence on held-out FLORES-200 text (lower is better). The reactive optimizer initialized with that recipe ends at 81.11 in one shot; Swahili shows a similar gain. Sweeping the standard recipe's learning rate gives a static recipe at 80.89, ahead of the reactive optimizer, but the reactive optimizer initialized from that swept recipe ends ahead again, at 80.73.

On image generation, the reactive optimizer ends ahead of the official AdamW recipe on LightningDiT-XL/1: FID-50K 1.83 against 2.18 with guidance, and 4.33 against 5.15 without. It also ends ahead on RAEv2-B, 3.19 against 3.39, where the official recipe uses a Muon variant. All these runs used Claude Opus 5.5 to power the agent.

The visible part of the post then shows how agents set learning rates. The schedules are intricate, with separate multi-stage schedules per parameter group that differ from run to run, and several raise a learning rate mid-run. Four examples are given. In 'Deduced from clues', the SiT agent measured the cosine between consecutive gradients; where it was negative it lowered the learning rate, taking the timestep embedder to 0.3x and then 0.15x of the official rate, and where it stayed slightly positive (the Muon matrices) it raised the rate. In an 'A-B-A test', the RAEv2 agent alternated two learning rates over stretches of a few hundred updates, compared the training loss, and switched to 2x from update 3,100. In 'Trial and error', the same agent set every group to 1.3x, returned the backbone to 1x when its gradients began changing sign each step, raised every group to 1.5x at the next decision, then tried 2x, 2.5x and 2.75x for about 650 updates each and kept 2x; two deep dips in its backbone attention plot are 75-update test reductions. In 'Re-planned cooldown', the SiT agent at step 190K moved its planned cooldown from 320K to 280K and lowered its learning-rate floor from 0.1 to 0.05 of peak, while the RAEv2 agent began cooldown at update 11,250, which is 45% of the run instead of the planned 75%, citing its loss trend and a paper on decaying the learning rate.

The text available here is cut off partway through this learning-rate section, so the parts on loss-spike suppression, escaping overfitting, and cost and practicality at scale are not covered.

Key facts

  • OPAL has an LLM agent rewrite two files during one training run: updater.py (the per-step update rule) and analysis.py (the agent's own diagnostic probes).
  • On Goldfish Amharic, the official recipe scores 83.48 bits per sentence and OPAL starting from it reaches 81.11; a learning-rate-swept static recipe gets 80.89, and OPAL initialized from that gets 80.73.
  • On LightningDiT-XL/1, FID-50K is 1.83 against the official AdamW recipe's 2.18 with guidance, and 4.33 against 5.15 without; on RAEv2-B the result is 3.19 against 3.39.
  • On modded-nanogpt variants (Chinese, Thai, Kyrgyz, and a quarter of the batch size), the authors say OPAL outperforms or matches the best of five adapted leaderboard recipes.
  • The agents built multi-stage schedules per parameter group, for example the RAEv2 agent started cooldown at update 11,250, 45% of the run instead of the planned 75%.

Why it matters

Training recipes are normally fixed in advance and run unchanged, so a new architecture, language or dataset often has to live with a recipe tuned for something else. The authors propose spending LLM tokens instead of GPU and human hours: the agent adapts the optimizer to the one run it is in, including reacting to accidents such as loss spikes or stalls. They expect the method to grow stronger with each generation of agents and to be most useful where no recipe exists yet. That is an expectation, not something the post demonstrates.

Who it affects

Researchers and small teams who can afford only one serious training run on a new architecture, modality, language or scientific dataset, where a large hyperparameter sweep is out of reach. The post frames this with a PhD student who has one GPU for one day. Teams that already have a well-tuned recipe for their exact setting are not the target case.

How to use it

The post invites readers to try it on their own training run with the OPAL package. The setup it describes: let the agent rewrite updater.py, which runs in every optimization step, and analysis.py, its own probes, then initialize from a standard recipe if one exists. On Goldfish, starting from the official recipe or from a learning-rate-swept recipe both gave gains. The visible text gives no cost figure in tokens, dollars or GPU hours.

How solid is it

These are the authors' own reported results in a blog post, with five baseline recipes for the language tasks and official recipes for the image ones. The gains over the official recipes are clear in the reported numbers: 83.48 to 81.11 bits per sentence on Amharic and FID-50K 2.18 to 1.83 on LightningDiT-XL/1 with guidance. A static learning-rate sweep on Goldfish Amharic beat the one-shot optimizer (80.89 against 81.11), and OPAL's edge over that tuned recipe is small (80.73). The number of runs or seeds, and any variance or significance, is not given. Results are with Claude Opus 5.5 only. No peer review or independent replication is mentioned.

Risks and caveats

The visible text is truncated in section 3.1, so the loss-spike, overfitting and cost sections cannot be checked. The claim that the agent prevents loss spikes and escapes overfitting appears only in the summary here. The agent's stated reasons for its decisions are referenced but their text is not present. Because the method is zero-shot and relies on one specific agent model, results with other agents are unknown. The metric for the RAEv2-B comparison (3.19 against 3.39) is not named in that sentence.

“A reactive optimizer offers another way to use that budget: it spends LLM tokens in place of GPU/human hours.”

— Reactive Optimizers blog post