Claude Opus 5 leads EEBench, a new AI circuit-design benchmark
The team behind atopile built EEBench after OpenAI put a demo of GPT-6 Astra working on a circuit board in KiCad on the front page of its launch post. The demo raised a question the team had already been chewing on: how do you measure whether the electronics an AI model produces are actually any good, rather than just plausible-looking?
EEBench has agents work in atopile, a declarative-code representation of a circuit, rather than clicking around a graphical CAD tool. That lets an agent change a design, build it, run a simulation and inspect what failed without leaving the project, and it means the benchmark spends its time testing electronics rather than testing computer use.
One public task is based on a residential energy meter: when its 5 V supply disappears, the circuit has to keep the processor alive for another 20 ms so it can save the accumulated reading, and the protected rail must stay above the processor's 3.0 V brownout threshold the whole time. Most models reach for a capacitor immediately, but a real ceramic capacitor can supply much less than its rated capacitance once voltage is across it, parts carry tolerances, and oversized capacitors cost more, take up space and slow the rail's recovery once power returns. A harder analog task asks an agent to synthesize a multiple-feedback low-pass filter around an op-amp, solve the resistor and capacitor ratios for the required poles, and keep gain, cutoff frequency and Q inside spec after every component is pushed to a worst-case tolerance corner; the harness rebuilds the SPICE deck for those corners and runs AC and transient captures against named probes.
Grading is fully deterministic. EEBench builds the submitted design, constructs the circuit graph and bill of materials, and runs SPICE simulations and design checks against real manufacturer parts, with specifications pulled from datasheets. Each requirement produces a measurement against a limit, and a technical score is combined with cost efficiency measured against a reference bill of materials, with cost only counted once the circuit works. EEBench V1 covers analog and digital design through simulation only; it does not yet test whether a model can lay out, manufacture or bring up a complete product, which the team wants to add later.
On the September 1 leaderboard (EEBench's own runs), Claude Opus 5 scored 61.6% across the 13 tasks in EEBench V1, ahead of Grok 4.6 at 57.1% and Claude Fable 5.1 at 56.4%. OpenAI's models trailed: GPT-5.5 scored 42.3% and GPT-5.6 Sol scored 39.4%, and the team has not yet tested GPT-6 Astra itself. Separately, xAI's own published run of Grok 4.6, run with xhigh reasoning effort and included in the Grok 4.6 model card's "engineering acceleration" section alongside 3D-modeling and parametric-CAD evaluations, put the model at 60.0%, a different figure from EEBench's own 57.1% run of the same model. In its Grok 4.6 launch post, xAI said the model was trained on high-quality engineering data with reinforcement learning in domain-specific environments including computer-aided design.
The same pass/fail checks EEBench uses for grading double as reward signals for reinforcement learning, since a failed run shows which voltage missed its limit, which corner failed, or whether the model just solved the problem with an unnecessarily expensive design. The atopile team says it is also working directly with frontier labs on larger, non-public evaluation suites and training environments. Elon Musk has said Grok 4.7 is coming within weeks after further training on a large collection of SpaceX data aimed at making it especially strong at engineering, though the model is not out yet and the team notes the schedule may move.
Key facts
- EEBench grades AI-designed circuits with SPICE simulation on real manufacturer parts rather than by inspecting schematics, using atopile's declarative circuit code instead of a graphical CAD tool.
- On the September 1 leaderboard, Claude Opus 5 scored 61.6% across 13 tasks in EEBench V1, ahead of Grok 4.6 (57.1%) and Claude Fable 5.1 (56.4%); GPT-5.5 scored 42.3% and GPT-5.6 Sol scored 39.4%.
- xAI's own published run of Grok 4.6, with xhigh reasoning effort, scored 60.0% and appears in the Grok 4.6 model card's "engineering acceleration" section, a different figure from EEBench's own 57.1% run.
- The energy-meter task requires keeping a processor alive for 20 ms after its 5 V supply disappears while a protected rail stays above the processor's 3.0 V brownout threshold.
- EEBench V1 tests only analog and digital simulation, not layout, manufacturing or bring-up, and the atopile team is separately building larger, non-public evaluation and training environments for frontier labs.
Why it matters
OpenAI's decision to show GPT-6 Astra working on a circuit board in KiCad as part of its launch post pushed the atopile team to publish a way of checking whether an AI model's circuit actually works, rather than just looks plausible. EEBench's authors argue that most of what separates a working circuit from a broken one is not the equations but the trade-offs: parts have tolerances, cheaper components cost accuracy, and a design that works with nominal values can fail with the parts that actually arrive. They read both OpenAI's KiCad demo and xAI's decision to publish EEBench in the Grok 4.6 model card as early signs that AI labs are starting to take electronics engineering seriously as its own discipline, not a side effect of coding ability.
Who it affects
The comparison directly concerns Anthropic, OpenAI and xAI, whose models were benchmarked and ranked. It also concerns hardware and electronics engineers deciding whether an AI agent can be trusted with part of a design, and any lab building post-training environments for engineering tasks, since the atopile team says it is already working directly with frontier labs on larger, non-public evaluation suites and simulation-backed training environments.
How to use it
EEBench's methodology and a sample result explorer are public, and anyone can try the same approach by pointing an agent at atopile with a board they want built. The benchmark itself is not a paid product: the atopile team says it funds and pays for the public benchmark runs and does not sell benchmark scores. Labs that want larger or non-public evaluation suites and training environments can contact the team directly, though no pricing or terms for that work are given.
How solid is it
Grading is deterministic: EEBench builds the submitted design, constructs the circuit graph and bill of materials, and checks it against SPICE simulations using specifications pulled from real manufacturer datasheets, covering measurements such as voltage during an outage, gain, thresholds, ripple and transient response at component-tolerance corners. That said, EEBench V1 tests only the requirements, design and simulation loop, not layout, manufacturing or bring-up of a physical product. The two published figures for Grok 4.6, 57.1% from EEBench's own run and 60.0% from xAI's own run at xhigh reasoning effort, do not match, which shows how much a model's score can depend on who runs the evaluation and at what reasoning effort. The source does not give a year for the "September 1" results.
Risks and caveats
The authors are explicit about the limits of what they have shown: they are still far from asking an AI to design an entire product, such as a phone, in a single prompt, and they say they still would not let a model design a pacemaker and blindly install the result. EEBench is built and funded by the atopile team itself, so the leaderboard is not run by an independent third party, even though the team says it does not sell benchmark scores. No GPT-6 Astra score exists yet despite it being the model that prompted the benchmark's creation, and Grok 4.7, which Elon Musk has said is coming within weeks after additional training on SpaceX data, is not released and its schedule may move.
“We still would not ask it to design a pacemaker and blindly install the result.”
— EEBench/atopile team, on the current state of AI circuit design