AutoDesign beats Claude Design on new PosterBench benchmark
A new paper argues that turning multimodal source material into a finished, structured output, such as a poster, can be treated as a long-horizon agentic process built on a model-harness system. The authors say an ideal harness should align with human design priors and accumulate reusable experience through empirical exploration to drive recursive self-improvement, and that existing approaches stay static and fall short of that. Their proposed framework, AutoDesign, tackles this with a meta-harness optimizer that guides a code agent to recursively improve the harness itself, using feedback from the agent's own rollouts.
To test the framework, the authors focus on academic paper-to-poster generation and introduce a new benchmark, PosterBench: a 100-paper Main Track spanning five disciplines, plus PosterBench-mini, a shared 10-paper subset for controlled evaluation. On the PosterBench Main Track, AutoDesign scores 78.32, the highest of the systems tested, beating the closed-source commercial system Claude Design by 7.45 points. Across seven controlled code-agent-model configurations, adding the learned component the authors call DesignHarness consistently raised performance, lifting the average PosterBench score from 54.99 to 67.39, a 12.4-point gain.
Run as a fully autonomous long-horizon loop, AutoDesign executes 253 tool calls and 11 editing turns in 40 minutes for under $3, and reaches average conference-poster quality by human evaluation. A separate system-blind human study, where evaluators did not know which system produced which poster, found AutoDesign the most preferred among the systems compared.
Key facts
- AutoDesign scores 78.32 on the PosterBench Main Track, beating the closed-source commercial system Claude Design by 7.45 points.
- Adding the learned DesignHarness lifts the average PosterBench score across seven code-agent-model configurations from 54.99 to 67.39, a 12.4-point gain.
- PosterBench consists of a 100-paper Main Track spanning five disciplines plus a 10-paper PosterBench-mini subset for controlled evaluation.
- In a fully autonomous run, AutoDesign completes 253 tool calls and 11 editing turns in 40 minutes for under $3, reaching average conference-poster quality.
- A system-blind human study rated AutoDesign the most preferred system among those compared.
Why it matters
Most agent harnesses, the scaffolding of prompts, tools and workflow wrapped around a base model, are built by hand once and then left static. AutoDesign's meta-harness optimizer instead lets a code agent rewrite its own harness using feedback from actual rollouts, which the authors present as a step toward harnesses that keep improving on a task rather than staying fixed after deployment. They frame this as closing a gap between how an ideal harness should behave, aligned with human design priors and able to accumulate reusable experience, and how existing static paradigms actually behave.
Who it affects
The work targets researchers and engineers building agentic pipelines that turn source material, papers or other multimodal input, into a finished structured artifact. PosterBench, with its 100-paper Main Track across five disciplines, gives that community a shared, reproducible way to compare systems on the concrete task of paper-to-poster generation, rather than relying on one-off demos.
How to use it
This is research output rather than a released product. What the paper documents is the operating cost of a full autonomous run: AutoDesign executes 253 tool calls and 11 editing turns, finishes in 40 minutes, and costs under $3, reaching average conference-poster quality in human evaluation on that run. The abstract does not describe what the learned DesignHarness consists of beyond being the meta-harness optimizer's output, so applying the approach elsewhere would mean reproducing the method rather than reusing a published artifact.
How solid is it
The headline numbers, a 78.32 Main Track score, a rise from 54.99 to 67.39 across seven configurations, and a system-blind human preference study, all come from the authors' own paper and their own PosterBench benchmark. The abstract does not name the seven code-agent-model configurations tested, and it does not describe how the comparison system Claude Design works or who built it, so that specific comparison cannot be checked from the text alone. No publication or evaluation date is given either.
Risks and caveats
The abstract carries no author names or institutional affiliations, so the retelling above refers to the authors generically. The cost and timing figures, under $3 and 40 minutes for 253 tool calls and 11 editing turns, are stated only for the separate fully autonomous long-horizon loop, not for the PosterBench Main Track runs that produced the 78.32 score; the two should not be read as the same measurement.
“an ideal harness system should align with human design priors and accumulate reusable experience through empirical exploration to drive recursive self-improvement”
— AutoDesign paper abstract