Hugging Face reproduces the viral watercolour-painting model, in the open

On 23 August, Surya Narreddi posted a video of watercolours painted by a language model that had gone viral, drawing over 1.5M views. The model works by writing about 150 lines of JavaScript using p5.brush, a library that simulates paint rather than drawing shapes: pigment bleeds past the edges of a fill, paper has texture, strokes have mass. Narreddi's own blog post explained the training behind an earlier, narrower stage of the project (close-up flowers rather than the full compositions in the video), but without open artifacts, and his full technical report has not appeared yet.

Sergio Paniego, in a Hugging Face blog post published September 3, 2026, reproduced Narreddi's recipe from the engineering side using TRL and OpenEnv, and published everything: the reference pool, the RL environment, training scripts, trained models and rollout data. The whole pipeline runs on Hugging Face end to end, with training on Jobs, the environment and scorer as Spaces, and the pairwise judge through Inference Providers.

The reward is aesthetic preference rather than a checkable answer, mixed from a gate check (0.05, does the sketch compile and avoid cheating), a length nudge (0.05), a pairwise judge (0.60) and HPSv3, an open 7B preference model (0.30). The pairwise judge, Qwen3-VL-30B-A3B-Instruct, compares a candidate painting against four references drawn from a hand-rated pool of 178 paintings split into 'love' and 'okay' tiers; its score is the share of comparisons the candidate wins. The pool itself was built by generating p5.brush sketches with four open-weight models (GLM-5.2 contributed 64 paintings, Kimi-K3 57, Qwen3-Coder-Next 35, Qwen3.5-122B-A10B 22) from real hibiscus photos, refining each over three rounds under a vision critic, then having Paniego rate every final render by hand.

The system prompt restricts the model to 10 of p5.brush's 47 methods (scaleBrushes, noStroke, fill, noFill, fillBleed, fillTexture, beginShape, vertex, endShape, circle), following an allowlist Narreddi had converged on through 200 iterations of the GEPA prompt optimizer; a longer API reference makes models invent methods that do not exist. Paniego's one addition was a single instruction to paint each petal two or three times, which made outputs a lot more colorful.

Paniego trained three runs with the base model Qwen/Qwen3.5-35B-A3B using LoRA: hps-only (pairwise weight 0, HPSv3 weight 0.90, stopped at step 60, a validation run), judge-led (pairwise 0.60, HPSv3 0.30, stopped at step 110) and hps-led (pairwise 0.30, HPSv3 0.60, stopped at step 110). Mean group reward rose in all three: hps-only from 0.58 to 0.71, judge-led from 0.45 to 0.72, hps-led from 0.57 to 0.82, measured over each run's first versus final third. In every run, most of the early gain came from bad paintings (scoring under 0.3) becoming rare: they fell from 99 to 16 across judge-led's thirds and from 37 to 4 in hps-led. In hps-only, the best painting of each step barely improved (+0.034) while the median moved much more (+0.155); once HPSv3 saw a recognizable flower it stopped asking for more. With the pairwise judge active, the top of the distribution moved too, and paint coverage doubled in both judge runs (0.11 to 0.23, and 0.13 to 0.30), against almost no movement in hps-only.

A hyperparameter bug cost real time before any run worked: the default LoRA target_modules list assumes a dense model, but Qwen3.5-35B-A3B is a mixture of experts, so the adapter was training only ten layers out of forty. Switching to an 'all-linear' target list, raising the learning rate from 2e-5 to 5e-5, and changing the reward-scaling and the LR scheduler unlocked the first successful run. Separately, infrastructure failures (render timeouts, an unresponsive scorer) had been silently scored as 0.0 reward, training the model on noise; this affected about 1.5% of rollouts on average and up to 5.2% in the worst run, and was fixed by excluding those rollouts instead. Paniego also found and fixed a bug in OpenEnv itself, where a closed websocket stayed cached and silently failed every subsequent call.

A training step (eight rollouts) took 15 to 18 minutes, 70 to 80% of it spent rendering headless Chromium canvases in software, with a single render taking 69 to 96 seconds against a 90 second timeout. The full trainer run took about 18 hours on one H200 GPU for 60 steps, and about 34 hours for 110 steps, on top of an always-on HPSv3 scorer Space. Paniego's own verdict, having built all three: judge-led produced the most diverse and artistically interesting paintings, hps-led converged on a distinctive wet-on-wet look, and hps-only converged hardest onto a narrow palette. Next steps on his list include multi-step training where the model sees its own painting before continuing, and testing whether a much smaller model (his 4B side experiments already produced valid sketches) can match the results at a fraction of the cost.

Key facts

• Sergio Paniego reproduced Surya Narreddi's viral watercolour-painting video (over 1.5M views) using TRL and OpenEnv, publishing the full pipeline: reference pool, RL environment, training scripts, models and rollouts.
• The model writes about 150 lines of JavaScript using 10 of p5.brush's 47 methods; the reward mixes a gate check, a length nudge, a pairwise judge (weight 0.60) scored against a hand-rated pool of 178 paintings, and the HPSv3 preference model (weight 0.30) scoring the render alone.
• Three training runs (hps-only, judge-led, hps-led) all raised mean group reward, mostly by eliminating near-blank failed paintings; the pairwise-judge runs also doubled paint coverage (0.11 to 0.23, and 0.13 to 0.30) while hps-only barely moved it.
• A LoRA misconfiguration for the mixture-of-experts base model trained only ten of forty layers until switched to an 'all-linear' target list; infrastructure failures had also been silently scored as zero reward for up to 5.2% of rollouts in one run.
• A full 110-step run cost about 34 hours on one H200 GPU plus an always-on HPSv3 scorer Space; Paniego's next steps include multi-step training where the model sees its own output, and testing whether a 4B model can match the results.

Why it matters

Most recent reinforcement learning work on language models rewards verifiable correctness: math with a known answer, code that passes tests. This project instead trains on pure aesthetic preference, with no correct answer, closer to the older RLHF approach than to today's verifiable-reward methods. It shows that a hand-curated pool of 178 images is enough to steer a model's visual taste, and it does so with every artifact published, unlike the original viral video, whose training recipe and code remain unreleased.

Who it affects

People building creative or preference-based RL pipelines get a working, fully open reference implementation to build on, including the reward code, the environment, and three trained model checkpoints. Anyone who follows Surya Narreddi's original work benefits from an independent, documented reproduction while his own technical report is still pending. The method also generalizes beyond flowers: Alex Yango applied the same recipe to paint animals, and Brendan Hogan used it to train canvas animations against a pool of hand-rated clips.

How to use it

The entire pipeline runs on Hugging Face infrastructure: training via HF Jobs, the RL environment and HPSv3 scorer as Spaces, and the pairwise judge through Inference Providers. Once the two Spaces are duplicated and running, launching a training run is one hf jobs uv run command with the base model, reward mix and hyperparameters as flags; the reference pool, environment, training scripts, and all three trained adapters with their rollout datasets are published on the Hub for direct reuse.

How solid is it

The reward curves for all three runs are backed with concrete numbers (mean group reward, fraction of failed rollouts, paint coverage) computed from published rollout datasets, and the author states the per-rollout numbers can be recomputed independently. He also documents a subtle failure mode he caught and corrected: a LoRA configuration bug that silently trained only a quarter of the model's layers, and an infrastructure bug that fed noise into the reward signal for a measurable share of rollouts. Both fixes are described with before/after evidence rather than asserted.

Risks and caveats

No human-made painting is in the training pool, a limitation the author flags directly, since accessible human work in this niche library is too scarce to build a corpus from; the model may therefore be learning to imitate other models' outputs rather than genuine human aesthetic judgment. Within a single run, paintings converge toward visual sameness, a known GRPO effect when the reward pool centers on one subject. The model also learns to ignore an explicit prompt instruction on shape count once it finds that obeying it does not raise reward, and the author cannot confirm whether his own engineering choices match Narreddi's unpublished implementation.

“The more weight the judge carries, the more the reward means my taste instead of everyone's, and the harder it should be to climb.”

— Sergio Paniego, Hugging Face blog