Glonce

ldraw-nova: open-source tool has AI agents build LEGO models

toolsThe Glonce editorsHacker News🔥65Glonce rating65Impact45Novelty70Relevance90Surprise50Players20Credibility85
ldraw-nova: open-source tool has AI agents build LEGO models

ldraw-nova is an open-source project, presented on Show HN, in which an AI agent takes a model idea and builds a LEGO model in LDraw, a simple, low-level, executable language for 3D CAD models of LEGO parts. The author, who publishes under the handle anteloc and gives no personal name, says the goal is to get agentic LLMs capable of designing buildable, physical things.

When a build finishes, the user gets the model's LDraw source, several views (a 3D viewer, a 3D player, an interactive VR view for Meta Quest 3, and images), a Blender-editable glTF file in .glb format with metadata stored as Blender Custom Properties, and the chat history together with the agent's thinking process.

The author's route to the design is told in the README. LDraw is heavily focused on math (rotating and positioning parts), which LLMs are usually bad at. Even so, agents did pretty well in initial tests, and later projects (ldbuilder-ai as initial research, then py2bricks and py4bricks as the first and second attempts at agentic Python tooling) also gave good results, but never enough to consider the generated models correct. From these attempts the author draws two conclusions. First, agents do much better at writing Python code that produces the math than at producing the math themselves, so tooling that generates LDraw sources sidesteps the geometry. Second, agents tend to learn better from Python code that produces models than from the models themselves. The remaining job was a Python toolkit with the right primitives, verbs and constructive vocabulary, which the author says was really hard to get right, even by vibe coding, until GPT-6 Astra and Claude Opus 5.5 arrived and vibe-coded it right.

The working process is: the agent takes a prompt, reads instructions.md and related documents on LDraw and LEGO building, and plans the model (required parts, submodels, aesthetics). It then loops: render images of the model or submodels, inspect them, adjust positioning and aesthetics, and render again until it considers the model finished. The tooling helps it find suitable parts and example models and submodels to start from, detect collisions and gaps when placing parts, and render headlessly. The agent does not start by placing parts, apart from prototyping and learning by altering existing example models. Instead it builds one or more plans that fully describe the model and submodels, including geometry (the example given is atlas-crane.plan.json), then writes one or more generator scripts (for example generate.py) whose execution produces LDraw files (for example atlas-crane.mpd). The author sums it up as an agent creating a generator that produces a 3D model in an assembly language, "a compiler of sorts".

The agent's part search relies on jev-rerank, a semantic search tool with re-ranking that the author also wrote, backed by TypeSafe's Jev System One AI model. With a TypeSafe API key (TYPESAFE_API_KEY) set in the web app's Settings section, reranking works. Without one, agents fall back to full-text search, which could (maybe) give worse models.

To run it you need Git and Docker. The web app runs dockerized from two sibling repos, ldraw-nova and ldraw-nova-docker, cloned side by side at the same tag (v0.6.0). You run docker compose build in the ldraw-nova-docker folder (the first build takes a while and needs about 5 GB of disk space), then docker compose up -d. The app opens at https://localhost:8443, which is needed for VR on Meta Quest 3 and uses a self-signed certificate, or at http://localhost:8765, plain HTTP without certificate warnings but without VR. Other devices on the network can reach it by the computer's IP. The app has no login, so the author advises running it only on trusted networks.

The author calls this a first release and lists the work still to do: VR on Meta Quest 3 has model-handling and performance issues; the tooling, docs and instructions need adapting for low-end models such as Luna and Haiku; currently only expensive, high-end models can generate large, correct models; the generative process is slow; more model families are wanted, including minifigs (humans and animals), Technic machines and engines, and spaceships, whose generated models are not very good; building models from manuals partially works, better when manual pages are given as images; and fine-grained inspection of submodels and their step-by-step building is planned. The README thanks the LDraw community and the authors of LDView, LeoCAD, LDCad and Shadow Library, ldraw.rs and pyldraw3, and carries the disclaimer that LEGO is a trademark of the LEGO Group, which does not sponsor, authorize or endorse the software.

Key facts

  • An AI agent turns a model idea into a LEGO model in LDraw by writing a plan (plan.json) and a Python generator script that outputs the LDraw file.
  • Outputs include LDraw source, 3D viewer and player, a Meta Quest 3 VR view, images, a Blender-editable .glb file, and the chat history with the agent's thinking process.
  • The author's conclusion: agents do better writing Python that produces the geometry math than producing LDraw math directly, and learn better from generator code than from finished models.
  • It runs as a Dockerized web app from two repos cloned at tag v0.6.0; the first build needs about 5 GB of disk space, and the app has no login.
  • The author calls it a first release: generation is slow, only expensive high-end models build large correct models, and VR and spaceship models are weak.

Why it matters

The project tests a specific idea: LLM agents can design physical, buildable things if they are given a language and tooling that keep them away from raw geometry. The author's two conclusions are that agents write Python that produces math better than they write the math, and that they learn better from generator code than from finished models. The result is a pipeline in which the agent plans, writes a generator script, and lets it emit LDraw, which the author likens to a compiler. The author says the Python toolkit only came together after GPT-6 Astra and Claude Opus 5.5 arrived.

Who it affects

LEGO and LDraw hobbyists who want to generate models from a prompt, and developers curious about agents that design physical objects. People with a Meta Quest 3 get a VR view, and Blender users get an editable .glb file. The author names the LDraw community and the makers of LDView, LeoCAD, LDCad, ldraw.rs and pyldraw3 as the foundation the work builds on.

How to use it

You need Git and Docker. Clone ldraw-nova and ldraw-nova-docker side by side at the same tag, v0.6.0, run docker compose build in ldraw-nova-docker (the first build needs about 5 GB of disk space), then docker compose up -d. Open https://localhost:8443 for VR on Meta Quest 3 (self-signed certificate, so accept the browser warning) or http://localhost:8765 for plain HTTP without VR. Run docker compose down to stop it. To use the semantic part search, set a TypeSafe API key (TYPESAFE_API_KEY) in the web app's Settings; otherwise agents use full-text search.

How solid is it

The source is the project's own README, so every claim about results comes from the author. The author says initial tests went pretty well but never well enough to call the generated models correct, and that only expensive, high-end models can currently produce large, correct models. No benchmark, success rate, generation time or cost figure is given. No user numbers, stars or independent reviews are given either. The author describes it as a first release with work still required.

Risks and caveats

The author lists known limitations: VR on Meta Quest 3 has model-handling and performance issues, generation is slow and expensive for large models, spaceship models are not very good, and building from manuals only partially works. Without a TypeSafe API key the search falls back to full-text search, which could (maybe) give worse models. The web app has no login, so it should only be run on trusted networks. LEGO is a trademark of the LEGO Group, which does not sponsor, authorize or endorse the software.

“I did this in order to get agentic LLMs capable of designing buildable, physical things!”

— anteloc, author of ldraw-nova (project README)