A robotics coder gives up on letting his LLM debug visually
A programmer who writes robot code describes trying to hand off one of the most tedious parts of the job to his LLM coding assistant: visual debugging. Robot code, he explains, is normally debugged by watching a simulation or viewer render the robot's behavior and eyeballing the result, then tweaking parameters and rerunning until it looks right. He already likes offloading drudge work to an LLM assistant, praising its limitless patience for the more repetitive parts of coding, so he tried extending that to debugging itself. His reasoning: the models already ship with an image encoder, and his visualizer tool exposes an MCP server, so in principle the assistant should be able to look at the scene and adjust the robot's behavior on its own. In practice, he found the models had no real feel for what a working robot looks like, no matter how much text they had been trained on. The tooling made things worse: he can navigate to a specific spot in his debug viewer and inspect it in about five seconds, but an MCP-connected assistant took roughly five minutes just to reach the right view. A full round of the assistant's debugging loop took him about thirty minutes, after which he would typically get back a new answer that was still wrong. He also frames the frustration in personal terms, comparing it to arguing with a bad-faith commenter online: after about six replies you conclude you are talking to someone incapable of thought, and he says that exchange is actually more pleasant than trying to get an LLM to do what he wants. He ended up falling back to a manual workflow: running the debug viewer himself, spotting anomalies by eye, then taking a screenshot and describing the mistake to the language model. Eventually he stopped involving the model in the debugging step at all and did the visual debugging entirely on his own, reasoning that if he was going to do the tedious inspection work anyway, he might as well keep the satisfying part of fixing the bug for himself too.
Key facts
- A robotics programmer tried using his LLM coding assistant, connected to his visualizer via an MCP server, to run the visual robot-debugging loop on its own.
- He can jump to a specific spot in his debug viewer and inspect it in about five seconds; the MCP-connected assistant took roughly five minutes to reach the right view.
- A full cycle of the assistant's debugging loop took about thirty minutes and typically still returned a wrong answer.
- He concludes that no amount of text-only training gives a language model a good sense of what a normal, working robot looks like, and that the tools for letting coding assistants control GUI software are limited.
- He abandoned the automated approach, first screenshotting the viewer for the model himself and eventually just doing the visual debugging entirely on his own.
Why it matters
MCP servers are pitched as the bridge that lets coding assistants operate real tools, including graphical ones, instead of just writing text. This account is a concrete data point on that promise for a task that depends on spatial and visual judgment: connecting a visualizer's MCP server to an image-capable assistant did not translate into competent debugging, exposing a gap between text-trained reasoning and reading a 3D scene the way a person does.
Who it affects
Robotics engineers and other developers who write code debugged through simulation or visualization, and anyone evaluating agentic coding assistants and MCP tool servers for tasks that require spatial or visual judgment rather than pure code generation.
How to use it
There is no product or release here, only a workflow lesson from one practitioner: LLM assistants can still be useful for the repetitive parts of writing robot code, but for visual debugging through an MCP-connected viewer, expect the tool-use loop to run in minutes rather than the seconds a human takes, and the final answer to often still be wrong. Manual debugging, with the model at most seeing a screenshot, was faster in this case.
How solid is it
This is a single first-person account with no benchmark numbers, and no LLM model, coding assistant, visualizer tool, robot platform, or company is named, so the specific tools involved cannot be checked. It reads as a genuine practitioner anecdote rather than satire, but it is one person's experience, not a systematic evaluation.
Risks and caveats
The essay does not say how long the MCP-based debugging experiment ran before it was abandoned, does not name the model, assistant, or visualizer involved, and does not state the author's job title or employer. The conclusions describe one workflow on one setup and may not generalize to other tools, other robots, or newer model versions.
“It seems that no amount of training on text can give a language model a good sense of what a normal, working robot does.”
— the author, in the blog post