Software engineering fundamentals matter more with AI coding agents
In a first-person blog post, the author frames the piece around a specific version of imposter syndrome: working out what it actually means to be a software engineer at a moment when online discussion of agentic engineering is mostly noise rather than signal. The author pushes back on two extremes at once, the marketing fervor around major model providers and the narrative, seen elsewhere online, that the profession is ending. Watching friends work with agent harnesses and models, the author says the people doing the most impressive work are not the ones publicizing it or predicting the death of the field; they found what the author calls a 'big damn stick' and are quietly exploring where it gives them leverage.
The author writes that in the past year, agent harnesses crossed the 'can it be done' threshold for software work. That comes with reservations: discomfort with how large language models' training data was gathered without consent, and with what the author calls 'the lunatics' engaged in economic self-dealing that is, in the author's own words, 'peanut buttering over the otherwise tanking US economy.' On the technology itself, the author argues that the economics of large models are not viable by any report seen so far, but that the underlying capability keeps shrinking rather than disappearing: open-weight models now let capable personal computers do similar work, with only a small remaining gap in effectiveness and speed.
The essay's central argument is that 'can it be done' is only the start of software engineering, not the bulk of the job. The author compares this to learning to weld in their 20s: quickly able to join pieces of metal that could not then be lifted or carried out of the shop. Applied to coding agents, the author writes, the lesson is the same: how something goes together is what makes the difference. Using agent harnesses with some foresight, including prompting them to work in red/green test-driven development, gets code that works and is testable, but, in the author's words, 'not very solid much above that.' The harder part is what the author calls the 'seams': how a piece of code's interface behaves and how it fits with the rest of a system, part art and part science, resting on the engineer's own judgment and experience, both for the problem at hand and for living with that software over a long period. Making software debuggable, maintainable, layered and composable still takes extensive, thoughtful reasoning, and that, the author writes, is exactly where even today's leading frontier models fall short.
The reason given is that large language models do not reason: they predict, since the models are, in the author's description, effectively compressed written human knowledge, so they can echo human reasoning patterns already present in what they were trained on. The author points to a paper called 'The Illusion of Thinking' as an accessible account of just how limited that reasoning actually is. Separately, the author flags an unrelated, ongoing research direction aimed at having models predict the outcomes of actions, and points readers toward 'JEPA' models, world-model research, and recent talks by Yann LeCun as a starting point, while noting plainly that none of that is what today's coding agents actually do.
Despite those reasoning limits, the author sees real room to make large language models more effective in practice. Most of the gains observed so far come from feeding a model good, concise, well-timed context and pairing it with deterministic validation tooling that returns natural-language feedback the model can act on to correct itself. What stands out most to the author is not that models can predict what to write, but how effective they have become at tool calling and following instructions. That same strength has a cost: what Simon Willison termed the 'lethal trifecta.' Because large language models cannot reliably tell good instructions from bad, the author writes, they are foundationally unable to consistently block prompt injection attacks. Alignment work, safety harnesses and sandboxes help but leave real gaps, and the author describes something that tirelessly follows instructions without good reasoning as 'nightmare fuel.'
The post closes on a hope for near-term training advances: post-training reinforcement learning from human feedback that bakes in more of what it means to build software with clean interfaces, debuggability and maintainability, as part of the reward signal itself. Until then, the author argues, carefully reviewing, planning and fixing the seams of software and systems remains a critical skill for engineers to keep practicing, with or without agentic help, and matters more than ever given how often people now reach for AI agents, which the author calls 'clankers,' the moment something looks easy to build. In a final aside, the author notes personally writing every em dash in the piece, framing the habit as a personal writing quirk rather than a sign of AI-generated text.
Key facts
- The author argues that AI agent harnesses crossed the 'can it be done' threshold in the past year, but says that is only the start of the software engineer's job, not the majority of it.
- Using agent harnesses with foresight, including prompting for red/green test-driven development, produces code that 'works' and is 'testable' but, in the author's words, 'not very solid much above that': the harder part is designing clean, debuggable, maintainable interfaces, the 'seams,' between a piece of code and the rest of a system.
- The author's core technical claim is that large language models do not reason, they predict, since the models are compressed written human knowledge; the author cites the paper 'The Illusion of Thinking' on the limits of LLM reasoning, and separately points to 'JEPA' and world-model research, including talks by Yann LeCun, as a different, unrelated line of work.
- The biggest practical gains the author has seen come from giving models concise, well-timed context plus deterministic validation tooling that returns natural-language feedback for self-correction; the author is most struck by how effective models have become at tool calling and following instructions.
- That same instruction-following strength carries a risk the author calls 'nightmare fuel,' Simon Willison's 'lethal trifecta': large language models cannot reliably tell good instructions from bad, leaving them fundamentally unable to consistently block prompt injection attacks even with alignment work, safety harnesses and sandboxing.
Why it matters
AI coding agents have gotten good enough, fast enough, that the public debate about them has split into two loud camps: hype about what agent harnesses can now build, and predictions that the software engineering profession is ending. This piece stakes out a third position from someone using these tools daily: agent harnesses have genuinely crossed into 'can it be done' territory, but that is the easy, early part of the job, not most of it. The argument matters because it reframes what 'AI can code now' should mean for a working engineer: less a threat or a shortcut, more a power tool that still needs a skilled hand on the parts that decide whether software survives contact with the real world, its debuggability, maintainability and design.
Who it affects
Software engineers and teams already using agent harnesses for day-to-day coding, especially anyone tempted to treat 'it compiles and passes tests' as the finish line rather than the starting point. It is also relevant to anyone deciding how much to trust agent-produced code in systems meant to last, and, in the author's closing hope, to the people who train these models, since the piece argues that post-training methods themselves should start rewarding clean interfaces and maintainability, not just working output.
How to use it
The author's own practices: prompt agent harnesses to work in red/green test-driven development rather than accept a single-shot answer; feed the model concise, well-timed context instead of dumping in everything at once; pair the agent with deterministic validation tooling that returns natural-language feedback it can act on to correct itself; and keep ownership of the design of interfaces, the 'seams' between a piece of code and the rest of the system, plus long-term debuggability and maintainability, reviewing, planning and fixing them by hand rather than delegating that judgment to the agent.
How solid is it
This is a first-person reflection on a personal blog (rhonabwy.com), not a study, so its central claims rest on the author's own experience and judgment rather than data. That said, the piece leans on named, checkable references rather than vague authority: Simon Willison's 'lethal trifecta' concept for the prompt-injection point, a paper called 'The Illusion of Thinking' for the claim that large language models reason poorly, and Yann LeCun's public talks on JEPA and world models for the aside about a different research direction. No statistics, benchmark numbers or company-specific claims appear anywhere in the piece, and the extracted text names no author, byline or publication date, and no specific AI company, model or product.
Risks and caveats
The piece's own flagged risk is the 'lethal trifecta': because large language models cannot reliably separate good instructions from bad, they remain fundamentally unable to consistently block prompt injection attacks, even behind alignment work, safety harnesses and sandboxing, a gap the author calls 'nightmare fuel.' The author separately flags unease, without elaborating, about how large language models' training data was originally gathered without consent. Readers should also weigh that this is one engineer's personal opinion and experience, not peer-reviewed research or an industry consensus, and that no specific company, product or model is named as its subject.
“It helps to know that LLMs don't "reason". They predict, and the models themselves are effectively written human knowledge compressed.”
— the author, writing on rhonabwy.com