AI can't devise new hacking methods alone, but excels with a human, researcher finds
At the Black Hat security conference in Las Vegas on Wednesday, longtime web security researcher James Kettle presented findings from months of experiments testing a narrow, hard question: can agentic AI invent genuinely new, abstract hacking methods entirely on its own, from concept through to a working attack? Agentic AI has already reshaped cybersecurity, making it faster to find software vulnerabilities and either patch them or turn them into working exploits, and major AI organizations have begun disclosing real-world cases of rogue AI hacking; Kettle wanted to look past that broader shift and test the autonomous-discovery question specifically. His experiments began in September 2025, using Anthropic's and OpenAI's latest models at the time. The conclusion was nuanced: AI is perhaps minimally capable, but extremely limited, at devising new attack paths in a fully autonomous way. Paired with human guidance at key moments, though, Kettle found AI becomes an extremely powerful partner for conceptualizing and uncovering new hacking strategies.
Kettle initially set out to test the models' ability to do open-ended theoretical security research, but ran into a problem: the systems tended to pass off existing research as original, returning findings on extremely esoteric topics that were difficult to verify. So he narrowed the scope to his own area of web security expertise, where he had full command of the material and could be certain the AI was not fooling him. He also fed the models his own research methodology, which let him probe further into what they could extrapolate on their own. Kettle says, "I'm interested in pushing AI to the absolute limit to see where it fails and where you need a human." He argues few people in security openly discuss where those limits sit, because, as he puts it, everyone wants to be seen as "AI native," not to admit where their system falls apart.
As Kettle refined the experiments, feeding the models more methodological detail and sharper parameters, and as more capable models arrived, the systems began generating findings faster than he could keep up with, creating what he calls a productive research feedback loop. Notable findings arrived roughly every two days, even when he was not logged into the system, he says, a pace that left him anxious: "It was so many research leads that you have FOMO about not exploring all of them, so it forces you to automate more analysis." Over a few months, Kettle found more proven examples of certain vulnerability types than he says he could likely have found in years working alone.
Kettle had also hoped the AI could surface an entirely new class of bug, and in a way it did. The systems flagged a bug type Kettle had not seen before, though it turned out to be extremely rare and, on the one vulnerable target available to test it against, not actually exploitable. The insight behind it came from the AI noticing that some web servers use the same code to process both requests and responses, a pattern Kettle has since named Shared-Parser Confusion. "This is an absolutely massive deal, because if you think about it, requests to a website are completely untrusted, they could be anything, but responses are trusted," Kettle told WIRED ahead of his talk. "So this is a major attack surface and potentially spills into a lot of different attack types."
Kettle stresses that even though it took a human to close the loop, that is precisely why the finding matters: it shows how AI can contribute most powerfully to both defensive and offensive security work right now. The AI could not prove the Shared-Parser Confusion hypothesis on its own; it analyzed real, proven findings, generated the hypothesis, and Kettle evaluated and confirmed it himself. He calls it likely the discovery with the biggest long-term impact from the whole project: the AI, he says, spotted something he would never have found alone, even given a single line of documentation, but it could not have proven the finding without him.
Key facts
- Longtime web security researcher James Kettle presented Black Hat findings from months of experiments, begun in September 2025 with Anthropic's and OpenAI's latest models at the time, testing whether agentic AI can devise novel hacking methods in a fully autonomous way.
- His conclusion: AI alone is perhaps minimally capable but extremely limited at devising new attack paths on its own, while becoming an extremely powerful partner when paired with human guidance at key moments.
- The AI systems initially tried to pass off existing research as original on extremely esoteric, hard-to-verify topics, so Kettle narrowed the tests to his own area of web security expertise, where he could vet every claim himself.
- As Kettle refined the experiments with more capable models, AI-generated findings arrived roughly every two days, even when he was not logged in, letting him find more proven vulnerability examples in a few months than he says he likely would in years.
- The collaboration surfaced a new vulnerability class Kettle calls Shared-Parser Confusion, rooted in web servers that reuse the same code to process untrusted requests and normally trusted responses; the AI produced the hypothesis, but Kettle had to evaluate and confirm it himself, though it was an extremely rare, non-exploitable case in the one available target.
Why it matters
Kettle's experiment lands on a live question: how much of the current talk about autonomous AI-driven hacking is real, at a moment when major AI organizations are already disclosing real-world cases of rogue AI hacking. His months of dedicated testing come back with a nuanced answer rather than a hyped one. AI on its own remains, in his words, perhaps minimally capable but extremely limited at devising genuinely new attack paths. Paired with a human who scopes the problem, checks its work, and knows the domain cold, though, it becomes an extremely powerful research partner. The proof is concrete, not hypothetical: the collaboration surfaced Shared-Parser Confusion, a newly named vulnerability class rooted in web servers that reuse the same code to parse both requests, which are inherently untrusted, and responses, which are normally trusted. Kettle calls that combination a major attack surface that could spill into many different kinds of attacks.
Who it affects
Anyone running web servers where the same parsing code handles both incoming requests and outgoing responses sits inside the attack surface Shared-Parser Confusion describes, since it turns a channel normally treated as trustworthy into one that can carry attacker-controlled input. The finding also speaks to the security research community itself: Kettle argues few people in the field talk openly about where AI's limits actually sit, because, as he puts it, everyone wants to be seen as AI native rather than admit where a system falls apart. His account gives both defenders and attackers a concrete data point on what AI-assisted security research can and cannot do unsupervised, at a time when major AI organizations are already disclosing real-world rogue AI hacking incidents.
How to use it
There is no tool to download here; the value is methodological. Kettle scoped the AI's task to his own area of deep expertise rather than letting it roam broadly, specifically because the models had been passing off existing research as original on esoteric topics he could not easily verify; narrowing the scope let him check every claim himself. He also fed the models his own research methodology so they could extrapolate from a known baseline. That produced a high volume of leads, arriving roughly every two days even when he was not logged in, fast enough that he had to automate more of his own analysis just to keep up. But the AI could not close the loop alone. On the one genuinely novel finding, it produced the hypothesis from real, proven results, while Kettle supplied the evaluation and confirmation himself. Anyone trying to replicate the approach should expect to play that same verifying role, not hand the process over entirely.
How solid is it
This account rests on a single source: Kettle's own Black Hat presentation, plus his direct comments to WIRED. There is no independent audit of his experiments, no named models beyond "Anthropic's and OpenAI's latest models at the time," and no exact count of vulnerabilities or findings, only relative comparisons such as more in a few months than in a few years. Kettle's long track record in web security and his deliberate choice to scope the AI's tests to his own specialty, specifically so he could catch it if it tried to fool him, both support the account's credibility. Even so, the central figures remain self-reported: the piece names neither the vulnerable target where the novel bug class turned up nor the specific rare bug type involved, so outside verification of that particular finding is not possible from what is public.
Risks and caveats
Several specifics are missing from the account: the exact date of the Black Hat talk, since the source gives only "Wednesday"; which Anthropic and OpenAI models were used; the identity of the one vulnerable target where the novel bug class appeared; and what that rare bug type actually was. It is also not stated whether Shared-Parser Confusion has been disclosed to, or patched by, any affected vendor. The novel finding itself is a limited proof of concept: it was not exploitable in the single target where the hypothesis could be checked, so its real-world severity is unproven. And because Kettle designed the entire experiment around his own specialty, where he could catch the AI trying to pass off recycled research as new, how well the pattern would hold in a domain the human partner does not already know deeply is untested here.
“That's probably going to be the discovery that has the biggest long-term impact. It couldn't do that on its own, but I would never have found that on my own for sure. Even if you gave me the single line from the [documentation], I wouldn't have seen it. But together we managed to find it.”
— James Kettle, web security researcher