Glonce

Report on the malicious use of AI resurfaces, last revised in 2024

securityThe Glonce editorsHacker News🔥66Glonce rating66Impact70Novelty45Relevance100Surprise35Players45Credibility90

An arXiv listing titled 'The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation' surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent and mitigate them. The report analyzes how AI could reshape the threat landscape across three domains: digital, physical and political. From that analysis, it makes four high-level recommendations aimed at AI researchers and other stakeholders. It was first submitted on 20 February 2018 and was revised for the first time, to a second version, on 1 December 2024; the listing does not explain what changed between the two.

Beyond the four recommendations, the report says it suggests several promising areas for further research that could expand the portfolio of defenses, or make attacks less effective or harder to execute. It also raises, without settling, the long-term balance of power between attackers and defenders: in its own words, it discusses, but does not conclusively resolve, that equilibrium. The abstract does not enumerate the four recommendations themselves, nor does it name specific threats, technologies or incidents within the digital, physical or political domains it covers.

The arXiv page names Miles Brundage in the submission's 'From' field as the submitting author, but its Authors line credits 26 people in total; the page gives no institutional affiliations, so readers cannot judge from this listing alone which organizations stand behind the report.

Key facts

  • The report was first submitted to arXiv on 20 February 2018 and revised for the first time, to a second version, on 1 December 2024.
  • It surveys potential security threats from malicious AI use across three domains: digital, physical and political.
  • That analysis leads to four high-level recommendations aimed at AI researchers and other stakeholders.
  • The report also flags several promising areas for further research that could expand the portfolio of defenses, or make attacks less effective or harder to execute.
  • It discusses, but says it does not conclusively resolve, the long-term equilibrium between attackers and defenders; the four recommendations themselves are not enumerated in the abstract.

Why it matters

The report set out to map how AI could reshape security threats across digital, physical and political domains, then turn that mapping into four high-level recommendations aimed at AI researchers and other stakeholders. First submitted to arXiv on 20 February 2018, it was revised for the first time on 1 December 2024. The abstract gives no account of what changed between the two versions, so what is notable now is that the report is resurfacing years after that revision, rather than any new content on the page.

Who it affects

The report names its intended audience directly: AI researchers and other stakeholders, the group its four recommendations are written for. Because the survey spans digital, physical and political domains, that audience runs wide in practice, from people securing networks and infrastructure, to those responsible for physical systems that AI could be used to attack, to policymakers and institutions shaping how the political domain responds to AI-enabled threats. The arXiv page names Miles Brundage in the submission's 'From' field as the submitting author; its Authors line separately credits 26 people, though it gives no institutional affiliation for any of them.

How to use it

The paper is a free, open-access PDF on arXiv, with no price or tier attached, released under arXiv's non-exclusive distribution licence. By its own framing, its practical use is as a starting map: the four recommendations aim to guide how AI researchers and other stakeholders forecast, prevent and mitigate malicious use of AI, and the report separately flags several promising areas for further research that could expand the portfolio of defenses, or make attacks less effective or harder to execute. Neither the four recommendations nor those research directions are spelled out in the abstract, so using either means reading past it into the full report.

How solid is it

The claims sit on an arXiv preprint listing, not a journal page, and that listing does not show peer review status or institutional backing for its 26 named authors, which limits what a reader can judge from this page alone. What the listing does show is continuity: the paper has stayed live on arXiv since 2018, and its authors returned to revise it once, in December 2024, rather than replacing it with a new submission. The abstract states its conclusions without walking through the analysis behind them, so the underlying evidence for the four recommendations cannot be assessed from this page.

Risks and caveats

The abstract leaves real gaps for anyone relying on it alone: it does not enumerate the four recommendations, does not name specific threats, technologies or incidents in the digital, physical or political domains it says it covers, and gives no timeline or resolution for what it calls the long-term equilibrium of attackers and defenders, stating only that the topic is discussed rather than settled. It also does not explain what changed in the 2024 revision relative to the 2018 original, and though the arXiv listing credits 26 named authors, it does not identify their institutions. Readers who need the specifics, rather than the fact that the survey and its recommendations exist, should read the full PDF.

“We discuss, but do not conclusively resolve, the long-term equilibrium of attackers and defenders.”

— the report's abstract