AI chatbots provide usable bomb-making info in nearly a third of queries, report finds
A new report from Tech Against Terrorism, an online watchdog supported by the United Nations counter-terrorism directorate, reveals that artificial intelligence chatbots often provide would-be extremists with actionable information. Researchers sent over 2,300 requests for information based on "real terrorist use cases" to 27 different AI models and found that 32% of the queries resulted in "genuinely usable" information. When the same questions were reframed as being for research purposes, that figure rose to 42%.
The report highlights a growing concern among digital security and terrorism experts that AI is being used not just for propaganda but also for planning attacks. Over the past three to four years, extremist groups such as the "Islamic State" and al-Qaeda have primarily used AI to generate propaganda, including videos, memes, podcasts, and disinformation spread among adherents and used to radicalize potential recruits.
But the new findings suggest that AI models can also serve as a resource for operational planning. The report's authors note that "jailbreaking" — a technique where users craft prompts to bypass safety restrictions, described by OpenAI as "attempts by a malicious actor to prompt the model into providing disallowed content" — can yield useful instructions.
Earlier tests by various media organizations had already shown that some AI models, when given the right prompts, could provide guidance on making bioweapons, bombing a sports arena, or covering a terrorist's tracks. The Tech Against Terrorism report quantifies how prevalent this issue is across a wide range of models.
The report also cites real-world incidents illustrating the potential danger. A 16-year-old perpetrator wrote a manifesto with the aid of ChatGPT before stabbing classmates at a school in Pirkkala, southern Finland. Additionally, Cambridge University released research featuring interviews with Boko Haram members in Nigeria, who discussed how the group used AI models to plan attacks.
While the report does not suggest that AI is the sole cause of terrorist activity, it underscores the need for improved safeguards on AI platforms. The findings come amid ongoing debates about the balance between free expression and security in the development of artificial intelligence.
As AI technology continues to evolve, experts say it is crucial for developers and policymakers to address these vulnerabilities to prevent the tools from being exploited by malicious actors.