Kevin Leyton-Brown isn’t just worried that the future of AI misinformation will be about tricking people. He’s worried it will be about making them distrust other people . 

Leyton-Brown, a University of British Columbia and Amii Canada CIFAR AI Chair, was part of an international team of researchers exploring how multi-agent misinformation networks might influence elections and weaken democratic institutions - underscoring the importance of more research into AI trust and safety. 

“It's really not all that hard to do, and it's pretty hard to detect,” he says. 

The paper, How Malicious AI Swarms Can Threaten Democracy, was published in the prestigious journal Science earlier this year. In addition to Leyton-Brown, the authors were made up of researchers from nearly 20 institutions, including the Max Planck Institute for Security and Privacy, Oxford, Harvard, Cambridge, MIT, and ETH Zürich. The authors argue that recent advancements in Large Language Models and the multi-agent AI system could lead to “swarms” of malicious AI agents, working together to spread misinformation in a way that is more effective and harder to detect. 

Leyton-Brown likens these potential misinformation methods to how photo manipulation has changed how much people trust images.“For many decades, a photograph was irrefutable proof that something had happened. Now we've just lost that, and we're never going to get it back. And that's a humongous change,” he said. 

Old tactic, new technology

Online misinformation has been a concern for decades. But, as the authors note, earlier attempts have had limited impact. Organized misinformation campaigns used to be a labour-intensive task: “troll farms’ would have to pay hundreds of people to spend hours writing and posting misinformation on social media sites and blogs. Not only was it an expensive endeavour, Leyton-Brown says, but it also made the false information easier to identify. As a result, their impact was relatively small. The paper notes that an  assessment of Russia’s 2016 campaign found that only “one percent of users saw 70% of its content, with no measurable effects on opinions or turnout.”

“It was all pretty smoking gun stuff once you saw it. These accounts are all posting at the same time, and they have no friends but each other," Leyton-Brown says. But AI tools have changed that. Instead of having people manually write and post misleading information, LLMs can generate text that reads like it was human-written. That has led to more complex campaigns - the paper points to the use of AI in attempts to influence public perception on topics such as climate change and foreign aid in India and Taiwan in 2024, which appear to have been much more effective. And they worry that the use of multi-agent AI could make these misinformation efforts even more sophisticated.

Multiagent AI is a branch of research that focuses on systems with multiple AI agents interacting. Different agents might have different goals and capabilities that can affect one another. Depending on how they are set up, they can also cooperate towards larger goals. Autonomous vehicles are a good example of multiagent systems: while they share the road and follow some of the same rules, a self-driving vehicle will have its own destination and instructions.

According to Leyton-Brown, a multiagent swarm of bots could spread misinformation in ways that a self-contained agent couldn’t. A swarm could make a series of posts that appear to be a conversation between different users, he says, making a fringe idea or conspiracy theory seem more widely accepted than it truly is. He notes that LLM-based swarms would also beharder to detect than previous forms of misinformation. Someone who is wary of online misinformation might have their guard up if someone replies to them directly, but they might be fooled by what seems to be a natural conversation between other people. 

Multiagent swarms can also be very persistent, posting all day without the need for a break. This can not only put misinformation in front of more people, the paper argues, but it can also be used for other methods such as harassing journalists and opposing parties. There’s also the concern that AI swarms could inject misinformation into other LLMs, which are often trained by info that is posted online. If that training data is contaminated with bad information, it could be passed along inadvertently by other models. For Leyton-Brown, the concern isn’t so much that AI swarms could trick people into believing misinformation. Instead, he’s worried that they will muddy the waters, making people lose trust in any news or information that they hear. That could cause people only to trust a small group of verified celebrities and sources, cutting other valuable voices out of the conversation. Or, they could simply disengage from the conversation entirely, which could lead to things like lower voter turnout and weakened elections. 

"I think the biggest problem is that they might just destroy our ability to talk to each other. On social media, we might just stop believing that anyone we don't know is a real person," he says. 

Using data to combat misinformation

These AI swarms are a potential threat, Leyton-Brown says. But there are steps that can be taken to address them. The paper’s authors suggest several ideas to combat misinformation swarms. Some of the suggestions focus on improving the platforms that could host misinformation. The paper suggests that “AI Shield” tools could be developed that would offer real-time scoring of posts to assess the likelihood that they originated from a swarm while protecting users' privacy. As well, they say simulations could be used to stress-test platforms ahead of major events like elections, which could then find weaknesses that AI swarms could exploit. 

Methods like persuasion-risk tests, which evaluate how effective an LLM is at influencing human belief and behaviour, are an existing tool that could prevent models from being used for malicious purposes. And watermarking, which makes it clear when something is generated by artificial intelligence, could be part of the solution. 

Since misinformation is a global problem, the authors argue, fighting it requires global cooperation. They suggest that a global AI Influence Observatory, backed by an international organization like the UN, could not only keep track of proven misinformation campaigns, but might provide an early warning system for new attempts at using AI to influence public perception and voting behaviour.  

But for Leyton-Brown, the most obvious first step to fighting misinformation is real information: there is a gap in understanding the impact these multi-agent swarms could have on democracies. More research is urgently needed into this and other areas of AI safety. And since a lot of that misinformation happens on private platforms, the data on it isn’t easily accessible to those looking to find out more. He calls for great attention and resources dedicated to studying the risk of AI swarms and developing solutions to combat them. “I don't think this is a silver bullet that would make the problem go away. But it's something you could do that would be a response, and would at least start assessing the scale of the problem."

See the Full Paper

The full paper referenced in this article, How Malicious AI Swarms Can Threaten Democracy, was published in Science