‘These risks are real’: the threat of AI bioweapons

0:00This is The Guardian.

0:11According to a top safety researcher at AI company Anthropic, there's more than a 10 % chance that AI could kill us all within the next decade. But how? One way is in the development of bioweapons, something that has got scientists and security agencies around the world very worried, including the head of MI5. Not yet realised, but the misuse case to which I'd attach greatest weight is AI's potential to change the stakes on existing or even new bioweapons. These risks are real.

0:50so how could ai be used to develop lethal pathogens and how serious is the threat i'm the guardian science editor ian sample and this is science weekly

1:08dr tom inglesby you're director of the johns hopkins center for health security and the Bloomberg School of Public Health. And one of your specialties is the prevention of and response to biological threats. Tell me first how you would define a bioweapon and what kinds have been used in the past? Yeah, well, I think there are different definitions we could use. I think back in the Cold War days, there were large state-sponsored biological weapons programs that had munitions and they had pathogens and storage. A range of pathogens were used in the original offensive weapons programs in the second half of the 20th century.

1:52So anthrax, plague, smallpox, other kinds of weapons that we worry about. I think the way that people talk about it today would include those kinds of weapons, but also it's kind of used as a term for the deliberate misuse of biotechnology on a smaller scale. Could be somebody who releases a virus without any kind of formal weapons program, but just intending to cause illness or harm, either on small scale or on very large scale. In his annual speech on threats to the nation recently, Sir Ken McCallum, the head of the UK's intelligence service, MI5, he warned that AI had the potential to, in quotes, change the stakes on existing and even new bioweapons.

2:41This is an area that you and your colleagues have been watching, I know. When did this threat associated with AI and bioweapons first sort of hit your radar? How long has this been a concern? Over the last few years, I think people in the field have been watching the capabilities of models rapidly progress and to see what models could potentially do in a number of ways related to risks in biology. I think you break them down into two categories. One is to make more accessible the knowledge that people could gain, the know-how to make things that already exist in the world. And the second category of concern is the potential for AI tools to modify or create new versions of viruses that are not known to nature or to science that could do things that are as dangerous or more dangerous than exist now.

3:37I was wondering whether we could look at this as threats coming from, say, nation states and capable organizations, places which will have the skills, they will have the modern labs, And then these other groups, maybe terrorist groups, maybe your rogue individual bad actor that maybe doesn't have the skills, doesn't have the access to those sort of lab equipment. What is the risk of AI at that sort of nation state level where they have a lot of capability already? No country really openly admits having a biological weapons program, although there are intelligence reports that some countries do.

4:13The goal of creating something entirely new from scratch, that's going to remain quite a challenge because the AI models themselves have to train on existing data. And so at the moment, I think the greatest concern is innovations, changes to existing viruses or pathogens, but in ways that are somewhat unpredictable or outside the usual design space that humans would be working in or that nature has worked in before. I can give you an example that received a lot of attention this summer, which is a group of scientists, very talented scientists and engineers. Their goal was to use very powerful biological A-models to design novel bacteriophage viruses.

5:00These are viruses that infect bacteria. They don't infect people. But they were able to design novel bacteriophage viruses that do not exist in nature, that were not anticipated. You know, it wasn't as if humans were already working towards those. And those viruses infected and killed bacteria and in ways that overcame the resistance of that bacteria that they usually have to these kinds of viruses. And it's a proof of principle that biological AI models will be able to help scientists create novel viruses that could overcome resistance or immune responses. This model wasn't trained on human information, but there's no reason why others couldn't train that model or other models on human information.

5:55Which I think brings us to this question of why bother, essentially? Why would a nation state look to do this? What sort of properties would they be trying to optimise for? On the more dangerous side, humans could look for ways to make pathogens more transmissible between people. They could look for ways to make viruses, pathogens more lethal or more immune evasive. We would hope that the 99.999 % of scientists would never go in that direction. But we do know from one large language model company, Anthropic, they provided two public examples this year that two state programs were trying to access their model for help in making chikungunya virus and H5N1 virus, the ones they reported, more transmissible, more immune evasive.

6:52They also reported a scientist seeking information on orthopox viruses and how to make those viruses able to avoid antiviral drugs. Orthopox viruses are the family that includes M-pox and smallpox. So why people would do this? Hopefully many of those people were trying to do something beneficial in the world around vaccines or medicines, but we don't know. We've been talking about the sort of the high level work, the advanced work that probably is the best that could be done in the world. But there was also this question mark over what could others do with it? Terrorist groups, individuals who probably don't have the high level skills, probably don't have the access to the laboratory.

7:35Does AI pose a risk on the bioweapons front at that level? Yeah, it's a different kind of problem. The concern there is the field calls that concern around uplift. What are AI tools able to do to help uplift the skills and outcomes of a small group of people, an individual who would seek to use them to do harm? And we've had benchmarks that have been created over the last couple of years to assess how well AI tools help novices or PhD level trained scientists, people at different stages of training. We've seen that frontier models are raising what is possible, providing know-how uplift in a way that they keep breaking through benchmarks, biological, biotechnology, bio-know-how benchmarks, one after the next.

8:30For the most part, I continue to be very excited and eager to see what these tools can do to help us with breakthroughs in cancer and maybe infectious disease and other areas of biology. But right now, there really aren't any clear rules of the road for preventing the potential dangers. For people listening here, it doesn't mean that now everyone accessing a tool can suddenly do something very powerful and wrong with biology. But it does change the dynamics for those who do have skill or access to some skill and would be seeking to do harm. We've been talking so far about scientists doing this kind of work intentionally, trying to use AIs to develop bioweapons or to look at tweaking existing pathogens intentionally.

9:17But I'm wondering if in this field, especially when AI comes in, the field of synthetic biology, are there risks of researchers creating pathogens or modifying pathogens or creating new toxins accidentally while they're doing experiments which may be completely benign? Yes, that was a concern before AI and AI kind of probably amplifies that. scientists even working for the good that are trying to do useful things, or at least just exploring science, might inadvertently stumble into something that creates a lot of potential public danger. And another example of that is governments that are working to protect the public, maybe in some kind of government laboratory or some research-affiliated lab, that might find something from an AI model, some output, some blueprint that seems very dangerous, there may be kind of an impulse to test it out, to create that thing in a laboratory.

10:21We could inadvertently create the thing that is the most dangerous in an attempt to protect ourselves. And I think that's a bad road to go down. Obviously, that's another thing that we want to protect against.

10:35Coming up, how to stop people using AI for bioweapons.

11:09And did. Thomson Reuters builds the technology that sharpens insights, speeds up decision-making, and powers the outcomes that matter. So when professionals act, the impact is felt by everyone. Be a changemaker. Visit tr.com forward slash changemakers.

11:31Tom, in your view, what do we need to do? What do we need to bring in now to protect us as much as that's going to be possible? Yeah, well, I don't think it's super complicated. I think for other industries where there's great benefit by potential harm to the public, we have oversight. We have checkpoints, airline industry, nuclear industry, making medicines. So in the case of AI models, just a couple of elements could really reduce the risk dramatically. One is creating some kind of access control for people working on the most dangerous data with AI tools. there's public data that's being developed all the time around how to make pathogens more transmissible or what makes something really particularly lethal.

12:19That data is the data that models need to make predictions about how to make things worse. The world manages access to human genomic information, other data sets. And so tiering or kind of having access controls for people who really need to know that data for something really beneficial, like that's one intervention. The second intervention is for powerful models, whether they're language models or biological models, above a certain threshold of power and capability, they should be evaluated before they're released into the public. And it should be something that governments approve. They set the standards for.

12:56It shouldn't be a voluntary practice that a company just decides what's safe and not safe to release them to the public. And a third intervention of three, I'm just giving three and they would be, it could be a dramatic reduction in danger. The third is that we make sure that we're kind of protecting the digital physical divide. And by that, I mean, it's one thing for the AI models to make digital instructions, blueprints, ingredients, but they still have to be translated into physical viruses in a laboratory. And one principal way of that happening is scientists ordering nucleic acids from companies in various parts of the world.

13:40That is an industry. And that industry right now does not have rules about screening those orders or screening the people who are ordering those nucleic acids. There is a movement to try to require screening. and I think we should push hard on that. In the US, there's legislation that could be passed as soon as this year. Other places are considering legislation. The industry, for the most part, who makes nucleic acids is supportive of it. They think it's a good idea. So I think that would help add another layer of really important protection. And just finally, Tom, you live and breathe this area.

14:20How worried are you? What do you see as the biggest threat in this area? Yeah, I have, I mean, I kind of have a steady low level of worry. I think the biggest threat is that we just kind of assume there's nothing that can be done. And we kind of remain frozen and watch these technologies rise in capability and speed and don't do anything about it. I am worried that they could be used to do something novel that could hurt a lot of people, potentially initiate large-scale events that would be difficult to control. But if we start with oversight and governance of the model's potential to do widespread harm, I think that's a public imperative.

14:58And that's where I place a lot of our energy right now. Tom, thank you so much. Really interesting. Thank you, Ian. Really appreciate it. Thanks again to Dr. Tom Inglesby. You can find all our coverage of AI on theguardian.com. And before you go, I'd recommend checking out the award-winning documentary, NASA. Directed by two of the Oscar-winning directors of No Other Land, NASA reveals the secret systems used by the Israeli military in the mass killing of Palestinian civilians in Gaza. Produced by The Guardian, it's the documentary that the Israeli government doesn't want you to see, and it's now available in cinemas around the world.

15:37I really urge you to go and watch. This episode was produced by Madeleine Finlay. It was sound designed by Ross Burns. And the executive producer is Ellie Bury. We'll be back on Thursday. See you then.

16:00This is The Guardian.

16:07Thank you.