AI poses biosecurity risk, and the ice that doesn't shatter

0:00This ad is sponsored by ZocDoc. Hi, it's Danny Pellegrino from Everything Iconic. Have you been putting off that annual doctor's appointment because you don't want to make the call? Or maybe you've been meaning to find a new dentist, but you have no idea where to start. I find that's especially true when you move. It's hard to find your people. I've been there, which is why I use ZocDoc. Now, as a busy podcaster parent and someone who's always juggling a million things, I love anything that makes life easier. And ZocDoc lets you search and compare local in-network doctors with over 250 ,000 providers across more than 200 specialties.

0:33You can read reviews from real patients, find a provider who feels like the right fit, see actual appointment openings, and book instantly. I recently had a last-minute health issue pop up and needed to see someone fast. Instead of calling around or sitting on hold, I found a doctor on ZocDoc and booked an appointment in just a few minutes. It made the whole process so much less stressful. Ready to be seen? Download the ZocDoc app today. That's ZocDoc, Z-O-C-D-O-C, and book that appointment on your to-do list.

1:07All engine running. Absolute genius. Get this. Welcome. Welcome. This is the show where we bring you science. What that essentially means is discovery is advances research technology unbelievable. Without further ado, this is The Naked Scientist. Hello, welcome to The Naked Scientist podcast, the show that brings you the biggest breakthroughs and talks to the major movers and shakers in the worlds of science, technology and medicine with me, Chris Smith. and coming up, high risk or hyperbole. Lawmakers clamber to prevent AI from being used to develop biological weapons. We hear from scientists who've brought a Second World War building concept in from the cold, why ice might be the construction material of tomorrow, they're saying, and how researchers are tracking weather patterns on faraway planets to look for life.

2:06Lawmakers worldwide are considering emergency legislation to address concerns about AI's rapid advancement. This comes as tech giant OpenAI said it will support a raft of proposed bills in the United States that could help to prevent artificial intelligence models from supercharging the threat posed by biological weapons and synthetic viruses. The company told the news site Politico that AI poses a grave threat to biological security, with senior figures at rival AI firm Anthropic also recently warning that AI could soon be able to engineer dangerous pathogens. Gemma Bousher is an expert on global health security at the Centre for Conflict and Health Research at King's College London.

2:51What most people are concerned about are the use of these technologies to make pathogens, so viruses, bacteria, or even fungi, more dangerous to humans. And that can happen in a number of ways. Either it can make them more easy to spread, so you can take a fairly benign pathogen, but make it more infective, so it infects more people from sort of bang for buck, you might say. It could also take a pathogen and make it more lethal. So it might be a question of taking a single pathogen that infects an individual but makes it kill them more quickly or make the symptoms or signs that they experience more significant and more dangerous.

3:28And the questions that we're asking now around many of these technologies is how the technology actually can be targeted and focused to create these conditions on demand, as it were, so that you can essentially pick up your menu of attributes in a virus or a bacteria and say, I want it to do this and I want it to be more powerful, more potent, more lethal, more transmissible. What does AI bring to the table that we can't do already? Because people are already manipulating organisms. 15 years ago, researchers reckoned that they could turn flu into a pretty nasty flu that would transmit very readily.

4:05They even published how to do it, although it did get some redactions a bit later on for safety reasons. So we already sort of know how to do some of these things. So where does AI come into it? So the first is really around reducing the level of expertise required to do these kinds of manipulations. So if we think about some of the chatbot style models that lots of us use on a daily basis, the concern isn't so much that they hold any particular secret information. It's more that they make it more accessible for anyone on the street, perhaps, to be walked and talked through the process of manipulating pathogens, right?

4:39The second is speed. We already have the grand designer of virulent pathogens and that's nature. Nature has been doing this for thousands of years. But the concern now is that these tools can take all of the learning and the published literature that we already have and move at speed to focus in on the more dangerous attributes that may cause issues of concern in the biosecurity space. Is there evidence that this is happening? This is really where the kind of question around and some of the tech giants becoming involved as it has come up, because we can't really document any specific real threat in this space.

5:13But certainly these actors are concerned because they've run their own tests. So what these tests did show is that some of the models did trigger safeguards that have been introduced. So we can see that the risks that people are talking about are potentially possible, and that the internal tests of these systems have in some ways been failed in the sense that they did trigger issues. Now, we've not seen AI-engineered pathogens of concern anywhere in any attributable event. It remains a kind of a speculative risk. That said, we have seen these companies say that they have put the kibosh on some inquiries coming through from certain sources, which their network has caught, and they've stopped it.

6:00I mean, And obviously, that begs the question, well, what ones didn't they catch? Or that's a group of companies being transparent. What about the ones that are not being transparent? It's the fact that it does appear to be happening then that's got people worried. And I think that's the question with any of these novel and evolving technologies. And I think there's two perspectives you can have on this. I think the first is that yes, in some ways, we can see this as a very positive step in transparency from these AI companies. You know, they've set up these benchmarks, they've set up these protocols, this process of red teaming, it's called, is where you test these models in this way.

6:34And they've published it, right. And it shows that some of these discussed risks are real. And that's important, because as we've seen, we live in a world where all of the complexity of biological events has caught the attention of the media because of the pandemics and continuous outbreaks that we continue to live through. I mean, there's also another cynical perspective, which is my own, which is that, you know, we can measure these risks in biological space in the context of AI tools. In fact, there's a lot of discussion at the moment around the regulation of these AI companies, and perhaps they might see biological risks as a safer territory to engage in some of these conversations in, because very specific proposals for regulation can be put forward, rather than perhaps some of the more diffuse, catastrophic global risks that are being discussed around really the legitimacy of the technology on a wholesale basis.

7:22I suppose one of the tricky things, though, is that very often scientists need to do this kind of thing for all the right reasons. So if we're thinking about flu, you might want to ask very legitimate questions about, well, what could flu evolve into to become much more virulent or to make people much iller? And what would I have to do to counter that? Now, they're perfectly legitimate medical questions, medical inquiries and the kind of thing we'd like a virologist to be worrying about to make sure that we're not going to get taken down by another pandemic tomorrow. But at the same time, if we block the sort of nefarious work that could involve many of the same approaches, we prevent that good work in the course of blocking the bad.

8:03Absolutely. And this, I think, is one of the fundamental challenges in biological research and in its kind of partner area of biosecurity, because at its core, this kind of work is dual use. We refer to it as dual use. It's the balance between the promise and the peril. There are completely legitimate and important types of research that we could be doing. You mentioned the work from 15 years ago. We saw the construction of essentially a flu virus from sequence data to show that it was possible. That's a scientific advance of significance, both to humanity and to the medical sciences. And many of the advances that we rely on in cancer, in infectious diseases, in vaccine technologies rely on working with very dangerous pathogens and understanding what makes them more potent and what makes them less.

8:54If we don't do that kind of work, we will not be able to benefit human health. of course, the perpetual concern is around misuse and malign actors. And that could be at the level of an individual. And that's really where some of this AI work might be a problem, where individuals are able to do more and more and more with less and less and less, or at the level of states. And we know that there are states that have had historically well-established biological weapons programs, and there is a whole convention that deals with this issue. And so one of the areas that I am concerned about is how we match AI and the regulatory powers of the convention and new regimes that might apply to manage research in a way that balances promise and peril.

9:37Gemma Bousher at the Centre for Conflict and Health Research on the debate about the bioweapon risk posed by emerging AI technologies. To cancer now and a new study has found that chemotherapy in childhood can leave lasting genetic scars on the healthy tissues of kids treated for cancer with certain platinum-based chemo drugs. These genetic changes effectively age the affected organs by the equivalent of many years, making new diseases and especially new cancers that much more likely in the future. The findings, which have been published in the journal Science, could help to explain some of the long-term health problems that children who survive cancer face later in life.

10:20Fouad Rouhani at King's College Hospital and the Francis Crick Institute was one of the senior authors on the study. We've known for a long time through clinical experience that children who've had previous cancers who then go on and survive can have medical problems further down the line many years later, secondary cancers for example that arise or diseases related to the liver and so the question we're really trying to ask with this work was can we somehow understand what happens at the time of the initial treatment which may in some way explain what then happens years down the line. Are you sort of saying that the treatment itself carries a risk?

11:02Yes, we've known for a long time that chemotherapy, which is the backbone of many treatments for a range of cancers, we already know that chemotherapy causes a level of damage to background healthy tissues around the body, but we didn't really have an idea of quite how extensive this damage was up until now. What's the mechanism of that damage? Is it that it's just flogging lots of cells and making them grow very fast to repair damage that we're doing with the chemotherapy? Or is it actually damaging the cells genetically like it is the cancer? With this work specifically, we were focusing on the liver and the way that the chemotherapy that we focused on, platinum containing chemotherapy works is exactly that it damages DNA.

11:50And that's really how it's effective against cancers. But what we've shown with this paper, and what we sought to clarify with our work was whether the type of damage that we were seeing in the background tissue was exactly the same genetic changes that the chemotherapy is good at against the cancer. How have you done it? We had an experimental design where we took liver tissue from children who had a rare type of liver cancer called a hepatoblastoma. And we were taking lots of different biopsies and little samples of the liver of a number of children and then applying some quite sensitive DNA sequencing technologies to this.

12:32What are you doing comparing the genetic sequence of the liver tissue before and after the treatment for the cancer? In a lot of these cases, we only have the sequence of the DNA once the tissue comes out. But what we're comparing this to was, for example, other tissues such as blood from the same patient, from the same child. And what you're asking, have we caused any new damage to the DNA in these individuals in the course of trying to treat their cancer? Exactly, exactly. So one of the techniques that we apply in these sort of scenarios is to try and detect patterns of changes within DNA mutations, the change of the DNA and mutational signatures of these patterns.

13:15And we know from other work that lots of people have done that certain types of damage, such as due to tobacco from smoking or ultraviolet light, causes characteristic patterns of mutation. It's not totally random. So the changes in the DNA are characteristic. And hence, by looking at the DNA and the changes, we can then link to a potential cause. and so what we were looking at with the tissue that we were taking was to see whether we could see a pattern which was consistent with the platinum chemotherapy that the children had been treated with. The field already knew of signatures or these patterns of DNA changes associated with platinum so we not only saw those but the interesting thing was we saw a completely different type of signature which was only present in the children who had had or being exposed to platinum chemotherapy for one.

14:11And secondly, the new signature that we'd found associated with platinum was only seen in liver cells and not in other tissues. And what are the implications of that then? Have we effectively artificially aged these individuals in the sense that what we've done is add damage that is like a notch further towards a disease in weighting in these individuals? Yes, we know from lots of other work that DNA damage or mutations are associated with age and they arise over time. And so what we were able to do in this study was to compare the amount of mutations or DNA changes in the livers of these children and compare that to adults.

14:53And to our surprise, we saw that the ones who'd had a chemotherapy, the damage had actually aged the livers of the children to approximately the same as that of an adult. Does this happen in adults as well then? So if we use the same drugs in say ovarian cancer cases or something, do those people succumb to the same accelerated ageing so an adult ends up with a liver of a really elderly person? It's a very good question. That wasn't something we looked at in this study but I think it does follow that agents which can damage DNA such as chemotherapy would do this in an adult as well as a child.

15:29And what are the implications of that then. Now you've found that in the liver and shown that these children appear to have much older livers than they would have based on their chronological age. What does that mean for them? What we've really uncovered with this work is something which was there all along but was invisible before some of the latest sort of sequencing technologies and so really our work has highlighted that these patients do need continued follow-up. There may also be opportunities to perhaps design different types of chemotherapies for these sorts of patients. An example, perhaps, of the pill sometimes being as bad as the ill.

16:10Fascinating findings there. Ferd Ruhani at King's College Hospital and the Francis Crick Institute.

16:18This ad is sponsored by ZocDoc. Hi, it's Danny Pellegrino from Everything Iconic. Have you been putting off that annual doctor's appointment because you don't want to make the call? Or maybe you've been meaning to find a new dentist, but you have no idea where to start. I find that's especially true when you move. It's hard to find your people. I've been there, which is why I use ZocDoc. Now, as a busy podcaster parent and someone who's always juggling a million things, I love anything that makes life easier. And ZocDoc lets you search and compare local in-network doctors with over 250 ,000 providers across more than 200 specialties.

16:51You can read reviews from real patients, find a provider who feels like the right fit, see actual appointment openings and book instantly. I recently had a last-minute health issue pop up and needed to see someone fast. Instead of calling around or sitting on hold, I found a doctor on ZocDoc and booked an appointment in just a few minutes. It made the whole process so much less stressful. Ready to be seen? Download the ZocDoc app today. That's ZocDoc, Z-O-C-D-O-C, and book that appointment on your to-do list.

17:26produced in association with Spitfire. Cost-effective voice, internet and IP engineering services for UK businesses. Find out how Spitfire can empower your company at spitfire.co.uk.

17:43This is the Naked Scientist podcast with me, Chris Smith. Now, we're halfway through today's show, which gives me an opportunity to tell you about our sister programme, Ask the Naked Scientist. Each week, we endeavour to answer your questions, the things that have been keeping you up at night. And you can find that programme on our website or wherever you get your podcast. Just search up Ask the Naked Scientist. No two programmes are ever alike. Still to come on this programme, how scientists are tracking weather patterns on distant planets. Before that though, ice is plentiful, inexpensive and surprisingly strong.

18:20In fact, it can make an excellent building material under certain situations. but for one problem placed under significant load cracks can readily spread through it causing it to suddenly shatter and fail with obvious consequences historically engineers sought to fix this by mixing substances like wood pulp that contains cellulose into the forming ice and that has the effect of binding the crystals together and interrupting the propagation of cracks making ice that is incredibly strong in fact halfway towards the resilience of concrete Now back in the day this was dubbed Pycrete and in World War II a plan was even hatched in London to use it to build a submarine resistant aircraft carrier to patrol the Atlantic.

19:06Now those plans were shelved at the time but Pycrete technology remains on the table as an attractive environmentally friendly building option for cold places. And now scientists in Israel have discovered how to make it even harder and even stronger with a recipe called Bio-Pycrete. This uses naturally occurring proteins to stitch the wood pulp material much more tightly to the ice crystals. It's the brainchild of Hebrew University of Jerusalem's Ido Braslavsky. So what we have is a mixture of ice and cellulose, which is basically chopped wood in a sense, but on small crystals. And what we add to that is a special double-sided glue of two proteins, basically one stick to ice and one stick to wood or cellulose.

20:00We're investigating ice-binding proteins for many years, and we know their ability to stick to ice and hold it firmly. and we know about this mixture of ice and cellulose so we thought maybe there's some application for that one once we have a stronger mixture i mean flippantly is this stronger houses for eskimos or is there more to it than that well it's more into that because this is not make of a of a snow which is packed and this is stronger material basically and it's called a pike right Pike rod is a program that was used many years ago in the Second World War, with the thought they can use this kind of material to build ships.

20:47So there's some history for this material of mixture of ice and wood. So this kind of material is really, it's much stronger than what you can use for igloo also. So you can really build things from that. one of the inherent problems with ice is it has a sort of brittleness to it if you push it too hard it will actually propagate cracks and then just break apart so does what you've got here surmount that problem then because you've got these different things integrated into it that that then bind it together exactly so ice is quite strong but but if you push it hard it's it will shutter. And it was known that if you add to that additive, such as wood, it will become stronger.

21:35So it's a kind of a ductile material. And the energy to break it is much, much higher than the usual ice. So eventually what we got with all the composition, it is 70 times more energy to break that than the regular ice. And it's just by how much you can push on that, it's 10 times stronger than ice. How do those proteins do that? So I'm envisaging a giant ice cube for want of a better phrase. I've got ice crystals, they've got wood material, that's the cellulose in there, and then these proteins that bind onto the ice, bind onto each other and bind onto the wood. But how are those proteins doing that and how does that turn into this interesting materials characteristic that you've got with the extra strength?

22:21So we take two proteins from nature. One is from fish which live in cold environment and there the proteins attach to small tiny ice very firmly and not let them grow. So they're doing that by organizing water on the surface in a particular way which is very similar to ice and then all of this kind of complex stick twice very firmly. This is one side. The other side is a protein that is a part of a bacteria complex that the bacteria use it in order to hook to cellulose and then to digest it. So, and this actually comes from bacteria that like heat. So, they are very stable protein. And there, again, they have some structure which have a very flat area with a lot of connection that can make the connection to the cellulose itself.

23:20So now once we put two proteins together, one of them stick to ice, the other stick to the cellulose, now we have a kind of double-sided duct tape that can glue these two together. So why does that particular configuration, protein stuck to ice, protein then glued the duct tape to the other protein, protein stuck to wood or wood component, why does that make the ice really strong how does it stop the inherent physical problem you have with ice shattering for example and make it into this much stronger material what's the physics of that so there are two steps for that the first step is that if you take cellulose and freeze cellulose solution and freeze it in a directional way so you're cooling for one way and then the ice grow in one direction, it creates kind of a net of cellulose which make the ice between this kind of net.

24:21And now with some crack running, instead of running without stopping, it's stopping this net. So this is without the protein, so it's already much stronger. But once you add this kind of extra glue, it's double the strength of the material. So to break further the crack, to go further it's a it needs more energy to move on so altogether so we have the structure of the cellulose ice and then on top of that you make the connection between the cellulose and the ice stronger and is that how you make it you sort of have a giant ice cube tray where you you mix all the ingredients together and then freeze it and it forms an ice cube with these things in or is there a special way you have to layer this to get the right three-dimensional structure so that works What we use is a directional freezing.

25:15That means that instead of just putting the whole cube into the freezer, we are cooling from the bottom. So that means that we have a directionality for the growing of the ice. So it grows from bottom to top. So that means that once the ice is growing, instead of slowing down, it keeps growing in a constant velocity, which keep the size of the crystals quite uniform and then you have the whole structure quite uniform and organized. So it looks like a sheet of ice that grows up from the bottom to the top. What can you do with this then? I mean, I flippantly said, are we going to build better igloos for Eskimos earlier?

26:00But is there a realistic prospect that you could use this as a building material? I mean, you've obviously got to think about temperature, but is that the aspiration with this? So the big advantage is in a remote cold place and this could be in the Antarctic or Arctic regime which it's expensive to bring concrete and also concrete in order to harden you need to keep it warm for a long time. So if you go to some places which you have water in the cold. So this is basically free. So all what you need is to bring only about 3 % of the cellulose and about 0.1 or 0.15 % of protein. So it's a tiny fraction of the mass you need to bring.

26:50And this is a huge advantage. And now you can build with that different constructions that you need. It could be part of a housing or road or bridge, for example, a frozen road, which you make it stronger. So this kind of construction abilities is feasible and the advantage is when it's remote and cold. It's an ingenious idea, isn't it? Ido Breslavsky there. Up into space now and scientists say they can now track weather patterns found on faraway planets starting with a nearby brown dwarf. Now this is basically a massive body that's not quite big enough to have become a star and it's located about 20 light years away.

27:40Being relatively close and a heat source makes it a convenient study subject and the team at Trinity College Dublin have been able to watch the patterns of thermal radiation coming from the object as it rotates and map those onto models of how we think atmospheres, clouds and storms work in general. It is an important step towards being able to read the weather on remote worlds, helping us to home in eventually on the ones that might be habitable. Merle Shrader. We know on Earth vaguely what drives the weather. So what we've found is what drives weather on planets outside of our own solar system that aren't as resolved as our solar system planets.