Government regulation of AI has arrived

0:07Welcome to Practical AI. If you work with artificial intelligence, aspire to, or are curious how AI-related technologies are changing the world, this is the show for you. Thank you to our partners for helping us bring you practical AI each and every week. FASI.com, fly.io, and typesense.org.

0:39what's up friends ai continues to be integrated into every facet of our lives and that remains true because you can now index your database with ai you can write more code become that 10xer you always wanted to be and you can even draft a letter for a lease on an apartment or a new property AI is everywhere and it might be time for us to start questioning is AI our friend or our worst enemy and that's the focus of the three part season opener of the award winning podcast called Trace Route Podcast you can listen and follow the new season of Trace Route starting November 2nd on Apple, Spotify or wherever you get your podcasts and this show is all about the humanity and the hardware that shapes our digital world in every episode of Trace Route a team of technologists seeks to untangle the complex question who shapes the internet.

1:26Seasons one and two gave us a crucial understanding of the inner workings of technology while revealing the human element behind tech. And season three tackles not just AI questions, but also how can we use technology to preserve the earth? Who influences the technology that gets made? And what happened to the flying cars we were promised? I think it's safe to say that the future of AI is both exciting and terrifying. So it's interesting to hear the perspectives of experts in the field. Listen and follow this new season of TraceRoute starting November 2nd on Apple, Spotify, or wherever you get your podcasts.

2:22Welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist and founder at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. Today's a fully connected episode with just the two of us where we're trying to keep you updated with everything that's happening in the AI community and maybe learn some things ourselves that help us level up our own understanding of these topics and yours as well. So, yeah, how are you doing, Chris? Are you keeping fully connected? I'm definitely fully connected, and there's been a lot to fully connect to this week.

3:04You know, there was a bit of homework going into this episode here, so it's been interesting. We got a lot to talk about today. Yeah, there's a lot happening in the world. Normally in these episodes, or at least it feels like in recent times in these episodes, there's been a lot of updates on new models and other things like that. And that's still happening. So, you know, things like Mistral and other models have come out. But I think the interesting thing maybe that I've seen people talking about this week in particular circles back to government interactions with the AI community and in particular the White House here in the US, the White House, the president's executive order on safe, secure and trustworthy artificial intelligence, which is kind of timed interestingly with other things as well.

4:01But yeah, I know that you're very in tune with the public sector, Chris. Are you seeing a lot of discussion of this in your circles? Yeah, we are. I would say that we're actually, as we parse through it and we'll talk about the different sections and stuff, I would say a lot of the stuff that would affect my day job in the defense and intelligence world, we're already kind of doing. Not kind of doing, we're already doing a lot of that stuff. And so there's a lot of specifics in this executive order, but it's not starting a new process for us in that world. It's something we've been working on for quite a lot, for years.

4:38So it's interesting. I was pleasantly surprised with this executive order because, you know, we've talked many times on this podcast about the fact that how long it's taken for governments to start getting a bead on these AI issues and what does regulation mean and who's going to participate and, you know, how are you going to do it and all this kind of stuff. We've been saying that for years. And finally, on Monday of this week, as we're recording, Monday, October 30th, 2023, we got this executive order issued. And then I believe we got the Bletchley declaration issued as well later in the week.

5:13You want to talk about that a little bit? Yeah, sure. It might be useful. I know we have a wide range of listeners. And if I'm being honest, even, you know, myself, we had a friend that just became a U.S. citizen the other week. And congratulations. Yeah. Talking through things with him is like, of course, it's fresh in his mind, but all of these ways about how our government works and the various ways in which things can be legally enacted can be quite confusing. So maybe before we jump into things, let's maybe just touch on an executive order what might that imply and how it may be different than certain things that have preceded it because there's been statements on ai and government thinking about ai in the past here in the u.s but from your perspective what makes an executive order maybe different than some of the things that we've seen in the past sure so noting that I am most certainly not a constitutional attorney or any such thing.

6:19Just a dude who likes AI. I would still vote for you, Chris. Oh, that's really nice of you. But an executive order in short, and I'm sure if we have listeners that say I'm slightly off, they can correct us on this. But the president of the United States can issue an executive order, which is a legal device, which essentially has the effect of law, it can be overridden a couple of different ways. The U.S. Congress can override it by passing an actual law. So if an executive order is in conflict with a law that is passed in Congress, that law in Congress will trump that. And in addition, an executive order, I believe, can be the U.S.

7:03Supreme Court can also override it on constitutional basis. But unless one of those two things happen, my understanding is executive orders otherwise have the effect for all practical purposes of law in the United States of America. Yeah. And apparently these actions are, quote, the most sweeping actions ever taken to protect Americans from the potential risks of AI systems. It's funny how the use of adjectives in government has come quite interesting over the years. But this is the most sweeping actions ever taken to protect Americans from the potential risks of AI systems. There are some interesting ones, and I think what might be interesting about at least a couple of these is the way that they might influence the AI industry in the US in particular, but also some ways that government agencies and other entities might become involved in the AI world.

8:08So I guess before we jump into the specifics, you also mentioned the Bletchley decision. What's that for our listeners? How might that relate to things going on? Sure. So there was a summit. It was called the AI Safety Summit that took place on November 1st and 2nd of 2023, which was just a couple of days ago this week. They issued what they call a policy paper, which was the Bletchley declarations by countries attending the AI Safety Summit. on that date. And it's fairly short. It's a few paragraphs long. And my understanding, not being a legal mind, is that it would not be binding in any way legally.

8:49But there's a number of countries listed. There's a couple of dozen that attended, including the United States, United Kingdom, and a lot of other countries around the world that basically said, we're acknowledging, the short of it is we're acknowledging AI safety is important to us all. it's by definition an international concern and that the way to deal with these concerns going forward is for us to all work together and you know share information and such as that without reading the whole thing to the audience on the show which i don't think we have time for yeah we will link these things in the show notes absolutely it's a good thing i agree with everything they said uh and it's a good uh kind of a kumbaya of saying we need to work together but it's otherwise, you know, it's just saying, hey, let's go do these things.

9:34That's very important because it is indeed an international concern. I'm definitely applauding that. I've been more focused a little bit on the executive order simply because it's binding, having the effect of law. And we'll talk about the details, but it gets quite specific in the executive order itself. There's a fact sheet that kind of gives you a high level and it doesn't give a lot of the detail. And I read the fact sheet first and I was a little bit, I was like, okay, that's all great, fluffy stuff. But when I read the executive order afterwards, it gets down to who's responsible for what, how long they have to do it, and what they have to do it.

10:07And there's a bunch of specific results expected or standards applied that clearly were people from the AI community that were very, very knowledgeable. So it wasn't a political-only group of people that did this. They obviously had expertise available to them. So I was, I ended up being more impressed than I expected with what they came up with. I haven't done all of the research and I don't know if it's even published anywhere about who was involved in the process of developing this, but we can go through some of the high level things and there's a few really helpful. So there's the fact sheet that you mentioned there's also some good articles that I've been looking at over the past days that give some summary information that might be relevant for people as well like the MIT tech review has an article and some other ones if we don't dive into the the specific wording quite yet and I just look at kind of the high level what people are are saying about it what's standing out to them about it.

11:20One thing that I see really driven home is the real focus on safety and in particular safety and security and the real focus on kind of like labeling and watermarking the output of AI systems, which I think if I'm understanding right is framed within the executive order as a safety thing in terms of protecting our citizens from potentially fraudulent or harmful material that might come out of AI models, as well as giving, whether it's entities related to defense or education or whatever, the ability to identify and discriminate between AI-generated assets or content or text and non-AI-generated, or I guess human-generated.

12:14You know, we're talking about a specific point here in it, but it takes both sides of that. It both issues the directive that you, in essence, you must identify AI-generated content so that you can't have deceptive representation there. But it also issues the appropriate government agencies to come up with mechanisms by which they may detect people who are not subscribing to that directive or foreign. Obviously, there are all the other countries in the world and they are not held by our executive order. So the means to detect all that. But I also would say that that's not a new idea to the DOD and intelligence community.

12:53And am I understanding this right, Chris, that the executive order would put the obligation on certain government entities to figure out how to enforce this mandate on the other organizations, companies, entities, teams that are within their jurisdiction? Is that a good way to put it or am I misunderstanding? Yeah, it leaves a lot undone. And at some point, we probably should jump in and kind of hit the highlights on what they were. But it basically puts the burden on various agency leaders. It'll outline what they have to accomplish and when it must be accomplished by, and in some cases, what the output of that is.

13:41But it doesn't tell them how to accomplish all that. And I have yet to see any place where it ascribed any budget to any of those items. So there are strengths. It was good thinking in many areas, but the dollars that would go into making some of these things happen, I did not see assigned. So positives and negatives. When you read this, when you see this coming through and how things have trended over the recent years with respect to the government's thinking on AI, is this like taking us up from a level two of priority in government and action up to, you know, 10? or how do you see this as escalating the kind of involvement from the government in the AI industry in terms of practical things that we'll see in, let's say, the coming year?

14:34Oh, I would say that they don't have a choice given that it's now in an executive order and it directs them with timelines and specifics on that. If I was leading a federal agency that was being called out to do this, I would probably be scrambling and trying to figure out who I had and who I could get and where I was going to pull the money from to accomplish those things. And I'm hoping maybe a listener knows that there's budgets described that we haven't heard about yet, and that would be good news. They would have to juggle a little bit in terms of their priorities to make some of these things happen.

15:08But I think it's good. It forces it right up to the top of our list of things to, you know, the government could happen. And I don't think it was going to happen from industry alone. We've watched for years as commentators on the industry every week. We've seen individual companies kind of do their own thing. They kind of compete with each other because there's a bit of a marketing tinge to AI safety as well. But nothing that has ascribed the entire industry universally. And so this will clearly do that with external standards.

15:44this is a changelog news break hugging face released a distilled variant of whisper for speech recognition it's english only and optimized to the hilt which resulted in running six times faster while being 49 smaller and performing within one percent word error rate from the original model. It's designed to be a drop-in replacement, and the Hugging Face team cites five reasons why you might use it. Faster inference, robustness to noise, robustness to hallucinations, designed for speculative decoding, and permissively MIT-licensed. This looks great, but I'm still waiting on speaker diarization.

16:24You just heard one of our five top stories from Monday's Changelog News. Subscribe to the podcast to get all of the week's top stories and pop your email address in at changelog.com slash news to also receive our free companion email with even more developer news worth your attention. Once again, that's changelog.com slash news.

16:51Okay, Chris, let's dive into some of these specifics, which I think are really quite interesting. The one that I thought was probably most interesting to me, although I think the others have both good and interesting implications, as my wife and her business would say, there's wins and opportunities. But I think one of the ones that stood out to me was, I'll just read the wording here and we can talk about it and some of the wider implications. But one of the things is a requirement that developers, which is interesting to me because it's, I'm an AI developer, I guess, a requirement that developers of the most powerful AI systems share their safety test results and other critical information with the US government.

17:45How does that strike you? What are your thoughts? Depending on the model and the specifics of that, it will often be the U.S. Department of Commerce that's receiving those. In some cases, it will be military or intelligence, depending on the nature of what the concern is and what the model can do. But that will have to be created. It doesn't exist today, to the best of my knowledge. If I was leading commerce, I'd be going, okay, how are we going to receive this information and store it? Because they're probably going to be getting quite a lot of data coming at them with the amount of development in this area.

18:19That's one of the things that certainly here in the U.S. we're going to have to learn how to do because as of now, once the timeline, I don't have in front of me how long they have to put that into place, but after a certain number of days, it will be required by law. If we're specific on the thing that would be required by law, if I'm understanding this right, It would be someone that is developing a new, maybe one thing, I don't know if you've seen this, maybe I just missed it. I'm not sure if this would be new in the sense of training from scratch or new in the sense of a fine-tuned model. But those that would release large models like, you know, in recent weeks, we've seen Mistral and Llama 2 and Zephyr and all these models coming out.

19:07that models that are significantly large, like that foundation models that are significantly large, as those are released, after they're released, before they're released, maybe we can talk about that, that the people that are producing those models, the teams, the developers that are producing those models perform some sort of red teaming to probe the models in terms of potentially harmful outputs and kind of behavioral tests and perform risk assessments that would be, you know, gathered in some sort of coherent way and shared with the U.S. government. One of the interesting pieces of this part of the executive order in particular, if I'm understanding right, is that it falls under the authority of the Defense Production Act.

19:58Correct. Which I think in addition to the executive order status might bring this point maybe a little bit higher up in terms of firm legal footing. Again, not an expert in that thing, but that's my understanding from what I've read. Yeah. And as a non-expert, but the Defense Production Act, in case listeners may recognize that because they've been hearing about it a lot over the last year because it's been a point of in the Russian invasion of Ukraine. It has been repeatedly cited as a mechanism that the U.S. government can use to increase production to support that effort, our allies, the Ukrainians.

20:41And so you may have heard that before. And from that, I know as a non-attorney, non-legal mind, that it gives the U.S. government broad powers on requiring commercial companies in the United States or that we do business with to meet a certain set of criterias. And there are things that they can be told, you must go do this because it's in the interest of our national security. And so it's a fairly sweeping thing, is my understanding. Since it has been referenced here, I agree with you. You not only have executive order, but you also have reference to a fairly strong point of law from Congress.

21:19One of the interesting takes that I saw on this is this is just a blog post that I'll link in the show notes. But the comment in the blog post was that this might be one of the things that would kind of lead to a firming up of the players within the AI market as we see it. because not only is now there a computational data infrastructure burden on those who want to produce these large models, but now there's more of a regulatory burden that would actually be an additional kind of step here in terms of being a player in the foundation model space. Now, there's irony to that, by the way, and that they explicitly note they're trying to address equity in the executive order, but by adding the regulatory burden that will be exclusionary.

22:16Correct. Yeah. You don't get anything for free. That's for sure. So this will be interesting. It's hard for me to think that the progress will slow very quickly around releases of these models. What is very interesting to me is how they will end up deciding, like if I pull down llama two and use a small data set that i have access to to fine tune it and then i release that as another foundation model that people can use well you know at what point so i am modifying the weights right at what point between there and training from scratch which you know even large models I don't think often are they might start from a starting point with their weights in certain cases but yeah at what point in there are you really a developer of a significantly large model because both of these models are large when is it adaptation when is it fine-tuning when is it training or releasing building as the executive order says so yeah that that's all kind of mushy in my mind, I think.

23:31Yeah, they do attempt to kind of address that. There's a section 4.2, ensuring safe and reliable AI, and they talk about a time frame. And this is one of those that calls out the Defense Production Act specifically. And within 90 days, there's a set of things that are being required, which is a very short timeline when you think about it. Part of that is they They have a set of criteria, which is certainly not comprehensive, to your point just a moment ago. It's better than I expected, but it's not entirely sufficient. There's a lot of nuanced questions, as you just raised. I noticed that one of those, they talk about the quantity of computing power used for training.

24:15and they have picked, interestingly, 10 to the 26 integer or floating point operations for kind of as a general point of computation. It's like a threshold. And if you're above that, you're kind of in that large range that they are particularly focusing on. They have reduced that down to 10 to the 23rd integer or floating point operations for models that are based primarily for biological sequencing and things like that. they have some other aspects in but they have an interesting threshold that they call out in the in the executive order so you know exactly what's going to happen here chris is that whatever that number ends up being so it's just like in in so the town where uh my dad grew up in in kentucky is a um actually still is i believe a dry county meaning in the u.s this is where you can't actually purchase packaged liquor in a liquor store.

25:16Of course, what happens then is just on the edges of the county, you have these at the sign where the county ends, there's like 14 liquor stores, right? We'll see something similar here, right? We've been choosing our numbers of parameters and such 7 billion, 13 billion for various reasons for our models. What's going to happen is people are going to get really, really good at training models right under that threshold, which is 10 to the 25th will be the magic number going forward. Yeah, which, you know, I think it could actually have a, even though that's kind of gaming the system, it could actually have a really nice effect that instead of us trying to always think about more data, bigger model as the way to incrementally improve, this does put a burden on those that want to operate at the lower level under the threshold of regulation, the burden to say, hey, what if we're creative either in our model architecture or the way we train it or the way that we fine tune or whatever that ends up being to actually do more with less, which I think overall will be a good, would be a good thing.

26:27And, you know, academia is already thinking about these things with things like the baby LM workshop and other things like that. So I think that could actually have a follow on effect that's quite positive for the model landscape. You'll have a set of players that remain out of necessity, you know, they must play up in that area because, you know, the large, the true LLM range isn't going to go away. But that will also be dominated by large players who are already doing regulatory stuff anyway. Maybe they weren't in this, but they're accustomed to that. It is exclusionary to those large, you know, things like cloud providers and such will be doing that, but you'll probably have a whole range of mid-sized players that are below the Amazons, Googles, and Microsofts of the world and the open AIs of the world that will play in that just below that and build foundational models.

27:19It may be that, yeah, as you said, it'll be interesting to see what kind of innovations come from there. And one other question that I have coming out of that is, how do I know that I hit the 10 to the 26? And how do you, along with this sort of restrictions or legal implications throughout the executive order, this kind of naturally brings up a lot of questions. Like they also talk about watermarking, which we can talk about here in a second. But the general thought is like whether you're talking about this computational power, red teaming, behavioral tests, watermarks, labeling, you need standards and tools and tests to help you ensure that you can do these things.

28:07Like how do I know when I hit that threshold? How do I watermark things, et cetera, et cetera. And so one of the other things that's drawn out right away is the developing of standards, tools and tests to ensure that ai systems are safe secure and trustworthy and this specifically calls out the national institute of standards and technology or nist that you might have heard of before because they have like one of the most precise clocks to you know help keep the standard of time and the most precise weights yeah the most precise like this is exactly what a kilogram is. And yeah, I actually in my undergrad when I was doing research, I did research at NIST with one of our collaborators.

28:55So we were theoretical, they were experimental. And I think mostly all I succeeded in doing was spilling a bunch of carbon nanotubes on the floor. Not very good at experiment, but that's what they're experts in minus the occasional intern that spills carbon nanotubes on the floor. But they're specifically called out to help or set the rigorous standards for what's phrased in the executive order, extensive red team testing to ensure safety before public release. What are your thoughts on this, Chris? This goes back to something I mentioned earlier. There's a lot of figuring out the how that's undetermined.

29:35So, you know, you have clearly some bright AI minds that help construct the executive order, but they've left wide open what that means and what is red teaming? What is red teaming trying? They hit some things that red teaming should be trying to do at a very high level, but it's up to NIST and the Department of Commerce to come up with what the specifics are on that. And I think we're all going to be learning. I think the key thing that I would take away from that is that this executive order is the first of many things to follow over the next year from various agencies as they are trying to fulfill the executive order's intent.

30:14Well, Chris, the next thing I see in here is biological materials, which I don't necessarily think about that much, even though I'm made up of biological materials. I guess I don't consider my own biological self very much. But they talk about protecting against the risks of using AI to engineer dangerous biological materials. What is a dangerous biological material? I guess a bioweapon? Is that what we're talking about here? It would be bioweapons. And it could be something that we've all heard about. When we were in the height of COVID, there was all the theories about whether or not it had been created in a lab in China or elsewhere and such as that.

31:01And so it could be a weapon by design. It could just be a virus. It could be lots of different things. There are a lot of international laws and domestic laws against these things, but we also have actors around the world who don't necessarily subscribe to the same values. And so it's still something that the intelligence and defense communities of both the U.S. and our allies spend a lot of time thinking about how to address and defend against, though we follow those laws that our adversaries may not. So, yeah. Yeah. People might be wondering, well, how might you practically think about protecting against the development of dangerous biological materials with AI?