EU Passes AI Act, But Generative AI Provisions Remain Controversial

0:00Today on the AI Breakdown, we examine the EU's newly passed AI Act. Before that on the brief, new tools from Google, a lofty McKinsey prediction, and AI Jesus on Twitch. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Like, subscribe, and share, and go to breakdown.network for more information. Welcome back to the AI Breakdown Brief, all the AI headline news you need in five minutes or less. Today we kick off with a really interesting report from consulting firm McKinsey, which argues that generative AI could add up to$4.4 trillion a year to the global economy.

0:35And by the way, the minimum that they think it will add is$2.6 trillion. Their new trend report is called the Economic Potential of Generative AI. And beyond just that top-level economic impact figure, they also dig into just how much of the current work activity that people are engaged in can be automated away by AI. In fact, this McKinsey study thinks that AI could automate about 60 % to 70 % of what we spend our time on. Now, McKinsey takes the optimistic view that this doesn't mean job loss, but means that people's time will be able to be used for more advanced purposes and more creation. That's part of where they see these trillions of dollars of new economic value coming from.

1:10Overall, they think this could add 0.2 to 3.3 percentage points annually to productivity growth, which would be a productivity boom the likes of which we have not seen for many, many decades. To put together the report, McKinsey analysts looked at 850 occupations as well as 2100 detailed work activities across 47 countries that they say represent more than 80 % of the global workforce. Now, given that type of wide-ranging potential impact, it's no surprise that AI is in the regulatory and policy hot seat. In the US, a new bill has been introduced that would hold AI companies accountable if their tools were used for harmful content.

1:43Specifically, it would say that Section 230 of the 1996 Communications Decency Act would not apply in this case. Section 230 is a controversial provision. For internet freedom advocates, it is one of the most important things in the history of the internet and has been a key piece of why the internet has flourished in such an open and diverse way. Section 230 effectively says that platforms can't be held liable for the things that users on those platforms do. Democrat Senator Richard Blumenthal and Republican Josh Hawley have now introduced a No Section 230 Immunity for AI Act that basically has everything it does right there in the name.

2:16Over in the EU, the European Parliament has voted for the AI Act. This includes things like prohibitions on live facial recognition software, a banning of biometric data scraping, and a review regime before major models are deployed. Now we're going to dig deeper in today's main AI breakdown, but this is a huge deal. It represents one of the first, if not the first, comprehensive AI legislation from a global power, and so is likely to have meaningful influence on other regulatory regimes around the world. Bringing it back to the realm of the commercial, Google has announced another slew of AI features.

2:45Some of these are for advertising, shopping, and commerce. One, for example, is a more sophisticated ad placement tool that allows Google advertisers to automatically have their content placed where it's most likely to drive impact. And virtual try-on is a new way for people to customize how different clothing would look on models of all sorts of various sizes, skin tones, and with different hair types. Google also announced an update for its lens feature, which can help identify skin, lip, nail, and hair conditions. Google says that this is not meant to replace medical diagnosis, But given that the company receives over 10 billion searches annually for skin conditions, this could be something useful for helping people at least understand what they might be working with.

3:24In the research realm, NVIDIA has shared a new text-guided image editing tool. This allows users to take a source image, in the case that we're looking at here, a horse in a field, and then customize it using natural language. So for example, taking that horse and placing it as a pink toy horse on the beach, or as a bronze horse in a museum. This continues the trend of more pinpoint image generation and image modification tools that we're seeing pop up basically everywhere. Now, when it comes to the efficacy of AI-generated content, an interesting update on that front. To understand better how generative AI might impact the fundraising process for businesses, Clarify Capital compared a slate of successful human-created pitch decks, i.e.

4:03companies' decks that had already secured funding, against GPT-4-generated pitch decks. They asked 250 investors and 250 business owners to rate these decks without knowing that AI was involved. Overall, those investors and business owners found that GPT-4 generated decks are two times more convincing than those that are made by people. Those same investors and business owners were three times as likely to want to invest after reading a GPT-4 pitch deck than after reading a human deck. Of the decks created by GPT-4, 77 % were rated as high or very high quality, while only 26 % of human-created decks were rated at that same level.

4:39Still, while these changes show just how much generative AI is working its way into the normal business flow, there are rumblings that we might have jumped the shark when it comes to funding hype. This week, France's Mistral AI announced that they had raised$113 million at a$260 million valuation. Now, what made this notable, aside from the massive dilution that the company was taking, is that this company is only four weeks old. It is currently pre-product. Now, the team does have deep experience in this space, including alums from DeepMind and Meta, and they're playing in the lightning-hot area of enterprise AI development.

5:14Still, anytime people see nine-figure funding rounds for companies that don't have products yet and only just started, it gives them tremors of bubbles past. Finally, one of the more wild stories from this week is that for the past few days, an AI Jesus has been giving advice on a 24-7 Twitch stream. Basically, this is a chatbot version of Jesus created by the Singularity Group that answers basically any type of question users throw at it, from breakup advice to game tips. Now this year, we've also seen other types of AI-generated entertainment, such as the Seinfeld parody show. However, that hasn't always gone well.

5:47Twitch, for example, temporarily banned that Seinfeld show after it started making transphobic remarks, And at the time of recording, this Twitch stream had been taken down for violation of Twitch's terms of service. If nothing else, it's clear that the world is going to get weirder, not normaler. Anyways, that is it for this AI Breakdown Brief. If you're enjoying this, please like, subscribe, and share. And I will be back soon for the main AI Breakdown. Welcome back to the AI Breakdown. On Wednesday, June 14th, the European Parliament advanced a draft of the AI Act, which could become the first comprehensive global legislation when it comes to AI.

6:21Today we're looking at what the act includes, what the response is among industry and other advocacy groups, and what the precedent could mean for other regulation around the world. The first thing to know is that while the generative AI hype has been around for the last seven months or so, of course since the launch of ChatGPT in November, the EU has been working on this AI act since April 2021. That was when the European Commission first proposed the EU regulatory framework and started working on what would become a slate of something like 43 technical meetings, 12 political meetings, to come up with this draft, which has 85 different provisions.

6:56The European Parliament website says Parliament's priority is to make sure that AI systems used in the EU are safe, transparent, traceable, non-discriminatory, and environmentally friendly. AI systems should be overseen by people rather than by automation to prevent harmful outcomes. Now, the approach overall of the AI Act is to have different rules for different risk levels. The categories that they include are those systems that have what they consider unacceptable risk and should be banned, systems that have high risk and so require a different level of consideration, those with limited risk that need a lighter touch, and then a separate set of considerations for generative AI all on its own.

7:32Now, a lot of the discussion so far has been on the things that fall in that category of unacceptable risk. The EU includes in this real time and remote biometric identification systems such as facial recognition, social scoring, which means classifying people based on behavior, socioeconomic status, or personal characteristics, and cognitive behavioral manipulation of people or specific vulnerable groups, an example being voice-activated toys that encourage dangerous behavior in children. Now, in terms of this draft law specifically, there are a few of those technologies that are directly banned.

8:03One of those, for example, is live facial recognition. And even though the European Parliament has voted to ban the use of live facial recognition, there are still discussions that could change that in practice over the next bureaucratic steps around whether there should be exemptions for national security or law enforcement. Another activity that is banned is companies scraping biometric data from social media to build out databases. Now, one of the areas that's contentious has to do with transparency requirements around generative AI. Some of the provisions in this bill would include companies having to publish summaries of the copyrighted material that they use for training their systems, which publishers support because they think it means there could be a path to revenue.

8:39Obviously, right now we're dealing with lawsuits in the UK and the US from Getty Images around Stability AI's supposed use of their private database of images to train their stable diffusion model. AI developers, meanwhile, have said that this is technologically infeasible and also could create another set of problems that the European Union isn't anticipating. This is obviously a different approach than we're seeing in some other places. Japan has been seeming to indicate that they will not enforce copyright when it comes to model training as a way to increase Japanese competitiveness when it comes to AI.

9:09Now, there are some other parts of the generative AI slate of rules that are a little bit more agreed upon or consensus among many people across industry and policymaker sets. Those include things like working to design the model so that they prohibit the generation of illegal content, as well as AI-generated content disclosure so that people know that they're actually interacting with content that was created by AI, not humans as we normally think about it. Now, when it comes to the high-risk activities, a recent research paper called To Be High Risk or Not To Be defined the risk conditions across five concepts that includes domain, such as law enforcement, administration of justice, or critical infrastructure, purpose, in other words, what the AI is meant to do, its capabilities natively from a technology perspective, who the AI user is, and who the subject of the AI is.

9:55For example, one thing that has been determined to be very high risk and thus prohibited in the AI Act is law enforcement using AI tools to predict future criminal behavior minority report style. So you can see here the domain is law enforcement, the purpose of the AI is predicting the occurrence or reoccurrence of criminal offense, the AI capacity is profiling, the user is the law enforcement authority, and the subject is the perpetrator or suspect. Other uses, however, while in the same domain may still be high risk, but not high risk enough to ban. So for example, law enforcement using AI tools to investigate criminal offenses, such as evaluating the reliability of evidence.

10:33That's still high risk, but not so high risk to ban. Instead, it just requires a different type of regime. The European Parliament writes, AI systems that negatively affect safety or fundamental rights will be considered high risk and will be divided into two categories. One, AI systems that are used in products falling under the EU's product safety legislation, including toys, aviation, cars, medical devices and lifts, and two, AI systems falling into eight specific areas that will have to be registered in an EU database. Those include biometric identification and categorization of natural persons, management and operation of critical infrastructure, education and vocational training, employment worker management and access to self-employment, access to and enjoyment of essential private services and public services and benefits, law enforcement, migration, asylum, and border control management, and assistance in legal interpretation and application of the law.

11:21So this is what the EU means when it says that they're taking a risk-based approach. These are riskier activities, so they have more rules around them. Law enforcement won't just have freewheeling access to AI. AI systems that relate to it have to be registered in an EU database and will likely have a different set of disclosure regimes and review processes as well. Now, in addition to just registering these high-risk activities, the EU is also looking towards a set of pre-deployment risk assessment processes that effectively create some formal go-no-go process before advanced or high-risk models enter the public domain.

11:53This isn't so far from something that Sam Altman said in front of Congress that when it comes to advanced models, it makes sense to potentially have a regime for, if not research, for approving when a sufficiently advanced model can actually be released to the public. Sam Altman and OpenAI haven't commented on this version of the EU bill that was just passed, but did get in a little hot water last month when they said that the regulations might be hard to comply with and might be a case of overregulation. Indeed, Altman said that OpenAI might not be able to comply and would actually have to leave, causing quite a dust-up with European politicians before walking it back and saying that OpenAI had no intent to abandon the European Union.

12:30Now at this stage, there still haven't been many comprehensive reactions to it. Amjad Massad, the CEO of Replit, did say it's heartening to see that the latest draft of the EU AI Act has a carve out for open source, and much of it is limiting government usage of AI, such as prohibiting Chinese style social scoring. I'm starting to think that these legislators are largely reasonable. Hot take. However, as Jeremy Howard pointed out, that's not all open source models, and there is a carve out from that carve out for open source foundation models. And Gennaro points out that part of what we're seeing with this AI Act in terms of its shortcomings has to do with how much of it was formalized before the rise of generative AI.

13:07Gennaro writes, this is the proposal of the AI Act, which was formalized in 2021, which is why it's so skewed towards government risks with a couple of adaptations for generative AI. We'll need to see the legislation they'll do on top of it to have a final call on whether it's good or not. Gennaro specifically points to those issues of documenting all of the copyrighted materials from which a model was trained. He argues copyright should be tackled in court, i.e. companies suing OpenAI, not via legislation. And here I guess no foundation model would pass the test. He concludes that's why it's critical always to go beyond the rhetoric of the legislator, where you hear all nice words in two hours of discussion, and at the last minute when discussing practical application, you stumble upon the real pitfalls it can cause.

13:49Our rules are on three levels. First, so if the rules will be enforced today, what CHETGPT will need to do is, number one, mind the interests of those downstream of the value chain to make sure that they pass on the information they need for them to comply if they go into high risk areas. Number two, they'll have to be transparent about the data sets that they used in training CHETGPT, how CHETGPT functions, the basic parameters of the system. And number three, most importantly, the two specific things for generative AI, for example, for chat GPT, i.e. they would have to show that they have, in the course of developing and designing their algorithm, they have seriously considered the lawfulness of the content that might be produced.

14:34Now I have a question mark if they did ask that question when they developed. And number two, they will have to reveal all of the copyrighted material that they use in the training of the algorithm. They have to document it, we say comprehensively document, and be transparent about it. This is what Chagipiti will have to do tomorrow if the legislation will come into force. Thank you. I agree with Gennaro that this could be an extremely problematic provision of this bill. I don't think it would be very beneficial to anyone if all of a sudden, overnight, every LLM and chatbot was brought into noncompliance because of these rules.

15:07Frankly, the gambit there for Europe is that every other country in the world would make similar rules, and so those companies wouldn't have the ability to just abandon Europe because they found those rules too onerous. I think that at this stage, it's far more likely that Europe would effectively just be banning its citizens from ChatGPT, while ChatGPT went off and thrived elsewhere. Now, the good news is that there is still a lot of bureaucratic process before this draft bill is finalized. And more than that, the recent crypto legislation passed by Europe, the Markets in Crypto Assets Framework, or Mika, went through something similar, where a lot of this was drafted in the ICO era and so deals with things like initial coin offerings, even though they're not really a thing anymore, and figured out how to punt until Mika 2 on some really thorny issues of NFTs and DeFi and other more recent advancements.

15:54Now, Europe does have an interest in getting rules on the books as relates to generative AI. I don't think they're going to punt that entirely. However, I also don't think it would necessarily be all that difficult to come to some common sense agreements about the things that everyone can agree on and save more difficult and complex issues such as copyright for a future session. Ultimately, this is a big deal. It's a big step in the first comprehensive AI legislation. Even if it's not copied entirely, it will set precedent for other nations. And the question you have to think for EU citizens is the extent to which this makes Europe a leader in AI versus just a leader in AI regulation.

16:27Of course, if it just makes them a leader in AI regulation, but without the benefits that AI can bring to society, to work, and beyond, it might have a very unintended set of consequences. Obviously, we will continue to monitor it closely. But for now, that is it for today's AI Breakdown. If you're enjoying, please like, subscribe, and share. Check out the podcast and the newsletter version of this. And until next time, peace.

16:57Thank you.