E532 | Eoghan O’Neill, European Commission AI Office: Unpacking the AI Act & Europe's Global AI Play

0:00You know the saying, the US innovates, China replicates and Europe regulates, but in AI, is that still true? To explore that, today on the EUVC podcast, we have Owen O 'Neill, Senior Policy Officer at the European Commission's AI office, to shed light on the act's nuances and its impact on European innovation. We explore the EU AI Act. Europe's bold move to shape AI responsibly? Or is it just another bureaucratic overreach? First thing to keep in mind with the AI Act is it's not competition law. It's a product safety regulation. We break down why it's closer to medical devices or escalators than a digital clampdown.

0:35If you're a startup in that space, the first question to address is, are we in the prohibited systems game? If you are, you need to make sure you're not releasing anything in the EU because it's now prohibited under the Act. How Europe's pulling startups into the process and why most won't even hit the high-risk rules. We also asked the hard questions about regulatory capture, comparisons to GDPR, and whether Europe is stifling or enabling innovation. In a simple sentence, the AI Act can deliver for wider society trust in AI-driven products and services, and that can drive market uptake. For the founders who navigate it early, this isn't just compliance.

1:13It can also be an edge. Access to that compute power, free, no charge, gratis. Europe is a player here. Tune in now for the real story on AI landscape in Europe. Explore whether Europe is regulating or leading. with Andreas and Owen on the EUVC podcast.

1:32Before we start the show, a quick note. If you're building or running a fund, you know it takes the right partners. At EUVC, we only work with sponsors we truly believe should be part of your tech stack. Please do take a moment to hear about them. And if you do, reach out, mention EUVC. It's the best way you can support what we do. Thank you so much. Starting off, HSBC Innovation Banking. If you're a founder, a scale-up or a VC, you need a bank that actually understands your world. HSBC Innovation Banking backs innovation globally from seed to IPO. And if you ask me, a strong banking partner like HSBC belongs in your stack.

2:08If your portfolio companies are scaling, they need infrastructure that won't slow them down. Google Cloud Starter Program offers$2 ,000 to$350 ,000 in credits, plus technical support to build better and faster. It's a key boost every fund should bring into their ecosystem and oh my god are we thankful to be partnering with them. Now legal is a space you cannot lag on. Legal needs to move at the speed of venture. Goodwin's team has decades of experience with startups and funds. They're trusted at every stage from formation to exit. Goodwin definitely is a legal partner every series manager should have in their stack.

2:43For Luxembourg based VC, PE and fund of fund managers, modern funds means going digital. Funcrafts gives you a full service, digital native platform built for today's European managers. It's a must have if you're scaling smart. So we all hear about the Middle East. How about you go there? From AI to deep tech to summer funds, Guy Techs in Dubai is where global future of tech gets negotiated. It's not just a conference, it's where East meets West, capital meets innovation and the bolts set the agenda. If you're playing on the global stage, join us going to Guy Techs this year. If you're gearing up for your next fundraiser and want a placement agent who truly understands emerging managers reach out to cfunds their boutique placement agency that has helped gps across here brace capital from top tier lps we've been on the other side of the table here they are actually good ones to work with so i do urge you to go to cfunds.io to go and check them out and hey before you go if you're looking to discover startups raise capital or connect with innovation leaders do check out dealflow.eu the eu-backed platform bridging founders vcs and corporates.

3:44There's no better place to find the startups that have received significant funding from the European innovation ecosystem.

4:03This show is not investment advice, and the hosts of this episode may be invested in the funds and companies featured. Welcome back, everyone, to another episode of the EUVC podcast. As you know, we try and connect and champion European venture. And today we're diving into a topic that's been stirring conversations across the continent, the EU AI Act. With Europe's ambition to lead in trustworthy AI, understanding the implications of this regulation is absolutely crucial for all the startups and investors, as well as the broader ecosystem out there. Joining us is Owen O 'Neill, Senior Policy Officer at the European Commission's AI Office, to shed light on the act's nuances and its impact on European innovation.

4:45Owen, welcome to the pod. Andreas, delighted to be here. Greetings from the austere offices of the AI office here in Brussels. No pool tables, no ping pong tables, no juice bars to distract us from the important work we have to do here. But we do actually get some very nice headphones, as you can see. And I can't see, I don't know if you can see, there's a very nice red trim hidden away there inside the matte black. So we do get some treats. to do our work. I can tell you I've done podcast episodes with people in more or steered places. Places like basements are definitely a go-to for many VCs.

5:24So, Owen, let's get into it. Let's start a bit with your background so everyone understands why you lined it up at the European Commission working on AI Act. Well, I joined the AI office of the Commission was established almost a year ago to the day, middle of June last year. And I joined it from the very, very beginning. I moved to it. I was previously on the enforcement team for the EU's Digital Markets Act, Digital Services Act. But I've been around the commission, working here for over 20 years. If we go way back, I'm an engineer. I did civil engineering in Trinity College, Dublin. Started out my career in, of all places, working in the pet food industry.

6:07I was a king in the world of designing gravy systems for pet food. on this continent many years ago. But then I ended up moving from, in a non-linear fashion, from the pet food industry to the IT industry, working for one of the big companies before joining the commission. And all of the time in the commission, I've been working on the digital side, digital research in innovation programs, innovation policy, then the regulatory side, the DMA, the DSA, and now the AI office, where we're doing, yes, regulatory stuff, but also non-regulatory stuff. But I'm sure we'll get to talk about it. Andreas?

6:44Owen, you have done what few have ever done, which is go towards the more sexy space of AI regulation. Not a lot of people go to regulation and end up in a more sexy place than they were before, but I guess designing pet food systems counts as less sexy than regulation. Don't underestimate the attractiveness of the pet food industry. That's all I'll say, okay? Yes, but there are some VCs that make real money there. Let's get into it. Let's talk about the mission and the responsibilities of the AI office, just so everyone fully understands why it's such a central body for us. The AI office was set up to bring together a number of different teams we had in the commission looking at AI, looking at the regulatory side, looking at the funding of research and innovation, looking at the infrastructure side.

7:38We had teams involved in the financing and strategies for supercomputing facilities across the EU. All of this was rolled together into the AI office starting a year ago so that we would be working together in terms of driving the AI in Europe in terms of its trustworthiness, its use, and advancing Europe's capabilities here instead of us working in different silos. but yet started one year ago. I think everyone understands the importance of AI. Just before we go into the act, let me just give a huge shout out to Thais at DealFlowEU for connecting us because he was the guy who said, Andreas, I think you should talk to Owen.

8:22So Thais, if you're out there listening, I hope you are. Thank you so much for this introduction. Now let's go to the AI Act. I'd love to ask you, could you just outline the primary goals of the EU AI Act and how it aims to balance innovation with regulation? First thing to keep in mind with the AI Act is it's not competition law. It's a product safety regulation. And product safety regulations are something that are well established in modern developed economies. They underpin some of the marvels of modern living in a modern developed economy. the aviation industry, medical devices, escalators in metro systems, high-speed rail, food production.

9:08All of these tech-driven services and products are being used en masse because of product safety regulations. There is trust in these products, in these services. So product safety regulations are a feature, not a bug, of modern developed economies. And they're non-controversial. There's no pushback in these sectors saying these product safety regulations aren't helpful. They're widely accepted. And what the EU has decided audaciously is, well, this proven approach for bringing new cutting edge tech driven products and services, we're actually going to also use that for AI driven products and services.

9:54It worked in other areas, this product safety regulation approach, and that's exactly what we're bringing now in the EU through the AI Act, a product safety regulation for the AI era. How would you though, because there's a lot of discussion in this area and of course, a big critique from many, which say that AI is just not at a place where you can yet regulate it without creating a bunch of over-regulation that's not necessary at this stage. Right now, we just got to let it run. Well, let's keep in mind of one thing. AI is now being used at scale by the masses. This is no longer a fringe technology being used on the edge of society by some early adopters.

10:45AI is in literally the pocket of almost every citizen in the developed world in some form or another. We're not at the, you know, the Orville Wright, the early days of the aviation industry back on a beach in Carolina at the beginning of the last century. We're now at a moment where probably the order of a billion or more people have access to AI and the capabilities of AI models and systems continue to grow. And I think it is broadly accepted this is not a zero risk technology, just as flying in an airplane is not a zero risk proposition or interacting with a medical device or even an escalator in a metro station.

11:33And the EU, based on its experience with product service regulations, has decided we're going to regulate in this space. But we will do so in very close cooperation with the key stakeholders in shaping the regulation, how it is enforced, how it is implemented. This is a key feature of how the AI Act is going to impact in the world. You shared a good story last time we spoke on this very involving process or open process of creating the AI Act. So maybe you can talk a bit about how you've structured that, the parties that has been involved and so on. Yeah, if you take the AI Act, you break its focus down into two populations.

12:21One population being generative AI model providers and the other population being those developing AI systems, which could be downstream making use of AI models. Let's focus for a moment on that first population, the providers of generative AI models. They will be the first community who will have compliance obligations under the AI Act. It's in August, August this year, their compliance obligations kick in. The question was, how do we go about implementing the act for those providers? What the commission did was, last year we announced, we called, we created a code of practice working group, open call for stakeholders to apply to join.

13:05We have about 1 ,000 individuals who are in this code of practice group. And what is this code of practice group doing? It is developing a code of practice to be used by providers of general purpose AI systems, gen AI systems, models, should I say. We have everybody, the full spectrum of stakeholders are there shaping this code of practice. First key thing to mention is the commission is facilitating this. We, my colleagues here, we are not authoring the code of practice. We used our convening powers to bring together the community of stakeholders. And among these 1 ,000, we have the full spectrum.

13:47Yes, we've got the antropics, the mistrials, the open AIs, the metas of the world who are delivering models at scale. We've also got downstream users of these models, SMEs, startups. We've got academia. We've got global AI safety experts. We've got lawyers. We've got the full, we've got NGOs, you've got civic society representatives and organizations. We've everybody on board. Now, it's broken down into four working groups where the chairs of those working groups are globally reputed, respected figures of consequence who are chairing those four working groups. So this is a, if I do say so with institutional immodesty, I put this forward as being an excellent example of collaborative work to support a regulator in enforcing a regulation in a way that is helpful to the wide community of stakeholders, including the technology providers.

14:51And it's going really well. The chairs of one of the working groups, the chair of the test risk mitigation working group, Joshua Bengio, who is a during prize award winner. He only about three weeks ago on the public record said that he considered this code of practice approach to be exemplary. It's bringing in voices from every corner of the community to come up with a code of practice that then can facilitate the commission in enforcing the AI Act in a way that is helpful to this wide community. And it should be concluding soon. We're expecting probably in July, the fourth and final draft of the Code of Practice.

15:34This is all in the public domain. The previous three drafts are out there, accessible to the public. I can see the direction this is taking. The chairs asked the commission if they could have more time for drafting. We were happy to say, sure, yes, take more time. Let's do our best. Let's help you, give you what you need to try and do as good a job to bring all of these voices on board with a text that can be accepted by the community of a thousand people. You always, yeah, now you mentioned maybe a magical number there, because one of the critiques or risks of running an open process, so to say, where you get input is that the places where you get most input are from the stronger organizations, which then leads to regulatory capture because the only people that have the means to influence are those that have good lobbyists and good people working on this type of thing.

16:26And someone building something in a garage will, by nature, not have their focus there, meaning that startups get squeezed out. How have you ensured that the startup community is properly listened to in this process? First of all, we facilitate. We're not in there micromanaging the discussions and the exchanges. We did make sure that the working group leads were strong figures who would be able to bring balance to all of the voices that are coming that would be able to take on board the considerations and the points being raised by the wide community we're not in there micromanaging but we have we made this open we made it very broad in order to bring in the voices of not just big players but smaller players to the chairs recognize the need for more time and we as i to repeat what i said earlier, we were happy to do that, to let them have that more time.

17:22Let's go to the impact on startups here. If I ask you the question, what practical steps startups that are developing AI-powered solutions should take to comply with the AI Act? Okay. The first thing is to try and establish if the AI Act is actually relevant to your business. The main provisions of the Act do not need to be addressed or to be obliged, complied with by all actors in this space. Let's take that second population. We discussed a few moments ago the general purpose AI developers and the court of practice. Now let's move to the second population, those developing AI systems. If you're a startup in that space, the first question to address is, are we in the prohibited systems game?

18:16That's scraped facial recognition data, social scoring, biometric categorization, crime prediction. If you are, you need to make sure you're not releasing anything in the EU because it's now prohibited under the Act. So let's assume you're not in that space. Excellent. Next question is, are you offering solutions in any of the high-risk categories for AI that are in the AI Act? These are AI systems in terms of biometrics. It's all listed in the Act, but I'll share it with you now. Biometrics, critical infrastructure, AI solutions for education, such as evaluating students, the admission of students into universities or the monitoring of students, employment, such as in the recruitment game, AI for law enforcement is a high risk, as well as AI for migration, asylum, board and control management, and finally, AI for the administration of justice.

19:12If your offering is in that space, it's still perfectly legal to launch it, acceptable in the EU, but you do have compliance obligations. Those are obligations. Let me just ask you, because you gave an education example there where you said it was monitoring students or it was the admission process. What if I offer a training or a teaching solution using AI, basically an AI tutor, would that count as an educational tool that falls under the AI? I'm going to dodge your question. I'm not going to preempt here. All right, but hang on a moment. What we do have, and I go back to our fundamental approach here with the AI, is to be open and to solicit inputs to help us with our enforcement of this.

20:03As we sit here talking to each other, we have open at the moment until I think it is next week, a public consultation on high risk systems, on how to classify them, seeking from the community of innovators out there examples of AI systems in these areas and to help us with the possible classification of high risk systems in these different categories. So we're not just sort of deciding here in our austere offices on our own on what could be classified under these. We're also engaging with the community of innovators. So I don't want to preempt that process. We have the categories. We want to understand better what will be considered as something that falls under them.

20:51That process is underway. So just because you're delivering a service in education doesn't make you in the high-risk category. It's specific items within… Indeed. And following on the consultation we're having at the moment, we are going to be issuing guidelines. Guidelines that will make it much easier for me or anybody else to answer the very question you posed. We're not there yet, Andreas. It wasn't planned that we would be there at this point in time. It's one of the things we're working on. We will be issuing guidelines. In fact, I know the timeline. February next year, which is a full six months before the compliance deadline obligations kick in, we will be issuing the guidelines so that there is clarity and understanding among innovators as to what would be considered high risk and what would not.

21:41It's coming. Hang in there. Can I ask you a question? And it's a potentially provocative one or it potentially troublesome one for you to answer because it touches on other parts than what you are working with yourself. So you end up commenting potentially on what some other people are working with. But GDPR is something that I think many in the European Union would say, we did a good job there. It really like that insured privacy. We're at a better place today than we were before. However, there's a lot of startups that say there's a lot of innovation that we're not able to do because GDPR is blocking us.

22:21Just because compliance, making sure that we are compliant, makes it abundantly difficult for us compared to if you're building outside of the EU. And I don't want you to comment on this. I'm just saying that this is a situation and an observation by many in our ecosystem. What steps are you taking to ensure and do you feel confident that you will be able to ensure that, as an example, we don't lay waste to everything inside education, inside employment, inside. Now you said law enforcement, but I imagine that that also will in some way include or encompass anything in the legal space. So I'd be very curious to hear how do you make sure and what is your confidence level that we don't end up in a similar situation where the EU considers we've done everyone a favor here.

23:13We're making sure that the AI products are safe. But in fact, the startup ecosystem is saying we're experiencing the opposite. Okay. With GDPR, it's not something I'm working on. It's not part of my responsibilities here. What I will say about GDPR is, yes, it has been a big success because it encodes values of the European Union in legislation. It gives agency to citizens of the Union in terms of their private data. It's also something that's been a big success in terms of it has been copied by very serious other global economies and societies out there. South Korea, Canada, to name a few, Norway, Iceland, have drawn direct inspiration from the GDPR despite the criticisms that come up to it from some corners.

24:03Implementation of GDPR, is it optimized? No, I don't think it is. And does that have an impact on, for example, startups, as you say? Yes, it does. But there is recognition that we should hear institutionally that we need to look at GDPR and see what we can do. I don't want to preempt where that may lead, but what I do want to reassure you and the listeners is that there is recognition that it also has an impact, not necessarily positive, on some important stakeholders and actors out there. With the AI Act, we are from day one engaging very actively with communities about what it means, consulting with them, trying to get this right.

24:47But I also think for the vast majority of startups out there, I do not think they will fall into, for example, the high risk category. And therefore, their obligations under the AI Act would be actually quite minimal. They would have obligations, for example, in terms of transparency. Which upstream AI model are we using? The transparency requirements, what they would need will be provided from what is decided in the code of practice. we're working to try and make this easier and less burdensome from day one, not as an afterthought. It's baked into our approach. You know, even just you and I talking today, this is not happening by accident.

25:31This is part of an effort to engage with the wider community and raise awareness, understanding, and provoke thoughts, reflections, and ultimately questions from this community about what we are doing. I understand that the AI Act is first and foremost intended to affect the very big systems and where there's real things at risk, so to say. As you say, it's a safety provision. That said, as an example, in the U.S. or in the debates around how do we build AI, we saw two very different approaches with Grok on the one hand, that just says, no waiting. You can say whatever you want. You can get the AI to say whatever you want, answer any questions that you want.

26:19And then you had the mess up with the founding fathers being all portrayed as being minorities in the first rollout of Gemini. And because I understand that some of the critique from those that are, I don't know if we should call it AI purists or at least very liberal AI thinkers say, well, we should be able to have AI do almost anything. And that's, of course, being championed by the team behind Grok. And you have others that say, well, we need to take a more responsible approach. How do you ensure that the AI Act does not embed what some people call biases or specific viewpoints by a group of elites today that might in the future be considered, as an example, what some people would say is we've had that with the woke mentality, which three years ago, everyone would have expected everything to be very locked down in terms of what you can do and what you can say and have the AI do and say.

27:26And today it looks like we're coming towards a very different point of agreement. Do you see the AI Act in any way forcing the model developers into taking a specific stance on this? Now, the AI Act, it comes in, and I'm proud to say this, at about 90 ,000 words. It's an absolute huge piece of legislation, which I think is necessary given the sophistication of the technology that we're trying to bring some product safety regulation to. I am not aware of anything in the AI Act that goes in the direction of UR in terms of what is right, what is wrong inside, for example, in terms of training data and biases that may emerge from a model.

28:12The code of practice, I don't want to preempt what comes. I don't know what's going to come out of that. As I say, we're not micromanaging. There are working groups there on the internal risk management and governance of general purpose AI providers. That's one of the working groups in the code of practice. Let's see what comes out of their consultative process in terms of those models and how that could be relevant to the point you're raising here. But I am not aware of any provision in the AI Act that goes in the direction of, for want of another expression, thought police. What we want is we want a product in the AI world that will not cause harm to citizens in the EU.

28:59Just like product safety regulations for escalators, we do not want people to lose their toes when they're going to get the metro. That's the fundamental philosophical core behind the AI Act. And the difficult thing is, of course, that with AI being artificial intelligence, if you police the AI, do you not, by definition, police thought, at least by the machine? I think that it's, I definitely get that it's not the intention. And I think that we all agree that at least you and I agree where we should land on this spectrum, but it is one of the big things that people are afraid of, of course. One point I will make though is, I mean, the AI Act and its implementation is not going, it's blindingly obvious what I'm about to say.

29:51It's the way of the world in terms of legislation and regulations coming out of democratic societies is a compromise to compromise from our legislative process and that's a feature not a bug that we managed to get a regulation that is acceptable to almost everybody to some degree and now we're taking it one step further through the collaborative process of the code of the code of practice to come up with a way to implement this that is going to be satisfying, sufficiently satisfying to most actors. There will always be people who will not be happy. That's the name of the game. It's impossible to get everybody fully on board, but we're doing our very best to get as many voices actively involved in the shaping of what we are doing.