Johannes Lenhard: AI Governance

0:00We have an idea of as ecosystems where we should be aiming, but the kind of move fast, breaking things, former mentality of Silicon Valley for the last two, three, four years since the release of ChatGPT has caught us to an extent off guard and pulled us again in a direction that is dangerous. And that's where we very actively work against trying to show people as lucrative, but much greener, nicer future for all of us with Responsible AI decor.

0:40Welcome to the Future in Sound podcast. I'm your host, Jen Wilson. This is a podcast where we discuss people, planet, and profit. In each episode, we'll learn from world-leading experts who can help us see the future we want and our role in it.

1:06This is episode 49, AI Governance.

1:15Dr. Johannes Lenhard is the CEO of Reframe Venture, a nonprofit working on integrating sustainability across the practices of venture capital investors and their limited partners globally. Reframe Venture works with 200 VC and PE funds and 120 LPs on advancing meaningful responsible investment integration. He is also an affiliated lecturer at the University of Cambridge, where he earned his PhD. He is the co-author of three books on homelessness and venture capital. He also contributes regularly to journalistic outlets such as the FT Sifted and Impact Alpha. Johannes, it is so good to have you on the Future in Sound podcast.

1:58Welcome. Thanks for having me. Looking forward to chatting.

2:07So I've just shared a bio that's sort of, you know, on the website. It's how some from the outside describe you. How do you describe your background and your role at the moment at Reframe? I think there are still a variety of ways of how to think about what I do, given that I speak to a lot of very, very professional asset owners and private equity folks, and to the first-time VC fund manager who's really an entrepreneur. it still changes a lot depending on who I'm speaking to. But I think a uniting descriptor is I'm a kindergarten teacher. We try to play, or actually even more so, we try to help everyone play with each other.

2:45Everyone likes us. Everyone wants to be in the room that we organize. And then we just make sure that people are safer and more sustainable and more responsible playing in this ecosystem that happens to be early stage tech investing. And that's still connected to my former full-time job, which was teaching and doing research at Cambridge. So there's a through line, even though it doesn't sound like it, when you read out the bio that is ultimately about helping people do something slightly better. And this particular iteration of my life, that's investing, investing in tech specifically. That's great.

3:24And you work with hundreds of venture and growth equity executives at Reframe. When it comes to AI risk, safety, and governance, where would you say that we are as an industry? You're jumping right into AI, the topic that everyone wants to talk about, only perhaps rivaled by defense. AI, yeah. So I think one very important contextualization is the world actually knows what good and bad AI looks like. And the world has known that for decades because they're very, very, very smart people, way smarter than most of the investors I've encountered and us at the best universities in the world who have worked on making AI and at the same moment worked on making good AI or responsible AI or safe AI for at least the last 50 years.

4:10So as a world, in the nooks and crannies of the libraries and research institutions that make up part of this world, and we have a very, very good understanding of what we should be building and what we should not be building and how we should do this. As an investment ecosystem, and especially as a, let's say, epitomized by Silicon Valley early stage VC investment ecosystem, we are unwilling to do some of the work of digging that up. fortunately you encounter some entrepreneurs like the folks at Anthropic or some other folks in the European ecosystem trying to build companies in the AI space that say no actually we're researchers at heart we do want to build stuff full stop but we also want to build stuff that is good and that isn't repeating the same mistakes that the social media companies have kind of have as have hazardlessly brought into the world that wasn't built by researchers at heart right Mark Zuckerberg is a lot of things, but he's definitely not a researcher at heart, right?

5:10He was a small kid that was interested in meeting women. And that's how Facebook came about, right? The generation of AI founders that we're encountering today, I think many, many of them are very well-educated, smart, and research-driven. And I think the epitome of that is anthropic. And they are at least saying, and I can't check this, that they are building with more responsible principles and practices at heart. And I think that's very important because the stuff that gets built now just gets multiplied into the kind of foreseeable future, right? I think we're going to force similar path dependency as we are with Google still, right?

5:48Once you're sucked into one model, you're going to continue using the model. And that takes us again back to the VC ecosystem. I think the VCs, there's perhaps two things I want to say. A, it's not their draw up to be very, very, very, very deep in the responsible AI. by piece because their job is first and foremost to make money. But if an LP comes to them and says, why would I give you money? The answer has to be because I'm going to make more of it. That is still a very blunt version of describing what we see to their asset managers and PE even more so. And they have to do this in the PE space, especially reliably.

6:24So first and foremost, they have to find investment opportunities. So they are dependent on the entrepreneurs telling them in turn another story. And if you're then meeting entrepreneurs like the anthropic folks who say, look, I can make more money by building responsibly in AI. They're like, oh, that sounds great. Because some of the LPs I'm encountering, they're like, oh, can you make lots of money but not destroy us in the meantime? And the VC is like, yeah, I mean, I can do the first. I don't know about the second, right? Because I just don't know. So that's an easy excuse for the VCs, which I obviously fundamentally disagree with.

7:00But on paper, that's their job. They don't have to think about not destroying the world. But as long as they made money, nobody can fault them. I did my job. Don't kill me. I made you more money. That's what you hired me for. Now, what we are fortunately seeing is that some bits of the ecosystem, perhaps a little bit too slowly for our desire, but some bits of the ecosystem, including some of the biggest asset owners, and I think that's an important piece of the kind of understanding of the hierarchy of where we're operating and the asset owners really hold the power because they can say to the asset managers, so the kind of Stepstones, Cambridge Associates of this world, where they want their owned assets to go and then together they can say to the other asset managers, the PEs and the VCs, how that should look like.

7:48So some of the largest asset owners, say in California or in the Netherlands or among some of the European government LPs, they understand, A, their constituents, which are taxpayer, pension pairs, they don't want them to invest in something that destroys the world or even makes the world a worse place. And that's actually bad for their bottom line because they think in 100-year chunks and not in one-quarter chunks. And if I'm the biggest of seven biggest pension funders of the world, I'm going to think about the next 25 years. And if then a VC promises me, oh, yeah, we're going to invest in AI and that AI is going to just make workers obsolete, I have a real problem.

8:28Because suddenly I'm like, oh, but that's a bigger issue than you losing all my money because suddenly I don't have any people paying into their pension anymore. And that means I don't get money anymore. Right. And that is where we're seeing some dots connecting now, where some of the biggest LPs and the most powerful LPs are coming up with concrete ways of tasking their GPs, PE and venture alike, of investing in AI that is better, responsible, whatever words, adjectives we want to use. that is only slowly starting. And again, that's where my impatience comes in. I think we know where we want some of them to go and not to go, I think very importantly, but the pressure hasn't been building enough.

9:08So a very long answer to your question, which is we have an idea of as ecosystems where we should be aiming, but the kind of move fast, breaking things, former mentality of Silicon Valley for the last two, three, four years since the release of ChatGPT has caught us to an extent off guard and pulled us again in a direction that is dangerous. And that's where we very actively work against trying to show people as lucrative, but much greener, nicer future for all of us with responsible AI at the core. And again, the asset owners are starting to really, really, not listen to us. I don't take ourselves that seriously, but feel the same.

9:52Before I get into what's being missed and where you're feeling impatient and wanting to see more progress, in terms of where we are now, for the leading and larger LPs that are connecting the dots between their business model, their returns, and responsible or safe AI, what are they asking for? What are the best of the best right now asking their GPs to show, demonstrate, etc.? I think you have to start one level before. What have they been doing is the first question. I think the LPs that have been thinking about this in the most clever and leaning in way have said, I'm not going to write 150, whatever, 500 million checks into an AI company if I do not understand what the risks are with this company.

10:40And I don't right now. So I'm going to task my research team to come up with what does responsible AI mean for us and what does that translate into when it comes to guidelines for both investment decision-making for an LP to the GP level and for perhaps also guidelines for our GPs. So I know of at least two or three big LPs, both asset managers and asset owners in this case, who have done this step. So there is a much better understanding in-house, and I think that is very important. How are you going to ask good questions or you just copy them from somebody, but what do you do with the answers?

11:14If you don't have the expertise in-house, you can ask whatever you want. you can make them report on whatever you think might be important or whatever anyone else has said to you is important. You need to then be able to scrutinize the answers. So I actually think the first step starts before, and we are still there, very much so, Jen, right? We're not, oh, everyone has done their homework, which is, I think, the stage we're at. We now know where our boundaries are, where our red lines are, and what we're looking for, what the good stuff looks like when it comes to perhaps impactful AI. That's only starting now.

11:46And then the first pieces that we're seeing LPs push into the ecosystem are about reporting. There's an initiative that we were part of in the continental European context to get the big government LPs to harmonize their reporting requirements across all sustainability channels. Last year for the first time, they included some very high level responsible AI questions. And we advised them to keep them very high level because we didn't know and we included didn't know what good looked like so it's more a tell us what you are doing when it comes to responsible ai question than it is here is seven kpis that you have to fill in precisely because we wouldn't know what the kpis are and even if we did we wouldn't know what good looked like so that's the starting point and then i think the best lps now are able to have conversations we have gps that claim we are ai I first, we only do AI investment and poke holes in the right directions, again, aligned with what they're thinking about, right?

12:43If I'm a pension fund, I'm going to talk to these people about worker replacement, right? But that requires me to know something about that, right? And that takes me back to this homework phase, which we're seeing the best going through right now. So before there's a noticeable real impact across market segments, not only by the top, whatever, 1 % of LPs, I think it'll be another 6 to 12 months. I have two LP-only Responsible AI trainings, one in New York, one in Amsterdam in my calendar within that timeframe. And I think that for me is a good indication of who's moving, how quickly are we moving, and what direction are we moving in.

13:20I appreciate this point about applying principles ourselves before asking other organizations to apply. It's really interesting. at Ricoh, we were doing a responsible AI, you know, training at the moment. And when we're looking at dealing with confidential information, if we didn't go through a process of, I think we're on our like fifth confidential information training linked to AI now, and, you know, have pretty strict requirements around, you know, which third party AI tools, like how we vet them, how we release the ability to use enterprise models, etc. If we hadn't done that, it's really hard to put a bullet point on a slide and make it come to life.

14:04And to have a conversation with a GP around confidential information or, you know, data rights in particular. So I really appreciate this point about, you know, all organizations grappling with similar questions and fostering conversation versus artificially trying to come up with KPIs too early that don't fit the need over time. And I wanted to pull on the thread of impatience. So Johannes, you were saying earlier that there's this feeling a bit impatient at the pace. And I'm wondering from your perspective, what's being missed when you look at AI risk, safety and governance right now in the PE and venture community?

14:42Like, what are the big areas that leaders need to be challenging themselves to go deeper now? Oh, I think you need to start one step before that. Most of the folks think big name venture, and I would think lots of big name PE2. They don't even know that they're missing something, right? That's always the fundamental beginner's mistake. And then the LPs are like, oh, yeah, but I can't tell what to do and not to do. because if I ask them any questions, they throw me out of their allocations. That was the answer we often got across sustainability issues over the last five years. That has changed a little bit.

15:19Venture capital fundraising is hard. Even the big guys have been struggling to raise their AUM targets. I think the LPs are seeing, especially the big asset owners that have come down a little bit more to write large direct checks themselves, they are seeing the opportunity and they can actually and they do ask questions. but I think a lot of it is still we haven't even really started to think about what responsible AI means and and how I integrate that into my say due diligence decision making right it's one thing and everyone is jumping in this direction right now to talk about AI governance AI governance on the portfolio company level is really something that you're likely building after the first one two three, four, five rounds of investment were raised, right?

16:02Especially if we're talking AI university spin-out type thing. The first level of governance that we are seeing through our GPs is very, very low. They don't even have a board, right? No, they're not going to do an AI audit because there's no AI yet, right? There's no product yet, right? So the kind of governance conversation, which lots of LPs are jumping on now, will start a little later. So the GPs can scrutinize the data set, for instance, right, which again would require technical expertise, which many GPs don't have. So I think the kind of understanding of the fundamental principles, which if you're looking at the NIST guidelines, for instance, to use something very neutral, because it's American-made and scientific, rather than say the EUAI Act, which is actually fundamentally aligned with the NIST guidelines, but raises alarm bells because it's called the EUAI Act, and there's clearly some Brussels overreaching regulators behind it.

16:52Not the case. I mean, yes, the case, but still aligned with the American ideas of what good responsible AI looks like. It's really about building a trustworthy AI system, right? And that's a way that lots of people can fill with different meanings. And I think that is the job that first both the entrepreneurs need to do and then the GPs need to scrutinize. And it does remind me of a similar type of conversation that we've been having with the defense FOMO wave investors. because there was this FOMA, and this is even, even, even stronger in AI, obviously, and people have just stopped asking questions.

17:27And I'm like, but that's your job, my friend. That's your job as an LP. That's your only job, really. And that's the only thing you can do as an LP and as a GP down the value chain. That's what we need to take back. No, it is not about just getting into the round, whatever the current hyped round is. It's about getting into the right round and still upholding your due diligence. And again, I've been starting to inquire more and more LPs who write direct checks also in companies who say, I was offered a slice of, I mean, you name it, whatever big AI company that's about to IPO. And I said, no, knowing well that there is a financial, massive financial opportunity there, but I wasn't comfortable with our internal understanding of the risks.

18:09right so i wouldn't want to pinpoint at any subject area whether that's the whole worker replacement issue which is a big issue for the lps which the gps will think about very very little i think unless pushed by the lps because it's that level of complex and long term we have good data on that for instance coming out of stanford but that's really a longer piece of research that we will learn more about as this ecosystem will develop. And so I wouldn't want to pinpoint at any of these substantive issue areas. I would more pinpoint at how many GPs you encounter have an adequate set of questions they can pull from in their due diligence when it comes to specific responsible AI slash AI questions.

18:54You don't even have to call it responsible AI. And the answer is perhaps 10%, perhaps 20%. And that is a point of almost structure and process for an LP to say, no, I need you to have a defense-specific, an AI-specific, like you would ask different questions from a SaaS company than from a climate tech company, set of questions you're asking in the D. Why would you think this is a generic set of SaaS questions? There's totally different underlying value drivers, one of them being data, right? That requires you to understand the business model, meaning asking different questions. It's clear that the starting point then is having a wider penetration amongst LPs and GPs on the why for safe, responsible AI governance, and also having a clear set of nuanced questions that help to assess risk and the mitigation management of those risks.

19:50A shout out to you and your team, Johannes, Oliver, and others who pulled together the AI governance due diligence questions, which are very thoughtful. We'll include a link in the show notes to that particular resource. It's really helpful and very practical outlining what good looks like and what not so great answers look like as well. So we'll make sure to promote that. But it sounds like that's the first step of understanding why this is important and then starting to integrate more thoughtful, provocative, clear questions into the due diligence process. And I think the why is important because there is kind of this abstract understanding that is relatively easy to nurture, whether that's coming from an LP perspective or because I'm actually research driven and I read AI safety folks why I should be doing this.

20:40I think because venture in particular and PE slightly lesser but still is very gut driven and very full of emotions and there is FOMO is a real driving force and who is leading the round is a real driving force. And there is sometimes at least time constraints. So sometimes the rational stuff of why I should be asking these questions just disappears until you hit an actual concrete experiential point where the why becomes something that you can almost feel and touch. it'll stay too far away from actually having an impact, right? And I think one big, big role that a part of the ecosystem can play that we haven't touched on at all is the corporates.

21:19And we're starting to see about the potential role that they do play, very abstractly speaking, if you're mapping Anthropics enterprise sales and OpenAI enterprise sales, Anthropic marketing themselves as a WeBuild AI responsibly, and we're also releasing it responsibly in OpenAI basically saying, we build whatever anyone ever wants, whether that's porn AI, or atomic energy AI, they have been struggling selling to corporates because the corporates do have more boundaries. And they also, I mean, honestly, have the firepower internally to develop what they need and what they want and beyond compliance, right?

21:54We're doing a call in a couple of weeks on response to AI from a corporate perspective. It was complicated to find people to talk about it publicly, semi-publicly, but we do know, and they're all talking about this in one-on-one calls, that they actually have very strict guardrails and very strict DDs that they do for the actual products. And as a VC, I think we are always one level removed from the actual sale of the thing that our entrepreneur was very provocative and knows how to sell a story to us as an investor when they're encountering pushback. And I think the big pushback that is the pushback you need to overcome in order to be successful financially in the AI space is selling to corporates.

22:36and that voice can be a voice of reason for once at least because they have such big machines and so many things can go wrong that they're going to be very likely the first and again we have started to see this the first pushback on irresponsibly AI if you want to call it that. We simply cannot deploy this internally. We can dabble around with ChatGPT and actually we're going to stop this now because we know that this is, as you just said, we know that this can go wrong But in order for this to be used, client-facing, to develop real stuff that contributes to products that we're then selling, it needs to be tight.

23:13And it kind of almost needs to be our own, given the current standard of products that you guys have been better. But that's a very good corrective force.

23:34value through sustainability. That might mean identifying where sustainability improvements can drive measurable returns, helping you set the right targets, or making sure sustainability sits at the heart of your value creation strategy. We also work on some of the more complex challenges our clients are navigating right now, including how to approach AI responsibly as it becomes part of how we all operate. Ultimately, we help you do sustainability in a way that actually matters for your business and your investors. If that sounds interesting, come find us at re.co.com. All right, back to the conversation.

24:13I just want to say that one of the things that we see at RICO, and I know this is something that you and I have talked about, Johannes, is there are so many connections between value creation, operational excellence, etc., and responsible AI. just the data around you know customers being willing to pay significantly more if they know that their data is going to be how their data is going to be used and that's it if they know how their data is going to be used they are willing to potentially leave for a more transparent option or potentially you know leave an AI enabled platform if they don't know how their data is being used you know over time thinking about if there's the energy angle I know that right now the name of the game is adopting as fast as you possibly can on AI, but the prices are increasing on these models, particularly the frontier models.

25:08And thinking through, okay, well, which model for which task, this kind of thing, not only could save energy, but also cost over time. Thinking about, okay, I want to exit this business, making sure that you have alignment with ISO 42001. And, you know, it's in order to say, OK, you know, we've thought through our risks and how to manage these risks and how to govern over these risks. This is very similar, actually, to arguments around how value creation connects to sustainability. And so it seems sort of obvious to make these connections. But I think that right now it's it almost reminds me and this is the FOMO question.

25:46It reminds me of 2020 and 2021 when GPs that we were working with were being coached before LP calls to say ESG is my top priority, even if the role was value creation. And what Rico said was, look, your title is head of value creation. Your top goal is value creation, and it's okay to say that and make the connection to what ESG means for that. And now the pendulum has swung to, you know, many GPs being unwilling to say anything about sustainability unless it's very clearly connected to financial return. And part of it is, you know, we're all very connected at the moment. The zeitgeist is very clear.