E410 | Alvaro Alvarez del Rio, Dr. Andre Retterath, John Frankel & Steven Greenberg⁠: GP Roundtable on Leveraging AI in Portfolio Monitoring & Management

30:06On their website, we vectorize everything and so on and so forth. Second dimension, we have manually classified. And this is built in-house by your data sciences. We always try to think about make versus buy. And of course, we started a couple of years back. So we started our journey in 2017, 2018. By 2020, we started ramping up our engineering team. And back then, just like very few things were on the market. So we continuously think about make versus buy. And we try to get rid of as much as we have built in-house as possible. Because we want to keep the maintenance as low as possible. But at the same time, we still find that we don't want to end up with the kind of patchwork stack where we have multiple best of breed solutions.

30:56We want to have like one single source of truth. And that was one example. Other examples, we have manually classified more than 50 ,000 companies into our internal framework. So we have like an industry segmentation framework. With that, we train a classification model so we can classify all companies also for market mapping. But this is a bit in terms of how we reuse stuff that we built for the sourcing screening due diligence part for the portfolio value creation. Now, if you think about portfolio monitoring, there are different components. Number one, again, you need to collect information from the companies.

31:32So we collect tons of outside in data about the company and know a bit about their health already. But then obviously we have the regular reporting. So we want to keep it as easy as possible for the founders or the teams to just put in the respective information, get pulled into our system. It gets unified with other data sources. And then also you can use LLNs to just like redraft in your style, your way of how you prefer the reporting, your reporting to the APs. And you can push it through specific systems. So we try to work as much as possible with standardized input output with APIs, and then really put everything into the single source of truth and then put it to the respective stakeholders.

32:17Steven, how do you think about the, or how do you typically, when you talk to clients that are considering Totem VC versus building in their own versus something else, how do you think about the make versus buy decision? How do you typically say, and this, of course, has something to do also with the sweet spot for you, because we all have places where our tools are more relevant than others. A firm like Early Bird is probably not this space where Totem BC is the perfect match, but maybe you would say that I'm wrong there. I mean, I think a lot of the tools that are being built in-house today are tools on the sourcing side of the equation.

32:54And that tends to be where firms want to have their special sauce applied to their process. I see what we're doing on the portfolio monitoring side as being some of the nuts and bolts that you don't necessarily want to be recreating within a fund. And when I think about portfolio monitoring, it really is a game of cat and mouse. It's investors chasing their portfolio companies and investors wanting as much data as they can get and companies wanting to give as little data as they can give. and the truth lies somewhere in between as to what is a reasonable approach. And I think historically, it's been a very tough problem to solve because of the amount of unstructured data that exists, whether that's data coming in in the form of a board deck being shared with you or financial statements coming in at differing timeframes or calls that you're having with various notes that you're taking on those calls.

33:55And we've tried to really streamline that process. So I kind of think about what we're doing in two buckets as it relates to AI and portfolio monitoring. So on the first side of things is really helping you capture all of this structured and unstructured data. So on the structure side, we make it as easy as possible for portfolio companies to upload their normal financial statements, and our system will take care of doing all of the data extraction behind the scenes on behalf of both the company as well as the investor. And on the unstructured data side, it's really being able to pull in emails that may have context documents such as board materials and any call transcripts, obviously, that you're discussing updates with your founders, and being able to pull out all the unstructured data.

34:42And once that's all normalized and in the system, it's really then how you can create reports using that data. So, you know, one of the big challenges I think a lot of GPs face today is around, drafting their LP letters. That typically takes a ton of manual work to go and find all of this varying information. And I think just as easily as we can help ingest that data from both structured and unstructured, I think we can also help create reporting formats, such as an LP letter. So what clients can do is they can upload a past LP letter. Our AI can scan the document and figure out what are the different sections that are contained on your LP letter, figure out what data is needed to recreate that, and then actually query the database for the different data that's needed and recompose it into a draft that you can then take to the finish line as a team.

35:36So that's kind of how we think about AI in the context of... I may have one question, Stephen. I think that's very useful. How do you think about best of breed stack versus a single source of truth? because just like to explain what I'm referring to, for us, we also use a variety of tools. So for example, for CRM functionality or email calendar integration, we use still some functionality of affinity because they just did it. There's no point in us building it all. So cap table management, we can use Carta for kind of like, I don't know, a few reporting, Thunderbird for portfolio stuff. There are other, like we use best of read solutions, but then embed them in the stack and really merge it all together.

36:20But I haven't found this single source of truth. And what I see personally in the last, say, 12 months or something, is that many of the providers try to become this single source of truth. So essentially, some of the players that I mentioned earlier are expanding their value proposition. They enter into partnerships, having native API integrations, because one of the biggest blockers in the past was that systems don't natively integrate. So you end up with a messy best of breed stack and you need to move data from isolated silos from A to B to Z. How do you think about that and how would you approach that?

36:59It's clear that every fund manager is craving a single source of truth because even the funds that have these various best of breed solutions in place, like a CRM and like, you know, fund modeling tools and sourcing tools. They're ultimately taking all of that and either Frankensteining it together with an air table, you know, solution or in a more elegant way, perhaps building tools like what you guys are building at Early Bird. So it's very clear that people desire all of this to get pulled together. We don't think of ourselves as necessarily replacing some of those underlying pieces. I think a lot of these tools have built really great and deep utilities for various use cases within funds.

37:35we really see ourselves as the tool that stitches things together. And I think it's only really now becoming possible with the advancements in AI and how it has reduced the friction of capturing data. So I think it's historically something people have wanted and have tried stitching together with varying degrees of success, but I think it's now possible. So we really see ourselves as the unifying layer, not necessarily trying to replace some of the pieces that may already be working well within the funds. Yeah, it's very interesting. And I see this ambition with many, many VC or investment tech stack providers more recently, because everyone's here is like, there is this need for a single source of truth.

38:16And technically, now with LLMs, it's actually possible to string data together in an easier fashion. So the question is then obviously, who owns the interface to the customer eventually? So who consolidates, who gets consolidated in a way? because everyone wants to own and be the I-stitch-it-together kind of solution, which I find at this point in time very difficult to assess because many tool providers have this ambition. But it's super, super interesting and dynamically changing, and it's going towards the single source of crews for sure. Alvaro, I'd love to ask you a bit about your reflections when you've heard the rest of the panel talk here because I gave you the challenging task of opening it, which meant you provided broader context.

39:01Now I'd love to, you know, what here that you've heard kind of resonates with you? What, you know, how, what decisions have you made on these parameters? When it, because as you said, you've built it from the bottom up with Boost. Yeah. So a couple of thoughts of, you know, things I was thinking as I was hearing the rest of the panel. The first one is, I think it depends massively on the type of fund you're running. we're a smaller fund, we're a newer fund and therefore we have to be very mindful of the number of external tools that we use because they are pricey and normally the more value they add the more pricey they are and normally it tends to be the case that all of us know which service providers are providing the most value at the same time a lot of us cannot afford to pay for them and so So I wish I could use most of the tools that I've had conversations with, but we have to limit the number that we use based on the budget.

40:07And for us, what's really important is functionality. How critical is what this company is doing to boost? And so, you know, for instance, if you talk about sourcing and you've got a lot of tools out there, potentially you have to compromise by either not paying for the best tool or building it internally. And Andre was referring earlier to the fact that at the end of the day, what you want is to minimize maintenance and to have an external provider. And if possible, then again, to have that one single source of truth. but it becomes particularly in an environment where someone controls that one single source of truth, pricing becomes a challenge to new entrants.

40:53And I want to address that because probably in the whole panel, I am kind of representing the new entrant to the market. And then the other thing that I was thinking is, to me, something that has not been fully addressed today is there is a lot of manual input still here. So when we were thinking at the beginning of, look, you know, we're going to maximize the value that we give back to our ecosystem and particularly to our portfolio, you come up with these ideas, hey, you know, let's put all of our VC connections together, right? So that it's easier to identify for our portfolio companies. And we were thinking about, hey, let's record every interaction that we've got with our portfolio.

41:41And, you know, let's record every quarterly metric that the companies provide. Or, you know, let's provide an answer to everyone who sends a deck our way. And so when you think about that, there's a lot of manual input that it's either a one-off or it's a continuous kind of exercise of providing that information. And you can replace part of that, as we've discussed today with AI, but a lot of it actually can either not be replaced or has to be audited somehow by a human. And so you need to find that balance of how much artificial intelligence is replacing what you're doing. But in our case, in particular, at the very beginning, it has been a very manual exercise where we said, well, this is where we went, what we wanted to get.

42:34You know, how many corporates do we have direct access to? But, you know, how do you get that information out of your network? Sure, you've got unstructured data in your networks and different social networks, etc. That's fine. But that's still very unstructured, very dirty data that you need to clean. And so to me, one thing that is important is particularly when you are starting this exercise, this is time consuming. And then hopefully you get into that wheel of optimizing as much of it as you can so that it becomes, that's what we're all trying to do is to minimize the time that we spend on those more menial tasks on a regular basis.

43:14Let's actually double click on that question of cleaning data slash making sure that whatever you get from the LLMs is actually true. It's definitely something that I'm spending some time on. So maybe let's start with you, Stephen. how when you have Totem VC pulling from different data sources and different systems, how do you allow the user to very quickly audit that it's actually true? Yeah, I think that's especially critical given the current hallucination rate. And hopefully with the current trajectory we're on, AI will become better than humans and they'll be checking us. But for now, that is not the case.

43:54So there's a few things that we do on that side. So number one, anytime that we extract data, we will always link back to the source. So whether it came from a call transfer, a document, an email, obviously a human needs to be able to verify this, especially if they're ever going to be reporting that data anywhere. I think secondarily, anytime a performance metric is calculated, something like IRR, for example, I think a lot of these systems that exist are very much black boxes and it spits out a number. and a CFO is then forced to have to run a separate calculation to make sure that that aligns with the system because they don't have full visibility into what is driving that calculation.

44:33So anytime we calculate numbers, we give our clients the ability to export and actually see the underlying formulas that are driving that performance metric. I'd say additionally, anytime we extract structured data, such as, for example, from a financial statement, I think what's nice about financial statements is there are some cross-checking that you can do. to verify that things were pulled out correctly. So one simple example would be that a balance sheet is balanced, right? If something is not balanced, then clearly there's something wrong there. So putting in place double checks after the fact to ensure that what was extracted makes sense within the GAAP accounting standards.

45:14And lastly, and I think this is an ironic one, is self-reflection. AI is particularly good at checking its own work. So, you know, I think there was a study that was done that if you ask an LLM, you know, to do a task and you don't ask it to check itself. But then in a separate process, you have, you know, an AI that does the, you know, that does the work and then an AI that reviews the work. And they have this back and forth dialogue before the two are in sync with each other. The results are, you know, way greater. And I guess in the context of what we're doing, after we've extracted anything, having the process called reflection, where the AI actually goes back and does a sanity check against itself.

45:57And that seems to have surprisingly effective results as well. You're doing that inside the platform? Yes. That's cool. There's something to understand about this, which is AI is not an HP 12C. so for those who remember uh in finance there was a calculator that came out i don't know 30 plus years ago called hp12c which used reverse polish notation so everyone felt cool on trading floors using it and every generation of hp12c had a chip that didn't get faster so if you took an hp12c from 30 years ago and you took one today and you run the same calculation took the same time because they didn't want people who already had bought one previously to find that it now, what didn't work as a standard, it became a standard.

46:52That is not what we're having with LOMs. I mean, you've seen it with image work. What they did two years ago, what they did two weeks ago, what they did two days ago is rapidly changing. And we're seeing the same here. So today, it's sufficiently good. In two years, it's going to be better. And so, yeah, I think that's something difficult for humans generally to get their heads around because we tend to like standards and we tend to like and can drop linear progression. But this is exponential progression and capability. So I think if we look in a couple of years and we ran the same podcast, there'll be a whole series of other areas we've been able to apply this technology to because it's just got buttoned down.

47:43How, Andre, have you solved for this issue of hallucination within your models? How do you do that? I think it's generally a question of the philosophy. So there are two ends of the spectrum. The one hand side is just like traditional investing as we've done it for the past 70 years since the industry evolved. So it's just like humans meeting their bodies at the golf course and just investing in their friends. Like I'm exaggerating, but just representative of the old world. And there are a few extremists. So we could see that in our survey in the data per VC landscape that already 10 % of the people we surveyed believe in a pure quant VC, meaning there is no human involved in the data collection, decision making, investment process and portfolio value creation afterwards.

48:30So there are already people who believe it. And this number has exploded in the last year. We see ourselves in the middle of the spectrum. So I call it an augmented VC approach. And augmented means we let the machines do what they can do best, which is collecting structured and unstructured data, processing it, putting it in the right place and being directional for humans in terms of you should spend your time here, because there's a higher likelihood of success and value creation and not here. So this is how we use our systems. But eventually the humans are still in control, number one. So as soon as we focus on a specific opportunity, for us, it's a lot about sitting together with the right founders.

49:12Like we need to sit together in person. You need to see like if the fire in her eyes is burning and they are up to something really big. So for us, this is still very important. And also if you flip it around, like the founders if you are as early but we are a lead investor or co-lead in very few cases but we are not one of these tiny followers so if you are a lead investor you also need to convince the founders to go with you and the founders also want to have a human on their board like if shit hits the fan i don't want to call the algorithm and tell me what's happening so that goes also a bit into the direction of what john was saying earlier in terms of being like the altisons and so on.

49:50And this is our belief. That also means that the human is in control of whatever goes out. So we also played around with LLM generated outreach to founders. We measured conversion rates and so on and so forth. But we stopped it. We stopped it because there's also a massive downside in terms of harm towards the brand, founders not getting the right attention and so on and so forth. So you need to be very cautious in terms of what is your belief? What is your philosophy, how does that translate into the tech stack? And then really, how do you as a human want to create value in the long term? So that's on this part.

50:27And I just wanted to touch upon something that Ivaro said earlier. You mentioned that the value created by a tool provider mostly correlates with the pricing, which I agree, by the way. And the problem is not only that one once one single provider owns so much of your stack and creates so much value for you that the price is very expensive, but also the problem that you outsource majority of what you should do as a fund. Like if you outsource everything from the data collection to the decision-making, like which opportunities should I focus on? Which should I invest in? To the portfolio value creation, to the monitoring, and then the exit process.

51:09if you have outsourced all of that to one single tool provider? Like what's the point of giving you a 220 as an LP? I really don't see it. So the question is really secret sauce. And this is why the core of our tech stack, we've built ourselves. So we emulate human decision-making. Like our investment professionals, we send them recommendations. There's an interesting opportunity. And like dating app style, they can swipe left, swipe right, go deeper. We track all of this information to represent and codify the decision-making, and we will never give that to an external party because I think this is the long-term secret sauce, how we can codify our decision -making and scale it up so that we can apply it to a broader universe in a biased plus also unbiased, more sample approach.

52:02And I just want to add one thing there as well. I've heard an LLM described as a calculator for words. And I think today it's being used as a tool. You know, the expression goes, you know, if you have a hammer, everything looks like a nail. Everybody's trying to fit every use case into an LLM. I think, you know, you need to figure out when you're, especially when you're extracting data, what is native and makes sense for the tool. and what is something that should be done outside of the context of the LLM. So I think that's another way that you can at least downside the risks of hallucinations from these models.

52:38I was going to make the point that to me, we're not replacing ourselves. We're just trying to free up our time to do other things that we do best. And I was just making this example the other day where you've got your top of the funnel with say 10 ,000 companies and you've got the, call it 100 companies you decide to spend time on and you've got the 10 companies you can invest in. And then you need to be able to get access to the 10 ,000, but you need to select the 100 and the 10 that are best. And it's as much about getting those 10 at the bottom of the funnel as getting access to the 10 ,000.

53:17And to me, what's really interesting is at the risk of being mocked in five years' time for this, but I think that as long as we continue to invest in humans, there's a human relationship there. And so we always kind of break down the funnel and narrow it down to the 10 companies we're going to invest in. But guess what? Those 10 companies are probably being approached by a number of other people and they need to decide whose money they're going to take. And so you can build the best models, you can get to the best 10 companies, but if you don't have a winning rate that is better than the average in the market, and to me, there is a human effort, you're not going to create a top performing venture capital fund.

53:56I want to amplify something that Alvaro just said and sort of pull on a thread here. You know, if you are able somehow to distill the creative part of you into a machine, into an LLM, into a process, then yes, keep that proprietary for sure. but as VCs

54:26so and this was referred to earlier before certain distributions in the VC space are normal and certain distributions in the VC space are power curve, they follow the power curve and I have a very strong belief in early stage VC that deal filtration, deal selection, the companies you actually want to work on is incredibly artisanal. It is informed by enormous amounts of data you as a VC bring in, but it is also tremendous amount of judgment about where you think the world is going in the next five to 10 years. We like last mile drone delivery. And we've liked it for five years. We backed MANA five years ago.

55:25But even today, it's a little risque in the VC business. But we think the economics, et cetera, make it incredibly compelling. I can't imagine what AI looking at data five years ago would have sort of pulled on that thread with the passion and enthusiasm we have for the space. And so I think there are a lot of chores we do. I can't tell you how many emails I sort of kill in my video game of managing email every day.

56:06but the creative side of the business, thinking of the spaces you want to move into, how you want to work on it, the people you want to work with, et cetera, that to me is something that I can't see being reduced. What Totem brings to us is the ability to normalize the data. It's a single source of truth and allows us against that normalized data to take out a lot of the drudgery of what we do and give us as a team more time to focus on the creative. Steven, I want to close by asking you to just give some final remarks as our dear sponsor for today's session. However, before going there, Alvaro, I want to just ask you about the ethical implications.

56:54I think John put it well before. Let's think about when is it okay to have a camera in the showers? So, Alvaro, maybe you would just share your views on some of the ethical implications and considerations for us to have as VCs. And then after that, we'll go to Steven to round off. Yeah, I think what's important to understand is that models are built by humans. And every human, we all have our biases and we have our needs. And that's why I was alluding earlier to the fact that using what's worked for other people in order to build your own models is very risky because we all prioritize different things when we're looking for companies.

57:37But you do have to give that input to the model. You do need to prioritize whether an MBA from a top university is more important to you than having worked in big tech before or it is less important. and what weight each of those carries within your model. And I think the big, big risk here has to do with that bias applied to diversity in the ecosystem. If you give me a data set with the 20 ,000 most successful exited companies in the last 20 years in technology, it is very clear that you're going to have many more Stanford grads, white male Stanford grads than you will, females with no university education.

58:23And if I apply that to the future, then that is the data set I'm going to get. My conclusion is going to be, I need to invest in white males that come out of Stanford. And that is the best source of unicorns there is. And that the ethical implication there is very clear, I believe, in terms of providing value to the broader ecosystem, allowing others to get into the ecosystem, if at the end of the day you're trying to bet on disruptors and you're trying to bet on people that build the future because they look at the world differently, you cannot bet on the system. And so to me, being conscious of those biases and understanding the inputs that you're giving to the model so that it spits out the right type of information and you are aware of how you are making those investment decisions is extremely relevant.

59:22Yeah, absolutely. That's not all Invest in White Males coming out of Stanford. Stephen, I'll take it to you. Take us out of this. Let us know just a couple of thoughts from your end as the sponsor that made this happen. Tell us a bit about what you think and your reflections on today's conversation. Yeah, so first of all, thank you again, Andreas, for putting this together. Thank you to the rest of the panelists. It's been a great conversation. I think in thinking about, we spoke a lot about what the current state of AI is, but this is clearly an ever-evolving space. I mean, literally, if you're tuned off from Twitter for even 24 hours, you've basically fallen behind.

1:00:03And I think about really where we're at, zooming out a little bit. We've been really a couple of phase shifts. we started out you know probably about 10-15 years ago in being able to analyze patterns really well right what was called machine learning I think we now have LLMs which are really good at you know not only analyzing patterns but creating similar patterns and really where this is all headed I think is not only analyzing the patterns being able to create similar patterns but can we give you know the permission to be able to act on those patterns and I think that will have a number of implications within the space of venture capital.

1:00:40I don't think, I agree with you, John, I don't think the creative aspect of venture capital is going to go away, but there are some of the menial tasks that tools can certainly take actions on your behalf. And ultimately, I think we're in the bottom of the first inning with all of this. And I've been particularly excited about building, and we've been at this now for seven plus years, and I'm ever more excited in building what we're building for fund managers. Just very excited for what's to come. So thank you all for participating today. Amazing. I absolutely agree. I don't think it's ever been a more interesting time to either invest or to be running a VC firm or building a VC firm.

1:01:25It's incredible everything that we have at our disposal today, which we did not five, seven or 10 years ago. Everyone who joined today, thank you so much. Do make sure to subscribe on EU.BC, where we'll also be putting out the core learnings from today's conversation. And then I just hope to see you all around.

1:02:04problem requires a European response. Europe is a story of new beginnings. New beginnings. Let's start acting.