Ep 150: How AI Is Transforming Diligence, Decision-Making & the Future of Investing with John Melas-Kyriazi

0:00AI as a technical co-pilot for diligence is actually one of the most interesting use cases for AI as an investor right now. If you come across a really interesting company, you can get really, really deep within a matter of hours and then pull in human experts to help you to go the last mile. The mission for our business is to accelerate innovation in the private markets. If people had access to better data, they could make better decisions. Is it changing how venture capital firms operate when they're using tools like Claude and ChatGPT to poke holes in them? What am I missing? Where's my argument weakest?

0:30Investors for decades have not had access to the kinds of tools that they need to really operate their businesses in a data-driven way. Let's talk about how AI is changing things right now. You can go into Excel, hook it up with Claude, hook up Claude with standard metrics, and then ask you to do a discounted cash flow. It'll just build it for you. It's amazing. People are going to be able to spend more time on what they're truly passionate about.

0:58John Millis Curiazzi is CEO of Standard Metrics. Over 10 ,000 innovative companies use this technology with hundreds of venture firms to do their work and figure out what's going on, how businesses are doing, how to interact with them. John is on the leading edge. He's running a really high growth SaaS company. It's becoming an AI company. It's really interesting to hear from John about how AI is transforming business, how it's transforming finance, and what's going on in the innovation world. Welcome to American Optimist. Today, we have my friend John Millis Curiazzi. John, thanks for joining us.

1:26Thanks for having me. Hey, John, you're a co-founder and CEO of Standard Metrics. I think it's a good example of what's happening in AI with agents right now with all sorts of other things. So I want to talk about that. But first, tell us a bit about yourself. Where'd you grow up? I grew up in Brookline, Massachusetts, right outside of Boston. What were you into as a kid? What kind of stuff? So I was really into reading fiction and then over time, more and more science fiction. Fiction and then science fiction, because you studied physics eventually. Eventually, yeah. I think science fiction was maybe a precursor to that.

1:54And your master's was in material science. That's a different area than you're working on now. Is there cool stuff happening in these areas you studied? Is that something you eventually want to go back to? You know, I don't know if I'll go back. I loved it though. And it definitely inspired me a lot. I think as a kid, being interested in science, I got really into electric guitar in high school. And then that drew me into electronics and circuits. And I think I freaked out my parents because I ended up setting up like a soldering iron and like printed circuit board etching station in my basement.

2:23That kind of was my first foray into getting into engineering. What were you trying to build in your basement? I built a tube amplifier, a vacuum tube amplifier. And I also built a bunch of guitar pedals to help with like reverb and distortion and stuff like that. So you're an engineering kid, a physics nerd, went to Stanford, you started a PhD, you dropped out of a PhD. How come? That's right. You know, I was really lucky in my sophomore year in undergrad, I ended up at this material science labs, Mike McGee, who's now a professor at CU Boulder. And he was researching next generation photovoltaics.

2:58And I had this incredible PhD mentor, this guy named Ikang Ding, who kind of took me under his wing. We ended up publishing a bunch of papers together. And it was this really cool experience. Papers on what? The first one was around a new manufacturing process for a new type of solar cell we were working on. And the second was more of a device physics paper. So it was understanding really the loss mechanisms in the solar cell. Like when the sun shined on the solar cell, what happened to all the electrons and holes, which are like the counterpart to electrons and where were they getting lost and what was degrading their performance?

3:34So it was this really cool experience. And I thought, gee, like, this is awesome. I should keep going. I should get a PhD and become an academic. So that was where I was originally kind of pointing myself. And, and then, you know, I had a bunch of friends that went and worked at startups. I went and worked at a startup for a summer. And when I got back to my PhD, I realized that I wanted to be doing startups instead. It's interesting. I think this is a really common thing in our society right now, where I think it used to be maybe 50 years ago, a hundred years ago, the smartest people who wanted to pursue the most intellectually demanding and interesting things would stay in academia because it was just more interesting there and there's more challenges there and there's things you could do there.

4:12And now I feel like the last 30 years, especially the last decade, started to become so much more compelling. And there's like so many smart people there and you can build and solve so many problems. You actually get like more of your, you know, Aristotle would say that like, like the, that one of the things that's most satisfying for man is to, is to use the highest use of their faculty applied to, to successfully. And I feel like that's now more in startup world sometimes in academia. That was definitely your experience. That was my experience. And look, there are aspects of academia that I absolutely loved.

4:43I think that there was an element of intellectual freedom exploration that was incredible. It's actually something I loved about being an investor too. Post academia, I spent a couple of years helping run this early stage startup accelerator called StartX, and then became a VC. I was a VC for six years. You were at Spark. I was at Spark. That was one of the things that I loved about being at Spark was this idea of being able to explore many different things, being able to go very, very broad, and then choose when you went deep. It's very different from being a founder, which I also love for very different reasons.

5:16It's interesting though, because you like that about academia, but then in Spark, you're able to do even more things, it sounds like, and go even broader, right? Yeah. I think as an academic, as a researcher, I felt like I had a lot of freedom to explore different areas within a certain field. And part of the goal was to become, you know, the world's expert in that field or in a subset of that field. As a VC, and I think it's actually something that AI is now changing, which I'm psyched to talk about with you, it's possible to have a much, much broader scope in terms of the types of businesses that you're looking at, but then kind of gear up and start to get really deep technically where needed.

5:52Well, the AI thing is actually interesting. I feel like I've always had a little bit of an advantage because I do go pretty deep in a lot of areas. Maybe not as deep as the PhDs, but I go decently deep. But now with AI, everyone can just pretend, right? And by the way, it's useful for me too, because I'm like, oh, how does that, how does that really nuanced deep tech thing work again? And you just like ask it. And then all of of a sudden you're like asking really good questions. It's a little dangerous right now. It is. Yeah. I think like AI as a technical co-pilot for diligence is actually one of the most interesting use cases for AI as an investor right now.

6:19I don't think it's a replacement for human experts necessarily, but it certainly is a huge help. If you come across a really interesting company in a field that you know a little bit about, but you're not an expert in, you can get really, really deep within the matter of hours and then pull in kind of human experts to help you to go the last mile. Whereas, you know, five years ago, it would have been this kind of mad scramble, kind of calling people, trying to assemble folks that might've taken days or weeks that can be done much faster now. I actually, one of the hacks that I really enjoy when I have free time is to ask the AI, like, what are the most interesting and important or cited papers or new papers or controversial papers in a field?

6:56And then explain these to me as if I only have an undergraduate degree, like, like, so, so dumb it down a little bit from like the most sophisticated PhD level of whatever it's saying and explain it, how it works. And you actually can pretty quickly kind of learn what's going on in pretty much any area. For sure. Yeah. I think that as a VC, having the ability to tailor the output of a model to meet you where you're at from an understanding perspective is one of the most useful aspects. Opening up a 10, 15 page long academic paper full of citations is extremely intimidating. It's so annoying, these papers.

7:28And it's like they purposely, I think it's like they purposely sometimes making them so you can't understand. This is how I always felt about Emmanuel Kant and some of these philosophers. I'm pretty convinced they're purposely doing it to like, so I'm not one of the initiated. There's a proof of work there. You need to put in the time and effort. And they like that. They're purposely putting the barrier in so the AI can remove some of that barrier. It's dangerous because you probably are missing a few things still. Yes. So you were helping people build things. You were an investor. And then you went on and I was involved obviously in founding what's called Standard Metrics now, which is this awesome, of really quickly growing company.

7:59You have over 10 ,000 companies now that Standard Metrics reports on for people? That's right. What was the idea behind the company? Yeah. So when I was at Spark, when I joined the firm in 2014, the firm had already been fabulously successful. It's like taking Twitter public, Wayfair, Oculus was sold to Facebook. It was a really interesting time to join the firm because it was a group of very technical people who were really interested in investing in cutting edge products. At the same time, the firm didn't really have core software that it was running to operate day to day. And when I got to the firm, I was actually surprised by that.

8:36I went to talk to friends in the industry and started realizing that very few investors in the private markets really had access to great software. It was this kind of cottage industry that had suddenly grown a lot and become very competitive and very global, but software hadn't yet caught up. And so I started going on this quest. I actually ended up meeting Ray and Shuby from Affinity as part of this, looking for software for our firm that we could use. We ended up buying that software and using it to try to run and improve the way we were doing relationship intelligence with data. So kind of discovered firsthand.

9:07And some of the stuff that I went through when I was there was around, for example, have an LP meeting coming up. How to collect data from over 100 companies, clean all that data, have it be apples to apples, use it for a bunch of modeling exercise and reporting exercises. When you say apples to apples, for people, it's basically you want to show off the metrics where you actually compare the company's progress to each other. That's right. For example, let's examine revenue growth rates across the entire portfolio. Let's examine gross profit margin across the entire portfolio. And so going through those exercises, even things like, hey, let's keep a list of the companies that are pretty low on cash to make sure that we're jumping in to help them.

9:45Those things were incredibly manual and time consuming. And those experiences really inspired me, kind of gave me a prepared mind. Then when I ended up meeting you, Alex and Danny and the team at 8VC, who are also working on this problem, kind of gave me the inspiration to go and start this business and try to build a platform to automate and improve portfolio reporting and reporting management for the private markets. It sounds like pretty esoteric, great portfolio reporting management for private markets, but there's literally hundreds of thousands of the smartest people trying to solve these problems.

10:14And so you're kind of supporting. How many customers do you have now? How many people use this? Over 150. 150 firms. 150 firms. Yeah. And well over 10 ,000 companies now on the other side of the platform. And it's neat because Affinity, obviously, we built to solve one part of the problem. This is like the other biggest part of the problem for all these firms. And it started in venture. Are you doing private equity now as well? We're doing some private equity now as well. I'd say especially in growth equity, which is kind of an interesting in-between. Looks venture style in some ways, but then they also do buyouts.

10:42We see the mission for our business is to accelerate innovation in the private markets. We think that ultimately a lot of folks spend a lot of time doing things that ought to be faster and more data driven and more automated. And we also think that if people had access to better data, they could make better decisions. So portfolio companies, for example, when they report data on Cedar Metrics to their investors, they get access to aggregated and anonymized benchmarks that are relevant to companies at their stage in their sector. So the 10 ,000 companies actually in some ways are kind of your customers as well.

11:17Absolutely. Absolutely. Yeah. And from our perspective, if we can help those companies to build better relationships with their investors, have a better dialogue with their investors around their numbers, and also have access to better data, we think that they're going to improve their probability of success. And then on the other side, investors for decades have, we think, not had access to the kinds of tools that they need to really operate their businesses in a data-driven way, which extends from the back office and processes like working through the audit and valuations and LP reporting, all the way to, hey, our portfolio company's coming in.

11:57They're raising another round. Should we invest more? Should we invest less? Should we participate? A lot of my stuff right now, it's like, I was with a CEO last week and it's a new AI company that's, you know, a couple of years old. He's like, Joe, we're not just doubling revenue this quarter, we're tripling revenue this quarter. And I always say, that sounds crazy. It's like really good, but I'd be interested to know, okay, well, I'm sure there's like another 50 that are doing that too. Like, how does it compare based on the stage it's at? What are those raising rounds out? Like, so like you're based on the aggregate data, like, what is this worth?

12:24Like, I actually don't even know. Like, cause it's all changing so quickly, even for me in the middle of it, probably as much as anyone, like, what is that worth? And it's like, you would know a lot better than me because you have all the data. Like, how does that workflow work? Yeah, for sure. So I think there's a couple of things. I think one is getting all of the data in one place. And I think the way we think about it is that firms need a mechanism to collect and store all their information on their portfolio. As we said before, kind of in an apples to apples format, that's clean, that's auditable, that's kind of traceable back to where the data came from.

12:57they need a way to access that data. We think about it as we have this deep commitment to interoperability at our company. So we think that when an investor has their data, it's their data. They should be able to use it with their own AI agents that they're bringing to the table or with software that they're building on top of standard metrics or in a spreadsheet or on a platform. And then lastly is kind of robust tools that help them to leverage that data, as well as the corpus of data that exists in an aggregated and on a market. Yeah, because if I want to ask what that's worth, I need to use your aggregate data to see, to compare it to other things.

13:31Absolutely. So we think that the most powerful mechanism is if a firm has great intelligence on their portfolio company, they understand all the numbers. They also have written artifacts, notes, partner commentary. And they also have access to what's going on broadly in the market and how that's changing over time. They can take all of that data to bear to make the right decision for their firm. And that's something that we think is an opportunity for firms ultimately to generate more alpha, not just creating, like having really smart initial investment decisions, which is obviously mission critical.

14:05But then also, you know, many firms take 20 to 50 % of the capital of their fund and reserve it to invest in their existing portfolio companies. And we want to help them to do that better. So let's talk about like how AI is changing things right now for firms. Obviously you guys are seeing what's going on. You guys are seeing the numbers. Is it changing how venture capital firms operate? How are people using AI in general? Yeah. It's changing how firms operate all the way from the very, very earliest stages of sourcing and identifying companies all the way through things like portfolio management, LP reporting on the other side.

14:39It's actually really fun right now. I'd say when we started Standard Metrics with you guys, in the initial phases, a lot of what we were doing was evangelizing to firms like, hey, you should use software. Software will help you to run your firm better. And now that's still happening, but there's also a lot more that we're learning from firms. They're experimenting a lot. So for example, on the sourcing side, for the last decade plus, there've been a bunch of firms that have built data-driven sourcing engines. They're looking at things like GitHub stars, a number of employees on LinkedIn, social media followers.

15:11We're now seeing people use large language models to do a lot more qualitative research that complements that quantitative. For example, like what's the tone of what people are saying on Reddit about this company? Or much more nuanced. Is it good if they're attacking it? I mean, those guys are pretty crazy. Maybe, maybe. Well, there's also a lot that people are doing around the analysis of people. So for example, if you take someone's GitHub profile for their kind of public facing work and you take their LinkedIn profile, can you start to build a map of the type of founder that you want to identify back early that matches with your investment thesis?

15:44So we're seeing a lot of really interesting experimentation there. On the diligence side, as we discussed before, AI as a technical copilot is super powerful. Another thing we're seeing is that firms are writing investment memos, and then they're using tools like Claude and ChatGPT to poke holes in them. What am I missing? Where's my argument weakest? And then using that to help to guide the diligence process and then feeding that back in again. So you start to get into an iterative mode. On the portfolio management side, we have customers that are doing wild things right now. For example, we have one customer in Singapore that's using an agent orchestration tool and it's building this wide variety of AI agents that interface with standard metrics, grab data, and fulfill a bunch of really critical internal reporting workflows.

16:29For example, an LP asks a question, an agent automatically gets triggered, reaches into standard metrics, goes back, composes an email, flags it for a human, and then sends it off to the LP. Things that you wouldn't have even considered a few years ago are now starting to get built, which is all automatic, basically. Yes. People are talking to agents more than they realize. Oftentimes with humans in the loop. And I think human in the loop has been a really important part of our business, particularly on the data processing side and making sure that data is correct. And then, yeah, when it comes to the sort of technical diligence side, that's another area where I think the AI is really helping people to push deeper into deep tech.

17:04Can you not teach the AI to like use the common sense check to make sure the data is correct? Like how, like, like, it's like, how long do people are going to have an advantage in that? Isn't there like a common sense framework you can give another agent to do the check? I think there will be, I think there will be. And we're certainly seeing even just the, the, the, the kind of core models, the reasoning models get better and better and better at identifying issues. I think that when we think about the future, when data is really well organized, for example, the internal AI agent that we've built, which we call our AI analyst.

17:36When the data is really well organized. So this is an analyst that's part of standard metrics that a firm could use if they have standard metrics. That's right. Effectively. Do you have a name for it? We call it AI analyst. It's not the most creative name in the world. I guess standard metrics, there's no like little like standee or something. You gotta get something more clever. Atopar has Addison, right? So I'm like trying to like, standard metrics just has AI analyst. Yeah, we decided to not anthropomorphize the AI agent. It's not a person, it's just an agent. That's fair. But maybe we could use some branding help there.

18:03But the AI analyst, when it has extremely structured context with really clear harnessing guardrails, it's unbelievably accurate at pulling data across large amounts. So, for example, you could have hundreds of portfolio companies and ask a question like, make me a table with my top 10 fastest growing companies where I own at least 5 % that have at least$10 million in revenue and include these metrics. And it'll produce it with very, very high fidelity. We do want our customers to always go and double check the data, but that works really well. What's tougher is feeding in a completely unknown document and having AI extract data well out of that.

18:43The unknown documents is a little harder. We do that currently, but we use humans in the loop to make sure that we QA. And that's something that just is not a fully solved problem, obviously. It is not a fully solved problem yet. We think it will be in the future. For now, having humans in the loop dramatically improves. This is where a lot of cutting edge stuff is happening. like one of our companies, like with real, like hand-drawn landscaping documents, they can get it right, even though it's like different every time. Cause like it's just smart enough to like figure out that they're probably need a plan given what they're doing.

19:07They're probably need this type of plan, but it's a very hard problem, right? So you have to have a person check for most of these things. By the way, Chris thinks that Stan is a good name for his turn. Stan? Okay, cool. All right, good. We'll take that into consideration. So internally on your team, this is where you were all living in AI world. I think there's like probably a few hundred thousand of us that are part of this in a lot of ways. And obviously everyone's society is part of it, but a lot of people are listening and they're not building things they are right now. Like, how's it different for you?

19:32Like, tell us about this. Like you have a team, obviously you have a lot of technical people on your team. Like, how is the AI changing how your company works internally? Yeah. Dramatically. And I, and it really feels like the, the, the, the rate of change is increasing every day. The, the sort of the product releases, the conversations we're having it's super exciting and it's a lot to stay on top of. I see the first place where we saw it change the most on our team had to do with data operations. As I mentioned before, we sort of ended up building a human team to go and work with our customers and help them to process our data.

20:03And the first place we launched AI agents in our product was to assist the data operations team. Now AI is doing the vast majority of the work. So that's been a big, big, big shift. Have you cut people or do new things now? How does that work? Because our team's grown a lot, we actually have continued to grow that team, but we're just handling a much larger volume of data. So you didn't need to grow the team as much as you would have to do the same thing. And also what we tend to do with this team called Data Solutions Team is they're oftentimes focused a little bit more on breaking new ground.

20:31So we're really good at processing data from financial statements. We just launched board deck parsing with AI. And so our humans are like much more involved in running that process. So someone sends me a board deck. Like right now I have agents like helping us with emails and stuff too, of course. And like you went in the loop, obviously. Don't worry. When you hear back from me, it's not my agent. But it's helping. So if the board deck comes in, that's actually smart. There should be something where our agent knows to make sure standard metrics automatically gets it and just automatically goes in.

20:57That'd be a good work. And the data is extracted. But doing that from a board deck is really hard, right? From a financial statement, it actually is quite hard because the range of different formats and currencies and fiscal year ends. A board deck is even harder in some ways, but at least it usually should be labeled. Usually, but oftentimes data is in charts. So then you need to use AI to start extracting approximate values out of those charts. So there's a sort of a different set of technical issues that crop up. Well, it's super annoying when it's a chart that's not labeled perfectly or something.