Sarah Guo - The Power of Conviction - [Invest Like the Best, EP.383]

25:09Not yet. That's pretty far. People want a lot of control. They want a lot of creative degrees of freedom. Everything needs to be invested in. And one of the ways this company, HeyGen, thinks in working with their customers is for some type of video, asking customers, like, are we good enough? And one of the core breakthroughs for this company that's been growing really fast over the last year is if you're Patrick and you want a commercial use video to promote this podcast or Positive Sum or one of your companies, is their generated video of you speaking by yourself in a room good enough that you're comfortable using it?

25:46And until this past year, until Heijen broke through that minimum viable quality barrier, it just wasn't for like any player. Input ease matters. So for Heijen, it's like you give them two minutes of video, that's consumer quality. It could be you and me on a webcam, not recording something perfectly, and you'll still get good enough out. If you had 30 cameras in a studio in front of a green screen that are all super high def professional cameras, this problem is a lot easier. It is a little bit multidimensional, but trying to define within a company what's good enough quality as the customer experiences it is like this really important new thing as a moving target as you expand scope of use cases.

26:25If you go look at self-driving, like the scoping of minimum viable quality for a particular use case also seems really important. I don't know if they're allowed to call it full self-driving or not at this point, but stay in my lane. They reached a long time ago. Now you have minimum viable quality as provable to safety regulators in the city of San Francisco for a taxi with no driver. We just got to this year. You can use this concept across a lot of different machine learning products, and then it feels like quite different than scope for other types of companies. It brings to mind that classic idea of the uncanny valley in a lot of this stuff.

27:02Talk through where the valleys are still the most uncanny or the hardest to cross. It is crazy how so quickly some things just seem to have crossed it and just seem to work fine. We were on a road trip recently, and I had one of these audio models narrate our journey for my kids, teach us about what we're driving through as we're driving through it. And I had this thought of like, holy crap, it's good enough. This is fine. This is better than having a human guide in the car because it's more knowledgeable and we can tune it how we want and have it explain things a certain length. And the valley is crossed, so to speak.

27:37So I'm curious where you see the most interesting valleys that are left. Some of them have been crossed, some of them haven't. What hasn't yet been crossed, do you think? Oh, I think the world is full of valleys. It's all valleys from here. We've crossed three of them in little ways. Even in what should have been what is considered to be one of the very first domains, like writing, you're somebody who cares a lot about quality of content. One way to think of the uncanny valleys and how to cross them faster, what the foundational capabilities offer you. And increasingly, the labs are all working on a reasoning that should continue to improve writing output generation.

28:14It's a super multidimensional problem. And for example, if you look at a writing product, whether or not I use something for writing is going to depend on, oh, well, man, I'm so OCD about what I will actually publish. If you generate something for me and it's going to take me just as much time to edit it into what I want, then I'm not going to do this again. Versus part of the new product thinking for these AI companies is going to be how can I make the cost of managing errors or going from 80 % quality to acceptable quality really, really low cost for the end user. Another way to think of this would just be there's increasing interest in just how to do things that are really cheap for models to create a better end user experience.

29:06And so more and more model systems, they generate a bunch of candidates and they have some sort of verification or ranking against those candidates to get to an experienced outcome for the end user that's better. And like cross the valley that way. A really important valley, just to make sure I answer your question directly, is, well, how much code can you generate from a natural language specification? Not much today. There are a bunch of different benchmarks out there. But like if you look at Sweebench or something, it's like, oh, well, all of a sudden we got to 13, 15, even 20%. That's not good enough.

29:38We have some software engineering interns right now. If they generated stuff that was good enough 20 % of the time, they're fired. Go fix that. head. The question is, well, also classical problem in computer science and math, but like, is something more easily, more cheaply verified than solved? I think that's going to be true for lots of classes of model problems where people are designing verification and designing ranking, because this is, I think, one of the most recent progressions in code generation. But if I can, like the Google team did, generate a million candidates and then come back with a reasonable view of the top three, the end user experience is much better.

30:14If I have seven interns, look, 20 % right, but then I know which of the best two intern answers I should look at, that's better. And so we're going to have foundational improvements. We're going to have company-specific improvements, but they're going to be a combination of different systems approaches and research approaches to this. So I am very optimistic that lots of valleys, Lots of people pushing for values to be crossed really fast as experienced by us or other end users. What have been the biggest mistakes that you've seen application companies make that we haven't talked about so far in this fast-changing landscape?

30:55I know I just slammed this idea of the GPT wrapper narrative at the beginning, but there is a seed of truth in it. Maybe I'll backtrack a little bit. State a truism here, which is like, well, if something is really easy for you to build, unless your distribution is totally unique and defensible, you're probably not going to be able to capture rent or economic value on that for a long time. And so we still see plenty of entrepreneurs get into initial traction with something that amounts to weeks to months of work in a handful of prompt templates, essentially. But the idea that could that be an entry wedge to get customers to engage with you so you can serve them more deeply?

31:39Totally. We're not expecting that companies build billions of dollars of enterprise value with a software project that is three months old. But if they think that is the long-term answer, we disagree. Because if they have done it in that period of time, and it is that simple, competition is coming. There's a lot of people who recognize the opportunity. And so I'm somewhat surprised at how short-term people are in terms of approaching the strategy piece of this. What problems do you hope the next generation of frontier models solves? What do you most hope for as leaps forward in the next generation?

32:15This is more optimism grounded in a lot of really smart people at the labs and large players having confidence this is true. But if you could have models that were better calibrated on whether or not they are correct, they can tell when they have a good answer or not. They become much, much more useful. Hallucination management is the blocker for many, many use cases in the enterprise. And there's a lot of confidence that the next generation of models improves against this. The most obvious answer is just going to be multi-step reasoning. It is a core mission of what the labs are working on in terms of solving that in a more general way.

33:04So the things that I'm most hopeful for, multi-step reasoning and self-improvement, The things I am confident on that have commercial value, hallucination management. Let's say you get everything, like we get the best possible version of GPT-5. Where are you going to run towards in terms of seeking new company opportunities? What spaces will get unlocked or use cases will get unlocked in your mind by the perfect version of GPT-5 that would get you excited to invest behind? I would say, I mean, the perfect version of GPT-5 is a lot. So, you know, rooting for you, Sam, bring it out. But well, if all of those things are true, I think we're in a really different zone.

33:43If we just narrowed the scope of it to models that were more verifiable, more calibrated, better hallucination management, these are things that block the enterprise from adoption. There is, I actually think people are asking this question for good reason, but there's so much enthusiasm and so much CapEx spend against AI model development right now. The idea that we're going to hit an air pocket because adoption of these tools in the enterprise where there's real economic value is lagging. I think that is real. But if you talk to companies like, oh, there's the things that are just the enterprise overall, like change management.

34:18But when you ask them, like, what is the problem from a risk perspective, the adoption of AI in large foundation model based applications internally or externally built in large financials, traditionally a huge spender in technology is marginal. It is extremely low. And it's because from a risk compliance reliability perspective, the bar is really high. We haven't hit minimum viable quality. And so part of that is all the clever things we're talking about in terms of entrepreneurs designing systems and product scopes such that the experienced quality by the user is good enough. But part of it is just we need a little bit more from the models, but it's coming.

34:54We need lower hallucination from models. Maybe say anything more you've learned about who is overrepresented and underrepresented in the buyers of AI products. Like that financials point is really interesting. Who are the biggest buyers of this stuff already? And is there any other group like financials that you see as big laggards that will become addressable when the quality gets higher? This AI thing is making me personally rethink a number of my assumptions that were long held as an investor, but we may be surprised. I think we're going to get some leapfrog effect as we get over this minimum viable quality bar in different domains, because the areas where we're just talking about healthcare operations.

35:36I have not been super enthusiastic about selling healthcare IT over the last decade. Looked at it off and on. Healthcare is a quarter of the American economy. I mean, globally, it's very important to every single human being. It is not particularly efficient. It is especially inefficient in the United States. It is hard not to want to work on, except then you look a little closer at the companies that actually work in this space. You're like, ah, the incentives are a mess. It's super slow. The ecosystem is extremely complicated. There's regulatory capture in every zone. But there's so much to unlock there because the number of people employed and the number of hours wasted that in the administration of health care in the United States that causes a good deal of why our health care is so expensive is really high.

36:26If you ask me about another area that I wasn't excited about in terms of investing in, but generally, please still call me, especially now, but might be as the models get better is government services. These are all huge parts of the economy that are incredibly inefficient, where people are doing analysis and moving data around. The workflows you can picture in a way that make a ton of sense to apply these model capabilities against creatively. I guess maybe my fundamental optimism comes from this place of, well, if we make it a hundred times cheaper, no matter how complicated that ecosystem is, I think we can sell it.

37:01That's one thing that we've begun to see. And the leapfrog effect is the types of businesses and the types of functions that had the most inefficiency can be most ripe for selling new solutions. What's your best guess as to what the market structure will look like for the foundation model providers in like three to five years? What do you think the healthiest version of, Maybe you don't predict it. What do you hope it looks like? We work closely with all the large providers. And that means take our portfolio over to see the foundation model companies, make sure they have research connectivity, co-invest with them.

37:38And we are also first round investors in Mistral. I'm very loathe to make predictions here because AI is just really hard. Anybody who really says that they know what is coming more than six months from now, that doesn't need a large lab is likely. And even so is, I think the large labs were quite surprised by the success of open source over the last year. And so when you say like, what do I hope for? I want a million flowers to bloom because the last mile to humans with more joy and play and productivity in every corner of the economy and every part of the globe, that last mile is really long and there's a lot of them.

38:21And so it's very hard for me to imagine a single company or two or three walking that mile, essentially. And so what I would like to see is you have these amazing assets, you have multiple options, you have open source, and there is enough choice and competition and robustness at this layer that you can have companies built with true economic value on top and in partnership with them. I think that will happen. The alternative point of view is that the race for development gets increasingly expensive. I mean, that's going to happen anyway, but like that it becomes harder and harder for people to keep up.

39:01But an interesting dynamic here is the number of people who know how to train large models is increasing. And the second comer discount is very high. As that expertise is known, as you get to use modern hardware, as you get to not train all of the intermediate models and just try to be at the state of the art with the state of the art techniques. It is a competitive market. The commitment of Zuck to that market makes it also much more interesting. I still think that enterprises have some challenge believing that Facebook is going to be a great long-term partner to them. But the more they work with the ecosystem of deployment and inference and consulting partners, the more credible it is.

39:46And a rich and competitive ecosystem of models means that there's, you know, not total rent collection at that layer and lots of opportunity for many companies to walk that last mile. What's your model of thinking about Mistral versus Anthropic and OpenAI? How should we think about the difference between them? I know one's open source, but like maybe say a bit more about how you think about it, maybe even why you invested early on. Going back to the description of the ecosystem that I think is great for innovation and great for the end user, it is one of faster, more democratized progress and economic value at multiple layers in the stack.

40:26And I think that is Mistral's view of the world. Arthur and Guillaume and Timothy are amazing researchers with a bent toward both state-of-the-art performance, but also a view on efficiency. And again, going back to blindly believing some dominant narrative in the AI landscape is very dangerous. There was a narrative, maybe it still exists, I don't know, that efficiency doesn't matter. Model efficiency doesn't matter. It doesn't matter how expensive it is to run or how big it is. You just want the best performance. And nowhere in the history of computing has efficiency not mattered because somebody is paying for it in the end.

41:07And so maybe the model company can absorb it for a very long time if they've got like a great other business model like ChatGPT, Facebook, Google. But in a vacuum, more efficiency is better for latency and cost and scalability reasons. And so one big potential change in the ecosystem is whether or not people are going to start using a mix of distilled models, models of different scales, putting them in compound AI systems and getting to performance and efficiency. And so efficiency was not a huge consideration amongst research labs and application companies until relatively recently. But I'm sure you've heard this term towards intelligence too cheap to meter.

41:59And I I will make you a claim that there is no such thing as cheap enough because we've been working at compute for a long time as an industry, and it is not so cheap that nobody cares because we just keep doing more with it. There is no company that isn't like, oh, if compute resources were free, similarly, we'll look at this and be like, if intelligence resources were free, we wouldn't do more, but they're not free. I do think efficiency, especially if we start getting to the next generation scale of models, even limits to data center size and power. Thinking about using the resources that we have well, both in training and in inference for every particular application problem is just going to become a much bigger consideration.

42:40You see it now. A year and a half ago, I'd say focus on data quality wasn't considered the highest status part of research. People care a lot about it now because we run up against some limit, which is, well, We took the internet data and improved quality is going to lead to improved model quality. And so I just think the dimensions on which people optimize change over time, I'm very loathe to make any prediction, but I think it's going to be a rich ecosystem. And I'm hopeful of that. We've been told that software is like the ultimate business model for lots of reasons. And if you look at some of the growth for these firms, like OpenAI's revenue numbers are incredible for how young the company is.

43:20It's like this huge revenue business. Maybe the only thing most staggering is like their OPEX, like an incredible amount of cost in these businesses. So far, business models that don't look like the traditional zero marginal cost software business model. Say a bit about margins and the basic stuff and how you think that will evolve, given that so far we've seen these things are freaking expensive to run. I mean, we just had a conversation about how efficiency matters, will matter more and more. I am not particularly concerned about OpenAI's OPEX. The wins behind them, not that they shouldn't care, but I am not concerned that this is an existential thing for them where chat GPT can never be a good business.

44:04It depends on whether or not chat GPT is a differentiated product in market. But the wins behind them look like the rest of computing investing in efficiency against these workloads. And so we're super early. Right now, we're in this crazy, immature part of the infrastructure cycle. So let me say a little bit more about that. Look at the history of where'd you get a server for, let's say, a client server web application. You had it on-prem, and then you co-located it. Somebody gave you like power and real estate and cooling and you put your server there. And then you had hosted renting server and the data center from somebody.

44:47And maybe they're even running an application for you. And then you had virtualization. I'm renting an instance. And the kiddies, I graduate from computer science programs now. If they're not thinking about AI, serverless. I don't want to know that there's hardware behind that. Click button, Amazon, Google, infinite capacity. In AI, we're like step one or two. You are building and deploying your own clusters in data centers. You're in three-year reservation mode. You're managing your own depreciation cycle. And you're like, oh no, are B100s from NVIDIA going to totally destroy the amortization schedule on this?

45:24I don't know. That's a really hard environment to innovate in. And so power to the labs and every company inside, outside our portfolio that thinks about it because they're training on large resources in this super immature ecosystem. Think about the amount of compute that we have accessible in our phones when like a huge ecosystem is focused on the efficiency of getting more and more power into our hands here or in any other machine. Could we use as an ecosystem more competition to NVIDIA? Yes, this stuff will get cheaper if there are multiple players. I mean, NVIDIA is innovating pretty fast, but pushing NVIDIA and then eventually challenging a 90 % margin product.

46:07But overall, I expect the industry to get much, much better at hardware utilization and at efficiency in a bunch of different ways at every layer of the stack. And so when I think about the OPEX of OpenAI as an example, but any other application company, the optimizations that a company itself can do, then in infrastructure management. I'm on the board of this company called Base 10. You want to do inference on GPUs of different types that is serverless and not write all of your own scheduling and GPU failure management and whatever else. You can do that. And then there are people working on everything from memory bandwidth to chips to systems.

46:48The entire ecosystem has innovation that we expect to, as an industry, reap the benefits of over the next five or 10 years. There's no reason to me that won't happen. I can't predict how quickly the cost curve comes down for the same level of compute. And I think it will still feel like you always need more. Think about progression of PCs. You always want a latest processor. But I am not worried that the end margin structure of any application company is bad or any worse in this era. It's not clear to me that that's a structural problem versus the software era. That's probably the best answer I've gotten to that question.

47:28And I've asked a lot of people, which makes me wonder, what else do you want to tell us about infrastructure that you've learned? I'll claim one thing that's maybe interesting to a broad investing audience like you have. There are a lot of people thinking about why is there not a legitimate challenger to NVIDIA? And what are all of the structural advantages that NVIDIA has? And broadly, basic analysis of the landscape would land you at, okay, AMD is working on this and their software stack has not been competitive to date, but maybe they're getting closer. Software and hardware, but weakness in the software stack.

48:05Then you have a series of proprietary chip efforts, sometimes from acquired companies, at Microsoft, Amazon, Google, rumored now opening eye. And then you have upstarts who are trying to do either systems with a particular DGX boxes, like more like a mini data center than a chip, but either like a full system or a chip. And so you have Grok and Cerebris and Etched and Maddox and whatever else. I would love to see most of the industry that isn't a giant holder of NVIDIA, but even some of those would love to see more innovation at this layer. people understand the power of CUDA and the optimization and effort that's been going into that for more than a decade and a half.

48:51One of the core challenges that people don't recognize is we had a portfolio company who used a different chip, not NVIDIA GPUs, to try and train a large scale models. These are X large lab researchers who know what they are doing. And no one is saying that the engineering and infrastructure management work that they did was perfect, but it exposed me to an understanding of what is making this problem so hard for people to attack. If you don't have people running large-scale training workloads at multi-thousand node cluster size on your chips, you do not know if they are going to work. And so this isn't something that you can do as easily.

49:46Okay, we're going to do the test and simulation, test and FPGA. And then we have as much confidence as we can that chip will tape out, chip fits the spec. It is useful. The types of failures that can happen in large-scale training, which is a huge driver of consumption that a lot of people are going after, it seems to, at least with some of the new contenders, only come out at scale. And so that's a really difficult risk to mitigate where I'm like, I don't know, can you do that in SIM? Because it's a new problem. I think that's kind of interesting because then if you look at, for example, Google and their chip efforts, those work actually, because Google has the workloads where they're training on their own chips at scale.

50:30And I would ask you, do we know that the other chip efforts are working? You either need the patient zero or that patient zero to be yourself, because it is a very hard thing to answer without that workload. Doing it physically is really expensive because then you need to go deploy 10 ,000. Maybe that's just an interesting anecdote where we've been thinking a lot about, is there opportunity here and what are the barriers? And love and respect for Jensen and a video, spending time with him as well. I think the barriers are pretty high. Back to like the thousand flowers blooming. It's cool to imagine all of the smart people that are economically and just curiosity motivated to tackle all of this all at once.

51:12It makes you, of course, wonder about like past technology bubbles where lots of money has been spent trying a million things. And then we find our way through and money's lost and money's made. Where with your like bubble hat on, do you have a bubble hat? What does your bubble hat tell you? Talk to us through that specific lens. Oh my goodness. Of course I have a bubble hat. We have skin in the game. I'm a large owner in my own fund. And this is the main thing. If the promise I made to LPs was multiple, we got to go make good on that. I think most people should not be training general foundation models.

51:47Maybe that's obvious. That's a way to lose a lot of money. I would also say as somebody who wants, I do think that there will be multiple players at the chips and systems layer and chip systems networking, the new data center layer. The opportunity is too large. It capitalism works. It will happen. It's just barriers are high. But I don't think that the market is not infinitely deep for another LLM player that isn't particularly differentiated, but it's just a handful of great researchers working on something that feels very general or feels directly in the path of the existing large players. And so I would say companies keep getting funded that are of this shape because people look at and are envious of the value creation at OpenAI and Anthropic and others.

52:43It's the great game. It's really interesting. course, like you and I are interested in that too. But when the entry price is now more than half a billion dollars to do pre-training of something interesting, or at least get into the game of a general language model, that's a really easy way to lose money. Can I ask you the question I asked Martin Casado, which I quite liked his answer to, which is, what do you think the 2030s look like in the world as a result of all of this, intentionally creating like a little buffer between now and then. And we're always so focused on like the next three years, but your kids are young.

53:19My kids are young. What's our kids like teenage years going to be like? I think they're going to be a lot more intelligent and creative. So 2030 is really hard because I can't assign a probability to it, but there is a piece of me that says real probability of abundance. I don't know if we're all sitting on a beach partying in Europe or something. But I say that like a little bit glibly because the ability of society to ingest that much change in productivity is not great. There will be winners and losers. And the job displacement dynamic, I think, is real. I also think it's probably not quite as fast as people are worried about.

54:01So I'm splitting my mind between, okay, model progression happens very, very fast. And we solve a bunch of really important problems. And we have abundance. If I take a step back from that in terms of just optimism of timeline, I'd say the things that I can picture very clearly is the ability to learn and express a capability that would have been a very skilled thing before is going to just be completely changed. Your expectations around creativity of entertainment. Everybody has desktop Pixar. Everybody has the world's knowledge in the form that is most useful to them immediately. Andre is working on this educational tutor thing.

54:48Somebody can become an expert in something at an incredibly accelerated rate because of that. There are a bunch of areas that should market structures be willing, buying processes be willing magnitude of improvement being enough become a lot more efficient. So I don't think I have a clear version of the future. I can picture a lot of really amazing individual experiences. Is there any commercial or product thing that you're surprised we haven't seen yet that you're like got your eyes peeled for and it hasn't happened and you're surprised it hasn't happened? I'll describe two opportunities that I'm excited about.

55:25And one of them, I know why it doesn't exist yet, even though it's so big. The vast majority of the time, incubating companies is a terrible idea because my foreign partner, Jerry, used to say there's no such thing as inception. People have to live the problem and the ideas and go after and make it their own. The entrepreneurs that we want to back, they have the agency to go do it. But one of the places where I think this is not true is where essentially the context and capability to build a particular type of company rarely come together. This is one of the things that is so powerful about Harvey.

55:58How many times do you get somebody who is going to be entrepreneurial and forward thinking about workflow, that they are a lawyer working 90 hours a week at a good firm and are still like, can I make Chachupi T do my job? And then his roommate is a great researcher. It's uncommon because you ask your average researcher at a lab, what does the legal workflow look like? That's not part of their life experience. And so one of the really interesting opportunities that we think about and actually do support incubation around is places where the context of what people are doing that could be automated or improved is so far away from the engineering and research community that understands the capability at the state of the art today.

56:41So an example would be there are many billions of dollars spent a year on configuration, monitoring and maintenance of enterprise software systems, ERP, HR, CRM, workflow systems like ServiceNow. That gets done by large consulting shops in the US, in India. And I think if you asked, if we look at the capabilities that are coming in software generation, in code generation, I think some of these tasks are pretty tenable. Or you could at least create a services org that has a very different margin structure than today's orgs. Better margin structure, better SLA, better end user experience. maybe change the industry structure of software because this configuration and maintenance and depth of embedding in a particular enterprise system is the thing that makes it so sticky.

57:38The fact that you configured SAP 20 years ago, it's just all your business process you couldn't possibly get out of it. If you can replicate in a different system, that's a very high mountain to climb. But pieces of that that could change the industry of software, enterprise software, That's really interesting. But if you think about all of that work, that type of software engineering work, that's considered very low status work in Silicon Valley engineering circles. What's happening over there? Who's writing configurations of this platform? It's so far away. But if you think about the economic value and what technology is actually super important to the largest companies in the world that they feel like they can do nothing about or they can't improve, that's kind of it.

58:21That's, to me, the type of underexploited opportunity where the domain is so far away from the technologists. Even that's still software, never mind healthcare operations or government administration or call center operations or something like that. And so I think there in large a little bit of the opportunity. Another one that we're just really excited about in terms of potential value for society, and clearly the data set is not generated and owned at the large labs today, is material science as a domain for foundation models. But if you ask me, like, why doesn't that exist yet? Well, I think you need to have the right talent, you have the right thesis on data, hopefully some cleverness around efficiency of collection of that data.

59:07Would you get excited by some sort of pitch around, we're going to create the next Accenture, but built on top of... Basically, the question is like, would you get excited about a services business that just was built around using and deploying all this technology to make the offering better and faster and higher fidelity? Or a service business is just too lame of a business model for you? The answer is yes. But what would get me jumping out of my seat excited would just be somebody with the relentless ambition to figure out if we can make the margins as good as a software business. And let's think less about margins, more about scalability, because you can be like, okay, fine.

59:48Why are services businesses lame? Well, because you're fighting on human capital quality and volume. You can only scale so fast. It only gets valued X multiple in the public markets eventually. It is only so profitable. and hey, like to some degree, because you can't really tell, it's a brand game in the end, go with a big consulting firm. Those are some of the reasons people don't like services businesses. And so for a story I could believe around scalability and ambition of scalability and all the things that come out of that, like how much of it is a technology business? Yes, the answer is high enough, yes.

1:00:25That's true when you look at some of the most interesting application layer companies today. They may not frame themselves this way or even be consumed this way. And that might be the difference. What we're talking about is consumed as a services experience, but they're replacing things that were services before. Is there anything that scares you a lot? The things that are interesting to be afraid of, bio risk, runaway model that is optimizing paperclip production, China weapons system or something. I think we should go figure out if those risks are real. But I actually think they're like a huge distraction from the near term abuses.

1:01:08And so for you and I, investors in technology, technologists themselves, the ability to adapt to new tech is a built skill. But it will not be true across the entire world that people adapt very quickly. And so there are some incredibly basic abuses where I would be excited about solutions. and it's very easy to say misinformation and fraud, but I think those are quite large. I'm more oriented toward, wow, we have like a lot of abuse that's happening already with older technologies that just gets so much more amplified, gets much cheaper with leverage of this generation of models and improving capabilities.

1:02:00We should address that pretty immediately and in a sophisticated way. Is there any debate in this field that you find really interesting, personally? This is a little bit in the weeds, but I think the debate about how you improve multi-step reasoning in a general way is open question amongst the labs. And this is like a very fundamental question of where does the next step function of intelligence come from beyond just scaling? Because on both a compute and a data side, that is more difficult than it was in the last generation. People have varied points of view on this in the different labs. What are the sides of it?

1:02:40Like, how would you describe the sides of that debate or the different opinions? Labs and upstarts. I will describe one directional effort that people are investing in, which is I just had Oriol Vinials from DeepMind on our podcast to talk about this. But does it make sense to invest in math and computer science as a domain where it feels closer to pure logic? You are doing multi-step reasoning. And in terms of improving model capabilities specifically against this domain and using generated data in this domain as new training data for the model, lots of people are doing this. There's a contingent of folks that say that is not a general approach.

1:03:28Human reasoning is much richer than that. And so looking at code or looking at math, translating problems into lean and verifying them, that's to some degree a dead end because we cannot do this with the rest of think about the qualitative reasoning that you do on an investment. There's no proof. That's one open question of are there more general answers to improving multi-step reasoning that are not in a particular domain? because some people will look at code and math as an extension of RL and games. And is that the right path forward? I think lots of people are interested in it. It's like much less clear consensus in the research efforts than more scale, more data, human expert data, and better quality of data.

1:04:18Improvement of the quality and the pre-training as well. Do you think there are any important, I'll use the term, crossroads that we're at as industry in AI today? Or we have to go one direction versus the other? I think a point of change now that's kind of interesting is at the scale of clusters required for the next generation of training, these are for training essentially like single use, gigantic constructions of CapEx. And they're not easily extensible to the next generation. Obviously, you can use those for training smaller models, experimentation, inference, whatever. But the idea that we are just going to build larger and larger data centers consuming more and more power in order to improve model performance feels like it has some limits.

1:05:13And it probably goes back to the discussion we were having before of efficiency has always mattered. And maybe there is increasing investment in some sort of modularity or that efficiency of training matters in a way it hasn't in the past. Because that is when you start talking about the multi-billion dollar training run that gets you to state of the art for some period of time. It's a hard capex investment. If you could commission a research paper, like a crazy well-done in-depth research paper on any topic in AI, what would you pick? Three domains that I think are interesting. I can't choose one.

1:05:57But one is, the problem is nobody's going to write this research paper because it's all like proprietary knowledge. That's okay. You have a special magic wand. The comparison of generalization of multi-step reasoning, I think is an interesting area. A open question is, we have all of these benchmarks, exams that humans in different domains take, MMLU or a legal exam, a medical exam, etc. We are surpassing the benchmarks in a bunch of different ways where it begins to open the question of how do we do evaluation of these models as they are super intelligent, not in any super philosophical sense of the word, but only in the sense that they are better than our human experts at a particular task.

1:06:44And so human eval becomes hard. How do we understand progress as we go from there? Something like ARC, would that satisfy what you mean? That's one version of it. I think another example of an idea is, have you ever heard of the term centaur play? If you play chess? Oh, yeah, yeah, yeah. Sure, sure, sure. A lot of people in AI don't believe in this idea anyway. One theory of it would be Patrick with a model can come up with better answers than Patrick or the model, that will be the bar for human eval for the next set of models. But that's a version of it. But it's an interesting question. How do we continue to evaluate models as models progress?

1:07:25Understanding the different theories on reasoning traces leading to model self-improvement. What is it in your personal experience that has led to you caring so much and willing to work so hard in this field? This is the most important change to happen in technology in our lifetimes. And so if you believe the impact is very large, not in like a religious AGI way, but if you just think the opportunity for productivity and abundance is very large and you can create new economic platform players. And the landscape is so open because your opinion actually really matters now versus if you were investing in SaaS, like you and I were in like 2015, there were incremental discoveries and understandings about companies and technology that did matter.

1:08:19I'm not trivializing that, but it was at a very different scale. If software 1.0 was about human engineers writing explicit instructions and source code, And that was decades. And then Andre Karpathy wrote a essay about software 2.0 being like replicating and finding in search space a specific behavior from a data set and a neural network and then training the program from the data set. This is like quite a lot of work to do from scratch. I genuinely think of this era as a new era of software. We are manipulating models of different kinds where a bunch of the work has been done for us. The foundation models are so capable.

1:08:56And if you can build intelligent systems to do increasingly useful work, how could you not want to work on leverage these superpowers in every domain? Maybe do it better than humans in areas that really matter, healthcare and science, but also in daily work and taking the operational toil out of every industry. And so it's just really interesting. It's an ambitious era. I think that's super exciting. From a personality perspective, we had some founders doing references on us in an investment context recently. They walked away convinced that we'd go to work for them. And they're kind of like, oh, people are surprised.

1:09:35This is like why you're trying so hard when you don't have to. And I'm like, oh, anybody in technology, like above subsistence level, what does have to mean? It's like the funniest observation to me. I guess my reaction to that is, of course I have to. What else am I going to do? Let the world pass me by, be number four? Maybe they mean it like financially. Oh, like, doesn't money really matter? I'm like, no, but it's really fun to be right, go compete, work with great people. The discovery for me is maybe this resonates with you. The motivation of working with the most interesting and motivated and high leverage people in the world and helping them be a little bit more successful and being right about that over a long period of time.

1:10:15Of course, you have to. Hey, man, this has been so much fun. I think, you know, my traditional closing question. What is the kindest thing that anyone's ever done for you? There's too many to answer here. I'm very grateful for the number of people that have given me opportunity is not an acceptable answer. I'm going to name three people. The first is, I mean, you have my husband, Pat, on the podcast. He's my favorite person in the world. I'm like, oh, thanks for asking me to marry you. Like, that's great. Right. I just appreciate that. Got very lucky. And then I'd say two people that I will be eternally grateful to for different reasons.

1:10:54One is Anil Bustry, who used to be a partner leader at Greylock and was the founder of Workday, an amazing technologist, where I think in Anil's view, he was like rescuing me from finance at Goldman. He's, ah, at one point, like I was at Morgan Stanley and you like belong back in technology. And I'm like, I know, like I'm a technologist. I'm just here learning about the business side for a year and I'm like, I'm going to go work at Stripe or something. But the thing that Anil did for me was just say, working together for a small amount of time, like having complete conviction. And I did too, but you have potential.

1:11:27We don't know what we're going to do with you. Just come work here. And so it was like raw goods bet of an opportunity where I'm like, I haven't done anything interesting in the technology world. I am a kid from Wisconsin who is working in banking, who happened to work on your IPO. And it just takes like a certain conviction in your own judge of talent for somebody to be like, oh, come work here when on paper. Who cares about your profile? I'll forever be very appreciative to Anil for that. My former partner, Ashim Chanda, is legendary enterprise, in particular, security investor. But one of the things that I seek to emulate in Ashim is his ability to work with earlier career investors really comes from a place of amazing personal confidence.

1:12:16One more sentence about this. If you could be productive for... I think Sheen was actually like a little bit skeptical of me when I came in, which is totally okay, right? Trust is earned. But he became a really big sponsor for me. The attribute I tried to emulate is give your people as much and perhaps a little too much opportunity if they prove they can take it. when I say confidence, he was incredibly open with his network. He was incredibly meritocratic, where I don't mean to be cynical about this, but there are many people who would have been like, I'm investing in enterprise security. And you as a 24-year-old, really young looking girl from no particular interesting accomplished startup background, yes, we're going to go do all of this company building work together.

1:13:07And I'm going to take your judgments very seriously, very quickly, I'm going to open my world to you. I think that's an amazing thing to do for somebody. And in terms of giving the people that I work with opportunity, may we try to be like that. What a wonderful trio of closing thoughts. Sarah, thank you so much for your time. Thanks, Patrick. If you enjoyed this episode, check out joincolossus.com. There you'll find every episode of this podcast complete with transcripts, show notes, and resources to keep learning. You can also sign up for our newsletter, Colossus Weekly, where we condense episodes to the big ideas, quotations, and more, as well as share the best content we find on the internet every week.

1:14:07Thank you.