EP 121: Snowflake CEO Sridhar Ramaswamy on the Future of AI and Data

0:03Welcome to the Logan Bartlett Show. On this episode, what you're going to hear is a conversation. I have a CEO of Snowflake, Sridhar Ramaswamy. Sridhar and I talk about a number of different things, including his transition to leadership at Snowflake, how he's been able to make operational changes to the business to help them execute on shipping product faster, his predictions for the future of AI and where Snowflake sits in that world, as well as how his leadership style has changed from the early days of Google, where he led a major business unit, to Neva, the company that he co-founded, as well as today with Snowflake.

0:38A really fun conversation with a number of different predictions about how AI is going to play out in the future that you'll hear now. Thanks for hosting. This is a cool room. We're in the, what do you call this, the gaming room? This is the game room, yeah. Game room turned podcast studio. It's a hybrid. You don't typically see that. So you've been on board now as CEO for 10 months? 10 months, roughly. And how long at Snowflake? Like, let's see, joined the day after Memorial Day last year, so like 17, 18 months. 17, 18 months. But, you know, especially because you're inside, it's a little bit of an awkward process in that it was not like a formal offer letter.

1:20And so you're sort of like waiting. Guys, are we doing this? When are we doing this? Yeah, when are we doing this? And I also had a few other opportunities that I had to go say no to. And, you know, opportunities don't come all that often. And so saying no to some concrete things while saying this will likely work out, super awkward. So in the original plan, so Snowflake acquired your company and your original plan was to stay on for six months. That was my deal with Frank. that I would stay on until the beginning of December and then go figure out what I was going to do with my life. Was it at the six-month mark that you made the decision to hang on a little bit longer than that, or was it somewhere earlier than that?

2:08I started looking, I would say, I started talking to a few folks October-ish last year. These things take time. Senior roles don't come by that often. What was your initial mandate in that six-month period of time? I was SPP of AI during those six months. And I joke to people that I was like a cabinet position without an actual portfolio. You were kind of an IC, right? No direct reports. No direct reports. But roughly the mission that we set for ourselves was to make sure that Snowflake was kind of an A player when it came to data and AI. It was really to craft that product vision along with Vivek, along with others that came like Barish and Asim about how do we bring AI front and center to Snowflake in a way that stayed true to the product principles of Snowflake.

3:17One product, everything is tightly integrated, stuff just works. And so that was the bar that we set for ourselves. And so it was a lot of product design. And then there was also a lot of talking to customers about AI, what was coming. So that was roughly what I did. And so now, I guess you're 18 months into this AI journey within Snowflake. I'm curious, how would you describe where Snowflake is in AI today? First of all, I think of AI as a horizontal capability with broad application. It's a little bit similar in that sense to, let's say, like the mobile change, where every company that was involved in software, especially if you're a consumer company, knew that you better figure out what mobile meant to your business.

4:15And a different way in which I crystallize how it is a horizontal is by saying that for pretty much the first time in sort of computing's history, there is now a seamless fluid way to go between different kinds of data. whether it is unstructured text to extracting structured information from it or going from an image to the structure of images, same with video. Things that honestly that you had to do software engineering projects for or you had like specialty areas in computer science that just went and did one thing, all of those capabilities are now exposed to you via AI models. So to make it super concrete, let's say we have a health provider as a Snowflake customer, and let's say they have a bunch of clinician notes from years and years of doctors talking to patients.

5:18If you wanted to ask a question like, oh, make me a list of all patients that had, I don't know, cough symptoms, that would be like a custom job. You had to go find some software engineers. They would figure out what model to use. They would tune it. And then maybe three, six months later, you got like this other table that said, okay, this is the list of patients that had cough symptoms. Now a 17-year-old kid can write a little prompt and then run a SQL query on Snowflake. That'll be like, okay, here's your list of patients that had cough symptoms. And so first we said, we want to bring the power of these language models and make it available to anyone that touches Snowflake.

6:00And that is Cortex-AI. It's the model garden, but it's integrated into our Python environment, into our SQL environment, so that you and I can just write SQL and underneath its AI, you don't need to know about where and how the models are hosted, how fast, how slow. We take care of all of that. It's that baseline of capability. And then beyond that, we think in terms of what are key primitives that use language models, that use AI models, but provide additional functionality. So, for example, we implemented search, and this was tech that we had developed at Neva, but it's now part of Snowflake, where you can create an index on, say, a bunch of documents.

6:43We take care of things like chunking it and generating embeddings, being able to do fast retrieval, so on and so forth. But that's a key primitive that makes chatbots possible. Similarly, we have another primitive for how do you go from spoken text or written text to getting back structured information. And so our vision very much is that there is going to be this base layer. There are important primitives, and people can mix and match them to do all kinds of interesting things. They can build applications. They can do data transformations. And those are things that are still evolving. Absolutely, you can do agents, but part of what we're trying to figure out is like what is the boundary of where an agent is still reliable.

7:29We place a lot of emphasis, for example, with the chatbot. If you want a business user to use it, you want to be able to tell them, you know, you can engineer this. Like it'll be right 99 % of the time. And if you make a change, you can verify that it's still right 99 % of the time. Same goes for the analyst. And so, but we think that by building very, very reliable primitives, we unlock the power of AI for business applications. And so I guess as you think about instilling this DNA within the organization, or I guess it's probably pivots too strong of a word, but distilling down a focus on AI, I'm not sure how you would phrase it, what the right phrasing is.

8:07How do you actually do that from a change management standpoint within the organization? You know, when the Neva acquisition happened, we, first of all, you know, made sure that the team and us, me and Vivek, my co-founder, were very clear about sort of our mission. We shut down Neva, the consumer product, and we told our team, our goal is to make snowflake success. Okay, to me, that part is important. where everybody knows that you don't have an agenda, you're here to make things better. And then, you know, the founders, you know, Benoit Thierry, Christian, our product lead, were very cognizant of the power of AI.

8:55I mean, at the end of the day, you know, you don't have to be a rocket scientist to internalize what I just said about AI is like a magic data transformation tool. Like, they get it, and they're like, we need to make sure that it's available. But much of the work that we did, to be honest with you, was a nice and to what the rest of the Snowflake team was doing. It was not like that or this. We very much said, this Cortex AI makes Snowflake SQL better, makes Snowflake better. These operators make it easier for people to use the data that is in Snowflake. And in fact, the thing that I tell our customers is, you know, listen, Snowflake does not compete with OpenAI.

9:37when it comes to foundation models. Let's face it, you're simply nowhere as capable as them. But on the other hand, if you want to use the power of AI to create a chatbot, to do data transformations, to get access to structured data, we'll make it dirt simple for you. And this message of take Snowflake with all of its glory and power and enhance it with AI, it's like there's a positive cultural message. It's not us coming and saying, you people don't know what you're doing and let us show you how. That's the way in which a startup fails. I've had plenty of founders that have come into my teams with that kind of attitude at Google.

10:15And so in many ways, all of the dumb stuff that we did at Google were like good lessons for us in terms of how do you approach change culturally? You know, organizations adapt. And the Snowflake team very much sees this as a yes, and. And that's a message that goes well. At Cortex AI, have there been any non-obvious ways that people are using it that have jumped out to you? Anything you would point to? People are endlessly inventive. And it's the composability that's also super, super cool. We have run into applications, the obvious. Siemens using us because they make 100 ,000 devices and searching through PDF manuals of 100 ,000 devices.

11:03As anyone that's looked for a washing machine manual knows, kind of annoying, kind of hard. In some sense, that's an obvious sort of use case. But even internally, people have done stuff like, oh, we can annotate use cases sitting in Salesforce. This is just like we record every new thing that we want to launch within a customer as a use case. And tagging them with the right product, with the right feature is error prone. And also sometimes our sales teams don't do it. But what used to be difficult before and manual tagging, as I said, it's now like a single SQL prompt to be able to do this. We have hedge funds that are using it to grind through analyst reports.

11:54and say, hey, if this company gets mentioned in this report, we will be able to extract it. There are lots of different ways in which people are mixing and matching what you can do with different kinds of models. We also have great partners like Landing AI, and that is sort of truly magical because they run within our security perimeter, but they can do things like analyze images and extract structured information from it. it's that sort of composability that is incredibly cool to see. You actually launched Arctic, which is your own large language. It's our own MIE Foundation model, and it feels like we did it like 100 years ago.

12:37Is that right? When did it actually launch? I think it was March of this year. Okay. And how much of that was, I guess, a customer pull versus the ability to go in and get some native perspective and increase your bona fides in building your own large language model? 100 % the latter, which is, you know, like, I think that actually knowing how to use language models is a, I would say it's almost a required computing skill for any meaningful, any meaningful, like, infralvel company. Just like, you know, at like the turn of the century, like dealing with big data, like data that could no longer fit in a machine.

13:22Like, you know, you kind of needed to know how to do that. We thought of Arctic very much as what will it take to train a modestly sized model. Okay. We have talked about it. We spent a few million dollars on it. Again, nothing like the billions that are being spent now. But I think it was a very good experience for us. And it also taught us a lot about what does inference mean? What does inference optimization mean? And those skills continue to come in handy. We have folks from teams like VLLM, which pioneered a lot of efficient inferencing work. And so I'm pretty happy that we have expertise up and down the AI stack.

14:05You guys sit in an interesting nexus point and support a lot of different models, including Arctic, but Lama and Mistral, and you have a bunch of great partners, Rekha, a bunch of great partners or solutions that you all work with. Is there anything you're seeing that's interesting on the model selection side or how end customers are kind of choosing to work with different models? I would roughly say for very, very common functions, I don't know, like sentiment detection, people know that small models work just fine and they're sort of happy to go with defaults. It's very inexpensive. But then I would say that when the requirements are a little bit more stringent, people like the name brand markets.

14:51Interesting. Which is sort of, it's a weird setup. I think it's the reason why, for example, OpenAI continues to get like heavy, heavy, heavy usage. Yes, they make the process of working with them easy. But I think people like to go with something that is known and trusted, even if it's just a smidgen beyond what you actually need. But why bother figuring out what you actually need when you can just use the name? Yeah, the default. It's actually an interesting point that I've seen for sure as Anthropic or whoever it is will launch all these new solutions, models, and pick up a percentage point here or there.

15:33But it actually, people just seem to default. I'm sure you saw it in search. human behavior is defaults in some way. Defaults, just like they are very powerful. And it's surprising, but yeah, there's first mover advantage sort of even here. As you sort of look at people trying to get applications into production or leveraging all these different models, what do you see as the biggest pain point in the stack or the value chain to actually get the value or the potential of a lot of these things? I think there is still a big gap when it comes to here is an engineered solution. I think, and I don't think the kind of AI industry, if there is such a term, does itself any favors by simply not talking about things like hallucination rates.

16:26If you look, no one publishes hallucination rates on their models or on their solution. It's like, look, you're so cool. You should just use us. But I think once you, you know, I was actually showing my board members an app that we have that we can ask questions about our financial data. It's pretty cool because I can say things like, hey, how did James Petter, who is our JVP of EMEA, how has he been doing over the past seven days? and it'll give you a table. It'll run the SQL query and show you things like we can make growth and year on year growth and apps, you know, numbers and so on. But before we give it to my CFO, Mike, you know, I'll tell the team, you better be sure that you're doing the right thing.

17:19I think it is that engineering mentality that is still a little bit of an issue. And so the more, part of what we end up doing is educating our customers on, okay, so you need a release process. You need to actually be sure that something like 95 % accuracy is okay for you. Or if you use a chatbot, you need to make sure that you provide the citations or place in the document where you can go verify the answer. Because business use, people take seriously. You can't make mistakes, especially when it comes to critical applications, for example, like the one that queries revenue data on which people can actually make decisions.

18:06I think it's stuff like that, that, you know, continue to be obstacles for broad, broad use. But on the other hand, I think AI has a ton of use cases to solve problems where you will happily accept a certain amount of mistakes. So, for example, if, I don't know, somebody, people send articles all the time, right? I find it annoying. It's sort of virtual signaling. They're like, here's a great article you should read. You know, my head hurts. Why another one? So oftentimes, I'll copy the text in the article, paste it into Cloud or into ChatGPT, and be like, you know, can you give me a summary? If it's off a little bit, I don't really mind.

18:55It's not a huge deal. It's not a huge deal because just the time saving that I get from not actually having to crank through the article. there's like so much value. Similarly, the example that I gave you of my team, I think they updated 5 ,000 different use cases just by running a SQL query, figuring out, you know, a bunch of product categories and updating the underlying table. I could never have persuaded them to do this manually. And they're like, off by a little bit, it's fine. Do you think for mainstream enterprise or business adoption we need to get to a rate of, I mean, is it 99 %? Is it 100 % for it to actually be trusted across an enterprise?

19:37It depends on the application. So for example, with something like a finance chatbot, I'd be like 100%. Please be accurate or begin to think about how errors are evident. Remember, the insidious thing about hallucinations is not that the model is getting 5 % of the answers wrong. It's that you don't know which 5 % is wrong. And that's like a trust issue. And I think so it's going to depend on the application. But the good news for all of us as technologists is that there are just a whole pile of these applications where AI will do stuff that you and I have like zero patience for doing. I love the little, I'm a Gemini subscriber as well.

20:27And I love the little button that I get on my Gmails that says summarize, because for things that I want to read quickly, I will happily take the few problems that I run into. So I think there are lots of places where the stakes are not that high, where AI is already useful. But I think it's also important that we understand that there is a software engineering aspect to how do you create reliable applications in quite a few contexts. When you look off in the future, and I don't know what time horizon you sort of think about with the artificial intelligence industry, it's obviously moving pretty quickly.

21:12But is there a point in time you pick and say, at this point, I want Snowflake to be known as XYZ in two years' time or five years' time when you're sort of setting the vision for what's possible within Snowflake with regard to AI? Our aspiration is to be an end-to-end solution for data. We want to make it easy for people to ingest data, to be able to engineer it, because data that is collected at source systems, for example, logs, logs will never, you know, logs will always have IDs. Why? Because there's a lot of them and you don't want to be putting strings into it. And so we want to be there when people sort of engineer data, make it more usable, and then obviously when they store it and want to run analytics on it, or when they want to get insights with a dashboard or an AI tool.

22:07I think we are well on our way to having AI, first of all, play a role in each and every one of these. How do you make it easier to write a SQL query? How do you make it easy to set up a pipeline? How do you help people generate metadata for a table or update metadata for a table? I want to write a description for a table that says no one. like you just don't like doing stuff like that. How do we have tools that make it easy? And then, yes, how do we have business users get access to the data that is in Snowflake? You know, my take is that just like you and I don't really think I'm doing this on my phone.

22:47Just like do it because it's natural. That's my aspiration for AI with Snowflake. If you feel like you need to do something and AI is the way to do it, you don't actually think about it. You just go do it because it works out of the box. So to set it another way, when we stop talking about AI and it's just embedded into the process of which we do. And it's just embedded into what we do. To me, like, that is actually huge. There are just a whole set of things, again, like in my own work life, personal life, where AI is just embedded. Like, I don't think about it. I'll give you, like, any number of examples.

23:21If I want to generate a meme to make fun of someone or to just say something silly, I don't think twice. I just go write, I'm very good with words. I just go with like, you know, write out a bunch of words and out will come beautiful memes that I can just like text someone. You don't need to think about it anymore. Similarly, like I never ever write sheets or Excel formulae anymore. I don't know if you like writing them. Total pain. I always get them wrong. I just write English descriptions. And these models, all of them are magical at generating code to be able to do stuff. like that well. And so I think it just like blends into the background because a whole lot of especially soft programming things that like you and I always do, it's like so much easier.

24:09You obviously lived through the Google and the internet transformation that society kind of went through. I'm curious as you sort of look at the potential of AI as it currently stands and the the hype that's around it. Do you think we're at a technological point that the hype and the possibility are commiserate with one another right now? Or do you think that we need to make some substantive process on the models or whatever it is to actually meet where we are from a hype standpoint? My first answer to that question is like, I've been around enough to know that there's a lot that's unknowable.

24:48I ask people, like, you know, could you reasonably have predicted Uber from the beginning of the internet? Or even, like, you know, when the BlackBerry was around? It's pretty tough, right? And so I think there's just lots of stuff that is quite unpredictable. But I think that the predictable parts of AI have to do with what I said earlier, which is data transformations of different kinds, new kinds of interfaces. And it's a pretty easy and reasonable prediction that every app on our phone or perhaps the phone itself is going to have a very different interface, like much more of a voice interface.

25:34I mean, think about it. Even the act of saying, I want to go to this location and I want this kind of a car and once I get into the car, like I'm going to text my spouse and say, this is when I'm going to show up. It's like, that's the kind of stuff that is simply going to change in a pretty dramatic way, all driven by voice. I think that kind of, you know, data transformation is going to be different is an easy prediction to make. And I don't think it requires sort of rocket science technology. It's application of like existing models. Yes, with a lot of sweat to integrate them, and put them on the phone, figure out what is running locally versus what is central.

26:17So I think those are the easy predictions. On the other hand, will we evolve into this world of autonomous agents everywhere doing things on our behalf and arguing with each other and suing each other on our behalf? I think that's a little bit harder to predict. I think it's an area that's just evolving so rapidly that some of those things are hard to predict. My favorite one is like, what if you had an AI that was designed to get itself more GPUs so that it could train better versions of itself? Very referential, self-referential in there. Yeah, it's an interesting one. I'm curious, I guess a lot of the companies we referenced, be it Uber or Google, we sort of ended up in a fairly consolidated power or at least like user from a user standpoint.

27:16I'm not sure if the market splits and how it all broke out. Do you think we're headed for more federation in the large language model than what we saw in other technologies? Or do you think we'll sort of follow a similar trend? And, you know, earlier this year, like, I was actually hoping that we would end up with dozens of model makers in the world. Okay. As we get to the close of this year, it looks like, huh, three, maybe four. It's kind of sad. Yeah. I actually do sincerely hope that it goes back to dozens of companies. I just think it's healthier. Yeah. I think whether we like it or not, concentration leads to, yes, it leads to great companies.

28:08I work for Google. They paid for my house. It's like magical. But on the other hand, you know, I think of effectively call it Google, Facebook, and Amazon as kind of like basically a tax on our attention. all around. Like they sort of, just those three companies, you know, collect tax on how we get to information, how we get to entertainment, and how we get to products. You know, like, I just think that more competition is helpful in terms of how to make more of the benefits of technology flow more broadly than what can happen when there is such heavy concentration. Yeah. Yeah. I mean, it sort of comes down to antitrust and consumer harm and all those definitions, because Google, actually, you benefit in a lot of ways for Google being a better, more people using it makes it a better product, which makes it a better service for me.

29:12And Uber, same thing, the density of, but that's in the near term. Over the long term, who knows? I think competition is probably a good thing for all those businesses. That's correct. That's right. I mean, anytime there is massive concentration, I don't think broad society benefits. A small set of people that are part of the company, let's face it, you and I are sometimes part of them, and we're very happy that we are part of them, benefit. But I say that's more of a wish. And these things also seem to change every six months. So, you know, in another six months, hopefully we'll have a different prediction.

29:46Boomerang background, yeah. Yeah. I'm curious, when did you start diving into AI in a meaningful way? I've been in and out in the sense that I wasn't part of the team that created the first, it was a logistic regression model, except that the team that built it didn't know that they were building a logistic regression model. They were just like strung together a feedback loop and said, we want to estimate probability, and here is how we can do it. But I got to be part of the team like probably early 2007, and so I've lived through a number of revisions of how that product evolved. So we were doing 15-minute updates to these models back in 07 because we had to deal with actual problems like massive spam attacks.

30:42We'd have these bots that would just slam us with clicks. And so if you didn't react fast enough, the PCTR models would just go stir crazy. And so you had to do real-time updates. Or we would have launch events where a new product comes along, massive interest. What is your problem? PCTR model do. So we learned a lot. My team also launched the first generation of deep learning models for logistic regression. This was, again, 2011. You know, but by that time, I was also more involved in the business of advertising and commerce and less on the technology of it. But even by 2015, 16, you could begin to tell that there was magic in the air because that's when models started doing completely strange things like being able to translate from, let's say, like English to German while being trained only on translations from English to French and French to German.