Kumo’s Hema Raghavan: Turning Graph AI into ROI

0:00If you have your data laid out as relational tables, a coma just sucks it in. So you just specify and through connectors, tell coma what your schema is, and then you can just start writing predictive queries. So the graph is abstracted away. But if you have someone like a data scientist who loves tweaking the neural network parameters, in case Constantine is using a positive index. Exactly. You look out guilty. Exactly. You can look under the hood and the analogy we always use is we'll give you the self -driving car. But if you want to look under the hood or if you want a drive stick, we'll let you drive stick.

0:59We have a brilliant guest today on training data. Welcome, Hema Rogovon, co -founder and head of engineering at Kumo AI. Hema brings decades of experience leading AI initiatives at LinkedIn. She came up with the people you may know technology and other core features that leverage the power of graph learning. Her journey in AI predates many of the technologies we all take for granted today. She was working on NLP before BERT was even a thing. With Kumo, Heyman, her team are revolutionizing how companies harness AI by making advanced graph neural networks. These neural networks let you do auto -enail, automated machine learning on any platform, from snowflake to Databricks.

1:46Kumo's innovative approach allows companies to leverage their existing data warehouses in order to build sophisticated AI models faster, cheaper, easier. You don't require the deep expertise in graph learning or maintaining complex features. You can just go straight to business value. Welcome, Hema to Franning Data. Today we have the amazing Hema Rugovon. You are building Kumoei, which is AutoML on the Data Warehouse using advanced neural networks and graph neural networks. AutoML was incredibly promising a few years ago. is a major trend in the last wave of AI five, six years ago. It went through a little bit of a trough of disillusionment, a lot of the AutoML players receded from the forefront, and companies started to store their features and feature databases and the like.

2:39Why are you focusing on AutoML? What's different about Kumo? Okay, so there's AutoML, and then there's AutoML on GPUs. And I think that's the big difference for Kumo .AI. And let me give you a little bit of an example from my own career. So I started in NLP and when we would build systems back in the early 2000s to answer a question like when did Marco Polo land in Asia? We would be encoding features like Marco Polo is the subject of the sentence and it's going to be the subject of the answer and all of that. So we had to know a lot about language, about linguistic structure and so on. And then the GPU revolution came that enabled neural networks to come at the forefront of this technology.

3:28We don't write features like that anymore. Those intermediate layers in a neural network, really learn the parts of speech, the name densities, all of those properties of language. it's the same in other classes of problems. So in the class, the AutoML that was happening maybe a decade ago, we were looking at CPU -based models. So think of logistic regression, think of XGBO's SVMs and so on. And all AutoML did then was parallelize what a data scientist would have done, which was a lot of hand -computed features. And that required you to be, you had to write code to think like a data scientist.

4:15So you were trying to get the machines to think like humans. Whereas here, what we're doing is we use GraphNural Networks, so it's a neural network technology. And you can think of a GNN as a superset of a CNN which is used for images or a sequence model which is used for languages. GNNs, you know, allow for arbitrary structure and the GNNs are learning all of the features that you would normally use for predictive problems. So, Kumo, it's in the space of predictive AI and we're early bringing transformer technology to predictive AI problems. Can you say you mentioned GraphNural Networks and you gave it great explanation?

5:01Can you explain to me like I'm five years old because that might be where my level of understanding is? Like, our GraphNural Networks good for any class of problem, is it good for, you know, you came from LinkedIn where you were working on, you know, the social graph of LinkedIn, is it good for specific types of domains? That's a great question, Sonia. So let's say you're going to put this podcast episode out and, you know, it's going to be on some video streaming site. And we want to recommend the relevant podcasts for users of that video streaming site. YouTube to be explicit. Someone's watching this on YouTube.

5:40You want to recommend someone to watch it or not. Exactly. So user logs in. And you not only have the content of this podcast episode, But you also have what you might have watched in the past. So you can think of that the records of what you watched in the past are sitting in a views table. Collaborative filtering era. Exactly. Exactly. But the difference with collaborative filtering is it's just looking at views. How can we take, you know, the view data. So the view data is a network. So coming to Sonya's question, right? there's a podcast episode. There's all the users who are watching it. So you got it in terms of you know you have a bidirectional graph, the users and the podcasts.

6:30But then you have the organization, you have Sequoia Capital, you have the channels from Sequoia Capital, you have other metadata that you may have in. So all of that can lend itself naturally to a graph and start thinking about links across these nodes of a graph. And effectively, what a graph neural network is learning is let's look at what Sonya watched in the past. It seems like she really likes AI. AI and baby shark videos. Okay, so AI and baby shark. But and then the neural network also learns that Constantine likes AI and what would it be for you? Probably AI. That's that might be the end of.

7:17Just AI and AI. Like history. That's great. So there's AI and history, right? So the neural network can learn that there's an overlap between both the few on the AI pieces of content. You both engage a lot with Sequoia content and it's learning across this network, right? But the next one, so when you're watching a baby shark video, we don't want to be recommending that to Constantine, right? So how do you take that content that you engage with, the view data, the click data that you're engaging with, and learn across all of these edges. Think of clicks, views, every all your behavioral signal that you engage with with entities in this world as a graph and how do we learn across that graph?

8:06Yeah. So you don't have to be a social network to have a graph. Everyone, almost every enterprise I know has a graph. Fintech has graphs because they have customers, they have transactions, they have related data think of, you know, one of your delivery services, they have the inventory, the suppliers, the means of transportation. So they're all sitting as tables. They're all sitting as entities, and they're all linked across each other. So that's learning, let's you learn across that. That's a pretty key insight, the tables. Yeah. Before we go there, that was a very smart five -year -old. I think that you have a five -year -old.

8:49I have an eight -year -old. An eight -year -old. An eight -year -old. And a 12 -year -old. Well, they're very, very smart if they understood that explanation. Like, what would be, to Sonya's point, if you were five and you were going to say, graph learning versus any other type of machine learning, What's the difference? Ah, graph learning versus machine learning. Easy. Fast. I think those would be the two things. Just, you know, low code. I think that would be the key about Cool War. It learns all the weights. It learns all the features and discovers them over time. Exactly. And my eight year old has no machine learning.

9:30But if you were going to write

9:34the classifier, the old school way, you'd be writing features that say, okay, users in this platform, we need to look and click through rate data for the last three months and six months and eight months for every single video and we discover that Sonya has a preference for data that's for videos that are ever green. So six month windows really matter for Sonya. So imagine all of that code being written as features, draft neural networks eliminate all of that code. Do you think that means feature, feature engineering goes away as a discipline or what happens to it? I think feature engineering goes away and that's not a bad thing as such because prior to Kumo, I was at LinkedIn for almost seven close to eight years.

10:27And data scientists love finding opportunities for the business to make value, right? And it doesn't mean that feature engineering is the place where you spend that, you know, that's the time well spent. You'd much rather try out end different models on end different parts of the app or or whatever your business is and drive value. So trying out models in different parts of your application is where a data scientist needs to spend time. We started this episode. You said, you know, other ML on GPUs is different from other ML. Yes. And so what about GPUs specifically makes what you are describing possible?

11:12Like was it even possible to do this on a CPU? Or is it faster now? Or what's different than that you're doing on GPUs? Yeah, that's a great question. So it's definitely possible. It's much slower, right? So it's very similar to what neural networks brought to the text and image spaces in that we can scale these models to, you know, a large amounts of data. And while these models existed before the GPU revolution, it's we can actually take an entire enterprise as like FinTech data and learn GraphNural Networks of them. Yeah, in the previous era of auto -email, so much of the juice in the performance came out of ensembles.

11:56So you do these logistic regressions or you do these SVMs would have you and then you'd ensemble them together. Yeah. Frankly, in the Kaggle era, which was high first met your co -founder Yuri and the data science era of Kaggle and the like, always the ensembles won. Even in the Netflix prize back in the day, it was the ensembles at one. And there was something to the fact that these ensembles are just tons of little algorithms change together. And what is a neural network? Put tons of little algorithms change together. I mean, you could consider it billions of sigmoids or billions of logistic regressions.

12:27And really, the way I see GraphNural Networks is you're able to discover the features and the ensemble that you chain together to actually optimize towards the solution. So, it's the, to be a graph, they're the most general data type. Yeah. And a GraphNural Network is the most general. You kind of, you mentioned, it's a generalization where even a transformer is a subset of this generalization, the most general type of algorithm that can do some learning. Yeah, absolutely. And as you mentioned on Sombal, something that struck me was try maintaining that in production. You have end of front featured generation pipelines and an ensemble.

13:10And I've seen a world where you'd have one front end engineer change how we were logging the view data. Yeah. And everything either needed to change or something, you know, one pipeline breaks and breaks. And it's it's a mess to debug. So you so graphs give you a simple elegant framework to get at the same outcome. It also reminds me a lot more of our brain. Yes. Right, our brain, we think operates like a graph. And it's forming and pruning connections, more like a graph, even more so than a more structured neural network. And so have you ever, have you guys experimented or thought about that as an analogy and any ideas of the pros and cons of that analogy?

14:03I think it's very similar to the way I think about it is let's go back to that video watching example, right? And if I think of Sonia as a node in a graph and what these neural network algorithms are really good at is learning these embedding representations, right? And on this big graph, which has Sonia with her preference for baby shark and her household's preference. Exactly. Makes more sense. That checks out. Your embedding vector would be pretty close to both AI, so you close to Constantine, but you're also close in Euclidean space or in some big and dimensional space to all the baby shark loving folks, right?

14:59And we're basically learning these representations. So people or these and all the entities in the graph, like even Sequoia Capital in that case, becomes, you know, a representation. So in that sense, it's the idea is very similar, but what GNNs do and is allowed for arbitrary structure. And that's where I think it's a lot closer to the human brain, but I don't think the human brain is wired as a linear sequence, so there's a grid as an image. Yeah. Could you say a word about how it works under the hood? Like how are you able to, let's say you go and work with, I don't know, a food delivery service.

15:40Yeah. How does it actually work for you to go and automatically I'm going to be able to learn this graph representation and how are you training models on that? Absolutely. Given people might be watching it there and we're talking about AI, baby shark and history already. Exactly. So there's two pieces to come up. Historically, a graph learning has been restricted to I want to say PhDs in graph learning. Yeah. Because it's not easy to view the world as a graph. People think in terms of relational data, that's the most common data layout in companies. That's largely because of the analytics revolution that preceded the AI revolution.

16:31Everyone thinks in terms of relational data. But really relational data and graphs are have a one to one mapping because you have data laid out in tables usually an entities a primary key in a table and then you have all these relationships primary key foreign key relationships which encode. the edges in a graph. So that automatic construction from a table layout to a graph layout is one of the innovations inside code. The other bit is we've invented a language called predictive query language. And the language is allows you to specify any machine learning problem in a few lines that looks very much like SQL.

17:22So think of SQL with the predict clause. So we've created this very simple abstraction layer on top of relational data warehouses. There's already a universe of people who are writing SQL queries, and we've created a language that appeals, or you know, is one on, that resonates with them in some sense. So that's one of the innovations of Komodo. the other one is running these graph neural networks. So once you go from relational to graph, just running graph neural networks at scale. And that again is something that has not been easy to do. There are a few companies in the world that can do it.

18:04And it usually takes a huge infrastructure team to build that out. And because graphs inherently, unlike databases, where you can think of some logical partitioning. Graphs. It's all entangled in. So how do you split it across different machines with limited memory and so on? So all of these bits coming together makes Kumoi easy to use. But that's it. So when we go to a company, like a YouTube -like company, we'll often talk to a data science team that is looking to get faster our return on investment in AI. But then, Kumo becomes really easy to do, because if you have your data laid out as relational tables, a Kumo just sucks it in.

18:59So you just specify through connectors, tell Kumo what your schema is, and then you can just start writing predictive queries. So the graph is abstracted away. But if you have someone like a data scientist who loves tweaking the neural network parameters in case Constantine is decides to exactly. Exactly. You can look under the hood and the analogy we always use is we'll give you the self -driving car. But if you want to look under the hood or if you want a drive stick, we'll let you drive stick. So concretely in the YouTube example. Yeah, historically if I was an analytics YouTube and watching this video, I can look and say hey query all AI there'd be some Some tagging or some system to understand all AI historically.

19:50Let's see what the trends are over time That's querying the past. Yes What you're saying is once you have this in this database in the structure you're able to predict How many people are gonna watch AI videos in the next several weeks? Yes, I'm gonna watch baby shark videos. Yeah How much are they going to spend what is gonna be their Yes. Monetization. What are their ads? What else can you do with this? So you can say, is this user going to turn, for example? Yep. Right? And then you can say, what's the most relevant video that I want to show this user in order to retain them on my platform, right?

20:24So I want to drive value for my business. Given the past videos that they've watched, what's the next video to watch and so on. And we can also do demand forecasting. So we have customers in fact, we have in the health care sector. And they use Cuomo to forecast demand so that their wealth stocked on their emergency room. So the applications of using Cuomo go from consumer to health care to FinTech, where if in tech we see applications in fraud, for example, just is this user's behavior suspicious? Should we flag the user? So what's the thing of any question which says, how much? I love the user query the future.

21:21How much is this event going to happen?

21:29is what's the next best action for this user from an action space. Those are all the kinds of questions that Kumo can help answer. And I love that you said analyst because Kumo aims to be as auto ML as you want it to be. But we also have a Python interface. So you want to be a neural network expert, you can go all in. Cool. It's a brain. It's a brain and a lot of brain. Out of the applications you've discussed just now, I would imagine, you know, there's such classical ML problems that you discuss. Each of them probably has a five person fraud team and a 15 person demand forecasting team. What do those ML people think when, you know, when Kumo is pitching the company, like, walk me through that spiritual journey.

22:23And are you actually able to get results out of the box that are better than a 15 person team maintaining it can do? I'm okay with it as long as they don't have VC prediction.

22:36So for a lot of the companies we work with, the data scientist is excited about CooLan. As I mentioned, writing goes feature engineering pipelines comes with maintenance jobs to maintain those pipelines. And that's not where they want to spend their time. Data scientists in most companies are incentivized with direct business impact. So did I push that ad CTR model out? The squatter did a drive X percent revenue. So a lot of our customers will come to us and say, you know what I signed up for Xpercent Revenue, but I'm only one third of the way there. Can you guys, you know, help us accelerate?

23:25And we do a four week POC, so you know, and within four weeks we'll almost always, I'm trying to think of a case when we've not shown value. And I can't remember one, but we've always shown value within those four weeks. So you convert them into believers. And it's about where you want to spend your time. So I think once they get hands -on product, many times people will come in and say, oh, but feature engineering is where I spend all my time, right? And how can you say that I don't have to do it manually anymore? But we'll remind them, we'll remind them of the NLP journey. And then we'll also remind that once they get hands -on keyboard with the product and they realize that the journey in Kumo, it's not completely automated away, right?

24:22Because we say a data scientist knows their business well. So if you're going to define Sean prediction for your business, maybe on YouTube activity around in the last 30 days is a good predictor of Sean. So you want to bring your events table with a 30 day window. Ah, the schema, the actual structure that you use for the window, right? So because these are all queries and these are all parameters in the queries or you could play with 90 day or 365 day activities. So these are all queries. You can write five of these queries and say, oh, really, on my system, the best predictor of churn is behavior in a 365 day window and I didn't even know that because I'm spending all my time looking somewhere else.

25:09So the data scientists spends lot more time finding the relevant tables in their organization that are going to, you know, bring value and then finding those, the right query formulation or the right business formulation. In this case, you know, for example, churn, what's the right definition of churn for my business? And once they see that, actually, they really, they realize that it's a lot more fun than what I was doing before. Totally. Yeah. You mentioned tables, structured data, schema. That naturally leaves me to think about snowflake and data breaks. a lot of companies have spent the last five years heavily investing in their data warehouses.

25:52How do you work with the data warehouses? That's a great question. So at the outset, we started as a purely SaaS company, emulating a lot of, the principles from the Snowflake architecture looking at their success stories. One thing we realized is that data scientists, though, they need to see value on their own problem, because they're so KPI or business impact focused, showing them value on a caggle data set doesn't really count. So the easiest way to show values, of course, when they can connect to their own data, But connecting to your own data on a SaaS product means you go through a huge security review through the company, which can in some many organizations can take a couple of months.

26:51So that we wanted to reduce that friction. And we started partnering with the warehouses to think about deployment models where compute can be closer to the data. And we have a deployment with Snowflake, which is part, we use a combination of what is called snowpark container services and really cumocan deploy as a container in Snowflake's compute pool. So from a data scientist point of view, we're also a native app in Snowflake. So you a data scientist in an organization, let's say YouTube can go in and click install Snowflake. So it's like an app on your iPhone. It gets installed and then they can start writing those predictive queries and looking for value.

27:39And oftentimes the security team is completely okay with it because there's no data leaving the ecosystem. We have a very similar deployment model with data bricks, though in that case, we manage the GPU compute and but data residency stays completely inside data bricks. And that also led us. So we started that from the point of view of getting data scientists get hands on keyboard with Cuomo quickly. But we also realized that freed us up a lot to not have to think about security compliance, governance, and let the data warehouses as they're already building all of the tools and technology for management of data, let it stay there, let it be managed there, but Kumo just talks to data directly sitting inside the warehouse.

28:37So you talked about relational versus graph data, and relational data is kind of how many of our brains have been taught to think. Yes. Yes. We think about things in spreadsheets oftentimes. We might go down and say, if we have a series of AI videos, you have them as rows, and then you have some descriptors of them as columns. But really, when you start to see things as graphs, which I did, frankly, back in the day around Yuri's time as a professor, I think everything starts, you can start to see everything as a graph. It's the most general data type. And when you start to see things as graphs, it's actually kind of how our brain thinks.

29:14Yes. Hey, here's a video and that has a pointed characteristic that some other part of the graph, which is connected to another part. How do you ingest all of this relational data, which is the way that the world has been run for the way computers have been run for 50 years and put them into a graph structure? It sounds like a very heavy lift and doing that inside of Snowflake and Databricks is probably pretty hard. I want to say that's part of the magic of Google, right? And that's That was the friction that prevented graph learning from taking off and it's staying within and, you know, the big companies.