Arvind Jain: Why Now Is the Time to Solve Enterprise Search (Encore)

0:05Hey everyone and welcome to Generative Now. I am Michael Mignano, a partner at Lightspeed And this week, we're revisiting a conversation with Arvind Jain from a few months back. Arvind is the founder and CEO of Glean, one of the most sophisticated AI workplace assistants that's now a massive player in the enterprise search space. Arvind is an alum of Microsoft and Google, and he previously co-founded Rubrik, which he took public in the high-stakes world of data security. Arvind and I had a great conversation about the early days of Glean, how he sees AI shifting the way we work in the future, and what's next for the company.

0:39Check it out. Hey, Arvind. Good to see you. Good to see you, too. Glean has been, you know, an obvious success story in the world of AI and generative AI. The interesting thing about Glean is it seems to have started well before the current wave of generative AI and the AI explosion that we've now been going through really since kind of the launch of ChatGBT. I figure a good place to start would be to hear the story of Glean and how it got started in 2019, you know, from your journey at Rubric to starting this and like really what you saw before others did, before the AI wave took off. So we started in early 2019.

1:22And at that time, you know, we didn't have, you know, that much of an idea of how AI was going to take over the world like it has today. But the problem that we wanted to solve was an important one. We started with this vision of building a really powerful search experience inside businesses for people in their work lives. Before Glean, I was one of the founders of Rupric, which is an enterprise data security company. And one of the things, you know, like at that company, we grew very fast. we were very successful. But as the company grew, like, you know, as we sort of crossed 1000 employees, we started to see, like, you know, big drop in productivity, like across all of our different teams.

2:13And part of it is like, you know, you sort of like take it for granted that, yeah, like, you know, as things, you know, as companies get bigger, like, you know, like, like things become more complex and slower. But I thought I felt there was more to it, like, you know, we were actually seeing very, very steep drops in productivity. And I think part of it was caused by this issue of the company has so much knowledge, so much data, you know, spread across so many different systems. And, and also like, you know, as the companies grow fast, like, you know, people's roles, like, you know, it's not always clear, like, you know, who works on what, who's an expert who can help me on any given topic.

2:47And these were actually the largest complaints that we were seeing in our pulse surveys at Rubrik. People are complaining about these, that people are saying that, you know, I'm not enabled, I can't actually get my work done because of, you know, these, these issues. And, and so me being a search engineer, like, you know, before Rupert, I spent over a decade of my career building, you know, Google search, I said, like, you know, well, you know, people are complaining about not being able to find things and, well, we should go and, you know, buy a search product, like, you know, that can connect, you know, our enterprise data that's, you know, fragmented and spread across all these different systems so that, you know, it's easy for people to find things.

3:23And also it's easy for people, you know, people to find other people who can help them, you know, do their work. So we just had to, you know, that was sort of the origins of, we were trying to solve a problem internally, you know, at our company. And that's when I realized that there's no good search product that we could buy. And so we wanted to go and solve that. And one of the interesting things is that in 2018, in the search industry, you know among the search engineers we were already seeing the power of the transformer based you know neural nets and uh and these models like you know the there were some language models you know they're small like you know the the bird family of models that were already out there and uh in google search we were seeing like you know how you could take a you know every like you know component of the search technology and you could actually build a new system using these transformer-based models, and they would actually perform better than these previous generation of models that were built over the last 15 years.

4:24And so we knew that transformers was going to completely change how search works. You can actually do search at a much more conceptual and semantic level. And so that was an opportunity to actually also leverage and build a really good product in the industry. So that's how we sort of got started. In fact, Glean became the first company, I believe, that has brought the transformer technology to the enterprise in that sense. It's so cool. And it's such a great insight. I mean, even relating to my own experience, I was building a small startup, got it to a couple dozen employees, and then we sold to a much bigger company.

5:03And we show up at this big company, and there's thousands of employees, and there's all this context that we're missing. and you go and grab it in Slack or an email or in Google Drive or in any number of these systems, Confluence, Okta, but it's all disconnected and there's no way to sort of bring it back together and bundle it. And it sounds like that's the opportunity you saw. I mean, even though you saw that this opportunity was coming for LLMs, I imagine that you weren't necessarily leveraging them from day one in the product. So like, how were you tying all these systems together that I just mentioned?

5:41Like, what was the actual technical approach to connecting all of these endpoints? Yeah. So like, if you think about the search stats, the number one, like to be able to search over information, you need to have access to that information. If you look at a modern enterprise, like most of their systems where their knowledge and data is, is a SaaS, like cloud hosted system. And typically, like SaaS systems interoperate well with each other. They have good APIs available to them. We started to build these integrations with products like Confluence and Jira and Google Drive, SharePoint, whatnot, right?

6:15And all of these integrations were built using the already published APIs that these systems exposed to us. You guys should connect with these systems. You could see all the data inside them. And then it can bring it together in one common search system. So that was the first step of building the technology was building these integrations. But you have to also understand that enterprise search is very different from search on the web. On the internet, we can all access all information. Everything is available to us. In an enterprise, information is privileged. It's secure. There are certain documents you can access, but then there are many others that you can't.

6:54So when you actually build a search product, to actually build a safe version of search. You can't actually start to leak information to people from this search experience. So you have to sort of think about the problem differently. You have to figure out how you're going to actually solve, you know, secure, you know, permission-aware access, like, you know, within the search results. So we have to sort of build a lot of security infrastructure to actually handle that. But to actually, you know, on your question about transformers, actually the transformer models were part of our first version of the product.

7:26Like it was from day one. Yeah, so from day one. That's really cool. Because, you know, the technology was in front of us, you know, we saw these models were so powerful and you can build a much more smarter search. You can actually come in and you can ask questions in natural language and we could actually match it much better at a conceptual and semantic level, you know, to those documents in Confluence or Google Drive, as opposed to doing the brittle keyword-based search. So it was a great technology and it was available to us. So what we did there was we took BERT, which was a model that Google had put in open domain for startups like ours to use.

8:05And then we would take that model and for every enterprise customer of ours, we would actually build a model, you know, customized to that company. We would actually train, we would actually train, you know, retrain these models on the enterprise corpus, on that data in a safe and secure way again, so that the model starts to understand your company, your lingo, your code names, acronyms, all of that stuff. And then in our core search stack, we would use both traditional search techniques to figure out what's the best information to return given a question, but we'd also use this transformer-based semantic meshing technology.

8:50So it was a hybrid search stack that we built from day one. So what was sort of finding product market fit for that? Like, you know, I think about all the startups that have launched over the past couple of years, they almost have this truly magical experience the first time you experienced them because the AI is just so powerful, right? It's like the first time you used ChatGPT or the first time you used one of these image generators. Going back to 2019 or 2020, when you started to go to the market, the concept of transformer must have been so new for all these enterprise companies. Did you find that same sort of magical experience where you're like, oh, hey, Enterprise, let us show you how you now can, you know, prompt semantically against your entire organization and just have it, you know, come back to them like a human being?

9:35I mean, that must have just been really, really powerful for them. Yeah, actually, you know, it's interesting that we didn't actually talk about Transformers as a cool thing. It was not a hot thing, like people are not talking about it. I think what they cared about was what you could do with it, which was that, well, like, you know, in Glean, you can ask questions in natural language and it will conceptually match the information with your question. You don't have to, like, remember the exact keywords, you know, and that was sort of one of the big pain point of the previous generation search systems is that, well, I know I'm looking for this email, like it's in my outlook, but I still cannot, you know, get to it because I don't remember the exact words.

10:12So I think it appealed to people, but you actually, you know, ask a very interesting question. I don't think we had a product market fit for a long time, like in the sense that somehow the industry felt that search was a vitamin and not a painkiller. Everybody would acknowledge that, yes, we have a problem. We have lots of information. It's spread and fragmented across many different systems. And people do struggle with finding information. But, well, ultimately, they find it. And, like, you know, I guess, you know, things are working for us. And there was not a product that they were actually buying.

10:52Like, you know, they're not buying a search product that we would say that, hey, look, you know, we have a better one than what you buy. There was no concept of them buying a technology like this. So we had to create a market. And it took us time. We had to do a lot of evangelism. We had to talk about why it is important. Like, you know, it reduces frustration. It makes employees happier. You know, they spend a third of their time just looking for information. So you will save a lot of time. And so it took us some time before we started to gain that traction and gain that word of mouth from our initial customers.

11:24And what was the moment that it all flipped? Was it just you reached some moment that you had enough people talking about in the market that it just sort of snowballed from there? Or was it a key customer or a key sort of product unlock? Like what really did it? Yeah. I would say there were two distinct moments. Like one is where we actually had like about 30 or so companies and we were first focused on the tech sector. So we got most like, you know, the most iconic tech companies, you know, they started to use our product at a scale. And the word of mouth started to happen from there, which sort of created like, you know, that further momentum and inbound like, you know, to us in terms of more interest in the company.

12:07And then there was another big moment, which was ChatGPT. In some ways, you can think of Glean as the enterprise version of ChatGPT. It does everything that ChatGPT does, but it does it with that knowledge and context of your company. When the world saw ChatGPT and the power that it has, as an enterprise leader, you were thinking about, well, what if I had something like this inside my company? something that knew everything about my, you know, my employees, you know, my, you know, internal data. And it could also answer questions for me using all of that. That would be fascinating. Right. So, so that actually, you know, started to create that, you know, organic, you know, demand, you know, it was a second wave of like, you know, massive momentum for us.

12:52Yeah, that's really, really interesting. It's almost like ChatGPT kind of did a lot of the external marketing for you. That's right. But all the things you had previously been working on, like security and, you know, safety, and like you mentioned, permissioning between different users, obviously, like, so, so important to bring that technology into an enterprise. So that's fascinating. You mentioned early on that, you know, you started out, the first transformer based model you're using was Bird. Since then, like, how is your, like, how do you manage the sort of model component of the product?

13:22Do you train your own models now? Are you, you know, hot swapping things under the hood between, you know, OpenAI, Anthropic, et cetera? Like, Like, how does that all work? Yeah, more of the latter. So let me explain the architecture of this. We're not a foundation model company. We don't train these super large models. When you think about like the product, like how does Glean actually work? So it looks and feels like ChatGPT, right? You come in, you ask questions and Glean will actually use either all of the world's knowledge and context to answer those questions for you, or it's going to actually use, you know, your internal company's data and knowledge to answer those questions for you.

13:59Now, the way like the models themselves, like, you know, when you think about, you know, GPD or Cloud or Gemini or X or, you know, DeepSeek and whatnot, like, you know, none of those models are trained on your enterprise information. So they don't know anything about like, you know, how work happens inside your company. And so how do you how do you actually like answer questions, make these models answer questions, you know, you know, using data and knowledge that's inside your company? you know, the architecture for that is, you know, drag, right? Where you actually, you know, you know, it's a two-step process, like, you know, given any task or any question that a user comes and asks, you know, first we use our core retrieval and search technology to assemble like some relevant pieces of information from inside our company.

14:45And then we give all of that to the model so that the model can actually reason over it and actually generate, you know, the appropriate, you know, answers or responses back to you. So that's the architecture that's typical, like, you know, most enterprise agents or applications are built in this, you know, rack style architecture where search is a first leg. I mean, not search or retriever, whatever you want to call it. And then the model reasoning and inferencing is the second part. Now our architecture, you know, is, so we, we do build models and they are, they are, they're still built on like, you know, bird or modern bird, like, you know, these small, you know, I guess these are SLMs, you know, is what the industry calls them now.

15:24So we actually still train models on your enterprise corpus, but that is done mostly to actually do semantic matching of people's tasks with your enterprise knowledge. So it's used for that, you know, retrieval task. But then for reasoning and generation, we use the foundation models. And our architecture is like, we're not tied to one particular model. like what we're seeing in the industry is that there are many, many model providers. They're all in different models are getting better at different things. For example, like, you know, these days you'll hear a lot of people say that for code generation, like Claude is actually doing better than any other model for reasoning.

16:06GPT is doing better than others. And so they're all like all these models are getting better at different things. And our goal, you know, at Glean is to make sure that for our enterprise customers, like we can bring all the amazing innovation that's happening in the industry, you know, to our customers, they don't, they should not have to choose, you know, what model to use. And so like, so that's how the architecture is that behind, underneath, you know, we have access to all these different models and given, you know, given the task, given, you know, we can actually, you know, help an enterprise choose what's the right model to use for that.

16:39Yeah. Yeah. And what's really cool about that is as these models keep getting better And, you know, you know, and I'm sure everyone in the audience knows, like, feels like every week these things are getting better. Your product just keeps getting better and better as a result, right? Anything these models can do, you get to now offer to your customers, your enterprise customers. What must that be like? Because I have to imagine that as these models improve in capabilities, they probably unlock, like, completely new feature sets for you. You know, like, you know, one thing I keep thinking about with Glean is like, as these models become truly agentic and they move from, you know, they move from individual sort of retrieval of information to more sequential task oriented, things like that.

17:21Like, that's got to open up a huge service area for Glean and your customers. Is that how you think about it? Absolutely. I mean, I think the and these features are actually coming every day. Like, you know, like, you know, like, you know, you saw in the last month, you know, with, you know, operators like, you know, is a new thing, you know, which actually like really unlocks, you know, our ability to work with many, many more enterprise applications. applications. One of the important things, like the way I think about AI and its impact in the enterprise is that like, if I'm a business user and I have a question or I have a task, which requires me to play with or work with some data inside my enterprise, like AI is supposed to make it super seamless for me.

18:05Like, you know, like I don't want, like, you know, this, this, we don't, we're not going to be in a world anymore where we know that the data exists, but accessing it is hard. Like, you know, AI is going to make it super easy. Like, you know, a lot of it we do through our integrations, but a lot of it we can also do through these new LLM capabilities like computer use, operators where you don't even need those systems to have APIs. You can still go and drive a browser and get that data that you need to work on for your agents. So the innovation is going to keep happening at the fast pace. Our strategy at Glean is always very clear, you know, everything that the model providers, that the cloud hyperscalers, all the technology that they're building, you know, we basically don't compete with them.

18:52You know, we leverage that technology and we do the remaining part, which is like, you know, how do you make it easy to access that, like, you know, all that technology and make it work on your data and your enterprise. The third thing that you can do with Glean, like, so we have a Google like search, we have a chat GPT like AI assistant, but then we also have an agent building platform. So you can actually build all kinds of agents to transform your business processes using AI. And in fact, I think the best thing with AI is that it's actually bringing that power to, you know, to every business user.

19:28You don't have to be a developer anymore to be able to build something, you know, complex or like interesting. Like, you know, you don't need developers, you know, for everything anymore. Like, you know, there are a lot of like, you know, agents that you can actually just build by expressing, you know, a business process in natural language to AI. And then AI sort of like, you know, like does, you know, things like what you mentioned, like, you know, you don't do the reasoning is actually take the business process. It will create a multi-step workflow that you can then go and review and execute.

20:00So, so I think, yeah, we are definitely, you know, going, um, um, you know, towards a world where, um, a lot of like, you know, how, uh, like just how work happens is going to fundamentally change. Um, and yeah. Yeah. So, so, so this Asian builder, which you mentioned, um, like, what does that require from, from the models to, to work? Like, does that require models that are specifically capable, capable of reasoning or, or being able to browse the web like operator or computer use, or is Glean really the product in the system that enables to take these LLMs and sort of piece them in a way together that they're actually executing on tasks on behalf of your enterprise?

20:46Yeah. It's a combination of that. Like today, if you just work with an LLM, that's not enough for you to actually build an agent and automate a business, you know, business process with it. You know, you do need, you know, some more work, you know, that needs to happen on top of it. Like, for example, you need to figure out how to actually bring the right data to these models that's in your enterprise. And like over time, like, you know, there's going to be more and more features that you will get from the LLM providers. Like, you know, computer use, for example, is an example of that. This is a feature.

21:28This is not like really the model, but it's a feature that the LLM companies are actually providing to you. You know, it's a mechanism for you to sort of, you know, bring more data into the models, into the LLM reasoning. The way things will evolve is today, I think that you still need, you know, agent building frameworks and platforms that will actually sit on top of LLMs. The way we are helping our customers build agents is, you know, you get this, you know, agent builder UI, you know, where you can just talk in natural language, you know, converse with clean to build an agent together. Behind the scenes, you know, we're using, you know, both our, you know, API based integrations to enterprise data sources.

22:10But like that, the overall orchestration is being done by us. Got it. That makes a lot of sense. And, you know, speaking of computer use and again, these models getting more agentic and the fact that it does seem like a lot of the big sort of model providers, the open AIs, the Anthropics, et cetera, like they are leaning more and more over time into the enterprise, offering, you know, more and more ways to connect different parts of your business. How do you think about the landscape of competition when, you know, I'm sure many of your competitors are sort of building kind of like at the application layer on top of the models.

22:43And now there's also this, I don't know, new sort of a front from the model layer where they're coming at it as well. How do you keep the product sort of competitive when you have, you know, you have this competition coming from all different angles? Yeah, it's true. From a technology stack perspective, like an overall agent architecture, you're going to have the LLMs and model capabilities at the base level. you're going to have a data connectivity layer and a retrieval and search system and then you have basically like you know the overall like agent building and you know like the reasoning flows you know that you're going to build on top of them first i'll make make this assertion a lot of things are going to change about work in in five to ten years from now majority of the work that we do today, we won't do anymore.

23:39It's going to be done by AI assistants and agents that are going to be helping us perform those tasks. Our work is not going to go away. It's just going to change a little bit. It's going to become maybe more creative, more thinking-oriented, and the base level, working with data, doing some research analysis on it. The models can actually do a lot of that work for us. So the majority of our work is going to change. the way the organizations are built, the business processes, like most of these business processes are going to actually also get automated, you know, with AI. So one thing that I feel is that the body of work that needs to get done versus the companies that are actually doing that work, there's a big delta between that.

24:28You know, the white space is huge. And the, like, if all of us, you know, work super hard, like, you know, we're still only getting 1 % of, you know, what needs to happen done in the next one year. This is not the time where there are too many players and, you know, and not enough, like, you know, problems to solve, like, you know, within the opposite. Right. It's probably the opposite. Yeah. And so therefore we don't feel like, you know, any sort of concern, like, you know, with competition, you know, our strategy will remain, you know, what it is today, which is we will continue to partner with all of these players.

25:04We'll continue to partner with the language model companies, with the cloud hyperscalers, and we'll fully leverage all the innovation that they are making because, well, they won't be able to cover more than 10 % of what needs to get done. And so we have enough for us to work on. And so we basically think of us as sitting on top of the core technology from these players. As they build more, like we also rise and like you know we start to build like you know the other things you know that we haven't built yet so we we think you know of all of these companies more as our partners not as our competitors but now like practically like if you think about you know like as a customer you're being pitched like you know agents from 300 different like you know vendors and who do you pick who do you choose um the i think glean is has a very unique architecture like you know and a position like, you know, in the marketplace where, well, we'll say, you know, we're telling our vendors that like, you know, you're not a LLM company.

26:04So when you actually think about your AI stack, you know, you have to work with, you know, the language model companies anyway. And you can work, you know, with them through us or directly, you know, we're fine with both those options. But what we are doing is, you know, we're giving, you know, our customers a horizontal AI platform. a platform that's connected to all of your enterprise information and data and knowledge. So you can actually use this as a way to consolidate your AI work, as opposed to buying AI agents function by function, product by product. And if you use a thousand SaaS applications, you're going to have 10 ,000 agents in your company in the future.

26:49And so like one model, you know, is that you actually go very, you know, functional and you get a lot of these vertical agents. The other approach is you pick us as the horizontal AI layer and then use us as a central platform to build like many of these agents. So I think right now we, you know, definitely are unique compared to like any other approach that you see from other vendors in the enterprise. And it remains to be seen, like, you know, how many of these agents will be built on a horizontal platform like us versus, like, you know, fully dedicated verticalized, you know, agent solutions.