Arvind Jain: Why Now Is the Time to Solve Enterprise Search

0:04Hey, everyone. Welcome to Generative Now. I am Michael Magnano. I am a partner at Lightspeed. Enterprise AI is transforming how companies work, and few understand this revolution better than Arvind Jain. As the founder of Glean, Arvind has been building one of the most sophisticated AI workplace assistants since years before ChatGPT became a household name. And now it is a massive player that dominates the enterprise search space. Starting his career at Microsoft and then working as an engineer in Google's early days, Arvind co-founded Rubrik and took that company public in the high-stakes world of data security.

0:39Arvind and I had a great conversation about why he loves competition, how technical innovation informs product strategy, and how Glean found product market fit. Take a listen. Hey, Arvind. Good to see you. Good to see you, too. Glean has been, you know, an obvious success story in the world of AI and generative AI. The interesting thing about Glean is it seems to have started well before the current wave of generative AI and the AI explosion that we've now been going through really since kind of the launch of ChatGPT. I figure a good place to start would be to hear the story of Glean and how it got started in 2019, you know, from your journey at Rubric to starting this and like really what you saw before others did, before the AI wave took off.

1:28So we started in early 2019. And at that time, we didn't have that much of an idea of how AI was going to take over the world like it has today. But the problem that we wanted to solve was an important one. We started with this vision of building a really powerful search experience inside businesses for people in their work lives. Before Glean, I was one of the founders of Rupric, which is an enterprise data security company. And one of the things, you know, like at that company, like we grew very fast, we were very successful. But as the company grew, like, you know, as we sort of crossed 1 ,000 employees, we started to see like, you know, big drop in productivity, like across all of our different teams.

2:22And part of it is like, you know, you sort of like take it for granted that, yeah, like, you know, as things, you know, as companies get bigger, like things become more complex and slower. But I felt there was more to it. Like we were actually seeing very, very steep drops in productivity. And I think part of it was caused by this issue of the company has so much knowledge, so much data, you know, spread across so many different systems. And also like, you know, as the companies grow fast, like, you know, people's roles, like, you know, it's not always clear, like, you know, who works on what, who's an expert who can help me on any given topic.

2:56And these were actually the largest complaints that we were seeing in our pulse surveys at Rubrik. People are complaining about these, that people are saying that, you know, I'm not enabled. I can't actually get my work done because of, you know, these issues. And so me being a search engineer, like, you know, before Rubrik, I spent over a decade of my career building, you know, Google search. I said like, you know, well, you know, people are complaining about not being able to find things and we should go and, you know, buy a search product that can connect, you know, our enterprise data that's, you know, fragmented and spread across all these different systems so that, you know, it's easy for people to find things.

3:32And also it's easy for people to find other people who can help them, you know, do their work. So we just had to, you know, that was sort of the origins of we're trying to solve a problem internally, you know, at our company. and that's when I realized that there's no good search product that we could buy. And so we wanted to go and solve that. And one of the interesting things is that in 2018, in the search industry, among the search engineers, we were already seeing the power of the transformer-based neural nets and these models. There were some language models. You know, they're small, like, you know, the BERT family of models that were already out there.

4:15And in Google search, we were seeing, like, you know, how you could take a, you know, every, like, you know, component of the search technology and you could actually build a new system using these transformer based models. And they would actually perform better than, like, you know, these, you know, previous generation of models that were built over the last 15 years. And so we knew that transformers was going to completely change how search works. You can actually do search at a much more conceptual and semantic level. And so that was an opportunity to actually also leverage and build a really good product in the industry.

4:52So that's how we sort of got started. In fact, Glean became the first company, I believe, that has brought the transformer technology to the enterprise in that sense. It's so cool. And it's such a great insight. I mean, you know, even relating to my own experience, I was building a small startup, you know, got it to a couple dozen employees and then we sold to a much bigger company and we show up at this big company and there's thousands of employees and there's all this context that we're missing. And you go and grab it in Slack or an email or in Google Drive or in any any number of these confluence Okta.

5:27But it's all disconnected and there's no and there's no way to sort of bring it back together and bundle it. And it sounds like that's the opportunity you saw. I mean, even though you saw that this opportunity was coming for LLMs, I imagine that you weren't necessarily leveraging them from day one in the product. So, like, how were you tying all these systems together that I just mentioned? Like, what was the actual technical approach to connecting all of these endpoints? Yeah. So, like, if you think about the search stats, number one, like, to be able to search over information, you need to have access to that information.

6:02If you look at a modern enterprise, most of their systems where their knowledge and data is, is a SaaS cloud-hosted system. And typically SaaS systems interoperate well with each other. They have good APIs available to them. We started to build these integrations with products like Confluence and Jira and Google Drive, SharePoint, whatnot. not, right? And all of these integrations were built using the already published APIs that these systems exposed to us. You guys should connect with these systems. You could see all the data inside them, and then you can bring it together in one common search system.

6:40So that was the first step of building the technology was building these integrations. But you have to also understand that enterprise search is very different from search on the web. On the internet, we can all access all information. You know, everything is available to us. In an enterprise, like information is privileged. It's secure. Like there are certain documents you can access, but then there are many others that you can't. So when you actually build a search product, you have to actually build a safe version of search. You can't actually start to leak information to people from this search experience.

7:12So you have to sort of think about the problem differently. You have to figure out how you're going to actually solve, you know, secure, you know, permission-aware access, like, you know, within the search results. So we had to sort of build a lot of security infrastructure to actually handle that. But to actually, you know, on your question about transformers, actually, the transformer models were part of our first version of the product. Like, it was from day one. Yeah. So from day one. That's really cool. Well, because, you know, the technology was in front of us, you know, we saw these models were so powerful.

7:43And you can build a such, like, much more smarter search. you can actually come in and you can ask questions in natural language. And we could actually match it much better at a conceptual and semantic level, you know, to those documents in Confluence or Google Drive, as opposed to doing the brittle keyword-based search. So it was a great technology and it was available to us. So what we did there was we took BERT, which was, you know, a model that Google had put in open domain for startups like ours to use. and then we would take that model and for every enterprise customer of ours, we would actually build a model, you know, customized to that company.

8:25We would actually train, we would actually train, you know, retrain these models on the enterprise corpus, on that data in a safe and secure way again so that, you know, the model starts to understand your company, your lingo, your code names, you know, acronyms, you know, all of that stuff. And then in our core search stack, we would use both traditional search techniques to figure out what's the best information to return given a question, but we'd also use this transformer-based semantic meshing technology. So it was a hybrid search stack that we built from day one. So what was sort of finding product market fit for that like?

9:08Like, you know, I think about all the startups that have launched over the past couple of years, they almost have this truly magical experience the first time you experience them because the AI is just so powerful, right? It's like the first time you use ChatGPT or the first time you use one of these image generators. Going back to 2019 or 2020, when you started to go to the market, the concept of Transformer must have been so new for all these enterprise companies. Did you find that same sort of magical experience where you're like, oh, hey, Enterprise, let us show you how you now can, you know, prompt semantically against your entire organization and just have it, you know, come back to them like a human being?

9:44I mean, that must have just been really, really powerful for them. Yeah, actually, you know, it's interesting that we didn't actually talk about Transformers as a cool thing. It was not a hot thing, like people are not talking about it. I think what they cared about was what you could do with it, which was that, well, like, you know, in Glean, you can ask questions in natural language and it will conceptually match the information with your question. You don't have to, like, remember the exact keywords, you know, and that was sort of one of the big pain point of the previous generation search systems is that, well, I know I'm looking for this email, like it's in my outlook, but I still cannot, you know, get to it because I don't remember the exact words.

10:21So I think it appealed to people, but you actually, you know, ask a very interesting question. we I don't think you know we had a product market fit for a long time like in the sense that the somehow like you know like the industry felt that search was a vitamin and not a painkiller like you know well yeah like you know everybody would everybody would like you know they would they would acknowledge that yes you know we have a problem like you know we have lots of information it's you know spread and fragmented across many different systems and people do struggle like you know, with finding information, but well, ultimately they find it.

10:56And like, you know, I guess, I guess, you know, things are working for us. And there was not a product that they were actually buying. Like, you know, they're not buying a search product that we would say that, hey, look, you know, we have a better one than what you buy. There was no concept of them buying a technology like this. So we had to create a market and it took us time. We had to do a lot of evangelism. We had to talk about why it is important. Like, you know, it reduces frustration. It makes employees happier. You know, they spend a third of their time just looking for information. So you will save a lot of time.

11:24And so it took us some time, you know, before, like, you know, we started to gain that traction and gain that word of mouth, like, you know, from our initial customers. And what was the moment that it all flipped? Was it just, you know, but you reached some moment that you had enough people talking about in the market that it just sort of snowballed from there? Or was it a key customer or a key sort of product unlock? Like, what really did it? Yeah, I would say there were two, like, I would say two distinct moments. Like one is where we actually had like about 30 or so companies and we were first focused on the tech sector.

12:00So we got most like, you know, the most iconic tech companies, you know, they started to use our product at a net scale. And the word of mouth started to happen from there, which sort of created like, you know, that further momentum and inbound like, you know, to us in terms of more interest in the company. And then there was another big moment, which was ChatGPT. In some ways, you can think of Glean as the enterprise version of ChatGPT. It does everything that ChatGPT does, but it does it with that knowledge and context of your company. When the world saw ChatGPT and the power that it has, as an enterprise leader, you were thinking about, well, what if I had something like this inside my company?

12:44something that knew everything about my, you know, my employees, you know, my, you know, internal data. And it could also answer questions for me using all of that. That would be fascinating. Right. So, so that actually, you know, started to create that, you know, organic, you know, demand, you know, it was a second wave of like, you know, massive momentum for us. Yeah. That's really, really interesting. It's almost like ChatGPT kind of did a lot of the external marketing for you. That's right. But all the things you had previously been working on, like security and, you know, safety, and like you mentioned, permissioning between different users, obviously, like, so, so important to bring that technology into an enterprise.

13:17So that's fascinating. You mentioned early on that, you know, you started out, the first transformer based model you're using was Bird. Since then, like, how is your, like, how do you manage the sort of model component of the product? Do you train your own models now? Are you, you know, you hot swapping things under the hood between, you know, OpenAI, Anthropic, et cetera? Like, Like, how's that all work? Yeah, more of the latter. So let me explain the architecture of this. We're not a foundation model company. We don't train these super large models. When you think about like the product, like how does Glean actually work?

13:53So it looks and feels like ChatGPT, right? You come in, you ask questions and Glean will actually use either all of the world's knowledge and context to answer those questions for you, or it's going to actually use, you know, your internal company's data and knowledge to answer those questions for you. Now, the way like the models themselves, like, you know, when you think about, you know, GPD or Cloud or Gemini or X or, you know, DeepSeek and whatnot, like, you know, none of those models are trained on your enterprise information. So they don't know anything about like, you know, how work happens inside your company.

14:26And so how do you how do you actually like answer questions, make these models answer questions, you know, you know, using data and knowledge that's inside your company? you know, the architecture for that is, you know, drag, right? Where you actually, you know, you know, it's a two-step process, like, you know, given any task or any question that a user comes and asks, you know, first we use our core retrieval and search technology to assemble like some relevant pieces of information from inside our company. And then we give all of that to the model so that the model can actually reason over it and actually generate, you know, the appropriate, you know, answers or responses back to you.

15:03So that's the architecture that's typical, like, you know, most enterprise agents or applications are built in this, you know, rack style architecture where search is a first leg. I mean, not search or retriever, whatever you want to call it. And then the model reasoning and inferencing is the second part. Now our architecture, you know, is, so we, we do build models and they are, they are, they're still built all like, you know, bird or modern bird, like, you know, these small, you know, I guess these are SLMs, you know, is what the industry calls them now. So we actually still train models on your enterprise corpus, but that is done mostly to actually do semantic matching of people's tasks with your enterprise knowledge.

15:43So it's used for that, you know, retrieval task. But then for reasoning and generation, we use the foundation models. And our architecture is like, we're not tied to one particular model. like what we're seeing in the industry is that there are many, many model providers. They're all in different models are getting better at different things. For example, like, you know, these days you'll hear a lot of people say that for code generation, like Cloud is actually doing better than any other model for reasoning. GPT is doing better than others. And so they're all like all these models are getting better at different things.

16:20And our goal, you know, at Glean is to make sure that for our enterprise customers, like we can bring all the amazing innovation that's happening in the industry, you know, to our customers. They don't, they should not have to choose, you know, what model to use. And so like, so that's how the architecture is that behind, underneath, you know, we have access to all these different models. And given, you know, given the task, given, you know, we can actually, you know, help an enterprise choose what's the right model to use for that. Yeah. Yeah. And what's really cool about that is as these models keep getting better and, you know, you know, and I'm sure everyone in the audience knows, like, feels like every week these things are getting better.

17:00Your product just keeps getting better and better as a result, right? Anything these models can do, you get to now offer to your customers, your enterprise customers. What must that be like? Because I have to imagine that as these models improve in capabilities, they probably unlock like completely new feature sets for you. You know, like, you know, one thing I keep thinking about with Glean is like, as these models become truly agentic and they move from, you know, they move from individual sort of retrieval of information to more sequential task oriented, things like that. Like, that's got to open up a huge service area for Glean and your customers.

17:37Is that how you think about it? Absolutely. I mean, I think the and these features are actually coming every day. like you saw in the last month with operators is a new thing, which actually really unlocks our ability to work with many, many more enterprise applications. One of the important things, like the way I think about AI and its impact in the enterprise is that if I'm a business user and I have a question or I have a task which requires me to play with or work with some data inside my enterprise, like AI is supposed to make it super seamless for me. Like, you know, like I don't want like, you know, this, this, we don't, we're not going to be in a world anymore where we know that the data exists, but accessing it is hard.

18:22Like, you know, AI is going to make it super easy. Like, you know, a lot of it, you know, we do through our integrations, but a lot of it we can also do through these, you know, like new LLM capabilities, like computer use, you know, operators where you don't even need those systems to have APIs. You can still sort of go and drive, you know, a browser and sort of, you know, like, you know, get that data that you need to work on, you know, for your agents. So the innovation is going to, you know, keep happening at the fast pace. Like, you know, our strategy at Glean is always very clear. You know, everything that the model providers, that the cloud hyperscalers, all the technology that they are building, you know, we basically don't compete with them.

19:01You know, we leverage that technology and we do the remaining part, which is like, you know, how do you make it easy to access that, like, you know, all that technology and make it work on your data in your enterprise. The third thing that you can do with Glean, like, so we have a Google like search, we have a chat GPT like AI assistant, but then we also have an agent building platform. So you can actually build all kinds of agents to transform your business processes using AI. and in fact I think the best thing with AI is that it's actually bringing that power to you know to every business user you don't have to be a developer anymore to be able to build something you know complex or like interesting like you know you don't need developers you know for everything anymore like you know there are a lot of like you know agents that you can actually just build by expressing you know a business process in natural language to AI and then AI sort of like, you know, like does, you know, things like what you mentioned, like, you know, you don't, you don't do the reasoning is actually take the business process.

20:04It will create a multi-step workflow that you can then go and review and execute. So, so I think, yeah, we are definitely, you know, going, you know, towards a world where a lot of like, you know, how, like just how work happens is going to fundamentally change. And yeah. Yeah. So this agent builder, which you mentioned, what does that require from the models to work? Does that require models that are specifically capable of reasoning or being able to browse the web like operator or computer use? or is Glean really the product in the system that enables to take these LLMs and sort of piece them in a way together that they're actually executing on tasks on behalf of your enterprise?

20:55Yeah, it's a combination of that. Like today, if you just work with an LLM, that's not enough for you to actually build an agent and automate a business process with it. you do need some more work that needs to happen on top of it. For example, you need to figure out how to actually bring the right data to these models that's in your enterprise. And over time, there's going to be more and more features that you will get from the LLM providers. Computer use, for example, is an example of that. This is a feature. This is not really the model, but it's a feature that the LLM companies are actually providing to you.

21:42You know, it's a mechanism for you to sort of, you know, bring more data into the models, into the LLM reasoning. The way things will evolve is today, I think that you still need, you know, agent building frameworks and platforms that will actually sit on top of LLMs. The way we are helping our customers build agents is, you know, you get this, you know, agent builder UI, where you can just talk in natural language, converse with clean to build an agent together. Behind the scenes, we're using both our API-based integrations to enterprise data sources. But like that, the overall orchestration is being done by us.

22:22Got it. That makes a lot of sense. Speaking of computer use, and again, these models getting more agentic, and the fact that it does seem like a lot of the big sort of model providers, the open AIs, the Anthropics, et cetera, like they are leaning more and more over time into the enterprise, offering, you know, more and more ways to connect different parts of your, of your, of your business. How do you think about the landscape of competition when, you know, I'm sure many of your competitors are sort of building kind of like at the application layer on top of the models. And now there's also this, I don't know, new sort of a front from the model layer where they're coming at it as well.

22:58How do you keep the product sort of competitive when you have this competition coming from all different angles. Yeah, it's true. From a technology stack perspective, like an overall agent architecture, you're going to have the LLMs and model capabilities at the base level. You're going to have a data connectivity layer and a retrieval and search system. And then you have basically like, you know, the overall like agent building and, you know, like the reasoning flows, you know, that you're going to build on top of them. First, I'll make this assertion. A lot of things are going to change about work in five to 10 years from now.

23:45Majority of the work that we do today, we won't do anymore. It's going to be done by, you know, AI, you know, assistants and agents, you know, that are going to be helping us, you know, perform those tasks. Our work is not going to go away. like you know it's just gonna change a little bit like you know it's going to become maybe more creative more uh thinking oriented and like you know the base level working with data doing you know some research analysis on it like you know the models can actually do a lot of that work for us so so so majority of our work is going to you know change um the way the organizations are built the business processes like most of these business processes are going to actually also get automated you know, with AI.

24:25So, so one thing that I feel is that the body of work that needs to get done versus the companies that are actually doing that work, there's a big delta between that, you know, the white space is huge. And the, like, if all of us, you know, work super hard, like, you know, we're still only getting 1 % of, you know, what needs to happen done in the next one year. This is not the time where there are too many players and, you know, and not enough, like, you know, problems to solve, like, you know, within the opposite. Right. It's probably the opposite. Yeah. And so therefore, we don't feel like, you know, any sort of concern, like, you know, with competition.

25:05You know, our strategy will remain, you know, what it is today, which is we will continue to partner with all of these, you know, players. We'll continue to partner with the language model companies, with the cloud hyperscalers, and will, you know, fully leverage, you know, all the innovation that they are making because, well, like they won't be able to cover more than 10 % of what needs to get done. And so we have enough, you know, for us to work on. And so we can't, like, so basically think of us as, you know, sitting on top of, you know, like the core technology from these players as, you know, as they build more, like we also rise and like, you know, we start to build like, you know, the other things, you know, that we haven't built yet.

25:42So we think, you know, of all of these companies more as our partners, not as our competitors. But now, practically, if you think about as a customer, you're being pitched agents from 300 different vendors. And who do you pick? Who do you choose? I think Glean has a very unique architecture and a position in the marketplace where we're telling our vendors that we're not an LLM company. So when you actually think about your AI stack, you have to work with the language model companies anyway, and you can work with them through us or directly. We're fine with both those options. But what we are doing is we're giving our customers a horizontal AI platform, a platform that's connected to all of your enterprise information and data and knowledge.