The future of enterprise search and AI-powered work productivity with Glean’s Arvind Jain | E1916

0:00Who's going to win long term? Who will have the best models and which will be the most market share? These closed models or open source models? I think in the future, majority of the majority of the best models. models. I would say 80 % of all AI inferencing or people building AI applications is going to be based on open domain models. And some of those will be fully open domain. Some of them could be open domain, which are sort of supported by enterprises. And that's sort of really how the industry has progressed over the last two decades. It's just really hard to beat open source. This Week in Startups is brought to you by OpenPhone.

0:46Create business phone numbers for you and your team that work through an app on your smartphone or desktop. WIST listeners can get an extra 20 % off any plan for your first six months at openphone.com slash WIST. Lemon.io Need to speed up your product development without draining your budget? Hire vetted engineers from Europe at lemon.io Go to Lemon.io slash twist to get 15 % off for the first four weeks. And Gelt. It's time to take control over your taxes. Discover how Gelt can help you to manage and optimize both your personal and business taxes. Visit JoinGelt.com slash twist now. All right, everybody.

1:32Welcome back to the program. Today on the program, we've got Arvin Jain. He is from a company called Glean. What is Glean doing? We're going to find out today. They're trying to get corporations, enterprises, to use AI to help them sort, make sense of, and search their data. We'll hear all about it from Arvind. Arvind, welcome to the program. Thank you for having me. Tell me, what are you building and why is it important? Think of Glean like Google or ChatGPD, but inside your company. It's a product where people can go and ask any questions they have. and Glean will use all of your company's knowledge and data and information to answer those questions to you.

2:12That's what our product is. We are an AI-powered search engine. We are an AI-powered assistant that helps people get more work done. Got it. And so you're not asking the chat GPT, for example, to answer questions or make a marketing plan. This is specifically to search the data inside your enterprise. That's right. And ask questions against it. So I see on the site, you mentioned every department that any company could have sales and, you know, marketing, etc. Which categories? What's the beachhead market? Where are you being the most effective for your customers? So typically, Glean gets deployed company wide, typically Glean will sell to CIOs.

2:53Our top users do tend to be engineers, support people, folks in sales, like those are the three biggest user populations, you know, that we have. But in general, this is a product that actually is useful to every single employee in a business. And therefore, we don't go and sell the product to individual departments. We typically go and sell through the CIO. So the CIO is evaluating new technologies and saying, this is going to go across all the verticals in the organization. We need a solution to ask questions in every department. That's right. Well, that means you're going up against, I assume, like Intercom, HubSpot, Zendesk for support then.

3:35Or are you sitting on top of those systems? We typically sit on top of those systems. Think of Glean as an assistant. It's a layer, it's a connective tissue that connects your knowledge across all of your different systems. So while you may be using Intercom or Zendesk as your customer CRM and your support people use that as a system of record, but when they have a case that has come to them, so they'll open the case in Zendesk or Intercom and they need to actually resolve it. to resolve that big and actually help them find the right answers. And sometimes those answers may be in knowledge articles in Zendesk, but sometimes they may be answers that are in some Slack conversations or in some internal JIRA issue.

4:23And sometimes the answers are with people that you can actually go on. So Glean will actually help you find those people or that knowledge that sits outside of those systems and help you answer those questions. So it's sort of like... What's an example of that? Give me an example. What's the best example? Well, I mean, let's say that there is a as a support agent um you know somebody files a request that my product has stopped working for some reason and like you know what would have happened is that like maybe there's a needed release that got rolled out and there's a bug in that and right now people inside the company are actively discussing you know that issue in in some slack channel it has not made it it made its way into your knowledge articles here so when when you get a you know request you know from your customers you know you like will actually quickly help you tap into like you know hey is this like has other people run into it and like other conversations you know inside the company that would help you sort of figure out a quick answer you know back for your customer how do you deal with the fact that a lot of this data is confidential and maybe not everybody in the organization should see it there's permissions in each of these systems but you're going to index the whole thing so somebody could ask about salaries in the company or you know different things the language model has been trained on all this data i assume you're yeah training your language model and all this what what llm are you using yeah so so first of all like you know we are um llm agnostic so we can work with uh gpt4 or gemini or you know which one are you using right now we use all of them these all of them like typically we let customers make a choice like you know what language models you know they would like to use um which one do they pick most often actually like most of the times they will you know give back the choice to us so like you know we get to choose.

6:02I think right now we started out with, you know, I think majority of our deployments right now are using GPT-4. With GPT-4, when you put that data in, how do you know if the model is the GPT-4 model by OpenAI assures you that the customer data does not go in there? Or do you have it off-prem? How do you manage that issue with the language models? Because a lot of CIOs and CEOs are really concerned about giving OpenAI their data to train it on. Yeah. So see, you're absolutely right. If you think about using AI in the enterprise, first of all, your data inside the company, first of all, it's private to you as a company.

6:41But second, within the company, there's governance on that data. Not every employee can actually use all the information that exists within the company. So Glean actually solves both of those problems. So number one, we're not actually training or fine-tuning models like GPT-4. We're actually using them only as summarization and synthesis engines. The way our product works is that when you come and ask a question and you're one of the employees in the company, what we will do is first, based on that question, we're going to use our core search technology and we'll assemble the right pieces of knowledge and information that we think is going to be able to answer that question that you have.

7:19and we will actually restrict you. So like we know who you are and what content you have permissions for. So we'll only let you use the information that you are actually individually authorized to use. Now, once you've actually gathered, you know, this information safely, now we will actually take the snippets of this information and ask an LLM like GPT-4 to summarize that information. Do you trust OpenAI? So we actually work with Azure, you know, to use GPT-4. and the way we work with these model providers is that we have a contract with them where our customers are guaranteed full privacy for their data.

8:00Their data is never logged outside of their own clean environments and Azure or Google don't have the ability to actually go and train any models on that data. So our customers get full assurance. So you trust them with that function because a lot of CIOs have been a little bit concerned watching some of these, you may have seen, you see the viral video of the CTO of OpenAI talking about Sora last week, and she couldn't answer the question of like, what training data was there, and she wasn't sure. And it kind of felt like she was lying, I think, based most people's thing there. So it does seem to me like the big challenge here, you tell me if I'm wrong, is that using these third party models, even even Even on Azure or Google Cloud, are people nervous about that and they want to move to having, say, an open source one on-prem or just in their own cloud?

8:54Well, see, different customers are at different level of both paranoia and security requirements. A lot of the companies today, a lot of enterprises are now comfortable with storing their enterprise information in the big cloud vendors like Google or Microsoft or AWS. A lot of business technology and systems run in these systems. And so the trust level for these, the big three cloud providers is actually quite high. And if you think about it, like, see, AI is a new thing. First of all, already my business data is in these systems. Now I'm also using some additional AI models, again hosted within Google or Microsoft.

9:42As long as I get those VPC controls, I'm actually comfortable with that. Most of the customers feel that way. And if you don't trust Google or Microsoft, then, of course, you typically are running everything on-premises. And we, as a company, we support also hosting models ourselves. So if a customer wants to use an open-domain-based model as the core LLM that's in their clean experience, we also allow them to do that. Yeah, I mean, there was a big instance of a bunch of Samsung employees, I guess, using ChatGPT4, and then their source code and other information was then trained into ChatGPT4.

10:29I'm sure you've seen that, and I'm guessing that comes up with CIOs. Can you explain what happened there? Yeah, so in that particular instance, the employees in the company, they were actually using the standard ChatGPT product and they were actually pasting their code inside of that. Code or sensitive documents within the company. You were posting that within that ChatGPT interface and then asking ChatGPT to do some work on it. When you actually use these services directly, these are meant to be consumer services. If you use them directly, you don't have any controls. Every data that you actually put in that system, like OpenAI has is allowed to actually go and train their future models using that information.

11:15On the public interactions, right? On the public interactions. And that's what happened there. The way to make sure that you are not exposing your private data as a CIO to ensure that your employees are not sending information to these public consumer-based products, that's when you use a product like Glean. Because if you use Glean, now you have a very safe and secure environment where people can still go and ask questions. And we will make sure that any information that is being sent to Azure or to Google is actually following that contract and that security agreement that you have from them that they're not going to use that information to train.

11:57Juggling multiple devices and apps to run your business is a mess. OpenPhone is here to make it simple by simplifying your business communications with one easy-to-use app. OpenPhone has rethought every detail of what a modern business phone should be. And here's the magic. It works through a beautiful, elegant app on your phone, or you can just use it on your desktop, making it super easy to get a business phone number for your entire team. And you know how brilliant OpenPhone is? My teams use it every single day. My sales team loves it. My ops team, they use it all day long. And here's the features that we love.

12:33You can create a shared phone number, like customer support, with multiple employees fielding all the calls and all the texts to that one number. At my investment firm launch, we pride ourselves on replying to every single call or email instantly. And OpenPhone is the number one rated business phone on G2 for customer satisfaction. So here's your call to action. Super easy. OpenPhone is already affordable. Starts at just 13 bucks a month. But Twist listeners get an extra 20 % off any plan for the first six months at OpenPhone.com slash Twist. And if you have existing numbers with other services, no problem.

13:04open phone is going to port them over easy peasy lemon squeezy no extra cost head over to open phone.com slash twist to start your free trial and get 20 % off how do you compete against the the native tools you know getting more and more robust so you can't possibly write an llm that's going to work as good as you know the one that's built into you know salesforce eventually just Salesforce build in a language model yet or no? So typically, like, you know, companies like, you know, SaaS companies like Salesforce or Atlassian, you know, or, you know, any of these systems, they're not building language models.

13:44Language models are built by... No, no, no, but they're building language models into their product. So, and you're not building language models either, right? Well, so it's complicated. So we build language models. We build smaller models to sort of build semantic understanding of your company knowledge. But we, as well as Salesforce and other SaaS product companies, all of us use APIs to these large language model providers like GPT-4 or Gemini to do some work. So typically in that model, what happens is that you're basically sending prompts to these models and having these models do some work on that prompt and return back a response.

14:29And you sort of create these AI experiences within your application. So now to think about like, you know, if you think about Salesforce, they're going to actually have some AI features within their product. You know, Coda will have some features within theirs. Jira will have some. So, yes, so you're right that, you know, every application in the future, you can imagine that they will have some AI smarts. They might even move to the chat interface, right? You know, and yeah, they may have a chat interface in addition to actually do some of the work that people do with those products. For example, you know, today, like if you want to create a new issue in JIRA, like you like, typically you'll go in that app and click a button and then fill a form.

15:06but you can imagine that they may have a chat interface that allows you to go and create that new issue using a national language interface. Those things may happen in the future, but that's not what Glean is actually solving for. Glean is solving a different problem, which is that if you think about your work inside a company, it happens across many different systems. I'll give you one user journey as an example. So as an engineer, let's have to actually go and build this new technical component. And so that journey for me is going to start with first talking to people. I want to be actually having some conversations like this one, like on Zoom, I'm going to be talking to some other engineers, talking about design choices.

15:50I may have some conversations in Slack, you know, where we're talking about like, hey, like, what about, you know, this approach versus that approach is actually a, you know, a Jira, which actually tracks why am I even building this technical component? What were the problems that we're trying to solve? So first, you have all of these different things. Then at some point, I'm going to actually write a design document, like maybe in Google Drive, to describe my design. And then later on, I'm going to actually write code, and that's going to go in GitHub. And so if you think about this whole journey, all the information about this project, it actually spanned all of these different systems.

16:24So now think about six months down the line, somebody comes and asks a question that, hey, why did we use this programming language to build this component? Where is the answer? It's not in one place. It's actually, you get that answer by actually consulting, you know, all the knowledge that sits in like those five or six different systems. And so that's where the power of Glean comes in. Like if you think about, you know, your work, we are tying together knowledge from all of these different systems in one place. And we give you as a user, like we remove that burden from you, like, you know, hey, where should you go and find things?

16:57Where should you go ask questions? uh and it's a great place to start if i was joining the company or i'm the cfo and or the chief operating officer and somebody tells me about project bluebird and i don't know what's going on project hey give me an overview of project over bluebird and he gives me all those but if i'm not the the question i have uh the next question i have there because that's kind of a cool feature to be able to go across the entire enterprise so i totally get that but there's a discussion going on in a slack room that i don't have permission to permission for so how do you deal with that maybe there's a bluebird project bluebird slack room it's got 20 people in it but i am the coo and a cfo i don't have access to that room i was never invited yeah how do you but you have that in the llm so how do you let me know that there's a conversation there that i don't have the rights to see because of the way slack works or jira works like the coo doesn't even have a jira that's right they're not having a github account so how do you deal with permissions?

17:57Yeah. So two questions. First of all, the way our system works, there's no data. None of your enterprise data is actually in the LLM. The LLM is basically just the standard language model that is trained on the world's public knowledge. We're not using it to store knowledge. But now, the way Glean works is that when we actually connect with all of these different systems, we have actually built an understanding of how permission works in those systems. So, for example, when Glean connects with Slack, it knows the concept of channels. It knows that certain channels are private, some of them are public.

18:37If it is private, it knows who are the members in those channels. So now, we're going to index every single message or conversation, and we know exactly who are the people who have access to it. And all of this information is stored in our search index. Same for a document on Google Drive. We know who are the collaborators on that doc, and we store this permission. This is unique about the Glean technology, so it's fully permissions aware. Now, when you come in and ask a question in Glean, you have to be, first of all, signed in. You have to present to us your identity, who you are. We are able to now retrieve documents from the index, but only work on the documents that we already know that you have permissions for.

19:24Building that, that's part of our core technology is to understand these authentication and permission models in each one of these individual labs and make them work. So you know Jason, the COO, can't see Project Bluebird. I'm not in that group. So when I do a search, you won't show me anything. yeah we won't even tell you we won't even tell you that hey there's some useful information but we're not like we can't share it with you talk to somebody else because like sometimes even that is dangerous like even i was about to say like let's say you did a search who's on a performance improvement plan that's right it's like there are seven conversations about performance improvement plans happening with these people in them with these names you're like uh wait a second who are the seven people who are in a just the fact that there is a file in and of itself is information.

20:14And do these, you know, Slack has pretty robust AI coming on board notion and Coda also have AI built in, have they built API's into it? Or are you just having to rebuild all that from scratch against their services? Or are you doing the search in Slack, as if I was logged in? No, we actually returns, we do like our we do search natively in our system. So the way our system works is that we bring content from those systems and index them in clean so as content is being produced as somebody sends a new message we get a notification and we'll then use you know take that message and index that in our system so this is this is continuous this is done in real time all the time and now when you come into a search like that search is entirely served from within our system and that's important because for two reasons one like you know like you need to be fast.

21:06You can't actually in an enterprise, you know, you will have 1000 systems, you know, that you're using, you can't actually, you know, when a user comes and ask a question to you, you can't actually send a message to all, you know, hundreds of them and wait for responses to come back. Yeah, no, search correlates to search usage correlates directly with the speed of return. That was Google's big lesson, right? Exactly. Faster. Yeah, people use it more. Yeah, in fact, that's, that's, that's one of the things I worked on at Google, you know i was actually making it fast but the second thing is the second thing is also like search is a hard problem it's sort of like magic like you know you come in you type two words and like i need to sort of now figure out from those 10 million documents the one that exactly you're looking for so there's a lot of work that needs to be done to build a great search and i think like like what we have seen typically is that like each one of these individual sas products like you know they're they don't have so many resources to put on search like we have hundreds of engineers and to actually make search better.

22:02It's such a good point, right? Search is like an afterthought for them where they may just use some open source library. They never update their search. Even like I've been complaining about Twitter search since Twitter was born. And there was a third party called Sumise that they bought to make their search a little bit better. And that was 10 years ago. And it has gotten better. Because you think about like enterprise software companies, they don't win customers that way. Like, you know, Jira and Asana are competing, they're competing on features, not by saying that, hey, my search is better than yours.

22:31So I think that's another thing. So like, you know, to really solve the search problem, you have to you have to do a lot more work. And also just one more thing, you know, on this, like, you know, why, why, you know, thinking about search in the way we think is important, it's really important to sort of take all of your enterprise context. And that sometimes gives you signals on, like, you know, both, you know, what information is actually relevant and important and to whom. So one example, let's say that, you know, somebody writes a, there is a document that, you know, that talks about benefits.

23:06But whenever somebody asks a question in Slack that, hey, like, you know, where's our benefits policy? Like, you know, somebody in HR shares that document with you. So there's a lot of, you know, like there's exchanges that are happening in Slack, which tells you that, hey, this particular document is authoritative for that answer. So Slack's great training data in that way. Yeah. And Slack just being one example, if you think about there are these interconnections between how a Jira issue was created and how it's referenced in Slack conversations. When you think about your enterprise knowledge, it's a graph.

23:39There's knowledge, there's a lot of different pieces of knowledge, and they have all interconnections with them. And similarly, there are people. and people, of course, we're talking about there are engineers, there are support people, there are salespeople. We build these learnings that engineers are actually clicking on this or using this document a lot more than salespeople and vice versa. All of those learnings is what enables you to find out what is more relevant information for whom. That's the core of what cleaners. And that's why it's so important to actually have that full enterprise wide view of your people, as well as your knowledge.

24:20Right now, startups have to do more with less, we all know that it's rough out there, folks. So if you need great tech talent, but you don't have the time to interview dozens and dozens of candidates, you need to check out lemon.io. Lemon.io has thousands of on demand developers to choose from. And these devs are vetted, experienced, result oriented, and they charge competitive rates. Great developers can be incredibly hard to find. and when you do find them, it can be hard to integrate them into your team. Lemon.io handles all of that for you. Startups choose Lemon.io because they only offer handpicked developers with three or more years of experience and strong portfolios.

24:52In fact, only 1 % of candidates who apply get in. And if something ever goes wrong, Lemon.io will get you a replacement ASAP. You know what? A bunch of our launch founders have worked with Lemon.io and they've had great experiences, which is always good to hear. Go to Lemon.io slash twist and find your perfect developer or tech team in 48 hours or less. Go to lemon.io slash twist and find your perfect developer or even a tech team in 48 hours or less. And twist listeners get 15 % off their first four weeks. What a deal. Stop burning money. Hire developers smarter. Visit lemon.io slash twist. What's really great as a byproduct, and you can tell me if any of your customers are asking for this.

25:29There is a concept of compliance and legal reviews. So, for example, people think like their DMs on Slack or their emails, even if you delete your emails. Those things are stored if you have the settings done properly, like in Google Docs or Microsoft Teams. And so because you ingest everything, you do have the ability to do a god mode where the compliance could say, hey, did anybody say this? Let's say it's insider trading. You know, did anybody share Project Bluebird? Let's say Project Bluebird was an acquisition. Did anybody say the word Bluebird? And you could actually see across all documents.

26:04Does that exist like a super god compliance mode? There is a compliance mode which is highly restricted and it's available only to your governance and legal teams for exactly the use cases like that, eDiscovery. Also, for privacy compliance, for example, a big use case that's out there is you have an ex-customer or an ex-employee who comes and tells you that, hey, delete all my data. right, you know, and then you have to sort of, you know, there are laws that actually require you to do that, like, you know, and so how but how do you figure out like, no, where is all that information? Where is that data?

26:47Facebook had this issue because Facebook had been doing backups and backups and backups. And that's why when you delete your account, people are like, oh, you know, they say it takes 30 days. I think that's because they have all kinds of mirrors and mirror images of data so many different places. They want to be thoughtful and thorough about it. It would seem like a CEO God mode would be one of the great features of being able to look across this whole amount of data. If you're working at a company, you should just know by default, anything you do on your laptop is your company's. And every email you send, never use corporate devices to order from Amazon or do private communications.

27:27Gosh, it's 2024. I don't know why I have to say this to people, but I'm shocked that people will, because I'm on the board of so many companies or investments. Some story will come up that people were sending things to each other on Slack or Teams or doing something on a corporate device that is completely insane that you should not be doing. That's right. Yeah. And I think from our perspective, we help companies from a compliance perspective there. But Glean is actually not a system of records. So that's not like, you know, another system that you do worry about, like in the sense of when you need to delete data for somebody, like, you know, you don't have to actually go and explicitly go and delete that in Glean.

28:08Well, you do have to get rid of the data in Glean if it's indexed it, right? Because you have to be indexed? We stay in sync with the actual system. So like if deleted in Slack, it's automatically deleted in our system. Wow, that's super complicated to take care of all that, huh? Yeah, that's where the complexity is. But actually, it's an interesting thing. Like, you know, you talk about AI, you know, like everybody wants AI. and this is one of the key problems that businesses are running into which is like look you know we have all this information in our company and yes like you know we've set some rules you know permissions but you don't always get it right like you know oftentimes you know there'll be some document somewhere that like you know sensitive and the person in hr like they didn't know how to set correct permissions they made it open to you know everybody in the company and and you start living with that like because nobody can find anything anyways like who cares like there's a talks somewhere that so you know but let me ask him yeah we know get finished i was just going to say that that's so that's a that's a big issue today like you know with like ai because now ai does all of that work for you like you know like in this new world you just get to ask questions right and and there's this ai you know like for example our product is connected with all of the company information and it's going to actually answer questions back for me so it sort of makes you know these governance gaps you know like you know you're going to pay for it now like you know they're going to become a big problem problem and so that's one of the things that we hear a lot from, you know, CIOs, you know, they feel AI is powerful, but they're also scared of it.