RAG is the key for smarter productivity tools with Notion CEO Ivan Zhao

0:05Hi, listeners. Welcome to Notion. Today we have Ivan Zhao, co-founder and CEO of Notion, the the beloved productivity application for notes, tasks, and knowledge base. They recently launched an AI Q &A interface as well as a calendar application. We're super excited to have Ivan. Thanks for being here again Ivan. We are going to start with the hardest question, which is what is Notion? Notion is always pretty hard to define because it can do so many different things. But that's also our goal. We want to give people one tool that they can do their most work with. For a personal user, that means all your personal notes, all the planning for a trip or for your wedding.

0:42For business, for enterprise, for a company, that means all your documents, all your tasks, all your issues, calendaring, knowledge base in one tool. The reason we want to do that because there's just so much fragmentation in the market today. We wish like it wouldn't be nice as one place to do your most work. And our approach here is rather than try to cram all the use cases into one product, what are the underlying software building blocks? What are the Legos that power those use cases? Can we give users those Legos so they can be creative with software themselves? They can create and tinker their perfect workflow for their personal life or for their company.

1:25And none of this is new, by the way. like people back in the 80s, even 70s, tried this kind of building blocks approach to software. We're just trying to take a modern spin with cloud and with AI. What it's like to break the prison of application-based software. It's dramatic to think we've been living in a prison of SaaS fragmentation for the last two decades. But I do think it's actually surprising to hear eight points of view that is so obvious, which is like, of course, we want one tool where the data was interconnected. Why do you think people, why do you think more people don't try that to have unified tools and unified data underneath?

2:09I think people try for different angles. Like even fairly recently, there's this thing called NoCo, right? NoCo is like coming from this kind of like power user developer angle of wouldn't be nice everybody can modify this underlying software they use every day. That's one angle. It wasn't coming from the angle of the knowledge and data wants to be in one place. Language models sort of give another angle is the underlying knowledge in embedding space wants to be one place, wouldn't be nice in one place. The macro is also coming from the budget place. Wouldn't be nice rather than pay for five different vendors and all seed-based business just to pay one vendor and save some money.

2:53So they have different angles from different times. I would say we are more come from this kind of computing and medium and literacy angle. Like you and me go through school to learn how to read and write. English and Chinese, we've spent years to do that. We all know how to do that. The world, the same MacBook for most people are very rich. It's more like a machine to do typewriting or watching YouTube, not much more beyond that. It's not very creative, right? Wouldn't be more nice that more people can use their software more creatively, right? Because there's a separation between people who can make software and people who use software, that's why SF's rent is so expensive because we're the modern day Detroit or Manchester, right?

3:47We're the factory of the world. Notions largely come from that angle, which is the original angle. We were inspired by early computing pioneers. They thought about that angle, right? They thought about computing could just be like literacy. One day everybody can do it. I guess they didn't expect AI might make that even give a really interesting twist to it. Because now language model AI can not only to create software, but also do a lot of thinking working for you. So the future is pretty interesting. So for someone who thinks on, you know, span of like decades of, you know, what should computing look like and what were the most ambitious plans for personal computing, you know, three, four decades ago, like what are you most excited about seeing from AI broadly over the next decade?

4:42I think three, four decades, a bit too long. If AGI happened that time, computing might not be necessary. For this decade, I think one sleeper category is the drag, the embedding space. Decades might be too long, I would say in the next year or two. Now the language model can understand what you put into a computer. Understanding. So rather than you do the organization to make you retrieve the understanding more easily, machine can do that better than anybody else can. So before that, we use keyword-based search where you find your coworker who will remember that, that queue, where does that information sit?

5:23Now, just ask Notion AI and you get that in seconds. So that's one I'm personally really excited about. I think not enough people talk about it. And of course, the other one is like the agent, the workflow side that has a lot of buzz already. So that's interesting too. You and Simon said bet the company on AI and have real conviction. And as you are building out the team, what does the talent look like you have or need to make Notion an AI-first company? I kind of argue you folks are one of the earliest adopters of AI at scale as an application. So part of the question in some sense is you've built so many interesting things like what are the people that you now need to sort of build the next level of stuff in addition to what you already have on the team?

6:09In the early days, it was kind of just brute force. Simon's a really good thing. I built a lot of things and learned really quickly, right? I would say Notion is a company where largely people interested in interface and design a lot of full stack. From then, heavy folks and back-end people who scale. We have somewhat a small team of search, but we don't have too many ML folks, almost nothing. And in at least my learning, our learning in the past year or so, building for AI is okay you MO folks are important it's kind of like you no longer do a this deterministic thing that you can see how it works it's almost like a I don't bake but feels like a baking right you have to like do something get the thing ingredients ready run through rinse press the button and wait for a while as you does it come up or is it a different way a different sense of patience and different type of personality to do that well a lot of massaging a lot of Some of my friends call that probabilistic software engineering.

7:10Kind of like it. I think it's morphed into this sort of stochastic world, or at least partially stochastic. Yeah. So one is like maybe gardening feels like that way. I don't garden either. So that category of people are to me is pretty necessary. The other category is like people who are curious and learn really fast, right? It's like, okay, like the group of prompt engineer, language model sort of make everybody like a real time machine learning engineer. You just prompt the right, then you can get your stuff, right? And there's a lot of trick and techniques. And how does that plug into user interface?

7:54I think this category of people call AI engineer or something, there's some terminology form. They tend to be pretty young. they tend to be like we have someone like under drinking age working at Notion. They fit into that bucket. And I think both seems to work quite well. We don't have too many researchers at Notion. That's another one I think will be important, but we're fundamentally sitting in the application layer so it's more about applied side of things. So we're manipulating the models and making sure you can scale them to like user outcomes and think about that. And another part is like the scaling part.

8:33How do you scale to like tens of millions, 100 million users? It's a problem on its own too. It's beyond just a demo on Twitter, right? It's tricky. So you have often said that Notion is less a productivity company than an application building company. How do you think about the initial use case and what makes you believe people want to build more applications? I don't think people want to build more applications. What got me started in Notion, got us started in Notion, it's last year college, I read a paper by one of the computing pioneers, Douglas Engelbar. He talked about his papers named Augmenting Human Intellect.

9:15So every day we use software today, very much like application, when you go into one application, do one thing. But for that generation of computing people in the 60s, 70s, 80s, computers are a lot more, software are a lot more malleable. You can actually tinker and modify, right? Small talk, you can go into it and change the hot operating system work on the fly. That really inspired me. It's like today, people with software are so rigid, can we create a new breed of software that people can modify, can change and customize, and bring back some original ethos of those early computing pioneers?

9:51That's why we started Notion. The hard lesson for us is like you mentioned, most people don't want to create software. They don't wake up and say, hey, I want to create my perfect project management tool, my perfect knowledge base. The boss asks for something, they just have to get the work done. So in some sense, our learning and pivot is instead of giving people those software building tool, we have to package the software building blocks together as ready to use templates, as ready-to-use use cases that people can adopt really quickly. So you were one of the earliest adopters of AI in terms of application with any real scale.

10:30And I think it's impressive how quickly Notion ended up starting to work in this area. How do you think about how that impacts different aspects of what you built and what you're building going forward? And how does that impact that vision of saying, okay, we have this effective platform that allows people to both interact with documents or core use cases in simple ways, add things like calendar, but then also go in very interesting directions in terms of both the set of applications and templates they can use. Yeah, I think we're lucky. Like I mentioned, we're not trying to build specific use cases.

10:58We're trying to build a Lego bricks that power those use cases. What are those Lego bricks? Text editing is the one fundamental Lego bricks. Most software have that piece. Relational database, a table, is the one fundamental Lego bricks. Different form of permission, commenting. So we've been spending five plus years building those Lego bricks and feels like, boom, AI just jumps in almost like a brand new car engine and can power those Lego bricks in brand new ways. So it feels very lucky in that way. And that, because we've been building those Lego bricks and refining those, allow us to ship features, plug in with AI really quickly.

11:36We're one of the earliest one to launch AI writing for productivity software at scale because we've been spending years building a text editor. We can do AI-powered database table features really quickly because we've been building relational databases. We've been building a knowledge base for a long time, so we launch AI Q &A really quickly, fairly quickly. The Rack system on top of Notion, because we have those Lego bricks. So in some sense, it's kind of like just the right moment, right time for us. How did you begin to resource and prioritize this effort? because you're like, ah, magic. We have this engine.

12:14It applies for our Lego bricks. And then you start shipping pretty quickly. But I think there are a lot of organizations right now trying to figure out what to do with AI. And so, you know, in terms of like designing the features, prioritizing that effort versus everything you're already doing in a rapidly growing software company. Yeah. I think I had the conviction. My co-founder, Simon, actually, they all had the conviction. Fun because we all live in the mission, right? And OpenAI initially still is in the mission. And some of my friends, especially Simon's friends, work at OpenAI. I remember we'd go to their office.

12:50They do in the Dota days. They were like, what is this company doing? Kind of interesting. And Simon and some people in Notion saw early demos of GPT. It's like, what is this thing? Spin out text, sometimes gibberish, sometimes useful. I personally, I have to admit, I slept on it. On GPT-3, even saw GPT-3, it feels like, what is this thing useful for? It's like, yes, for marketing, for content writing, for creative first draft. Didn't really click for me. Personally, for me, it was fortunate enough to saw early preview GPT-4. And that's like, oh, wow, this thing can think. It can reason. It can know how to do things.

13:32has this little bit workflow power to it. That's a big aha for me. And like it just give me, it personally give me so much conviction that this is gonna change everything. If you think about what knowledge work is in, why do we use software? Fundamentally, SaaS, software, this is all we're all in the same information, people paper pushing activity, right? It's like a piece of paper coming in front of you, a human, like change a couple bits, push to another human. Language model can do some form of this now. So that just like gave me the conviction, like this is gonna completely change everything we do with the computer.

14:14And after that, we sort of just bet the company on it. Like we're lucky enough to have those Lego bricks. And then which Lego bricks can work well with AI, which doesn't, which we're trying to figure it out. Who inside a company are good with this technology? We have search, but it's not like we don't have a lot of ML folks So you need to hire more ML folks need to Get people inside a company to have similar convictions so we can move in the same direction. It's quite interesting. It's kind of like So must have dinosaur feels like when astral hit the earth and what do they do? Yeah, there's a lot of change coming for sure.

14:51It's a lot of change. Yeah What do you think is missing from the capability set? Because to your point, I think a lot of people weren't really thinking about AI too much until ChatGPT and GPT-4 came out. And there was a period of time where 3 and then 3.5 and you started to see the capabilities incrementing up and entirely new businesses are suddenly enabled with each sort of step with the next GPT-level model. You know, GPT-5 or to your point, RAG adds a lot of capabilities. What are the biggest missing gaps for you to take full advantage of this technology? Is it future reasoning? Is it better thinking and knowledge?

15:25Like what's the... Yeah, I think all of that above, to me feels like technology is... We're in the tech business. Technology is fundamentally about trade-offs, right? It's like the plastic can do things that wood cannot do. We discover plastic and then we figure out new things. We can bottle water like this. Before you cannot bottle water with a wood table, right? So all of a sudden we have this thing called language model. They have a new characteristic that deterministic software cannot do in the past. And we don't really know how it's made fully. So every month, every week, if you're on Twitter, people discover new techniques to get more out of this.

16:07And for companies, entrepreneurs, they're also making trade-offs discover how the market reacts to these capabilities. this new what, this new language model. And so it's a constant evolution of cycle happening really, really fast right now, right? I think with that mindset, what are the dimensions? On the technical side, on the technology side itself, yes, the model gets larger context windows, more reasoning, better speed, smaller footprint. Those are all great. like for Notion to power the workforce, I would really need like we learn like GPT-4 is smart, a cloud too is smart, we need that intelligence to do reasoning or for the tech summarization, cheap, fast, it's better, right?

16:59That's the technology side. And in my opinion, there's so much about human behavior as well, just like inertia in our personal behavior, companies' risk tolerance, And that's slowly evolving as well. Like Steve Jobs always talk about, you cannot make something too new. You have to be largely the same and change one thing and two things. Virtual App below, the off-white guy is like 3 % difference. Just push the boundaries so people can accept it, but still also new. To me, it feels like language model, power, application are kind of in the same phase. If it's too different, people are like, what do they do with it?

17:40It's such alien behavior. It has to, the Rack is pretty nice because largely existing behavior but better output. Can you describe the AI Q &A product for people who haven't experienced it? Right. Essentially everything you put in Notion, Notion helps you remember. And this is not just applied to Notion, it really applies to most Rack systems. Like why do we use a computer? We need to store things and we need to recall things. Before language model and RAC, the recall largely happened based on keywords. The keyword has to be precise or there's some lexical tech tricks that you can recall easily, imprecisely.

18:22What RAC happens, language model can actually understand what you're putting into there. So you no longer need to organize your information in Notion. Whatever you throw in there, you can find it later. What that means is for a person or for a company, for a team, you can have perfect memory. And not only have perfect memory, the right piece of information, if we design our software right, can push to the right person at the right time. That's probably more than 50 % knowledge work. We're still perfecting the system. I think we're one of the first on the market that apply at scale. We still have somewhat a waiting list because it's hard to do this at scale still.

19:02But for a company, for a team, before search is one of a weaker point, but with the Rack, you completely change that. I change how I use Notion. I can just ask a question to Notion, like how large, when are we moving out of the SF office to a new office? As someone in the company wrote in some documents, I don't have to ping three different people to find the answers. If it's in Notion, go find it for me, right? Everyday engineers, designers, operation people just keep asking each other on Slack or in email such a question. Each question is 10 minutes writing the answers, 20 minutes to find answers, and there's a delay in the middle.

19:43With Notion Q &A, you can completely cut that into seconds. We're just at the beginning of what RAG can do for work. It's pretty amazing. I feel like ragged and embeddings are very under-discussed or under-appreciated in some sense relative to the impact that they really seem to be having or starting to have. And I think Notion Q &A is a great example of that. I guess the other thing that you folks just launched is calendaring. And if you can't talk about it or if there's nothing to talk about, that's fine too. But I feel like one of the really interesting things that people are talking increasingly about is agents and sort of the agentic world.

20:15And there's a lot of capabilities missing to really make those valuable. But in the context of a calendaring application, you could think of all sorts of ways that having AI act on your behalf or help understand things can be incredibly valuable. And so I was just curious how you think about the application of AI relative to calendar versus, you know, some of the core information related things that you just talked about. Maybe we can group AI stuff into, at least in my mental model, it's direct knowledge information retrieval is one bucket, knowledge bucket. then there's this workflow bucket right use the word agent that's in that bucket counter somewhere in that bucket why do we need to meet because why do we need to counter because we need to meet and when you schedule time we need to figure out exchange some kind of bits between my brain to your brain right can that bit can I exchange to be dumped by a language model maybe and can the meeting Any time be done by scheduling be done, that's like a baby step, right?

21:15And most things we do has this kind of time dimension to it. Can language model help us shuffling our schedule? Yeah, it feels like there's also the information retrieval piece of it because if my calendar auto-populated everything I need to know about the meeting or the people attending or other things that's incredibly valuable as a user of a calendar. And so I just feel like there's a lot of these things that kind of tie in together both in terms of the coordination which you mentioned and the workflow and then separate from that there's just. What do I know about this person? The calendar part is the simpler part of the workflow.

21:45The holy grail is kind of like, can just the agents, robots do all our knowledge work for us? It's a really interesting framing that I didn't have before of a bunch of the work you're doing at Notion actually eating into communication. It's sort of obvious in retrospect, but if you look at what you describe of like, Am I really going to slack back and forth about this thing about when we're moving, if I can just know, I'm in motion, help me know. Or with calendaring, the most intelligent version of it is, well, do I need to have that meeting or can you tell me what Ivan was going to tell me? I know.

22:18Why do you need to communicate? Because there's something, the work cannot be done asynchronously or by the software itself. Then that's why you talk. Yeah, it's kind of interesting. I think maybe it's an interesting question like, are we going to communicate more or less with language model? I probably feel it's probably less. The agent side essentially bet on language model, that's the communication. One question I have for you, just going back to the implication of RAG and you can be my brain and do my organization for me. What if my brain is really disorganized? Do you think that this changes the amount of work people should do in systems like Notion at input, right?

23:04Like, you know, should I be designing my knowledge base in the same structured way or kind of just dump it all in stream of consciousness? I think organization might be, we might be moving away from the organizational world. Why do you need to organize because you can retrieve? why do you have index? Like index initially are file cabinets and the little index are sitting on top of it so you can find things quickly, right? And they're indexed based on certain names or certain dimensions. But embedding and Rack sort of, it's just you have semantically connection of all the things you throw into this file bag and you can bring it out however you like.

23:47So I think we might be moving past the need for organization. organization, that's really liberating. That means on my phone, imagine this experience. I'll have a new idea where I see a whiteboard behind me. I just take a picture or write something, dump it, and Notion is going to organize for you, right? So then that's become my perfect memory to start. Later, it could be my perfect assistant to help me do something with this knowledge. That's the vision we're moving towards. That's super exciting. So you are, this is a question from Michael Mal, you've been a long time for us with. You are a student of history.

24:25You mentioned Doug Engelbart earlier. I know you think about the transition in terms of like Alan Kay and what he did in terms of simplifying many of those concepts for a broader audience around computing. Vernon's question was, what lessons in history do you take that inform your point of view of how to treat AI strategy with Notion now? From a prior revolution in computing, how does that help you decide what to do? A lot is intuition. I think understanding history gives you a sense of, history doesn't repeat itself, but rhymes. So, okay, which phase are we in? I personally think we're sort of in this kind of bundling phase.

25:09like who said this like there's only two ways to do business bundling and unbundling and actually during the break I was reading a Chinese novel Romance of the Three Kingdoms and the opening sentence for that is the empire long divided must unite long united must divide that's as always been business is the same way too We're in the bundling phase. I will say the SaaS, it's sort of this unbundling fragmentation phase. If we trace back to SaaS, why is SaaS happening? In the mid-2000s, before that, everything's running on Microsoft. That was like a bundling phase. Early days of PC, there's so many different applications.

26:01The first version of the world-star, world-perfect, different text editors, There's D-base, different database software. The funny fact of D-base, it's like they start with D-base too because there's so many company go bust that it sounds like if they start with D-base too, people has more credibility. It feels like this product has been around for a while. So that's the eighties. Nineties was this kind of bundling phase because Microsoft has OS layer underlying it. And the saddest is because the web becomes good enough to run software, right? Then we have this unbundling phase, a fragmentation phase, and then with the last 10, 15 years, the money is cheap, easy to create a company.

26:43There's so much. Too much now, it feels like. There's information so fragmented. And now the new technology happening is AI language model. And if you build more with it or just think more with it, language model wants information to be one place. wants the endpoints to be connected so it's easier to, it's hard enough to like a conversion language model to do what you want, but imagine top with different endpoint, that's even harder. And so we're in the bundling phase because the macro, but we're also in the bundling phase because language model, I believe wants the things to be together. I think that makes sense.

27:22I also feel like we're in the bundling phase because the nature of how founders think about their businesses shifted. How so? I think that it's interesting because Because I remember, I don't know, 10 years ago, I used to argue with people about, oh, you should really buy other companies or integrate or sort of pull all these things together. And in consumer, that actually happened, right? Like Facebook bought Instagram and WhatsApp and other things. And they effectively created like a bundle of social products that they could cross use in different ways for distribution or other things. But I feel like what happened is we had a series of highly technical founders because we shifted in the Facebook era from Cheryl becoming CEO to COO.