Rahul Roy-Chowdhury: AI as a Tool for Co-Creation at Grammarly

0:05Hey everyone, welcome to Generative Now. I am Michael Magnano. I'm a partner at Lightspeed. From the moment you sign onto your computer at work, you are inundated with emails and messages. And that's exactly where Grammarly comes in. Grammarly has been using AI to streamline and improve communication well before Gen AI was a thing. I sat down and talked with the CEO of Grammarly, Raul Roy Chowdhury. He's a great perspective on AI products. He's Grammarly's former global head of product before he came CEO. And then before Grammarly, he spent 14 years at Google as a VP and product leader across many different categories.

0:41We talked about the future of AI agents and tools, privacy and security when it comes to AI, and why Raul sees Gen.AI as a tool for co-creation. Check out this conversation with Raul Roy Chowdhury, the CEO of Grammarly. Hey, Raul, thank you so much for doing this. Michael, great to be here. Thanks for having me on. You know, I'm really interested in hearing a little bit about your experience. I feel like you've had you've had quite the journey. You spent many years at Google and now obviously you're the you're the CEO of Grammarly. Would you mind just taking us taking us back a little bit and giving giving us a little bit about your story?

1:17Yeah, yeah. So if you go way, way back, I was actually in a PhD program in AI. So my initial goal in life was to be an AI researcher many years ago, before AI was cool. Yeah. I dropped out of the program. I didn't finish my PhD. I think it's required to drop out of some sort of an academic program. That's what I did. Especially if you're going to be a CEO. That's right. That's right. So I did that. But I've always been thinking about AI. It's always been something that's on my mind is something I've had a lot of interest in for many decades. And man, just seeing the changes that AI is driving today is absolutely incredible.

1:53Incredible. In my career, it's been interesting. I've had a pretty interesting kind of vantage point to see some pretty big changes in tech. So, for example, like one of the big changes was I was working on the Chrome team as I spent many years on Chrome. and we saw the change from desktop to mobile. And, you know, we were a successful product. We had built this browser for desktops and we were doing quite well. And then mobile came along and it represented a really interesting kind of way to think about what is a new platform? What does that mean? Initially, we thought, well, I guess we'll just shrink down the desktop product and just ship the shrunk product onto on mobile devices because it seems like a smaller version of desktop.

2:39And it took us a while to recognize, no, it's not a shrunk down desktop. It is a completely new platform. And it seems obvious in retrospect, but it wasn't obvious then to us. It took us a while to get that journey. And then I just saw the incredible opening up that mobile presented to access to computation. People got their first devices in many parts of the world through a low-end Android smartphone. And it was transformational for billions of people. So that was awesome. Then I saw the big shift from on-prem to cloud, you know, kind of seeing how that opens up the aperture for any entrepreneur with a good idea, any founding team with a good idea, getting it built without worrying about all the operational details, being able to scale their business extremely efficiently.

3:27Again, transformative. So it's really awesome to see these waves of tech transformation. You know, I saw mobile, I saw cloud, and now AI is bigger and more consequential than any of those other things. And we're living it. I was actually going to ask you that exact question. Like, how would you compare what is happening or what will happen with AI to these other platform shifts? Sounds like you feel like it's going to be even more consequential than the shift to mobile, the internet, cloud. Is that right? I think it's going to be way more consequential. But I also think that it's going to take a while for AI to fully play out.

4:02There's always a bit of a hype cycle that ends up happening with these things. And we end up always overshooting in the short term and undershooting in the long term. Like, you know, even if you look at cloud, you know, AWS launched in 2006. This thing has been going on for 18 years and it's still not done yet. There's still kind of people are moving from on-prem to the cloud. So these platform shifts are massively disruptive, but they also take a while to play out. And so we just got to recognize that, you know, there's going to be a journey for adoption and value creation. You got to always keep your eye on the prize on what are we actually doing to help make users' lives better?

4:39Because ultimately that is the test. Yeah, it's so true. I mean, those of us who are in tech, whether we're on the operating side or in venture or, you know, whatever, we're all living, breathing this stuff basically 24-7 right now. All we talk about is AI. Yeah, yeah. But like, you know, the litmus test that I like to I like to apply to things is like, well, are my parents talking about this stuff? Are my kids talking about it? You know, people that I don't know in tech. And yeah, I mean, I think per your point, not everyone's talking about it yet. Not everyone. I mean, a lot of people are using it.

5:10Not everyone realizes they're using it yet. So I think I agree. Like we're probably a few years off from where, you know, everyone in the world has an iPhone or an Android device. And it's, yeah, people talk about it, but also really to be super crisp on how is it making my life better? Like give me very concrete examples in which AI is actually helping me do a thing better, faster, more accurately, giving me superpowers. Like we got to really nail those use cases and kind of expand that set of use cases. That's how what's going to lead to widespread adoption. Yeah, I think that's right. Flip side, just thinking out loud of it a little bit, I don't know that most people think about the cloud in that way.

5:51And so the thing that I wonder is, is AI going to be the type of shift that's very top of mind for everyone in the world? Or is it going to be more in the background, like the cloud where it's had, obviously, massive, massive consequential impact, but it's kind of invisible. It's behind the scenes. You know, I think, so let's take cloud as an example. The way I see it is, initially, my very simplified view of cloud adoption over the last 15 years is it started out as, I don't know what this cloud thing is. I'll take some low risk workloads and stick it in the cloud. Like I remember most common use case was I have a disaster recovery site.

6:27I have some servers sitting, maybe I have an investment bank in New York. I have servers sitting in Hoboken. That's my disaster recovery site, you know, where you are. And let's move that to the cloud. And there's some cost savings and it's very clear. And then you're like, all right, that worked. That's great. Now, what else can I do? And then you start moving more workloads to the cloud. And then it becomes sort of a, all right, I'm removing CapEx. I'm moving to OpEx. I have elasticity in my compute capacity, et cetera. That's great. Then you realize one day that, wow, 60, 70 % of my critical business apps are in the cloud.

7:02That enables me to rethink how I do my business. Like it enables me to connect my apps together. It enables me to get insights from what used to be previously siloed. And so over time, new opportunities open up. And I think ultimately cloud did transform how businesses run and operate, but it didn't happen overnight. It's like I put in the cloud, I got some cost savings, great. And now I see what cloud can actually enable for me. And I think AI is going to be similar. It's an incredibly powerful piece of tech. We're going to start using AI in different ways. and then we're going to realize, you know, this workflow that I have that has like 15 steps, now that I've got AI enabled in every step of this workflow, I can actually rethink the workflow from the ground up and I can do my work differently.

7:46So I think we're going to go on this journey on AI and it's not going to be a quick journey, but I think that is the journey and that's what's going to lead to transformation. Yeah, yeah, I think that's right. That makes a lot of sense to me. Your career has been fascinating. You know, you spent many years at Google. We touched on it very briefly. I think initially as a product manager, which I've always viewed the product manager role as sort of kind of like being the CEO of a feature or a product. And now you're the CEO of Grammarly, a company that's I think glass valued at nearly$13 billion.

8:20Like what was that journey like? going from a product leader to, I don't know what your first thing was that you focused on as a product leader at Google to now leading this incredibly beloved product and brand? It's been a phenomenal journey. I spent many years at Google, as you said, almost 15 years at Google. And the reason I joined Grammarly was I really felt a deep sense of affinity to the mission of the company. Our mission at Grammarly is to improve lives by improving communication. And that's always been my North Star is, you know, how do I use my time in the most impactful way possible?

8:59Am I working on something that I can really be proud of? Maybe it will be successful, maybe it will not, but like, I really want to align on if we are successful, the outcome is something that I'm really, that's good for the world, that's good for in terms of the impact we can have in a positive way. Like thinking back on, you know, early days of Chrome, when I joined Chrome, Chrome success was very far from assured. It became very successful, but back in those days, definitely not obvious at all that that was going to happen. But I was really attracted to this idea of making the web platform better.

9:33And I felt like, you know, the web is great. No, there are no gatekeepers. You don't need permission. You just innovate. That seems like a very worthwhile mission. So I'm in. I'm just going to sign up and we're going to see what we can do with it. And so Grammarly was very similar to me in that sense. It was a really amazing mission that I wanted to sign up for, join Grammarly. And it's an amazing company. It's just people love the product. I've never encountered this much product love. And I keep telling folks who've been at Grammarly a long time that this is not normal. Like, you know, people genuinely just have this incredible affinity for the product.

10:11So we built something really special. I can't take credit for it. This has happened way before I joined. The team has built something really special. And now AI has created a huge tailwind for us. And now we take our mission, we want to improve lives by improving communication. We can execute on our mission in a much more substantial way using AI. So this is great. And so my journey to CEO was really rooted in that. I joined Grammarly to make this mission come to life. And if I can do it as a head of product, if I can do it as a CEO, I'm in and I just want to get this mission, you know, moving forward.

10:50That's awesome. When I think about Grammarly, is it fair to say that Grammarly has kind of always been an AI product, even before AI was cool, right? I mean, like, you've always been there, right at the cursor, right where the user is communicating them and leveraging some level of intelligence to help them communicate better. is it always been AI? It's funny, actually. Some of us maybe curmudgeonly get a little annoyed that AI has become equated to Gen AI. AI is a lot more than that, but I think that ship is sailed, so I'm not going to fight that. Don't fight it. Don't fight it. So let's move on.

11:29But if you think about the journey of Grammarly, it started out, so it's always oriented to the mission. We want to help people improve lives by improving communication. We want to help people communicate better. So back in the early days, there was no gen AI. There was no deep learning either. There were no kind of scaled out neural networks. What we did have was NLP, natural language processing with rule-based systems. And so Grammarly said, all right, well, what can I do with the technology that I have available to me? And it sounds like NLP is very amenable to grammar correction because it is all really based on these grammatical rules.

12:07So that's where we started. We said, okay, we can actually start with helping people be correct in their communication using NLP. That's the technology we have. We can make progress. As deep learning and sort of scaled ML pipelines became more prevalent, we realized, hey, you know what? We could actually help people with more than just grammar checking. We could do things like helping people with conciseness or clarity or tone. And so we started annotating data sets. In fact, we brought a lot of linguists. We hired a lot of linguists to help us with our NLP rules. And then those same linguists who are language experts, world-class researchers in language evolution, the same linguists said, were kind of tasked with, let's start annotating high quality training data for ML pipelines to enable tone detection, tone correction.

12:59And so then we started moving from NLP to adding on a bunch of ML-based features, moving beyond grammar. And now with LLMs, I feel like that's the next evolution, which is LLMs enable us to do way more. One of the things that enable us to do is move beyond the revision phase. It's like, think about the life cycle of communications. you know, it's, I'm thinking about what I want to say. It's kind of a conception phase or ideation phase. Then I kind of compose it. I write it down. Then I revise it. I make it better. And then I send it to some people or a person or group of people and they comprehend it.

13:39So there's this entire life cycle. We've always focused on the revision phase of this life cycle, but Gen AI enables us to actually extend beyond that to the entire life cycle. And so that's the journey we're on. So it's always been an AI company, to answer your question, and we keep taking in new AI techniques and improving on them, but it's always in service of these use cases that we care about for our users. So with NLP and sort of rule-based NLP, it almost sounds like you're looking at every part of the sentence and you're just following the list of instructions effectively. You mentioned that the LLM is sort of the next level.

14:17Does LLM, does using an LLM almost just like kind of wipe out all of the groundwork you had to lay with NLP because it just kind of does everything? Like, is it that powerful that it can kind of do it all compared to what you previously did? LLMs are extremely powerful. And I think we haven't even seen kind of, we haven't seen the asymptote on the power yet. It's still seemingly like getting more and more powerful with every generation. But I think everything we do ends up being accretive. So for example, we create these rules. We kind of fine tune these rules based on user feedback. So users will accept or reject suggestions.

14:56And we serve 30 million users. So we get a ton of user feedback. All of that user feedback data helps us fine tune LLMs and fine tune our existing models and our new models based on user feedback. So it all adds up and it's all accretive and it just makes the quality of our experience better. To the extent you can share, do you leverage some of the bigger closed source models out there or is this all your own stuff? We do a mix of things. I think the dust hasn't settled on what the right architecture is. Is it going to be one model to rule them all? Is it going to be multiple models? My personal sense, I don't know if I have a very scientific defense of it, but my personal sense is there's going to be multiple models and they're going to be kind of fine-tuned for specific tasks and use cases.

15:46And in any case, that's kind of the architecture we have put in place. So we have multiple models. We use some open source models. Big fan of open source. Happy to talk more about it, but I think that is the future that I wish for, for AI. So we take some open source models. We fine-tune it on our data. We also use closed source models. We use GPT as well. And so we kind of are seeing a plethora of options with the kind of foundation models. And I think that's great. That's great for people like us because we can just take from that great foundation layer and then tailor it and fine tune it to our specific use cases.

16:23Got it. You mentioned that open source is the future you wish for. Say more. Tell us about some of your thoughts on open source. Yeah. You know, so back in the early days on Chrome, we also open sourced. Chromium, yeah. Chromium. and funnily enough, years later, like after I left Chrome and left Google, Microsoft Edge is now based on Chromium. Right. And I think that's a wonderful testament to open source. Like you kind of keep contributing to this product. You keep hardening it. You keep making it resilient and kind of by just exposing it to daylight and ultimately you get a very, very strong product.

17:04And now you have amazing browser engineers from Microsoft and from Google, making the web safer for everyone. I think that's just a great outcome. So in general, I feel like open source is our path forward. AI is extremely powerful technology. We don't quite know the parameters of where the different vectors of risk and harm are. We're kind of figuring this out as we go. And I think we'll never fully know. I think we'll always be kind of like learning as we deploy this in the field and get feedback. and we're like, oh, this is a new thing that we never thought of. Someone kind of created a new attack surface that we never even imagined and like we got to deal with it.

17:45The best way to navigate through all of that is by just making these models open so everyone can test it, probe it, push for it. And there's some people who push back on this, but ultimately in my experience and my firm belief, really open source is the way to make these models safe and useful in the longterm. So I'm actually seeing a ton of progress on it. Like Lama 3 was an exciting development for people like me who really care about open source. And so we open source our own models. We built something, Seismograph, which is like a sensitive text detector. So it's like, you know, a model is emitting some output.

18:27What if it's sensitive output? And so we actually have done a lot of work to detect that. and then you as an app provider can say, well, it looks like the model has generated some sensitive output, but let me take some evasive action. Like maybe I won't show it to the user or maybe I'll do something else. And so we built that technology, we've open sourced it. So we are contributing as well to the community. That sounds like it could be really, really useful for responsible AI and other models and companies, as you mentioned, products leveraging it just for AI safety. That's really smart. And I'm sure the community appreciates you putting that back out into the ecosystem.

19:08Since we're on the topic of browsers, maybe this is getting a little bit away from Grammarly, but there's plenty for us to come back to there. You know, I've heard a lot of people in the topic of AI talk about how the browser and sort of the data within a browser is going to become such a pivotal playing field when it comes to AI, right? If you just think about how much information flows through your browser on a daily basis, are we going to see the main battleground for consumer attention shift away from devices like iPhone and Android devices more into the browser where there's just such data density for models to train off of?

19:47Maybe, but I don't feel like that is going to be the main battleground. I think the game is going to be much richer and fine-grained telemetry at the user level. But if I just look at my own use of my MacBook, I'm sitting here, I use a bunch of tools at work. Some stuff is in the browser, some isn't. I'm using a Zoom app. It's a native application. I use Slack. It's a native application. I use a couple of other native applications. I also use browser apps. I use different devices. And so I think it's less about what is the kind of the window and the container in which I'm collecting data, but it's more about how can I make sure that I'm getting all the relevant data?

20:32How can I make sure that I'm actually getting user permission and I have the user trust to collect this data? Because clearly there's value back. and then how can I actually generate insights to provide that value to users? If I'm collecting data from a bunch of different applications, how do I then synthesize that to provide value back to users? I think that's going to be really the key thing. One of the key pillars of Grammarly to connect the browser Grammarly thing is I do think user trust is one of the key pivotal kind of attributes that are going to kind of make or break success in this area.

21:13It's something we've been focused on for a decade. It's also something that's kind of hard to retrofit after the, you can't like launch a thing and then come back later and say, all right, now let's figure out trust. You really got to have that kind of baked in from the ground up. So that's something we mix and, you know, and it takes investment, privacy, security. There's a whole bunch of things that go into it. It's not an abstract thing. It's actually very concrete and specific. And so we've made a ton of investments in user trust. I think, A, because we care about our users and want to earn their trust, but also because I think in the age of AI, it's more important now than before to really enable that trusted partnership because there's a lot of data moving around in this world of AI.

21:55It's so true. You know, a lot of the success over the past year in generative AI has actually been on the consumer side, not yet on the enterprise side. I think for the reason you're saying, I think in order for this technology to be deployed in the enterprise, which I think is where everyone ultimately believes is the most opportunity. Yeah, you got to have that trust. Right. And so you have to have like many, many layers of security and privacy that I think, per your point, for a lot of teams, it's going to take some time to do it correctly. So it's such a good point. You know, your point about from going from the browser to Grammarly made me realize, you know, by amassing such a scaled user base of Grammarly, you're in like this really interesting position in the user's workflow to now do some really interesting things with AI and LLMs, right?

22:46We talked about the core product and the main thing that Grammarly does earlier in the conversation. But like, what can that what can that access layer now be a wedge for through AI? Yeah. So it's super interesting. Right. I mean, you think about people are expecting a ton of efficiency gains through AI. You look at, you know, the suppliers of AI companies like NVIDIA. Incredible. Like there's just, there's a presumption built in that the demand is very strong and the demand is going to be predicated on tons of efficiency gains. And the question is, where do those efficiency gains come from? And it's very clear on the supply side that we're going to produce a ton of compute.

23:26But I feel like it's pretty nebulous on the demand side where the efficiency gains actually will come from. I think coding is one clear use case. I think it's because it's quite easy to measure and it's kind of, you know, engineering is a very kind of well-defined set of telemetry and metrics and engineers, early adopters, high value demographic. But I think that's going to happen everywhere. And so when I think about the world of AI and kind of how it's getting deployed, here's kind of some observations I have. And actually it's backed up by data, but forget the data for a second. Just imagine our own experience, right?

24:03So I'm a knowledge worker, professional. I spend more than half my work week just communicating with other people. And in fact, engaging in written communication with other people across a wide variety of tools. I'm writing in so many different text boxes. I'm writing in Slack, Gmail, in Word, in Salesforce. We use a tool for candidate feedback when we're hiring people. All these different places I'm writing in different contexts. and the data suggests that we are spending more time, like that time we spend writing is going up and we are increasingly fragmented across an ever wider array of tools because people want best of breed tools to run their business.

24:47Totally. So you've got that one reality, right? And I think we all live it. Then you've got the other reality of I'm consuming a lot of information. I'm reading all of these documents and I'm reading all of these messages and all of the candid feedback and all this stuff. And I can barely keep track of it. Like it's a lot. So you put those two things together and then you say, all right, so now let's imagine AI gets deployed in every single text box that I'm writing in. You've reduced the cost of content creation to zero or near zero. Simple law of supply and demand. There's going to be a lot more content.

25:24Like you're going to get 10X more content. That's a horrible outcome. Like who's going to read this stuff? Not me. I'm like, can barely keep up as it is. And so I feel like that is not the path. I feel like that is not the path to actually get the efficiency gains that everyone is hoping for. So the reason I kind of say all of that is this is the path at Grammarly. We are this layer across all of your fragmented business applications. And that position gives us a couple of ways to help you as an end user. So one is we can give you a consistent user experience. So you're not navigating through, okay, what is Google's AI thing again?

26:06And what is Microsoft's AI thing again? And what's this other company's AI thing? It's like, it's a consistent thing, no matter what tool you're using. And then because so much of our work is fragmented across all these different apps, we can start connecting the dots and bringing context along across all these different apps. So we've actually done some interesting things in this regard. One of my favorite examples of that at Grammarly is this feature we launched called Knowledge Share, where if you're reading a document and you come across some project code name or some acronym and you don't know what it is, Grammarly will highlight that term for you and kind of you hover over it and it will share information.