#274 Luke Behnke: Why Grammarly Is Going All In on AI Agents

0:00We've achieved the success that we have. 40 million people use Grammarly every day. We have been continuing to grow. And I think it's largely because what we've focused on most is how a human and an AI work together and how the AI can really support you. And we just believe AI detection is not going to be enough. Clearly, the answer is not no AI allowed. That is just silly. Institutions that are doing that are not setting their students up for success. What is better is for institutions to be able to say, this is what you can do, and then this is what you can't do for this assignment. Can you use an open-ended chatbot or not?

0:38And if so, what are the parameters that you can and can't do? And then we can produce a report that says exactly what the student did, that both sides can verify. Really, we think the future of how human provenance in the AI era will work. Building multi-agent software is hard. Agent-to-agent and agent-to-tool communication is still the Wild West. How do you achieve accuracy and consistency in non-deterministic agentic apps? That's where agency comes in. A-G-N-T-C-Y. The agency is an open source collective building the Internet of Agents. And what's the Internet of Agents? It's a collaboration layer where AI agents can communicate, discover each other, and work across frameworks.

1:33For developers, this means standardized agent discovery tools, seamless protocols for interagent communication, and modular components to compose and scale multi-agent workflows. Build with other engineers who care about high-quality multi-agent software. Visit agency.org and add your support. That's A-G-N-T-C-Y dot O-R-G. Hi, this week I talked to Luke Benke, Vice President of Enterprise Product at Grammarly. I first heard of Grammarly at an AI conference six or seven years ago, and I remember thinking that that product will never fly because Microsoft Word has all of the same features, or so I thought.

2:30But since then, I've seen the company grow, much to my surprise. And finally, talking to Luke, I realized that it does much more than what I initially envisioned. I've since gotten a Grammarly account, and I'm a writer, and I'm frankly surprised at how many times Grammarly catches things in my writing that need improvement. Since our conversation, Grammarly has announced its intent to acquire Superhuman, the AI native email app, and that takes it a step toward becoming a full-fledged AI productivity suite. Combined with their recent acquisition of Coda, they're knitting together an AI-powered workflow across documents, meetings, email.

3:24So stick around to hear Luke's vision for making Grammarly a collaborative workspace supercharged by AI. I hope you enjoy the conversation as much as I did. So yeah, I'm Luke Benke. Thanks for having me on today. I lead our enterprise product team at Grammarly. We've been moving the Grammarly product that people know and love now for 15 years into the enterprise over the last few years. We've had a lot of success, you know, selling our advanced AI capabilities for customizing your style rules and brand tones and the way that you communicate with customers, helping English as additional language sort of customers that are working directly, you know, in maybe a customer support use case, helping them all communicate more effectively.

4:15And we've had a lot of success moving Grammarly into the enterprise. And I've been lucky enough for the last 14 months or so here at Grammarly to lead that effort from the product side. And now we're in the midst of another big set of changes, which we're going to talk about today that has been really fun to be at the forefront of. Prior to that, I have led product at a number of different companies, particularly a long stretch at Zendesk in the customer support space. And we were, you know, using previous gen AI at that time, previous generation AI kind of machine learning NLP agents that, you know, maybe if we were lucky back then, they were solving, you know, 10 % of customer support issues.

4:58And it's just amazing what's happening in that space. And Grammarly is a part of that, given that we do sell to a lot of customer support teams to help them better communicate with customers. So it's been kind of fun to see how all of this tech has evolved over the last few years. Yeah. Can you sort of back up and give us a bit of the evolution? I first became aware of Grammarly at an AI conference, must have been in 2017 or 18. and I was surprised. I hadn't heard of the company. I know that you've been around for a bit already at that point. At that point, my understanding, it was really targeting students or individual users.

5:46And I remember, and I'm sure you've heard this, I said this to you when we spoke earlier, that my initial reaction was, you know, doesn't autocomplete and you know all those things that's built into microsoft word already do that uh and here we are uh you know 10 years or nearly 10 years later and you guys are a massive company and um i i'd love to know how how you got started what the initial vision was as you say you were using ai from the beginning uh and how the product is is evolved and then we can talk about how it moved into enterprise and what enterprises are using it for and then of course now uh your support of uh agents uh but but can you sort of give us a little history of grammarly definitely yes we were founded in 2009 and our goal really from day one was helping people build confidence when communicating when particularly when writing you know most people don't actually think of themselves as great writers we have a lot of data on this they don't feel confident you know that they're saying the right things they don't want to look silly in a a critical email or, you know, in a, in a, before they hand in homework at school or, you know, before they have a conversation, if they're having a conversation with a customer or something like it's just, it's a real challenge and it remains even a challenge in the age of LLMs, which we can, which we can talk about, but obviously 2009, long before generative AI.

7:32And so we were in many ways, probably a lot of people's first touch with like an AI. Now we call it kind of an agent. I mean, it does it, you install it in your browser or on your desktop. It shows up in every text box where you're working, where you give us access. And it's just helping. It's just providing a consistent experience. You are right that there are a lot of spell checkers built into OSs or built into different walled gardens. Microsoft has their own system. Google has their own system, etc. But Grammarly kind of creates this very familiar, easy to use, really beloved experience that isn't too pushy but comes in at the right times helps you correct your words and now of course with gen ai it helps you generate words helps you make improvements like you know make it shorter make it sound more like me make it friendlier all these sorts of improvements that you can do and it just shows up while you're working you don't have to go to a new destination to write it's contextually aware of what you're doing.

8:36And we've, we really feel like we've kind of, we've achieved the success that we have. 40 million people use Grammarly every day. We have been continuing to grow despite, you know, all the questions you were asking, Craig, is there, is there really a need for this in, in this crowded space? We've been continuing to grow. And I think it's largely because what we focused on most is how a human and an AI work together and how the AI can really support you. You don't have to learn anything to use Grammarly. You don't have to learn how to be a prompt engineer. You don't have to install complex sort of like, you know, customizations or anything.

9:25You just sort of start using it. It starts to learn what kinds of mistakes you often make and can help you correct those. It can help learn your tone and the way you speak. And it's just this very personal, you know, assistant that helps you write. And that's really what we're known for today. We have, I can talk about our move into the enterprise. you know mostly we've been for those 15 years since 2009 focused on serving individuals and you are right that students was a big initial use case helping students write you know papers and work on their homework currently our business is actually increasingly and in fact now more more professionals using our product than students, but both continue to be really, really important users of the product and personas on the product team that we build for.

10:29And then increasingly since 2019 or so is when we launched our first Grammarly for Business this Grammarly Teams offering. And now, like I said, we serve a pretty good percentage of the Fortune 500 and have had a chance to really, you know, actually launch our product offering at scale into enterprises. And that's been a fun journey, which I'm happy to talk more about, but I'll pause there. Yeah. Yeah, I'd like to know, on moving into the enterprise, do you sell seats or do you do, Is this an enterprise-wide application? I mean, a subscription? How do you do that? Yeah, we start with seats. We're often purchased by the marketing department who wants to write better content and superpower their group or by the customer support team who maybe English is not their first language.

11:34and they are communicating in real time with, you know, their name of the game there is speed and they don't have time often to correct their text via a chatbot, an LLM chatbot or something. So having Grammarly show up right in line and kind of ensure that they're able to respond to cases, tickets or chats in real time is a big use case. And then also IT often binds us where they see that employees are bringing Grammarly to work. They love Grammarly. They use it. They've used it sometimes since they were a student and now they've brought it to work. And then IT comes in and says, oh, let's see, we've got to decide what this company is doing.

12:22We've got to see what they're up to and make a decision on whether we want to provide licenses of Grammarly for a subset or in increasing cases for giving all employees the chance to access Grammarly. And of course, they get advanced security controls and ability to really ensure that it's configured in the way that they want. They can even encrypt any data that we do store with their own key now. Of course, we never train on enterprise data ever. That's just becoming the norm. hopefully that's what every vendor is doing these days um and so you know they have an opportunity to really understand what we're doing and and control um the settings that they want um and so yeah we've seen success kind of across those different motions mostly a seat based model today though of course as we've had a chance to um sell more across an organization you know we do we do have like um enterprise license agreement models that become much more flexible as we do larger deployments yeah and i guess the because the the natural question is how do you compete with copilot for example which is alive on the browser uh or in microsoft word I mean, first of all, I can see the value, particularly for non-native speakers of a language, or people who are not, just cannot handle language well for whatever reason.

14:12It's wonderful to be able to to send something in in absolutely correct language and have confidence in it but again the the various uh assistants uh that are integrated into many platforms uh do something similar is it that grammarly is is following the keystrokes so you're not leaving your sentence to click on an assistant? I mean, what do you see as the reason Grammarly has been able to survive? Yeah, I think a couple of reasons. Definitely the fact that we are working where you're working in any application. It's not just going to work in one suite of tools. You know, our goal is not to just sell you our core productivity tools that we make.

15:11and AI is an add-on to that. And then once you leave that tool, it doesn't work or it isn't supported, right? We're just there to get to know you as an individual user or organizations can customize us to their needs and we show up everywhere where you're working. And we've even found that in so-called sort of like Microsoft shops that we sell to. We have data on which apps employees are using And in many cases, still 60, 70 % of the time, employees are not working within a certain walled garden, right? And so having a consistent experience everywhere you're writing is the biggest reason why people use Grammarly.

15:55They know that they're going to get that everywhere. They're going to get that support regardless of which apps they are using. And so we've really managed to kind of carve out that neutral position on any website, any application where you've allowed us to run, we'll be there to help your employees or your, you know, within an educational institution, we'll help students do work. So that's probably the biggest one. I also think, you know, there's something that's been true since day one for Grammarly, which, you know, LLMs obviously solve so many different interesting use cases, and we're so excited about the future there.

16:29But this idea of just a consistent final check on the work that you're doing and have that again consistent everywhere you're working but that last sort of confidence that I don't have any of these like red underlines that I just made a silly mistake and it goes even further than just basic spelling and punctuation it really says like here's you know this these sentences are a little awkward let's fix them or you know really gets into some advanced suggestions and kind of having that final check before you hit send, I think is still unique for us. There isn't really any provider who's doing that and doing it across every system.

17:07I mean, some of these tools have built-in spell checkers, but we think, like I said, we're a familiar experience. So the way that the AI and the human work together in Grammarly, I think, is very unique. We've managed to find this very fine line between popping up and telling you, hey, you should really fix this without being annoying. And That's kind of like this very unique, very Grammarly thing. And as we transition and look forward to what we are doing and can do with LLMs, we don't want to lose that history because it is such an important part of our product. And I think it is why people really do fall in love with Grammarly.

17:47Yeah. And we'll get to the agentic stuff in a minute. But I'm really interested right now in this feature. you came out with uh i i don't know if i think last month uh called author or authorship authorship yep yeah that that will scan a document and and give you uh and i haven't used it so i don't know if it's uh visually by color coding sentences or something uh which sentences were uh written by human which were written by ai which were ai edited and and that's particularly relevant to me because uh you know at this point i work with uh and i haven't tried grammarly i'm sorry to say i i will i've got a you should try it it's free you can try it for free and see if you like yeah and i've got it on my list uh but uh but i do use a couple of different uh models uh chat gpt4o is the one i use the most and uh you know i'm back and forth all the time as i'm writing doing research or or sort of brainstorming with the model on language uh and on you know doing outlines and on occasion writing a draft or asking for help in smoothing out a sentence or greeting a transition.

19:29It speeds up the writing process enormously. And now a lot of places are sending these very draconian messages saying, if you know no ai allowed and if you're caught using ai you're out and that sort of thing uh but it's much more complex than that because uh you know it's at this point it's it's part of your workflow it's not like you're writing and telling a model to write something and then copying it and using it directly. I mean, certainly I understand the concern about that, but I don't know what parts of my stuff is AI written because it's just such a spaghetti process. So how does authorship work?

20:31Yeah, great, great question. So you're exactly right that the, you know, as these LLM tools become more prevalent it is becoming less clear what is mine, what was written with or by, and then improved by me. And we have seen, obviously, a rise of a lot of these AI detection tools, which basically are trained on patterns that typically come out of an LLM tool, and they can detect whether it's likely that this text was generated by AI. And we believe those tools are definitely useful. We offer an AI detection tool for students to pre-check their work before they turn it in. But to your point, this is very problematic for universities, particularly, where students have been accused of cheating, even maybe perhaps been expelled from the school over allegations of cheating.

21:36And there's a lot of gray area there on, I didn't know what was allowed, what wasn't allowed. Actually, I didn't use AI and this was falsely flagged, which definitely can happen with AI detection. And this is serious business. People, you know, these students' kind of lives and reputation are at stake. And it's also true increasingly in the workforce as AI detection is being used on resumes and cover letters coming in, for instance. So it's making a first round of decisions around, you know, a hiring situation. Or if you're writing a report, a research report or something, you know, you want to be able to understand how much of this was just generated with AI versus generated by a human.

22:20And we just believe AI detection is not going to be enough. It's certainly one of many tools, but it's not going to be enough. And so it actually all started. There was a student named Marley who was accused of, she used Grammarly to write her paper. She believed her school told her it was okay to use Grammarly, which most schools do. But then when she turned it in, she was accused of using AI and she posted this TikTok about it, which went viral. And we saw it and it gave us an idea to really rethink this whole space. And the way we did is with our authorship capability, which allows a student, it can be pre-configured by their university or the student themselves can enable this feature.

23:05It's a little fingerprint icon that they can click. It starts tracking from the start of their document. Works now in Google Docs and in Microsoft Word. It starts tracking how you created that document. Did you use AI? Did you paste in from external sources? If you paste it in, it brings in some metadata. data we understand it came from Wikipedia, we can help you actually properly cite it, automatically cite it for you so that you're being clear that you pulled in something from another source. Or if you copied and pasted it from an LLM, you know, chat GPT, et cetera, we can see that we can tag that this was generated by AI.

23:44You can also cite that increasingly AI citations is becoming one, you know, in some cases, an acceptable form of using AI. But then as you change it and make it your own. We're actually watching you do that. And then it's not just inferring based on language patterns. It's really understanding what did this human type versus what did they use an AI tool for? And at the end, it produces a report that says percent of text that was generated by a human that was copied and pasted that was generated with AI or improved with AI? And then how was that text that was generated by AI actually then enhanced by a human after the fact?

24:27You can even watch a playback of how the student or the individual created that document. And then you can turn it in almost like a certificate alongside of the piece of work that says this was really generated by me. We just came back from a large educational conference since ASU plus GSV a couple of weeks ago. This was sort of, we launched this generally available. We're really like leading the way here. And it was really the bell of the ball there. Institutions really believe this is the future. And like I said, I actually think there's a lot of business use cases for this as well. And so we're really excited to kind of be leading the way on that.

25:12And I think it's just one of many examples where I think our thoughtful, approach to how humans and AI work together in this future. Clearly the answer is not no AI allowed. That is just silly. Institutions that are doing that are not setting their students up for success in the whole world. And what is better is for institutions to be able to say, this is what you can do. And then this is what you can't do for this assignment. Can you use an open-ended chatbot or not? And if so, what are the parameters that you can and can't do? And then we can produce a report that says exactly what the student did, that the student can verify.

25:57Both sides can verify. And it's just really, we think, the future of how human provenance in the AI era will work. Yeah, it's funny. From my point of view, I think schools should teach how to use LLMs or other AI systems because they're not going away and they're only going to become more integrated. And I also don't, I mean, I understand in education because you want students to be doing the work. But in other settings, in an enterprise setting, I mean, as long as a document is factual and not plagiarized, I don't understand why it would matter if it's written by AI or not, unless, I mean, there's been a lot of research about AI-generated content filling up the internet and then getting into the training data and there being some, you know, some collapse to the mean in generation because the training data is so similar.

27:23But in the enterprise, do you find enterprises concerned about this as much as they might be in education? Again, as long as the output is factually correct and not plagiarized. Yeah, I think the incentives are better aligned in the enterprise where, you know, whatever tool gets the job done, productivity is the goal, right? And we're going to talk about agents and all of that. But there is a really critical thing that you said there, which actually I loved this. We had a customer advisory board with a number of our biggest customers. And we had someone from a large higher ed institution who said that they actually get a lot of businesses.

28:12They do a lot of their feeder school for many of the big employers in the area. And they have increasingly had those employers say the most important thing that they want when they're hiring someone out of this institute, out of this university is who are your best users of AI? And he was laughing and he said, we call those cheaters. Like, you know, this is who you want to hire. Our job is to produce people who are AI fluent, theoretically AI fluent so they can get these jobs. And yet we're actually labeling a lot of these people as cheaters. And I think this is a huge, huge problem and one where I think Grammarly has a really critical role to play in bridging that gap from students to professionals.

28:54But then once you're in the professional landscape, you are increasingly being performance reviewed on your use of AI. Toby from Shopify and a number of other companies have followed suit on this is going to be a critical part of how you're evaluated as an employee. You know, this is you're going to be managing agents. agents. You're going to be using this technology day in and day out, and it's going to be expected of you that you are fluent. And we are not doing a good enough job training people. Even within organizations, I think we have a bigger burden to think about this like a learning and development situation and to put the same amount of care that we would into onboarding new employees or teaching them how to use certain systems or helping them understand our security and privacy policies, you know, helping them become AI fluent, especially in this transition period that we're in is just so important.

29:51And I think Grammarly has a really important role to play, again, with that thoughtful AI human interaction, how we can bring some of this AI to the masses, instead of it being this hard to use tool that you, you know, we're in this like early stage where we've all been given all of of these raw materials really with AI. And some people are emerging as super users. Some people are afraid and self-admit that they're just not touching this stuff and they don't really know what it means for them. And I think Grammarly can do this great just because of this thoughtful way that we've always approached bringing AI to a wide group of people.

30:30I think we can be an interesting bridge there in this kind of exciting moment in AI. Building multi-agent software is hard. Agent-to-agent and agent-to-tool communication is still the Wild West. How do you achieve accuracy and consistency in non-deterministic agentic apps? That's where agency comes in. A-G-N-T-C-Y. The agency is an open source collective building the Internet of Agents. And what's the Internet of Agents? It's a collaboration layer where AI agents can communicate, discover each other, and work across frameworks. For developers, this means standardized agent discovery tools, seamless protocols for interagent communication, and modular components to compose and scale multi-agent workflows.

31:35Build with other engineers who care about high-quality multi-agent software. Visit agency.org and add your support. that's a-g-m-t-c-y dot o-r-g tell me how uh agents are being integrated in the grammarly or agent management or agent creation or what are you guys doing there yeah so um i think it you know really starts with this understanding that while grammarly has been known as this kind of ubiquitous ai proofreader as i was talking about something that we've built behind the scenes that just works when you download Grammarly, either as a browser extension or on your Mac or Windows PC. Like we've built this incredible, what we call the AI superhighway.

32:28You know, Grammarly, our proofreader agent, we think of as like the OG, the original agent, AI agent for many people. And we had to do a lot to make that agent work in every single situation. You know, the way that Google Docs renders text is very different from the way that Microsoft renders text. And for Grammarly to work seamlessly across those two surfaces to make it work in Excel and Google Sheets, to make it work in almost every text box on the web, every desktop application, we put a lot of effort into this AI delivery mechanism, essentially, that brings these really thoughtful, helpful AI helpers right to your fingertips while you're working.

33:10It can understand what's happening on your screen. It can understand you personally. It can understand the kinds of things you're trying to do. And it brings it right there. And so we call that kind of our superhighway, this ability to have someone configure something in Grammarly and then have it go right to their employees' fingertips as they're working. And we kind of laugh, but we've been driving one car down that highway so far. This is our little spell checker proofreader agent who comes down and tells you, hey, you know, you should fix the sentence. It doesn't make sense. Or this word's spelled wrong.

33:43And we actually believe that there's an opportunity for us to have a lot more agents drive on that superhighway. Those agents will be configurable by the end user or by the organization. I can give you some examples of what we're thinking about for these agents. But to really allow the end user to bring in more of these agents, they're powered by the knowledge they've given it access to. of course knowledge becomes such an important part just as it does with any human doing work you have to understand you know what are what's the knowledge that this agent has access to what are the skills that this agent can do and then have I been clear about the instructions for this agent and so when we really put together kind of knowledge skills instructions and then bring those agents directly into where employees are working I think there's some really interesting opportunities You know, one, for instance, that we've been thinking of is, let's say you have your CRM system of choice.

34:47Something that we do often is, you know, you want to understand, I'm about to go meet with a customer. What's the latest? What's happening here? And we have this really cool tech that we built at Grammarly that we call Knowledge Share, which actually highlights an acronym on your screen and defines that acronym for your organization. Historically, you had to set that up. You had to give us a list of all your acronyms. You had to give us definitions of all those acronyms and keep those up to date. But with knowledge and ingestions and integrations and with LLMs, we can actually be much smarter about saying, when you hover over a customer name, you're hovering over Acme Co.

35:28that you're just reading in a document. You're like, oh, are they a customer? You can hover over it. And let's say you use Salesforce. We built you a little Salesforce agent. It can pop up automatically right where you're working and say Salesforce or sorry, AcmeCo is a customer. Here's all the information. There's a current open deal, you know, an open opportunity. Here's the current spend. Here's the current contacts. Here's the latest communication. You can just bring that context directly, you know, to your employees. Of course, it could go much further than that in terms of the ability to actually orchestrate all of the things we do every day around managing our meetings and agendas, managing our calendars and setting up time with other people, taking notes, writing documents, sharing documents.