Emailing like a superhuman

0:03Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, productive, and accessible to all. If you like this show, you will love The Change Log. It's news on Mondays, deep technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for The Change Log wherever you get your podcasts. Thanks to our partners at Fly.io. Launch your AI apps in five minutes or less. Learn how at Fly.io.

0:44Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? Doing very well. Looking forward to talking some fun stuff on this beautiful spring day. Yes, yes. Well, I've always hoped that AI could make me a superhuman. So really, really excited to hear about maybe something in that realm today from Loic Oussier, who is head of engineering at Superhuman. How are you doing, Loic? I'm doing great. I'm super excited to chat with you guys.

1:31And I was with a pretty humbling set of guests in the past. So I'm super happy to have this opportunity and discuss in length. Yeah, that's awesome. Well, I know this is kind of interesting because I know Superhuman, I think, was one of maybe this sort of first, really integrated AI first kind of engineering tools that I remember seeing. And of course, the AI space has advanced a lot in that time. Maybe could you give us a little bit of a kind of state of AI in email or productivity more broadly, if you want to think about that. But really, I mean, obviously, we're going to talk a lot about email and messaging.

2:14So could you give us a little bit of a sense of what that landscape looks like right now and kind of how superhuman fits into that? Yeah, totally. And it's like it's incredible, like the time that we're living right now. of course like everyone has been shocked like when we had like the first version of those llms like doing some crazy crazy stuff and like analyzing text summarizing and doing like all sort of magic and and of course email i mean is all text-based for the most part and so it was like a really nice testbed like to to try out like all the the cool stuff that you can do. And interestingly, that's also helped this category, the email client, to thrive for quite some time.

3:05Superhuman was almost the only one supercharging Gmail and Outlook. We were the only one on the space making people faster going through their emails and all of that. And with the rise of LLM and agents and everything, now there's a bunch of people now like, oh, damn, this is a great, great, great environment to play around and to make things better. And right now, this is what we see. We see a bunch of, I would say, other tools trying to do stuff with LLM and to create a better experience for email. And this is indeed an interesting time for us because this is the proof that that category needs to exist, was existing before, but we are the only ones there.

3:51And now like more and more people are getting there showing that there's deep interest in it. And it's challenging. And it's like super interesting. And we'll probably talk about it, but it will also like help us understand what makes like a good product. And is like just the LLM and AI sufficient? Or like do you need some sort of like a secret sauce on top of it? And happy to discuss about it. As you, I'm kind of curious following up on what Daniel was saying with, you know, with you guys being so early into the space and obviously we're, you know, not only LLMs, but just AI in general has been going at light speed, you know, increasing steadily over that time period.

4:34how is that how is the the space change for you guys from being kind of the early only player you know into into the space where um there's others you know it's becoming uh you know somewhat congested across not just the space you're in but just like everything how has that changed the world for you guys in terms of of staying differentiated and all that yeah so so it's very interesting because like there's multi-dimensions uh that we can talk about like the the the first one is this is like the the raise of those ai features and capabilities are bringing like a new set of features that you can implement that you couldn't do in the past in the past ai was mostly classification like adding labels and and stuff like this and that was kind of like the limit of what ai could do really for everything that is text based so typical classifiers typical models like this.

5:30And more and more now you can do some intelligent stuff. So we moved from a place where we were making things faster for our users compared to Outlook, compared to Gmail. But now there's more that we can do so we can make things smarter, which is probably a paradigm shift in terms of the value that we're creating for our users. The other dimension is like, this is raising the expectations for, I would say, the different users. Like for a long time, they were like, damn, this is so fast. And I'm winning like four hours a week to go through my emails. But like everyone is used to chat GPT. Everyone is used to perplexity.

6:14Everyone is like crafting images or like even movies with Sora and all of that. So like the level of awareness and the level of understanding of what the technology can do is raised dramatically. So for our users, the level of expectation is like, hey, superhuman, I expect this now. I expect this now. The other dimension is like from an engineering standpoint and like a building standpoint, our tool set is totally different. Like the tools have changed. And engineers that were working like in some ways three years ago, even two years ago, even six months ago, like right now the tool set and like your flow and all your setup to work has dramatically changed.

6:58And maybe the last dimension that I think is really tricky to apprehend is the perceived quality. So superhuman was seen and built on the one single dimension that was like, it's highly qualitative. We were in charge of the quality because we master everything. So you can have a zero bug policy. You can take the time to deliver the value, but it needs to be perfect. And now with LLMs, a bunch of the perceived quality depends on your prompt. So you have users that are prompting with different skills or different level of skills, and the outcome of that prompt may be perceived as low quality. But that's something that is really hard to control.

7:46And it's creating something that is sort of mind-blowing from an engineering standpoint. I mean, we've all been working in tech and the craft, the bugs and everything. There are some processes to limit the number of bugs. But now quality is not only bugs. It's also like this perceived quality based on the user. And that's an interesting thing to tackle. And I'm curious, as you kind of mentioned the fact that with some of the prompts and having different users, you know, skill level and stuff like that, Could you talk a little bit about kind of how you tackle this is one of those interesting things from my standpoint to hear about where there's all these little gotchas in this world that a typical person isn't going to ever have thought about going ahead of time.

8:37And so as one of those things where prompting itself is fairly diverse in terms of the skill set, can you talk a little bit about how do you deal with that when you're trying to put together a product and focusing on the quality issues and stuff like that? Because I'll be honest with you, that would not have occurred to me to have to think about addressing that kind of issue. Can you talk a little bit about that? I will tell you about one specific feature that we released in Q1. so we have those auto labels so automatic labels that will basically flag your emails and based on the label you can decide to skip your inbox altogether typical stuff like random pitches from a company that want to get in touch with you to sell their product I receive probably 30 of them every day do I want to take a look at those 30 and answer all of that?

9:33Probably not Probably not. So I'd love them to basically be skipped altogether. So for those, we build classifiers that do not rely on user prompts so that we control the quality, precision, recall, like the typical stuff. But we also allow our users to create and craft their own labels. Let's say you want to have like, Like, oh, I want all my podcast invitation to have the same label. But like you cannot just have like a deterministic rule to say because I don't know all the podcast people and everything. So you cannot just do like the filter like Gmail would do where you say like if, then, then, this.

10:17So you have to prompt it. And you have to basically allow the user to craft a prompt that will surface all of those. But then that prompt is tricky because if you have someone that is just putting just a one-liner, you start having some issues because the precision and recall based on the one-line prompt is not great. And we know, as I guess your audience have been working with chat GPT or prompts in general, the more structured and extensive they are, the better the result. And there's a bunch of hallucinations that can happen if you are like just one liner because lack of context and lack of all of that.

11:00So of course you do like some like system prompt to basically surround this user prompt to try to avoid like too much issues. But there's also like a part of education that you need to have. And we're working on this now, which is like, huh, your prompt seems interesting, but like probably you want to structure it that way. So there's some stuff like this that we will be working on. Also sharing prompts, like libraries of prompts is something that we're thinking about more and more because not everyone is able to craft a nice prompt and maybe someone in your team will have done like a prompt that you would really use happily if you get access to it.

11:42So it's sort of like, I mean, it's very product centric, so it's not AI centric and you need to work around this new problem. And I wish we'd have a civil bullet and like the answer to that problem. But I think we are like learning as we walk. But it's super interesting. I'm wondering, like, I'm always intrigued by, I read a book by Richard Hamming and one of the things that he talks about is how if you rethink a process that was very human and manual before, often the way that you would make that an augmented or machine-driven process is very different from what the original human process would look like.

12:22And I think in the email client, we all sort of expect a certain process, a look and feel to the email client that's developed over time. What have you found in terms of presenting an email client to a user that is drastically different? What sort of needs to be preserved? What's kind of up for grabs in that experience? What should stretch the user? What needs to be preserved? How do you think about that? No, that's really interesting. That's a really interesting point because we are at that moment where the user interaction with the computer, with the system is like dramatically changing. Like people don't expect to click in different windows anymore.

13:08Like the expectation is different. Like chat EPT or the other like clones, like from like different providers, you basically have a chat box and you ask everything there. Like even if you're working on a document, you ask on the chat bot and like, oh, modify my document and rewrite my exact summary. Oh, make my tone a bit more like X and Y and Z. You don't expect to have like a button like Word would have like Microsoft Word back in the days. So and we are only at the beginning of this shift. So I think that and it's kind of like coming back to like competition and all of that, the barrier to entry to like pretty much any SaaS application.

13:48or given a consumer application is very low now because it's very easy to at least to build a POC, at least to build a POC. I wouldn't go further than that. And what will make the difference is the product test and how you want to understand your users and how you understand their user interaction. And this is where I feel pretty proud to work at Superhuman because our CEO is a freak in terms of user interaction and vision. And he's already thinking about that and how the future of interaction will be. And it will change. It will be different. So what will stay? What will be slightly different?

14:33I'm pretty sure that the conversational aspect would be a strong paradigm. Right now, you don't talk, whether it is through your keyboard or through a mic, you don't really talk to your system. You don't talk to the application. Maybe you start talking with ChatGPT because they have this nice voice interaction. Maybe you use WhisperFlow or like this type of tools to basically write your email or like write in Slack and your messages. But you're not exactly commanding the device to do things as you talk just yet. But more and more people are doing so. I probably talk to my computer now more than I type, interestingly.

15:18So there's a change. And everything that we've done in the past was mostly click and click and click. Superhuman started with the common K and keyboard-centric access to things for people that wanted really productivity because switching with a mouse is pretty slow. And now more and more people are starting to engage with the voice. So all of that will change the way you think, the way you surface the data, the way you interact with the data, the way you bring the focus. So this is an interesting, I would say, area. One thing that I do believe will stay, though, to your point, Daniel, and I would talk about email especially, the concept of inbox, like the concept of having like some sort of like a timeline of things that you need to go through and get rid of the stuff that are top of mind some sort of like a task list to some extent will stay now how it will be surfaced how you will go through it will dramatically change over time and we're already like seeing this

16:38Okay, friends, build the future of multi-agent software with Agency, A-G-N-T-C-Y. The Agency is an open source collective building the internet of agents. It is a collaboration layer where AI agents can discover, connect, and work across frameworks. For developers, this means standardized agent discovery tools, seamless protocols for interagent communication, and modular components to compose and scale multi-agent workflows. Join Crew AI, LangChain, Lambda Index, BrowserBase, Cisco, and dozens more. The agency is dropping code, specs, and services. No strings attached. You can now build with other engineers who care about high-quality multi-agent software.

17:28Visit agency.org and add your support. That's A-G-N-T-C-Y dot org.

17:40So as we were kind of going into the break, we were talking about kind of, you know, the notion of rethinks of that. And I'm kind of curious, as you're thinking about not only like the rethinks, but you're also having to respond to the evolution of the technology itself that's available for your teams to implement stuff. And one of the things that we've seen over time is kind of, you know, it's not this smooth increase. You may have evolutionary increase in the model capabilities for a bit, but you also have these jumps that'll occur along the way. And with your product teams, as you're looking at what the future of your products are going to be, and you hit these moments where it kind of goes from predictable improvement in the models and you make these jumps, how does that affect the product development cycle that you have internally when you're saying are those moments you know as we were talking about rethinks do you have moments where you kind of go maybe it's time for kind of a deliberate rethink because something just happened in terms of the technology capability that we weren't expecting last week and we're going to do that how do you guys handle this kind of an industry being in it for that no it's it's interesting okay and so daniel you were mentioning a book but one book that comes to mind as you're asking this question is Zone to Win by Joe Fremour.

19:10And it talks about like continuous innovation and like disruptive innovation. And this is probably what we're talking about. Like we continuously innovate and we continuously add more features and new stuff into the product. And sometimes you have this opportunity to provide something that is disruptive, whether it is like the underlying technology that is disruptive or because you have like some sort of a wow moment and you have someone, I would say, with a vision that is like, this is the direction to take and we need to either pivot or we need to do something drastically different. What we've seen, especially with AI, is the rate of those disruptive innovation is mind-blowing.

19:51I would say before AI, to some extent, the technical innovation where maybe once a year, once every two years, you have something that is brand new and like, holy shit, we need to use this. and pardon my French. But what is interesting with LLMs, every two weeks or three weeks, if you're not on Twitter, you're not on Hacker News, you can miss the new big stuff. Like LLM, multimodals, reasoning, MCPs, that came in six months. And all of that is coming with a new set of capabilities that you can decide to implement in your product. So to come back on your early question, what is the impact on the product development how do you handle this uh one you better be agile meaning like the true agile um so you better be able to uh stop what you do and say wow focus uh we need to sit down for a moment because this is coming what do we do about it and uh you need to have like um and that's why i love like small companies uh to some extent because it's very easy to have like everyone listen there's this new thing we need to do something about it uh Let's change the roadmap right now.

21:03When you're in a bigger company, it's way harder to do it because you have your yearly planning that is like coming into quarterly planning and you have all those OKRs that you need to report on and everything. So you need basically almost a six-month business plan to explain why you want to pivot and do something else, which is obviously not the case when you're a company that is of a small size. superhuman engineering and product and design is probably like i don't know i don't have the the strict number but like 40 40 60 maybe 50 but that's about it that's the the size where you can be like super agile you can stop everyone doing something because something is coming up and we need to to focus on it of course we can do better if my engineers are listening to this podcast they would say loik maybe you're like you're caricaturing a bit so probably i'm caricaturing a bit and of Of course they are, right?

21:58I mean... And of course they're right. Of course they're listening to it. And of course they're listening to it. No, so it's having this understanding that everything is changing right now. So you need to reassess your priorities like almost every two weeks. Almost every two weeks. MCP is coming. People are standardizing on it right now. What do we do with it? What do we do with it? Should we invest like crazy? Should we stop everything that we're doing? Should we, I would say, do we still believe in the vision and it's providing more value? You need to make those decisions every two weeks. So, or like almost every week.

22:34So, being close to, I would say, a close-knit team that is talking like basically on a daily basis to make sure that you're making the right decision is key. And by the way, just for listeners, you may have heard MCP in there. If we did an episode explaining what MCP is. So anyone who's not familiar with it, you should jump back a few episodes and hear that out. It'll give you some context around that. Yeah. Thanks. Thanks, Chris. And I'm sorry if I use, I would say, some jargons. Jargon's fine, but we always try to jump in and point people to it. This is perfect. This is perfect. And I think kind of looking forward, one of the things that I'm really curious about is we've kind of tackled some of the bigger issues of AI and email.

23:23But I'm kind of curious, you know, if we dive down into specific functionality at Superhuman, you know, how do you see kind of the most, maybe the most useful AI email functions that you're currently either kind of releasing or kind of thinking about forward? Or, you know, how do you, when you get kind of granular on the product, how are you starting to think about that now? I would just like the feature that all our users are basically talking about because they just love it is a feature called auto draft. You receive an email as part of a thread. Someone is asking you some questions and or you send an email basically saying like, hey, can we meet next week or whatever?

24:06And after two days, you don't have an answer. You usually want to bump that into their inbox and everything. We build this feature where we create those drafts for you ready to be sent. It's not mind-blowing in terms of usage of LLMs. You provide the context. You use the tone of that person and everything. And you craft a draft that could sound like a good way to reply to it. And the results are just mind-blowing. The users find it so addictive because it's relatively accurate and they win a lot of time. It's just about winning time. Our users are mostly CEOs, CXOs on the sales side as well, some consultancy firm.

24:52They leave basically day in and day out in their emails. So every 10 seconds that you can make them win in their day is a huge win for them, given the amount of emails that they have. So this is one of those features that is super effective, even if it sounds simple. So, Loic, even with what you're just describing there, creating an auto-draft per email, maybe an LLM call, doing classifications, auto-labeling, maybe other calls. I don't know how many calls or chains of LLM calls are happening per email, but that could potentially be a lot. And if you do that for one email, that's fine. You do that for all my emails, that's more.

25:41If you do that for all emails of thousands or hundreds of thousands of people, that's a lot of Gen AI workload. How does Superhuman, as more of an AI application company, think about that in terms of optimizing infrastructure or AI, Gen AI use, consumption, hosting your own models, fine-tuning your own models, using smaller models? How do you think through some of that? No, so this is a great question. And this is a real challenge to some extent, if not a problem, sometimes. Indeed, I guess like my engineers are like very much into like the finance, like they understand like the cost of inference, the cost of the input, the cost of the output.

26:27they understand the difference between the different models so we had to put some high level principles to keep moving fast so that they know how to default and only escalate if they have some questions. I will give you some examples but if it's a new feature we don't know if it will be working or whatever so still testing we want it to be great so we take the most expensive model it's working and we have traction great, good problem to have. And now this is the moment where you start thinking about optimizing the cost. And maybe you will switch to a cheaper model, maybe more fine-tuned. Maybe you would switch to a different type of model altogether.

27:12So for example, the classification that we discussed, LLMs are okay with classifications, but you can have way cheaper for the same quality with a better type of models. and inference cost is like a fraction of it, a fraction. So long story short, this is the way we provide, I would say, the value to our end users. We try with the best working. We do optimization after the fact. Does that answer your question? But this is like more generally, I think this is like always like the right approach is like don't care about the cost right now if it's not becoming a problem because you always want to provide like the best experience and if you don't have traction, too bad.

27:56Because the risk, if you try to start small because you're afraid of the cost, you will use like a cheaper model and the feedback from the users would be meh and they won't use your feature and then you don't know if it's because the feature is, I would say, not well targeted or if it's because of the model. So targeting the best, you have like better answers and better insights. That was a really interesting answer from my standpoint. I just kind of you explicitly kind of called out, you know, as you're getting to the feature and going ahead and going with the best, the most expensive thing and then pulling it back to what what the efficiency will be.

28:38And once again, one of the things that we often call out on the show is kind of the the fact of kind of software engineering being applied and, you know, kind of the analogies on that on the AI side. So I just, I really wanted to kind of call that out because I thought that was a great insight that you made there. And it has impact. So I'm sorry to cut you off, Chris, but it has a significant impact on the way you build your application because you want to be able to switch models, to switch the heuristic associated with the output that you want to have. So you have to invest some time to have a way to do this switch relatively easily, potentially do A-B testing with different population to measure the difference of perception.

29:22Because again, not everything is black or white. There's nuances of gray now in terms of perceived quality. So you need to have more of a statistical approach in terms of understanding the impact of one model versus the others. And of course, we have internal evals and all of that to do our own testing in terms of with our golden data set. But the reality is we have a diverse set of customers and everyone is different. So we need to have a broader perspective than just relying on our own data set. Yeah, Loic, I appreciate you getting into the technical side of things a bit and talking through some of those optimizations and how you think about them.

30:06Obviously, you're leading the technical efforts with Superhuman. And I'm wondering if you have any sort of hard lessons learned from doing AI engineering over time. We have a lot of practitioners that listen in. Any kind of general principles or lessons learned that you'd want to impart? That's a good question. Maybe one thing that I've learned is to, like, as a CTO, I need to discuss with the rest of my leadership team. and we talk about the success of features and everything. And the typical way to talk about quality is typically in terms of number of bugs and everything. Now, and I was at touch on it early on, but the perceived quality depends now.

30:50We're in a world with way more subtleties with LLMs. So setting the right expectation, basically explaining that the way the feature can be built and sometimes failing because the feedback is not great might not be because it's not well implemented. But maybe there's, I would say, more to it. Maybe there's a part of the perception. Maybe there's too much latitude that is offered to the end user. Maybe there's some work on the prompt side. So that's something that hit me in the beginning where the perception of the feature was like, oh, this is terrible work. It's not working. People are complaining.