Lessons in product leadership and AI strategy from Glean, Google, Amazon, and Slack | Tamar Yehoshua (Product at Glean, ex-Google and Slack)

27:33The things that stood out to me from the meetings I had with him were a couple things. One, you know, a lot of people have written about these six pages. So you have to, he doesn't believe it. PowerPoint, you write a six -pager about, I mean, it's like studying for a final exam. It's writing these six -pagers. So you go into the meeting and there's people around the table, his executive team and him. First, he does not speak until everybody around the table speaks. So he goes around to all his leads and said, what do you think? What do you think? What do you think? I'm sitting there like, I don't care what I do.

28:06I just want your basis things. But he wants to, he wants to make sure that it's a team effort and that he's listening to what everyone in his organization thinks. And he always spoke last. He is by far the smartest person I've ever met in my life. In that I've worked a lot of smart people, but he's ability to go deep in any domain and nail the core issue and remember, like we would have quarterly meetings and from quarter to quarter, he would like remember things if he had talked about before. And then he would go into like the architecture of search and And why are you doing it this way or that way?

28:49And you're just blown away that he knows that. But the thing that for me the biggest takeaway from this meeting was his consistency, which is he had principles that you, it made it easier for you to operate in his company because you knew what he cared about because he always had these principles. Everything had to be customer driven. Everything had to be relevant for the customer. He did icons. the, it's just the thing, you had to like write what they were because people couldn't figure out what they are. So anytime you show an icon, who you get annoyed. But they were like, so when you would go in and after a couple meetings, you're like, okay, I know what he's going to ask about.

29:27I know how he's thinking and I know what his principles are. And I think those that consistency enables you to operate a large organization more effectively. And then there's one other thing that I really remember was very one of a few really small meetings I was in with him and we were presenting working on a new product and I was like the our competitors have like ten times as many people as we do on this and he looks at me and he said that is your advantage. And then he goes into his talk about how it's a hill and it takes seven years to build a product. You can't look at it in the near term.

30:04It's going to you have to be in it for the long term. And you can be sure I never went in and said I need a lot more resources. So that was based us. Oh, see. Stort, again, I went to Slack because I wanted to work with Stort Butterfield. I think he is the best product thinker in the Valley or he's not working in product right now. He's taking time off. but he's got this combination of long -term thinking in the details. So in 2014 he wrote a master plan for Slack, which was build a product that people love, build a network that's Slack Connect, build a platform that makes all of your other SaaS products more valuable, the Slack platform, and then do some magic AI stuff.

30:51The magic AI stuff took a lot longer. But it was part of his plan early on as magic AI stuff. literally was there was a there was a grid with four boxes in 2014 and it never changed. That was his master panel and what we worked on each year changed. But somebody recently asked me, well did he always like you know you guys did slack connect much like him like yeah because always part of plan. It was always part of his vision until he saw forward in the vision but he also was very much into the details. And I think the thing that I learned from him the most was the power prototyping. And that even though he was such a great product thinker, he would always say, I can't tell you if this is going to work.

31:32I have to feel it. I have to try it. And a mockup doesn't tell you what it's going to feel like. And he would push people to do prototypes, not incremental of just to get a feature out, but really to think very soon after I started we launched a hard -to -designly, and he led a redesign of the new information architecture for some reason. I worked with Ethan at R &B. Oh, yeah. He was head of design for the search experience in the search team. Yeah, Ethan was awesome. And he came in and his first task was to work with Stuart on this redesign. And Stuart came in and said, I want you to take everything in the interface and put him behind one button.

32:18And everyone's like, that's never going to work. And he's like, do it. Just do it. And so we had our prototypers. We had also engineers, front end engineers, who were really good at prototyping, like literally took everything in the interface and put it behind one button. And he said, this is how you're going to see what you really need in the interface. So we were never going to ship that. But it was the beginning of the redesign. Let me tell you about a product called Sidebar. The most successful people that I know surround themselves with incredible peers. When you have a trusted group of peers, you can discuss challenges you're having to get career advice and just gut -check how you're thinking about your work, your career, and your life.

32:59This gives you more than a like up. It gives you a leap forward. Having a group of trusted and amazing peers was key to my career growth, and this is the Sidebar ethos. But, it's hard to build this trusted group of peers on your own. Sidebar is a platform for senior tech professionals, director to sea level, to advance in their career. Members are matched into peer groups to lean on for unbiased opinions, diverse perspectives, and raw feedback. Guided by world -class programming and facilitation, all running on sidebar technology, sidebar enables you to get focused, tactical feedback at every step of your career journey.

33:34If you're a listener of this podcast, you're already committed to growth. Sidebar is the missing piece to catalyze your career. 93 % of members say Sidebar helped them achieve a significant positive change in their career. Check them out at sidebar .com slash Lenny. As a product leader, how do you think about just the time it takes to create a prototype? And something like this. You know, as a PM, I'm just like, a lot of times, we don't have time to build this whole prototype. We got a ship stamp, we got these calls, we got experiments to run. We'll just build it and see how it goes. How do you think about making time for something like that?

34:08If you're doing it right, it'll be faster. You need to have an engineering infrastructure that enables prototyping. Some engineering infrastructures are too heavy and they don't actually enable prototyping. We had a problem with our mobile apps that it was too hard to prototype and we actually redesigned our mobile apps until we got to the point where it was easier because our desktop app was pretty easy to prototype, but you have to have a layer of abstraction that enables you to do that and you have to have engineers who have a prototype you meant set. And if you build multiple things and you have this mindset as I'm willing to throw it away, you write code that you know is never going to make it to production.

34:50So you can get you can just crank it out much faster. And then you can see what works and then you build the production And so you actually get to your end goal as something working faster. But you need the engineering team to have this same mindset. The product and engineering have to work together and design because design is just in it. In it as much as sometimes you can get design engineers who are doing the prototyping. So your first prototypes are like Figma prototypes. And then you get prototypes on real data. When I was at Google, one of our teams at Frontend Team, I remember we got, we hired a bunch of prototypers in our head of front end engineering said to me one day, this is my secret weapon.

35:31This is how we move faster. So I do think it's a mindset shift and a tech stack shift. It's also we're going to talk about AI later, but it's also getting easier to build prototypes with AI. There's this video recently that went around on Twitter where an eight year old girl was building an app and in like 45 minutes, she built like a chatbot using this protocol cursor. So I think that'll unlock a lot of great product opportunities and just accelerate the sort of work. I asked about Jeff Bezos and Stuart Butterfield. I'm curious if there's another leader you've worked with that maybe is less known that you also learned a ton from that might be worth talking about?

36:11I think that there are people who are really, really good at what they do. So Mark Benioff is an amazing marketer. He's so at like his marketing mind. So after the acquisition, I got the opportunity to be on stage with him at Dreamforce for, cause you know, Slack was a new shiny thing. So of course, Slack was gonna be in the keynote. And so I was in the, yeah, Benioff's keynote two years in a row. And so I watched how he approaches his keynote and the whole thing around Dreamforce. And Dreamforce is incredible at the impact that it has in the ecosystem. And so I think that as a product people don't think of him as much, but as his marketing abilities are amazing what he's done.

37:06I've written posts about how various companies got started and Salesforce's story always comes to find where they go to conferences with us like no software mascot walking around. And I remember they did something around what are the competitors where they just like created some real controversy around someone. Well they went into their competitors conference and stood outside. Right. That's what it was. Yeah, this is in the beyond software. So this is in his book that he wrote about the early phases of Salesforce and it takes guts. He He pushes the envelope. I love that. It's a really good point that people don't think of Mark Benioff as a marketer.

37:44That's interesting that that's one of the, maybe the main thing you took away from him is just the power of marketing and the skill. He approaches his marketing presentations like a product person approaches their building a product. Amazing. Okay. Speaking of former colleagues, I asked one of your former colleagues, his name is Fuzzy He's now the CTO of Notion. You worked with him. I believe it's Slack. And Google. And Google. Wow. Okay. So I asked him what to ask you. And he said that you're amazing at building strong cross functional relationships, especially with engineers. I know you used to be an engineer.

38:21So I get, where that skill comes from. What can you teach people about building stronger cross functional relationships, especially to build better relationships with our engineers, designers, other folks on our team. Probably the most important thing that a product leader does, because if you have great ideas of what to build, but you can't get them built, then you go nowhere. So one, make sure you're somewhere where you have a good engineering partner. So Cal Henderson was a co -founder and CTO of Slack, and I couldn't have asked for a better engineering partner. He's just awesome. And you need to, that has to be part of your evaluation criteria that you meet and value your engineering partner before you join.

39:05Or you know that it's not the right one and the organization is willing to make a change. So that can happen too. You can go in and understand that something has to change. But that is a very, very important thing of what you're doing and what you're assessing when you go in. And then I think what's really important is that you're aligned. You understand your roles and responsibilities and where you're going to divide and conquer and where you're going to be aligned. You don't want any of this, like people in the organization, they asked mom, they asked dad, and they got different opinions and playing one against the other, like that doesn't work.

39:38So one you have to know that you're not going to do that. So if somebody would ask me something that was in Cal's domain, I'd be like, did you have to call? I would never try and go around him. So we were very clear on, you're going to drive this. I'm going to drive this and if it was unclear, we would talk and we would say, okay, who's going to take this one? And we would do all our reviews together. And so all of the OKR reviews, all of the, we had weekly exec reviews, we had the updates on our OKRs. So we did them all together, but I knew like, here's the things that he was going to ask the questions on a dig deep.

40:15And then when I was, he would take a back seat, but of course we could ask questions in each other things, but I knew that he was taking ownership. And he knew what I was taking ownership of. But I think the bottom line with respect is that you have to respect and trust that they actually will follow up on what they say. They will encal and fuzzy. We're like, amazing dad. I would go to fuzzy and be like, hey, we need more mobile engineers because this one product is not going to ship and we and he's like, I'm on it, got it. And that was all I needed to do. And like, obviously, he couldn't do it.

40:47He'd come back to me and like, Hey, there's going to be a problem. But it was like, just things got done. That's, that's the best part of it. You talked about being aligned, which I love and I fully have seen that power of that of you and your management, your desire manager being aligned on. And you tell me if I'm wrong, but specifically on what goals you're trying to achieve what success looks like, things like that. If there are any tactics you found to create that alignment. And also if there's anything else you want to add to the point I just made about what it is, you're specifically aligned on, that'd be great.

41:19One, you gotta spend a lot of time together. I mean, there isn't kind of a way around that and you have to document things and make sure that you've talked it through. And if you don't agree with something and you're not sure it's a priority, you have to speak up and you can't just be like, like, okay, whatever, and then go to somebody in your team and be like, oh, God, that CTO, like, why did he make this decision? Like, that just doesn't work. So I'm a very direct person. So if I don't think that something is the right priority or is working, I will be very clear. We had different forms. So I'll be very tactical here on the forms that Calle and I had together.

41:56Perfect. So we had, we used OKRs to drive our processes. and we would have the teams present OKRs to us. When the team got too large, it got to be too much time to go through every team's OKR review. So we had them get a Slack channel for each team. They're OKRs. They're a planning channel for each one. And people would post a doc and then a Slack video of going through the major points. And we had a time limit of how long they were allowed to be. And they would say, here's the OKRs. Here's the things that you would pay attention to. and then Cal and I would do a marathon and we would watch them all together.

42:36And we're gonna room sitting there watching them together. Correct. And then we would like say, do we have any follow -up questions that we put in channel our follow -up questions to the team and sometimes there would be like five to ten teams that we would then have a follow -up meeting with. We would say there's things that we, this is like a really high priority project or there are a lot of questions that we have and then we were deleted, but we were always doing those meetings together. So that was the okay, our reviews of getting the alignment. And by asking the questions, we could then, by adjusting us, we could kind of dig into the team.

43:10And we had, we had a chief of staff. So it was the two of us, plus or two chiefs of staff, and which they also did a divide and conquer, and they worked really well together. They were both long times Latin police. So for years, they had, one had been an engineering director and one had been a TVM. And then every Monday we had a Monday meeting where we reviewed the progress on the top of hairs and red yellow green and don't talk through the green ones, only talk through the red ones and what are the issues. And again, both there. And then we had a weekly meeting with the four of us where we would just go through any issues in the organization and what's going on, what's not.

43:51And the four of us is you, the CTO, the Chiefs of Staff. Yeah. And sometimes we would invite people like there's an issue with QA so we'd have the QA leader come in and present us. But we tried to limit the number of meetings with like teams. So it'd be like the okay Our the Monday meeting was the big meeting that you had to be there and you had to be able to talk to your project and that was it There's so much Awesome this here. I love the idea of this async share your plan and a video instead of meetings with everyone in real time and you could just do a lot of the stuff I think. We iterated every quarter, just like you iterated on a product.

44:26So every quarter, we would say, did the OKR planning work? Or not? And then we would adjust. So we got to the point where at one point, we added up all the hours of OKR reviews and all the people in them. And it was like some insane number, like 300 and something. And we're like, this has gotten out of hand. And so then we're like, we have to do something drastic. And that's when we moved them to Aysink. It was also right after Slack clips launched. I got it. That's the video feature. Very cool. And then I don't know if you're you launch huddles right? Slack huddles here. Did you use that as a part of this where eventually you just get people into a little huddle like synchronistically and talk sometimes absolutely we would be in the reviews and we would huddle with somebody to ask a quick question.

45:06We use huddles all the time. I still do I love how it's love it. One crazy thing about Slack is people in Slack don't actually use email internally right? it's like all in Slack. It's like the actual vision of Slack working within Slack. No email unless you're dealing with somebody external and now it's mostly Slack and F. Anyway. Wow. That's amazing. OK. I have one more question around product stuff and then I want to talk about AI. So I was reading your newsletter on Substack, which we'll link to. And you share this really interesting insight that I've experienced myself that I'll quote you here.

45:40One of the mistakes that I see a lot of product managers make is they over index on people who are going to be unhappy with the products they're launching. And you basically, your advice is not to worry so much about making users unhappy, which I think is countertuitive. And countertuitive to some people. You just talk about this lesson, and I'd love to hear what product you launch that made people unhappy that you realized, oh, maybe we don't need to worry about this as much. I saw it more when you launch things. You tank things away. There's always some set of users that are using a feature that nobody else does and Then you take it away and they're super unhappy, but there are more people you're gonna make happy so you get a product manager gets caught in this trap of the vocal minority and The number of people using your product depends on what phase are you a Google?

46:34I use lac or you a glean, but the number of people using your product today is is usually, unless you're a Google far smaller than the number of people you're going to be using your product tomorrow. So design it for the people, the bigger number of people who are going to be using it tomorrow. Like if you have to redo the UI and you know that, like with my cheese, people will be unhappy. But all the new people are going to be like, this is so much easier. Then do it and deal with the people who are unhappy. But the trick is you have to be respectful When you have to be transparent and you have to explain.

47:09You have to go to people and say, this is why we made this change and you have to be authentic. You can't be dismissive and you can't have marketing speak. You have to really say, here's for real why and you have to listen to your audience. You don't want to alienate your early users. Because most people, if you made a good decision on why you moved this or why we stopped using slack calls and moved to huddles and like you have to do it over time and give people choice and then give them enough time to move. So you have to do it in the right way. But if people feel heard, it makes a difference.

47:50And I have an example that's not a product example, but I think is a really good one. It's a leadership example. So when I was at Google, there was always a controversy about something. But there was a controversy about it was like blogger or something. I can't even remember what it was. It was like we made a change. And there was an engineer in my, in my, or, we were like 50 ,000 people at the time. There was an engineer in my team and I see an engineer was like super unhappy about the change. And I knew Rachel Letztenge. She was in charge of all of PR and global policy at Google. So a huge job.

48:22And I emailed Rachel and I'm like, hey, do you have like an FAQ or something that I can help me because I don't know why you made this change and that I can help explain to the engineer in my team. She did not respond to my email. She picked up the phone and she called him. I had no idea. She was she did this. She just called this IC engineer and she listened to him. And she heard why he was upset and she explained her reasoning. He was so blown away that she called him that he completely changed his opinion and then he told everyone else in the art. And so it had this effect. And I learned a lot from watching her do that.

49:02Like she never even told me she did it. Like later she just did it. You just act. You're authentic. You listen to people and you're transparent. It's so funny. This reminds me of a book I'm reading, a parenting book. I'm reading right now that a former guest recommended called listen and the core Theses of the book is when your kids are acting up or there's their getting off track so much of what their need is of sense that you're connected to them a connection which is rooted in you listening to them and so all these my favorite You're an interesting book. I don't know if this is the same one or a different one.

49:38It's how to talk so your kids will listen and how to listen so your kids will talk. Maybe it's the new name of that book, but it's so good and it's so true in everything and also in products. So whether you're in a forum and explaining to customers, whether you're enterprise customers, you're explaining, you're hearing them out, people want to understand. Amazing. So many applications of just the power of listening. Okay, well, not quite a segue, but let's talk about AI. You're kind of at the epicenter of AI now with Clean. How do you anticipate AI will change our jobs in product? What do you think people may be armed, recognizing it, realizing it, what have you seen?

50:20I'm going to give you a little bit of my history with AI to get to that point. So I actually, when AI was a completely different technology stack, I have a master's So I started working in AI when it's evolved so much. And then of course at Google, using it for autocomplete and search. And it's transformed so many times. But then with this last transformation of Gen AI and that's what brought me to Glean of seeing this like meeting lots of AI companies and like, wow, this is really gonna transform how it worked. And it's just fascinating seeing these products. I was kind of one of those people like, oh yeah, it's going to be so far away until like I saw GPT3.

51:05And I think AI, we are underestimating how much it's going to change how we work. It's not going to be sudden from today to tomorrow because people haven't figured it out yet. They kind of haven't figured out how exactly to leverage it, but the people who have are going to be so far ahead. They're going to be far ahead of everyone else because they're going to be working faster, they're going to be working better. And in five to ten years, I think the lines between product managers and engineers and designers are going to blur because AI will enable product managers to build prototypes, to build designs, designers to build a product like Sigma already has their FIGMI eye, you can press a button and you can get your initial prototype working.

51:53You've got all the cold pilots. So they're not quite there. You still like with co -pilot or with cursor, you need to be an experienced engineer to know when it's getting it on, but they're going to keep getting better. So I think people have to be careful about not getting left behind, but their jobs aren't going to go away. They're just going to change. And we're going to, I'm a believer that we're just going to love more stuff. But I talk to engineers into PM and say, yeah, I tried that. It doesn't really work. And then go back to how I worked before. And that's a, that's a dangerous spot to be in, I think.

52:33For people like getting society here and this, where they're feeling good, they're going to be left behind. And like, oh my god, I don't know enough time to do this. Or I don't know what I'm doing here. Give me advice for how, so with something someone can do to. not fall behind. Use the products. This is what good PM should do, period. Always be using new products. It's not a unique thing for AI. When mobile came out, PMs needed to be using mobile apps all the time to try them out, see what the UI's are, see what's working, what isn't. And the same goes even more for AI. Use chat to PT if your organization has clean, use clean, use claw, try them all.

53:13like try them and see what they do. I was talking to a product manager I know who is more forward thinking and loves playing with new products and he had this use case that like blew me away. So he said and this was like a couple months ago. So before it was right when Gemini had expanded the context window. So his product had a discord channel and he took the transcript from the discord channel, which was huge, right? And he fed it into Gemini, the entire channel, and then used it to ask questions. Like, what is the sentiment of my product? What is the most requested feature? What are the things people are unhappy with?

53:55Like, this never worked hard to make. And it's like, that is so smart. And he's like, it was like a gold mine. You know, how long it would have taken him to read? And he just wouldn't have done it, Right? So you're think about like for the argument, oh, I'm too busy. Well, if you use these products, you're actually not going to, you're going to, it's going to be a leverage on your time. So you get a lot of articles sent to you, summarizing. You use AI to summarize the articles. We use GONG at Glean to record our all our sales calls. We have a Glean app that will read all the Gong transcripts, put them in a spreadsheet in a certain format of who the AE is, etc.

54:39and then summarize all the top requested features from all the Gong calls. And we took a while to get it right. Like at first it would, the summarization, the prompt, wasn't good enough and would give us features that like our salespeople would recommend and didn't distinguish that this was actually the customer. See, I have to tweak it. It's not going to work out of the box. But then we got to the point where it worked. And like these things really save time. And you have to use your creative juices as a piano to figure out how it can help you and have patients to iterate and keep trying because the models that we have today can do a lot already.

55:22Yeah. I love your example of the PM and what they did with the Discord channel. Is there anything else along those lines that either you've done to leverage some AI tool to be more productive or folks in your team have done to be more productive you their product leaders or other folks? So many examples. So one that I did really recently is I wrote a prompt in Glean to help me get the status of features and like you have a launch cowl and you can look at launch Cal and we'll say a day. But then, like, is it really the day? What are the outstanding issues? So it will look at our launch Cal and it will look at see if there are any open -gear tickets, what the Slack conversations are, to see and the customers who are beta testing it, like, bring all these together to tell me, okay, launch data is this according to launch gel, but here are all the open issues and here are the open conversations that people are talking about.

56:19So then I can give me the confidence level of the feature with it. So I can run the prompt, just put in the name of the feature and get like, so I don't have to read all these channels. So this is a prompt that I built like two weeks ago. Because we're advancing our prompting capabilities. And so I was testing it out. And I was like, oh, I could do this. So that's like, and that's another example. And engineers are using it for automating part of the incident management of like I got an incident, how do I see all of where their previous incidents that were similar to it, where they're not. And so these are the type of things you can look at to say to help you.

57:01But like the simplest, simplest is there's so much news. Let me just like paste in all these things and summarize them like as a product manager, Gleam, hear all the latest news, what do I have to care about? What's impacting me and potentially competitive to any of the products that I have? And putting that into say, collater jet JPT you're saying. Yeah. Yeah, I think that just, I think that beginning of the prompt is something a lot of people don't know is the power of just giving it a role. Like you are a product manager, a glean from that perspective, give me this summary versus just summarize this.

57:40And that ends up being really powerful, right? 100 % and you can compare and like what is Quad, you know, PR saying that's like they just launched Quad Enterprise. How is Quad Enterprise different from OpenAI Enterprise? And what are like because these things, again, you can do it yourself, but these micro improvements in your productivity help. There's a, I, for my newsletter, I interviewed Claire Vogue who came out with chat PRD And so, product managers are using it. We're just starting to evaluate it internally in late. So, I haven't personally used it yet, but it's super cool. And you can use ChatGPT to do a PRD.

58:23And ChatGPRD gets more templatized, more frameworks of how to do that. And these, again, these things are going to keep getting better. There's been on the podcast. She's going to be speaking at the Lenny and friend summit coming up October 24th. Oh, cool. Yeah, and she a crazy statue sharder. She's making six figures off this product that she builds on the weekends chat priority So incredible, but it also shows what you can build with that Right, and it's just her I think she just recently hired some engineers to help because she has she has like three kids CTO launched darkly and just is building this on the side making hundred thousand plus dollars Incredible I want to add one a couple things here.

59:03So one is for folks looking for ideas for how to use AI tooling for their PM job. There's a couple of posts I've written that I'll link to in the show notes just to put that out there. Just a bunch of PM sharing here's what I've done with these various tools. Another thought I love you get your take on. There's a lot of fear that the PM job will be replaced by AI and I recently realized that it's the opposite. I think the PM role is the best positioned to thrive in a role of AI because if you just think about you have a tool that can just build a thing for you, just like you're staring at this blank thing that can build anything for you, which function would you think would have the best chance of building some of asking it of what to build and how to articulate what to build best.

59:47To me, it's clearly product people. They're best at figuring out what to build, what matters most, what the impact is going to be, what customers need, not to say other functions don't also have these skills, but I feel like evolve functions PMs have the most of that specific scale. I love to get your cake on that. I think that AI, the one thing it's not good at, has been creative. So if European who's doing the kind of route work, it's going to take your job away. But if European who actually is strategic and can pull the pieces together and be creative and think how you do something that it's going to differentiate, because it's not going to give you that leap, it'll say, here's what customers are asking for.

1:00:25Here are the problems today. But you have to figure out how to solve it. So in some ways, it might weed out the good from the bad PMs. Because there's our significant number of PMs who are more, like, just execution. And I think that part of the job hopefully will be lowered because I hope a lot more of the execution will be able to automate, like, updating Jira and, like, you know, all these things that just take time and creating even lower, like, launch cows, which PMs have to do manually now. So hopefully a lot of that work goes away and then people can be more creative. And I think designers and pms are kind of going to blend because the best designers I've worked with are product thinkers.

1:01:07And a lot of really good pms can also design. It depends on the what kind of product you're pmming for. But so I agree with the caveat that it will become harder to be a great pm. Wait, say more about that. It'll be harder to be a great PM because the many PMs are doing things that are mostly like project management and that's the stuff that you use. Yeah, let me rephrase. It's not going to be harder to be a great PM, but to be a PM, the not -to -good PMs jobs will go away. The great PMs will still have great jobs. Yeah, it's totally here. So there in the sense there might be fewer. You might need fewer PMs, but I think that applies in theory to every function.

1:01:47I don't think there are engineers, fewer designers. I don't think you'll need fewer. I think you'll be able to do more things. So, think about every company in our head of sales came to the other day. You need to hire more engineers because we just have so many things we need to build. Like when have you ever worked in a company where you thought that you didn't need more engineers? Like you always want people to build more stuff. So I don't think you're going to need fewer. I think you're just going to get so much more done. A lot of people are building AI into their product. Gleenas is obviously an example, which integrates LLMs, which in a layer, non -deterministic and hard to know if they're going to provide anything good, sometimes something really dumb comes out.

1:02:30Do you have any advice on working with these very complicated systems that don't necessarily, you can't control and building them into your products, anything you've learned that might be helpful? My first week at Gleene was eye -opening in learning some of these things, but let me first just explain what Glean does for people who may not know. So Glean was started as Enterprise Search, which reads content of all your SaaS apps, so it reads the content from Microsoft, Google, Slack, Salesforce, do you know like any SaaS tool that you use, it indexes it and enables you to search. So it started as just Enterprise Search, then when, and using AI, so it was an AI search using BERT models and using vector embeddings in 2019.

1:03:15because the early engineers at GLEAN came from Google and Google created BERT to enhance search. And so it was obvious that they would be using ML techniques for search. Then GPD3 came along and added a natural language interface, a chat interface, you can ask a question in a national language and get an answer. And it basically is a knowledge graph of your organization, so you can ask any question, think of chat, GPD3, if you're an employee. Ask any question about your enterprise. So search, people understand search because they understand Google and you put in a query and then you refine it.

1:03:53But chat interfaces, people still don't really know how to use. If you look at the stats from chat GPT from what I understand, the retention is fairly low. Like people use it, they play with it and then they don't get back to it because it's not in their workflow and they have a hard time figuring out what they can and can't do for it. When I got to Gleeve, the first week I met with the assistant quality team and one of the biggest issues they have is people you're trying to use Gleeve for things that there's like no way it could know. Like what should my top priority be next week when like we don't even know what your priorities are and but then there's golden cases that are just amazing.

1:04:42So we're not the example of refining queries and search to years for people to understand how to do. And I took a lot of features of like auto -complete and refinements at the bottom of the page. So we need to build in those things to guardrails to help with the change to help suggest, oh Lenny, here's what you could do. Here's a here's a prompt to find out the status of your feature that tomorrow built. So how do you give people guardrails so that they understand what is going to work and what isn't going to work? Because this whole, I don't know what I can ask. And then on top of that, the time is terministic.

1:05:23Like an enterprise CIO will go use chat GPT on the weekends, but they come to work and they expect their software to be deterministic. So how do how do you help educate users about that? And then the other thing I'll say about using LLMs is the industry is transforming so rapidly. You need to make sure that your product gets better as the LLMs get better. And that too many people are building things to make up and compensate for the LLMs that all that work is gonna go away. So it's okay to do it to understand that it's gonna go away, but that can't be your differentiator. You have to understand that your differentiator is something that will continue to be there as the LLMs get better and smarter.

1:06:11And as part of that, because the LLMs get so much smarter, everyone else will become awesome. And to keep up, you need something that is actually outside of the LLM that continues to differentiate you and is better than what other people are doing. Well, yeah, that your whole product gets better as the LLMs get better. Yeah, and that I think is it's a it's a frame of a frame of mind of How you approach the value add you're having Awesome Tomorrow to close out our conversations. Is there anything else that we did not cover that you think is important to share or you think might be helpful to leave listeners with Before we get to our very exciting lightning round I in my decades of working tech have never been working in environment that's moving so quickly like, and it's really exciting.

1:07:00I mean, it's super energizing, and it's also scary. But you have to change how you're working. You have to change how you're working so that you can keep up, because it is going to be, it's going to be an interesting decade ahead with all these new tools that are coming out, and staying ahead will be hard, but it's also, there's so much, I think, richness and opportunity here. So I advise people to get in the thick of it and try it out, because you'll be surprised at how much, how many products we can build. I love that. I can't help but drill in one level. Deeper, is there anything you found to help you stay ahead and help you stay aware of what's happening?

1:07:46Are there like newsletters you find useful chat podcasts that just like help you keep up the data and where things are going? Is it like a person you look to like, hey, what's new? There are definitely like, hey, I newsletters that I look at. There's AI podcasts that I listen to. I now have a commute. So in some ways, that's good because I get to keep up on the AI podcast. So I just try and listen. I'm trying to build some prompts for myself to make it easier to say, like, take in to, I haven't perfected this, but the chat GPT voice mode where you can and loaded somebody who just, I just hired a Gleam with saying he does this.

1:08:23He loads up stuff before his commute and then hopefully like summarize these articles and then he can ask questions to it. So I need to up my game there but I definitely have a list of like Ben Spites and Minoran and I love, I like those are good summaries and like I like Gleensera goes podcast I listen to cognitive revolution. There's kind of too many of them right now, but I pick and choose. Also, okay, we'll link to those ones you just mentioned in the show notes. The log, Gill and Sarah Goes podcast is called No Priors. I was actually just listening to it on the way here. Sarah is going to also be at Lenny's in -friends summit.

1:09:02She's going to be moderating a panel between Kevin Whale and Mikey Krieger, who are the CPO's of Anthropic and Open Eye. So there we go, another quick plug for Lenny in -friends summit .com. I think it's Lenny's summit .com. Awesome. With that tomorrow, we've reached our very exciting lightning round. Are you ready? I am ready. Okay. Let's do this. First question. What are two or three books that you have recommended most to other people? So one book was recommended to me by Shishir the CEO, Koda. When I started at Slack, he recommended the book Switch, how to change things when change is hard by chip and Dan.

1:09:41and he's and it's such a good book. And it's about like how do you set a path for people to follow the whole elephant and the rider, so setting the path, but yet motivating people to go down the path. And I read it and we had an all hands about, like there was, I don't even remember the topic. It was something that we were like all up in arms about that we had to do. And I just read the book and after though all hands, I went up to the store and I'm like, like you did that all wrong. Like you need to read this book. That is not how they get people motivated. And he read the book and he's like, you're right.

1:10:19So it just changes how you think in organizations to affect change. So that's on the kind of organizational leadership. And one book I really like was Team of Rivals. It's a book about Lincoln and putting together his cabinet and during the Civil War, one I just learned a lot about the Civil War that I didn't really know. And it's about, again, a book about leadership and it's fascinating. Don't think anyone's recommended either of these so. Love them. Next question, do you have a favorite recent movie or TV show you've really enjoyed? I don't know if you like British murder mysteries. I don't know either.

1:11:04It's kind of a niche thing, but there's a guy named Anthony Horowitz and the latest series he did was called Magpie Murders and it's just like kind of an intricate story, so I joined it. Very niche, but amazing. Do you have a favorite product you've recently discovered that you really like? Well, I'm going to give two. One is tech and one is non -tech. So non -tech, I really like dark chocolate. I actually just like a dark chocolate bar. A really good dark chocolate that's simple. Like no frills none of this just dark chocolate. And I discovered this chocolate called the zoo chocolate. And it's a guy in Oakland who makes it himself super niche.

1:11:53He was selling at the farmers market. And it's just like if you were like a simple, like no, not, no anything just in. Not too sold. great pride in the beings and resources in front. Wow, it's called BeZoo with a B. Yeah, BeZoo as in Kiss and French. And then on the tech side, I mean that honest answer is gleaned, is I was listening to your podcast with Nikita who said that people over 22 don't use new products except at work and that like stuck with me. The products, the new products I usually use are at work. And I use them 10, 15 times a day. Like I use it so much and it changed the way I unborted, it changed the way I work.

1:12:41I like, I just, even like the simplest questions, you don't bother people, you don't, you don't interrupt people on Slack. You're like, you know, what's the latest status with this deal? What's the last time we talk to them? I need somebody at a conference and I can quickly say, have we ever talked to this company before? and I can just get an answer and or without asking an engineer, where's the latest design doc? It's just, it has really transformed how I work. So I know it's cheating and that it's the product I work on, but it's the actual honesty answer. And for folks that haven't heard it glean, it's one of the most successful B2B AI companies out there, right?

1:13:18It's like a very large successful business and company that if you haven't checked it out, you should definitely check it out. As our investors say, it's one of the ad companies that's actually making money. Very few of those. Awesome. Two more questions. Do you have a favorite life motto that you often come back to? Repeat yourself share with friends or family and worker in life? I have one that my father told me when I was being really indecisive about what college to go to. He was really kind of bored of the conversation and he said there are no right decisions. you make a decision right and it is so true because you never know what's going to happen in life.

1:14:00You just have to commit to whatever you're doing and have no regrets about it. You can't be like, oh, you know, 10 years later, what if I had taken that job over there? It's like you make you make your success based on how you approach the decisions that you've made. But so it's if you feel regret about something this is a good one to pull out of just I will make this the best I can make it and that's all the best I can do. You can move forward and move forward and love that. Last question. I know your parent, I'm a new parent, I have a 14 month old at this point. Is there a piece of parenting advice that you learned early on that you think might be helpful to me or folks that are new parents or something you've just learned yourself that you think might be helpful.

1:14:46Fourteen months, I think the best parenting book I read, besides the How to Talks to Your Kaila Listen, How to Listen to Your Kaila Talk, is I think it's called Healthy Sleep Habits, Happy Baby. Like, we are much happier when we sleep well. We perform better at work when we sleep well. Children need to sleep. So making sure that they sleep well, and sometimes that's like, I did the whole sleep training like cry until he falls asleep and my kids still speak to me so it was okay. And so you just like that is like the basic things like make sure that their basic needs are met and then as they grow up share your life with them.

1:15:33So a piece of advice I was given was kind of analogous to talk so your kids listen is when they come home from school you can't just say How was your day? Say, you know what? I did a podcast with this person and it was super interesting because they talked about this or that or I'm massively screwed up and I should have asked them this or that. And they'll be like, Oh my God, I was its full today and this is what happened. Like if you share your life with them, they will share their life with you. That's such good advice. I really appreciate it. Tomorrow, thank you so much for being here. You're awesome.

1:16:07We got through so much great stuff. Everything else happened when we get through two final questions. Working folks find you if they want to read up on more stuff that you share and just follow you online. And how can listeners be useful to you? I find me on LinkedIn and I have a subschat called practical intelligence where I've been interviewing practitioners who are working with AI. It was my way of learning. I started it when I was a VC trying to continue doing it. And how can they be helpful if you're a customer of Lean? I'd love to know what you think, what works and what doesn't work. What's the best way to share that with you is it like messaging on LinkedIn, subscribe, subscribe.

1:16:45Message on LinkedIn or comment on your, on your own. Yeah, those probably be the two ways. Awesome. Tomorrow, thank you so much for being here. Thank you for having me. Bye, everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify or your favorite podcast app. Also, please consider giving us a rating or a leaving review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lenny'sPodcast .com. See you in the next episode!