20Product: How Linkedin Does Product Reviews, A Post-Mortem on Stories, Linkedin Messenger and Spam & Why the Data Advantage in AI is Diminishing with Tomer Cohen, CPO @ Linkedin

0:00If you ask me a couple of years ago, I would have told you that data is everything in AI. What's happening right now is technologies like GIPETE are always betraying on all public data. The whole idea of the whole pre -train is one of the biggest infection point of this technology. It's always trained on every available public information out there. The data advantage that used to exist I think is getting diminished. This is 20 product with me Harry Stabbings. Now 20 product is the show where we sit down with the best CPO's in the world to reveal how they build products and product teams. Now today I'm thrilled to be joined by Toma Cohen, CPO at LinkedIn.

0:37Since joining in 2012, Toma has served in several leadership roles, helping launch and scale new innovative member and customer experiences. Prior to LinkedIn, Toma worked as an entrepreneur with Greylock Partners and founded a company in the Personal CRM space. But before we dive into the show today, this episode is brought to you by Linear. Let's be honest, the issue tracker you are using today, it's not very helpful. Well, linear is different, it's incredibly fast, beautifully designed and it comes with powerful workflows. There's Dreamline your entire product development process. From issue tracking, all the way to managing product road maps.

1:10Linear is designed for the way modern software teams work. It's powerful workflows that you customize and automate processes to fit your teams unique needs. And they have integrations with your favorite tools like GitHub, Slack and it takes just minutes to set up. Linear is the default tool of choice for high -performance teams from startups to a wide range of large established companies like Vacelle, Ritool and Cash App. So see for yourself what makes Linear magical. Visit linear .app -20vc to try for free with your team and get 25 % off when you upgrade. That's linear .app -20vc and speaking of tools we cannot live without.

1:47I need your input on the 20vc mirrorboard. About new guests we should feature for 20VC in 2023. Just head on over to Miro .com for slash 20VC. Miro is a tool, like it's a truly game -changing tool. It's a visual collaboration tool, packed with the right tools, tech and templates, to help you think of and create that dream product. That means you can brainstorm the perfect product with your team, vote on the best ones and explore with your customer journey roadmap all on a Miro board. Whatever you need, Miro's infinite whiteboarding capabilities help you get that. It asks your team's single source of truth, and now I'm using it to hear from you.

2:25Go ahead and add your suggestions for 20 VC guests on our Miroboard at Miro .com for slash 2 .0 VC. And finally, did you know that 90 % of information processed by the brain is visual, and with Canva, it's never been easier. Canva empowers teams to design impactful documents, websites, videos and presentations. There were all clients and colleagues with no design experience needed. Teams of all sizes can design together, from anywhere on any device, and you can start designing to stay for free. What will you design today at Canva .com? You don't need any design experience, there are hundreds of thousands of free templates and a huge content library.

3:03And that's why the largest teams trust Canva. It's used by 85 % of Fortune 500 companies, whether you're a team of 2 or 2 ,000 Canva empowers teams everywhere to create that best work together. What will you design today? Start designing for free at Canva .com. What? Chilo. You have now arrived at your destination. Toma, I am so excited for this. I've spoken just so many of your friends before, so I know an incredible amount about you, and I'm ready, so thank you so much for joining me today. I'm excited to be here. I'm a big friend of the show, and I'm excited to be on it. That is very, very kind, but I'm fascinated.

3:42CPR linked him one of my favorite products. How did you come to BCPR linked in, Tomah? Ever since I was a kid, I love building. I love every aspect of building. I love the entire cycle. I love the problem solving side. I love the design side, technology side. I joined it to LinkedIn was actually very special one. So I became a LinkedIn fan long before I joined the company. I came to the VAL in 2008. I went to a lecture at a Stanford Engineering School. It was about social networks. This is 2008, Social Networks are a big deal. They're not as big as the R2Day, they're like the hot topic. All the big founders of Social Networks that you can think about were there.

4:16And it was all the rage. The hottest topic was Facebook time spent on the internet. Onstage, there was more of an older founder. His name was Reed Hoffman. This is where I got to meet Reed for the first time. Reed talked about the power of online professional communities and how it can create economic opportunities. The first time I heard it and it deeply resonated with me. The idea that a professional community becomes a powerful growth engine for the economy just inspired me on a whole new level. And over time, Read Himself became a personal mentor of mine. And it was only several years later, I had a conversation with who back then was the CPO of LinkedIn.

4:52And this was early mobile days and I came from a startup. And if you came from a startup in the valley, it was all about mobile. And larger companies were still trying to deal with this mobile thing that was happening. And he asked me, he said, how would you rebuild LinkedIn as a mobile product? I was excited to share my thoughts. And then he said instead of talking about it, how about you come and build it? And the rest is history. I joined the company 2012. And in 2020, I became the CPU myself. So it came full circle since then. I do have to ask you and I ask quite unfair questions because they're generic.

5:22But I'm fascinated on this one always, which is like when we think about product state, especially given your experience working kind of within semiconductors, within consumer and then also like income and LinkedIn style. Do you think product is more an art or a science and if you want to put numbers on it attach it to them. Where would you put it, Thomas? I heard you ask this question on the previous podcast I listened to and I must say credit some tension for me because I think it's impossible to delineate science from art. I think they're deruven, they play off each other. There's a lot of science in art and there's a lot of art in science.

5:52Some of the best scientists in the world are very imaginative. After I thought about But I think when we talk about science and product, I think there is a tendency to think about it more as the best practices of product, the skill set. How do you learn the know how around design and data and experimentation and business? And that for me is a foundation of what it means to be good at this job. And honing that craft, the scientific part about encroachs can take you a long way. And it has to be applied learning. You can't sit in a classroom learning how to do product, you have to build. But I think what sets you apart in the craft of building product is your ability to bring vision and creativity and intuition and the judgment and the imagination.

6:31I think that's what sets you apart from the group It's where you listen to customers or users or you just observe them and some would just go and say oh, this is what they want And some would come I think that's is what they need is that uncovering profile needs that they haven't even talked about It's being able to anticipate or technology is going where the industry is growing because you see a few steps ahead and that takes a lot of intuition and imagination and I think that 40 you grow in your career there is an expectation that's what you'll bring to the company. The best product people that I enjoy working with have this combination of knowledge and creativity.

7:05They can do vision, they can go all the way to execution and when I think about the intersection of product people who are dreamers, do our doers or learners, you really get the holy grail of product craft and ability. Tom and we chatted before about content creation on the platform and how much I loved it. I think the shows are also successful because I'm also quite honest, more so now than I have been before. Can I ask a blunt question, which is when you look at the desktop site and you look at the mobile app, do you not feel that it feels outdated from a UI perspective? I think there's a lot that we can do there to improve and there's actually what we're working on right now to really innovate within the constraints of what we've done.

7:44In fact, the use case of LinkedIn in dramatically evolve over the last few years. I do feel like in many ways, the current experience constrains us. And there is a lot we can do to improve it and to rethink it. That's what's happening right now. In a world of generative AI right now, there is a lot of complexity you can potentially unwound by building a simpler experience, what playing AI into it. If you play the last years for us at LinkedIn, as a company, there's been tremendous evolution in the product that led to some pretty remarkable results revenue for LinkedIn more than triple our memo base more than doubled our engagement is reading record levels we're growing the fastest we ever grew right now Can I ask an interesting one I love to come and you said what is something when you think about product you've been wrong on but not confused on and what did you learn from that?

8:31Yeah, so taking a step back on being wrong but not confused is my notion is if you're confused then only luck will save you It's like there is very little chances you'll be successful, but you have conviction rally around that conviction. Yes, you might not be successful. You might be wrong, but ultimately you have clarity and focus. And that for me, are critical for execution. So it starts with clarity of thought about the problem you're trying to solve and goes all the way to the clarity of the solution with some strong principles. I'll give you an example of one that we were not confused about, but we were wrong.

9:04We launched stories a while back and it was a short test in many ways. But we launched stories because we thought that we can basically unlock more creation on LinkedIn by allowing for a familiarity to play a role. We would sometimes hear from people that they think that it's sometimes risky to share on LinkedIn because their boss is on LinkedIn, their colleagues are on LinkedIn, their customers are on LinkedIn. We thought a familiarity might alleviate that concern. And we launched it and we saw some pretty, I would say, mediocre performance. It was far from the unleashing of sharing that the team thought they would get.

9:38When we did follow -up sessions with members, and we were clear about what we were trying to achieve, we thought that the similarities specifically would unleash more sharing because it would alleviate the concern around sharing on LinkedIn. When we did follow -up sessions with members after that, it was clearly that we completely missed understood the job to be done for creators on LinkedIn. It wasn't about wanting things to disappear. It was almost like the opposite. We wanted things to last on LinkedIn. When people share on LinkedIn, they wanted to be seen. In fact, they want to attach it to their professional identity.

10:06They want to feature it on their profile. Honestly, it was a really big ground -working insight for us because we started doubling down on the role of creation as part of your identity. We also learned that when they share on LinkedIn, yes, everybody enjoys their likes and views. It's part of the chemical response you get in your brain. But what they appreciate most about LinkedIn versus other platforms, it's the reputation apart. It's the opportunity. It's been able to showcase my values. It's very different than any other social platform. How important is being the first? I had a Alex Shulson from Metta, C .M .A .M .A.

10:37and I asked him this question, but how important is being the first? You mentioned stories. Facebook obviously followed Snap and Snap, followed Kekow. There's always a hooded first, but how important is being first in product release? I don't think it's first to launch. I think first to launch is not the right concept. I think there is a halo effect of who's launching first right now. that sometimes and as you would see product managers getting really excited about the launches that they've made. But for me, that's not interesting, the launches. I think it's first product market fit that is amazing.

11:07If you're first product market fit, you build an amazing leg up in terms of both insights, the momentum and speed. When we look at stories, we've got a snap product market fit first and LinkedIn were very far behind in like, race to product market fit on the stories. Do you think that was a cool component as well? I don't think it's the same dimension because we were not trying to compete with Snap. I don't think it was the notion of being first to the market that made it special. You mentioned like the data not being amazingly exciting. And I just wanted to ask, how do you know how much data is enough to make a decision?

11:41Is it two weeks of data way or like, okay, we see the writing on the wall? How do you know post -product launch? When enough data tells you the answer versus when you need more? Yeah, I always like to start with before you launch what is success? Once you launch it, what should be going up and through the right that should be excited about? Don't tell me in retrospect because then you're just trying to fit the data into what you're trying to do, but tell me ahead of time and it's okay to, again, you might be wrong, but not confused, just have a strong opinion around it. So we can see for building towards the intuition we have.

12:11I think for me, there is an notion of enough data is that the hypothesis you have isn't being validated. it. Like ultimately you have success with a product when you have real adoption and retention. That's when you know you really have something. And adoption by itself, if you have large enough base, it could also be in a way, a game in a while because discovery is really strong and you can always get people to try out something. But will they stick around? Will they come back to use it? That is your real test. We do a lot of processing around the notion of evidence versus conviction. You can build with strong evidence that you had before, or you can build it with conviction with a hypothesis of showing evidence at a certain amount of time.

12:49And ideally, over time, you start showing evidence. But some of our biggest bets we've made, for example, investing in skills as a way to connect the talent marketplace, started with some big bold bets around idea of conviction. And then over time, we started measuring, are we doing a good enough job there? Are we seeing enough validation for the early hypothesis to invest more and more? It's really a gradual and incremental process that starts with conviction when you don't have enough data, but ideally come back with data to showcase the responses again. As for you of that conviction and data, I do want your advice.

13:21Product reviews are so cool to all product teams and to all companies. How do you do product reviews? How often do you do them? Who's invited? Yeah, so we do multiple product reviews every week. It starts each quarter. We actually review all of our big rocks, our biggest investment areas across the product and the business, and then we determine what do you want to see as a product team coming in. And for that, I decided what's the products I want to recover for review. And what I tell the team is you've put your best thinking forward. You know, you worked hard on the product, you worked hard on the thinking, you worked hard on the design.

13:52Now you're putting it to the team. And our goal as a team in this meeting is to make that thinking better. Is to make that product better. So the only currency really is feedback. And what's nice about this team is you really have kind of the street 60. You have the team presenting. They provide that local expertise, these are the ones who are recommending and they're highlighting a problem. You have my team, which is a product executive team, they cover their whole business. It's multiple areas of the business coming together. It's a very broad. You have cross -functional leaders who can provide other perspectives.

14:23So by the time the meaning is done, they're getting some pretty diverse deep feedback from multiple areas of the product. And that allows for that deep thinking to come to it. Do you see a difference in product review, discussion, quality and feedback quality when comparing remote and online versus in person and whiteboarded? I do. I actually do my product jams in person. I feel there is an energy of creativity and discussion that you get in the room in person. It's you can do it remote as well. It's not that remote does not work, but I think there is an edge to doing it on a wide board, discussing opening, pointing, having free -form conversations, that allow for that creativity and discussion to be a lot more natural.

15:08So I moved my product gem sessions to be in -person and with COVID we moved them to be obviously remote because there was not a way. But once we came back from COVID we moved them to be in -person and across the room people will tell you the energy levels are just higher. The creativity, the ideation, the velocity of discussion is elevated to a whole new level. He sets the agenda and he's invited. I said the agenda for the quarter in terms of what I would like to see as the public area is being covered. And we have a structure for the session itself. So you start with the problem, the finish and ideally as articulate and nuanced as possible.

15:46You share your job to be done. What is that inside that leads your work? And then you share your principles. If those are there, that's already a great session. And then ideally we spend most of the time on the demo. What are your principles? So if the job to be done is lower amount of spam, say on -site, what are the principles? What would that be? Principles are basically your opinion of approach for how you would solve the problem. They ideally have teeth, they have trade -offs. If you're goal is to alleviate the amount of spam or bad activity on the platform at the cost of what. So for example, there is a trade -off between trust and growth because you want to a lot for a lot more growth in the system, you want to reduce friction.

16:25When you reduce friction, you also allow usually for bad activity to engage as well. And the more friction you have to alleviate bad activity, you also take down good activities. So there is kind of the opposite that there's usually a trade -off between the notion of engagement and growth, which trust. And when an idea for Lili, you find a great way to build an efficient frontier where you have high growth and high trust going together. For me, the principles is that the product leader for that area comes with an opinion of the approach of how they will solve it and their stiff to it, right? They're basically making an opinion of how they solved it and what's the trade after going to make to actually solve it in a great way.

17:01Once you have that, you can actually react to something really well. When you have bluntly this structure and you have a lot of ideas thrown around, you also have to make decisions and you have to prioritize. When you come out of a product review, how do you prioritize what to do, what not to do, and what's a luxury but next quarter. So if we just finished a create product jam, then what happens is there's tremendous amount of feedback being shared. Usually the way I end the meeting is I summarize the areas that we covered and where I would like to see progress on. I try to make sure it's between one to three, so it's not the whole list.

17:34And then the working team, we have a new process we started a while back, which is called BriefBack. The team itself, which presented, they sent a summary of the session with the feedback and all the key action items they have and the ETAs for the key areas they're going to work on. So it's really absolutely team presenting to take that feedback and act on it and we literally have those sessions that really allows for clarity and execution but it's really absolutely team leading it to go and do it. Terminal, what's the most controversial product decision you've made? If I go back, the feed at LinkedIn, now in hindsight doesn't seem controversial by the time it was very controversial.

18:09A link that was actually one of the first social platforms to ever have a feed. But it was more of an activity feed at LinkedIn. The feed was really a promotional feed for teams to showcase their products. The growth team would show people you should connect with. And the job team would show jobs you might be interested in. And everybody was using the feed as a promotional way to show members recommendations about that they can do across LinkedIn. The first change I made there was that the feed was first that I thought most about people that matter to you. Talking about things you care about, the feed belongs to the member.

18:40In some of the organizational chart, we start from the member's job to be done. And I can tell you internally, that was a controversial decision, because the feed was already used by multiple teams as a way of discovery. That was a pretty big pivot internally and externally. It's an innovative dilemma question. Is it why you've got to cannibalize your existing product with a new product? 100%. The feed was massive discovery engine for so many products at LinkedIn. And this was a revenue engine for so many products of LinkedIn. For many companies, it was their main discovery channel. So then comes this that confused.

19:12It might be wrong. Productly there and says, I'm going to change it. I'm going to change the core of how this works. But there was high conviction there. And I had to show evidence along the way. But that was a highly controversial decision early on. I think we should do an internal review, me and you here. Well, I say a feature or product. And you can give me a rating out of 10 on how you've done. OK? So if we start on feed, what would you you're rating out of 10. I think we still have a long way to go. I think we're far from building the product that we know we can. What I also think of when I think of that is, shit, AI fundamentally changes how we build products.

19:47Do you as a product leader need to fundamentally change how you lead product teams and product organizations with AI moving faster than ever as we touched on when we chatted before? Yeah, this is something which is we're in into my heart because I've been working on AI with AI for many years and same as we had the mobile revolution happening 15 years ago and how we changed dramatically, how we build and how we use products, AI is going to be much larger. And if you're building product today, I have this analogy when you have a re -erafting boat, you have the guide on the boat sitting on the back and that guide usually has two big pedals.

20:23And those pedals pretty much navigate the boat, they pretty much dictate success of failure for your product. Those pedals for me are AI and that guide better be you as the product leader. So if you read products in your company, you'd have to have the knowledge and skill set to know how to use AI, how to use those pedals to navigate your team in your company towards success. What do you need to unlearn? What are the bad habits that you need to forget is we embrace it. Oh, I love that question. There's a lot of things when it comes to AI, where you have to need to unlearn and relearn again. The, for me, one of the biggest ones, and this is where I usually see the hardest mindset shift for product people is that with AI, you don't control the experience.

21:02AI is not deterministic. If you ever had an interaction in JGBT, when most of the time, you're trying hard to anticipate how it would respond. And I give the analogy of a chef in a restaurant where before, imagine as the product leader, you were the chef of the restaurant, you dictated every part of the experience from the music, the ambiance, the ingredients, how the plating is done, the flavors, the amount of salt, and now with AI, it's as if you were giving the ingredients. You're giving the guidelines, the philosophy, the principles, but AI really learns on itself. You can always refine and help it understand, but it learns on itself.

21:36So you have to forego some of that control, trusting you can build a much better experience with AI and how you do it. Tom, do you and the team use co -pilot? We do. I've been using this technology all the time. It's incredible. Yeah, I had dinner last night with two of our founders and they said they're now 50 % more productive And my question to you is if we think about now get up published 41 % of new code creators a car AI code creation What is that in five years? Wow? Five years is a long time and we ever seen pace like this not in my lifetime I can give example I've been teaching how to build with the a first mindset for many years But it surprised me how fast in fact when you think about what's coming I think we're all going to be shocked by how more and more invads more and more capable because you are setting new baselines that allow for a lot more innovation.

22:24You know, if you go with all the constraints of AI today, algorithms, investment, computing power, allowing for more information to flow in and out in the form of tokens, those will all be alleviated. And when those are all alleviated, you can build faster and better and more accurate. You can allow for more creativity. That five -year horizon is almost like far too long. But I think we're going to see massive amount of productivity coming up. There's a brilliant quote. We've mentioned some great quotes. A big quote that I always think about is, will the incumbent acquire innovation before the startup acquires distribution?

22:55Brilliant quote. The really interesting thing now is, I think incumbents are innovating so well with AI. You look at Microsoft, you look at Adobe, and how Adobe are integrating into core products and product suite. And so my question is, where do you think value of crews in the next wave of AI? Do you think it is actually incumbents? Would you think it startups? So every tech revolution brings with it new winners and new losers. This one is no different. In fact, I think this one, the window for many startups to actually start playing with ingenuity and innovation. To your point, I'm sure many incumbents today have learned the mistakes of the past and they're racing to innovate with this technology.