20Product: Product Secrets Behind Uber and Opendoor | How AI Changes the Role of the PM & The Product Development Process | How to Hire the Best Product Teams & What No One Does That Everyone Should Do with Brian Tolkin

0:00This is 20 product with me Harry Stebings. Now 20 product is the show where we sit down with the best product leaders in the world to deep dive on product strategy, growing product teams and hiring amazing product people. Now joining us in the hot seat is Brian Tolkien. Brian is the head of product at Open Door where he spent an incredible six years and is responsible for product strategy and managing the product and design team. Before Open Door, Brian spent an amazing 5 years at Uber through their wildest growth periods and share some pretty incredible product stories and lessons from the Uber days.

0:34But before we dive into the show's day, are you struggling to beat model benchmarks or implement Gen AI in your product? If so, you need churring. Churring is an AGI infrastructure company backed by incredible investors like Foundation Capital and Westbridge Capital. And they do two things. 1. They help leading companies in AI labs, like Salesforce and Thropic and Meta, enhance their LLMs with advanced reasoning, coding, multilinguality, multi -modality and more. 2. They combine human and artificial intelligence expertise to deploy cutting edge AI systems for awesome companies like Rivian and Reddit.

1:12Right now, Turing offers a free 5 minute self -assessment to help you pinpoint your place in the Gen AI journey, get tailored next steps to optimize your model strategy, and then finally learn how cheering can refine and implement your models for better performance. Take the guesswork out of Gen AI, visit churring .com -4dslash -20VC to start your free assessment today. And talking about scaling seamlessly, let me tell you about Work OS. So Work OS is the modern identity platform for B2B SaaS, helping startups move up market with ease. Selling to enterprises means honestly really complex security requirements like Samo, single sign on, like SCIM, provisioning, audit logs, honestly a ton of stuff that is just a nightmare to do.

1:59Features that take months to build and maintain. While work at Streemline's this with flexible, easy to use APIs that may enterprise readiness quick and painless. Its sweeter features include auth kit, a complete user management solution, Free up to a million monthly users with built -in MFA, R -Back, Bot Protection and User Impersonation Enterprise SSO, my god these enterprises love their acronyms, supports any identity provider using Samil or OIDC while Diretry Sync enables seamless user provisioning and deep provisioning Oh no, for SCIN compliant directories. Find grand authorisation powers, complex permissioning, and it comes again.

2:45The admin portal simplifies SSO and SCIM onboarding 4 IT teams. Trusted by companies like Kerser. Truck Kerser uses. If Kerser uses this, it's gotta be great. Come on. Poplexity, Vacell, Sierra. I mean, these companies are awesome. And they've raised $95 million in funding from amazing people like 20 VC. Try it today at workos .com. Fords slash 20 VC. Now that you've nailed enterprise features, let's talk about creating amazing product experiences. Get your users to do what you want them to do. That's the simple power of Pendo, the only all in one product experience platform. Pendo combines analytics, in -app guidance, session replay, feedback management and road mapping, all purpose built to work together seamlessly.

3:31Trusted by over 10 ,000 companies, Pendo is transforming how businesses understand and engage their users. Plus, they're the creators of Mind The Product, the world's largest product management community. It's awesome. See the magic for yourself. Visit pendo .io forward slash 20 product hyphen podcast to get started today. You have now arrived at your destination. Brian, dude, I am so excited for this. I've wanted to make this happen for a while, so thank you so much for joining me today. Thank you for having me. I'm super excited as well. Let's be great. Dude, I spoke to so many people that worked with you at Uber, and I heard that you were instrumental in the China pool uber launch, which allowed uber to compete directly with Dede in major cities.

4:13This is what everyone told me. What are your biggest lessons from that time and launching China pool so efficiently? We were launching in China at the same time we were standing up a Chinese data center in China and there was all sorts of technical issues with the day before launch getting everything to work properly and we were launching in Chengdu, by the way, a city of 20 million people that most people at Uber had never heard of. And we were launching for rush hour because the product realized heavily on liquidity to make efficient matches and all of that stuff. And it wasn't working in those 9 PM, 10 PM, 11 PM midnight 1 AM.

4:48I think it slept 30 minutes on the floor of the Chengdu Eater office, launched at 5 .30 or maybe 6 AM. And knock on wood, we were able to figure it out and it worked. What are your biggest product lessons from doing that launch? from that time with Uber and China. Yeah, so I think my biggest product lessons are one, understanding the components that make your product work really matter. So in our case, we are trying to do matches between two riders for Uber pool. The thing that works, the thing that customers care about is the quality of the match and the price of the ride. The thing that drivers care about is also somewhat the quality of the match.

5:28And those things depend on having really good mapping and routing data. And the reality is in China, there's no Google Maps. It's much more difficult to have routing and mapping data. And we had to work really hard to try and figure out how we can make good matches work. And the reality is when we first launched the work that could have matches and the road infrastructure in China is very challenging. You have massive highways and overpasses and all this complexity that is just a little bit simpler and in places like the US. And I think we underappreciated the complexity of how you would make good matches without awesome underlying road data.

6:03What do you know now that you wish should know then when you think back to that time? We probably could have done a better job at the start in acknowledging the cultural differences of building apps in China versus the US. And the reality is you go to China and it's different. It's not as like the sleek, elegant design isn't where design falls into the background, isn't as prominent and it's a lot more color and red and big buttons and that type of stuff. If you think we've seen the globalization of product design, when you look at TikTok, when you look at red mode, when you look at the intrusion of kind of Asian influence into actually Western product design, are we becoming more like them, or are they becoming more like us?

6:42I think there's absolutely a convergence. I think probably the influence is a little bit more being pulled towards us, but the reality is like apps like TikTok, there's a lot of things going on, things flying around, scrolling all that stuff. As our attention spans shrink, Yeah, I think we're certainly meeting in the middle. What was the worst product decision you made at Uber? And how did that impact your mindset going forward? Back in the early days of Uber Pool, the way you accessed the product was as sort of this subset of Uber Act. So was it all on the slider as it is today? You go to Uber Act and then above it, you see this little toggle where you can pick Uber Pool, you can pick Uber Act and you see some differences around the price of the time that you would get there.

7:24And for a little bit, the default was Uber Pool. And I think that was a poor product decision, because even if you had chosen Uber X on your last ride, it defaulted back to Uber Pool. And so the UI wasn't clear enough what was happening. And so people were accidentally choosing Uber Pool. And they thought they were getting Uber X. That's maybe what they were accustomed to or whatever. And then someone else would show up in the car. How does that impact your product decision making? I think we prioritize the need and the desire to test out the bounds of liquidity. I .e., we prioritize the needs and in some ways of the business above the needs of the user in that case or respect for the user's choices and wishes.

8:05And I think that's a lesson of deeply internalized. I think good PMs you have to do both. You have to understand what works for the user and works for the business. I think in this case, we prioritize to aggressively to under action. And you said that about good PMs being able to balance between the needs of the business but also the needs of the user. The role of the PM itself is changing so much it would seem, especially in a world of AI. Can you talk to me about how the PM role changes in a pre -versus -a -post AI world, most significantly? The tools and means and mechanisms, I think, will change, right?

8:37And so your primary tool or artifact of writing a PRD versus building a quick demo may change as it makes it easier and cheaper and to build. And I think we will see that. We'll see a collapsing or a converging of the end product design tree at. I think they will never be the same, but I think everything will be pulled in tighter. So I think all of that will change. I think what won't change is actually the core of the PM job, which is you gotta go talk to users, you gotta go figure out what people want, you gotta go build it, and you gotta go build in a way that makes sense for the business, right?

9:08And I think trying to actually figure out the creative part of, okay, I've got some customers telling me this. I've got some customers telling me that. I've got my CX team telling me this. I've got my user interviews telling me this. I've got my data telling me that. What do I actually build? What do I do? How do I make a good decision that works and how do I do it with velocity? How do I know type one from type two decisions? Those core components, I actually don't think change in a pre or post AI world. I think what changes is your ability to communicate indicate those ideas, your ability to do more upfront, your ability to do better user research because you can show prototypes easier.

9:41So I think the tool chest gets bigger, but the core skill set of like actually figuring out what people want and does it work for the business is the same. Does Figma get replaced by a rut plate in this world? I'm seeing more and more people jump that. Yeah, I don't know on the company level. I think maybe the generalized version of that is do more PM start in prototyping land than they do in strictly design land? And I think the answer to that is yes, but at the company level, I think Sigma continues to build for designers, pms, and engineers and make the product closer and closer to being able to be implemented in production code.

10:16So how does the product development process change in the world of AI? I had for many people that've worked with you that you were like the master of the actual process of product development. How does that change in the world of AI? That's super kind of people to say. I don't know if I would consider myself a master. If you go back 20 years ago, it's like pretty waterfall -y, and I think we've, as an industry moved away from that, but you still have the PM owns this artifact of the initial PRD or one -pager, and design owns the Sigma file or the design prototype, and engineering owns the actual implementation in the code.

10:48That creates a process in and of itself, right, where the PM is at the top of the funnel and developing the idea, then the designer comes in, New York, the designer, and then it sort of moves down funnel. And I think AI completely collapses that cycle and accelerates a lot of the upfront, up funnel work where again, you can just, you can build a prototype and you can show that to customers. And like the PM and the designer might just work collaboratively to say, let's just build a prototype. Let's skip the document stage and let's just do that instead of maybe talking as a first step and then showing some flat files and then maybe building one or two prototypes to sort of refine your hypothesis.

11:25And so I think the development process just accelerates a bunch. You mentioned the one page of that. Say we keep it just for now. So you've seen many different types of one pages. What is a great one -pager? And why do most people go most wrong? Yeah, so I think if you're early in the process, the important thing to get right is the problem definition, the why? Right, what is the core insight that is the reason that we're even talking about this project? And that's usually a user insight or a business insight and or both. And then anybody should be able to pick up that one page and say, okay, I get where I get why we're doing this and I get the problem.

12:04It's totally fine. Slash expected that the solution is probably pretty under explored in those one pages. I think people get wrong where the one page is the solution description, not the problem statement. How do you think about prioritization of problems to go after? There are so many different problems that you could choose. A lot of teams have a lot of different opinions. How do you determine we are going to commit resources to this and not this? the standard way would be like impact confidence and effort, right? And I think that's a reasonable good framework and those are reasons it's been around forever and it's reason that people use it.

12:33I think that the part that has to mash up is something about time horizons and the priorities of the company at any given time. And so what I mean by that is like one tension that often arises is how much do we build new features now versus maybe stabilize the existing features versus paid on tech debt. And I think these can become spiritual debates. They sound like they come much more challenging when you're public. It is more challenging when you're public, but it doesn't change, I think, the core principle, which is you still need to decide at the highest level what time frame you are optimizing for.

13:07And so I think things like experience debt and the tech debt, the debt part is actually like a really apt description where you trade paying it down now for pain later or you trade taking it on now to pay later. You're an angel investor in my company in a hypothetical situation. I've got investor and I'm series A style. So I've got investors and they want me to go from two million to eight million in a year, but I'm mindful that I've got growing technical debt. Do I focus on new product expansion and just let the technical debt ride it out? Or do I solve technical debt at the cost of maybe lower growth and lower new products?

13:45I think when you're early -ish stage like that, at seed sales, I do $2 million trying to get to $8 million, like you have to earn the right to exist in the future and paying down tech that doesn't pay the bills. And so I think at that stage, it's probably a little bit early to start paying down technical that. Now, it depends. Are you in like a land grab? Uber was in a land grab for many years, right? It had to be first. It had to get riders. It had to get drivers, right? And so there's very much a land grab competitive dynamic that pushes a certain philosophy. I think most early stage companies tend to be that where you have to figure out if the thing that you're building in that 2 million AR you may not know yet actually has value and people want it and people will pay for it and so I think early stage slowing down to to pay down that debt is oftentimes a bit challenging.

14:35Should the CEO always be the CPO? Always know but in the early stages yes there are certain scales of companies where that maybe doesn't make sense anymore. But in the early days, yeah, I do think they should. If you think that the most important asset your company has is the product that is delivering, I just think of the early stages outsourcing that to someone else can be really challenging. We mentioned like the focus on new products who are expansion of products versus technical debt. You have gone from single to multi -product. Was your biggest lessons on how to go from single to multi -product?

15:08Well, it's a challenge so many face. There's a product challenge and then there's like a cultural challenge. The product challenge is you have this chasm to cross, which is you can't degrade your core product. Right? It's a classic innovator still on me. You can't degrade your core product to build a new product on top of it and until you know that the new product is working, right? You have a bunch of users who care about your core product. And I think the best way that I've seen this approached is by giving new products a little bit of a sandbox, a contained sandbox that relaxes a bunch of the constraints of company, but does it in a way that if it's completely fails or whatever, it actually doesn't harm the overall core product experience.

15:46So Uber and Open Door have an advantage where you can do this geographically by city. In Uber's case, even more extreme is you do what Uber eats was, which was like a totally separate app, a totally separate organ, a totally separate everything. It doesn't harm it. Does it have to benefit it? A lot of people say if you're going to do a second product, it has to benefit the first. I don't think it has to benefit the first, but it has to take advantage of some competitive advantage that your company has. And so if you think about a classic two by two matrix, where you have your customer set and your competitive advantages as a company, your core product is your existing customers with your existing product and competitive advantages.

16:23If you have a new customer set and a new set of competitive advantages or capabilities, that's probably just a new company. And so you're either playing in, do we take our core capabilities and attach to a new customer set or do we take a new customer set and attach to our core capabilities? I think you can play in either one of those, but in both cases, it doesn't have to make the core product better, but it does have to make the core experience for your customers better and it does have to make the business obviously better. What fuck -ups did you make in going from single to multi -product and very delicate with my words?

16:57I think the biggest challenge with going from single product to multi -product was probably the one that I described previously. That was Uber going from Uber X to Uber Pool in one. And that was actively, I think, carving the overall user experience. So that was probably the single biggest. What about it, I think? At Open Door, we had initially started building heavily on the buyer side of the business. And we knew we wanted to be multi -pronically had a bunch of initiatives over the year of building a retail buyer business, building a mortgage business. We could have been better about focusing on what are our core strengths, and how do we not be in that new customer set, new capability quadrant, but new capability setting existing customer.

17:35And I think that's what the company is having now with a lot of this new products focused on helping sellers and how sellers can sell their home in a variety of ways. You said to me before, the most important product skill set is simplification, speaking of a product expansion, simplification and finding the kernel of truth. This is a pretty one. It's a sea of cacophony. I read this and I was like, my God, it's like, it's having ways fooling into my email. So can you explain that to me of why a colonel of truth is a sea of cacophony? The product job is challenging, right? You have executive pressure, you have your independent thoughts, you have what customers are telling you, you have maybe a scaled customer feedback from your CX team, you have maybe your sales team yelling at you with some other feature requests.

18:19It's like it's the prioritization exercise that we talked about. So you have all of these different pressures and they often come in a variety of different forms, but oftentimes they come in the form of solutions. Hey, we need this feature. Hey, it would be great if the product did this. Hey, we lost a deal because of that, whatever. The core of that product job is say, okay, there's all of this feedback, all of this noise, what actually matters to user. In the Uber example for a second, there's a ton of things that make that product work in the early days that made that product work. But at the end of the day, is there a car available?

18:49Can it get to me quickly within five minutes? How to price that's reasonable? And if you can do that, all the other stuff, all the other product nuances, like kind of fade away. And so that's the simplicity that matters for that product. And so trying to align and say, okay, how do we prioritize everything that moves one of those particular dimensions? But actually, like trying to understand that when maybe someone's saying like, hey, cancellations are a big problem. And our pack is too high. And like all these other things that are problems, real problems or potential solutions. And find like, okay, that's what I really need to focus on.

19:20That's what matters. Who is the one to set that okay or of that is what you need to focus on? Is that the CEO? Is that the CEO? Is that the head of product? That level where you're talking about that needs to come tops down and say this is the the success metric for the company and then that cascades to the rest of the orange. So for example in the Uber case just to extend trips may just be the metric right? That's the okay or that matters but I'm doing Uber pool right? And so like okay I can see that as a product reader for my area right which is not the whole company, obviously, for my area and say, okay, how do I ladder to that?

19:54Right? And so what matters to me is, okay, in this case, it's easy. Uber pull trip count. But I can match my OCRs and everyone else can ladder their OCRs to the top one. So OCRs to me are like cascading trees. Wherever you are in the organization, like it should be layering up to the one or two above that. What are the big mistakes people make with OCRs in product, do you think? I think the biggest one is having too many. How many can you have? I think teens can generally have a few like three at any given time that really matter maybe three to five depending on the size of the team But if you focus on everything you're not focused on anything and so I think in the okay our process It's important to a having manageable number But then be able to articulate what important things you're not doing and the tough trade -offs that you had to mix if you're doing Everything and you're doing it all and you're doing it all at once.

20:42It wasn't necessarily a regress prioritization that's what it says. Where did you focus at Open Door? Where with the benefit of hindsight, you shouldn't have. And how does that impact your mindset? Open doors had different variations of a market product that didn't succeed. And I think those were like articulated reasoned decision. So I think the outcome isn't necessarily what company wanted at the time. But I think it's hard to divorce like bad decisions from bad outcomes. So the hindsight becomes very easy to say, we should focus elsewhere. But I think there were well -reasoned decisions at the time.

21:11But I think focusing on the right to win for sellers is where a lot of the magic is today And I think that's the right focus here. Can I ask you when you think about we've spoken about single to multi -product We've spoken about technical debt. Do you think people are destined for certain stages of company development? It's often instead do you agree with that? No Because I believe in human agency and the ability for people to adapt their skill set I think certain skill sets are better adapt at certain stages of company But you agree that people can adapt skills as I do think people can adapt skills I think people can grow with companies.

21:45I think people can grow in their career I do think people can adapt their skills that people want to have to do that That may be the hardest challenge because people can be very good at a thing and it's easy to stay good at that thing and do that thing How often should you change your chaos? We were talking about having three to five per team How do we think about how often we should change them and how should they be communicated to the team teams should be developing their own all cars, they should be defining the work that matters the most to the strategy and the top level Ocares that are outlined.

22:14A couple of philosophies here. One, generally, the quarterly planning process or whatever. I don't think you should be changing your objectives that frequently, right? Otherwise, it's an unfocused strategy. It's unlikely you officially completed your objective in a quarter. The more nuanced answer is you burn the right to set OKRs on longer time horizons by proving you can execute on shorter time horizons. And so super early stage companies, like having an annual year long OKR process when like it's not clearly going to build in three weeks. I don't feel like a super worthwhile exercise. And so I think you earn the right to plan on longer time horizons by proving that you can set and execute over shorter ones.

22:52How do you think about the importance of speed of execution in product versus the importance of taste of refinement of beauty of kind of human preference? How do you think about that balance and what's ultimately more important? I probably personally fall a little bit more on the velocity side. I think the reality is as good as your product intuition is as the reality is, you throw something on it into the world, you ship something, and you figure out if people like it or not. And the more shots on goal you take with that feedback loop, the more likely you are to succeed. And so I don't think you can throw out crap, right?

23:27Because the risk is that you have like a negative signal where actually your execution was just bad. Right, and that's the reason it didn't work. So you have to have a minimum bar, right? And that's what viable and minimal viable MVP is, but velocity matters a lot. Dude, how do you determine whether your new product change, your product update, your redesign is shit versus uses, it's just not used to the transition or the change. I remember the iPhone losing its home button, me and all of the people around me were like, this is terrible swipe up. Now it's preposterous to think of that. How do you think about whether it's a bad decision or it's just use a preferences changing.

24:02The iPhone one is particularly challenging because it's hardware. And so hardware's hard. In software though, I think you tend to have the ability to give people time. I think what you need to do is be a little bit more rigorous in how you think about your metrics and your definition of success, where the classic, okay, we're gonna make a change, we're gonna roll it out until we get static and then we're gonna pick the winner and then move forward. Like that may lead you in the wrong direction here, right? because it may be you have a novelty effect and numbers decrease or frankly even novelty effect and the numbers increase but it's temporary and so if you think there's the risk of that change being just a change to customer's behavior you may just set up your experiment differently and you say okay we're going to look at all the data but we're not going to make a decision in the first two weeks we have to have the fortitude the guts the gumption to say we might see patterns in the behavior where behavior is slowly changing over time so we actually need to evaluate this experiment on week sides through eight data, not week one through four data.