Unpacking Amazon’s unique ways of working | Bill Carr (author of Working Backwards)

29:10Where people get confused about it is they don't maybe understand like when they're supposed to stop disagreeing is one thing. And so hopefully that explanation made people clear like this is when you're supposed to stop. And then the commit part done well means that it's not just like I'm gonna commit, I don't really agree with what we're gonna do but I'm gonna get behind this. It's a, ideally it's, oh, now I've heard the argument, I've actually now fought about the argument and hopefully that person has now understood why we're taking that direction and so their commitment is based on that understanding because then they can reflect that understanding back to their organization too.

29:59Because the worst thing to do is to say, yeah, we're committed to this. I don't really agree and I still think it's wrong but I'm committed to it. That's not actually commitment. So it's really about, this is really about decision making and understanding the facts or an information that people are gonna use to make a decision and then they will reflect that back. Imagine there are many times I've gone through this why I still don't agree. What's your advice to a manager to a report of just like, okay, when you actually still don't agree, how do you behave? Do you still behave like, yes, I agree with this and don't really voice your concerns or something else?

30:37I work with Jeff. All kinds of different new ideas. And Jeff doesn't think like an old person. He's this level of sort of creativity in the way he thinks, the timescale, which he thinks. There's many ways with the way he thinks that there was no one else in Amazon thought that way. And so there'd be times when even after we've had that discussion, I would maybe still disagree. But then what I would do is I'd focus on, okay, well, what is the kernel or the core of like why Jeff thinks that we should do this? And I would focus on that kernel. I got great advice actually from one of my managers, one point Steve Kessel, who said, you have to look for what that is.

31:27And then your job is to then take that kernel and try to run with it and expand it and try to see how I can take that idea, that concept, and then make it into something viable. And it doesn't always work, but it's about then having that understanding of what it is, not just sort of like going through the motion of like stom -stomp -stomp through it. Like that's not gonna work. And so I've seen people who try that and their career doesn't go very far. You have to have some degree of faith that there's something there, and I'm gonna try to do the best I can to make that part. How would I productize that idea?

32:13How would I make that viable from a business point of view or whatever the different constraints are? Awesome. So the advice there is focus on the parts you agree with and think about how you can find out if it's actually right or not. Agree with or even just like, you mean not even agree, but like what is the core of like what that person is thinking is the big benefit or good guy or what is the thinking vector that they're on that's causing them to want to go in this direction? Thinking vector, love that term. Along the same lines, another principle that I love is leaders are right a lot. And I feel like this is a term that almost goes unsaid, like almost can't say this in a lot of companies.

32:55Yeah. And I'm curious just the origin of why that became an important principle and then how it's implemented at Amazon. Yeah, so going back to this last discussion. So one, you know, one fallacy we should all acknowledge is that when you're making these decisions, you know, you can kind of, and you're trying to use data to make decisions, like you can make the data kind of look however you want it to look to sort of try to meet your decision. You know, if I'm looking at some issue, I've got some big data set. I can come up with, you know, ways of looking at a data set to support this idea and ways of looking at a data set to, you know, not support it.

33:33So the data doesn't, you know, rarely makes the decision for you. What is happening is that a lot of judgment and interpretation of the data, weighing that weighing various factors to then come to a decision. And that is sort of the right a lot part. The right a lot part comes from, you know, having experience, you know, having what we call sort of sound judgment, which generally come, you know, some people maybe are born with this, not a lot of them, mostly they get it through experience. A lot of experience is actually about being wrong, by the way, about making mistakes. And by having looked at a lot of problems, made decisions or observed others making decisions being a student of that, and then using that to understand that how to weight different information when making a decision.

34:27So right a lot is that you're good at that and that then it proves and that generally speaking, people want to follow someone who ends up by and large like, you know, going in the right direction, right? You're the leader of a team, the team is, you know, petitioning you on multiple sides. And if you keep kind of going off in some direction where most of the team is scratching their heads saying, like, I don't think that that was the right decision, you're probably not going to go very, they're not going to want to follow you very far and you're probably not going to go very far. So this is something that you develop through, you know, experience.

35:02And I'd say from having the opportunities to, you know, observe and work for others that are good at this. I love that it's a lot. I like that it's not just leaders are right. It's like right a lot. Yeah, there's still be, you know, no one is right every time. That's, that is totally unrealistic. Let's talk about the Taylor concept of your book and that's a word I've never used, but I think it's appropriate, which is working backwards. First of all, just what does it actually means to work backwards versus working forwards? The title of the book comes from two things. One is, you know, one of the leadership principles, which is that, you know, customer obsession and the principal states, something along the lines of that, you know, great leaders start, you know, start with customers needs and work backwards from there to sort of, you know, meet those needs or solve them.

35:58And then also because we created a process in this, in this window was talking about earlier, the 2004 to 2007 window, we created this process for new product innovation called the working backwards PR FAQ process. And they both refer to the same idea, which is that as you're guiding, you know, star or the point from which you're going to start is what are the customers' problems or what are the customers' needs and then figure out, okay, well, what would be the solution to that? What are potential solutions to that? And to do those things, to starting with, without the constraints of my financial constraints, my resource constraints, my legal constraints, my engineering constraints, whatever all those constraints may be, because the problem is what most of us do is we start with those constraints and work forward from there.

37:03Or we start with like things like, I got to increase revenue. How do I increase revenue? I need to increase active customers. How do I increase active customers? For customer oriented behavior, we tend to start with those things, which may often lead you in the wrong direction. Whereas we had, as Jeff would say, we took it as an article of faith. If we served customers well, if we prioritized customers and delivered for them, we took it as an article of faith that then things like sales, things like that, things like revenue, and active customers, and things like the share price and free cash flow would follow.

37:42And so this is important because there's no, I still can't give you objective proof that that is true, I don't know who could. And so it was saying, this is an article of faith that if we do that, we think those other things will work out. So therefore, when we're making a decision, thinking about a problem, we're gonna start with what's best for the customer and then come backward from there. And then in that coming backward process, we're gonna have to figure out, well, to do that, gee, I'm gonna have to solve this engineering problem or I'm gonna have to figure out how to make this thing cost less or make this thing faster or solve some one or more problems.

38:26And that's the backwards product. That informs like, what's the work you have to do to then create this new solution for customers? Awesome. So just to summarize, you start with, what are the customers' needs and problems? And I think a big part of Amazon to purchase, what are the lasting problems they'll always have, which is I think it's like lower prices, faster, shipping and all those things. And then think with no constraints. When you work with companies to implement this idea of working backwards, is it always what is the customer problem in need versus revenue or growth or something like that?

39:01Is there other examples of where you work backwards from and different sorts of companies? Well, the working backwards part is strictly about the customer's needs. We don't wanna work backwards from revenue. I guess there is, we didn't really use this term for sort of other things like cost structure. Cost structure was actually a part of working backwards from the customer that if we had a low cost structure, we could afford to give customers lower prices. Therefore, let's go ahead and have a low cost structure. Because in itself, driving out cost, doing things more efficiently doesn't inherently benefit customers because you could just use to take more profit.

39:48It only does if you decide that I'm doing so, I'm going to lower my prices to customers or provide some other benefits. So, we used it in this method of like, I'm starting from the customer and then very specifically we used it in this method of new products and features that I'm gonna go build on behalf of customers. Awesome. This episode is brought to you by Wix Studio. Your agency has just landed a dream client. You already have big ideas for the website. But do you have the tools to bring your ambitious vision to life? Let me tell you about Wix Studio, the new platform that lets agencies deliver exceptional client sites with maximum efficiency.

40:30How? First, let's talk about advanced design capabilities. With Wix Studio, you can build unique layouts with a revolutionary grid experience and watch as elements scale proportionately by default. No code animation at sparks of delight while adding custom CSS gives total design control. Bring ambitious client projects to life with any industry with a fully integrated suite of business solutions, from e -commerce to events, bookings and more and extend the capabilities even further with hundreds of APIs and integrations. You know what else? The workflow just makes sense. There's the built -in AI tools, the on -canvas collaborating, a centralized workspace, the reuse of assets across sites, the seamless client handover, and that's not all.

41:12Find out more at wix .com slash studio. Okay, so then when you go work with a company to implement this idea of working backwards, what are the very tactical things that you do to help them here? You know, PR FAQ is a part of that sort of chat about how to actually implement that. What are the steps to shift to working backwards? Yeah, so the first shift is to take this, you know, so that's just a concept, right, working backwards. Well, how do I turn that concept into a scalable, repeatable process? And that's exactly where Jeff's mind went. And eventually, without getting into the origin story, we came up with this process called the PR FAQ process.

41:48So what it means is that whenever we're divided, devising a new product or feature, we're gonna start by writing a press release describing the feature and describing it in a way that speaks to the customer into some degree, you know, the external press and world where the idea is in my description of this, it better jump off the page or something like, wow, you know, I really, you know, as a customer, I will really need this. And so what I work first is to say, okay, for your product development process, let's start by, you know, using this method to, as the method to decide, what am I gonna go build and know, by the way, to use it as a method to sort between a lot of different choices of what you might build.

42:37You know, in summary, the way that process works is that you're in the PR, you're going to describe very carefully and clearly like, who's the customer, what's their problem and what's the solution that you're planning to build. That sounds really simple and easy, but if you, it's actually very hard to do, to do that well, to crisply and clearly define those, the first two things are the things that are hard to define, like, who's the customer, like anyone says, like, all restaurants are my customer, okay, well, that's a mistake now. I mean, like, which kinds of restaurants are your customers in what kinds of cities, in what kinds of, formats, et cetera, et cetera.

43:19And then what is the specific problem you were solving? And ideally, you would some way have like, want to find that problem, or there's some data or customer insights that have led you to understand that problem to know that it is a meaningful and big problem. Ideally, a problem that people would, you know, pay money if you could solve that problem for it, because you can just look at the economics of that problem and, you know, if instead they use your solution, this would be beneficial to them. So, I worked, if have them first implement this PRFAKU process, is the first step. And then the next step really is to go from there to say, okay, writing PRFAKU is one thing, well, how do I actually use them?

44:02How do you actually develop them? Because there's this iterative nature to writing PRFAKU's where it's sort of a concentric circle review, like you start off small, like with one author, and with low fidelity writing these things. And then you start to share them with, you know, a small group and get feedback and improve it, a wider group, get feedback and improve it, and onward and onward until, you know, depending on the size and scale of your company, you get up to the CEO as a way to strengthen and improve and really codify this idea and determine whether it's a great idea or not. So I help them understand, like, how does that work?

44:42How do you do this iterative process? And then once you've done that, then what do I do with these PRFAKU's, like once I got them, how do I then think about that with respect to my roadmap? Awesome, okay, that was an awesome overview. I'm gonna fire up a couple questions around the first part. You still would suggest people do it as a press release? Feels like press releases aren't a thing anymore. Do you ever suggest people do it as a tweet or as TikTok video or a blog post? Good question. So the first thing is it's not a real press release, okay? We could change the nature of it, and if instead we wanted to call it the customer problem solution statement, right?

45:23We could just change it to that because there really are, you know, you know, three money paragraphs in this, this is, first of all, yeah, it's not meant to be a real press release. So don't use the language you would use if you were sending an actual press release. This is like an internal document, okay? So that's the first thing. The second thing is the heart of it really is that, like first paragraph, it's a short description, that second paragraph that's the problem statement, and that third paragraph that's solution statement. If you wanted to like ditch the rest of it and the artifacts of the press release, you could, I think there are other benefits to it.

45:58Like the headline is this headline, like long and drawn out, and like I can't even tell what the heck this thing is from this reading this headline, that's generally a bad, if you used a tweet, that wouldn't work very well. The date is also a meaningful thing. When you write the press release, the date is meant to be a hypothetical timing on which you're envisioning, you know, launching this thing, which tells the reader something, are you thinking that this is something that's so simple and easy, we're gonna launch it next month or so complex that we're gonna launch it in a year from now? So there are some other directional cues within it.

46:33Like I said, with everything, these are tools that people can use and maybe end, you know, I'm sure that companies will find other ways to improve upon these tools, but if you don't use those parts of them correctly, you're kind of missing out on like, what's the main benefit of you're getting out of this? And do you try to write it in a way that would be announced like a press release feel or is it mostly just like, who is the customer? What is, like do you try to pitch it as a part of this experience? So you try to write it in that way, but the one thing is you don't wanna use like hyperbole and it would be like very factual like with like, you know, numbers, like data rich document too.

47:16So again, not like a real press release, a lot of like internal confidential data would be like in this press release, kind of. So it's a tool that's, you know, has a very specific, you know, use to it. Is there template that we can point people to in the show notes to help them craft this? I think there's a version of your book maybe, but is there some online that we can point people to? Yeah, so we have a website related to the book, which is www .workingbackwards .com and there's a resources section within there and you'll find a template. Amazing. Okay, and then the concentric circle piece.

47:55So the idea there is basically get feedback from an increasingly larger swath of the company and it sounds like a big part of that is also get buy -in as you go along the way. Yes and no. So first of all, there are some things where you may write it and you the author will, you know, if we were in the old world, would like take the piece of paper, crumpled up and throw in the trash can, which is in your own, you've realized, now that I put this down on paper and read it, this is not actually that good of an idea, I'm gonna try something else. And by the same token, you may then have written one, you think this is a pretty good idea and you show your up here or your manager and they give you feedback that makes you want to then ball it up and throw it into the trash can.

48:38So part of this concentric circle thing is not just that everyone you write lives on and gets all the way to the CEO, like there are no stats on this, but let's just say in some virtual world, some imaginary world where yes, all these things, you're a product manager and you've got a director or product manager report to who reports to some senior vice president of division reports to a CEO. Well, if you truly run this out and you write 100 PR FAQs in a year, maybe 20 of those make it their way to the CEO, right? The point is not every single one of them is destined to go that far. They're the numbers get narrower and this sort of leads me down the concept of what you're really trying to create is a product funnel, not a product tunnel.

49:29And with a funnel, meaning lots of things at the top, fewer things at the bottom, the tunnel means that everything that comes in is also going to come out the other side and the problem with that method is the means that you have no consider, you're not actually having a method of like consideration and comparing it against other things that you might build or how you deploy what are frankly, in most companies, you're most precious resources, which is your engineering team, you should be looking at various choices, you should think of yourself honestly as like a venture capitalist. They don't fund every company that they meet with, they actually fund a very, very low percentage of them.

50:07And at Amazon, we had lots and lots of PR FAQs, they were a great idea, but we didn't ship them because we had other ones that were just a better idea, which had bigger potential impact. So you want that, you want to create this like, corpus of ideas that are well thought out and select the best ones. It feels like a lot of these processes are basically just waste a stop, stupid shit from happening. I think the narrative is a good example where you have to like expose your thinking deeply. This is a great example of that. Yeah, and it's also, I would say an example, where this is a process to prevent the other process, which is the product development process from becoming the thing, right?

50:51Where you just get locked in on, what are we doing in this sprint, what are we trying to get done and like focused on shipping stuff? What I recommend is you try to break that into two different processes. One is the process of deciding like what you should go build, and that's what the PRFAQ is designed for. And then once you've decided that, then yes, by all means, you saw that good thinking trait. And now how can I ship it efficiently and effectively with you know, few to no bugs? I was just reading this Harvard Business Review article. I think that's called the thinking to doing gap where a lot of companies just spend a lot of time talking about ideas and solutions and not actually doing anything.

51:33And so I'm curious how you try to avoid that at Amazon considering there's this period of just like what's exploring, exploring, exploring, refine. There's a couple of ways. And of course, I'm somewhat having to imagine, you know, what are the problems in such companies where that's going on. So one such version of this problem is what I'd call the big idea that's not fleshed out problem. So I'm sure that every single person listening to this podcast has either themselves done this or have witnessed others in their company who come up with a concept of like, oh, I think if we built this, boy, that would really solve things or that would really work well or that would really grow things.

52:17And it may sound good to everyone. It may sound good to you to everyone and then maybe you start then working on building it. But the reality is that actually once you've spent some time looking at that idea more deeply, you then start to identify several roadblocks or maybe a fatal flaw with this idea. And in fact, no, you shouldn't waste any of your time going into building it, I think, because it has a fatal flaw. So one problem is that companies get stuck, I think, where they never actually go do that documentation. And so it's a debate and discussion about concepts that aren't really well fleshed out.

52:58And so people's ability to actually evaluate them in any realistic way is they don't have a good way. And so in those situations, what gets done is probably more of a function of politics or will or a culture of a completely top down. I think the other way is where they're debating and discussing things that they just don't have good methods where then they can take things and then go build them, meaning they probably don't have the right org structure or processes in place to then go, take the good idea, assign it to someone who will own it, go look at it, and after they have owned it and gone and looked at it, if it works, then they and their team can go actually build it.

53:50What I always found is once I became more senior in the company and my role became bigger and bigger is that when something came up, some idea that didn't neatly fit within my org structure, I couldn't necessarily delegate it to someone that there were only two things I could possibly do which is just sort of set it aside altogether because otherwise there's just be real distraction to people. Or I had to decide this was a compelling enough idea that we were going to take a resource, could be one person, could be a whole team doesn't, depending on the idea. And I have to assign that resource to actually go, look at this and work at this, otherwise it will never happen.

54:31I've been through those many times. Okay, so there's two more concepts I want to try to touch on before we wrap up. The next one is the idea of input and output metrics. There's something that Airbnb we super implemented became a very defective way of thinking. And actually there's a lot of Amazonians that ended up at Airbnb, a lot of leaderships. So there's a lot of stuff that we ended up doing like the memos. And so on the input, output metrics, to just describe what that is and why that's so important why people kind of think about metrics in the wrong way often. Yeah, so the origin of this one really was, again, kind of in our early years at Amazon, 99, 2000, 2001.

55:11We would, we're a public company then, we were growing, but then growth started to like, it wasn't just all up and to the right and like, woohoo. Every company is going to hit a wall eventually and it's not going to be, you know, if you're so lucky to have even been in a company where it's just like going up into the right with no gravity, good for you because most people don't have never experienced that. But most of you experience the reality is there's a lot of gravity pulling against your revenue numbers and you've put a plan out there and you wanted to grow 15 % or 20 % or 75 whatever it was.

55:48And now you'll look like you're not going to hit that number this quarter. And so what ensues then is they, we're not going to hit our number. What should we do about that to hit our number and this often happens with, well, you know, there's like a month or month and a half of life in the quarter? And then we would run around like, chickens with our heads cut off and come up with a bunch of ideas. They tended to be like promotional in nature and tended to be like price reduction in nature or like we'll send this extra email or extra ad or whatever it might be. Another prank maybe. Right. And the reality is we did that, you know, we went through that enough time, several quarters and we started to realize, huh, these fire drills don't really work.

56:30We didn't really like get meaningful progress against the number with these like last minute things we decided to go do. And oh, by the way, they were kind of a big distraction. They probably, if they did work at all, they pulled revenue that might have just gotten in the next month or next quarter into this one. So it wasn't really kind of a zero sum game there. And we realized, you know, we're not really actually working on things that matter to customers that are gonna like move the needle over the long term. And this is about the same time when Jeff and the S team were reading the book, Good to Great.

57:06And I would, you know, you have to ask Jeff what it is, but if you ask me, I think that this was the single most influential and effective management book of our company. Because what it caused Jeff to do, and I won't describe what, you know, most of you probably know what it is. If you don't know what it is, go read Good to Great. It is, my opinion, has the best, you know, most important management book you'll read. Because what it did is to help us codify our growth flywheel. Meaning what are the inputs that if we improve these things, which in our case was, you know, how do we have broad selection?

57:47How do we have a great customer experience where great customers experiences in retail things? Like how easy was it to find what you wanted to buy? How easy was it to buy it and how fast did it get to you? Where the prices low, do we have lots of merchants on our platform? And by the way, could we drive out costs? So we identified these things on our flywheel. And this identification of these things was such a critical moment for the company because then it realized, okay, well, what we need to do is spend our time focusing on like how do I measure each one of those things and then how do I improve each one of those things?

58:23So it shifted our focus away from like this short term thinking of pushing the revenue number up to this longer term thinking that if we just improve these things, you know, whether it's, there's no day that people will wake up, 10 years, 20 years, 30 years from now and say, all else equal, I'd rather shop at a store with fewer items than more items or a store with higher prices than low prices or a store where things get to me more slowly versus, you know, more quickly. So if we can just improve these things, you know, this is our path to winning. So those were all inputs to the customer experience and so we then figured out ways to measure them creating a set of input metrics.

59:03And so then when we would develop our operating plans and review our business each week and set our goals, we were hyper focused on those inputs and input metrics. As a simple example, there was one tool that Jeff and the leadership team, the S team used called S team goals, which are effectively a list of what they would harvest would be like here are the most important goals for the company that I've harvested from all of our operating plans. And I came up exactly what you're something around like Tucson, 2007, 2008. They looked at that list, which is about 500 items long, by the way, and they counted it up.

59:42And of that list, only 10 of them actually had a financial metric in it like revenue or free cash flow or like gross profit. These other things were generally speaking all, you know, one of those inputs, like I mentioned to you about low prices in selection and speed of the customer experience. So yes, the point was, again, it's this other article was, so we took it as an article of faith that if we can just improve these inputs, the outputs will take care of themselves. The inputs are the things that drive the outputs, which are our revenue, customer activity free cash flow. And so one of Amazon's, you know, it's not really a secret, but one of Amazon's great strengths was there was focus on those things and make, you know, just continuous process continues to improve on each one of them and measure them, you know, rigorously.

1:00:37The flywheel reminded me that was another, it feels like that's another concept Amazon perliferated through all of companies is everyone's trying to create their own little flywheel. And imagine everyone has that image of the Amazon flywheel in their head with the little orange circle in the center and the black arrows. On the topic of input metrics, just briefly, what is an example of a good input metric? Because I'm imagining people are listening like, oh, shit, I got to think about my metrics as input input now. What's a sign? It's a good input metric. Sign that's a good input metric is the first of all map your end and customer experience like I know what did Airbnb, but okay, step one is that they clicked on some ad somewhere and showed up in the website or the app.

1:01:16I'm in the app, now you're looking at this first screen. Well, the first thing what they're doing is they're browsing and they're into or they're searching. Okay, how are we measuring the speed, quality and ease of that browsing and searching? Now they've got onto a different detail page for an individual property. How are we measuring the speed, easing quality of the different actions they may take like, you know, reserve? Forgive me if I get any of my terminology wrong. I'm not an Airbnb. You are, but it doesn't matter. It's close enough. And then, you know, so then you've reserved. Now you have interactions with a property owner.

1:01:54How do I measure the quality of those? Am I going to, you know, how many messages go back and forth? Is a lot of messages a good thing? Is that a bad thing? At first, you may not know the answer to that question. Same thing. Every step of the way, then there's the actual rental experience. How do I instrument and measure every part of the customer experience? So you know, it's an input metric. If it is measuring something with respect to the customer experience, which ones are the right metrics, which ones are the most causal to the outputs? I couldn't begin to tell you this is actually, this is actually where you're getting paid for.

1:02:35If you are, you're being pleased to figure that out. And then you basically through an iterative process of measuring observing, improving and looking at what the effect is on your outputs. So again, we didn't, we didn't really create this concept. This is a concept from six sigma, which is using, you know, DMAIC, which is I have a process. There's an output of this process, but the inputs are a black box to me. So how do I understand those inputs? Well, DMAIC stands for define, oh boy, define, measure, the A is going to come back to me in a minute. Improve and control. And I'm going to have to, oh gosh, the A is lost.

1:03:23I've lost it for a second here. But I'm looking at, we can define, measure, analyze, improve. Analyze, thank you. Yeah, analyze. So, you know, we just use that process, which was, and by the way, the way we think about it first is like, well, you need to throw a lot of things at the wall. You don't really know which of these things are going to be the most causal. So you know you're doing input metrics. If it is, do you control it? Meaning can you apply resources to make this thing better or worse? Is it, does it touch customers? It doesn't always have to touch customers, but if it is affecting the customer experience, it's almost certainly is an input.

1:04:05And then, you know, which ways you're going to measure that input, you know, you need to try, you know, more than one way. Because like again, we tell a story in the book about one of our most important input metrics, which was how much selection do we have? And we were actually measuring that right for several years. We had to refine that measurement. So I don't know if you saw this, but I asked on Twitter what questions I should ask you and tell people you were coming on. And something that came up a bunch is with working backwards. Obviously, some products Amazon has launched have not worked out.

1:04:34Say the fire phone is a classic example. What have you learned from that process of just like, okay, here's signs. Maybe this won't work out. Also knowing many things are not going to work out. There's no way to really know. Yes. So the one important thing to share is that all these tools that are describing this book that Amazon is using, whether it's using documents and meetings with a pure FAQ process or input metrics is that none of these things give you the answer. They are tools to help you make decisions. So sometimes you're going to make the wrong decision. Fire phone is a great example.

1:05:10It comes up often people ask, well, you know, if you've got this great peer FAQ process, like how did you get fire phone? So I was sort of tangential to the fire phone team and I work on it closely. And you know, different people have different opinions. So I'll just share share my opinion, which is that if you think about, again, how does the peer FAQ process works? Well, there's a customer problem. Well, what was the problem that the fire phone was seeking to solve for customers? I would argue this is a case where we made the mistake of what we had a technology solution in mind, which was 3D effects.

1:05:50Then we took that solution and were then in search of a problem. I don't think it solved any meaningful problems for customers. And Canada, like I was, you know, we had to build a version of the music application and the prime video application for this phone. And I don't, I couldn't figure out like how this 3D part would make it, you know, better for the customers to discover, watch or, you know, play back any of these, these media. Maybe there were games that could have been, you know, great solution. I don't know. But I think, you know, the simplest place to go when you see a failed product is to ask yourself what problem to dissolve.

1:06:31And I could, you know, I could get into all kinds of other examples outside of Amazon too. But, you know, nine times out of 10, I think, you know, that's, that's where it, if it wasn't poor execution, if the product was executed correctly, what was wrong with the concept of the product? Imagine there was a lot of a disagreeing committing on that concentric circle process. Is there anything that you've found of just like the number of disagreement and commits in this process of PRFICU filtering out? I don't know. That tells you like, maybe this is not a good idea. Not necessarily. So I'll tell you partly also why the fire phone happened was, you know, for my point of view, I think, you know, I think that we had had a number of successful products where in some cases there were a lot of people who doubted whether it worked, a lot of people doubted whether the Kindle inside Amazon doubted that the Kindle was going to be a good idea.

1:07:24I remember contentious, you know, board meetings on this topic. So even, you know, even within a company that was considered innovative, you would have a lot of people, you know, who would doubt things. I can tell you that for years is working on prime video. I would tell people about, you know, what our ambition was, you know, watching our TV set and we're going to have our own motion, you know, studio. They'll make our own movies and TV shows. And they would laugh at me. That was crazy. So that's not necessarily the sign of whether, you know, the product is right or wrong. And so that's a problem, actually, that makes it harder to know.

1:08:02And I think something Amazon's incredibly good at is being okay with a lot of failures. And I think that's part of the reason there's been so much innovation. Is that, is that true? I'd say it's partially true. I mean, again, it's hard for me to do a compare and contrast with other companies because, you know, but I can tell you, did we have a lot of things that we launched that failed? Yes. Some of our, some of them are, you know, very, you know, public and obvious. I'll give you one that people don't really realize it's something called we had a feature in the early 2000s called slots. And what it was was basically third parties could, you know, bid on different search terms and put like a little ad in there.

1:08:42Well, obviously that works now on Amazon, but it didn't work then and didn't work then because we simply didn't have the scale. And that we had that Amazon has today. So a lot of times I'm proud of this idea. Perfectly good idea. You just have the wrong time or the technology isn't there. I mean, Jeff, Jeff wrote about a product, you know, there was a puck that set on your on the, in your kitchen that you would talk to and ask it for things and, you know, we could shop from it. You wrote about that in 2004. Well, the technology wasn't there to be able to create that little puck, which one day would become echo was sort of a decade away.

1:09:21But we had a lot of things we launched that failed. We were, we were not afraid to take what we considered a well calculated risk. I think many, many companies are, are less willing to do so, less committed to product innovation and really do not want that fear failure. Did they do fear failure and they're really focused on sort of their nerd term, financial goals. It's no, it's sort of not their fault. It's kind of the way, you know, the public company and Wall Street sort of interact with each other sort of creates this dynamic. Just to pull in that thread a little bit more, it feels like a lot of companies talk about we're okay failing.

1:10:00We're okay launching things that don't work. But then in practice, their performance reviews impacted. Teams get shut down by just get pulled. There's something that you recommend for two companies that want to actually improve in this, like what could they actually change in actually do this well? Yeah, I just spoke with actually a senior executive at a well -known Silicon Valley company about this topic of the day. And so, well, what is it? What is it we had structurally to Amazon, especially from a people point of view that would enable or encourage people to take these risks? Because yes, in a lot of companies, if you go work on the project that fails, like then your career is in the garbage can and or your compensation system, like you're going to lose out on that bonus.

1:10:48So there were two things. One was our compensation system. So there were no performance bonuses. So if I was running the book business and I had a killer year from a financial point of view, there was no like extra kicker for me. And if I were in the book business and had a bad year, there was no financial penalty for me either because our compensation was based on the stock price. So we all had an incentive to do was write for the company, frankly, over a long term because trying to weigh off of short term fluctuations off Wall Street is kind of a losing proposition, which meant that therefore if I am, you know, so I mean, because I have that situation, I moved off of working on our largest PNL and then the book business and music and video business.

1:11:34Now I'm going to go work on digital media. There is zero business there. This might not work. Well, my, that didn't, my compensation didn't change as a result of that. Didn't change, you know, one way or the other. We tended to also have a performance management system that then would change compensation based on evaluating what did you actually deliver kind of more in an input method? We cared about the outputs too, but the, but you know, just there, there are plenty of people that could be in a business that's like up into the right, but there's nothing to do with them. And so we tried to focus more on what did you actually build and contribute, you know, ways you improve selection or lower prices or whatever that might be.

1:12:17So those two things about the compensation, you know, mattered a lot. And then the second thing was having, you know, a CEO who was really committed to it and it wasn't something that they delegated to someone else. So, Sophie Bacall, I think writes about this in his book, Loon Shots, where part of the conditions that are necessary for innovation to occur are that you actually create different structures of decision making, of approvals, of all kinds of things. If you create some, you know, team that's going to go build something new and innovative because the most of the structures inside a big company are, are kind of designed to crush and impede a small, innovative team that's trying to go build something new.

1:13:03They need speed, but you know, some get approval here, approval there. It's going to like get in their way. And one of the ways we solved that two ways. One was when we went to go build, you know, digital media in AWS, we put like two of our smartest, you know, leaders in the company on those things, Steve Kessel and Andy Jassy. And number two, they were meeting with Jeff regularly. Jeff was deeply engaged with them, reviewing like, what are we going to go build? Part of the decision to decide where we're going to go build. And so he could then also between their seniority and of course, him being CEO, they could run interference on these sorts of things too.

1:13:39So even if you want to have innovation, even if you really, you know, you really do crave it, you're willing to take the risk. If you don't set up the organization in the right way, like you're just not going to get it. Amazing. I'm glad we didn't go into that. I wasn't planning to talk about that. And I'm glad we did. Final topic, this concept of barraisers, it feels like it's been such a core way of allowing Amazon to scale successfully. And I think there's something a lot of people can implement. It's like a very one off thing. You could just implement a your company. You just talk about what this idea of a barraiser is in the hiring process.

1:14:14And then what people can do if they wanted to add this to their hiring process. So the barraiser hiring process is a process is actually one of the first ones that was established. The book was pretty early in the company's history back in 1999. And we created it for a simple reason to quote one senior leader at Amazon. We had new people hiring new people hiring new people. We were our hyper growth phase. Okay. You know, we were the company was only, you know, what three, four years old. And we were growing like a weed at that point. So what this started off actually in our tech work and what, you know, our senior leaders in tech realized my gosh, we, you know, we hire some new engineering leader.

1:14:57And the next thing is that their job is to go hire like, you know, the senior managers and they'll go hire like managers. And all these people have been here for, you know, like a week. So we don't really even know our company yet. They don't know our culture yet. They don't know our standards yet. So what information are they using to make these hires? And, you know, what differentiates they were using is obviously they're just using their own personal judgment and their personal judgment combined with whatever criteria they used at prior companies that they work for. So let's say they came over from Microsoft.

1:15:32They, you know, if Microsoft had some methodology or criteria, they probably would just apply that. Well, is that methodology or criteria relevant to our company? Because every company has a different culture and, you know, I'm here to tell you that if someone's been a super successful vice president, Microsoft does not mean they can be a super successful at Amazon or at Google or Facebook. Sometimes they can, but these companies are very different. They all do work very differently. The way leadership happens and decisions are made are very different. So how do we fix this problem other than letting it run rampant and basically are high or a bunch of people who are, we don't know if they fit our culture and we don't know if they fit our high standards we have for what we expect of, you know, engineering leaders or engineers.

1:16:21So they created this bar raise or process, which by the way, they borrowed from Microsoft, which had a process called as appropriate. And the concept was that on every interview loop, there's one person who is not the hiring manager who doesn't report to the hiring manager is not the recruiting manager. They're, you know, a line, they're in the business like they're a software developed manager or they're a marketing manager and they are on the interview loop and they're a bar razor, which means when we get to the debrief meeting, they will run that meeting, not the hiring manager, not the recruiter.

1:16:54They will run the meeting. And it also means that they technically have veto power over the hiring manager, which by the way, a good bar razor never uses or I never saw a bar razor use. I was a bar razor. And in my 15 years of Amazon, I never used it, never saw it used. And then finally, which actually was not true in 1999, but later became true. It was once we established our leadership principles. We created a set of objective criteria that would be used and an interview methodology would be used in every interview, which was the definition criteria of your leadership principles and the methodology would be behavioral based interviewing.

1:17:33So this bar razor was basically would be a subject matter expert on how this process worked. It conducted debrief to make sure that we were actually adhering to the process, that people were sticking to the objective criteria, other than saying, I don't think we should hire this person because I don't know, they don't seem to want to work here enough. That might maybe that's a valid reason, but it's actually not part of our objective criteria. And so the bar razor was there to act as a balance also on the urgency bias that every hiring manager has, which is like, I got to fill these roles. And but rather than filling them with the next warm body they find, make sure they fill them with people who actually meet our standards, fit our culture and meet our standards for, you know, functional excellence to Citricle process.

1:18:22Two questions, longings lines. One is who is the final decision in hiring? Is it in the hiring manager? And this is just a spice from the bar razor. Yeah. So this often gets confused. The decision maker is the hiring manager. The whole interview loop and the bar is are actually just there to help the buyer hire manager make the right decision. Now oftentimes the hire manager could, you know, feel like this is actually a bureaucratic process and a group of people that I have to sell. And they're just in my way between me and hiring this person, which is kind of a natural feeling to have. But one of the feedback I would always give managers who were new to this is like, no, no, no, that's not the way to think about it.

1:18:59Think about these people are helping you because the amount of time you're going to put into the hiring process may seem like a lot. But if you hire the wrong person boy, that amount of time you're going to have to deal with managing that person. That's going to be a lot more impact on the team, impact on you. So making a great decision here is important. They're here to help you. So yes, the final decision was with the hiring manager. Technically speaking, the bar is your could block them from a decision to hire someone. But they would a while well done. They would help the hire manager see the reasons not the hire the person through kind of a secratic method and how they would guide the discussion.

1:19:39And then when you're choosing a bar, is there any suggestions you have of who to choose and how often you pull them into these things? Because it could also be a huge time suck. It is a huge time suck. And it sometimes could be up to 10 hours of my week spent. Wow. Actually, the R -Raser. The selection process is, you know, you start with as a company, I would recommend. If you wanted to do this, you pick out apartment, pilot it with, pick some, pick people who are a, care a lot about your hiring process, be, appear to be good interviewers and see, seem to have high standards. It's also a great role for people who are earlier in their career, like giving them this additional sort of leadership opportunity.

1:20:21It's a great way to grow and develop leaders, by the way, because this added responsibility is a great way for them to start testing out leadership. And you have to train them properly and you have to have dedication to the process. But I generally would try to pilot it with them one group at first. One last question before we get to our very exciting lightning round. Many people are listening to this. They're considering implementing some of these things, trying to figure out how to actually make these real. If someone were trying to move along the path of becoming more Amazonian, which of these elements and processes do you think often has the most impact?

1:21:02And or is there something fundamental that needs to change to allow for some change like this to happen to a company in your experience? Good question. And the first thing I'd say is one thing to be careful of is a lot of times when I'm talking to a company about these processes, they say, well, does this mean we need to turn into Amazon? And first thing I tell them is, well, first of all, I couldn't turn you into Amazon if I wanted to, because you have your own culture. And secondly, no, that's not the idea for you to try to become Amazon. The purpose is to sort of look at these processes in best practices they have and consider adopting parts or all of them into your organization to improve the same, every company of a certain scale has these same processes.

1:21:46So this is just a different way to do them. So you should have scalable repeatable processes for each one of these. You know, pick one, here's one choice of ways to do these things. The other piece of advice I give is that a lot of these changes are relatively profound. They really require buy -in all the way up to the CEO, because you're really going to change the way you do product development, or if you're really going to change the way you do hiring, that probably requires buy -in to the CEO. And so I would, you know, seek to get that probably before I would move too fast. Some of these things, though, can be piloted in your own little group, like your one little product development group.

1:22:25You want to decide you want to start writing PR FAQs. You probably can decide to do that, but again, you know, try to check with your leadership. The other thing I would just tell you is that for any of these processes, these in our book or any book, implementing a new process is not, you know, easy. And if you kind of go into it lightly and kind of dip your toes into it and sort of try it out, it's probably not going to work for you. Because it'll be hard first, and it requires some level of commitment to actually work through that hard part, and say like, no, I'm really committed to doing this.

1:22:58And it will take, you know, a few months for you to get good at it. So you have to have commitment and discipline to get through it. It's, you know, anyone can really do these things. It just requires commitment and discipline. And in our chat, we've basically just scratched the surface of a lot of these things. If people want to dig deeper, there's obviously your book, Working Backwards, which we'll link to in the show notes. And you also work with companies to implement a lot of these practices. You could just talk about what it is you can help folks with, and then how to potentially engage if they're interested.

1:23:27Sure, great. Yeah, Colin and I, one of the reasons we wrote this book was to, you know, pass on what we learned to the next generation of business leaders sort of at scale with a book, but also because we had a passion to work with companies, you know, directly one to one. And so we are advisors, consultants, call it what you will, but not traditional. Like we don't have a team of people working for us. We, each of us just work directly with the companies who engage us. And generally speaking, what we do is we are the right kind of company for us to work with. First of all, has to achieve a certain scale.

1:24:02Companies that are sort of in the product market fit phase, like they need to focus on getting product market fit. They probably don't really need to focus much on like how they put in scalable, durable processes. Like sure, some of these could definitely be helpful to you, even if you're in that phase. But really these, these are designed for like I, my company's kind of become complex now. I've got multiple product lines. It's well over 100 million annual run rate, growing fast, complex. So most of our clients are either large, well -passed series C, private companies or their public companies.

1:24:39And in most cases, a C level leader or the CEO themselves has read our book and recognizes that they have a lot of the same problems that we had at Amazon. And looks at these as useful solutions and wants us to help us to help them implement them. So we tend to usually first actually go in and do an assessment of how they do things today because to help people move from one place to another, we have to mention it where they are. And then we come up with a prioritized list along with that. The CEO and C level leaders of like what are the things that would be most useful? What are the symptoms and problems you're having and what are the sort of the root cause solutions that could be found in these processes.

1:25:21And then we sort of prioritize those and come up with a plan to work within the organization to help them implement those. And what's also sort of different is that we're very hands on working at all levels of the company. And we will, as we do it, we will, you know, be there in the meetings with the teams to help coach them and teach them along so that we make sure that it actually gets implemented properly. And then we will go into spec and they get the outcome they want sounds amazing. How would people engage with if they wanted to explore this? Simple way is you can just send an email. I'm Bill at working backwards .com and Colin is calling it working backwards .com.

1:25:57You can also just check out our website. www .workingbackers .com. We have some information there with a contact us form. Those would be the best ways. Okay, well with that we reached our very exciting letting round. I've got six questions for you. Are you ready? I'll try. Interestingly as I look through the list, many of them relate to using Amazon, which is pretty funny. The first is what are two or three books that you've recommended most to other people? So it's a in the management world. I'm not surprising you'll hear good to great. I'd say Drucker on management or Drucker the effective executive.

1:26:32And then the other one I'd say that's a little bit different as I'd recommend the steep jobs biography. I've never worked at Apple but looking at that arc, a lot of the way those things worked was not that different from what I experienced at Amazon. It's a good window into what it's like to be inside some tech company that goes through product innovation and big growth. On a personal basis, recent books would be 70's by Neil Stevenson is a favorite and gentlemen Moscow. Amazing. Can get them all on Amazon. Yes. Another Amazon related question potentially is do you have a favorite recent movie or TV show?

1:27:12Might be on prime might not. Yeah, my favorite recent movie is the latest June movie and I can't wait for the new one to come out. What does that come out? It seems like I've been waiting a long time. I think it's supposed to come out next month but I used to know this. I used to have to know the answer to this question but I don't anymore. But I anxiously wait the next one. I thought that last one was awesome. I even liked the original doing movie so I'm probably unusual that way. And I just watched along with my wife we just enjoyed watching the TV series Aspire Among Friends which was on MGM+.

1:27:49MGM+. I have not even heard of that. I had not actually heard of it either. You can basically go on to prime video and you can find this show and you can kind of subscribe to MGM+. Thank you, Prime. What is a favorite product you've recently discovered? Maybe you bought an Amazon. Maybe not. This one did not buy on Amazon and this one is most of you may not understand this one but I'm an avid cyclist. And I got myself a new set of wheels from my road bike this year. Actually my road bike and my gravel bike. It's the Zip 303 Firecrest. The latest model and these are just fantastic wheels that are light.

1:28:28They're sturdy. They absorb all the bumps well. I can use them on a road bike. I can use them on a gravel bike. Awesome wheels. Wow. That might be the most obscure random product that we've had yet. Recently we had a humidifier. So I like this collection of products. We're building here. Create a wish list and Amazon maybe. Do you have a favorite interview question that you really like to ask? Yeah. It's actually quite basic. It's, you know, tell me about their most significant professional accomplishment. And I was always clear for this. By this I mean not some award you won or some promotion you got.

1:29:07I mean something you built or some products, some process, some organization you built. Something like that. And I could basically then, you know, once they get into that example, ask a lot of, you know, probing and follow up questions. Like it fill an entire hour interview just sticking with this one example to really understand how, you know, how they using the star method. Which is to try to understand everything from, you know, the situation to the result and everything they did in between. Who they influenced, how they influenced what decisions they make, what roadblocks they encountered.

1:29:42If I just pull on that one string, I can learn a whole lot about it candidate. Next question. What's a favorite life motto that you often find yourself coming back to sharing with friends. Are you find useful? Well, one that I end up coming to, a lot professionally and to somewhat personally is this one called slow as smooth and smooth as fast. This is a, I believe the origins of this one are actually from the Marine Corps for the Scout snipers. So not trying to promote, you know, that particular craft, but the point of it is that, actually, and we did this all day Amazon, it really, oftentimes to really to go fast, you actually need to kind of go slow first and to be very clear on like what you're doing and where you want to go.

1:30:34And most people confuse, you know, speed with velocity. And the difference between the two is that, you know, speed, velocity has both speed and a vector to it, meaning you're some specific destination. And so I see a lot of people who are going very, very fast, but the destination isn't very clear. They haven't really sort of thought that out well. So slow as smooth, smooth as fast. There's a similar quote that I've always thought of of. You've got to go slow to go fast. And I always thought it was Stephen Covey, but I just Googled as you were chatting and someone named Peter Singh Singh in the book fifth discipline.

1:31:10It took me a while to, I always wanted to go fast first and not go slow first. So I'd say a lot of my personal development and growth and that's a big thing that I learned from Jeff at Amazon. Final question. I don't know if you'll have an answer to this, but is there a pro tip that you could suggest for using Amazon? Something that people may not know about how to get the most out of using Amazon .com. I, I, sorry, I have no, I have no secret insights. It is not like something like if you go on Monday mornings at this time, the prices are lower or something, I don't think stock is better. No, I know, I know of, I know of no such thing, which I think is great because it's built exactly as it should be for, for customers.

1:31:53I guess so. But I, you know, it used to be that you, you know, maybe in the days of the slower internet that you, I would tell you to go at, you know, non peak hours, but this isn't really an issue anymore. I would be, I'd be wild if that was still an issue. Bill, this was everything I hoped it would be. Thank you so much for being here. You already shared where folks can find you online. So I'll skip that question. So final question is, how can listeners be useful to you? You know, we're always looking for, you know, feedback. You can post a review on Amazon for our book. That's probably like the best way.

1:32:27You can fill out the contact us for a send us an email telling us what, what you found most useful in the book or what you, what you actually found that is missing. What would you like to learn more about if we were to say, write another book or write more, what would you like us to tell you about? Amazing. And that's Bill at working backwards .com if they have that feedback. Go by the book working backwards on Amazon and other places and working backwards .com to learn more. Bill, thank you again so much for being here. Thanks so much. I really enjoyed it. Me too. Bye everyone. Thank you so much for listening.

1:33:05If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify or your favorite podcast app. Also, please consider giving us a rating or a leaving review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lenny's Podcast .com. See you in the next episode.