Thinking in Bets by Annie Duke | with guest Jordan Staab

0:01Annie Duke:Welcome to Business Book Club, where top entrepreneurs break down the most powerful insights from the world's best business books. I'm your host, Sam Brown, and today we're diving into Thinking in Bets by Annie Duke.

0:17Annie Duke:Most founders treat business decisions like chess, assuming that if they make the right move, they'll get the right outcome. But business isn't chess, it's poker. You can make the perfect decision and still lose. You can make a terrible decision and get lucky. We need to know how to play the game. So what is thinking in bets? The author of today's book, Annie Duke, isn't just a world-class business consultant. She's also a World Series of Poker champion who won over$4 million in tournaments. She's mastered the art of making smart calls when you don't have all the facts. Joining me today is Jordan Stapp, the CEO of betterfinancers.org.

0:55Annie Duke:Jordan is a serial entrepreneur. He's helped raise over$200 million for VCs and led ventures worth over$30 million. What I love about Jordan is that he doesn't just read this theory. He uses this to project revenue, manage risk, and make smart decisions every single day. In this episode, he's going to teach you how to separate your results from your decision quality why saying I'm 100 % sure is actually destroying your strategy and how to build a truth-seeking inner circle. So let's get into it. Jordan, you chose Thinking of Bets as the most impactful book that you've ever read. Tell me why.

1:33Jordan Staab:I can't think of a book that represents how I got here more than this book. My business understanding and baptism by fire came from playing far too much poker in my college years and early 20s, continuing to be the most applicable business MBA that I could have ever gotten. When I found Thinking in Bets, it crystallized all of my thoughts around how I was taking the lessons from poker and applying them to the business world in just a much cleaner way than I was doing at the time. It allowed me to supercharge a lot of those skill sets and make them applicable for growing my successful businesses.

2:09Annie Duke:Let's dive in. So what have you got for us for lesson number one?

2:12Jordan Staab:Lesson one is separate outcome from decision. In business, there's always limited information. Everyone often jumps to, did we win or did we lose? And saying that that's the definition of success should really be the how you got to the decision. How are you thinking about things so that you can get consistency and start to develop a system of how to improve. And so in poker, you can make all the right decisions and then lose. Didn't mean you did anything wrong. It also doesn't mean you should have done anything different. Sometimes you just were unlucky. I think the big point of this lesson as well is start to think more about doing that consistent decision making process and less about what the result was.

2:53Annie Duke:How then do you take this lesson and try and apply it into your own life? Like you talked about decision making systems. What does that look like?

3:01Jordan Staab:It's constantly understanding that you don't have perfect information. And so if you wait for perfect information, you lose. So you just don't have enough time to obtain enough information. So building a good system and also fine tuning that system allows you to progress significantly faster. So if you're trying to wait for perfect information, you're too slow and moving quickly and getting a feedback loop going of like, OK, that wasn't a good idea. And this is why and improving and improving allows you to advance the idea much, much faster. And so part of weaponizing this lesson is also making sure that you understand that failure is a part of that feedback loop process and like how to learn quickly and adapt and keep moving.

3:47Jordan Staab:Everything we do is always walking through like, why are we doing it? What outcome do we expect to happen? And what is our confidence? Where do we think things can break and go? And then evaluating. And it's not necessarily evaluating, did we win or lose? It's were we correct in our original assumptions, how much of this was correct and can we start to get better and better at that first stage? And the better we are at that, the more efficiently we spend our time, which is always the battle as a startup.

4:17Annie Duke:On how efficiently you can spend your time to be able to make a decision, do you have a rule? If you have 50 % of the data, that's enough or 80 % of the data, that's enough to try and start making decisions and moving. How do you balance action versus confidence, I suppose, in your decision?

4:30Jordan Staab:We think in math problems and it depends. So if we're trying to buy traffic to one of our sites and we want to know if we've targeted the right audience and if some ad sets are starting to work, part of it is like, you'll never have perfect information on like, yes, this is the correct ad set. Like this is the right audience. And we know it's going to win from day one. When we start placing that bet, we're going to go spend$10 ,000 on this test. It's not to see if it works. It's to learn something deliberately in that equation. We are testing this ad set within these parameters and we want to see if this is correct.

5:07Jordan Staab:And so hope is not a strategy. We deliberately make a bet and we're trying to learn from it. If we are learning something, I'm like, oh, this doesn't work. This audience is not good. This ad set doesn't work well with this audience. That's learning. That's advancing the strategy. And we try to do that upfront as defined. These are the things that I think I could learn. I'm out of this or I'm not confident enough to solve the problem yet, but this variable, I can get to a 90 % confidence in that variable with this test. And so how much of that can we continue to clear up and get a very finite understanding of that math problem and that we're trying to solve?

5:44Jordan Staab:And in my head, I'm not sure if you guys have ever done the iPhone fingerprint thing where you have to like push your fingerprint down like in every possible direction and then you get a whole picture of your fingerprint. that's what we're doing is that we usually just have like the left side of the fingerprint and like we got to go try it a little bit on the right to get that side of the problem and like how can we get in that arena and learn that side of the fingerprint to start to understand and so going back to the title of the book thinking and bets you have a limited number of bets and limited number of dollars that you can spend on the test or hours in the day and that you can invest and so we have to be very deliberate about what is the most efficient way to obtain that new information to create that full fingerprint of that math problem we're solving.

6:25Jordan Staab:And you're just trying to increase your percentage confidence within a problem because 100 % is extremely rare. And the only thing you're 100 % on is that we will never be 100%. Let's move to lesson number two. Think in probabilities, not absolutes.

6:43Annie Duke:Okay, great. So we've just been talking about not thinking about the quality of our decision-making based on the outcome. So we shouldn't say things like that was bad choice because it failed. What I think you're saying now is we're moving beyond that to assigning probabilities to your decision-making process. Like if a strategy has a 90 % success rate, but you hit the unlucky 10%, the decision was good, but you just got unlucky. So within this, Annie talks about assigning probabilities before you kick off a project, not just when you're running the post-mortem after. I guess you've got a few different ways of how you do this internally.

7:17Jordan Staab:Like a really tangible exercise I do internally whenever we're doing projections and where are we going, which is just a difficult task, especially when you're having younger resources in your company build out year long projections. Like we just started this project, I'm guessing. Yeah, no, I get it. But we still need to do these because what gets measured gets managed and every other cliche there. But you have to have a direction that you're running down to do these projections. And what I always ask them is just do 80%. What do you think you can accomplish 80 % of the time? If I gave you 100 shots at this, do you think you could meet or exceed this 80 of those shots?

7:54Jordan Staab:For some reason, that simplifies the project in their head to think more in probabilities. Okay, I don't have to be perfect here because perfect is always the enemy of progress. And it lowers that bar and 80 % feels much more comfortable to make educated guesses. And then you can move a lot quicker. And often those are remarkably accurate when people are thinking that way. tying it back into lesson one of separating the outcome from the decision and like understanding that process, your goal with the probabilities is to get tighter and tighter and tighter and shrink that variance within the problem.

8:25Jordan Staab:I think it's a very important lesson because there are no absolutes in business.

8:30Annie Duke:In the last chapter of the book, she has quite a few different tools and tips for improving your decision-making by changing your timeframe. One of them is reconnaissance and to explore lots of different future scenarios before committing to any one of them and then assigning probabilities to each of these scenarios or to the different outcomes that you might get from each of these scenarios, which allows you to kind of prep and plan for this uncertain future that we live in. Have you put that sort of thing into practice before? Is that part of planning your new product launches or new marketing efforts?

9:04Jordan Staab:Yes. Really understanding why are we doing this? What are all the possible outcomes within this and understanding where this could go before you start down that path? A lot of problems can be avoided in startup world, especially. Are we backing ourselves into a corner by making this decision? Because like a year from now, the decision tree is like not great. We don't love any of the outcomes. And so we might be climbing the wrong mountain. I always look at choosing the problems you're working on. It is like climbing a mountain. You don't really know where the top is. And when you start, usually you have limited information on that.

9:36Jordan Staab:But if you choose the wrong one, you get to the top and you look over and there's a bunch of other mountains that you should have climbed instead. And you got to go all the way back down to start on the next mountain. And so that's part of that game theory there is like thinking through like, where is the ceiling of this idea? And so we always go through effort and upside on things. And so we say, how big is the opportunity? And then how difficult is it? Like, how much effort do we need to put into it? And that's just a very simple version of like how we can stack rank ideas. is, and we sprinkle in how likely are we to be the ones to solve it as an accelerator for it.

10:08Jordan Staab:And so let's say this idea can be a billion dollar idea, but it's a nine out of 10 on difficulty and one out of 10 on where the people to solve it. Probably not a great idea. How do we look at everything we're doing under that same evaluation criteria and make sure that we're spending our time as wisely as possible and that it also is on the path to something that we like all of the branches of the decision tree later on.

10:34Annie Duke:Do you have like a template or a document that you always use to plan this sort of thing on? So it's easy to compare one idea or one plan against another. Like I know Amazon has this PR FAQ for everything that they ever release. And I know a lot of companies do something similar or have like a different version of that. Do you have anything that springs to mind there?

10:52Jordan Staab:Yes, we do have a standardized approach. I mean, again, it's about trying to keep everything as standardized as possible, because if it's not consistent, you can't improve it. It's very difficult to improve it. If you're trying too many things, it's hard to course correct. Absolutely always working from a template. It doesn't have to be a great template. Just make it up and then find the things that were terrible about it and fix it. A lot of it is just like get in the arena and like if you're not documenting, if you're not trying to create the process, you're not going to get better. It's literally going to be pure luck if you get better at this.

11:25Annie Duke:what have you found are some of the key things that kind of work no matter what kind of company you're in and what maybe are the couple of things that you would avoid avoid complexity so it should

11:34Jordan Staab:be extremely simple when you start to get into book report territory where like the objective becomes communicating your ideas in the cleanest and most articulate way possible and you feel accomplished by finishing an idea like that's not great and so the idea should not be rewarded the The idea should not be a dopamine hit. The accomplishment of the idea is the dopamine hit. And so making sure that everybody is always aligned on like, I don't really care if it's bullets or paragraphs or a chart or PowerPoint slide. Just clearly communicate it and make sure that it is as dense in value as possible because you're going to do a lot from there.

12:13Jordan Staab:And so if it's a complicated process, you're going to waste a ton of energy. And that's not a guess. That's from failing at this and experimenting in the wrong direction many, many times. And so minimize complexity is the number one thing. Everything in your formula and the standardized should be there for a reason. The simpler it is, the easier it is to edit and create understanding. Just cut all the fat and just focus on the tightest model possible. So opportunity and effort with a little bit of like, are we any good as our model? And that's going to get you 80 % accuracy really fast. And so we can go through 10 ideas in five minutes.

12:52Jordan Staab:and get cardinal directions and then you can dig in from there you can realize very very quickly this is a bad idea we should stop wasting energy on it and just move on like a version of that like where people be arguing about well like we need to do this um and we got to get this software for this problem and like no we don't need that software i'm like guys how much is the software and they're like five dollars like i'm like buy the software i don't care why are we talking about this like it's such a waste of energy and so there's so many of those problems within this If you can just eliminate those conversations by only deep diving on things that matter, you just get so much more time back in your day, so much more energy to devote to the problems that will actually move the needle.

13:31Annie Duke:My co-founder has this thing that he doesn't believe in any plan if it's not survived the test of writing. You have to force yourself to go through the act of writing down exactly what the plan is. He's done this to me a few times when I've been quite enthusiastic about a plan and in my head it makes sense. He just says, okay, I'm not fully getting it, write it down. and then in the process of writing it down, I found all the holes. Then I've come to them and been like, yeah, I got excited. Let's just move on. That wasn't the thing. You'd be surprised at how many times this comes up and people just haven't got this on paper, haven't written it down and haven't thought it through fully because you can't until you've actually written it down.

14:05Annie Duke:Putting it into words and documenting it, it forces you to think it through in a different way.

14:09Jordan Staab:One of my favorite things I always talk about is a problem well-defined is half solved. So it's the same flavor of that. It's like if you can't clearly define the problem, like you don't stand a chance at solving it. It's way harder to do than people think. People like complaining, but they don't like really defining the problem. Like what is really the problem? And finding every possible nuance. And like at the end of the day, that's a business plan. It's like really clearly defining the problem. And then you can go attack and solve it.

14:36Annie Duke:Let's move to lesson number three. What have you got for lesson number three? Resulting is a trap. Can you explain that term resulting first?

14:45Jordan Staab:Yes. So the concept of resulting from Annie Duke is judging decisions solely based on their results. That's what we were talking about before. Like, did I win? Did I lose? So if you made horrible decisions and you won, was that good? No, it was not. Like, you're going to lose the majority of the time going back to thinking in probabilities, not absolutes. And so this is just a deeper dive into lesson two. I played a 2-7 in poker and ended up winning the biggest pot ever. Like, should you play 2-7 all the time? Absolutely not. Statistically, that's like the worst hand you could possibly have. And you're going to lose the vast majority of the time, no matter how you play it.

15:20Jordan Staab:That circumstance, you got lucky, or maybe you won the skill like understanding something within that, but not over-indexing on because the outcome happened this way, this is how I should change the course of everything. And this is easy to say, hard to do. Results matter. And if you want control of results, though, you need to focus on the process, not just the outcome.

15:42Annie Duke:How do you really evaluate your decision making to make sure that you're not just scaling bets that have won because of luck?

15:50Jordan Staab:Again, in your initial process, what are the possible outcomes from this? And what are the likelihood of the outcomes? If something happens outside of our predicted outcomes, we pause because something was very wrong with our initial process, or this is a complete anomaly and we just don't understand it yet. And so if it fell within the initial bounds of our decision making, it was like, yeah, I'm playing two seven here. I'm going to lose 95 % of the time if this person has better cards. But the 5 % chance is that like a seven comes out and it's a messy board and I can win on a pair. And that was a predicted outcome.

16:25Jordan Staab:if getting four sevens was the result. I'm like, I am playing seven two and I'm gonna get four of a kind every time I play seven two. Like, no, that was not remotely in our outcome set of like how we would interact on this project. And so like, wait a second, this is weird. Let's not ever index on this result. I mean, it's not 10X our effort on this and scale it because like we have no idea what we're doing. If you own the decision-making process and then evaluate your results around your decision-making process, you can start to differentiate luck and skill. In the 4-7 scenario, that's luck. If we won on a pair because it was a predicted outcome and we were doing it trying to get lucky on that one thing, that's skill.

17:07Jordan Staab:That's part of our original process. That's how we apply it on a regular basis. If you're not going in with a scientific process of saying, this is what I think will happen, and evaluating the results in that way, you're kind of lost as you're going into it. Again, you're just hoping.

17:22Annie Duke:There's this thing, even if you're looking at an illusion, you can't unsee it. I feel like it's a little bit similar to this. Even if you know that something is possibly luck, you're still kind of drawn to try and do it again. Everyone who's ever played a little bit of poker knows that 7-2 is, that's the worst hand statistically to be dealt. But I think maybe because it's the worst hand and it's got this thing about it, a lot of people play it just for a bit of fun. And then obviously you sometimes win on it because that's just the way luck happens. And then you get people playing it. I think Andy Duke tells a story actually in the book around some guy who says he always plays seven deuce and then lost a hell of a lot of money.

18:00Annie Duke:But he kept doing it because he had like a mental muddle in his mind that it would work for him.