Sample size (E)

0:00Hey, it's Seth, and this is an Akimbo announcement.

0:08I know I've been gone for a while. I've missed you. I've been writing. My new book, This is Strategy, comes out in October 2024, right at the end of the month at a local bookstore near you. Starting in September, we will be back with new episodes based on the audiobook, based on the Udemy strategy course and a special Q &A series where you can volunteer to go over your strategy as part of the show. If you'd like to share your journey, what you're up to, what you're challenged by, visit akimbo.link and press the appropriate button and we will be in touch. Thanks for holding on. Thanks for the work you do.

0:57Keep making a ruckus. We'll see you at the end of September. During the 10-year period between the year 2000 and 2009, the per capita consumption of mozzarella cheese in the United States had a 99 % correlation with the number of doctorates awarded in civil engineering. Or, if you want more correlation than that, Between 2000 and 2010, the per capita consumption of margarine had a 99 % correlation with the divorce rate in Maine. Hey, it's Arav, and this is a special archived episode of Akimbo.

1:48You may have heard some of these spurious correlations before. 95%, 99 % correlations between obviously unrelated factors. But how do we know they're obviously unrelated? Statistics is a fascinating science. It is a science that helps us see the past, understand the present, and predict the future. If you want to know if Oreos get soggy in milk, you don't have to dip every single Oreo ever made into a glass of milk to come to the conclusion that they do. Statistics with the appropriate sample size helps us see that there is in fact not just a correlation but causation between dipping your cookie in milk and the cookie getting soggy.

2:41Where we get in trouble is this. There has to be correlations of unrelated items over a 10-year span because we have so many to choose from. We can look at every statistic about divorces, about lawyers, about other factors in our society. Per capita consumption of cheese has a 98.9 % correlation with the total revenue generated by golf courses. Well, we didn't come up with that by looking at two things that we thought were related and then running the correlation. We simply matched up some numbers. That leads us to the challenge of survivorship bias. One of the greatest statisticians of all time was someone from Transylvania, Abraham Wald.

3:35Abraham Wald was a pioneer in statistics and one of the people who invented operations research, the whole idea of looking how systems worked and figuring out how we could understand to make them work better. The U.S. Army hired him to look at the fact that a lot of planes were getting shot down. Not only is this a tragedy for the family of the pilot, but it's really expensive. How, they asked him, should we make it so planes get shot down less? here's a whole bunch of planes that came back. And he looked at the planes that came back, the ones that had survived, and they had bullet holes in them.

4:14And what the other engineers had said was, put shielding over the spots where the bullets are hitting the plane. Because if we put shielding on the spots where the bullets are hitting, the planes will do even better and more of them will come back. And what Wald says is, no, no, no, no. Put the shielding on the parts of the plane that don't have bullet holes in them because the planes that aren't coming back aren't coming back because they're getting hit in the engine. We're seeing the survivors. The survivors don't have the problem. Let's look at the ones that do. Consider the idea of stock market funds, funds that purport to beat the stock market.

4:57Turns out that more funds than you would guess have had excellent results over the last five or ten years. How do we explain this? Are they all really smart? Well, in fact, almost none of them are smart if we're going to define smart as consistently beating the market because nobody is consistently beating the market. Nobody is consistently getting better returns than the risk that they are taking. In fact, what we are seeing is this. Funds that got unlucky and didn't do well, they disappear. Because they disappear, they're not in the pool that we are comparing. So back to this idea of spurious correlation.

5:40How do we know that margarine and divorce are ridiculous? The answer is because there isn't an explanation we can understand. And our need to understand is essential because we can't make good choices going forward if we are simply basing it on data. This becomes super important when we begin to scale up artificial intelligence. Arvind Narayanan is a professor who is researching how we can use AI successfully. He has pointed out that there are three problems that AI is purporting to solve. One of them is very straightforward. This is things like perception, facial recognition, or identifying a song from just a few notes.

6:31We know there is a definitive, correct answer. If we feed the system enough data and we give the system enough right answers, not only can it look for correlation between data points, but it can begin to match up what actually works with what over time. The second category is automating judgment. Things like spam detection or copyright violations or grading an essay. And it turns out there's a judgment call here, but over time, because we are using enough data, the systems can figure it out. But the third category, predicting social outcomes. Predicting social outcomes, things like predictive policing or terrorist risk, all the stuff that big government and big companies would like to use on humans, it turns out that's really, really difficult.

7:30Because in the words of William Goldman, no one knows anything. That predicting the next hit song is much harder than it looks. The thing about statistics is that statistics will tell us on average what to expect. That if there's a 78 % chance that someone's going to win the election, it doesn't mean they won the election. It's not like a test score where if you get more than 65, you win. 78%, 95%, it doesn't matter. What 95 % means is that 19 out of 20 times, person A is going to win the election, and one out of 20 times, person B is. And if you had a pair of dice with 20 sides on them, and you rolled them, then you would have a decent understanding of what the chances are of it all working out one way or the other.

8:27But we act like statistics are true. The thing is, ironically and sadly, flying on an airplane is super, super safe. Flying on an airplane is far safer than driving. But that didn't keep Abraham Wald from dying in a plane crash at younger than 50 years old. because statistics simply tells us the range of what we can expect to happen, not why or how it will happen in any given moment. What we need is understanding. Now, on to this idea of the future, using the past to predict the future. Earlier, I pointed out that stock funds like to imply that they will beat the stock market. But yet, in 100 years of people trying to consistently beat the stock market at a rate better than the risk they are taking would indicate, we have not found somebody who has done it persistently in a way that we can understand.

9:33Because as soon as someone understands it, other people will do the same method. Now, this rubs a lot of people the wrong way because they have seen people who are good at predicting the future, people who are good at making psychic predictions or who's going to win the next football game. Well, if you want to run a scam, here's one thing you could do. You could start 200 sites, each one of which predicts the results of football games. And in week one of the football season, have 100 of the sites predict team A is going to win, and 100 of the sites predict Team B is going to win. And each game, as you go further and further through the season, if a site is doing poorly, as you branch out, close that site down.

10:24So by the time we get toward the Super Bowl, there are going to be sites with a perfect record. There have to be sites with a perfect record because all they are doing is going down one branch. But sooner or later, they're going to be wrong because all they are doing is moving numbers around. They don't actually understand what works and what doesn't. Nobody knows anything, not when it comes to who's going to win a football game. Bob Lefsitz runs one of the most important and funny newsletters about the music industry. He recently wrote a piece about a singer from Australia who calls herself Tones and I.

11:11Tones and I has had number one hits in dozens of countries around the world, except for the United States, at least to date. Well, he talked about how she was on fire, how she was doing great. And then his readers wrote in. His readers are music industry executives. His readers are people who for a living get paid to predict the future. Bob Davis wrote, watched it. Dreadful music. Meh. D. Hamill wrote in, pointless. Guy named Bill Seiple wrote, I listened to it. I don't get it. Your first instinct in not liking it was correct. Billie Eilish gave us mumble rap. I hope the genre of baby talk singing doesn't catch on.

11:57And on and on it went. And they're all wrong.

12:17They're wrong because correlation isn't what's on offer here. what's being discussed is do you understand? And no, you can't possibly understand. That the people who are being seen as successful as predicting pop culture, the John Hammonds of the world, the Jason Flombs of the world, they have an ear, but mostly what they have is the guts and confidence to double down on the things that work. Because we don't pay attention to the ones that don't work, But we do pay attention if you double down and double down and double down on the ones that do, because then we have a chance to amplify the small signals of what's working.

13:04Back to the irony of Abraham Wald. After he died, Ronald Fisher, one of the other great statisticians of the 20th century, attacked his work, saying he didn't know how to set up an experiment. Ronald Fisher did brilliant work at understanding issues of Darwinism, including sexual selection. Ronald Fisher also spent a lot of time hanging out with psychics, meaning he was completely fooled by false correlation appearing real. At the end of his life in 1950, Ronald Fisher made a tragic error in that he spoke out against a UNESCO study that showed that people of all different races and backgrounds had the potential to do any sort of work.

13:55Fisher, confused because he was looking at false correlation, not at understanding why, said that different races had different abilities and that black people couldn't possibly do the intellectual work that white people like Fisher were doing. Again, what we're stuck with is margarine and divorce rates. What we're doing is looking at false correlation. Correlation is not causation. That there are tons of factors at work that influence whether or not something is going to happen. And just because things line up for a week, a year, a decade, doesn't mean that one is causing the other. And so the hard work of statistics is not to do a t-test, not to make sure you have the right sample size.

14:47The hard work of statistics is to understand. Because if we can't understand, then we're going to get seduced by the seemingly accurate predictions of artificial intelligence. We're going to be seduced into believing that the future of music is going to sound like the past of music. We're going to write off populations of people simply because in the past, other factors prevented them from doing the work that needs to be done now. We can do better than this. Odds are, anyway. Thanks for listening. We'll see you next time. Hey Seth, it's Maria. Hey Seth, my name's Kyle. Greetings, Seth. This is Stephen out in Madison, Wisconsin.

15:29Hi Seth, Alicia from Charleston here. Hi Seth, this is Anupam. Hi, this is Caitlin. Hi Seth. Warm greetings from Curacao. Hey, Seth. My name is Nick Ryan from Pittsburgh, Pennsylvania. Hey, Seth. This is Rex. Hey, Seth. Hi, this is Vasilis from Greece. Hi, this is Roberta Perry. My question is... And that completes my question. Thank you, as always, for listening and for your questions. I love to hear from you. To ask a question, just visit akimbo.link, that's A-K-I-M-B-O dot L-I-N-K, and press the appropriate button. While you're there, you can find show notes, ways to subscribe, and where the transcripts are.

16:11If you were to start over now in 2020, what steps would you follow to build an audience that you go deep with? What is the right medium to expose yourself? And at what pace would it be okay to do it so that you also build momentum and at the same time take advantage of the media algorithms that are affecting our work? Thank you for everything you do. I love your work. It's extraordinary that someone in New York is talking to someone in Romania about how to start a business that matters. In 2020, the rules are fundamentally different than they were just 20 years ago. Access to world markets, the ability to find employees and contributors from around the world, a shift in what people are buying, what they care about, and how they connect.

17:07It's profound, probably the most profound shift in our culture in over 100 years. And you, like so many of us, have a lot of choices to make. The choices go way beyond what should I sell at a little store in my village, way beyond what should I make in a factory with a machine that I could possibly afford. Because we don't need a little store, and we don't need a factory, and we don't need a machine. So with that said, I would begin with this. What's the smallest viable audience, the smallest group of people that you've could connect with, that you could inspire, that you could change, that you could make a living with and for that would sustain you on your journey?

17:54Because the answer is not a billion people. It might not even be 100 million people or 10 million people or a million people. And the place we begin getting trapped is saying, this is for everyone. It's so good. It's for everyone. How do I get the word out? The alternative is to say it's for someone. How can I be specific? The next part is Google and people like Google are making it almost impossible to get found. You can't count on a search engine to help people discover you. That the way you win SEO is not by getting it so that a generic term finds you. It's getting it so that people don't type in a generic term.

18:41They type in your term. They're looking for you. How to get people to look for you. You make something that other people want to talk about. You make something so that people benefit when they tell their friends about what you're doing. And so that thing is probably not a commodity. That thing is probably not, you're looking for something and we've got something. That thing is specific. That thing is hard to copy. That thing might very well be about connection, about who are the others, who else is here. I don't want to miss out. And so, ironically, one of the oldest professions, the idea of a musician standing in front of a bunch of people live, that industry is back.

19:30It's back because you can't make money really selling records, but you can make money putting on a concert if the other people in the room are worth being in the room with. And you can do it at scale. Rocky Horror Picture Show generated tens and tens of millions of dollars years ago because people wanted to go together to see a movie. So I could go on and on, but I'm going to begin with this. smallest viable audience and build something that a small group of people can't go to bed without telling someone else about. Hi Seth, this is John from Cape Town. In the 2012 London Olympics, a South African swimmer called Cameron van der Berg won a gold medal in breaststroke 100 meters and then afterwards admitted to cheating by using underwater kicks that were illegal.

20:23And he came out in a statement saying that the reason he cheated was because everybody cheats. And if you don't cheat, then unfortunately, you're not in the game. So I was wondering how in this climate where you have to cheat to stay in the game at that level of competition, how do you reconcile morally justifiable action with this competitive nature of business? Your last episode on sportsmanship really got me thinking about how to put these two together. I would absolutely love to hear your thoughts on it. Thank you for this question. It's poignant and important. I would argue this. There are competitions where you cannot win without cheating.

21:12It could be because it's easy to cheat. It could be because they are so competitive that creative cheating actually pays off. And it could be because you're not monitored very closely, and so cheating becomes the new normal. But I'd like to argue that you don't have to pick one of those competitions. You don't have to seek to be in competitive bicycling knowing you're going to have to dope to win. Pick something else. That to live a life worried that you're going to get caught cheating or complaining that you're losing because you didn't cheat isn't necessarily the life you might want to choose.

21:51There are plenty of other areas, plenty of other ways to make a good living, to enrich the people around you, to do work that matters for people who care, where you don't have to cheat. These ways are harder to find. These ways probably have less glory associated with them, but they exist. And if, by example, we choose them and we devote ourselves to them, if we pick something where it is not winner take all, if we leave the monopolistic network effects to others, what we will be doing is weaving a better culture. You can smell it. You can tell when cheating is your only option. And if it's your only option, I don't think it's your only option.

22:37You can choose to do something else. Thanks again for listening. We'll see you next time.