#428 How Claude Shannon Worked

24:02Bell Labs was spun out of the phone company too. So there's a lot of thinking and similarities and thinking between spinning out R &D between what is happening here in this story and what the founder of Honda was doing in Japan as well. So Shannon is headed to the world's most foremost technology company, and it's the home of the best communications minds in America. The goal of Bell Labs wasn't simply clear and faster phone calls. The labs were tasked with dreaming up a future in which every form of communication would be machine aided. Right place, right time, right set of skills, right person.

24:34This is exactly the perfect place for Claude Shannon. So the amount of innovation that comes out of Bell Labs is insane. Let me just read this paragraph to you. In the span of a few decades, Bell researchers engineered the first ever long-distance phone call. They synchronized the sounds and images in movies and demonstrated some of the earliest facts in television systems. During World War II, they improved radar, sonar, and the bazooka, and they created a secure phone line connecting Franklin Roosevelt and Winston Churchill. They would invent touch-tone dialing and the solar battery cell, And they would also pioneer the communications satellite.

25:04And in 1947, Bell researchers also created the transistor, the foundation of modern electronics. And so this is why Shannon said he wanted to work at Bell Labs. I had the freedom to do anything I wanted from almost the day I started. They never told me what to work on. He also had no responsibilities other than to tinker and to think and to publish papers and to do science. Why is this important? because Shannon was allergic to administrative work and bureaucracies of almost every kind. He has a very deep, he's like a level 10 introvert, which we'll get to in a moment, but he's naturally a loner.

25:40He long solitudes, Spartan apartments, empty offices. This is what he like where he does his best work. His freedom from obligation played into his lifelong tendency to isolate himself. Most days were spent shut indoors, alternating between his notepad and the clarinet and back again. So he, I already mentioned it a few times and he's going to mention a few more times that he loved working, uh, back and forth on multiple things at one time. Again, there's no separation between, you know, work and play. So he might be thinking about, you know, how can we, uh, he did a lot of work in cryptography or how can we, you know, improve the telephone network.

26:17And he'd think about that for a little bit. And then he put his pad down and he would listen to jazz or he'd play his clarinet or he juggle, or he'd ride his unicycle. And then he'd go back and forth, back and forth. Like this is essentially, you know, his entire day and how he lives his life. Then it goes back into, oh, it says it right here. He was actually writing this letter to Vannevar Bush and he talked about, you know, essentially what we'd call this today is like his productivity hack. He says, I've been working on three different ideas simultaneously. And strangely enough, it seems a more productive method than sticking to one problem.

26:47Now, Now, this is early 1940s. He's, you know, in his 20s. And so he's going to be drafted into the war. It terrified him. Now, you know, you could think, okay, might terrified him because he was worried about dying. Obviously, he didn't want to do that. But what was interesting is a description of his introvert nature. It's like, wait a minute, I'm going to have to live in barracks. I'm going to be in the army. I'm going to have all these strangers around me. Freaked him out. So he wanted to find a way to use his mind to serve the country, not his body. So it says Shannon worried not only about the dangers of an overseas deployment, but also the close quarters of army life.

27:21I think he did the work with the fear that he might have to go into the army, which means being with lots of people around, which he couldn't stand. He could not stand being in groups. He was phobic about crowds and people he didn't know. He would put his mind rather than his body to work on the country's behalf. So this is not only Shannon, but almost everybody at Bell Labs is now transitioned into doing work for the war. and one of the things is interesting it's like nothing really goes when you study shannon nothing that he works on ever goes to waste he'll find other ways to essentially make an abstraction or analogy and use it in his later work and so this description of what he's actually working on uh it's called fire control which i'll read to you in one second actually plays a role into him uh because he's writing the the paper on information theory i think it takes him like eight years on and off so he's doing a bunch of this these things at the same time and then he'll draw these analogies.

28:10So one analogy he drew was from fire control. So this is what fire control is. Fire control was essentially the study of hitting moving targets. The targets were anything and everything the enemy could hurl through the air to cause damage. Planes, rockets, ballistics. So imagine a gun firing a single shot at a target. Okay. Now imagine that gun is the size of a two story house and it's placed on a moving Navy ship in the middle of the ocean. And that it's trying to shoot down an enemy fighter moving at 350 miles per hour. This is the math he's got to figure out. That's a rough description of the challenge of fire control.

28:42It goes back to this idea of these always finding analogies. There are surprisingly close and valid analogies between fire control prediction problem and certain basic problems in communications engineering. At the most basic level, the speed and quality of information was vital to both phone systems and fire control systems. A phone call reaching its attendant recipient was a struggle against noise. An anti-aircraft missile hitting its target presented the same conceptual challenge. This is what he said. He's like, he's just a man of abstractions. He's like, oh, I'm actually working on the same problem.

29:15You might think many people, again, I think it's part of his genius. I wouldn't even see the similarities between these two problems. And to him, everything was the same. He says both required high level statistical inference. Both presented the challenge of building machines to accurately translate math into action. Then it goes into some of the other things that his colleagues at Bell Labs were working on. Some of the questions the scientists at Bell Labs had to tackle for the war effort. And they're working six days a week on this. And so again, the army and the government's giving them all these problems.

29:44Here, find solutions to these questions. How many tons of explosive force must a bomb release to create a certain amount of damage? In what sorts of formations should bombers fly? Should an airplane be heavily armored or should it be stripped of defenses so it can fly faster? At what depths should an anti-submarine weapon drop from an airplane explode? How many anti-aircraft guns should be placed around a critical target? In short, what the government is trying to figure out is precisely how should these new weapons be used to produce the greatest military payoff? Now, he's doing this because obviously they're compelled to.

30:16He doesn't want to physically serve in the war, but he hated it. So this is his reaction to all this war work. The whole atmosphere left a bitter taste. The secrecy, the intensity, the drudgery, the obligatory teamwork. It goes back to, I don't know if you caught on that. There's like this one sentence I said earlier that very few of his papers were co-authored. He does not like working on teams. He likes working by himself alone. And so he's like, I'm being compelled to work on a team. I hate this. And it winds up getting to him. He has, you know, he's also going through a divorce at the time.

30:48So he kind of has like this, you know, temporary breakdown. Now, another thing that they have him working on during the war is cryptography. They're trying to figure out how can we down, not only do we want to be able to send secure communications, you know, across long distances, but we also want to hack into, you know, the Germans and Japanese doing the same thing. We want to find out, we want to find their secrets and make sure they don't find ours. And so Shannon is tasked with checking the algorithms so to allow the messages to be securely reproduced on the receiving end. The work gave him a window into the world of encoded speech, transmission of information and cryptography, a synthesis that at that moment in history may not have taken place anywhere other than at Bell Labs, as Shannon observed.

31:31Not a lot of laboratories had voice encoding devices for scrambling speech. And so this is when Claude Shannon meets Alan Turing, who's also working on very similar things, and they wind up getting along so well. They met daily in the Bell Labs cafeteria, and they'd have tea. And later in life, Shannon's being interviewed, and he was being asked all these questions. He's like, well, with your own passion for, you know, cryptographic puzzles, why didn't you probe Turing, you know, deeper and further? And Shannon's response was simple and to the point. Well, in the wartime, you didn't ask too many questions.

32:03And but he did talk about some of the stuff that Alan Turing and him would talk about. Remember, this is happening in 1940. So it's pretty wild because now we are literally living in the future that these two guys envisioned, helped envision too. And we're essentially, you know, having these daily, you know, talk conversations about. And he says, we would talk about the notion of building computers that will think. And so Shannon was obsessed with artificial intelligence. He says, we had dreams. Turing out, he used to talk about the possibility of simulating the human brain. Could we really get a computer which would be the equivalent of the human brain or even a lot better?

32:35We both thought that this would be possible in not very long time, in 10 or 15 years. Such was not the case. And so in this book, there's a lot of quotes from Shannon about his deep desire to invent and to have artificial intelligence, to have AI. In fact, he called it his fondest dream. He says, my fondest dream is to someday build a machine that thinks, learns, communicates, and manipulates its environment in a fairly sophisticated way. Goes on over and over again. There's multiple quotes throughout the book like this. I believe we are going to invent machines which are smarter than we are. would talk about the fact that when he would discuss this, you know, it had to seem like crazy talk, especially back then.

33:14He says that he was adamant and he thought the people that thought he was wrong were actually the ones that were incorrect. The thought that a machine could never exceed its creator was just foolish logic. He called it wrong and incorrect logic. Now, Shannon winds up meeting, you know, all the other great minds of his time. He met John Van Neumann. He said Shannon called John von Neumann the smartest person that he ever met. Later in life, when he was asked a lot of questions, like, you know, they were all these scientists were advising the CIA, the Defense Department, the NSA. And Shannon was very reluctant to talk about that even decades later.

33:52But this surprising story will give you an idea of the type of classified work that Shannon was involved in because John von Neumann was involved in as well. One of Shannon's fellow NSA scientific advisors, John von Neumann, was watched around the clock by uniformed military personnel when he was on his deathbed. Impressive though von Neumann's mind may have been, it wasn't immune from infiltration, or so the government feared. And what better time to infiltrate it and grab the precious state secrets it held than when it was in a medically induced haze. So it's so important that he was guarded around the clock on his deathbed because they did not want anybody hacking his brain.

34:30So there is, I don't know, I bet you 50 to 100 pages of this book about information theory. I think for our purposes, I'm just going to give you a really simple overview because I think more important than that is the way he approaches his work. I think that's what you and I are most interested in. But I do think, you know, there is a couple paragraphs that give you, I think, a simplified version of a simplified understanding, rather, of information theory. So information existed before Shannon, just as objects had inertia before Newton. But before Shannon, there was precious little sense of information as an idea, as a measurable quantity, an object fitted out for hard science.

35:06Before Shannon, information was a telegram, a photograph, a paragraph, a song. After Shannon, information was entirely abstracted into bits. information theory was summed up by his recognition that all information, no matter the source, the sender, the recipient, or the meaning could be efficiently represented by a sequence of bits. He felt that bits was information's fundamental unit. So he writes this paper. Like I said, I think it's published in 1948. So his paper is going to introduce many of the concepts that underpin the modern digital world, the world that we inhabit. The fact that most of our, probably most of our communications is digital today.

35:45And Shannon's work is widely regarded as one of the most influential scientific papers of the 20th century and laid theoretical foundation for things like the internet, data compression, error correcting codes, digital telecommunications, modern computing, and much of today's AI infrastructure. Now there's a couple interesting descriptions in the book of him writing the paper. Again, it displays that over many, many years. Napkins decorate the table. Strands of thought and stray sections of equations accumulate around him. He writes a neat script on lined paper, but the raw material is everywhere.

36:14Eight years like this, scribbling, refining, crossing out, staring into a thicket of equations, knowing that at the end of all this effort, they may reveal nothing. There are breaks from music and cigarettes and bleary-eyed walks to work in the morning. But mostly it's this ceaseless drilling, again, alone, at night, by himself, back to the desk, where he senses perhaps that he's on to something significant, something even more fundamental. But what? But and then he talks about there was many flashes of intuition that his work was not linear. Ideas came when they came and he had no control over it.

36:47One night, I remember I woke up in the middle of the night and I had an idea and stayed up all night working on that. And one of the most remarkable things about Shannon is he publishes the paper. He knows it's good. He never doubts that his work is good. And even to the point where, you know, he kind of was like he trusted he trusted his judgment so much that when he said it's like, you know, if you want to argue ideas, like I'm not even going to argue with you. But once it comes out, then we see again goes back to his personality. He's like, he wants to do great work, but he's doing great work because he's just curious about it.

37:12He's not doing it. If you go back to what Munger says about the importance of having an inner clock, Buffett calls it an inner scorecard. Just like, man, I'm just doing what I want to do, regardless of what's going on outside in the outside world. And we see this because it says, having completed his pathbreaking work by the age of 32, he might've spent his remaining decades as a scientific celebrity. Instead, he spent all his time tinkering. So he'd build, he's a funny dude too. He'd build an electronic maze solving mouse. He had a chess, these are all things he built with his hands, by the way.

37:42He built a chess playing computer. He built the first ever wearable computer. He created a calculator that operated in Roman numerals. He had a fleet of customized unicycles. He spent years devoted to the scientific study of juggling. And his whole point is that working on what naturally interests you is time well spent. Shannon would be adamant on this point. After the effort of discovery, the effort of communication was secondary by far. He had solved a problem to his own satisfaction, and that, as far as he was concerned, was enough. Shannon explains this viewpoint later on. After I had found the answers, it was always painful to write them up or to publish them.

38:21Again, this is why it's so important. The writing up and the publishing is actually how you get the acclaim. He was just interested in solving the puzzle. So winds up, and now he's, you know, a scientific celebrity. Bell Labs does not want to lose him. essentially just do whatever you want Shannon you're on the payroll you can do whatever you know you can work from home you can come in you can do neither or both he still decides to wind up leaving because he's getting poached by by MIT and he talks about one of the reasons there's really two two main reasons that he winds up doing this and what's funny is even after he leaves Bell Labs they thought he was so important and he was so disinterested in money for money's sake that Bell Labs kept him on the payroll so he winds up getting paid from MIT and Bell Labs And so now at this point in the book, he's talking about why he leaves Bell Labs and decides to go to MIT.

39:09The general freedom and academic life is one of its most important features. There was a certain restlessness on Shannon's part after spending more than a decade and a half in a single institution. Having spent 15 years at Bell Labs, I felt myself getting a little stale and unproductive and a change of scene and colleagues is very stimulating. And so then I love this paragraph, which describes the result of that very important decision saying, hey, I had 15 years here. I loved it here, but I need a change of scenery. I need a change of place. I need a change of colleagues. And so it says, once he got to MIT, what resulted were some of Shannon's most creative and whimsical endeavors.

39:44There was a trumpet that shot fire when played, handmade unicycles. There was a chairlift that took guests down from the porch to the edge of the lake, a machine that saw Rubik's cubes, more chess playing machines, more handmade robots. Shannon's mind, it seems, was finally free to bring its most outlandish ideas to mechanical life. He loved building machines. What I would describe the activities that he's doing now, and you know, especially now at this point in the story, before earlier in life as well, his activities were autotelic. So autotelic, it's one of my favorite words. In fact, I've told you this before, but in case you don't remember, the original name of this podcast that I started 10 years ago next month was autotelic.

40:24Why was that such an important name? Autotelic is an activity done for the sake of itself. I was saying with the title of the podcast that I never thought I was gonna listen to, by the way, I'm going to podcast even if no one listens to it. I just feel compelled to do it. I love it. I'm obsessed with it. You see the exact same idea here. All of his ideas, all of his activities, they're just auto-telling. They're activities done for the sake of himself. Shannon summed all of his work up, the work that he's doing now, as happily pointless. I've always pursued my interests without much regard to financial value or value to the world.

40:55I spent lots of time on totally useless things. He made no distinction between his interest in information and his interest in unicycles. They were all moves in the same game. This is just and this is just great writing here. What other people called hobbies, he thought of as experiments, exercises in the practice of simplification, models that filed a problem down to its barest interesting form. He was so convinced of a machine-enabled future and so eager to explore its boundaries that he was willing to tolerate a degree of ridicule to bring it to pass. He was preoccupied with, quote, the possible capabilities and applications of large-scale electronic computers at a time when nothing even existed.

41:44Considered in the light of that future, which is our present, his machines were not hobbies. They were proofs. That is great, great writing. Here's another great quote from Shannon. A very small percentage of the population produces the greatest proportion of the most important ideas. There are some people, if you shoot one idea into their brain, you will get half an idea out. There are other people who produce two ideas for each idea sent in. Now, another thing, we go back to puzzles. He didn't really give a shit about money, but he was very rich. How do you get very rich? because he just started getting obsessed with stocks and stock market.

42:20He's like, oh, this is just another puzzle. And they wind up making, there's this documentary. And no disrespect to documentary makers, but everybody's like, you got to watch this documentary. It's called The Bit Player. And I think I watched it before. Rewatch it again to prep for this episode. Not good. But if you want to maybe watch it on 1.5, I don't know, or maybe 2x, maybe it gives you an insight. You're probably just better off just listening to the audiobook of this book or maybe listening to this podcast again. But one of the interesting things about documentary was they interview his kids.

42:53You know, Shannon's now passed on. I think he died in 2001. But they talk about the fact that they made investing a family hobby. And I thought that was interesting. And so his wife, Betty, says Betty and Claude did play the markets obsessively. The process became a family affair. And so this is what his daughter, Peggy, would talk about this. Much of the conversation around the home would be about the stock market because much of my parents' focus was on what the market was doing. They taught me to read the Wall Street Journal and stocks. And they taught me about stocks very early. You'd come down and open the newspaper and they'd have me read because at that time, their eyesight, my eyesight was better than theirs.

43:24And it was a way for them to engage their kids. Then eventually they set up a small personal computer to carry out the quotes during the day and then check again at the end of the day. So there were computer printouts floating all around the house with stock quotes on them. Now there's a couple of crazy stories. And again, this is one of my favorite parts of reading his biographies is like, you know, the higher you go, the world gets smaller and smaller. And so, you know, he's talking to John Van Neumann. He's meeting Albert Einstein. He's having tea with Alan Turing. And a twist of fate and just remarkable is the fact that Henry Singleton, who again, if you just go back and listen to Buffett and Munger, Buffett and Munger talk about Henry Singleton.

43:59Henry Singleton built a conglomerate before Buffett and Munger did. And they took a lot of ideas from Singleton. The ideas that I thought were Buffett and Munger's, they actually learned from Singleton. Munger said that Singleton was the single smartest person that he ever met in life. Buffett said that it was a crime that business schools did not study this guy. Claude Shannon and Henry Singleton wind up meeting in college. He was his friend in college. The conglomerate that Singleton builds is called Teledyne. Claude Shannon is going to be on the board of it, okay? And Shannon does something really smart and he makes a large investment into Teledyne.

44:35And that investment winds up compounding, gave him a compounded return of 20%, sorry, 27%. over 25 years. And as Shannon retold his story, he made the investment simply because I had a good opinion of him. And there's all these stories in the book where, you know, Singleton benefits from when he's doing a series of acquisitions, you know, being able to balance those ideas off of Claude Shannon. So I want to go, there's another book I read a long time ago called Fortune's Formula that I think actually has a better description of just how remarkable Shannon was at investing and trying to solve the puzzle of the stock market.

45:14I'm going to read a paragraph from that book in one second to you, but I just want to give you an overview of why he's doing this. Again, it was just another puzzle. Shin's interest in money resembled his other passions. He was not out to accrue wealth for wealth's sake, nor did he have any burning desire to own the finer things in life. But money created markets and math puzzles, problems that could be analyzed and interpreted and played out. So then let me read just how good he was at this. This is a paragraph from Fortune's Formula. In 1986, Barron's ran an article ranking the recent performance of 77 money managers.

45:48Claude Shannon, though not mentioned in the article, had done better than all but three of the pros. The Barron's money managers were mostly firms with up to 100 people. Shannon worked with his wife and an Apple II computer. Barron's reported on the recent performance of 1 ,026 funds. Shannon achieved a higher return than 1 ,025 of them. This is incredible. Over 30 years. From the late 1950s through 1986, Shannon's returns on his stock portfolio was 28 % a year. Another legendary person that Shannon winds up meeting. This is a young Ed Thorpe. Ed Thorpe and Claude Shannon wind up building the world's first wearable computer.

46:32they're trying to solve, Shannon was obsessed with trying to solve gambling. Thorpe wrote the book on how to count cards and blackjacks. Thorpe's still alive. It's episode 222. One of the best autobiographies I've ever read. It's called A Man for All Markets, if you want to learn more about Thorpe. But Thorpe, he's all in this book, but I thought this one description of Thorpe on how Shannon dealt with problems was very fascinating. And he says, Shannon seemed to think with ideas more than with words or formulas. A new problem was like a sculptor's block of stone, and And Shannon's ideas chiseled away the obstacles until an approximate solution emerged like an image, which he proceeded to refine as desired with more ideas.

47:14It reminded me of this great quote from Michelangelo when he was saying how he carved the statue of David. He says, I simply removed everything that was not David. Back to following his curiosity. He says, I don't think I was ever motivated by the notions of winning prizes, although I've had a couple dozen of them in the other room. I was more motivated by curiosity, never by the desire for financial gain. I just wondered how things were put together or what laws or rules govern a situation or if there are theorems about what one can or cannot do, mainly because I just wanted to know myself. And it goes back to that one of my favorite quotes of him.

47:49He says, I think the history of science has shown that valuable consequences often proliferate from simple curiosity. Later in his life, you know, he would pick, you know, he'd be invited all over the world to travel and to speak. Many times they wanted to give him an award. You know, he didn't really just like being at home. He liked eating the same thing all the time. He liked just being in his, you know, room building gadgets. But he did go and accept this prize in Japan, which I thought was very interesting. And he talked about the fact that he actually thinks that, at least in the United States, that we're actually teaching history incorrect.

48:21And it's just the wrong way to go about it. And he's got some really interesting insights about how he thinks history should be taught, but also tells you a lot about what he admired and what he thought were valuable. And he says, most of the time was spent on the study of political leaders and wars, the Caesars and the Napoleons and the Hitlers. I think this is totally wrong. The important people and events of history are the thinkers and innovators, the Darwins, the Newtons, the Beethovens, whose work continues to grow influence in a positive fashion. Talked about we need in the same talk, he says, we need more engineers.

48:52We need to encourage people to go into engineering. He thinks they're some of the most important people that ever exist. One category of innovation he signaled out for special mention. The discoveries of science are wonderful achievements in and of themselves, but would not affect the life of the common man without the intermediate efforts of engineers and inventors. People like Thomas Edison and Alexander Graham Bell. And then in a cool twist of fate, somebody who was born with one of the most gifted minds that has ever graced the planet. Unfortunately, the last decade of his life slowly starts to degrade.

49:23Uh, winds up having Alzheimer's once we're having to put into, you know, full-time care, but before he died, he outlined what he wanted his funeral to be. And I think, again, this gives you a great insight into just a very unique personality. This person that was obsessed with play and following his own curiosity. And so this is Claude Shannon's idea for his own funeral. Shannon had set to his mind the question of his funeral and imagine something very different For him, it was an occasion that called for humor, not grief. He outlined a grand procession, a Macy's-style parade to amuse and delight and to sum up the life of Claude Shannon.

49:59The parade would be led by somebody playing a clarinet. Behind them would be a jazz combo. Next in line would be six unicycling pallbearers, somehow balancing Shannon's coffin. Behind them would come the grieving widow, then a juggling octet, and then a juggling machine. Next would come three black chess pieces bearing$100 bills and then three rich men from the West, California tech investors, following the money. They would march in front of a chess float. Atop that float would be British chess master David Levy, who would be squaring off in a live chess match against a computer. Then, to the scientists and mathematicians, a phalanx of joggers and a 417 instrument band would bring up the rear.

50:46And that is how Claude Shannon wanted to be remembered. That is where I will leave it for the full story. Highly recommend reading the book. That is 428 books down, 1 ,000 to go. And I'll talk to you again soon.