The dawn of search engines

0:01NPR.

0:11This is The Indicator from Planet Money. I'm Adrian Ma. Today, we've got a special guest episode for you. It's an excerpt from a series called Who Broke the Internet from our colleagues up north at the Canadian Broadcasting Corporation. It's from their understood feed at CBC Podcasts, and the host is Cory Doctorow. You might have heard of him. He's a journalist, blogger, science fiction writer, and internet commentator. And his series digs into his criticisms on the state of the modern internet and what we can do about it. So in this excerpt you're going to hear after the break, Corey's going to talk about how search engines got started.

0:50Support for this podcast and the following message come from Ameriprise Financial. Chief Economist Russell Price shares a key investment principle. Market trends tend to tell us that time is on the side of the investor. remaining invested through periods of highs and lows is generally one of the better ways to build wealth over the long term for more information and important disclosures visit ameriprise.com slash advice past performance is not a guarantee of future results security is offered by ameriprise financial services llc member finra and sipc this message comes from npr sponsor capella university Learning doesn't have to get in the way of life.

1:32With Capella's game-changing FlexPath learning format, you can set your own deadlines and learn on your own schedule. That means you don't have to put your life on hold to earn your degree. Instead, enjoy learning your way and pursue your educational and career goals without missing a beat. A different future is closer than you think with Capella University. Learn more at capella.edu.

2:02Eventually, the internet got too big for people to know where everything was, so waves of clever people created search engines. Early search engines just looked for pages containing the words you typed, giving priority to pages that contained more of those words. This worked okay, but when it failed, boy did it ever fail badly. If you wanted your page to rate high on the search results for a query like Mexican food, you could keyword stuff it by adding the words Mexican food a thousand times in tiny white-on-white type to the bottom of the page. The primitive search engines would count these all up and conclude that your page was the most important Mexican food resource in the world, which is useless because most hungry people aren't looking for a site that just has the words Mexican food a thousand times.

2:53To weed these bad hits out of your search results, you'd have to master all kinds of arcane search engine syntax so you could exclude words and phrases, putting minus signs in front of the kind of junk that was typical of spam sites. It took forever, but those search engines were all we had, so we kept using them.

3:16We were using, you know, really awful search engines that we thought were pretty good. And you saw Google, and all of a sudden, the film cleared from our eyes. We thought, wow, you could actually find what you were looking for on the internet right away. Bang. This is Stephen Levy. He wrote the canonical history of Google, In the Plex, how Google thinks, works, and shapes our lives. Back in the early 90s, Stephen was Newsweek's tech reporter, and he knew right away that Google was a story. When I first saw the Google search engine, my mind was blown. So he tracked down their PR person. I called her up.

3:57I said, I got to meet these guys. These guys being Google's founders, two Stanford grad students named Larry Page and Sergey Brin. So Stephen booked a ticket from New York to San Francisco and flew out to meet them. It was near Halloween. Everyone was dressed in Halloween costume. So Larry was dressed like a Viking, and Sergey was dressed like a cow. He had these big plastic udders coming out from his chest. So the Viking and the cow took me into a little room and explained how PageRank works. PageRank is the algorithm behind the magic, a new way that Google had developed to deliver its search results.

4:41It was totally audacious. And I say audacious because the way they were able to locate the best result for your query was by basically, not basically, literally downloading the whole web, all of it, which some people thought was ridiculous. But Larry Page understood this could be done. All Larry and Sergey needed to pull it off was big, powerful servers, which they could get. because they had access to servers at Stanford and they would literally hijack them as they came in the loading dock meant for other departments. With all this yoinked power, they set up their first server, building the case for it out of Lego.

5:24Yeah, Lego, because it was cheap and cool and colorful, like the eventual Google logo. And then they would do math, which was like Sergey's specialty, to analyze all the links that happen between websites and figuring out what important websites link to other sites. So that way you could filter out which were the perfect answers for the queries you put into the search field. And that turned to be just a quantum leap better than the previous things.

6:07The early internet wasn't perfect. Google came into existence because the search landscape sucked. But the point is, Google could come into existence. There was oxygen and sunshine that reached the forest floor so new things could grow. If you could code, if you had some money, a computer or two, a couple of bins of Lego, you could. You were allowed to build something new that made the web better. Clive Thompson again. They start off by having a really good search engine, the best search engine. And the reason why it's so good is that they are in ferocious competition with like 12 different search engines.

6:45And they have to be better than everyone else. They have to fight to the top. And they do that. Just be the best. Make the best product and people will switch. And we did. We all switched to Google because it worked. And that mattered to Google. in their early days. And there's, you know, the very famous motto that Google had was don't be evil. Mm-hmm. You know, which was coined by an engineer when, you know, someone from HR had a meeting. And, you know, so what is our motto? What is our value? And, you know, one guy said, what are you talking about? It's just like, don't be evil, you know? I mean, that's it.

7:31So back in 1999, Stephen wrote that first story for Newsweek, and he kept writing about Google. Google's world was our world. You know, things didn't exist, but it wasn't on the web. It's like when Google did well, we all did well because we could find the stuff we were looking for. Right. And that came into play even more when they came up with their first big successful business product. That was ads. Yeah, yeah. When Google met advertising, everything changed.

8:09The introduction of ads to Google marked a major shift. Emanuel Goldstein again. Google in their early days, you didn't have ads on the front page. It was relatively fast. And they just seemed like cool people inventing cool things. Larry and Sergey actually hated advertising. And the paper they wrote talking about PageRank, they had a little thing in the end saying, boy, we hate advertising. It's terrible. Stephen's talking about an article the Google founders published in a journal in 1998. And in it, they explicitly wrote that, quote, advertising income often provides an incentive to provide poor quality search results.

8:53But they had to make money. So then they figured, well, maybe there's a way to do ads that wouldn't be terrible. So in 2000, the founders assigned an engineer to figure this out, one of Google's first eight employees. And he came up with something. It was called AdWords. So what it would do was when you put a query into the search engine, it would start an auction instantly. And, you know, your search term would be auctioned off to advertisers who wanted to sell a product that was, you know, related. If you search for Mexican food on Google back in the early aughts, the results would have been the most linked to websites for Mexican restaurants in your area and reviews from local blogs and newspapers.

9:43And now with AdWords, ads would appear in a colored bar atop the normal page ranked results. You get the same results, but you get an extra little set of possible results that actually might even help you if you were looking to buy something related to the search term. And the advertisers would get an audience which was primed to buy something. And it worked for Google. It worked for advertisers. And most of all, it worked for users. That is, until it didn't. Thanks to our colleagues over at the Canadian Broadcasting Corporation. If you want to hear more of Understood, Who Broke the Internet, you can find a link in our show notes or find it wherever you listen to podcasts.

10:32The original version of this episode was written and produced by Matt Mews, A.C. Rowe, and Corey Doctorow. Roshani Nair is the coordinating producer. Julian Uzielli did the mixing and sound design. Story editing by Veronica Simmons. The executive producer is Nick McCabe-Lokos. The indicator version of this episode was produced by Cooper Katz McKim and engineered by Sina Lafredo. Kate Cannon is our editor and the indicators are production of NPR.

10:59This message comes from BetterHelp. To mark World Mental Health Day, BetterHelp is thanking the therapists who change people's lives all around the world by providing accessible mental health support. With over 12 years of experience matching clients with therapists and one of the world's largest online therapist networks, BetterHelp can help you find the right therapist. Visit BetterHelp.com slash NPR for 10 % off your first month. This message comes from Grammarly. From emails to reports and project proposals, it's hard to meet the demands of today's competing priorities without some help. Grammarly is the essential AI communication assistant that boosts your productivity at work so you can get more of what you need done faster.

11:42Just a few clicks can tailor your tone and writing so you come across exactly as you intend. Get time back to focus on your high-impact work. Download Grammarly for free at grammarly.com slash podcast. That's grammarly.com slash podcast.