Google’s Crazy AI Overviews, OpenAI’s Superalignment Drama, NVIDIA’s Unstoppable Run

0:00Google's AI overviews are crazy. OpenAI super alignment team falls apart and has to sign wild NDAs. and now the company is fighting with Scarlett Johansson? And yes, NVIDIA remains unstoppable. How long does that go on? All that and more coming up right after this.

0:20You're used to hearing my voice on the world bringing you interviews from around the globe. And you hear me reporting environment and climate news. I'm Carolyn Buehler. And I'm Marco Werman. We're now with you hosting The World Together. More global journalism with a fresh new sound. Listen to the world on your local public radio station and wherever you find your podcasts.

0:47Welcome to Big Technology Podcast Friday edition, where we break down the news in our traditional cool-headed and nuanced format. It's a holiday weekend here in the U.S., but that's not going to stop us. We have so much news to cover. It feels like this week is kind of an even more eventful week than usual. We have Google's AI search errors coming out with this AI overview situation. We have the big drama at OpenAI, what's new? We have NVIDIA turning in major results. Microsoft has this AI PC. It continues. So it's going to be a fun... I think we're going to have some laughs this week because some crazy stuff happened.

1:24But let me introduce Ranjan Roy and we can get the show on the road. Ranjan, welcome to the show. I think this will be a fun one, especially starting with these Google AI generative searches. One week ago, I was promised that the web was finished. One week ago, I was promised that Google had just destroyed the web with its AI overviews, that a generative search would prevent anybody from clicking through to a story ever again. Let me give the scorecard of how things look right now. If you ask Google how many rocks you should eat, Google will tell you you should eat one small rock a day. If you ask Google, why is my cheese not sticking to my pizza?

2:04It will tell you. You can add about one eighth of a cup of non-toxic glue to the sauce to give it more tackiness. If you ask Google, is everything you see on the internet real? Google will tell you, yes, everything on the internet is 100 % real. And listeners, I can report that at least one blogger, Katie Natopoulos from Business Insider, has made and eaten the glue pizza. Ranja, what do you think about this? Is this a sign that these large hot takes were overblown or should we actually interpret it to mean that maybe a rock a day is actually healthy for you and glue pizza isn't that bad? I knew you would start with this because this does run in the face of a lot of what I've said recently.

2:48So I'm going to try to defend myself here. And for regular listeners, you may remember, I have agreed with a Gartner claim that I think it was like 26 % of all search will be synthetic or generative in the next couple of years. I've said Reddit is the greatest training data set in the world. And the source of a couple of these posts, especially the glue on pizza, was apparently a kind of shit post from Reddit from 11 years ago that somehow made it into these generative search results. So for me, the way I'm looking at this is this is definitely going to be disruptive in the near term for generative search.

3:29And it's a reminder that this is a major transition for Google because the thing that always allowed them basically plausible deniability from the actual information they're presenting is they sent you elsewhere. They didn't present the information other than in snippets, which they'd been introducing more and more, as their own. And now this is Google's essentially publishing when they give you an answer. So they are going to be responsible for it. And there's been a lot of shit posting on the internet over the years that can be misconstrued. There's even a couple of people who had examples where it was from The Onion or other satirical information that's on the internet that clearly the model had not been trained properly to handle.

4:15But I don't think this is actually going to change anything. I think this is going to be a little bit of a kerfuffle at the moment. But I actually, these are, for the most part, ridiculous queries. that clearly people have been setting up for the screenshots and for the tweet. But I don't think this set changes anything in the medium term. I'm sticking by my generative search is the future. Okay. And this is Google's company line kind of echoes what you say, which they said that this is generally very uncommon queries and aren't representative of most people's experiences. However, it's sort of the whole kerfuffle brings into relief this big question, which is, is this something people actually need?

4:58And is this actually going to improve the search experience? Like, I get that these are uncommon queries, but I'm also just like, is Google actually doing this because it believes that the experience is going to be better? Or is Google doing this because it feels like it needs to make a move on artificial intelligence, seeing the perplexities out there, seeing the Bing searches? And remember, Bing hasn't been able to inch up beyond 3.6 % of the worldwide search market, even though it has these AI generated responses. And I wonder if Google is doing something not because it believes it's right for the users, but because it believes it has to.

5:37And one piece of evidence I'm going to put out here to sort of back up the point of maybe that's the case, is even after this AI thing, the Google AI overviews told people to eat rocks. Google's stock went up on Friday more than the S &P 500. It's almost as if the market says, we don't really care if this product works, if it's monetizable, if it's good for users. As long as you're doing something AI, we're going to reward you. I think that Google does realize that search is different. It's going to be different. And again, these kind of queries and basically hacking the system like this is at the moment kind of hilarious.

6:17Again, how many rocks should I eat? One small rock per day. It's the recommended - They're 92. It's above the daily recommended value. No, no, no. Two gets a little bit ridiculous because that's just overindulgence. But no, I think I still, this is going to be really interesting because how Google kind of pushes ahead, I don't think it's just to get a little bit of AI driven stock, you know, stock pumping in the near term. I think they recognize that search is going to be transformed again. Perplexity is just so much better than Google searches right now. Even asking ChatGPT for any kind of evergreen information, ChatGPT is better.

6:59Perplexity is far better. that Google is going to have to do something because when you have a monopolized business model like they do with search and it's threatened like this, you have to do something and they should be able to still be the leader in this because they still have the best data set in the world. So I think they're going to still keep pushing ahead and I think that they should. Can we at least admit that the web is still safe for the next little bit or do you think the web is toast? No, no, this is my, the more I've been thinking about this, because a lot of people I really respect and like the way they think are the one, are, you know, the web is going to die.

7:42This is the end of the web as we know it. I still have argued that the web has already been dead for a while. The web is that we kind of like the web that we are nostalgic for of the late 2000s and like finding information and having like discovery and magical rabbit holes on and going web page to web page has been long dead. I think things have been walled off on different platforms for a while. I think there's some, and I say this myself as essentially as a blogger and newsletter writer, and you as well, like, a lot of us are still doing it. But overall, the idea that there is this massive ecosystem that's magical, I think, has been long replaced by crappy SEO optimized websites and like even larger spammy publishers and all of the above.

8:34that I think that that idea, it's any kind of generic evergreen information, I think should be synthetically generated. And there's still always going to be room for good original information. And how it gets discovered is going to be different. But already it's been, you know, it's SEO and Google search has not been the way to find good, exciting information. Do you think the web's dead? Do I think the web is already dead? Do you think it's alive and thriving? I think it's, I wouldn't say alive and thriving, but the web is the web and the web will be the web would basically be my answer to that.

9:11Let me ask you, the web is the web. Like the web, the web has not been disintermediated. I mean, it has to some extent been disintermediated, but it's not gone and it's not going anywhere. Like I do think that because people who say the web is dead mean that what they mean is we're going to just live within platforms all the time. And this idea that we'll have independent websites and stuff like that is gone. And I just don't think that's the case. No, I think we will. I think they just will not be discovered via search. That's the difference. And they don't need to be the same way, you know, write a newsletter and publish it on Substack and have people within the Substack network or your newsletter subscribers shared across different platforms.

9:51Like that's where already discovery happens for the most part for me and for us, I think, and then an actual community building. But the idea that someone is searching for technology news on Google search and is ever going to find me or us, I think has long been a dream. Well, we do have. I mean, I did recently set up big technology for SEO and we do have some SEO results coming in. but typically they just flood in for like one story. Like one is a story that I get a lot of visits from is like does NVIDIA have actual competitors? And I have a story about the competition coming for NVIDIA and actually that does quite well in search.

10:32But I take your point and I think this is important and I'll agree with you here. If you're going to set up a media business or a business on the internet right now, you aren't going to like live through the website. Like the domain name isn't the most important thing. And for me, like after coming from Buzzfeed, setting up big technology as a newsletter, And as a podcast, which is effectively subscription media through subscription platforms, right? The podcast catching apps and the email inbox sort of that goes around this whole filter of like the search engines and the Facebooks to get you to links.

11:08So I guess de facto, I'm kind of agreeing with you. You think about the business I've set up. But if your web is dead, I don't know. How many times a day do you visit a website? Like websites are still important and I still have a website, even though I'm going through the podcast and the email. Yeah, that's true. I think the tech meme homepage is still a daily go-to for me. I sometimes go to NewYorkTimes.com, I guess. Via platforms end up on websites, certainly. So yeah, I'll say, I guess the big question is all information around gender to be AI search that I have wondered about and that I don't quite see a path for is like all information up to today has been covered or written about or exists on the internet.

11:57But if there is no economic incentive to create new information for anyone, then it does pose a question when new things happen. And I'm going to put news as one of these things, but just like a new discovery happens what about entertainment yes later in the show yeah a little foreshadowing yeah basically if things are synthetically generated and returned to users but but i also think it's about time i think a total like rethinking of economic incentives for digital content creation is it's not the worst thing it's just will it all accrue only to open ai and google and a couple of other tech giants or will there be completely new models um that to create it that that could be an exciting thing yeah let me ask you one more question as we close out this segment about it why does google botch every single one of these introductions i mean the gemini thing was terrible their initial introduction of like uh bard was bad it cost them 100 billion dollars off the stock i mean this one i guess if you do it at scale at the scale of google search you'll get some rock eating answers but it seems like every possible what time that google has an introduction they mess it up in some novel and extremely embarrassing way where it just doesn't happen with their competitors this is what's so fascinating to me is like we've all joked about sundar and is he gonna have a job and with google are they lagging on this what i think is impressive or fascinating is that they keep pushing ahead i mean after the the gemini image like with the diverse Nazis, Bard itself, just the name alone, much less the outputs a year ago, they're still pushing through versus, and we're going to get into this, open AI that seemed to every single launch and rollout was beautifully orchestrated.

13:52And the products themselves actually worked right on launch. And everyone just marveled at them. And even the ones that were demos like Sora and Video, everyone was just like, this is going to change the future. We love this. Sam Altman, you're great. And Google, meanwhile, is just going strong. Left and right. Yeah, Leroy Jenkins. But they're going. And to me, this is such a fast-moving, unpredictable space that I do believe in is going to fundamentally change technology, the entire industry. So I'm still going to give them a bit of credit that they're still pushing forward because any one of these you could imagine any one of these mishaps in the past like if you're a large giant corporation playing from a position of fear these are the kind of things that could shut down projects and lay off divisions and i mean these are these are bad like these are these are not great but but it shows that they're actually they're not afraid right now and that they're gonna keep pushing on this not and i don't think it's just for any kind of short-term stock momentum.

14:56I think they really, and even it was interesting to me that I am a believer in this technology. But also listening to Aaron Levy at the live podcast, and then you'd publish the episode on Wednesday, like, it's sometimes even fascinating to me that I'm bullish and a believer. But when I see like people who, like him, who running a multi-billion dollar company and have been in Silicon Valley for since the early days of the like the mid 2000s of Web 2.0 like the look in their eyes in terms of how transformative this is it almost makes me like am I missing something myself am I because I'm already bullish I think this is going to be really interesting and change business models and but like they're like this is really going to change everything so clearly the Kool-Aid is strong among the Silicon Valley folks.

15:49Yeah, let me just sum up what your position here is. Google's good, so is failure. The haters be damned. And the journalists are making a mountain out of a molehill here. I mean, I think that like some of the coverage has been just totally laughable. Like the Times, the headline on this story was Google AI search errors cause a furor online. It's like, were people actually furious? Are they kind of laughing? and the headline is the company's latest ai search feature has erroneously told users users to eat glue and rocks provoking a backlash and someone like tweeted that subhead and said it's not erroneous if you're brave enough to eat the rocks exactly and it's only one only one rock one small rock that's day though that adds up yeah a day come on that's uh it's just a pebble i uh one thing that one thing that was interesting for me is i i have been a big proponent and we've debated this that i still think reddit is the most valuable data set on the internet it is funny to me and this i will recognize this reminded me that like there's entire subreddits there's one circle jerk nyc where people the whole thing is just shit posting like like entire threads of totally sarcastic commentary of people just, I mean, it's hilarious and you recognize it as satire.

17:08That's why you subscribe to it. That's why you read it. But how large language models are going to try to process that, this was a reminder to me that, yeah, it's still going to be a challenge in an uphill climb for this. Yes. Let's say an appropriate bit of nuance, but still, Ranjan, you've argued well and i think you've got the best of me on this one so let's give you a point there just a pebble just a small one pebble for dinner the um just after we went off air last week we talked about how this one uh co-head of the super alignment team on open ai his name's jan like it just uh he resigned and he just tweeted i resigned and we talked about how he had a bigger responsibility to the world if he cared about the safety of these AI systems to tell us more than simply I resigned.

17:58Then over the weekend he did and more than that we learned a little bit more about why these OpenAI folks that are quitting with safety concerns aren't saying anything. So very quickly, Laika says, I joined because I thought OpenAI would be the best place in the world to do this research. However, I have been disagreeing with OpenAI leadership about the company's core priorities for quite some time until we finally reached a breaking point. I believe much more of our bandwidth should be spent getting ready for the next generations of models on security, monitoring, preparedness, safety, adversarial, robustness, super alignment, confidentiality, social impact, and related topics.

18:40These problems are quite hard to get right, and I am concerned that we aren't on a trajectory to get there. Over the past few months, my team has been sailing against the wind. Sometimes we were struggling for compute and it was getting harder and harder to get this crucial research done. And he goes on and he says, OpenAI must become a safety first AGI company. So first, before we get into a little bit deeper what's going on here, what did you think when you saw his deeper explanation that you had called for on Friday? It just actually pushed me back into the what is actually happening behind the scenes.

19:16That's all I can think about. Because, again, open AI moving too quickly, maybe like the, and we'll get into using the voice of actresses that they don't have proper licensing for. Google launching products out into the wild that tell you to eat rocks. I get a lot of this stuff is not great. And a lot of this stuff is it's risky behavior and it's going to require taking risk. But the ideas of like super alignment, AGI, like what has them afraid? Is it just personal problems and kind of infighting with Sam Altman? Or is there really, really something that they have seen that they are, that they, that they're worried about, that they're like, that they're willing to very publicly leave about, I still don't understand what's going on there.

20:10So here's the deal. There's two possibilities here. One is that Jan and Ilya, Jan Leike and Ilya Sitzkever saw something pretty scary in what was going on within OpenAI and wanted to raise the alarm and ultimately just found like they were hitting a wall internally and nothing has happened. And like you mentioned last week, these people are basically folks that have been dedicated to making AI safe. So, you know, if they didn't talk specifically about what they saw, that that's kind of weird. The other possibility, so I guess I'm sort of biasing in terms of what I believe, because I think the other possibility is quite likely.

20:48The other possibility is that we are still at this elementary level of LLMs. And yeah, it's going to be powerful for enterprises and powerful for businesses. And who knows, maybe it will like be useful in search one day and we'll make Mark Zuckerberg a bunch of money. But it's not anywhere close to achieving supernatural or superhuman capabilities. And then these folks just kind of got frustrated after OpenAI was basically like, can you justify what you're doing? And they kept yelling safety without any like practical application. And this isn't just me. This is Jan LeCun, who's the chief AI scientist at Facebook.

21:23He says, it seems to me that before urgently figuring out how to control AI systems much smarter than us, we need to have the beginning of a hint of a design for a system smarter than a house cat. Such a sense of urgency reveals an extremely distorted view of reality. No wonder the more based members of the organization seek to marginalize the super alignment group. It's as if someone said in 1925, we urgently need to figure out how to control aircrafts that can transport hundreds of passengers at near the speed of sound over the oceans. It would have been difficult to make those long-haul passenger jets safe before the turbojet was invented and before any aircraft had crossed the Atlantic nonstop.

22:03Yet, we can now fly halfway around the world on two engine jets in complete safety. It didn't require some sort of magical recipe for safety. It took decades of engineering, careful engineering and iterative refinements. What do you think, Rajan? Yeah, I loved that quote from Jan. Like, again, the idea of after the first flight saying like, we need to start preparing for safety for this far off vision versus actually just getting there and along the way, having the right people very carefully and, I mean, you know, the economic and capitalist incentives to not have planes that crash all the time, though Boeing, as of today, you know, it's a different story right now.

22:47But I think, I think it's not because of some oversight in the 1920s that doors are flying off of Boeing airplanes. Yeah, it's the exact point. And I actually, it's kind of been fascinating to me to see how this develops. And all right, I think maybe I am liking this because it seems a little more sometimes uh it's not some they q star they saw like uh llm come to life and start running around the open ai office and like just something crazy happening but it actually it's mundane organizational politics that they have this vision and required resources and in reality at this point uh when google's generative search is still telling us to eat rocks and uh even open ai and chat gpt is temperamental sometimes but just gives just doesn't respond the way and as smartly as it could you're right maybe that is they just weren't getting resources anymore and then yeah they it's in their incentive to still go out making a big deal about it and still reminding us that safety is a big priority it is interesting to me in this whole, if that is the case, like, are we in good hands right now?

24:00Are we in the right hands? Because one thing I always think about is the development of social media in the early 2010s, there was no one responsible at the wheel. Like no one was thinking about safety of information. No one was saying it was just pure Chamath style at Facebook as head of growth, like growth. It was just as fast as possible, as tech driven as possible with no societal awareness versus at least now that conversation, every company is thinking about it. Everyone is thinking about it. Again, Google is being mocked as we speak because of issues. So I don't know. Do you think we're in good hands and we're in the right hands to steer us through this if the Jan-Lakoon framing is the right one?

24:47Yeah, I would say that we're in a very different situation than we were with social media where basically one platform or two platforms controlled everything. Whereas like this is much, much more distributed. And we already have like an open source social media model. What do you do with that? Nothing. You need a network effect. Whereas with AI, you can use open source to build things. You can use open source to sort of distribute the impact here. And so it's much, much, much less likely that this is going to be something that's controlled by, let's say, an open AI in Microsoft alone or controlled by Amazon or a meta or a Google.

25:24It's going to be distributed. And therefore, I think that that has a better chance of landing in, you know, with reasonable application versus this sort of crapshoot when you just have one company or two companies control things. So to me, I think that's that is probably where we are now. Like the question is, like, are we in good hands with OpenAI? I don't know. I mean, they have definitely been very quick to release things that everybody else has not. and so far as far as I can tell the world is still standing and we haven't had like mad uh you know mass uh hysteria and madness due to people's exposure to chatbots and people are this is one of the things with misinformation and AI safety uh and even this Google rock thing I think people are often much smarter than uh a lot of the commentators give them credit for being like if Google tells you to eat rocks you're not going to eat rocks if an AI tells you that it wants to take your wife away you're not going to go to your wife and be like excuse you know you should go date bing right now because the ai told me to like people actually do have good defenses against this stuff and so far so good i'm not sure if i'm gonna fully agree there i think i have a feeling there's probably people eating rocks as we speak just from seeing the tweet because it was funny and then deciding i'm gonna eat a small rock but they're doing it ironically right so they're not doing it because Google told them to.

26:51Actually, that is fair. What does that say about humanity? But I guess in terms of that good hands, OpenAI, certainly their behavior this week, I think there's a couple of things that we saw that definitely dig into the question of, are we in good hands? Right. Yeah, so that is a perfect lead into this next bit of things where, okay, so first of all, just I'll set it up a little bit and then I'm going to hand to you. But basically, there is a vertical on Vox. It's called Future Perfect. And it's inspired by effective altruism, which to me is a very interesting thing that there's actually a part of a publication that is like says it's EA inspired.

27:33Like, I don't realize what like that's very bizarre to me. I still haven't fully wrapped my head around it. But they have a great reporter there, Kelsey Piper, who then went on after the I resigned tweet to dig up why all these OpenAI ex-safety people have been so quiet and said nothing about their concerns. And that is, you know, thinking about whether we're in good hands or not and whether OpenAI really should be trusted for handling like the safety of this stuff is a little bit concerning. So what happened, Rajon? Yeah. So when they leave, there's a nondisclosure agreement where there's a provision that you can never speak ill about, you can never talk about the things you did or saw or speak negatively about OpenAI forever.

28:22Otherwise, even the vested equity that you had and certainly unvested equity will be clawed back. And you can imagine for anyone who sat at OpenAI at any time, especially if you're an early employee. That's a lot of money for a lot of people, especially at the current valuation. What is it,$90 billion or$95 billion? You imagine you're someone who was there when it was in the tens or hundreds of millions even. That's insane amounts of money. So people have kept quiet. And already the whole world around non-compete agreements, NDAs around when you leave a company. It's already been one that there's been a lot of chatter around, are these ethical?

29:06Are these the right way to go? But this has almost never been seen to have an NDA in perpetuity preventing you from talking badly about a company. I mean, you report on finance, you've been in the finance world. Has there ever been a situation where you can get your vested shares clawed back if you don't sign an NDA and a non-disparagement? that to me is nuts i mean these people work for you for years you're gonna claw back the shares because they won't sign an additional document about disparagement that's no no but see that part to me for some kind of time period actually does make sense and that's the kind of thing you'll see this because especially if you have proprietary information when you leave a company even if it's vested equity then for a private company that to me is actually not totally unreasonable that for a one year period or something that you cannot use any of the in proprietary knowledge you had to speak ill about the company that is not that crazy to me because again it's like you are you're and you almost have the incentive to not do that because otherwise your equity is going to go to shit anyway so i think like that part is not the it's the in perpetuity it's the that's the kind of thing you never see.

30:24Right. And this was a line that stood out, stood out to me. And I'm curious what you think about it. So Kelsey writes, all of this is highly ironic for a company that initially advertised itself as open AI, that is as committed in its mission statements to building powerful systems in a transparent and accountable manner. So Rontan, you tell me this, I mean, is this like a disproof of the fact that they, cause they, they will still, and they do talk a big game on safety is this almost like a refutation of their interest in that if they're you know you know obviously non-profit dedicated to advancing ai safely but don't freaking say a word or your equity's gone yeah no i mean that i feel i think we had talked about they should just rename themselves to chat gpt as a corporation like the pretense of openness and helping the world and all that stuff i feel is so long gone um that to me this is just another sign that they're a ruthless corporation and that is no but but that has and ruthlessly been delivering excellent products and you know finding customers and doing pushing the industry forward which is fine and it's just just again to move away from this whole open ai thing but also and it is a sign that not only are they a ruthless corporation.

31:45They're among the most ruthless to have these kind of NDAs. Now, here is another thorny ethical question for you that's not as easy to answer, which is, is it incumbent upon the researchers on these safety and alignment teams to give up their equity, not sign these non-disparagements? And if they're really concerned about what this AI is going to do to us, then to actually go out and say it. Absolutely. 100%. If you know that you have worked on something that is potentially going to end and kill all of humanity, I think giving up your shares that will be worthless when we are all taken over by robots and artificial intelligence is probably a reasonable thing to ask ethically of people who claim to care about safety anyway.