Mark Zuckerberg’s Personal Superintelligence, Layoffs and Payoffs, Writing With AI — With M.G. Siegler

26:51Is your enterprise ready for the future of Vigentic AI? Visit agency.org to explore use cases now. That's A-G-N-T-C-Y dot O-R-G. And we're back here on Big Technology Podcast, M.G. Siegler, Monday of first Monday of the month edition. You can find M.G.'s writing at spyglass.org. And speaking of the definition of super intelligence, MG, you actually have a piece on this. The endless rebranding of AI. Basically saying that for a while AI was good enough. Then people went to Gen AI. Then they started talking about AGI. And now AGI isn't enough. And now we're talking about super intelligence. What's next?

27:35What does super intelligence mean? Or is it purely marketing hype? I mean, this is so obviously the term has been around for a bit. It first really came on my radar, I think, actually, when Ilya Sutskiver spun out and made his company called Safe Super Intelligence. Right. And presumably he did that. Obviously, he did that on purpose, but presumably he did it also to help differentiate from what OpenAI was doing. Right. They were at the time it was all the talk was the march towards AGI. Famously now they had it baked into their contract with Microsoft, which is still an ongoing dispute, it sounds like.

28:12And so, you know, they're deciding like, OK, well, AGI nebulously defined as sort of computers reaching human level intelligence. Though some would disagree with that. And then superintelligence is a level above that. Right. It's it's it's going beyond what humans are capable of. Again, there are, depending on what you read, depending on who you talk to, people disagree about this. No, the part of the problem is no one ever really defined what AGI itself was. And then, you know, going back again, as you did, like with AI itself, it's like that was the all encompassing thing. But then we got all different sorts of like subsections of it because people were wanting to be more focused on and they thought AI was too nebulous a term.

28:57and then AGI came along and then AGI was too nebulous a term. And so now superintelligence and now superintelligence is too nebulous a term because we have personal superintelligence and we have safe superintelligence and Microsoft is working on their own variants of superintelligence that they want to talk about. And so it's getting a little silly. It just is obviously all marketing at this point. And so I do feel like that, you know, you got to at some point pin people down to say, like, what are you actually trying to build and what what is the outcome of that going to be? Yeah, I think we can agree here.

29:37And this is something that we've been talking about in the first half. It's the same damn thing. It's the same thing. Everyone's building the same thing. But people would argue with you about that, right? Because they'll say, no, AGI, again, is sort of human level intelligence. And we're close to that maybe, but we haven't quite gotten there yet with all the various math tests and the different tests that they've built to prove that. And regular super intelligence is just a level beyond that. There's lots of people who would argue about that and lots of people who would argue for that they're the same exact thing.

30:09I'll stand on the table here and say it's the same thing. I think it's, if there were different techniques, I think this is really what it comes down to. If there were different techniques to build AGI and superintelligence, I would say, OK, sure, you know, Facebook's building superintelligence over here and OpenAI is stuck building, you know, stale old AGI over there. But they're just training large language models. Yeah. And it's wild that we're trying to define the sort of offshoots of any of them when we haven't defined the first part of it and dating back to the, you know, original sort of definitions of these things.

30:49So it's a, it's a little bit silly. And that's obviously why I wrote the post once I saw that Microsoft, you know, was jumping in the ring as well to try to, to come up with their own individual branding for super intelligence. And then of course you could see it. So Zuckerberg's new memo that we're talking about, it's titled that, right. But he started dropping this, the first time I heard him mention this was in an, uh, an interview with Jessica Lesson of the information where he just started like to casually mention personal super intelligence. And it's like, Oh, did he, is he just trying to like say something offhandedly about how they're, you know, they have a sort of micro focus, but no, he just kept saying it over and over again that they clearly were going to try to make this a branding thing for meta.

31:33Yeah. I mean, it's, I think that it's very exciting when the word super is in there. I mean, really. All right. When are we going to get ultra intelligence? I'm ready for ultra. Super ultra mega intelligence. It's going to just blow away what we had. Remember when we were talking about AGI? Forget that. We were such losers back then. Now super mega ultra intelligence. That's where it's at. One interesting thing I saw in your story was that Microsoft cannot actually work on AGI legally. This is what you write. The pesky OpenAI contract says as much, which sheds light on the talking points that Mustafa Suleiman delivered over the past year or so, that Microsoft is happy to see the frontier of model work to OpenAI.

32:13So maybe this is meaningful that like, well, Microsoft can't build AGI because OpenAI is doing it, but it can work on superintelligence because that's not in the contract. Yeah. And so this, I feel like, is definitely under talked about. It's been reported before, but I feel like now because OpenAI and Microsoft are maybe in the midst, in the heat of a new negotiation over the AGI term and how that's going to play out for their relationship going forward, it has come to light again that, yeah, apparently a part of their original contract with the quote-unquote clause in it around AGI was that Microsoft said that they would not themselves go to work on the AGI.

32:59go after AGI and working on that. And that, in hindsight, brings up all sorts of interesting points. I think when you've talked to even Mustafa Suleiman, he's brought up the point of like, yeah, we're happy to let open AI work on the cutting edge foundation models. And we'll just sit back and work on sort of more tailored, custom tailored one. When the reality was, yeah, they're technically not allowed to work at the cutting edge if it's going to lead to AGI. Now they could say maybe it wouldn't, but, you know, everyone else would probably disagree with that. And I also think at an even higher level, part of the reason why I would imagine that you even saw Sam Altman sort of pivot his talking points from AGI to super intelligence at one point in various memos that he's written himself was because of, yeah, the clause in the Microsoft contract.

33:45And it was clearly a point of tension, you know, between those two companies. And it's like you've heard Satya Nadella talk on, I think, on podcast about like the, you know, I think at one point he called it ridiculous, the notion of sort of that we were close to AGI, right? Because obviously he's very incentivized for them not to be close to that because it would potentially sever the business terms with OpenAI. And so it was almost like was Sam Altman sort of throwing him a bone by starting to talk about other things other than AGI and rebranding as super intelligence. But, yeah, if the clause is in there originally that Microsoft is not able to work on AGI and now they're happy to have this rebranding, too, where Mustafa Suleiman can work on all the variants of super intelligence that he wants to because it's not AGI.

34:34It sort of brings us to this point that we're talking about how Microsoft and OpenAI are linked and they have these like weird contractual terms. And you also point out that something that's not talked about enough is just that big tech owns so much of this AI moment, even if they go by, you know, different names. So this is the list that you put together. NVIDIA owns stakes in OpenAI, XAI, Mistral, Perplexity, Cohere, Scale AI. Oh, that's a bunch. I didn't realize they had stakes in all of that. It's even more than that. Those are just like the major ones that I was pulling out. They own so many stakes in everyone.

35:09So Microsoft owns stakes in OpenAI and Mistral. Apple owns stakes in none of the big AI startups. But, you know, there's potential for, you know, maybe perplexity. OK, I'm going to let go of that dream at this point. Google owns a stake in Anthropic. Amazon owns a stake in Anthropic. Meta owns stakes in scale AI and safe superintelligence now. And Oracle owns a stake in Coher. You say this is a lot of hedging. Does that mean basically that like if this AI moment takes off, like big tech is just going to be, you know, they're going to be good, whether it happens in a start, whether they build super intelligence at home at Meta or whether they a startup actually is able to do this?

35:49I mean, obviously, to be clear, they want to do it themselves, right? And they want to capture that full opportunity. But it is interesting to me that this isn't talked about all that often, except around, you know, when new fundraisers happen, in particular with Anthropic, right? Because Amazon and Google own such relative large portions of that company. And famously, Anthropic, you know, apparently built into their fundraising documents that those companies can only own up to a certain threshold, because they didn't want to be beholden. Famously, when Dario left OpenAI, there was some talk that he didn't like the Microsoft element getting involved with OpenAI and what that was going to do to the mission.

36:32But yeah, all the big tech companies, aside from Apple, as mentioned, basically have their stakes in these different companies, and some of them with multiple stakes now because these companies have had to raise so much money. And I do think if you were just to sort of extrapolate out in the ultimate success state of AI and say that OpenAI and Anthropic and maybe a couple others are the next multi-trillion dollar companies. If, say, Microsoft, again, in the negotiations that they're going through right now, if they end up owning one third of open AI and it's one day worth$10 trillion, that's meaningful money even to Microsoft.

37:15It's already meaningful money to them because that company is so highly valued. But it's not more meaningful than a lot of their other sort of core businesses. But again, this is relative early days. If you believe this is early days of AI and these players own these giant stakes, it will end up being a huge, huge element of those companies. And I was reminded of when back in the day when Microsoft invested in Facebook before they went public and people were like up in arms about the deal, thinking it was such a bad deal. Like Steve Ballmer was doing another boneheaded thing and it being one of the best deals that he ever struck.

37:51certainly, you know, easy, of course, to say in hindsight, but I think it was a savvy move, you know, because they did it alongside a partnership. I think it was involved with ad sales and various other things at the time. Maybe Bing was involved in it as well. It was a long time ago. But I do sort of view it through that lens now. Their mistake was selling off that stake in Facebook, you know, a little bit after perhaps it went public, whereas now that would be worth billions and billions of dollars. And again, if Microsoft, if Google, if Amazon, hold on to these AI stakes and these companies end up being worth trillions of dollars, these stakes themselves could be worth trillions of dollars down the road.

38:29Yep. You always got to get Bing involved. That's important. I can't leave Bing out of the equation. But it's interesting because it really underscores how much money, resources, attention. Like we just spoke about how much money and attention Zuckerberg is putting into this. We talked about, of course, like these massive investments, like when we talk about Google has some of Anthropic, Amazon has some of Anthropic, actually, it's, I think, three and eight billion dollars respectively. They own something like you had this in your story, something like 30 percent or more of the company. Actually, Google and Amazon own more of Anthropic than Microsoft owns of OpenAI.

39:08So it's wild. And I just wonder, I mean, if we go back to like everyone building the same thing. Is there a way that these bets pay off? Like, what would it take? So you've spent a lot of time as an investor. What would it take for these bets to pay off? Like, what do we need to see for this money to be money well spent? So and just to be clear, yeah, Google and Amazon combined, I believe if my math is correct from all these court documents that I sort of went through, but I believe it's it's roughly correct that I think them combined those two combined own something just north of 36%, whereas Microsoft, again, potentially in this scenario, would own 33 % or around that percent of OpenAI if and when they can convert to actually have equity ownership in the company.

39:57To answer your question, though, what would it take? It takes a business model, first and foremost. Both OpenAI and Anthropik certainly, it seems like, are growing well, you know, from an ARR perspective, you know, their models are obviously slightly different, but they're doing a good job in different markets sort of selling their wares. But it's certainly not enough to turn either one of them profitable at the moment. And so they're going to need, they're going to have to do a number of different things. Either they have to keep growing at the incredible growth rates that they've been growing at, and that's, you know, possible up to a certain extent, more likely that they're going to have to add other layers to their business, other stools of their business to figure out how to make sort of the money work.

40:47I think the main wildcard, though, is much like Google back in the day. It might end up being that these companies in the ultimate success state, if they're able to reach it, end up creating sort of new models and not necessarily new that we've never heard of before, but much like Google and even, you know, meta to some extent, right, they basically created new forms of advertising that ended up being more lucrative than previous forms of advertising, because of search ads, because of intent, right, and then with meta because of the feed. And so these are different forms of an old school version of monetization.

41:21But so what does that look like in the future version of AI? And, you know, open AI is famously sort of said, right, that Sam didn't want to go down the path of advertising. And I mean, I would chalk that up to I do think that they will eventually go down that path. But I also chalk that up to the fact that, again, I think that if they go down that path, and if it's meaningful to them, it will look different than what sort of search advertising has historically looked like. I think there will be a part of that because obviously, search is a part of ChatGPT right now. But I think it just probably has to look different.

41:53And again, you ask about what it takes to make these companies ultimately successful. I think it's going to be multi sort of stools of their business. So not just selling APIs or not just selling access, you know, in premium versions and whatnot of their products. I think that they're going to have to have a lot of different things, including probably devices, right? Obviously, we know OpenAI is working on that. But I do think that there will be different flavors of advertising, newfangled versions of different old business models is what leads to their success. With the investor hat on, with the VC hat on, how do you try to model that out?

42:30I mean, again, you can model out where you think the growth rate is, you know, just of the current business and what that looks like. And I think OpenAI has said in their models that they're going to get to profitability by 2029. Again, having had some experience now with OpenAI in particular on the investor side, but with other companies, just, you know, generic X companies, say those models that are five year, 10 year horizon models, those never play out the way that they're modeled out to. Now, there can be upside to that. Maybe they play out better in some cases, but they never exactly like it.

43:06Right. It's just it's just to try to paint a picture of how you can get to a space where you could say be profitable, even when you're burning billions and billions of dollars. This is just the extreme version of that with AI. Right. So I spoke with Ravi Matra from Lightspeed for the Dario profile that I wrote. And I said, tell me how you thought about this investment. And I'm going to get it like just directionally accurate. It's not going to be precise. But he basically is like, well, you know, we took a look at the entire pool for like knowledge labor and then said, well, could this just kind of like, you know, either augment or replace that?

43:41Yeah. And he said, yeah, the market is 15 to 20 trillion dollars. He goes, You work backwards and just say at 60 billion or 100 billion, could you get a venture style return? You absolutely could. Sometimes it's about how you size the markets top down. And then he told me that they put a billion dollar check into Anthropik. So it's just fascinating how these how these it's it's so unknown and and such wild projections to get to the numbers you have to get to. I'll give you one just example of like, you know, an last generation of company that sort of this this played out at that, you know, that I was a small part of, which was, you know, around an Uber investment where it's like you're looking at the market.

44:22They just had the black cab market at that point. And it's like, how do you do what you're talking about? Basically, the TAM analysis, right? The total addressable market. And how do you think about that? And with Uber back in the day, it always had to be like that there would be something else to it. And, you know, there was the notion, I think the early notion of what UberX would become where it, you know, more democratizes the model. And it's not just about black cars, but it wasn't fully baked yet. It wasn't rolled out yet. And so you sort of had to buy into this vision of it. It ended up obviously working out that way.

44:54But there still were other elements that were key to what they were doing, including things like Uber Eats, right, which came along, which you never would have sort of envisioned was necessarily a part of it. There was some inkling like, yeah, could they be like the next version of, you know, again, like a democratized FedEx or something like that, right? Because they have all these drivers on the road and you can move parcels around and move different things around. But it's all just theoretical. And so, yeah, you try to do like this, these market analysis and figure it out. But it's all very silly to look back upon at the end of the day because no one actually knows, including the companies, right?

45:33It almost always goes back to the notion of like, build a good product. If people love using it, they're going to continue using it. And that doesn't necessarily mean that you're going to figure out the right model for it. But if you have enough users and you figure out a way to monetize those users, it's going to be a massive opportunity, obviously. And at least the good news for these AI companies, unlike some of the other sort of, if you want to call them VC subsidized businesses back in the day, they have models that are working, right? There were so many different companies even just a few years ago where, again, it was VC subsidized.

46:10They had no actual models. And it was just like, let's get to scale, dot, dot, dot, profit thing, right? And now at the very least we have money coming in and it's just a matter of are they going to be able to ever slow down the spend that they have to do to train these models in order to get to a scale at which they can actually finally sort of equalize that a bit. And in that context, how would you analyze the investments in people that they're making? because think of it this way. It's something a mantra told me. Amazon went public at$400 million. Mark Zuckerberg is handing reportedly billion-dollar offers to people.

46:55Yep, yep. So what would you need to do to get the payoff on that? This is where I am sympathetic to Zuckerberg's arguments. You know, he's made the argument that basically, it essentially boils down to, and I wrote a post about this, that it's all relative, right? Like, okay, if you are meta and you're spending whatever it is,$75 billion on CapEx right now per year on this infrastructure build out for AI, does it not make sense to spend some subset of that on the people who can actually make this work? And then it's just, it sort of becomes like this math equation of like, well, what makes sense?

47:36And you could certainly make the argument that to date, the human level of investment has been way under index versus what the, again, the capex or the infrastructure has been. And it's a weird dynamic, right, for reasons that we were talking about before, where he's creating this weird disturbance in the market because it's totally changing, you know, what the historical norms have been in terms of compensation. So you can have people who are, you know, getting paid what the quote unquote normal rate was for an engineer before, which is say in the hundreds of thousands of dollars at one of these tech companies.

48:12And now you have people getting paid hundreds of millions of dollars potentially to come work on this. And again, trying to be sympathetic to Zuckerberg's position here, if you're needing to catch up in AI, you're spending$75 billion plus a year on infrastructure. The amount that you're spending on people is going to be, even with these$100 million contracts, is going to be such a small subset of what you're spending on infrastructure. and if they're a key ingredient to making it work, you got to do it. Right. And so we've already poured one out to the poor engineers who are working on the metaverse.

48:46But part of the title of this episode is layoffs and payoffs. And what do you make of the fact that we're seeing this moment in the tech industry where literally tens of thousands of people are being laid off as these mega offers come in? It seems like they don't really need to lay off those people from a financial standpoint. So why would they cut so many? I mean, thinking about Microsoft in particular. Yeah. And because Satya Nadella wrote a memo about this, which he released the internal memo, which he released publicly, I'm sure, because it was going to leak. And so, you know, they had they wanted to get ahead of this anyway.

49:23But he's basically trying to explain they've done. I think the number is in total now, 15 ,000 people have been laid off from Microsoft over a couple rounds or a few rounds this year. And that's, you know, again, in the same time as we're talking about that they're that they're spending insane amount of money on CapEx and and just to hire people like Mustafa Suleiman to come over, you know, and acquiring his company. And so how do you sort of square that circle? And Satya Nadella, you know, basically tried to do it, sort of tried to do it. He called it an enigma, I think, of a time and that it doesn't make a lot of sense on paper.

50:00But that basically, you know, we're in this transformational moment with AI, yada, yada, what everyone says. But I do think like there's a number of things going on here one level deeper, which is, one, I do think that we're businesses in general are sort of looking at the landscape and figuring out, do they need all of these people, tech companies in particular now, almost all of them are around 100 ,000 or more people, you know, on their payroll. And a lot of those people are doing things which, you know, the unfortunate truth is that the, you know, the internal powers that be, the leadership infrastructure probably views them as like, not necessarily, they're not bad people, but they're doing stuff that was last year's innovations, right?

50:41Or they're, they're doing stuff that they they might think that AI is able to eventually replace if not now, but sooner rather than later. I also think that there's a part of it where, look, they know that Wall Street at some point is going to get skittish on this CapEx spend. And especially if the AI revenue isn't, you know, growing in in concert with it at some point soon. And so I do think that this is a way to throw a bone to Wall Street and say, like, look, even though, you know, it looks like that We're being sort of wild on our spend when it comes to CapEx. We are being prudent financially.

51:13We're thinking about how we basically organize the business overall. And that includes letting go of some of the people that we've had previously that we feel like we don't need anymore while we continue to bring on new people that we do feel like we need from the AI perspective and building that out. And so the weird things of how this plays out to me, it's almost like a microcosm, right, of like the overall debate being had about, you know, as I titled my thing, like sort of the haves and have-nots. That's a great title. No, you write the have-lots and the have-nots, which I think is just – it really nails it.

51:52Because it's above just having. It's having more money than – as I put it, I think, at one point. It's basically they're getting offers more money than anyone gets offered for any position, including the CEOs of these companies. Maybe there's some star athletes who make something comparable when you consider their endorsement deals alongside whatever they make in salary. But it's just an incredible amount of money. And how do you weigh that against these layoffs? And again, I think of it as sort of a microcosm of the world in which we live, in which there's the extremes of society with the ultra billionaires while people are struggling on the ground on a daily basis.

52:30And the microcosm that can be within a company itself, what does that lead to in terms of internal politics and weird intercompany dynamics? And I think that's part of why I think Satya Nadella and others have written that basic memo of like, I think that they're trying to get ahead of that. And also, lastly, I would just say, because we've seen this a few times now, and I think from Andy Jassy and others, they're basically giving the message that, look, you've got to incorporate AI into what you do or you've got to get off board right now because this is coming and this is how we're all going to do business.

53:04So do it right now or else you're gone. Yep. You go full tale of two cities at the beginning of that piece. It was the best of times. It was the worst of times. I mean, that is the way to start that piece. Okay. So let's end here. AI for writing. It's pretty controversial. I think AI is getting to the point where it's starting to write adequately. Not well, but adequately. I don't use it for writing, but when I read AI written stuff, I no longer have a gag reflex. But you write, I think, something very, very spot on about what it means for our society or our lives. And it's that writing is not just a way to communicate information.

53:46It's a way to organize your thoughts. It's a way to think. And so we focus so much on the output of AI writing, but very little on the input. And that's where it's going to hurt if people start relying entirely on these AI tools to convey their thoughts. So kind of take us through your thinking there and talk a little bit about whether you think AI writing is going to be a net positive or negative. Yeah, this this came from me just trying to extrapolate out how I would view using AI to write, which I don't like you. I don't right now. And it's like, would I ever and I was trying to think through like, yeah, how good is the technology now?

54:22And to your point, it seems like, you know, it's getting good enough for a lot of different types of writing. And I've also been thinking a lot about one thing I hate is email, as many people do, but I've long written about my sort of dislike of it. A long time ago, 15 years ago, I quit for a long time doing email. And I wrote about it in TechCrunch when I wrote there back in the day. And I got a lot of flack for that story. But I think it also resonated with people because it's like, well, you actually need to do it. And I think we're getting to the point with AI where you probably don't need to do it anymore, right?

54:52Like because you could basically have AI write your emails for you. And then I do think eventually the way that it plays out is that then you have AI reading those emails on the other end. And sort of it ends up these bots sort of writing to other bots and then sending you like a to-do list of whatever needs the action items need to be from that. And so I was thinking about it from that perspective. But then I go to, you know, my actual love of writing and sort of what I do and why I love to do it. And I would never use AI to write, say, an article about even AI for me because that would take away the value I get out of actually writing, which is writing like many people.

55:31It helps me think. It helps me form my own thoughts. And so I do think that that's true of a lot of people, even if they don't necessarily realize it. I think that people who write for a living or write a lot do realize that real element to it. But I think people who don't probably gloss over that fact. But I, you know, I do think like as a society, I would imagine that it plays out in a way where maybe there's some level of writing like email, like the tedious stuff that does get automated away. But the sort of certainly the creative endeavors, but also even just, yeah, like the the memos that that help you sort of formulate your own thoughts.

56:10Those don't go away because it's just about the process. It's just as much about the process of writing as it is about what you put down on paper, what goes on in your head and how you formulate those thoughts. It does make me wonder about whether, you know, I see the value, obviously, in writing, but I also know what's happening right now. where like there's these great screenshots that go around Twitter of like professors scorning their students for like being late on their assignment and ending it with chat GPT. Ask chat GPT. Like they're copying and pasting it directly out of the chat bot.

56:44And those students are going to be like, OK, well, here's the assignment. And then they're going to paste their assignment and it's going to say ask chat GPT. So we're just going to be a society, I think, of just people shuffling AI written text back and forth, which like give it. I mean, maybe you're right, given the amount of tedious stuff that we've been writing in our lives, that might not be the worst thing. But I think there is that risk of the fact that, like, yeah, we're not going to think as deeply about things anymore without writing. Look, the last thing I would say about that is student cheating is nothing new.

57:15It's always existed, right? Just began with large language models. Calculators and everything, you know, before that. And so I think ultimately, you know, that will sort of suss itself out in some way. I do think it's a real issue, obviously, because of how good these LLMs are and how new this, you know, this this whole wave is right now. And so it's going to take a lot of learning curve to get to the point where we sort of, again, reach an equilibrium of where it where it makes sense, where it doesn't. But I do think at the end of the day, like. if you're doing something for your own, you know, purposes, be it writing, be it many other different elements of life, like you're only robbing yourself if you're using like a tool, you know, to do it in an automated fashion.

57:59So yeah, for maybe for some writing assignments that you don't want to do, fine, you didn't want to do them, that's fine. But ultimately, there's going to be something that you do want to do, or that you get value out of that, I think that that will work itself out. I agree completely. All right, folks, the website is spyglass.org. M.G. Siegler joins us the first Monday of every month. This is part two of what I hope will be a long series. M.G., great to see you again. Thanks for joining. Thanks so much, Alex. Talk soon. All right, everybody. Thank you for listening. Amjad Massad, the CEO of Replit, will be on with us on Wednesday to talk about vibe coding.

58:35What is it? Will it work in the long run? Is it sustainable? and what will it change. So stay tuned for that and we'll see you next time on Big Technology Podcast.