IBM’s $10 billion bet on what comes after AI

0:00The very best founders I know are brilliant at building systems. They connect teams, they remove bottlenecks, and they eliminate single points of failure. And yet, when it comes to their own wealth, most are running a disconnected stack. A tax accountant here and a state attorney there, a wealth manager who doesn't talk to either one of them. Creative planning was built to fix exactly that. One integrated team of tax professionals, estate planners, investment specialists, all coordinated by a dedicated wealth manager who sees your full financial picture and keeps every piece working together. Proactive tax efficiency, estate strategy, investments all under one roof.

0:44Creative planning where wealth works together. Learn more at creativeplanning.com slash mastersofscale. Hey, folks. Jeff Berman here. Exciting news. Applications are now open for the Masters of Scale Summit. It's happening October 20th through October 22nd in San Francisco, and it is a really special event. Please join our curated community of founders, innovators, and leaders shaping the future. Expect ideas that challenge your assumptions and connections that move your business and maybe even your life forward. It's an experience that can change literally everything. Apply now at mastersofscale.com slash apply26.

1:30That's mastersofscale.com slash apply26. Humans will never be more intelligent than AI. There's going to be two types of companies. Those are great at AI and those that went out of business because they weren't. How do we build a future that is human-centered? I'm Rana Elkhaubi, and on my podcast, Pioneers of AI, we answer that question and so many more. As an AI scientist, entrepreneur, and investor, I know what it takes to build AI that works for everyone. Every week, I sit down with the pioneers shaping our future, and we take you behind the scenes of the AI that's transforming our lives. Find pioneers of AI wherever you tune in.

2:23I'll say something provocative. I think foundation models are going to become commodities. I think that right now the token price on all of these is going to go way up. It just has to justify the capital investments. That's Arvind Krishna, CEO of IBM. And he has a strong metaphor for current AI systems that are just not the right size for all uses. I guess you could, for those in the suburbs, take your kids to school in an 18-wheeler every morning. You could go milk shopping in an 18-wheeler. Then you'd ask yourself, is it really the most effective vehicle for that? I think right now we're using the 18-wheeler for everything.

3:05This is Masters of Scale. I'm Bob Safian, your host. IBM is playing a distinctive role in the AI race, not building AI models, but betting on how best to use them and on what comes after them. In this conversation, recorded in front of a live audience as part of New York Tech Week at IBM's Manhattan HQ, we dig into why Arvin thinks most enterprises are using an 18-wheeler for every task. Plus, what kind of risk-taking businesses need to take right now, how to think about costs versus benefits when implementing AI, IBM's big bet on quantum computing, and much, much more.

3:47Please welcome to the stage IBM Chairman and CEO, Arvind Krishna.

3:58Arvind, first of all, thank you for hosting us at your house. We're together as part of New York Tech Week. IBM is a New York institution. It is a global institution. As a company, it's had to continually reinvent itself, from mainframes to PCs and consulting and cloud, and now AI and on the cusp of quantum computing. How do you think about staying fresh as tech moves? And are there things about IBM's legacy that are an advantage versus what isn't an advantage? Yeah, look, the advantage is client intimacy, knowing your clients. The advantages are around trust. I don't think we have knowingly ever done anything wrong with a client's IP or data or people.

4:53Those are advantages. I think our people are incredibly technically adept. I would say they're experts in the areas where they have spent time and energy. So those are all the advantages. Now, technology keeps changing. And I'll be the first to acknowledge, Sometimes we are really good at predicting where it's going to go, and we kind of get ahead of the wave and do that. You mentioned a few of those. The mainframe certainly, but this is now 60 years ago, was one of them. I would say the IBM PC may have been another one of them. I think embracing Java in the Internet era was another great one.

5:33And then you sometimes miss them. It's not for actually lack of knowledge. You miss them because the business model doesn't align. You don't quite know how to get the investments and returns to come together. I would say public cloud is one of them, candidly, that we missed. I would actually turn around and say, despite inventing the IBM PC, client server was another one that we missed. So I think it comes into, you could probably miss some. And you kind of know you're going to miss some sometimes? And you know you're going to, I think because if you don't take risk, you're never going to succeed.

6:07So part of it is, hey, I'm getting a lot from that one. I kind of want to keep my focus there. And kind of if you're not two, three years ahead of the wave, you're going to miss the wave. But the whole point then is, can you do enough of those where you can be with the wave as opposed to way behind? Yeah. Because it moves so fast nowadays that if you're two, three years behind, you're not going to catch up. And so right now we are very focused on hybrid cloud. That is sort of our answer to the cloud movement. And AI, where I think that our play is going to be much more of similar to a hybrid play.

6:43How do we help our enterprise clients take advantage of it fully? I mean, the first mover advantage is this sort of catchphrase in Techland. And I was thinking about AI and IBM Watson, and you were sort of ahead in some ways, but you didn't maybe make the splash that you wanted. Was that a missed moment or is it more that, you know, you were too early or the tech wasn't quite ready? There's always all those things. I think when we won Jeopardy with Watson, I think it woke the world up. Because I think for the first time in a long time, AI started doing something people thought it could not do.

7:24Okay. Unfortunately, it woke the world up completely in the sense that a number of other companies started investing very heavily. Now, we had an advantage. We might have been able to succeed, but I'm not putting on a 2020 vision looking backwards. The mistakes we made were actually much more of a strategic nature. As opposed to creating building blocks, we wanted to create solutions in verticals. That, I think, is a mistake as technology shows. In the beginning, it's much more... You went too quickly to the application. We went too quickly, and we wanted to make a monolithic application. Mistake number one.

8:06Mistake number two, we picked a domain, which is perhaps the hottest of them all, which is health. Mistake number three, how many IBMers sell to doctors and how many IBMers deal with the FDA? Like, none. So you picked the wrong solution set. In an industry you know nothing about. And with a customer you know nothing about. Other than that, it was pretty good. So as you think about IBM today and its role in the AI ecosystem, like what is it? I mean, you're not trying to be open AI or anthropic. You're not trying to be Google or Microsoft. like, are you ahead? Are you behind? How do you think about all of that?

8:52So we are not going to be a hyperscaler, which was two of the four you mentioned, and we're not a foundation model provider. I'll say something provocative. I think foundation models are going to become commodities. By the way, not that far out. Is it a year? Is it two years? Is it three years? Commodities doesn't mean that they don't have value. Gold is a commodity. So is iron. So commodities means that there is very little switching cost to go from one to the other. Second, I think that right now the token price on all of these is going to go way up. It just has to justify the capital investments.

9:34You put those two together, and there's going to be a huge motivation then from everybody who's using them to say, I need to optimize, I need to use each one, but in the most economic way possible. Our role is to, A, let our enterprise clients do that. Two, do it in a way that is safe. And right now, there is very little demand for on-premise or smaller models, which are effectively then one hundredth of the cost to run. So I'll use the analogy in a very crude way. In the end, an automobile, which I'll include trucks into it, is an automobile at some gross level if you step back. I guess you could, for those in the suburbs, take your kids to school in an 18-wheeler every morning.

10:21You could go milk shopping in an 18-wheeler. Then you'd ask yourself, is it really the most effective vehicle for that? But if you're moving homes, which you do every seven years on average in the country, it is the most effective vehicle for that. I think right now we're using the 18-wheeler for everything. And this is the transition that I'll predict will happen within 24 months. I'm not sure it'll happen within 12 months. Underlying GPU pricing, which is what all of these things run on, has doubled in the last six months on a por-habo basis. It's getting more expensive to use these tools. And right now, when you're pre-public, it's okay to lose money because you're gearing towards number of customers.

11:05I think you've seen this before, right? It used to be called eyeballs. And then it suddenly became the economics are important. So I think we're right maybe a year from that point. I mean, you said something at the IBM Think event a few weeks ago. You said it's day zero of the AI revolution. And I think for a lot of folks, it feels further along than that. I mean, you've got trillion-dollar AI companies. A lot of business leaders worry that they're falling behind. Does day zero mean like you're not too far behind, you don't have to rush too much? Or what do you mean by that? So first, let me be clear, because I can sound a little bit cynical about the economics, and I actually am.

11:47That said, I think AI is an incredible productivity tool. I think those who don't take advantage of it will be perpetually disadvantaged compared to those who do. So let me begin by saying that. It's going to optimize how you market. It's going to optimize how you write code. It's going to optimize enterprise operations. It's going to optimize how you sell. is going to optimize how you get your daily work done. So there is no question about it. It's going to make a profound and deep impact on all of those things. By day zero, I mean, it's time to sit down, take it seriously. You're not in the experimentation phase.

12:25This is not like you're in high school. Day zero, the race is about to start. Put yourself in the blocks and start sprinting. But by that, I mean, take three, four, five things, not 100, and learn how to do them at scale, because that'll teach you how do you get all your change management done? How do you get your data organized? How do you really get people motivated to change a process? So do a few things at scale, learn how to do that really well, now do 10, and then give yourself the confidence to do the next 20. I mean, there's this expression that's used to talk about the economy these days that's a K-shaped economy, you know, some households do great and some don't as well.

13:09Sometimes I get the sense that when it comes to AI, we're sort of having K-shaped businesses that like the tech companies, folks like you are super excited. And then there are a bunch of other companies and these may be clients of yours, I don't know, that are like falling behind. Yeah. So unfortunately, I think corporate performance is even more of a differentiated K. If you look at corporate performance, actually it tends to be a 20-80 rule, more of a power law. Then the K is really more of a 50-50, I think, if I follow the economists correctly. Here it's more of a 20-80. And so 20 % get it.

13:49They go forward. They're jumping into it. They kind of are going to get their returns. And 80 % are either not getting a return or don't quite know what to do. If you're in that 80, figure out what is it that you should do. that probably doesn't matter where you start as long as you're starting to try to do it at scale to make a real difference to your bottom line. So what you're saying, though, is like, you don't have to know what to do. Like, it's better to pick something and go than to just be like, I'm not sure what to do. Like, I have a client walk up to me and say, look, I get it that I need to do it, but I don't have the right people on my team.

14:25Can you give me a deep AI expert, somebody who's kind of done their PhD in AI? And I looked at them and I said, Actually, I recommend we give you somebody from a domain who doesn't really know the depth of AI, but who understands the difference AI could make to your domain so you don't need to know what to do because those domain experts exist in every company. Find that 20 % or 30 % of them who are motivated to say, I want to learn a new way to do things. So I think curiosity, willingness to adapt is more important. I think we're getting hung up on, I need to know AI like a PhD in computer science.

15:01I think that's the wrong thing because that's for the inventors of AI. That's not needed for the deployers of AI. There's also, you know, this idea that AI is going to save me a lot of money. It's going to be very efficient for me. And I think for a lot of businesses when they start implementing, those results don't necessarily come. Now, you guys have talked about that you've, you know, unlocked four, four and a half billion dollars of efficiency from AI. So you're doing something, but there are also these hidden costs, as you mentioned, tokens. How do you balance what are the costs versus the efficiency and what you should be expecting?

15:41Yeah. So this serves my point of scaling. I would probably turn around and say that for our first six months to a year, we were probably spending more than we were saving. because if you think about you're putting a couple of hundred engineers to work at it, that's an incremental cost. The underlying infrastructure, a.k.a. the tokens, if you're doing it on public, is an added cost. There's opportunity costs also of not doing other things with these people that could have resulted in revenue. That's a cost. Now, once we learn a rinse and repeat method that you're not doing it across two or three, but across 10 or 20, when you're saving a billion dollars a year, Well, that's a lot more than the cost of a couple of hundred people.

16:24After year two, we were definitely getting a return that was 10x compared to what we were spending. And now at year four, we'll be, I think, over$5 billion from a baseline of 22 spent. So that's not an incremental five over last year. That's an incremental saving compared to our year-end 22 spending. So that is tremendous. That is more than enough to offset any extra expense. There was a CEO I was talking to about some of these issues. We were talking about the inexact nature of some of the outputs you get from AI, right? I don't want to call them hallucinations, but, you know, the things that don't go the way you want.

17:03So unlike humans, right? So unlike humans. What he was saying was that the money that he was saving by having his engineers use AI, that on the few cases where it was wrong, he had to spend so much trying to find what was wrong and fix it that he wasn't actually coming out ahead. Yeah. So I actually think that that is an edge case of how you use AI that I think is actually wrong. I think that you should try to use AI in a case where you're not going to have to go undo six months of work or undo having spent hundreds of millions. Take customer service. If it gives a wrong answer, you got to undo one customer service answer.

17:49Then you can put all kind of evaluations and checks. So AI can check itself to make sure that you're not like way off in the wild. It may be slightly off, but you're not way off. So you can put checks and balances, and this is the sophistication of how you use it. So when we use it, for example, for our software developers to help them code, I don't think they realize it. We actually have checks built in to make sure that what it's suggesting is not absolutely crazy. Right? But don't you, I mean, as with a human worker, you have to expect that sometimes it will go wrong. It will go wrong. I kind of turn around.

18:27If I look at customer service, I think the stat, which would be a good one, is 85 % of the time the human people get it right. And 15 % of the time. And I say, of course, humans get angry. Humans get pissed off. Humans are not like the tone of the person on the other end. if it's a call. Humans are sometimes overconfident. I'm sure we all remember things perfectly, right? I mean, perfect. You and I do. I'm not sure everyone else. Pouses have never told us that we are completely in the wrong and remember it completely. So humans have all those too. I think AI is at least, if you keep it constrained to some extent, it's probably 95 % correct.

19:02So the evals I'm talking about is more like saying, when you think your way off, punt it to a human. Don't try to venture into the underconfident range. But I mean, but the AI is always confident. After that, no, it's not actually. You'd be surprised. If you tell it, don't pretend to be confident, you will get from it that, hey, I'm not quite sure that this is as this thing. And you can put something to check it, who's rewarded on actually pointing out the other's mistakes. So now you have - So this is the advantage of having multiple agents, multiple... Multiple models. Multiple models working at the same time.

19:41Just like humans, you put four eyes. We call it four eyes in software development. You put two people. One is coding, one is checking their work. Just like the models, one is doing something, the other is checking its work. But when you have two models instead of two people, does that mean that you need fewer people? I mean, that is one of the... You got some grief a few years back for saying your back office was going to get smaller, which seems like it's small change compared to the things that some other CEOs are saying right now. Do you think there's going to be a lot of job displacement? So I'll address both parts of the question because it's a and.

20:23So our software developers are probably 40 % more productive today than they were two years ago. So I'm not saying it's one month, over two years, they're 40 % more productive. So you could turn down and say, that means you need 40 % fewer developers. We actually tripled our college-level entry hiring this year. Tripled compared to last year. So you say, wait, that seems off. No, because this is what people are missing. If my cost of software development is going down, that means we can make products that were not economically affordable three years ago. If we can do those, we can get more revenue at an appropriate margin.

21:03So why wouldn't I get more people? Because these are value creating. Then there is the 20%, I'll call it, is what you need to run the operation. So is it compliance? Is it accounts payable? Is it procurement? Is it all those things? It's not going to go down to zero, but I would not be surprised if about 30 % of the total headcount in those areas is not needed within a few years. That's the statement I had made and I'm actually still consistent. Note, I just said we tripled our entry level hiring. So while there is some decrease on this side in about 20 % of the enterprise, there is a big increase in the remaining 80 % of the enterprise.

21:42So I actually think net will have increased demand for jobs, but there is some displacement, which is always a little bit painful. And always happens with new technology. And always has happened with new technology. What kind of responsibility do you feel like you have? Do you think other CEOs have helping to ameliorate the displacement that's inevitable? You know, tech folks are very excited about the future because they're beneficiaries of it. But not everyone is a beneficiary in that way. Well, actually, I think that if we can get five to ten points of productivity in every enterprise around the planet, everybody is a beneficiary.

22:18Tech may be the early, early beneficiary, but I will note everybody in tech right now is losing money in it. So we can claim, is it going to be a long-term beneficiary or not? I think there's open questions in that. I think everybody is going to be a beneficiary. I think that in our societies, at least in the West, the responsibility of business leaders is to provide an opportunity. So we want to help our people get upskilled. We want to help our people get re-skilled. We want to open up that there are other opportunities or jobs. We can't force them to do any of that. But then it's on them. Do they want to take advantage of those opportunities and step up to do it?

22:56And I would say that answer has always been about 50-50. Some do, but a lot of people say, I don't want to get retrained. I want my old job. Okay, I'm sorry. That's not going to happen. And those folks, that ends up being the responsibility of government and society, not necessarily of the business. Correct. We try to be compassionate. We don't force people out like in a day. but if over six months or nine months they are not willing to learn the appropriate skills where they're needed, that's actually bad for the other 90 % who are around them. Still ahead, why Arvind thinks the math behind today's AI bubble just doesn't add up, the cybersecurity question keeping his team up at night, and IBM's$10 billion bet on quantum.

23:40Stay with us.

23:45Hey listeners, Bob here. If you listen to Rapid Response on Masters of Scale, you may be missing half the show. Because every Friday, we release a second Rapid Response exclusively in the Rapid Response feed. The guests and topics are just as compelling and timely, from Ford's CEO to NASA's administrator to the lessons from the Devil Wears Prada. It takes about 10 seconds to find. Just search Rapid Response wherever you listen to podcasts and hit follow to make sure you never miss an episode. I hope to see you there. Hey, Jordan Harbinger here of The Jordan Harbinger Show. Subscribe to the only show that will show you how to apply the world's greatest ideas from the most striking minds.

24:30We've got spies and CEOs, athletes and authors from Kobe Bryant to Malcolm Gladwell, Tony Hawk and Howie Mandel, and the chairman of Google, founders of LinkedIn and Instagram, antiquities smugglers, con men, brilliant scientists, national heroes, even the head of the CIA. Come join and have a listen for yourself to The Jordan Harbinger Show, available on Apple Podcasts, Spotify, and wherever you're listening now. Welcome back to Masters of Scale. You can find this conversation and much more on our YouTube channel, and be sure to check out the link in the show notes to subscribe to our newsletter.

25:07Are there signs of a bubble that you see in different places? If I take all the verbal promises, and if you say there's 125 gigawatts of AI data centers that are going to come online in the next two to three years, that's the$8 to$12 trillion of CapEx in total. Not in one year, in total. That is where I come to. I don't see the economics of that at all, because that would imply close to a trillion dollars of profit, which is in the best case, that means$4 trillion of more revenue. Okay, where exactly is that going to come from? The math doesn't add up for you. Now, will it really, really work out well for at least half of them?

25:51Yes. Some are going to thrive, but some will disappoint. And I don't think in a commodity world, there probably isn't space for a dozen foundation models. Is there space for three or four? Probably. But since there's a dozen being run after globally, it tells you that they're not all going to work out. So I'm going to ask you a super basic technology question, which will maybe reveal something about me, but may help those in the room. What is the difference between a data center and a mainframe? I mean, aren't they both buildings with a lot of boxes in them? So first, for the few geeks in the room, a mainframe is actually one box.