Power ranges: AI faces supply crunch

0:00This is a Monday.com ad. TheSameMonday.com helping people worldwide getting work done faster and better. TheSameMonday.com designed for every team and every industry. TheSameMonday.com with built-in AI, scaling your work from day one. TheSameMonday.com that your team will actually love using. TheSameMonday.com with an easy and intuitive setup. Go to Monday.com and try it for free. Yes, TheSameMonday.com.

0:41Rosie Blau:Hello and welcome to The Intelligence from The Economist. I'm Rosie Bloor. And I'm Jason Palmer.

0:50Rosie Blau:Today on the show, a new leap in sophisticated scams and why Allbirds is stepping away from shoes.

1:07But first...

1:15Rosie Blau:It may not have escaped your notice that AI is everywhere. In Silicon Valley, techies apparently like to do something called token maxing, a race to burn through tokens, which are the fundamental units of data, usually a word or symbol, that power AI models. They are, of course, trying to show off just how much artificial intelligence they use. And it is a lot. Open Router, which is a marketplace for AI models, reports that from January to March, the number of tokens processed each week quadrupled. But there's a problem. The industry that supports AI isn't expanding at the same rate. Could we be facing a looming supply crunch?

2:05The AI companies are actually struggling to keep up with demand. And in recent times, they have actually been throttling access to some of their tools.

2:14Rosie Blau:Shalas Chitnis is a global business writer. So for example, Anthropic, which is a large AI model maker, recently changed the terms of its service so that it dissuades a lot of its users from using capacity at peak times. OpenAI, which is another big AI company, shut down Sora, which is its video generation tool, because it wanted to allocate its scarce computing resources towards more lucrative ventures. So across the entire technology stack, AI companies are looking at different ways in which they can meet the demand with the amount of processing power that they have. so you talk about processing power but what is there actually a shortage of yeah that's a good question so if you can think about it ai models need really powerful processors which are called gpus graphic processing units to be able to run and these are placed in really large data centers where you have thousands of these gpus lashed together using optical network and all kind of devices.

3:17Essentially, the tech world is running out of processing power to be able to enable the kind of demand that we are seeing. And the reason this is important is because inference, which is when an AI model responds to a user query, every time you are using an AI tool, it's using that processing power. So the more people that are using AI, the more computing resources you need. And since AI demand is growing exponentially, processing power is struggling to keep up.

3:43Rosie Blau:So what are the AI firms doing about that? I think most importantly, they're just pouring vast sums of money to build out these AI data centers. To give you some numbers, this year, five of the largest cloud providers in the US, companies like Amazon, Meta, Microsoft, they are spending close to$700 billion in building out AI data centers. And it's not just these big companies, even model makers like Anthropic and OpenAI, which are really well funded, they are splurging on different deals to be able to get access to these resources. So the answer for the tech companies actually build, build, build a lot more.

4:23Rosie Blau:So throwing money at the problem seems to be a sensible solution. Will that work? I think it will work eventually. The problem is building out this capacity takes time. Especially now, in the US, a lot of these data center construction are running into local opposition for various reasons. Citizens are concerned about its use of electricity, land usage, water usage, so on and so forth. So a lot of these data center constructions are running into opposition, which is delaying their bills. I think the more pressing problem is the tech industry is just running out of kit to equip these data centers, which seems really odd.

4:59But that is true. So if you think about these really old school tech instruments like transformers and switches, which I think we've never even thought about, literally the tech industry is running out of them and lead times for some of them stretching between three to five years. So that's one problem. The second problem is the processors themselves, the GPUs that I talked about. Companies just can't make them fast enough or make enough of them to be able to supply these data centers. So if you go across the entire so-called tech stack, which is the different layers of technology that enable AI, there are shortages that are showing up everywhere.

5:35And that is what is making this particular supply crunch more concerning.

5:39Rosie Blau:So if there's a shortage in the supply chain, essentially, what are those suppliers doing about it? So the suppliers are definitely starting to build a lot more. But I think we need to keep in mind that software and hardware move on very different timelines. We are all used to great improvements in AI every few months. In hardware, it actually takes anywhere from two to four years to build out excess capacity because you actually need to go in and build the physical plant that builds these things. So there is a fundamental disconnect between the pace of software and hardware. So that is one problem.

6:13The other problem is, particularly in the semiconductor industry, there are some really critical choke points where key components are controlled by a few companies. So the first one is, of course, NVIDIA, which I'm pretty sure at this point everybody has heard of, which is the world's most valuable company. They account for over two-thirds of the world's AI processing power. So a hugely influential company. Now, chips from NVIDIA are essentially sold out. And it's so severe that a lot of companies are actually resorting to using chips that are really, really old. And by really, really old, I mean two or three years in tech terms.

6:52They're actually using those chips, which is unheard of. So that's the first problem. The other choke point in the semiconductor supply chain is actually making these chips in chip factories that are known as fabs. And today, there is only one company, which is called TSMC, a Taiwanese chip manufacturer, that basically makes most of the AI chips used by the tech industry today. TSMC has been expanding capacity. Its own CapEx, capital expenditures, are increasing by$60 billion this year. But again, it's still not as much as companies would like.

7:24Rosie Blau:So presumably, all of that is frustrating the AI firms who are seeing surging demand and now can't meet them. Absolutely. I mean, they've been pretty vocal about it. So Sam Altman, who's the founder of OpenAI, there's a quote from him where he says that TSMC should just build more capacity, which kind of explains his frustration. Elon Musk, who's the boss of Tesla and SpaceX, his solution is to simply go out and build his own fab. and not surprisingly, he's called it TerraFab. And his ambition is by 2030 to build a fab that will have more capacity than all the current fabrication plants put together.

7:57And to give you some numbers, some analysts ran a back of the envelope calculation that would require anywhere between five to 13 trillion. I mean, trillion in capital expenditure. So clearly we're talking about huge sums of money and it's probably not going to happen. But that just tells you the scale of the problem and the frustration that a lot of the software makers are feeling right now.

8:18Rosie Blau:Shailesh, I'm interested, as you say, we've got used to AI improving so much every few weeks, every few months. And of course, there are concerns about AI nicking our jobs. How will this supply crunch actually affect these developments of AI? I think there is a danger that the longer the supply crunch continues, the more pressure it will put on the firms to raise prices because we have been used to a world where prices of inference keeps dropping basically every six months. But behind that is a fact that a lot of companies are actually burning cash to be able to enable that. Now, if the supply crunch continues and prices continue to rise, that's going to increase pressure on these firms to raise prices.

9:04Potentially, that could then slow down adoption as well. And so there have been those that have called the supply crunch a quote-unquote natural break on this reckless AI spending. That's one view. The other view is this could potentially slow down AI adoption in a very real way if it continues.

9:21Rosie Blau:Shalish, thank you very much for talking to me. Thank you for having me.

9:42Right now at Mattress Warehouse, save up to$800 warehouse wide. Get buy one, get one free deals, plus a free adjustable base on select mattresses, 0 % interest financing, free delivery, and our exclusive lifetime price guarantee only at Mattress Warehouse.

10:06I've been covering the scam industry for around three years now, but I've always thought that calling it the scam industry wasn't quite right. It's bigger than just scams. Su Lin Wong is the host of our podcast series, Scam Inc., which last week won a prestigious Peabody Award, which if you haven't heard of it is like the audio version of a Pulitzer Prize. I came across a story recently that shows just that. I heard about a spyware attack, one that illustrates the very, very concerning leap in the capabilities of the criminals I've been following for the past few years. I spoke to a victim, I'll call her Amber, It's not her real name.

10:52She's 52 years old and works for an accountant for several charities in Indonesia. And at the start of last year, she received a message from someone claiming to work for Indonesia's tax office, asking her to update some basic information. It was a very mundane kind of message. She's been an accountant for a quarter of a decade, so she's on alert sometimes for possible phishing messages. She texted this person and said, let me check and I'll get back to you.

11:26The person on the other end of the phone purporting to be the tax office got back in touch about an hour later and was like, have you figured this out? We really need your information and gave her a link to download. So she downloaded this app that looked exactly like the government's own tax app, created an account. But what she didn't realised was that the software was collecting all her biometric data and it had also gained access to her photos, her camera, her microphone, her contacts, her notes app and her bank accounts. While this was all happening in the background, she couldn't see it happening on the phone.

12:05It just looked like the tax app was running. She remembered she had a friend who worked in the local tax office and so texted her. The friend immediately wrote back and said, stop what you're doing, it's malware. But it was too late. Two bank accounts had already been emptied and withdrawals were starting from a third.

12:28And as is the case with many NGO workers in Indonesia, those accounts were actually in her name, even though it was money from the NGOs that was stolen. Roughly more than 450 million rupiah, which is around 26 ,000 US dollars, which is just a huge amount of money anywhere, but especially in a place like Indonesia, where that sum could have paid for the salaries of 10 employees plus electricity, internet or utilities for a whole year.

13:04that really is the stuff of nightmares just a bank account emptying while you're watching what does this represent though as you say about the sort of expanding criminal capabilities of the scam industry for years the online fraud industry mostly relied on slow labor-intensive methods like the ones i focused on in scam inc for example investment cons romance scams and pig butchering schemes. And in those kinds of frauds, victims are groomed often over several months before being robbed. But what happened to Amber happened in minutes, if not seconds. And Amber's far from the only victim of these new Trojan schemes that are enabled by rapid advancements in tech and AI and really mark a change in how criminals are approaching cybercrime.

13:54What's really scary about this isn't just that it happens so much faster than a slower burning scam. It's that once the criminals got access to Amber's phone, they were able to target everyone in theory in her network. But the idea of a cybercriminal sending links that are clicked on by someone who's not paying close enough attention is nearly as old as the internet itself. How is this different? This is all part of a global malware as a service operation, where the criminals don't actually build the technology themselves. Instead, they buy it from messaging sites like Telegram. And so they're not just buying the malware itself, they're also buying stolen data.

14:36They have quite personalized information because all of this is being sold on the dark web or on messaging apps. This is likely what happened to Amber. It's likely that her tax information was sold to the criminals who were then able to send her a very convincing text message. This malware is being constantly updated so it's more efficient and better to evade detection by antivirus software. It's operating in lots of different languages. AI is making that even easier and it's sold by Chinese speaking vendors mostly to transnational criminal organizations based in Southeast Asia, the kinds I've been tracking for the past few years.

15:18And these gangs are running vast, vast businesses. Some of my reporting suggests that every year, this industry as a whole perhaps brings in 500 billion US dollars, which is the equivalent of the global illicit drug trade. So about a year ago, Intoblox, a security firm, noticed this huge surge in suspicious internet activity. And it led them to uncover this specific software cluster that targeted AMBA as well as many other people. And the firm has since identified hundreds of domains that have targeted people in more than 20 countries. So given the pace of change that you describe, it sounds like this problem is only going to get worse.

16:02Yes, I think you're completely right, Jason. We're already seeing new forms of spyware being developed. So the criminals are trying to figure out what to do with all the stolen photos and messages that they now have access to. And Infoblox found signs of AI chatbot integration, deep fake voice tools, and tests of ways to evade facial recognition systems. So we should expect that the criminals are going to ramp up their AI usage. At the moment, it's mostly been countries in the developing world who have been targeted along with Japan, South Korea, Spain. But experts I spoke to said that they expect to see this kind of malware target people across the US and Europe within the next year.

16:48And when you look at cybercrime, like you look at any other kind of crime, I suppose, it's a bit of an arms race between the bad guys and the good guys. What are the good guys doing? What's going on on the enforcement end of it? So the good guys are doing a lot, a lot more than they were doing when I first started covering this beat. We've seen the US government come out and say that they were going to try and crack down on this industry. Governments across Southeast Asia are very, very aware of the huge harms caused. The problem is that the criminals are making so much money that they're incentivized to find workarounds to these government crackdowns.

17:21And the other issue is corruption. There are many, many countries where corruption is endemic and the criminals take advantage of this by buying off the political and business elites. So it's not really as if one country can clamp down on this and the whole problem will go away because the criminals would just move to another country where it's easier to buy off the political and business elites. And so these kinds of banking trojans and malware are just the latest business line of these really sophisticated criminal networks. As ever with your newish beat, Su Lin, terrifying stuff. Thanks very much for your time and congratulations once again on the Peabody.

18:03Thanks very much, Jason. If anyone's curious to really understand the scale and sophistication of these criminals, you can listen to Scam Inc. wherever you get your podcasts.

18:34I had this idea for a wool shoe.

18:37Rosie Blau:Tim Brown, the chief executive of Allbirds, used to tell the origin story of his company. And a series of really bad-looking prototypes. How would we make it? Could we scale it? And I was like, oh my gosh. He also shared the advice he got from his business school professor. But I'm concerned that maybe you're going to be one of those guys that tries and tries and tries and then never stops trying and it's going to be a failure. So fail quickly, like put it on Kickstarter or something so it can fail. And then you can get on with your life and go find a real job. But a decade after the brand was founded, those wool trainers once adored by tech workers seemed to have lost their soul, so to speak.

19:18So Allbirds has decided to make a pretty significant change to its business.

19:24Rosie Blau:Shira Aviona is a business writer with The Economist. So it's selling off all of its footwear assets, no more sneakers, and it's decided to rename itself Newbird AI and to pivot to Compute Infrastructure for Artificial Intelligence. That is some pivot, Shira. What's going on here? So I think what Allbirds is doing is maybe an extreme version of a broader malaise within the generation of brands that grew up alongside it. Many of them were founded between, say, 2010 and 2015, and they marketed themselves as what I like to call millennial brands. So their consumer base were upwardly mobile millennials living in cities who found older brands boring and wanted beautiful and minimal products that they could buy online that would be shipped to their door.

20:19So many of them had a direct-to-consumer business model, meaning that they did almost all or all of their sales just through their own website. And a lot of them were backed by venture capital companies and kind of had this, say, just a very millennial aesthetic.

20:34Rosie Blau:I mean, it sounds like a decent business strategy to appeal to up and coming millennials. So what went wrong? Part of the story is that they got very lucky when they were founded. So two things were going on. One is that interest rates were low and venture capital companies that we kind of associate with tech became really interested in the consumer sector. So they were willing to hand out lots of money to Casper, a mattress company, or Warby Parker to sell glasses or indeed to Allbirds to sell sneakers to tech bros. And that has changed. Interest rates have gone up, especially after the pandemic.