67% of Enterprises Scaling Generative AI Pilots

0:00Today on the AI Daily Brief, where enterprises are with generative AI as we kick off the fall. Before that in the headlines, what Wall Street thinks about AI currently. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes.

0:23Welcome back to the AI Daily Brief Headlines Edition, all the daily AI news you need in around five minutes. Last week, we did a show about the five questions that would shape the near-term future of generative AI, and one of the big ones was how Wall Street would think about Gen AI heading into this autumn season. Reuters did a little bit of a summary of the market as a whole, summing it up by saying tech market values fall on AI costs and recession fears. And basically the idea here was that in August, there were two big reasons, or seemingly two big reasons at least, that companies like Alphabet and the other Magnificent Seven companies had a bit of a struggle.

0:59One was macro, a rising risk that the market was placing on the possibility of a recession. But the second was the narrative concerns around the ongoing and even escalating AI infrastructure build-out cost. One of the best examples of this was NVIDIA, whose share price fell by 7.7 % in the last week of August. That came after it projected third-quarter gross margins below estimates and reported revenues that only slightly outperformed investor expectations. Interestingly, Reuters framed it as only met expectations, even though it was actually a couple billion dollars over analysts' average estimates.

1:32Still, only being a couple billion over, not being some huge blowout, did feel like it met expectations. One counterweight to this came from Meta that showed some ROI from AI investment based on strong digital ad revenue growth. Still, by and large, the narrative heading into the fall is summed up by this Bloomberg piece, shorts are circling some of the AI boom's biggest question marks. Bloomberg writes, It's the story of so many stock market manias. A transformative technology juices a few companies, a bunch of more questionable outfits follow in their wake, and Wall Street buys it all. Then time sorts out what's real from fake.

2:05Now this specific report looks at a group of companies that have been recently targeted by bearish research reports. Hindenburg Research, for example, went in on server maker Supermicro, ripping around$10 billion of the company's$36 billion market capitalization off the top last week. And other companies like Lumen and Symbiotic have also faced bearish reports, although perhaps nothing on the same level. Now, interestingly, this report makes it clear that there is a big difference between the NVIDIAs of the world, even though NVIDIA did fall last week, and the hangers-on that benefited from the tailwind set by companies like NVIDIA.

2:38Mahoney Asset Management CEO Ken Mahoney characterized what the market is doing as separating winners and losers, pointing out, quote,

2:53Hindenburg came back again on Thursday with another report, with an even more dramatic accusation against iLearning Engine's holdings, accusing it of faking its financial figures. While the company denied it in a release, shares were still down 53%. One of the things that I've tried to make clear on this show is that Wall Street repricing some of these companies is not the same as AI being the next great bubble. John Belton of Gabelli Funds put it this way, the stock market always has a way of moving too far too fast and then going through a period of digestion. For a lot of these companies, we're in a healthy period of digestion.

3:25Now, one interesting little bit of speculation is at what point some of these big private market AI companies will have to go public. Corey Weinberg from The Information wrote a piece this week called Why OpenAI Needs an IPO that outlines a set of reasons that the company might want to consider something like this in the future. That includes the incredible capital intensiveness of OpenAI's business model, the potential that private capital might be running dry or might be problematic with the U.S. not necessarily loving a big dose of funding from a sovereign wealth fund in the Middle East, for example.

3:54And while Corey is clear that he's, quote, thinking about the following set of facts rather than acting on any particular news tip, I wouldn't be surprised if we see a little bit more of this chatter in the months to come. Staying on news from these public market companies touching AI, people are digging in a little bit more deeply to Meta's recently announced AI numbers. We discussed last week that Zuckerberg had said that Meta's AI tools have more than 400 million monthly users and more than 185 million weekly users. But some wondered if that was driven by accidental usage, given that AI rolled out as just an extension of the search bar.

4:25Microsoft is dealing with its own challenges. From the Wall Street Journal, Microsoft rolled out AI PCs that can't play top games and there's no quick fix. The short of this story is that in the process of getting a new generation of chips that are good enough to actually run these AI applications, Microsoft has made some of its most cutting-edge laptops incompatible with games like League of Legends and Fortnite, which are made instead to work with Intel's x86 chip. An independent research analyst tested 1 ,300 PC games and found only about half ran smoothly on the new AI-focused PCs. Now, obviously, it seems like this will likely not be all that much of a problem in the long run, but in the short run, it's another sign of the potential bumpiness of adoption that we're likely to experience.

5:05Lastly today in the headlines, a follow-up from our long read about the homework apocalypse, Axios pointed out that even beyond AI, schools are having a rethink when it comes to homework. In 2012, for example, 21 % of 13-year-old students said that they had no homework assigned. In 2020, that figure had gone up to 29%, and by 2023, it was up to 37%. Another study they referenced found that 67 % of high school students cited homework load as a major source of stress, which went up to 80 % among those doing three or more hours of homework a day. Then there's recently a bill that was passed by California's legislature that would recommend that school districts evaluate the mental and physical health impacts of homework assignments as well.

5:42One assembly member, Pilar Schiavo, said, quote, as a single mom, I only have a couple of hours with my kid at night before they have to go to bed. Spending most of that struggling to get homework done creates a lot of stress on a family. The point here is that it's not just AI forcing this conversation, although certainly AI is forcing it a lot faster. For now, though, that is going to do it for today's AI Daily Brief Headlines edition. Next up, the main episode. Today's episode is brought to you by Fractional. When we wanted to build an AI-powered feature of Superintelligent, our AI tool finder, I went straight to Fractional.

6:15The Fractional team is a group of senior engineers in San Francisco working on some of the most exciting projects in applied AI. Working with them is basically like hiring an absolute top flight AI engineering team, but in a way that you can customize exactly for your particular needs. Like I said, that AI tool finder feature that we built with them is already a key part of the Superintelligent platform, and we are working on something new as well. Fractional works with everyone from startups to the Fortune 500. And to request a free consultation, you can go to fractional.ai. If you want help identifying and building AI projects for your business, then I highly recommend you hit pause, open a web browser, and go to fractional.ai to request a free consultation.

6:54Today's episode is brought to you by Venice. The leading AI companies store your entire conversation history and attach it to your identity forever. That's every question you ask, every answer you receive, every image you generate, every thought you share with the machine, it's all being spied on. If you trust all the company's hackers and NSA board members that will ever have access to your AI conversations, then rejoice, for you are well served. For the rest of us, Venice is an alternative. Venice is a powerful AI app for text, image, and code generation that respects you as a sovereign individual and believes privacy and free speech are not only human rights, but necessary for civilizational advancement.

7:27Private, permissionless, and uncensored, you can try it for free without an account. AI Daily Brief listeners receive a 20 % discount on Venice Pro. Visit venice.ai slash NLW and enter the discount code NLW Daily Brief. That's NLW Daily Brief. All one word. Today's episode is brought to you by Super Intelligent, which is, of course, our platform that helps you learn how to use AI tools and perhaps even more importantly, gives you ideas on the best use cases that are actually going to help you achieve whatever it is you want to achieve. To recognize the end of summer and back to school slash back to work, we are running our best promotion ever.

8:06When you sign up for Superintelligent, between now and the end of August, using code SOBAC, your first month will be 100 % free. The platform features over 600 fun, highly practical AI tutorials that get you using AI fast and with an eye to actually transforming how you get things done. We've just launched Super for Teams, so if you have a group of people at your company that want to figure out how to use AI together, I highly suggest you check it out. But for those of you who are using Super Intelligent as an individual, once again, if you sign up for Super Intelligent between now and the end of the month using code SOBAC, you will get your first month 100 % free.

8:43Go to besuper.ai and check it out today. Welcome back to the AI Daily Brief. In the headlines today, we discussed how Wall Street is kicking off this fall when it comes to attitudes around generative AI. And so it only makes sense that for our first main episode of September, we're looking into AI in the context of the enterprise, and specifically where generative AI sits within companies now, after nearly two years since the launch of ChatGPT. Every quarter, Deloitte releases an update to its state of generative AI in the Enterprise report. This latest report that just came out covers the period between May and June of this year.

9:20They call it moving from potential to performance, and sum up, investment is increasing, but the clock is ticking to scale and create value. So let's discuss what their big findings were. Broadly speaking, one of the things that you're starting to see everywhere is that we're moving out of the technology novelty phase of AI adoption. Individuals and organizations are both getting more nuanced and sophisticated in terms of how they're thinking about using generative AI, and along with that is coming more expectation and less doe-eyed interest in anything anyone has to sell them. Unsurprisingly, a big theme for this report is organizations wanting to move beyond pilots and into a phase where generative AI is actually driving real business value inside their companies.

9:59Deloitte sets up a bit of a path diverging in the Wood analysis. On the one hand, as you'll see, there is a ton of movement shifting from pilots to deeper integrations. But as they also point out, quote, the clock is ticking for organizations to create significant and sustained value through their generative AI initiatives. Promising pilots have led to more investments, escalating expectations and new challenges. During this pivotal phase, C-suites and boards are beginning to look for returns on investment. There is a chance that their interest in generative AI could wane if initiatives don't pay off as much or as soon as expected.

10:30It would take a whole episode to discuss why I think that enterprises are largely in a situation where it would be much, much riskier to turn away from experimentation with AI than to accidentally overspend on it. But for now, let's just stick closely with what Deloitte is presenting here. So what is actually going on inside these companies that are testing AI? A couple big blinking banner headlines for me. First, two-thirds of companies are increasing their investments because they've seen strong early value to date. That is an extremely telling statistic that suggests there is much more than just hype going on inside these organizations.

11:03What's more, the benefits that people are finding are getting more diverse. On the one hand, improved productivity and efficiency as well as cost reduction are still both the top benefits sought by organizations, as well as those most cited as the most important benefits achieved to date, but importantly, 58 % reported they realized a more diverse range of most important benefits, such as increased innovation, improved products and services, or enhanced customer relationships, meaning there's a certain growing sophistication that's not just about ROI measurement. Something that we discuss a lot at Superintelligent is the way in which the AI transformation is proceeding through a sequence of steps.

11:38The first step, the low-hanging fruit, if you will, is personal productivity among employees. This is where a ton of the experimentation is happening now. When some individual is just figuring out if their work happens faster, if they integrate ChatGPT with their email writing process, that's sort of this first step. The second phase is all about unlocking new opportunities that weren't possible before. Totally reconsidered ambition, for example, around a marketing campaign, because things that were simply outside the capability set of the participants are now firmly within their grasp. This to me is what this 58 % reflects when they see increased innovation, improved products and services.

12:12This is more than just personal productivity. It's wider benefit. Then, of course, there's an even higher level stage, which is organization level transformation. Some organizations have gotten here, but by and large, this remains something for the future. This is all about how we might see organizations redesigned and reimagined from the ground up based on the new capabilities that AI enables. I think where you're likely to see this happen first is actually with entrepreneurs who create totally different types of organizations and model how you can do more with less or much more with the same, and slowly those types of transformations will find their way into the enterprise as well.

12:46In any case, again, two big banner headlines here. Two-thirds of organizations are increasing their investments in Gen.AI because they're seeing strong value, and nearly 60 % are seeing benefits that are not just about productivity and efficiency or cost reduction, but are about innovation and improved offerings. There is lots of challenge here, though, as well. 70 % of the organizations surveyed said that their organization has moved 30 % or fewer of their Gen.AI experiments into production. In other words, there does seem to be a big scale barrier. Those barriers come in a few varieties. One is around data.

13:1755 % of organizations told Deloitte that they're avoiding certain Gen.AI use cases because of data-related issues. Some are worried about regulatory uncertainty. Three of the four things holding organizations back from developing and deploying Gen.AI tools are risk regulation and governance issues. And then, of course, there's the old chestnut of ROI measurement. More than 40 % of respondents said that their companies are struggling to define and measure the exact impact of their Gen AI initiatives, and less than half said they're using specific KPIs to measure performance, with many standard measures of success not currently being applied.

13:48There is also definitely a broad sense here among these organizations that they do not feel prepared. Only 45 % think that they have the appropriate technology infrastructure. Only 37 % said that they feel they have the appropriate strategy. Only 23 % said they have the right risk and governance systems, and only 20 % said they have the right talent. For the sake of this report, Deloitte decided to hone in on two particular areas around data and governance and risk and compliance. The TLDR on the data side is that data has come even more to the fore thanks to generative AI. 75 % of organizations surveyed said that they've increased their tech investments around data lifecycle management.

14:26Data-related concerns are frequently cited as holding back Gen AI implementations. And when it comes to risks and regulation, as we said, these are meaningful hindrances as well. And this is me editorializing a little bit, but likely to get more so, especially as the U.S. starts to deploy state-by-state type legislation, like SB 1047 if it gets signed by Governor Gavin Newsom in California, as opposed to federal regulations, which might be a little bit clearer. Now, of course, even beyond these specific risks, there was never going to be a period of attempting to move from pilots to scale where the challenges of measurement weren't going to start to become a big issue as well.

15:01The story that this report tells is a lot of experimentation around AI ROI tracking. 48 % of those surveyed have used specific KPIs for evaluating Gen AI performance. 38 % have attempted to track changes in employee productivity. 34 % have tried to track non-financial benefits. Only 6 % are doing none of these types of things. So clearly there are experiments here, even if there aren't really best practices yet. My best guess is that we are going to see a continued period of experimentation around ROI measurement. I think that Deloitte is right to call out that there is a world in which the difficulty in measurement leads to a hindrance in organizations' abilities to actually scale these AI pilots.

15:40Clearly, we're already seeing some of that. However, I believe that the continued pressure on basically everyone from a director-level position up to have some sort of AI adoption strategy will actually produce an incredibly fertile set of experiments around AI ROI measurement that might get us to some of those best practices a little bit faster than we might anticipate. If you are interested in going deeper and checking out past reports, I will include a link to the overall webpage for this on Deloitte.com. For now, thanks to the team over there for preparing another interesting report, and thanks as always to you guys for listening or watching.

16:12Until next time, peace.

16:20Thank you.