Where AI Spend is Already ROI Positive

0:00Today on the AI Daily Brief, where organizations are already seeing ROI on their Gen AI investments. and before that in the headlines, how much are current layoffs about previous overhiring versus a direct result of AI? The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.

0:23All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, Robots and Pencils, Assembly AI, Blitzy, and Area. To get an ad-free version of the show, go to patreon.com slash AI Daily Brief, or you can subscribe on Apple Podcasts. To get more information about the show, go to aidailybrief.ai. And while you're there, please take a minute or two to fill out our AI ROI benchmarking study. We're heading into a year where every organization is going to be focused on how to measure the performance of their AI deployments. And we are all going to be flying blind unless we start to have better baseline standards and benchmarks.

0:57This should take you no more than a couple of minutes. And anyone who participates is going to get a survey highlights report as well as be entered to win an Amazon gift card or$200 in AI credits. And if you submit your use cases, there's a variety of other things you can get as well, up to and including a one-on-one meeting with me. You can find this at roisurvey.ai. And thank you so much for checking it out. Welcome back to the AI Daily Brief Headlines Edition, all the daily AI news you need in around five minutes. Today, we kick off with a discussion about the AI labor disruption narrative.

1:27On Monday, sources started leaking that Amazon was about to let go a ton of workers. The number given by those sources was 30 ,000, about 10 % of Amazon's corporate workforce. The question was, of course, going to be whether Amazon would blame AI or acknowledge other factors at play with their business. Amazon conducted a string of rolling headcount reductions across late 2022 and early 2023, obviously too early to be a result of AI disruption. Instead, those layoffs appeared to be a reversal of overhiring during the pandemic. That reduction in force totaled 27 ,000 corporate employees in total, so was also a very significant size.

2:02In addition, Amazon has been one of the weakest of the tech giants during the AI boom. Growth in their cloud business slowed from 19 % last year to a forecast 18 % this year. Compare that to Microsoft Azure, which is growing at 39 % in the most recent quarter. In any previous era, this would be an obvious cost-cutting measure to trim a bloated workforce. But CEO Andy Jassy has already laid the groundwork to blame layoffs on AI. In June, he wrote in a memo to staff that AI agents, quote, should change the way our work is done. He said,

2:45Bloomberg sources suggested that in reality, it might be a little bit of both. Citing meetings and internal strategy documents, they said that Jassy has led a big push to do more to automate work using AI. At the same time, he's told colleagues explicitly that parts of the company are still unwieldy after a pandemic-era hiring binge. And indeed, when we got the official announcement on Tuesday, the number was lower than the 30 ,000 leaked, which feels to be very intentional. Instead, this number was 14 ,000. And while they hinted at AI enough for it to show up in the title of news articles like this one, Amazon laying off about 14 ,000 corporate workers as it invests more in AI, When you go read the announcement post from Beth Galetti, the senior VP of people experience and technology, they're sort of hedging a little bit and saying that this is about being nimble for the AI era rather than blaming it explicitly on AI.

3:35She writes, This generation of AI is the most transformative technology we've seen since the internet, and it's enabling companies to innovate much faster than ever before, in existing market segments and altogether new ones. We're convicted that we need to be organized more leanly, with fewer layers and more ownership, to move as quickly as possible for our customers and business. The second layoff announcement comes from education tech company Chegg, and this one is definitely more directly aimed at AI. The company plans to lay off 388 workers, which is around 45 % of their workforce. And indeed, they said that this was just part and parcel of the, quote, new realities of AI.

4:11Those realities are that Chegg was one of the first companies to get absolutely decimated by the introduction of ChatGPT. Students were no longer seeking out online homework help and digital tutoring, instead using ChatGPT to access similar services for free. Chegg acknowledged that the introduction of AI had led to reduced traffic from Google to content publishers and consequently a significant decline in Chegg's traffic and revenue. The goal now is to restructure to support its academic learning products. This is going to increasingly become a policy conversation, and so it's important that we understand what's actually going on.

4:42You might have seen this chart flying around the internet showing that since 2023, while senior employees have continued to be hired, junior employees have sort of fallen off a cliff. What's interesting to me about how this is being shared is that while it's being presented as a chat GPT issue, there's clearly something else going on here as well. The downward trend in junior employees is both at non-adopting AI firms and at AI adopting firms, even if it is, yes, slightly worse at AI adopting firms. The trend in both is the same, which suggests to me that there are other things going on as well.

5:13This, by the way, comes from a Harvard working paper. What some people are grokking is that it doesn't matter what combination of post-pandemic reset and AI this is, it's going to demand a policy response. Vivek Ramaswamy, who's running for governor in Ohio, tweeted that chart and said, AI is making the job market a lot tougher for young graduates. This is a big deal and ignoring it will become untenable. Now, his solution is to give people a chance to participate in the equity upside of these companies. But that, frankly, is secondary to the fact that this is going to become absolutely, increasingly in every election cycle we have from here on out, a bigger and bigger issue.

5:47Moving over into new product and new feature land, Anthropic has expanded Claude for Financial services with a new Excel agent. Claude for Excel is now available in beta as a research preview. It adds Claude in a sidebar for Microsoft Excel, allowing the agent to draw context from the application as well as modify or create new worksheets. When working as an agent, Claude provides a line-by-line explanation of changes. This allows the user to follow along and check the work in real time. Anthropic has also added seven new connectors for Claude to provide access to relevant data and news feeds for work in the financial industry.

6:19The update also utilizes Claude's new skills feature with a set of preloaded workflows accessible to the agent. These include comparable company analysis, discounted cash flow models, due diligence data packs, and earnings analysis. Skills allow Claude to tap into these workflows as needed without requiring the user to build any agentic scaffolding or infrastructure. Norgis Bank, the Norwegian Sovereign Wealth Fund, is already seeing some dramatic results. Their CEO Nikolai Tangen reported 20 % productivity gains equivalent to 213 ,000 hours in annual savings or the work of around 100 full-time employees.

6:51He said that portfolio managers and risk departments are now able to, quote, seamlessly query our Snowflake data warehouse and analyze earnings calls with unprecedented efficiency. AIG CEO Peter Zafino said that their use of Claude had compressed the timeline to review business by more than 5x in their early rollouts, while simultaneously improving their data accuracy from 75 to over 90%. Some once again thought that this was yet another indication of how hard it's going to be for junior employees. HeyBuilds writes, this just instantly took out 80 % of junior analyst jobs. Associates will take this, make their work 5x faster because they already have the expertise knowledge and can just edit.

7:25The longest piece of a deal prep was to build the model in the presentation deck. Others were just excited to not have to go deeper in Excel. Buchao writes, my decade of avoiding to learn Excel is about to pay off. In chip land, Qualcomm is launching a new line of AI accelerators to take a slice of the rapidly growing AI inference market. Qualcomm has dominated the market for mobile phone CPUs and plans to repurpose the integrated neural processing units from those chips into full-size AI accelerators. The new chips will be offered as standalone units as well as racks of up to 72 chips in a server, suitable for deployment in data centers.

7:57Interestingly, if supplied as a chip only, the component could be installed in existing hardware supplied by NVIDIA or others. The first release will be the AI200 chip beginning next year, with a more efficient AI250 model planned for release in 2027. Both chips are optimized for inference rather than model training, so won't compete in all aspects with NVIDIA's leading GPUs, but the choice highlights a growing bifurcation in the chip-making space. While NVIDIA continues to be top of the tops when it comes to model training, and the choice for most large-scale data centers, demand for inference to serve models to end users is growing much more rapidly, which is opening up space for high-efficiency chips designed for that specific purpose.

8:32Qualcomm has already secured an anchor customer to produce their first run, with Saudi Arabia's publicly-owned AI company Humane planning to deploy 200 megawatts of data center capacity using the AI200 chips. Unsurprisingly, the market cheered on the move to diversify away from the stagnant smartphone space into red-hot AI chip manufacturing, setting the stock soaring by 15 % on Monday. That, however, is going to do it for the AI Daily Brief headlines. Next up, the main episode.

9:02AI changes fast. You need a partner built for the long game. Robots and pencils work side-by-side with organizations to turn AI ambition into real human impact. As an AWS-certified partner, they modernize infrastructure, design cloud-native systems, and apply AI to create business value. And their partnerships don't end at launch. As AI changes, Robots & Pencils stays by your side, so you keep pace. The difference is close partnership that builds value and compounds over time. Plus, with delivery centers across the U.S., Canada, Europe, and Latin America, clients get local expertise and global scale.

9:35For AI that delivers progress, not promises, visit robotsandpencils.com slash AI Daily Brief.

10:04happen. Assembly AI's models lead the field in accuracy and quality so you can actually trust the data your product is built on. And their speech understanding models help you go beyond transcription, uncovering insights, identifying speakers, and surfacing key moments automatically. It's developer-first, no contracts, pay only for what you use, and scales effortlessly. Go to assemblyai.com slash brief, grab$50 in free credits, and start building your Voice AI product today. This episode is brought to you by Blitzy, the enterprise autonomous software development platform with infinite code context.

10:38Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Enterprise engineering leaders start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and precompiles code for each task. Blitzy delivers 80 % plus of the development work autonomously while providing a guide for the final 20 % of human development work required to complete the sprint. Public companies are achieving a 5x engineering velocity increase when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding pilot of choice to bring an AI-native SDLC into their org.

11:13Visit blitzy.com and press get a demo to learn how Blitzy transforms your SDLC from AI-assisted to AI-native. There's a reason most enterprise AI initiatives never make it to production. You can't find a platform that's both powerful and secure enough. The result? AI budgets burn with zero business impact. But not anymore. ARIA is the enterprise AI platform that delivers speed without compromise. Unlike other platforms that force you to choose between fast deployment or secure operations, ARIA brings speed and security together. Launch AI quickly without cutting corners on compliance. Scale rapidly without sacrificing governance.

11:47Move at the speed of business without moving past your security requirements. Fortune 500 companies across finance, healthcare, retail, legal, and more choose ARIA because they deliver what seemed impossible. Enterprise AI that's fast enough to beat the competition and secure enough to protect your most sensitive data. Ready for AI at full speed with zero compromise? Visit ARIA.com to see the platform in action. That's A-I-R-I-A dot com. Simplify Enterprise AI.

12:18Welcome back to the AI Daily Brief. As I have said a number of times on this show and will continue to say I'm sure, one of the key themes for AI at work next year is going to be actual demonstrated performance. The simple reality is we have shifted fairly aggressively this year from a paradigm of pilots and experiments into much more extensive and large-scale deployments. Which is, of course, not to say that there aren't still lots and lots of things that are still in that zero-to-one pilot stage. Even companies that are quite advanced in their AI and agent strategy still have tons of areas of their company where there's no AI or agents at all.

12:52That said, the companies that do have some number of months of full-scale deployments under their belt are increasingly going to care about how those deployments are performing in the real world when it comes to making decisions about how they do the rest of that zero-to-one AI deployment. Arvind Jain, the CEO of Glean, posted about this on LinkedIn this week. He wrote, Lately, I've seen a clear shift in my conversations with enterprise AI leaders, and with Glean, I can see it in the data too. I asked Glean to analyze our customer call notes from the past couple of years to track how the reasons for adopting Glean have evolved.

13:22From mid-2023 to 2024, improving general productivity was the top driver, behind 67 % of Glean implementations. A year later, that dropped to 37%. Why? Because leaders have realized that productivity for its own sake doesn't move the business forward. It only matters when it shows up in measurable business outcomes. Over the past year, adoption drivers have shifted decisively towards outcome-based goals. Revenue growth, faster ship cycles, better customer support, Accelerating sales revenue, for example, is now about five times more likely to be cited as the top reason for adopting Glean than one year ago.

13:53Every function has its own North Star metrics that tie AI efforts directly to business outcomes. The bar for AI has been raised. The new standard isn't general productivity, it's measurable business outcomes. Now what's interesting is that companies are not only optimistic, but getting more optimistic about their anticipated AI ROI. In 2024, KPMG asked CEOs as part of their annual CEO survey, when they anticipated to see a return on their AI investment. The vast majority, 63%, said that it was going to take three to five years. 16 % said more than five years, and 20 % said one to three years. Only 1 % said six months to a year.

14:29When they asked that same question this year, the number that said that they anticipated ROI in six months to a year had jumped to 19%, and the number that thought it was going to be one to three years represented a full two-thirds, 67 % of respondents. Those pull forward ROI expectations are, I think, part of what's going to be driving such a focus on it in the next year. If you anticipate a return on your AI investment in short order, you better believe you're going to spend a bunch of time while you have to actually go out and figure out what your ROI is. And while generalist expectation surveys are super useful, the way that AI is, of course, actually going to make an impact is within specific domains and functions.

15:08Not all different types of AI deployments are at the same level of maturity and are going to have different types of dynamics surrounding their performance profile. Artificial Analysis recently released their State of Generative Media survey report. This was based on a survey of personal and organizational users in Q3 of this year. The survey was meant to understand how organizations and individuals are adopting and using both image generation tools as well as video generation tools. Now, notably, this happened before the release of Sora 2 and the OpenAI Video API, but it's still a fairly contemporary study.

15:40Let's talk first a little bit about what they found in that sector, and then we'll get into the ROI part. When it comes to image generation, Google Gemini is out ahead of OpenAI. This doesn't surprise me very much. Although the embedded ChatGPT image generation tool can be good under the right circumstances, it has some foibles, and just in general, it's very feature incomplete relative to other image generation models. The fact that it's so high, I think, reflects the fact that it's just embedded in ChatGPT. What's interesting about this list is that while the models you would expect are right at the top, there is still a broad base of other tools that lots of different companies are using.

16:15Some might be surprised to see MidJourney down at 17%. I'm not necessarily, in the sense that while MidJourney is artistically and aesthetically top of the tops for me and always has been, it still lags in a lot of the very practical areas you need when you're using it for a business purpose at least, like for example, text reproduction. The one that is criminally underused on this list is of course ideogram at just 12%. Ideogram by a factor of about 10 is my most used image generation tool. On video, once again, Google is in the lead. 69 % of respondents were using VO3. A lot of the other top models though were from China, with runway all the way down and forth with just 30%.

16:50And when it comes to adoption patterns, as you might imagine, personal usage is leading organizational usage. Image generation has reached a pretty significant level of ubiquity for personal use at 89 % and is now used by over half of organizations at 57%. Video lags behind, although not as much as you might think. A little over 60 % of individual users are using video generation compared to about a third of organizations. I would have actually bet that was closer to a 20 % number, but I may be underestimating how quickly people have understood that AI-generated video could be really useful for some very discrete purposes like ad and social content.

17:26Video is definitely more of an experimental stage. While 53 % of respondents said that they had integrated generative images into their creative workflow, a full 58 % said that they were just experimenting with video. When it comes to organizations, the distribution between deployment, prototyping, and exploring was actually pretty similar across images and video. Deployed into production workflows was actually the leading category, again confirming what I was saying before, that we're quickly moving out of the pure experimentation phase. We'll race through a couple of other interesting nuggets when it comes to this category among organizations.

17:57It does appear that marketing and advertising are the key use case for organizations with generative video, at 55 % of organizations using it for that purpose. But there's pretty good distribution across other areas, including creative storytelling, design, and educational and training content. When it comes to why people and companies are choosing a model, quality remains at the top, which is kind of what we've seen across a lot of different categories, but cost is a still large consideration, with 53 to 55 % of organizations saying that lower total cost is a key factor for them. One other interesting note, it looks like organizations are pretty willing to design new workflows around these new tools, with only 27 % saying that integration with tools and workflows was a major factor for images, and just 14 % saying that it was a factor for video generation.

18:40This says to me that particularly video generation might be a new capability that is coming online for the first time because of AI, rather than is just replacing some existing workflow. There's a lot more in here. I would encourage you to go check this out. But this is the big chart. When asked when you expect to see a return on investment from your organization's media generation initiatives, 34 % said that they were already seeing ROI. Another 31 % said that they expect ROI within the next 12 months, meaning that a full two-thirds of organizations are already seeing ROI or anticipate it within a year.

19:15Another 23 % said that it would be one to two years. Once again, we see ROI expectations being pulled forward in a pretty significant way. Zephyr on Twitter described this as AI-generated video and image having crossed the ROI Rubicon, and I think that's a pretty good way of putting it. Now, beyond this one study, you're also starting to see reports pop up pretty frequently indicating where organizations are seeing the actual practical benefits of AI. On Monday this week, CNBC published a story called AI is driving huge productivity gains for large companies while small companies get left behind.

19:48This was based on a Wells Fargo research note that shared that productivity for the S &P 500 is up 5.5 % since ChatGPT. For the smaller companies of the Russell 2000, real revenue per worker is actually down 12%. Now, this is that particularly vicious macro approach to measuring productivity that's not getting in there to do studies around how much faster people are doing their work, but is instead just looking at revenue compared to headcount. But still, the narrative here is that at least for big companies, AI is having a positive impact on productivity. What's interesting is that another study, this time from Intuit QuickBooks, found that small companies were seeing productivity gains.

20:23In fact, more than 7 in 10, 75 % of those organizations using AI said that it was boosting their productivity, up from 46 % a year earlier in July of 2024. When you look at the chart, you can see a massive increase in organizations that say AI is very helpful or somewhat helpful and a very significant decrease in organizations who found it neutral. 56 % said that they're more productive than three months ago. This survey, by the way, tried to do away with the complexity of productivity by simply defining it as higher output for the same or lower input costs. There are some other interesting notes in that survey.

20:58For example, 24 % said that their workdays are shorter thanks to AI, all of which seems to suggest that while market performance for small companies might be off relative to the big guys, small companies are certainly seeing the benefits of AI in practice. I think we were just at the very beginning of this conversation around AI's ROI, and I would expect real results over the following weeks, months, and year. I will use this as an excuse to tell you once again about the big AI ROI benchmarking study that we're doing. We're having people share the use cases that are driving value and what specific value it's driving, be it time savings, cost savings, new revenue, new capabilities, or something else entirely.

21:33It's been live for about one day, and we've already had hundreds of use cases with their ROI shared. And I think that with your help, we could actually make this the biggest repository of AI ROI data that exists. Find it at roisurvey.ai. And for now, that's going to do it for today's AI Daily Brief. I appreciate you listening, as always. And until next time, peace. Thank you.