Can AI Be Normal and Transformative at the Same Time?

0:00Today on the AI Daily Brief, can AI be normal and world changing at the same time? The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.

0:18All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, Super Intelligent, Robots and Pencils, Blitzy and Rovo. To get an ad-free version of the show, go to patreon.com slash AI Daily Brief, or you can subscribe on Apple Podcasts. If you are interested in sponsoring the show, send a note over to sponsors at aidailybrief.ai, and we can send you all the information. I can definitely feel that people are getting to the end of the year because Q1 is filling up very quickly. So again, shoot us a note at sponsors at aidailybrief.ai to get all that information.

0:45Lastly, as always, we are coming into the last week or so of the AI ROI benchmarking study, at least the last week for submitting information. So if you want to be included and you want to get the full study readout when it is done, check out roisurvey.ai. Welcome back to the AI Daily Brief and happy Sunday. Now, of course, it being the weekend, that means it's a big think slash long reads episode. And for the first time in a while, we actually are basing the show on a long read. The piece is called Common Ground Between AI 2027 and AI as Normal Technology. And as you can probably tell from the title is a meeting of the minds of two very different visions of the likely futures of artificial intelligence.

1:24The two pieces in question are AI 2027 and AI as Normal Technology. The first was written by a group that included Daniel Cocotallo, Scott Alexander, Thomas Larson, Eli Liflund, and Romeo Dean. The second by Arvind Narayanan and Sayesh Kapoor, who also have a newsletter of the same name. Now what's interesting is that both of these pieces were released in April of this year, and both had significant resonance despite having very different visions for how AI was likely to shake out. The idea of AI 2027 is that by around 2027, we likely get superhuman AI that transforms the world faster and more profoundly than the Industrial revolution.

2:01We get there through massive trillion-dollar build-out of data centers, models being trained on 1 ,000x times the amount of compute used for GPT-4, models that stop being just chatbots and become agents and employees, doing coding research and eventually a big share of intellectual work, and critically, in their vision, AI is used to accelerate AI R &D. Agents speed research for the next layer of agent, creating an AI-driven feedback loop that very quickly pushes capabilities into superhuman territory. By early 2027, they expect a superhuman coder that's effectively an AI workforce equivalent to tens of thousands of top engineers running 30 times faster, fully automating coding and much of research.

2:36The economic shock that follows is big gains for firms that adopt, turmoil for roles like junior software jobs, and sharply rising geopolitical tensions as states see AI as a decisive military and cyber asset. And of course, the goal of the piece was to get people to think differently about the speed at which AI should be developed. Then we flip over to the idea of AI as normal technology. One thing that should be noted is that normal in this case doesn't mean insignificant. AI as normal technology means, in these authors' minds, treating AI like electricity or the internet. Powerful, general purpose, but still a tool embedded in human institutions, not a separate alien mind or species.

3:14This piece was specifically offered as an alternative to the superintelligence and fast takeoff narratives that imagine AI as an autonomous agent racing beyond human control. These authors see the speed of change as much slower than current hype, and while they imagine a future with much more capable AI, they don't find a coherent case for godlike superintelligence. They point out that the diffusion of AI is limited not just by the speed of technological development, but by human organizational and institutional change. To say nothing of the forced inertia from regulatory and guardrail apparatus in safety-critical areas.

3:44They argue that the economic impact is likely to take much longer to fully play out than, for example, the AI 2027 scenario. The policy upshot here is that by reframing AI as a normal technology, these authors hope to change how we think about accidents, misuse, arms races, and misalignment to have more continuity with past tech risks and less focus on the sort of sci-fi scenarios. Indeed, they argue that superintelligence panic interventions could backfire if AI's real harms end up looking more like past technologies. So these represent very divergent views of the world of potential AI outcomes, Which is why it was so notable that a group of these authors got together to produce a new piece called Common Ground Between AI 2027 and AI as Normal Technology.

4:27The authors acknowledge that there are substantial disagreements between these two worldviews. However, they say, we found that all of us have more in common than you might expect. And from there, they put together 12 areas of mutual alignment. Now, what we're going to do is we're going to read excerpts from this, and I'll do some discussion along the way. But I want to be clear about why I think this is valuable. We are very quickly heading into the political era for AI. It is abundantly clear from the fact that both Bernie Sanders on the left and Ron DeSantis on the right are coming at AI with all they got.

4:58It feels much more likely to me that in 2026, AI is going to be used as a populist cudgel on both sides of the aisle than opinions are going to break on strictly partisan lines. In my mind, the best remedy for hyperbole is nuance and common sense. And what these 12 points reflect is a whole big bucket of common sense and common ground that can be built upon when it comes to making AI policy. So let's dig into what they're talking about. The first point of agreement is that before strong AGI, AI will be a normal technology. And the point here is that right now, and indeed for some period to come, the AI that we have and will have is a normal technology.

5:36It is not the recursively self-improving sort of uncontrollable agent that the AI 2027 group are worried about. This is important because as we think about policy, we are going to have to think about not just future AI, but also the AI we have right now. Being able to distinguish between these two things is going to be extremely important. Indeed, the two groups reinforce that the diffusion of AI into the economy is going to happen more slowly, not because of technology limits, but because of just normal human and institutional inertia. For example, they write, The diffusion of AI throughout the economy will continue to be fairly gradual, with industries slowly handing over tasks to AIs as they become convinced that the AIs are reliable enough and as they build the necessary infrastructure, interfaces, and new workflows.

6:16Second agreement. Strong AGI developed and deployed in the near future would not be a normal technology. And here you have basically the two groups playing nice with each other. The first point of agreement acknowledges the correctness of the AI as normal technology argument on a today kind of perspective, while the second point of agreement gives the AI 2027 folks their theoretical dues. In other words, the AI as normal technology guys say, if AI does improve in the way that the AI 2027 people think it will, that would not be normal. Here's how the authors sum up the difference here. In Arvind and Saesh's view, the notion of strong AGI or any other notion of AGI is a joint property of the system and its environment.

6:55They think strong AGI won't be developed in the lab, say by scaling LLMs. Rather, building strong AGI will require a feedback loop with the real world that would set limits to the pace of progress. Thus, if strong AGI is developed and deployed in the next decade, it is a world in which the normal technology view has failed and or is no longer useful. On the other hand, a more gradual route to strong AGI could be one in which the framework remains helpful, though the endpoint of this route is something they consider outside the horizon that can be reasonably foreseen and planned for. By contrast, AI 2027 authors see current developments as harbingers of strong AGI.

7:27They expect progress to be continuous but rapid and to accelerate dramatically when AIs fully automate AI research. So you basically have it as a core difference here, just a very divergent set of beliefs around the way we get to AGI and the speed with which it happens. Back to points of agreement number three, they say most existing benchmarks will likely saturate soon. This is, obviously, if you are a regular listener to this show, something we talk about all the time. In fact, we just discussed it in the context of the GPT 5.1 release, which didn't do the normal slate of sharing all the different benchmarks, at least not initially.

7:59I have long been in the camp that to understand models, you have to just dig in and see how they perform on a variety of tasks. And it seems that the authors agree, but the implications are a little bit different for them. Basically, everyone agrees that our AI models are going to be able to ace all of these various benchmarks. The question is how much those benchmarks say about the AI's performance in the real world. They write, Arvind and Sayesh believe that these benchmarks have poor construct validity and as a result, that saturation does not mean that the underlying task will be easy to automate.

8:25Just because an AI system can resolve sweet bench issues with superhuman performance, this does not imply that it will be able to start replacing humans at the job of software engineering. At least for the next 50 years, they expect that there will be many jobs where humans outperform the best AIs. Thomas, Eli, and Daniel agree that there is an important gap between benchmark scores and real-world utility. Their disagreement centers on the magnitude of this gap. They feel that some of these benchmarks are an important source of evidence for how close we are to the automation of AI R &D. So again, summing up, both of these groups think that benchmarks are or will be saturated, but they have a very different understanding of the story that that's actually telling.

8:59A fourth agreement is that AIs may still regularly fail at mundane human tasks, and that strong AGI may not arrive this decade. They write, on the other extreme, we also all believe that it will probably still be the case that AI systems regularly fail at automating tasks humans find relatively simple. For example, by the end of 2029, none of us would be that surprised if AI systems couldn't reliably handle simple tasks like book me a flight to Paris using a standard human website. The reason they think that at core is, quote, robustly handling the long tail of errors is challenging. It is simultaneously possible that AI systems can solve tasks well on average and yet behave far worse than any human would in the worst case scenario.

9:34And so this is actually a conversation about the economic viability of these AIs. They write, we think that in domains where a human can reasonably verify the work, AI systems will be reliable enough to be useful in practice. But we all agree that it is possible that even by 2029, AI systems will not be able to be used in high assurance settings. And in some ways, what we have here is the AI 2027 group pushing their timeline out a little bit. They continue, we all expect that strong AGI will probably not arrive before 2029, and in early 2029 the world will probably still look basically like it does today.

10:06There will be AI systems that succeed at increasingly many tasks, but humans are still basically employed to do most things, and AIs will not be able to independently discover new science. Now the AI as normal technology authors think this strongly, whereas they say the authors of AI 2027 agree, though only barely. They think their strong AGI in 2027 scenario is plausible, but faster than their median expectations, which are for these three authors, 2030, 2035, and 2033.

10:36Today's episode is brought to you by my company, Superintelligent. Look guys, buying or building agents without a plan is how you end up in pilot purgatory. Superintelligent is the agent planning platform that saves you from stalling out on AI. We interview teams at scale, translate real work into prioritized agent opportunities, and deliver recommendations that you can execute on. What to build, what success looks like, how fast you'll get results, and even what platforms and tools you should consider. All customized for you. Instead of shopping for hype, you get to deploy with confidence. Visit bsuper.ai and book your AI planning demo today.

11:11Small, nimble teams beat bloated consulting every time. Robots & Pencils partners with organizations on intelligent, cloud-native systems powered by AI. They cover human needs, design AI solutions, and cut through complexity to deliver meaningful impact without the layers of bureaucracy. As an AWS-certified partner, Robots & Pencils combines the reach of a large firm with the focus of a trusted partner. With teams across the US, Canada, Europe, and Latin America, clients gain local expertise and global scale. As AI evolves, they ensure you keep peace with change. And that means faster results, measurable outcomes, and a partnership built to last.

11:46The right partner makes progress inevitable. Partner with Robots and Pencils at robotsandpencils.com slash AI Daily Brief. This episode is brought to you by Blitzy, the enterprise autonomous software development platform with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Enterprise engineering leaders start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and precompiles code for each task.

12:17Blitzy delivers 80 % plus of the development work autonomously while providing a guide for the final 20 % of human development work required to complete the sprint. Public companies are achieving a 5x engineering velocity increase when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding pilot of choice to bring an AI-native SDLC into their org. Visit blitzy.com and press get a demo to learn how Blitzy transforms your SDLC from AI-assisted to AI-native. Meet Rovo, your AI-powered teammate. Rovo unleashes the potential of your team with AI-powered search, chat, and agents, or build your own agent with Studio.

12:53Rovo is powered by your organization's knowledge and lives on Atlassian's trusted and secure platform, so it's always working in the context of your work. Connect Rovo to your favorite SaaS app so no knowledge gets left behind. ROVO runs on the Teamwork Graph, Atlassian's intelligence layer that unifies data across all of your apps and delivers personalized AI insights from day one. ROVO is already built into Jira, Confluence, and Jira Service Management Standard, Premium, and Enterprise subscriptions. Know the feeling when AI turns from tool to teammate? If you ROVO, you know. Discover ROVO, your new AI teammate powered by Atlassian.

13:29Get started at ROV, as in victory, O.com.

13:37Next up, an important clarification given this terminology of normal. All these authors agree that AI will be at least as big a deal as the internet. They write, While we disagree on the upper bounds of capabilities, we all agree that AI will be a big deal. The world will change as a result of this technology, and things that seem like science fiction will soon be possible, just like the world of today is different to the world 30 years ago. Indeed, in many ways, the technology we have today exceeds the capabilities of the science fiction at the time. They then talk about all the different ways in which the internet changed the world.

14:07Arvind and Sayesh agree that AI is a general-purpose technology that will have similarly transformative effects. In the long run, they expect AI could automate most cognitive tasks, just as the industrial revolution led to the automation of most physical tasks, in the sense that machines are now responsible for vastly more physical work than the human population. That said, they expect AI impacts to largely follow the path of previous general-purpose technologies, which are bottlenecked by barriers to diffusion and adoption rather than capabilities. And they expect that there would still be a lot left for humans to do, like controlling AI systems and deciding how they should be used.

14:39Continuing, they say, the authors of AI 2027 clearly believe that AI will be at least as big of a deal as the internet, but they believe that this will very rapidly be followed by AI that is more important than any other technology ever developed. So if those five points of agreement are agreements around specific predictions of events, they also found that despite their differing worldviews, they found more agreements about actionable policy proposals than they would have imagined. They write, we see many actions as sensible in any possible future world and encourage more research on ways to mitigate potential harms.

15:10So what are those policy-related agreements? The first and sixth overall agreement is that AI alignment is unsolved. They write, we all agree that AI alignment, that is the problem of training AIs to behave in a way that aligns with our values and expectations, has not been solved when it comes to current AI systems. We all agree that it is important to invest in research aimed at aligning current and future AI systems. We all agree, they write, that on the current trajectory, AIs will continue to be misaligned, often in ways that aren't detected by evaluations. And while they disagree around certain specifics and how far you can take alignment research, they all agree that we should be doing more of it.

15:43A next point of agreement is that AIs must not make important decisions or control critical systems. This one is the shortest, cleanest agreement in the entire thing, with the authors writing, We all believe that current AIs should not be allowed to have autonomous control over critical systems. This includes extreme cases like giving AIs control over data centers, nuclear weapons, tech companies, or government decision-making processes. Next, they agree that transparency, auditing, and reporting are beneficial. They think that independent auditors should regularly evaluate the safety of AI systems, that whistleblower protection should be strengthened, that safe harbors for independent researchers should be established to encourage safety research, and basically that we need a big group effort around risk mitigation.

16:22Related to that, number nine, they say governments must build capacity to track and understand developments in the AI industry. Now, this is not going so far as to say that the government should control everything about AI, but basically that it has to have the technical capability to have an informed seat at the table. The next one, certainly one to keep in mind as we do see the rise of populist rhetoric around this, number 10, the diffusion of AI into the economy they agree is generally good. They write, the productive impacts of AI will be realized as it diffuses across society. Governments and other actors can play many roles in enabling diffusion.

16:55Deploying AI may also help build resilience as defenders can figure out how to use these systems to enable better responses to risks like cyber attacks and other AI-enabled threats. To be clear, they write, we aren't recommending ramming AI into everything as fast as possible. We simply mean that it's generally good for AI products and services to diffuse through the economy. They will have many immediate benefits and also help us learn more about AI, its strength and weaknesses, its opportunities and risks. That one I think could probably be an entire podcast or series of podcasts and policy proposals on its own.

17:24Number 11. The authors of both these pieces agree that a secret intelligence explosion or anything remotely similar would be bad, and government should be on the lookout for it. They write, Tech companies like OpenAI and Anthropic are explicitly planning to automate their own jobs as fast as possible. That is, they are aiming to train AIs that can fully automate the AI R &D process itself. The resulting recursive self-improvement could result in an intelligence explosion of rapid capability gains. At least, that's what the authors of AI 2027 expect. Or it could be bottlenecked by other factors, such as the lack of real-world data.

17:55We all agree, though, they write, that if rapid AI capability improvements were to occur in secret, it would be dangerous and potentially catastrophic. Secrecy would stand in the way of oversight and coordination that may be necessary regardless of how transformative the technology becomes. Instead, they say, information about the latest AI capability trends, the guidelines and constraints that AI companies attempt to instill in their models, the alignment and control techniques they use, and safety incidents and evaluation results relevant into the above needs to flow quickly out of the companies and to the public.

18:22Transparency about AI development is broadly beneficial in a variety of worldviews, even if there is no RSI or strong AGI. So that is a quick summary of the common ground. And again, I want to come back to why I think this is actually a relevant conversation for all of us. As you might guess, I find myself somewhere between these two camps. I agree with the AI 2027 folks on the potential scale of the disruption, but where I leave this discussion and others from this side of the AI safety world is the presumption that I often find that we're just going to sleepwalk into all of this and not see signals along the way that would allow us to change course.

18:55Now, my guess is that my AI safety friends would say that the point of having these conversations is exactly so that we don't sleepwalk into it. But I certainly am not operating on the same timescales as, for example, the AI 2027 folks. And I don't think that the outcomes that they're talking about are guaranteed on any timescale. That puts me in some ways more into the AI as normal technology camp. However, I do think that there are some quite abnormal aspects of this. I think that the potential for short-term dislocation in particular is perhaps bigger than they're giving it credit for. And I think that is where you're going to see a lot of the politics of AI resolve.

19:26We're already seeing this right now. Job losses that are almost assuredly, and in large part a recalibration in the post-COVID, post-SERP era, are being laid at the foot of AI, even when companies say that that's not what it's about. The robots become easy villains in an era where the economy just isn't working for most people. And I think even if it weren't for those politics, the magnitude, breadth, and speed of the disruption of AI are happening in a way that makes it hard to compare to the internet. There's nothing in equivalent internet history to a tenth of the world using a single application of AI less than three years after it was created.

20:00We've never had a technology where in the business world, every single part of the organization is simultaneously thinking about how it's going to change what they do and very seriously asking the question of whether they need to throw out the org chart entirely because of it. I think that while the normal tag is good to calibrate some of the more speculative scenarios of the folks of the AI 2027, what it risks is underselling the potential magnitude of the disruption in the short term. But here's the positive thing, and why I thought this was worth an episode. In our current climate, based on both politics, but also the algorithms of outrage that dictate the networks that we all spend our time in, it is almost assuredly the case that the loudest voices on the accelerationist side and the safetyist side will get the biggest media share for their opinions.

20:43Meanwhile, there will be an incredibly vast common ground with tons of common sense alignment that can be used as a foundation to progressively tackle harder and harder challenges. There is no universe in which we agree on all of the right ways to handle a technology this unknowable and fast-moving. But by stacking up wins in areas of agreement, we have a much better basis to have those harder conversations around. I don't want the AI Daily Brief to become just a show debating the macro aspects of AI. I think that the core of audience of you listeners are here because you're bought into the idea that this is going to change the world that you operate in, and you want to get ahead in how you operate within it.

21:20That's why there will continue to be as much emphasis as possible on new models, new developments, and the information that is going to be relevant in a very short-term way for all of you. But these conversations are going to be floating around. And frankly, you as this audience of listeners are in a better position than many to help others around you understand them. So hopefully this episode and the essay that inspired it were useful. We'll do more of this as is appropriate, all the while marching ahead into this dynamic and exciting, if nothing else, future. For now, that's going to do it for today's AI Daily Brief.