Sam Altman on The Singularity

0:00Today on the AI Daily Brief, Sam Altman on the singularity. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.

0:15Hello, friends. Quick announcements. First of all, thank you to today's sponsors, KPMG, Blitzy, Vanta, and Superintelligent. For an ad-free version of the show, which starts at just$3 a month, officially much less than a coffee if you have been to Starbucks lately, go to patreon.com slash AI Daily Brief. No other announcements today. I am finally on my way back from Europe. Next week should be normal. But today we get to read this new blog post from Sam Altman called The Gentle Singularity. A couple of days ago, Sam tweeted, wrote a new post, The Gentle Singularity. Realized it may be the last one like this I write with no AI help at all.

0:48Now, as usual, we're going to read the piece first, and then we'll come back and discuss it. And once again, this will be me personally reading, not AI. Although I have to say, with the new audio tags feature of 11labsv3, which allows me to specifically instruct it where I want emphasis or laughing or giggles or whatever I might want to make it more real. I am definitely going to try some AI reading again, but for now, you got just me, as most of you seem to prefer. Sam writes, We're past the event horizon. The takeoff has started. Humanity is close to building digital superintelligence. And at least so far, it's much less weird than it seems like it should be.

1:24Robots are not yet walking the streets, nor are most of us talking to AI all day. People still die of disease. We still can't easily go to space. There is a lot about the universe we don't understand. And yet, we have recently built systems that are smarter than people in many ways, and are able to significantly amplify the output of people using them. The least likely part of the work is behind us. The specific insights that got us into systems like GPT-4 and O3 were hard won, but will take us very far. AI will contribute to the world in many ways, but the gains to quality of life from AI driving faster scientific progress and increased productivity will be enormous.

1:58The future can be vastly better than the present. Scientific progress is the biggest driver of overall progress. It's hugely exciting to think about how much more we could have. In some big sense, ChatGPT is already more powerful than any human who has ever lived. Hundreds of millions of people rely on it every day for increasingly important tasks. A small new capability can create a hugely positive impact. A small misalignment multiplied by hundreds of millions of people can cause a great deal of negative impact. 2025 has seen the arrival of agents that can do real cognitive work. Writing computer code will never be the same.

2:322026 will likely see the arrival of systems that can figure out novel insights. 2027 may see the arrival of robots that can do tasks in the real world. A lot more people will be able to create software and art. But the world wants a lot more of both, and experts will still probably be much better than novices as long as they embrace the new tools. Generally speaking, the ability for one person to get much more done in 2030 than they could in 2020 will be a striking change, and one many people will figure out how to benefit from. In the most important ways, the 2030s may not be wildly different.

3:01People will still love their families, express their creativity, play games, and swim in lakes. But in still very important ways, the 2030s are likely going to be wildly different from any time that has come before. We do not know how far beyond human-level intelligence we can go, but we're about to find out. In the 2030s, intelligence and energy, ideas and the ability to make ideas happen, are going to become wildly abundant. These two have been the fundamental limiters on human progress for a long time. With abundant intelligence and energy and good governance, we can theoretically have anything else.

3:32Already, we live with incredible digital intelligence, and after some initial shock, most of us are pretty used to it. Very quickly we go from being amazed that AI can generate a beautifully written paragraph, to wondering when it can generate a beautifully written novel. Or from being amazed that it can make life-saving medical diagnoses, to wondering when it can develop the cures. Or from being amazed it can create a small computer program to wondering when it can create an entire new company. This is how the singularity goes. Wonders become routine and then table stakes. We already hear from scientists that they are two or three times more productive than they were before AI.

4:06Advanced AI is interesting for many reasons, but perhaps nothing is quite as significant as the fact that we can use it to do faster AI research. We may be able to discover new computing substrates, better algorithms, and who knows what else. If we can do a decade's worth of research in a year or a month, then the rate of progress will obviously be quite different. From here on, the tools we have already built will help us find further scientific insights and aid us in creating better AI systems. Of course, this isn't the same thing as an AI system completely autonomously updating its own code.

4:35But nevertheless, this is a larval version of recursive self-improvement. There are other self-reinforcing loops at play. The economic value creation has started a flywheel of compounding infrastructure build-out to run these increasingly powerful AI systems. and robots that can build other robots, and in some sense data centers that can build other data centers, aren't that far off. If we have to make the first million humanoid robots the old-fashioned way, but then they can operate the entire supply chain, digging and refining materials, driving trucks, running factories, etc., to build more robots, which can build more chip fabrication facilities, data centers, etc., then the rate of progress will obviously be quite different.

5:10As data center production gets automated, the cost of intelligence should eventually converge to near the cost of electricity. People are often curious about how much energy a ChadGPT query uses. The average query uses about 0.34 watt hours, about what an oven would use in a little over one second, or a high-efficiency light bulb would use in a couple of minutes. It also uses about 0.000085 gallons of water, roughly 1 15th of a teaspoon. The rate of technological progress will keep accelerating, and it will continue to be the case that people are capable of adapting to almost anything. There will be very hard parts like whole classes of jobs going away, but on the other hand, the world will be getting so much richer so quickly that we'll be able to seriously entertain new policy ideas we never could before.

5:51We probably won't adopt a new social contract all at once, but when we look back in a few decades, the gradual changes will have amounted to something big. If history is any guide, we will figure out new things to do and new things to want, and assimilate new tools quickly. Job change after the Industrial Revolution is a good recent example. Expectations will go up, but capabilities will go up equally quickly, and we'll all get better stuff. we will build ever more wonderful things for each other. People have a long-term important and curious advantage over AI. We are hardwired to care about other people and what they think and do, and we don't care very much about machines.

6:24A subsistence farmer from a thousand years ago would look at what many of us do and say we have fake jobs, and think that we are just playing games to entertain ourselves since we have plenty of food and unimaginable luxuries. I hope we will look at the jobs a thousand years in the future and think they are very fake jobs, and I have no doubt they will feel incredibly important and satisfying to the people doing them. The rate of new wonders being achieved will be immense. It's hard to even imagine today what we will have discovered by 2035. Maybe we will go from solving high-energy physics one year, to beginning space colonization the next year, or from a major material science breakthrough one year, to true high-bandwidth brain-computer interfaces the next year.

6:58Many people will choose to live their lives in much the same way, but at least some people will probably decide to plug in. Looking forward, that sounds hard to wrap our heads around, but probably living through it will feel impressive but manageable. From a relativistic perspective, The singularity happens bit by bit, and the merge happens slowly. We are climbing the long arc of exponential technological progress. It always looks vertical looking forward and flat going back, but it's one smooth curve. Think back to 2020 and what it would have sounded like to have something close to AGI by 2025, versus what the last five years have actually been like.

7:29There are serious challenges to confront along with the huge upsides. We do need to solve the safety issues technically and societally, but then it's critically important to widely distribute access to superintelligence given the economic implications. The best path forward might be something like, one, solve the alignment problem, meaning that we can robustly guarantee that we get AI systems to learn and act towards what we collectively really want over the long term. Social media feeds are an example of misaligned AI. The algorithms that power those are incredible at getting you to keep scrolling and clearly understand your short-term preferences, but they do so by exploiting something in your brain that overrides your long-term preference.

8:03Two, then focus on making superintelligence cheap, widely available, and not too concentrated with any person, company, or country. Society is resilient, creative, and adapts quickly. If we can harness the collective will and wisdom of the people, then although we'll make plenty of mistakes and some things will go really wrong, we will learn and adapt quickly and be able to use this technology to get maximum upside and minimal downside. Giving users a lot of freedom, within broad bounds society has to decide on, seems very important. The sooner the world can start a conversation about what these broad bounds are and how we define collective alignment, the better.

8:35We, the whole industry, not just OpenAI, are building a brain for the world. It will be extremely personalized and easy for everyone to use. We will be limited by good ideas. For a long time, technical people in the startup industry have made fun of the idea guys, people who had an idea and were looking for a team to build it. It now looks to me like they are about to have their day in the sun. OpenAI is a lot of things now, but before anything else, we are a super intelligence research company. We have a lot of work in front of us, but most of the path in front of us is now lit, and the dark areas are receding fast.

9:06We feel extraordinarily grateful to get to do what we do. Intelligence too cheap to meter is well within grasp. This may sound crazy to say, but if we told you back in 2020 that we were going to be where we are today, it probably sounded more crazy than our current predictions about 2030. May we scale smoothly, exponentially, and uneventfully through superintelligence. Today's episode is brought to you by KPMG. In today's fiercely competitive market, unlocking AI's potential could help give you a competitive edge, foster growth, and drive new value. But here's the key. You don't need an AI strategy.

9:39You need to embed AI into your overall business strategy to truly power it up. KPMG can show you how to integrate AI and AI agents into your business strategy in a way that truly works and is built on trusted AI principles and platforms. Check out real stories from KPMG to hear how AI is driving success with its clients at www.kpmg.us slash AI. Again, that's www.kpmg.us slash AI. This episode is brought to you by Blitzy. If you're a technology leader, here's something that probably sounds familiar. Your organization's competitive edge is buried in legacy code that desperately needs modernization, but the resources required feel out of reach.

10:19That was the case for a global investment analysis firm. They needed to migrate 70 ,000 lines of complex MATLAB financial algorithms to Python. Algorithms that drive investment decisions for trillions in assets. Their estimate? Months of high-cost specialized engineering work. Instead, they partnered with Blitzy. Blitzy's autonomous AI preserved mathematical precision and generated over 80 % of the new codebase, completing the migration with just five days of engineering time. They cut the timeline by 95 % and saved 880 engineering hours. If your organization is facing similar modernization challenges, visit blitzy.com to schedule a consultation and discover how AI-powered development can transform your technical capabilities.

11:00Today's episode is brought to you by Vanta. In today's business landscape, businesses can't just claim security, they have to prove it. Achieving compliance with a framework like SOC 2, ISO 27001, HIPAA, GDPR, and more is how businesses can demonstrate strong security practices. The problem is that navigating security and compliance is time-consuming and complicated. It can take months of work and use up valuable time and resources. Vanta makes it easy and faster by automating compliance across 35-plus frameworks. It gets you audit-ready in weeks instead of months and saves you up to 85 % of associated costs.

11:33In fact, a recent IDC white paper found that Vanta customers achieve$535 ,000 per year in benefits, and the platform pays for itself in just three months. The proof is in the numbers. More than 10 ,000 global companies trust Vanta. For a limited time, listeners get$1 ,000 off at vanta.com slash nlw. That's v-a-n-t-a dot com slash NLW for$1 ,000 off. Today's episode is brought to you by superintelligence, specifically agent readiness audits. Everyone is trying to figure out what agent use cases are going to be most impactful for their business, and the agent readiness audit is the fastest and best way to do that.

12:10We use voice agents to interview your leadership and team and process all of that information to provide an agent readiness score, a set of insights around that score and a set of highly actionable recommendations on both organizational gaps and high value agent use cases that you should pursue. Once you've figured out the right use cases, you can use our marketplace to find the right vendors and partners. And what it all adds up to is a faster, better agent strategy. Check it out at bsuper.ai or email agents at bsuper.ai to learn more. All right, so back to NLW here. First of all, one of the reasons that I read anything like this from Sam or Dario, it wouldn't matter if I disagreed with every single word.

12:50These are fundamental architects of the future, and when they write things like this, lots and lots of people are going to pay attention. Now, people have had a generally positive response to this essay. Certainly, there have been some who have been sharing their general critiques that in many cases amount to this is too much power for one company to have. Jeffrey Miller at Primal Poly responded to Sam saying, democracy means absolutely nothing if people don't get to vote on whether we want this singularity, which probably leads straight to human extinction. Do you support running a global referendum on whether we allow you guys to persist in trying to summon these super-intelligent demons in the hope that they'll play nice with us and destroy our current civilization gently?

13:24Now, obviously, that is a very specific take coming from a very ex-risk-centric view, but the point is that there are a number of responses to it on that level. Another response that I saw, summed up by Professor Ethan Malek, was that boy, these guys are not being subtle in their predictions. Ethan writes, one thing you can definitely say about Sam and Dario is that they are making very bold, very testable predictions. We will know whether they are right or wrong in a remarkably short time. When people asked what predictions he meant, he pointed to 2026 being the year that we see the arrival of systems that can figure out novel insights, 2027 being the year of the arrival of robots that can do tasks in the real world.

14:02Now that systems that can figure out novel insights is a really important one. I think for many, that represents a key inflection point before which AI really is still just a token prediction machine. I think many people's sense of what AI can accomplish will change if we start to see those novel insights. There has been some commentary on Sam's paragraph about electricity use without commenting on where these numbers came from or how accurate they are. This was certainly the part of the essay that was the most pretending to just be a casual reference when it actually was trying to dunk on an entire category of critique.

14:33And frankly, if there was one thing that got the most attention, it was the example of social media feeds as misaligned AI. A lot of people resonated with that argument. And I actually think it's incredibly useful as a way to get people to care about this question of alignment to frame it in those terms. I read Jeffrey Miller's critique just a second ago. but there are a lot of people, myself included, who simply disengage when your starting point is a presumption, a 100 % assurance that this is likely to lead to human extinction. If you force people to buy into that before they care about any sort of alignment question, guess what?

15:11You're not going to get a real good conversation about alignment. Whereas I think right now, a huge number of people, perhaps the preponderance of people, feel deeply this idea of misaligned AI in social media feeds. whichever social media channel they have segmented themselves into because of their personal politics, there is clearly a brokenness in that system that leaves us feeling worse than before we were interacting with it. So what ultimately is the point of this? I think it is exactly as Sam says, to try to get people used to the idea that as they've just been living their lives day in, day out, something fundamental has changed.

15:47This was a soft call for more engagement on some of the big questions of how we changed the social contract, but more a preparatory nudge that that was going to come soon. This was basically the first alarm followed by a snooze button for some of the most important conversations we'll ever have as a human species. For now, though, that is going to do it for today's AI Daily Brief. Appreciate you listening or watching as always, and until next time, peace.

16:20clean