Ilya Sutskever is Back Building Safe Superintelligence

0:00Today on the AI Daily Brief, former OpenAI chief scientist and co-founder Ilya is back with a new AI startup. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes.

0:24Welcome back to the AI Daily Brief Headlines Edition, all the daily AI news you need in around five minutes. Today, we kick off with an announcement shared first on X.com yesterday by Michael Dell. He tweeted a picture of a warehouse with a bunch of server racks that said, we're building a Dell AI factory with NVIDIA to power Grok for XAI. 2.1 million views later, people were pretty excited. Bloomberg's Ed Ludlow reported this morning Dell plus 3 % in pre-market. MarketWatch also covered the news, writing that Elon Musk's XAI buildup plans spark gains by Dell and Supermicro. A confirmation from Elon Musk that he's using technology from Dell Technologies in a supercomputer for his AI company XAI sent shares of the two technology names higher.

1:04Elon confirmed on X that Dell is assembling, quote, half the racks that are going into the supercomputer that XAI is building. The Wall Street Journal was also positive on this, writing, Dell has seen its market value surge over the past 12 months as booming sales of NVIDIA's AI processors have also sparked strong demand for the types of specialized servers that run on those chips. Dell's most recent financial results bore that out as server and networking revenue surged 42 % year-over-year to$5.5 billion in the fiscal first quarter ended April. This was the best growth that segment had seen in at least eight years.

1:32They did point out that operating margins had compressed, but overall, WSJ didn't seem particularly concerned about that. If you are interested more in the market analysis side of AI, go check out yesterday's episode. We go deep into that after NVIDIA became the world's most valuable company. Next up, a follow-up from a previous podcast story. You might remember when we talked about two different candidates that were running for political office as AIs, one in Wyoming, one in the UK. Alas, they will have to find a new strategy as OpenAI has shut down access to their tools for those candidates.

2:05On Tuesday, OpenAI told CNN that it had shut down the account for 42-year-old Victor Miller, who had filed paperwork to run for mayor of Cheyenne, Wyoming, under his AI chatbot, Vic, which is short for Virtual Integrated Citizen. Even if OpenAI hadn't done that, it's perhaps doubtful that this could have gone through anyways. Said Wyoming Secretary of State Chuck Gray, Wyoming law is clear that to run for office, one must be a qualified elector, which necessitates being a real person. Therefore, an AI bot is not a qualified elector. The other AI candidate who was removed from using OpenAI was Steve Endicott, chairman of an AI company himself called Neural Voice, who was using a chatbot called AI Steve to run for parliament.

2:42Associate Professor of Media and Public Affairs at George Washington University, David Karp, said gimmick is the right word for it. Chat GPT is not qualified to run your government. At this point, I think there are more than a few people who would raise their eyebrows at that assertion. Staying on political themes for a minute, The information has a really interesting piece called China's Top AI Startups Enter U.S., Defying Political Tensions. They write, Chinese tech companies are facing more scrutiny in the U.S. than ever before, yet some of China's AI startups are making a rush to grab U.S.

3:11customers. The reason? It's too hard to make any money in China. One example they give is a company called Moonshot AI, which they list as one of China's most valuable AI startups. The information writes, Its employees have been working on products that were recently launched in the U.S., including Ohai, an AI role-playing chat app, and Noisy, a music video generator. Another example they give is an app called Talkie, which has 11.4 million monthly active users and was compared to Character.ai. And overall, they argue that this is a much bigger trend. They write China's crowded AI market is fueling the international ambitions of Chinese AI startups.

3:41Really interesting phenomenon to watch, given US-China geopolitical tensions. Speaking of that, while of course with Apple's recent announcement we got their partnership strategy with OpenAI for the US market, the Wall Street Journal reports that the company is looking for a local AI partner for China as well. Writes the WSJ, OpenAI's ChatGPT and other Western AI models aren't available in China, and that's prompting Apple to look for a Chinese partner to help offer its Apple intelligence services. So far, no deal has been announced with the next iPhone model released just months away. In China, Apple is falling behind local rivals that have already incorporated AI functions into their phones.

4:14The iPhone dropped to third place by handset market share among smartphone brands in China in the first quarter of this year behind two local brands. It should be very interesting to watch play out. Is this a counterforce to the growing tension between the U.S. and China? Or will this just exacerbate things from a political perspective? Finally today, over in the world of retail, Target has announced a new AI tool to both help employees as well as improve the shopping experience. Target announced that it's rolling out what it calls its store companion chatbot as an app for employees at all of its almost 2 ,000 stores.

4:44Writes Axios, it will be featured on worker handheld devices to answer questions about processes and procedures. The system will be able to answer questions about topics like the Target Circle loyalty card, the location of products throughout the store, and how to restart the cash register. Basically, this sounds like a customized chatbot specifically trained on the Target experience. Target says the goal of the store companion is not to replace workers, but to quote empower our team and deliver a great guest experience. And with that friends, we are going to wrap today's AI Daily Brief. Up next, the main AI breakdown.

5:15Today's episode is brought to you by Fractional AI, my go-to AI dev shop. When we wanted to build a specific AI product feature, what we called our AI tool finder bot, for super intelligent. We knew we needed incredible AI engineers that could build something fast. And so we turned to Fractional because they've got some of the best AI engineers on the planet. The team is a group of senior engineers in San Francisco working on some of the most exciting projects in applied AI. They work with everyone from startups all the way through the Fortune 500. To request a free consultation, head to fractional.ai.

5:46And again, if you want help identifying and building AI projects for your business, I highly recommend you check them out. Again, that URL is fractional.ai to request a free consultation. A quick note before we get back to the show, today's episode is brought to you by Super Intelligent. Super is, of course, the platform that we built and released a couple months ago to help people learn how to use AI. It's built around fun, fast tutorials that get you actually using AI in minutes, not hours, and certainly not days. The learning all happens in the context of an engaged community, and we've got a bunch of exciting features rolling out in the weeks to come.

6:20One of those is a new Teams version of the platform, which includes a custom curated playlist, as well as a showcase where people on your team can share their use cases and projects in AI across the organization. If you are interested in being a part of the Super for Teams beta, go to bsuper.ai slash partner so you can learn about the program. Welcome back to the AI Daily Brief. One of the most followed sub-stories in artificial intelligence has been the destiny of Ilya Sutskever. Ilya was, of course, one of the founders of OpenAI as well as its chief scientist, and had a very strange role in the Sam Altman ouster last year.

6:56As one of the board members, Ilya was initially involved in kicking Sam off, but then part of the way through the whole dust-up, he switched teams, which in many ways sealed Sam's coming back. However, Ilya's future at the company was very unclear. For months, it was one of the main questions that Sam Altman got asked. What was Ilya doing? On Twitter, there were memes asking what Ilya saw and where Ilya was. Altman, for his part, kept saying that he hoped that he was able to continue working with Ilya. However, in May, almost exactly six months after the whole thing happened, Ilya announced finally that yes, he was in fact leaving OpenAI.

7:31This led to many people thinking that perhaps there was some deal cut where Ilya agreed to stay on or at least not quit for some consolidation six-month type period, but maybe he was always planning to go do something else. Well, now we have learned what that something else is. Ilya yesterday tweeted, I am starting a new company. We will pursue safe superintelligence in a straight shot, with one focus, one goal, and one product. We will do it through revolutionary breakthroughs produced by a small cracked team. The company is called Safe Super Intelligence, Inc., SSI. The announcement post on Twitter slash X reads, superintelligence is within reach.

8:06Building safe superintelligence, SSI, is the most important technical problem of our time. We've started the world's first straight shot SSI lab with one goal and one product. a safe superintelligence. SSI is our mission, our name, and our entire product roadmap, because it is our sole focus. Our team, investors, and business model are all aligned to achieve SSI. We approach safety and capabilities in tandem as technical problems to be solved through revolutionary engineering and scientific breakthroughs. We plan to advance capabilities as fast as possible while making sure our safety always remains ahead.

8:36This way we can scale in peace. Our singular focus means no distraction by management, overhead, or product cycles, and our business model means safety, security, and progress are all insulated from short-term commercial pressures. We are an American company with offices in Palo Alto and Tel Aviv, where we have deep roots in the ability to recruit top technical talent. We are assembling a lean, crack team of the world's best engineers and researchers dedicated to focusing on SSI and nothing else. If that's you, we offer an opportunity to do your life's work and help solve the most important technical challenge of our age.

9:04Now is the time. Join us. Now, the other folks joining Ilya included Daniel Levy, who tweeted, Beyond excited to be starting this company with Ilya and DG. I can't imagine working on anything else at this point in human history. If you feel the same and want to work in a small, cracked, high-trust team that will produce miracles, please reach out. Serial entrepreneur and prolific AI investor Daniel Gross tweeted, It is a great pleasure and honor to co-found this new endeavor. Alongside the X.com announcement, Ilya also spoke with Ashley Vance at Bloomberg. What did we learn from that interview?

9:33Well, to some extent, it was just a repeat of what we'd heard, and there was still a lot of caginess. Ilya reiterated that he was trying to shape this company in a totally different way. He said this company is special in that its first product will be safe superintelligence and it will not do anything else up until then. It will be fully insulated from the outside pressures of having to deal with a large and complicated product and having to be stuck in a competitive rat race. What we didn't learn is anything about who was backing the company, how much had been raised, or even more technical things like how Ilya is currently thinking about AI safety.

10:03He did take a chance to shoot at OpenAI, saying by safe, we mean safe like nuclear safety as opposed to safe as in trust and safety. Nate Chan pointed out that one of OpenAI's principles was be a pioneer in trust and safety. So what has the commentary been? One theme is that this definitely creates competitive pressure with OpenAI, at least in terms of people. Didi from Menlo Ventures writes, OpenAI is about to lose a lot of talent. Theo Jaffe writes, The year is 2021. A group of OpenAI employees are worried about the company's lack of focus on safe AGI and leave to start their own lab. The year is 2023.

10:35An OpenAI co-founder is worried about the company's lack of focus on safe AGI, so he starts his own lab. The year is 2024, dot dot dot. Another common response was sheer excitement. May at MultiplyMatrix writes, No one on the planet has contributed more to AGI than Ilya Sutskever. He triggered two of AGI's acceleration events. He proved to the world we could use GPUs to train state-of-the-art models, bringing a beacon of hope to the field of AI which was withering in an AI winter. Second, Ilya supported Alec Radford's scaling of GPT as an early believer in Ash Vaswani's Transformer. May continues, Today most thinking is done by humans, but soon most thinking will be done by machines.

11:11Over the next few years, humans will build powerful machines that can reason and act. The chatbots we use today are nothing compared to the superintelligence we are about to train. Superintelligence will very soon win gold medals in the Math Olympiad. They will make fundamental discoveries in math and science. and they will act agentically in our multimodal world over long horizons. We're at the steepest inflection point in the history of compute. We're racing to bring trillion-dollar clusters online to train billions of recursively self-improving superintelligent agents. We will arrive at a takeoff point.

11:37The singular mission that Ilya has dedicated his life to, safe superintelligence, is the most important challenge in our lifetimes. And most importantly of all, Ilya is a deeply kind and compassionate human being who is building superintelligences that can behave kindly and compassionately to human beings. Victor Talon writes, No users, no revenue, no business. One shot, one goal, one product. Research a technology and change the world. Ilya is a legend beyond our times. Still, if many people were excited about that focus, there was also some amount of skepticism. Matt Wolf writes, I'm excited to see what they do, but also curious about the sustainability of it.

12:10Will investors dump millions or billions into a company to build data centers and giant GPU clusters knowing there's no revenue generation on the other side? Will this new company leverage other companies' GPU clusters? And if so, how can they be sure it's as safe as possible while playing in someone else's playground? So many questions that I'm sure we'll learn more about soon. Shaquille writes, You probably need north of$100 billion for a cluster capable of training superintelligence if such a thing is even possible. That's not including salaries, etc. With a business model that says no products until then, where on earth is that money going to come from?

12:38Perhaps a little bit more cynically, Daniel Jeffries writes, The first question I'd ask a superintelligence if I had such a thing is this riddle. How do I build ASI with no revenue and no product plans? Professor Ethan Malek writes, how do you price an option on superintelligence or value a company trying to do everyone's job forever? Wild times. And yes, if you think superintelligence is impossible, the value is zero. But the people investing obviously view this as possible, so... And I think this actually gets a little bit closer to the nut of this. Ultimately, this is an incredibly binary bet.

13:07The reason that an investor would be willing to make this bet is basically that they believe that literally whatever it costs to get there, the value on the other side of this creation is so enormous that it justifies it. They probably buy the argument that, in fact, worrying about revenue in the short term is a net distraction. To Ethan's point, many investors would think that is crazy, but for the ones who don't, it sort of makes a type of sense. There's also a bit of skepticism about the capacity to actually achieve this. Pedro Domingos writes, Ilya Sutsgever's new company is guaranteed to succeed because superintelligence that is never achieved is guaranteed to be safe.

13:45Zestula writes, call me crazy, but now we have open AI Anthropic, Google Meta, Mistral, and now SSI all taking in billions of dollars and hiring the smartest people in the world. And they're all building more or less the same thing, scaling up Transformers? How does this end? Well, one way that people noted that it would end was with Nvidia getting even richer. For now, it is all speculation. What's undeniable is that Ilya's project represents a new force in this space that will be taken seriously and that cannot be ignored. I will, of course, keep you guys informed as we learn about what they're doing, although I don't anticipate us hearing all that much very often.

14:18For now though, that is going to do it for today's AI Daily Brief. Welcome back, Ilya, and until next time, peace.