OpenAI Quietly Abandoned Arrakis Model Earlier This Year

0:01Today on the AI Breakdown, we're looking at all of the latest news from OpenAI, including the fact that they had to shut down a promising model earlier this year. Before that, in the brief, rumors that Apple is going to release generative AI features in iOS 18. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our Discord, our YouTube channel, and our newsletter. Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. Today, we start with some juicy rumors in one of the areas of discussion that has really been confounding for people this year, which is, of course, what the heck Apple is going to do in the generative AI space.

0:44Now, what we know for sure is that Apple has been pretty unconclusive, even amongst itself, around what it wants its approach to be. Throughout the year, we've had reports that Apple had been involved in training their own models and even experimenting with them internally, but that they just weren't sure exactly what they wanted their approach to be. Now, part of this is probably due to the fact that Apple has very different feelings about privacy as relates to other companies. They also like to run software directly on their devices, rather than having to rely on the cloud. Of course, the state-of-the-art in LLMs right now doesn't really allow for that type of approach, at least not with some very different thinking.

1:19And yet, even with all of this, it has seemed like there has been more energy recently going into figuring out what Apple's approach is going to be. Specifically, in September, we got a report from The Information that Apple was now spending millions of dollars a day on training AI models. That sort of financial outlay suggests that they are moving closer towards doing something in the space. And yesterday, MacRumors shared an analysis from Jeff Poo, who's an analyst who covers Apple's supply chain for a Hong Kong investment firm, that generative AI on the iPhone and iPad is coming potentially in late 2024.

1:51Specifically, it sounds like there could be generative AI features as soon as iOS 18, which is of course the next operating system for Apple devices. From the piece, in a research note this week, Poo said his supply chain checks suggest that Apple is likely to build a few hundred AI servers in 2023, and significantly more next year. He believes Apple will offer a combination of cloud-based AI and so-called edge AI, which involves more on-device data processing. Now, back in August, another supply chain analyst, Ming-Chi Kuo, said that given how behind Apple's generative AI efforts were, it seemed like that late 2022 timeframe might not be realistic, and that Apple might not really be in the market with something until 2025 or beyond.

2:29Now, of course, whenever Apple does do something in the generative AI space, it's going to be a huge deal, and especially if it comes natively integrated into the iPhone, Even with hundreds of millions of new users between now and then, it's likely to still represent a significant mainstreaming moment. Now, while that Apple news is all still firmly in the realm of rumor, or if not rumor, analysis based on things like supply chains, some other news related to Apple is a little bit more clear. Apple scored a coup when they brought Jon Stewart out of Daily Show retirement to create a new news show, The Problem with Jon Stewart.

3:01Apparently the show was about to start production of its third season, and that is where they ran into challenging quote-unquote creative differences. Writes The Verge, Stewart's intended discussions of artificial intelligence and China were a major concern for Apple. Though new episodes of the show were scheduled to begin shooting in just a few weeks, staffers learned today that production had been halted. Apparently what had happened is that Apple came to Jon Stewart directly, and said that he and his team needed to be quote-unquote aligned with the company's views on those thorny topics of AI in China.

3:30Stewart, unfortunately for Apple, said kindly no thank you and decided to leave the show. Now, it is important to note that we don't actually know the specifics of their differences in opinion. In other words, we don't know what the planned coverage of AI or China was that Apple had a problem with. So take with a grain of salt anyone contending to have good information around that. But still, it feels likely and is the assumption of most that whereas John Stewart would perhaps be particularly critical of China, Apple is trying to walk a tightrope of not upsetting that country who represents a huge market for them.

4:01Hopefully we get more information about what opinions were of disagreement, but for now, discussion of AI and China with Jon Stewart are out, at least when it comes to Apple TV. Next up, let's move over into the world of hardware. IBM has just released research about a new chip that is specifically focused on AI, and which suggests that there are some pretty meaningful advances here. The processor is called North Pole. And the specific innovation is to take an approach that doesn't require the chip to as frequently access external memory, aka RAM, which means that it can not only perform tasks faster, such as image recognition, but it can do so while also consuming less power.

4:39Said Nanoelectrics researcher Damien Querliaz, its energy efficiency is just mind-blowing. I feel the paper will shake the common thinking in computer architecture. Basically, this chip puts memory in each of its 256 computing cores, which means less of having to shuttle data between chips. Writes Nature, the cores are wired together in a network inspired by the white matter connections between parts of the human cerebral cortex. This and other design principles, most of which existed but had never been combined in one chip, enable North Pole to beat existing AI machines by a substantial margin in standard benchmark tests of image recognition.

5:11It also uses one-fifth of the energy of state-of-the-art AI chips, despite not using the most recent and most miniaturized manufacturing processes. Now, importantly, this is just a research stage, And North Pole doesn't currently have the ability to work for large language models. As Nature writes, the chip can only run pre-programmed neural networks that need to be trained in advance on a separate machine. So what's the utility here? Well, one piece of it is that its architecture could be used in speed-critical applications such as self-driving cars, where the ability to pre-program those functions is there, and of course there's the possibility of continuing to evolve these new approaches to chip design.

5:43I think overall it's a reminder that alongside the rise in demand for AI software applications, it's highly likely that we're going to see a significant amount of hardware innovation as well. Last up today, another fun one from the world of science, AI has for the first time ever detected a supernova all on its own. So discovering supernovas is a really difficult and labor-intensive process. Astronomers and scientists basically have to hand-go-through huge, huge amounts of information and manually identify potential candidates that could be supernovas. Well, now a team from Northwestern, who are in fact my alma mater, have created something called the Bright Transient SurveyBot, which does all the painful intermediary work all on its own.

6:21Gizmodo writes, Researchers fed the BTS bot machine learning algorithm 1.4 million images from 16 ,000 astronomical sources. Those images included past evidence of supernovae, glaring galaxies, and temporarily flaring stars. Equipped with that training set, the AI model was able to identify a new supernova candidate and automatically request its spectrum reading from a robotic telescope at the Palomar Observatory in California. The system eventually identified the supernova candidate as a stellar explosion, in which a white dwarf star fully exploded and it automatically shared its finding with the astronomical community.

6:50In other words, the AI system identified and shared the new discovery all on its own. Great news to the humans involved. Now, this team at Northwestern says that in the last few years, researchers have spent 2 ,200 hours doing the work that now might be able to be done by BTSBot and other technology like it, which obviously frees them up for much more advanced research and digging deeper into other astronomical mysteries. Many folks, including notably Sam Altman, think that one of the most profound impacts of artificial intelligence is likely to be the way that it increases the rate and speed of scientific discovery.

7:20And this is certainly further evidence of how that might work. However, that is going to do it for today's AI Breakdown Brief. Thanks as always for listening or watching. Up next, the main AI breakdown. Are you interested in how two top-of-mind trends, AI and crypto, can work together? If so, I have the perfect podcast recommendation for you. Web3 with A16Z Crypto, the chart-topping show brought to you by venture firm Andreessen Horowitz. Web3 with A16Z Crypto is your definitive resource for the future of the internet, whether you're already building in these spaces or simply curious about what's next.

7:56If you need a place to start, they recently released an excellent episode with Stanford cryptography professor Dan Bonet and former Google Xer Ali Yahya in conversation with hosts on Al Choksi about the intersection of AI and crypto. From fighting deepfakes and proving humanity to large language models like ChatGPT, they cover it all. I highly recommend checking it out, especially if you'd like to learn more about how AI and crypto will impact our everyday lives. Beyond Crypto and AI, this show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics.

8:29So go ahead, listen to Web3 with A16Z Crypto wherever you get your podcasts. Welcome back to the AI Breakdown. Today we are covering a variety of stories that have to do with open AI. There are some fairly significant pieces in here, but more than that, and this goes for any time basically that I cover open AI, there are some times when a single company comes to define an entire larger industry or an entire larger movement. And right now that is certainly the case with OpenAI and generative AI more broadly. Now, I don't think that this will always be the case, but I do think that what is pretty inarguable is that this field leaped into general consumer consciousness with the launch of ChatGPT.

9:13Between that and the launch of GPT-4, which has defined the outer barrier of models we have access to, for better or worse, OpenAI is just the dominant force in this space. It is a big question, an open question to many, around whether when Google releases Gemini, which potentially is even more advanced than GPT-4, will that sense or sentiment change? And for that, we'll have to wait and see. But for now, what's clear is that what happens to OpenAI and what OpenAI does has a major impact and is covered as carefully as anything else by people who are trying to keep track of the AI space. Given that, it was particularly interesting to read a piece in the information earlier this week about a model that they had actually dropped work on.

9:53Now, this is the Arrakis model, which was named for Dune. And by the way, I'm not a Dune scholar, so forgive me if I'm getting the pronunciation wrong. But basically, the important thing to know here is that one, OpenAI has been developing a number of different models concurrently, and that two, Arrakis was one that was supposed to have some benefits that others did not. So basically what happened is that right around the same time that ChatGPT came out, and the whole world was racing to catch up and understand what was going on, engineers inside OpenAI began working on a new model that was codenamed internally Arrakis.

10:23There were a few motivations for the project. One, it sounds like there was a business strategy motivation. As the information writes, success with Arrakis would help OpenAI show Microsoft how fast it could create successive large language models, which would be valuable as the two firms finished negotiating a$10 billion investment in product deal. But two, and I think even more importantly, as we've seen how the LLM space has evolved, the goal of the Arrakis model was to allow them to run ChatGPT and other tools with less cost. However, when push came to shove, the model underperformed what OpenAI had been hoping for, and ultimately by the middle of this year they scrapped the project entirely.

10:59So let's talk about the technical aspect of this for just a moment before we get back into the implications. As we just discussed, the goal of the Iraqis model was to be as powerful as GPT-4, but to run much more efficiently and at lower cost. The goal was to leverage a machine learning concept known as sparsity. The idea of sparsity very simply put and probably reductively put, is to basically only use parts of the model that are relevant to generate responses, meaning that it's cheaper to run. Now, while the initial tests were promising, it wasn't too long before they realized that Iraqis was just not performing well enough to be considered for public release.

11:31Unfortunately, why it wasn't working as well as expected was not something the information could suss out. Apparently in late spring, after identifying that things weren't going well, OpenAI's team spent about a month trying to fix the issues before senior leadership decided to pull the plug entirely. Now, there are, of course, a couple challenges with this. One is on the business and partnership front. The information writes the failure also disappointed some executives at Microsoft, which paid for the right to use the startup's technology and its products, according to a Microsoft employee with knowledge of the matter.

11:57More broadly, OpenAI just lost time, and time is the thing that no AI lab has. What's more, for the first time, OpenAI appeared vulnerable in some ways. As the information again put it, the Iraqi setback could pierce OpenAI's aura of invincibility. after it humbled AI pioneer Google and built one of the fastest growing software businesses in history. It shows how the frontier of AI is riddled with pitfalls that can be hard to predict. Now, of course, there was other model work going on at the same time. A multimodal model called Gobi has been in the works, and once Iraqis was shut down, the team involved pivoted to trying to work on a version of GPT-4 that was specifically focused on generating responses more quickly.

12:32Now, even though Iraqis didn't work, the goal of bringing down costs and having models that run more efficiently remains at the very top of priorities for not just OpenAI, but for all of these labs. Indeed, it appears from this article that some of the news that we've seen around Microsoft working with other LLMs might have been prompted by the Iraqis' failure. What's more, even though OpenAI's first attempt at this didn't work, many still expect people to experiment with this sparse model approach. Said Jeff Dean, Google's chief scientist, sparse computation is going to be an important trend in the future.

13:03Now, many commentators pointed out that there was a lot of information in this piece. In fact, a whole lot more than OpenAI was probably happy about. Former GitHub CEO and now investor Nat Friedman said, this is a whole lot of leakage from OpenAI. Now, speaking of leakage from OpenAI, another thing that everyone has been talking about for the past few weeks is the fact that there have been conversations for additional fundraising. The one that has seemed the farthest along was a tender offer of employee shares at a valuation that was initially reported to be between$80 and$90 billion. Apparently now that has been honed down to$86 billion and we're starting to get more information about who might be involved.

13:38Apparently the lead on the round is Thrive Capital, which is of course led by Joshua Kushner, who is at this point perhaps best known as the brother of Jared Kushner, and apparently up to a billion dollars worth of employee equity will be sold. Now many are noting that this would be a 3x jump in the paper valuation of the company. Indeed Thrive Capital actually bought employee shares back in April at a valuation of$27 billion. Now, apparently, reporting suggests that Sam Altman has said privately that he expects before all is said and done that OpenAI will raise$100 billion along its path to building AGI.

14:09The company has jumped from$28 million in revenue last year to a revenue run rate of around$1.3 billion right now. A few more OpenAI stories before we get out of here. One has to do with upcoming product releases. You'll remember that about a week ago. we cover news here of speculation that one of the things that OpenAI might introduce at their developer day at the beginning of November is a new set of AI agent tools. Well, according to TechCrunch, one tool that they are not sure when they will release or if they will release is a tool to detect AI-generated images. Now, you may remember that OpenAI previously had an AI detection tool that was out.

14:43It was designed to detect AI-generated text not only from OpenAI's ChatGPT, but also from other models. They pulled the detector because it had too low a rate of accuracy and was contributing to a sense that you actually could mechanically figure out with any level of accuracy what was written by machines as opposed to written by humans. This is actually a huge problem in the education field where there are companies out there selling schools and colleges on the idea that their detector can determine which of the students is using AI, creating a huge problem of false positives where students have no recourse to defend that they actually were not using these tools.

15:16Now, a number of people inside OpenAI have talked very positively about this AI image detection tool, with one researcher telling TechCrunch that the tool's accuracy is really good, and with OpenAI's CTO Mira Mirati saying at a Wall Street Journal event this week that the classifier is quote 99 % reliable at determining if an unmodified photo was generated using DALI 3. Now, of course, while we don't really have an answer to how we're going to deal with AI detection, there are a ton of companies that are trying to figure out the right approach. DeepMind has proposed a spec they call SynthID, which would mark AI-generated images in a way that humans can't see, but which can be identified by software.

15:50Adobe announced something similar at their event a couple weeks ago. And ultimately, it feels to me like this is going to be a problem that is not just a technology solution, but also a social solution. In other words, I don't expect that we're going to see a tool that every single company and every single person adopts. I think it's far more likely that when it comes to important images and information, people simply will choose not to trust things of their own volition unless they have a recognized and respected validation. Although who does that and how remains to be seen. Now, speaking of DALI-3, Latent Space host SWIX posted some new research this week and said, in a surprising for these times moves, OpenAI actually published a research paper detailing the improvements they made for DALI-3, primarily caption improvement via a fine-tuned image captioner and upsampling descriptions with GPT-4.

16:35So there are two pieces that are interesting to this. One is the actual information that this paper has. Again, Swick sums it up this way. At the end, 95 % of DALI-3's dataset was synthetically generated by this captioner. Illustrates the power of being a fully multimodal foundation model org. Working on text, vision, and image gen models concurrently improve each other. There's also an answer as to why DALI-3 can poorly generate text and images. They simply made sure it was included in the captions. No extra work done on character-level conditioning. So all of that is fascinating, especially if you are keeping track of how these image generation models are evolving.

17:06But I think just as interesting is the fact that they chose to release this research at all. This is not something that OpenAI has recently done. Ivy Zhang wrote, we are so back, OpenAI is open again. To which Swix cautioned, I didn't see that. We only see what they want us to see. Now, speaking of what they want us to see, obviously the next big event in OpenAI's world is the Dev Day at the beginning of November. So much so that Logan, who does developer relations over there, tweeted yesterday going heads down until OpenAI Dev Day. See y 'all on the other side. I expect lots more interesting rumors and leakages before then, and of course, some interesting announcements at the event itself.

17:41However, that is going to do it for today's show. Thanks for listening to the AI Breakdown. If you haven't yet, go sign up for the newsletter or come join us on the Discord. You can find all that information at breakdown.network. And until next time, peace.

18:00Thank you.