Did Google DeepMind Just Revolutionize Materials Science?

0:00Today on the AI Breakdown, we're looking at some serious materials breakthroughs from Google DeepMind. Before that on the brief, the new open source Chinese model at GPT 3.5 levels, plus new models from Amazon, and a big investment in the UK from Microsoft. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information about our Discord, our newsletter, and our YouTube channel. Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. One of the things we've been talking about quite a bit recently is the fact that still no one can beat GPT-4.

0:37Even as we see companies like Amazon launch their own models, which by the way we'll be talking about one of those later today, GPT-4 remains the king. And so of course I took notice when AI entrepreneur Bindu Reddy tweeted, This is insane. And just like that we have open source Quen72B beating GPT-4 on some benchmarks. China roars back at toxic late-stage capitalism and corporatism plaguing the US. join the global open-source revolution. Now, I'm not going to comment on the late-stage capitalism piece, but I certainly took note of the idea that Alibaba's open-source model was beating GPT-4 on some benchmarks.

1:09However, upon closer inspection, the evals that Quen beats GPT-4 on are in Chinese. Now, that's still good, but it's a little bit more understandable. And basically, we have here yet another GPT-3.5-style model. Bin-Wan Hui from Alibaba writes, We are proud to present our sincere open-source works, QN72B and QN1.8B. QN72B has been trained on high-quality data consisting of 3 trillion tokens, boasting a larger parameter scale and more training data to achieve a comprehensive performance upgrade. Additionally, we have expanded the context window length to 32K and enhanced the system prompt capability, allowing users to customize their own AI assistant with just a single prompt.

1:45Now, all of this is over on HuggingFace, huggingface.co slash QWEN if you want to check it out. Now, as your heart palpitations come down, there is another interesting China-US competitive story that comes through the proxy of Saudi Arabia. Bloomberg is reporting that the US has forced a Saudi fund to divest from an AI chip startup that was also backed by OpenAI's Sam Altman. Basically, a company called Prosperity7, which is a venture capital arm of Saudi Aramco, invested in the$25 million round for Rain AI last year. However, the Committee on Foreign Investment in the United States has now forced Prosperity7 to sell its shares and unwind the deal.

2:21Now, this is not publicly announced. This is all from Bloomberg sources. This fits certainly with a larger pattern. As Bloomberg writes, the U.S. is heightening scrutiny over the activity of Middle Eastern wealth funds, part of a growing resistance towards entities regarded as having close ties with China. CFIUS is reviewing several multi-billion dollar deals this year on concerns they could pose national security risks. Now, the company in question, Rain AI, is a startup that, quote, designs AI chips inspired by the way the brain works. Obviously, AI chips are a central piece of the China-US tension, and apparently whatever Prosperity 7 is doing is too valuable in the eyes of the Biden administration to let the Saudis and by extension the Chinese have access to it.

2:59Now, adding even more complication to this is of course the fact that all the reports suggest that Sam Altman has been working to raise billions of dollars for another novel chip project, codenamed Tigris, and has been seeking out Middle Eastern funds to do so. Bloomberg also makes clear, quote, while Altman is an early investor in Rain AI, it's not clear whether he remains active in the company or how he views the technology today. Now, moving over to the world of big tech and their AI efforts for a moment, when all was said and done, we did not get the rumored Olympus model at Amazon's reInvent, but they also didn't just leave us with the Q Enterprise chatbot that we talked about earlier in the week.

3:31We also had the announcement of the Titan Image Generator, which is akin to a DALI 3 or a Mid Journey, but has a couple notable features. First of all, Amazon claims that it has built-in guardrails against toxicity and bias. Second, every image created with Titan Image Generator has an invisible watermark that can identify it as a generative AI creation. Third, Amazon is extending copyright indemnification to customers who are using Titan's foundation models, including text-to-image. Basically, if you're building with Amazon's tools, they're going to protect you against copyright claims. Now, all of this is part and parcel of the bigger approach here, which is that Titan is not a standalone application or website, but something that's available to developers from within Amazon Bedrock.

4:11Now it's quite clear between the watermarking, the guardrails, the copyright indemnity, and the fact that this is available through Bedrock, that this is an enterprise-focused offering. This is meant to give big companies who are nervous about using something like MidJourney the comfort to still get in the image generation game. A lot of these efforts are similar to features we've seen from Adobe's Firefly, who has at least in part a similar target in mind. Moving over to Microsoft, Microsoft is spending a huge amount of money on building out its data capacity around the world, and it appears that the UK is going to be one of the major beneficiaries of that strategy.

4:43Announced earlier this week at a conference with UK PM Rishi Sunak, Microsoft is investing£2.5 billion or$3.2 billion into the company, including opening up data centers that house more than 20 ,000 GPUs. Said Sunak in a statement, Today's announcement is a turning point for the future of AI infrastructure and development in the UK. Now, the UK and Microsoft have had a little bit of a tense relationship over the last few months. Back in April, the UK's antitrust regulator went against Microsoft in its attempt to acquire Activision Blizzard, but subsequent to that, it waved through a restructured version of that deal, which apparently, as Reuters put it, put Britain back in Microsoft's favor.

5:18Said Microsoft President Brad Smith, Microsoft is committed as a company to ensuring that the UK as a country has world-leading AI infrastructure. Now, in addition to just GPUs, this big investment in the UK will include what Microsoft called a training plan to help ensure Britons have the skills they need to build and work with AI. Still, of course, the biggest thing in Microsoft land remains their relationship with OpenAI, and one of the things that people have wondered is whether they will get access to the board after being shocked by the board's action to fire Sam Altman just a couple of weeks ago.

5:48According to reporting from the information, via statements from restored OpenAI CEO Sam Altman, Microsoft will be on the board in a way, but as a non-voting observer. Basically, this will allow Microsoft to have visibility into the board's debates and deliberations, but it won't be able to influence decisions in a direct way. This to me suggests some amount of resilience in the OpenAI structure. I think many folks would have thought that Microsoft would have demanded a board seat in some new governance structure after this whole hullabaloo, but apparently for now at least it's content with this additional visibility.

6:18Of course, the board story is still very much up in the air. There are three members currently, Brett Taylor, the former Salesforce CEO who is chair, CORA CEO Adam D 'Angelo, who's the lone holdover from the last board, and former Treasury Secretary Larry Summers. As part of his compromise to come back as CEO, Altman is not currently on the board, although that may not be the case forever. Now, also in Sam's first official days back as CEO, we learned that Mira Mirati had gone back to her role as CTO, and that Greg Brockman will be returning as president. Now, one big open question is what happens to former board member and chief scientist Ilya Sutskaver.

6:51He seemed to have been at one point an instigator of Sam's firing, but then also flipped sides and became one of the first people to sign that big letter demanding his return. In a memo to employees, Altman wrote, While Ilya will no longer serve on the board, we hope to continue our working relationship and are discussing how he can continue his work at OpenAI. Now, all of the messaging continues to be that this was not about AI safety. Outgoing board member Helen Toner wrote on Twitter, Our decision was about the board's ability to effectively supervise the company. We were not motivated by a desire to slow down OpenAI's work.

7:23That was repeated in statements from Microsoft. So, I don't know, man, for now, it still remains a mystery. Elon Musk used his Dealbook Summit interview to reiterate that he wanted to hear from Ilya about what Sam had done that was so egregious that he wanted to fire him, clearly intimating that there was something there. For now, we will just have to wait and see what happens, and that will do it for today's AI Breakdown Brief. Up next, the main AI breakdown. Quickly, a brief word from today's sponsor. As a listener of this show, I suspect you like to stay up to date on all things AI and tech, which is why you have to check out the chart-topping podcast Web3 with A16Z Crypto.

7:58Produced by venture firm Andreessen Horowitz, Web3 with A16Z is the perfect companion podcast to the AI breakdown. Web3 with A16Z Crypto is your definitive resource for the future of the internet, whether you're interested in the convergence of AI and crypto or simply curious about what's next. If you need a place to start, they recently released an excellent episode with Stanford cryptography professor Dan Bonet and former Google X engineer Ali Yahya in conversation with host Sonal Choksi about the intersection of AI and crypto. From fighting deepfakes and proving humanity to large language models like ChatGPT, they cover it all.

8:31I highly recommend checking it out, especially if you'd like to learn more about how AI and crypto will impact our everyday lives. Beyond crypto and AI, this show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics. Don't miss out. Follow Web3 with A16Z Crypto on Apple Podcasts, Spotify, or your favorite listening app. And now, a quick word from today's sponsor. I am a huge Notion user. We're talking multiple accounts for multiple projects. I use it for everything from applicant tracking to note taking to project management to sharing public documents to frantically capturing ideas I have while out hiking or just driving around.

9:14Given that, and given the topic of the AI breakdown, I was excited to learn that they've launched a new AI tool called Q &A. It's like a personal assistant that responds in seconds with exactly what you need. Notion AI can give you instant answers to your questions using information from across your wiki, projects, docs, and meeting notes. For someone like me who makes dozens of notes per day around a huge array of topics, having a built-in AI tool to help recall that is incredibly useful. Now beyond that use case, think about this. Have an urgent question you'd normally turn to a co-worker to answer?

9:44Just ask Q &A instead. It'll search through thousands of documents in seconds and answer your question and clear language no matter how large or complex your workspace is. Plus, you can trust your data is secure because Notion AI is designed to protect your information. No AI models are trained with your information, the data is encrypted, and answers will never use information from pages you don't have access to. With Notion AI, it's even easier to do your most meaningful work. Try Notion AI for free when you go to notion.com slash AI breakdown. That's all lowercase letters notion.com slash AI breakdown to try the powerful, easy to use Notion AI today.

10:19And when you use our link, you're supporting the show. One more time, that's notion.com slash AI breakdown. Welcome back to the AI breakdown. Today we are doing some big science stuff, and I will be clear that we are way out of my depth of expertise. I am just going off of what I'm reading, but what it seems like to me is a fairly significant announcement, and I don't think I'm alone in that. So what we're talking about is a new tool from Google DeepMind that is called GNOME that's basically designed to generate new potential recipes for new materials. GNOME came up with 2.2 million potential compounds, of which 380 ,000 seem theoretically stable.

10:58Now to get a sense of why this is significant, you might remember the whole conversation earlier this year about LK99. This was the new material that was, for a time this year, claimed to be able to act as a room temperature semiconductor. Now think about how much chatter there was around just one new material that might have really powerful properties, and now go back and apply it to hundreds of thousands. Now of course the challenge is, these could be useful for building new types of batteries, for that semiconductor use case. That is of course if they can actually be made. But let's take a step back and actually do an overview that comes from Google DeepMind themselves.

11:33They tweet, Introducing GNOME, an AI tool that helped discover 2.2 million new crystals. Crystals are found in everything from the chips powering our phones to solar cells creating clean energy. The model also better predicts the stability of new materials. GNOME, which stands for Graph Network for Materials Exploration, was trained using active learning, a technique to scale up a model first trained on a small, specialized dataset. Developers can then introduce new targets, allowing machine learning to label new data with human assistance. This makes GNOME well-suited to the science of discovering materials, which requires searching for patterns not found in the original dataset.

12:07As a result, the model has boosted the precision rate for predicting material stability from 50 % to 80%. Over the last decade, scientists have only managed to unearth 28 ,000 stable materials using expensive and time-consuming computational methods. Based on this record, our research is equivalent to nearly 800 years of knowledge. 380 ,000 of our stable predictions are now publicly available to help researchers make further breakthroughs in materials discovery. Teams at Berkeley Lab and others have already independently created 736 of GNOME's new candidates in the lab. To build a more sustainable future, we need to develop greener technologies.

12:41We hope GNOME's promising recipes for new materials could help improve products like batteries for electric cars, or even superconductors for more efficient computing. So let's talk a little bit about how this type of process used to work, before the advent of AI and the approach taken by Google DeepMind. They write, In the past, scientists searched for novel crystal structures by tweaking known crystals or experimenting with new combinations of elements, an expensive trial-and-error process that could take months to deliver even limited results. Over the last decade, computational approaches led by the Materials Project and other groups have helped discover 28 ,000 new materials.

13:14But up until now, new AI-guided approaches hit a fundamental limit in their ability to accurately predict materials that could be experimentally viable. You can see that human experimentation led to the discovery of about 20 ,000 of these, additional computational methods increased that to 48 ,000, and now GNOME has increased the number of potential stable compounds to 421 ,000 overall. So how does GNOME actually work? In a section of their blog post called Harnessing Graph Networks for Materials Exploration, they explain. Again, DeepMind writes, GNOME is a state-of-the-art graph neural network model.

13:45The input data for GNNs takes the form of a graph that can be likened to connections between atoms, which makes GNNs particularly suited to discovering new crystalline materials. GNOME was originally trained with data on crystal structures and their stability, openly available through the Materials Project. We use GNOME to to generate novel crystal candidates, and also to predict their stability. To assess our model's predictive power during progressive training cycles, we repeatedly checked its performance using established computational techniques, known as density functional theory, DFT, used in physics, chemistry, and material science to understand structures of atoms, which is important to assess the stability of crystals.

14:19We used a training process called active learning that dramatically boosted GNOME's performance. GNOME would generate predictions for the structures of novel stable crystals, which were then tested using DFT. The resulting high-quality training data was then fed back into our model training. Our research boosted the discovery rate of material stability prediction from around 50 % to 80%, based on an external benchmark set by previous state-of-the-art methods. They also have a graph that visualizes this. They write, GNOME uses two pipelines to discover low-energy stable materials. The structural pipeline creates candidates with structures similar to known crystals, while the compositional pipeline follows a more randomized approach based on chemical formulas.

14:55The outputs of both pipelines are evaluated using established density functional theory calculations, and those results are added to the GNOME database informing the next round of active learning. So this is a process that is both starting from random as well as starting from known structures, and then when promising results appear, it goes back to feeding both of those pipelines to further refine the model. Now of course, if these are just recipes, the question is, can they actually be created? Again from their blog post, The GNOME project aims to drive down the cost of discovering new materials.

15:21External researchers have independently created 736 of NOAM's new materials in the lab, demonstrating that our model's predictions of stable crystals accurately reflect reality. In fact, a team at the Lawrence Berkeley National Laboratory also published a second related paper in Nature, called An Autonomous Laboratory for the Accelerated Synthesis of Novel Materials, that is meant to, as they put it in the abstract, close the gap between the rates of computational screening and experimental realization of novel materials. To do this, they say, we introduce the A-Lab, an autonomous laboratory for the solid-state synthesis of inorganic powders.

15:53This platform uses computations, historical data from the literature, machine learning, and active learning to plan and interpret the outcomes of experiments performed using robotics. Over 17 days of continuous operation, the A-Lab realized 41 novel compounds from a set of 58 targets, including a variety of oxides and phosphates that were identified using large-scale abinito-phase stability data from the materials project in Google DeepMind. Synthesis recipes were proposed by natural language models trained on the literature and optimized using an active learning approach grounded in thermodynamics.

16:21Analysis of the failed syntheses provides direct and actionable suggestions to improve current techniques for material screening and synthesis design. The high success rate demonstrates the effectiveness of artificial intelligence-driven platforms for autonomous materials discovery and motivates further integration of computations, historical knowledge, and robotics. So going back to use cases, one of the things that I think is getting people excited about this is how many more shots on goal for potential use cases this approach to discovering new materials really could be. Now, when they say that their discovery of 2.2 million materials would be equivalent to about 800 years' worth of knowledge, one of the examples they give, quote, 52 ,000 new layered compounds similar to graphene that have the potential to revolutionize electronics with the development of superconductors.

17:03Previously, around 1 ,000 such materials had been identified. Additionally, quote, we also found 528 potential lithium-ion conductors, 25 times more than a previous study, which could be used to improve the performance of rechargeable batteries. So, like I said, this is heady advanced stuff. It is almost inevitable that there are going to be major questions around how readily creatable these things are, although the early results on that also seem promising. What's more, the identification of potentially stable materials does not guarantee that they will improve performance on any of these particular use cases.

17:33But still, seeing such a sea change in how discovery happens is, I think, a great example of why some people identify scientific discovery as potentially one of the most revolutionary aspects of the AI field. Now, whether this process can discover an LLM that actually outperforms GPT-4 remains to be seen. In all seriousness, though, very cool stuff from Google DeepMind, something I will certainly be watching closely. But for now, I appreciate you guys listening or watching as always. Until next time, peace.