The Biggest AI Announcements from Microsoft Ignite

0:01Today on the AI Breakdown, we're looking at everything that Microsoft just announced, including their first AI chips. Before that on the brief, new AI bipartisan legislation hits the Senate. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.net for more information about our YouTube channel, our Discord, and our newsletter.

0:24Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. We kick off today with the latest bipartisan legislation around artificial intelligence to hit the U.S. Congress. Republican Senator John Thune and Democratic Senator Amy Klobuchar have introduced a new AI bill that's largely focused on being a sort of middle-of-the-road, not exactly light-touch, but also not exactly onerous type of regulatory approach. Now, one of the big concepts from this bill is to identify certain types of generative AI as being considered, quote, critical impact. Companies and models that are this designation of critical impact would have different sets of requirements.

1:03There would be a self-certification component, but the proposed legislation also tasks the Commerce Department with coming up with a five-year plan for testing and certifying critical impact AI as well. Now, in addition to the Commerce Department being called on to develop new standards, the National Institute of Standards and Technology, or NIST, would also be involved in developing recommendations for other agencies around how to put in place guardrails around these high-impact AI systems. Now, of course, given all this new terminology being introduced. The bill also creates provisions for how to define those new terms.

1:34And basically the way the leaders who introduced this bill are positioning it is as a common sense approach. Thune, for example, waxed poetic about how revolutionary AI can be and its potential to improve healthcare, agriculture, and as he put it, countless other industries. In a statement, he said, as this technology continues to evolve, we should identify some basic rules of the road that protect consumers, foster an environment in which innovators and entrepreneurs can thrive and limit government intervention. This legislation would bolster the United States' leadership and innovation in AI while also establishing common-sense safety and security guardrails for the highest-risk AI applications.

2:07Klobuchar echoed those sentiments, saying, It will put in place common-sense safeguards for the highest-risk applications of AI, like in our critical infrastructure, and improve transparency for policymakers and consumers. Now, this is far from the only legislation out there. Back in September, Senators Josh Hawley and Richard Blumenthal unveiled another bipartisan AI framework that would be a bit more stringent than this one seems and would require AI companies to actually apply for licenses to be able to release their models. Now, on top of all of that, of course, Senate Majority Leader Chuck Schumer has been convening what he calls his AI Insight forums that are closed-door meetings between industry stakeholders, experts from outside the technology industry, and senators and congresspeople who are looking to learn.

2:46The anticipation is, of course, that that will also lead to comprehensive legislation. So far, however, it is worth noting that we are firmly in the stage where people are using the introduction of bills to start to lay a foundation for what they believe and different ways to look at the issues. Right now, none of these have any actual momentum, and thus are unlikely to be passed in anything resembling their current form. Now, staying on the theme of AI in politics, but moving specifically to geopolitics, yesterday, President Biden of the US and President Xi of China got together to discuss a variety of issues, and it was anticipated that artificial intelligence was going to be one of the big ones.

3:21Now, one thing that did not come from this meeting was any sort of shared commitments or pledge or actual wording or anything like that. However, AI was ultimately one of three areas of agreement that were touted by the parties after the meeting. The points of agreement and cooperation were around restarting military-to-military communications, cooperating around counter-narcotics, specifically against fentanyl. And the third was around artificial intelligence. Said Biden at a press conference after the summit, we're going to get our experts together to discuss risk and safety issues associated with AI.

3:51As many of you know who travel with me around the world almost everywhere I go, every major leader wants to talk about the impact of artificial intelligence. These are tangible steps in the right direction to determine what's useful and what's not useful, but dangerous and what's acceptable. Now, we didn't get any more details than that on how Xi and the Chinese government are thinking about AI and where they're willing to come together on specific rules. However, reports suggest that they are receptive, particularly when it comes to military applications of AI, and even more specifically, around the potential of renouncing using AI as part of the control of nuclear weapons systems.

4:23Now, experts in this field do think that given how much tension there is between these two governments, we're unlikely to see agreements around anything but very low-hanging fruit, and saying no to AI-controlled nuclear weapons could exactly fit that bill. Now, moving to our next story, one of the most contentious areas of AI politics is around questions of copyright. Are the creators of AI models within the rights of fair use to go out and scrape other people's data and creations? Or is that exploitative or even illegal? Well, one executive from Stability AI, the company's former head of audio, has taken a stand and quit the company saying that he believes that that company's approach to generative AI training is exploitative of creators.

5:01Ed Newton Rex tweeted,

5:11Now, he does have nice things to say about the company. Ed acknowledges that these are complex issues, and that the people, even those he disagrees with, are deeply thoughtful about these issues. He said, I'm thankful for my time at Stability, and in many ways I think they take a more nuanced view on this topic than some of their competitors. But he continues, Despite this, I wasn't able to change the prevailing opinion on fair use at the company. This was made clear when the U.S. Copyright Office recently invited public comments on generative AI and copyright, and Stability was one of the many AI companies to respond.

5:40Stability's 23-page submission included this on its opening page. Quote, We believe that AI development is an acceptable, transformative, and socially beneficial use of existing content that is protected by fair use. Now, why does Ed not think that this is fair use? He writes, One of the factors affecting whether the act of copying is fair use, according to Congress, is the effect of the use upon the potential market for or value of the copyrighted work. Today's generative AI models can clearly be used to create works that compete with the copyrighted works they are trained on. So I don't see how using copyrighted works to train generative AI models of this nature can be considered fair use.

6:13Beyond that, he says, he just thinks the way the big companies are going about this is wrong. Quote, Companies worth billions of dollars are, without permission, training generative AI models on creators' works, which are then being used to create new content that in many cases can compete with the original works. I don't see how this can be acceptable in a society that has set up the economics of the creative arts such that creators rely on copyright. To be clear, he concludes, I'm a supporter of generative AI. It will have many benefits, that's why I've worked on it for 13 years. But I can only support generative AI that doesn't exploit creators by training models which may replace them on their work without permission.

6:45I'm sure I'm not the only person inside these generative AI companies who doesn't think the claims of fair use is fair to creators. I hope others will speak up, either internally or in public, so that companies realize that exploiting creators can't be the long-term solution in generative AI. Now, I think a lot of the coverage around this is going to focus on the fact that it is yet another executive departure from Stability. But to me, this reads a little bit more in this case of a true fundamental disagreement where parties entered a relationship with one another in good faith and ultimately decided that their views on the world were too incongruous to continue working together.

7:15The CEO of Stability, Ahmad, responded and said, was great working with you and this is an important discussion. He also used this as a chance to share that 23-page document that they gave to the Copyright Office on why they believe fair use supports this type of generative AI training. Next up, a couple quick updates from Google. Jack K from the Bard team writes, Starting tomorrow, Bard will be available for teens to use around the world. We've made several updates to maximize its helpfulness and understanding of its capabilities. Basically, Bard is now going to help with math by sharing step-by-step explanations of how to solve math problems.

7:45Bart is getting data visualization at sounds along the lines of ChatGPT's code interpreter, or now what they call advanced data analysis. And Jack writes, before launching to teens, we consulted with child safety and development experts to help shape our content policies and an experience that prioritizes safety. We're announcing this in advance to help equip teens and parents for how to best understand the technology and talk about the best ways to think about using it. Google also announced a new music creation model from DeepMind that they're calling Lyria. They call it their most advanced AI music generation model to date and are releasing what they call two AI experiments designed to open a new playground for creativity.

8:20Those experiments are DreamTrack, what they call an experiment in YouTube shorts designed to deepen connections between artists, creators, and fans through music creation, and MusicAI tools, which are a set of tools they're designing with artists, songwriters, and producers to help bolster their creative process. Now, I think I'm going to do one of this weekend's episodes about Lyria more in depth and other music creation models, so I will leave it there for now, but very cool to see them pushing into this area. Speaking of AI and audio, Adobe has also announced a new project in that area that they're calling Project Soundlift.

8:48This is a hyper-functional tool that's basically like in-painting for audio. Effectively, you can bring audio files into the application and then choose which sounds you want to filter out. For example, if you want to get rid of applause or laughter or alarms or speech or crowds or traffic or typing, etc. Project Soundlift automatically detects those sounds and comes back with separate files that contain the background noise and the track that you want to prioritize, such as someone's voice or a particular musical instrument. On the one hand, this is not a flashy update, but it is a hyper, hyper useful one.

9:18Anyways, guys, that is where we will wrap the AI Breakdown Brief for today. Next up, the main AI breakdown. Hello, friends. Quick note before we get to the main episode today. You may have heard me talk about this in October, but once again, I have opened up a very small number of micro consulting slots for the rest of November. These are basically one-on-one sessions with me that are designed to dig into your big questions around generative AI and how it can impact your career, your professional goals, or if you're an organization, how your company functions. It's designed to be short and super high impact, and these sessions are paid.

9:51I've got about three or four sessions left for November, and if you are interested, email me at nlw at breakdown.network for more information. Excited to help you bring AI to your big goals for 2024, but now let's get back to the show. Welcome back to the AI Breakdown. Today, we are talking about all of the big announcements from Microsoft Ignite. That includes Microsoft rebranding Bing Chat, talking about partnerships with NVIDIA and other AI startups, and of course, maybe the biggest thing, confirmation that the company has indeed been working on custom silicon. Yes, Microsoft is releasing their own chips, and they are definitely focused on artificial intelligence.

10:34Now, we have heard for months about a project that was reportedly codenamed called Athena. This was meant to be a push into the AI chip space, and of course made sense in the context of just how much Microsoft is doing with AI, as well as how much need there is for these customized chips from other customers. So, first up, the biggest announcement from this event was the announcement of chips, and there were actually two. The one that we had been anticipating was called the Maya AI Accelerator. As Gary Explains writes, it's designed and optimized for artificial tasks and generative AI, and has 105 billion transistors and is built on 5 nanometers.

11:07Now, giving some more technical details, Patrick Moorhead writes, Racks are liquid-cooled, 4x per server, ASIC, not a GPU, which was expected, no cluster or model size limits, Ethernet connectivity embedded, and meant to power Microsoft Copilot or Azure OpenAI service. Now, they also introduced a second chip called the Cobalt CPU, which was an ARM-based chip for general-purpose computing. Morehead writes, this announcement on the custom silicon was more than I had imagined. Overall, I was very impressed and didn't expect this all at once. Now, Morehead also said, you have to assume today's and next generation's OpenAI models will be trained and inferenced on Maya.

11:42Sam Altman says this was a co-collaboration to produce more capable and cheaper models. Semi-Analysis writes, Microsoft is currently conducting the largest infrastructure buildout that humanity has ever seen. While that may seem like hyperbole, look at the annual spend of mega projects such as nationwide rail networks, dams, or even space programs such as the Apollo moon landings, and they all pale in comparison to the more than$50 billion annual spend on data centers Microsoft has penned for 2024 and beyond. This infrastructure buildout is aimed squarely at accelerating the path to AGI and bringing the intelligence of generative AI to every facet of life from productivity applications to leisure.

12:18Now, this is a really extensive piece, one that I will link to in the show notes, but there are a couple parts of it that I wanted to call out specifically in the context of this video. One point that Semi-Analysis makes is that, as they put it, Microsoft is currently behind on deploying custom silicon in their data centers relative to Google and Amazon. At the same time, however, Semi-Analysis writes they have a long history of silicon projects. Now, when it comes to how Microsoft's Maya 100 compares to chips from Amazon, Google, and Meta, they say Maya 100 isn't a slouch. Now, to me, the most interesting thing is certainly the fact that we are now operating in a world where Amazon, Google, Meta, and Microsoft are all building their own chips, and not only for their own purposes, but also lining up partners in the big AI labs like OpenAI, of course, and also Anthropic, to be customers of and co-developers on those projects.

13:05However, lest you think that this means that they are all racing away from NVIDIA right away, NVIDIA CEO Jensen Huang actually was on stage with Microsoft's Satya Nadella yesterday talking about their partnership and how they were even expanding it. The Benbytes newsletter summed up what NVIDIA announced as one, NVIDIA Foundation models, a new family of foundation models, Nemotron 38B with chat and QA variants. Two, NVIDIA announced an AI foundry service that gives enterprises an end-to-end solution for creating and optimizing custom generative AI models. Three, Azure Cloud offering new NVIDIA GPU virtual machines, and a couple of other announcements as well.

13:40Basically, the TLDR is that even as these companies are trying to break their over-reliance on NVIDIA, that doesn't mean that they're trying to get out of that relationship entirely. It also shows the extent to which, especially at the upper echelons of this space, everyone is friend, competitor, and frenemy all at once. Now, like I said, this hardware wasn't the only thing that Microsoft announced. There was also a set of conversations about foundation models. Of course, one piece of that is that all the models that were announced at OpenAI's DevDay are now available to enterprise customers through Azure, but there's also a new models as a service offering on Azure where customers can fine-tune with Llama 2, Mistral, or other options.

14:14They also announced PHY2, which is Microsoft's new dedicated LLM, and announced that it would be totally open source. Highlights from the presentation were that it was, quote, much more robust than PHY 1.5, that it was 50 % better at mathematical reasoning, and that it was designed to be ideal for fine tuning. However, in addition to all of the announcements for the enterprise, there were also a lot of announcements that were relevant for end consumers as well. The TLDR is that all consumer AI is coming together under the Microsoft brand of Copilot. So for example, that means no more Bing Chat, nor will there be any more Bing Chat Enterprise.

14:47All of that simply becomes Copilot. Now, when it comes to Bing Chat specifically, in addition to a free version of Copilot still being accessible through Bing and through Windows, it'll also have its own dedicated domain at copilot.microsoft.com, similar to ChatGPT. Now, interestingly, I've seen a lot of reporting pitching this as a way to compete more directly with OpenAI, and certainly I think that's a part of it. Again, this gets into that frenemy territory that we've talked about frequently on this show, where just because they're working with OpenAI and just because OpenAI is working with Microsoft doesn't mean they're not still trying to beat each other when it comes to some of the fundamentals.

15:19However, more than that, this just seems like a strategic evolution, where they're consolidating a set of various tools that are all sort of related to one another under a banner and framework, and more specifically a brand, that they can go out and push and build consumer awareness around. Now personally, I never found the arguments for why they stuck with Bing particularly compelling. Previously, Yusuf Mehdi, the CMO of Consumer at Microsoft, had said when it comes to Bing, It's a neutral vessel, so all the research from the branding team shows that people are basically neutral on Bing, which is generally a good thing.

15:49Medi said that awareness of the Bing brand was worth around$200 million, so quote, We said, do we want to start from scratch or build on that? It has all these positive things, it's four letters, it has one syllable, it's global, and it has equity. So we're going to stick with the Bing brand. I get that on one hand, but it's also a hyper-conservative position to take in what is going to be one of the most significant brand competitions in the world around which generative AI tools consumers use to bridge into the new world that's coming. I think that the other thing that the co-pilot branding shift does from a consumer perception perspective is that it ties together a set of disparate experiences as something that are all inherently linked.

16:21In other words, instead of there being lots and lots of different Microsoft-curated AI experiences across the apps that people already know and use, there is a new mental framework, a shift that people are undergoing that reinforces that this isn't just a set of small changes, but it's something bigger and much more fundamental than that. That's what having a single common brand helps reinforce. Now, some people aren't hot on the name. Brian Romley writes, this is a regrettable misunderstanding of how AI will play out, although he doesn't elaborate on why exactly he doesn't like that co-pilot terminology.

16:49If I had to guess, I would say that it doesn't create a lot of space for the agentic future, in which AI bots aren't just co-piloting, but are actually piloting entirely on their own. But who knows? Now, I think in many ways, the other part of the announcements from this event that are notable to me is just the sheer quantity of them. As I was preparing for this video, I kept seeing more other companies that were talking about their partnership with Microsoft that was just announced, such as this tweet from Langchain. They write, Today we're thrilled to announce our collaboration with Azure. Our joint customers will enjoy deeper product integrations that live up to our commitment to AI with the enterprise assurances that Azure ecosystems provide.

17:23Now zooming out a little bit in terms of where the state of the big tech competition is, one of the things that has been notable to some is how much more willing Microsoft has seemed to actually go out and try to implement AI tools quickly, as opposed to, for example, Google. Professor Ethan Mollick shared a set of charts and said, a lot of the questions about why Google is reluctant to deploy AI systems and Microsoft is not is answered by these two charts. Advertising is everything to Google and it's not clear how to integrate LLMs and ads. Microsoft does a lot of things that LLMs help. The two charts are revenue breakdowns and shows that Google's advertising properties make up 69 % of its revenue, whereas, for example, when it comes to Microsoft's revenue, 31.3 % of it is around Microsoft Azure, 23.7 % is around office products and cloud services.

18:0714 % is around Windows. And only 5.1 % is around search ads. Point being that to the extent that you think that there is a question of how LLMs are going to interact with advertising, there is perhaps less of a question of how all of these enterprise products are going to be able to start integrating generative AI features. Still, I'm not totally sold on the explanation. I think there are some fairly obvious ways that AI is going to impact advertising. And it feels like every other week or so that we get some announcement from one of the Alphabet-owned companies like YouTube around how AI is going into those advertising products.

18:38What's more meta is similarly or even more concentrated in advertising and yet has been an aggressive pusher when it comes to AI. So who knows? What is for sure is that Microsoft is not slowing down at all and is continuing to push to be at the very, very top of the generative AI heap. That's gonna do it for today's AI Breakdown. I appreciate you guys listening or watching as always. And until next time, peace. Thank you.