Amazon Gets In On AI Foundation Model Game with Nova

0:00Today on the AI Daily Brief, Amazon gets in the foundation model game. And before that, on the headlines, Google's VL rolls out to more customers. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes.

0:23Welcome back to the AI Daily Brief Headlines Edition, all the daily AI news you need in around five minutes. One of the big themes we thought that 2024 was going to be all about was video generation, and to some extent it has been. The year kicked off with a preview of Sora, which really reset people's expectations of what could be. And while Sora never came out, at least not in a way that was broadly available, we got great updates from Runway with their Gen 3, Luma Labs, Dream Machine, Pika. And now Google is rolling out their latest video and image generation models to AI Vertex customers in a private preview.

0:55Called Vio and Imogen 3, respectively, the models round out Google's Gen.AI offering into a full suite. Focusing on Vio, the video model, Google says that Quora has already integrated its features into their Poe chatbot platform, while Oreo's owner, Mondelez International, is using it to create marketing content and collaboration with agency partners. Unveiled back in April, the model is capable of outputting six-second videos at 1080p. Users can edit generated videos, including changing camera movements. Speaking to the long wait for API access, Warren Barclay, the Senior Director of Product Management at Google Cloud, said it was all about ensuring, quote, enterprise readiness.

1:29He added, Since Vio was announced, our teams have augmented, hardened, and improved the model for enterprise customers in Vertex AI. As of today, you can create high-definition videos in 720p, in 16.9 landscape, or 9.16 portrait aspect ratios. Similar to how we have improved capabilities of other models, such as Gemini on Vertex AI, we will continue to do this for Vio. TechCrunch writes, VO understands VFX reasonably well from prompts and has somewhat of a grasp on physics, including fluid dynamics. The model also supports masked editing for changes to specific regions of a video and is technically capable of stringing together footage into longer projects.

2:03However, they also say reflecting the limitations of today's AI, objects in VO's videos disappear and reappear without much explanation or consistency. And VO often gets physics wrong. I think in many ways, the state of video generation is all about what you can use it for. You have tons of creators who are pushing the boundaries of short filmmaking, but from a business perspective, this is a technology who's probably ready for prime time when it comes to advertising, social media, but not necessarily yet longer film content creation. Still with this update, it seems pretty clear that 2025 is going to have a lot more video generation than 2024 did.

2:38Next up, we've been following the recent FTC probe of Microsoft, and it appears that their deal with OpenAI is explicitly part of the concern. The information writes, FTC officials have been asking Microsoft's rivals about the impact of its AI deals and range of products. Specifically, the FTC is asking about Microsoft's deal with OpenAI. The FTC has questioned rivals about how Microsoft sells its OpenAI-infused co-pilot software and how it resells OpenAI's models to developers on its Azure cloud computing platform. They continue, the questions imply that the FTC is probing whether Microsoft's dominance in the cloud computing market has given the company an unfair advantage in sales of AI software.

3:13Of course, this is a very fluid situation. We have a new administration coming in, and many feel that it's basically inevitable that although the next FTC chair nominee has not been announced, it is very likely that they will be less antagonistic towards big tech than Lena Kahn has been. Still, according to the information, several of Microsoft's largest rivals, quote, believe they can convince the Trump administration to keep scrutinizing the company. Adding more credence to that is, of course, the close role of advisor Elon Musk in that administration, who has been a fierce critic of Microsoft's deal with OpenAI, going so far as to actually sue Microsoft as part of his lawsuit against OpenAI as well.

3:49Speaking of OpenAI, the company has just hired its first CMO, and that person is yet another refugee from the crypto space. Kate Roush is the outgoing CMO of Coinbase and represents the latest major hire as OpenAI builds out its C-suite. Sarah Fryer is in as CFO. Kevin Wheel is chief business officer. Although as of yet, they have not hired a CTO to replace Mira Mirati. Kate tweets, could not be more excited to help show the world what AI that benefits all of humanity looks like. Congrats to Kate and good scoop OpenAI. For now, though, we've got a slightly longer than normal main episode, so we are going to cut the headlines here.

4:25Appreciate you listening as always. And now it's time for the main episode. Today's episode is brought to you by Rocket Money. We are coming up on the beginning of the new year, and that is a perfect time to get organized, set goals, prioritize what matters most, which for many of us is going to be financial wellness. Thanks to Rocket Money, those goals, especially around money, feel achievable. Rocket Money shows you all of your subscriptions right in one place, helping you easily cancel those that you've maybe forgotten that you're actually paying for. Rocket Money also pulls together all of your spending across your different accounts so that you can clearly track spending habits and see where you can cut back.

5:05Rocket Money is a personal finance app that helps find and cancel unwanted subscriptions, monitors your spending, and helps lower your bills so you can grow your savings. Their dashboard gives you a clear view of your expenses across all of your accounts. You can easily create a personalized budget with custom categories. You can see your monthly spending trends in each category to know exactly where your money is going. Rocket Money will even try to negotiate lower bills for you. They automatically scan your bills to find opportunities to save, and then you can ask them to negotiate. They'll deal with customer service so that you don't have to.

5:35Rocket Money has over 5 million users and has saved a total of 500 million in canceled subscriptions, saving members up to$740 a year when using all of the app's premium features. Cancel your unwanted subscriptions and reach your financial goals faster with Rocket Money. Go to rocketmoney.com slash AI breakdown today. That's rocketmoney.com slash AI breakdown. down. Today's episode is brought to you by Vanta. Whether you're starting or scaling your company's security program, demonstrating top-notch security practices and establishing trust is more important than ever. Vanta automates compliance for ISO 27001, SOC 2, GDPR, and leading AI frameworks like ISO 42001 and NIST AI risk management framework, saving you time and money while helping you build customer trust.

6:19Plus, you can streamline security reviews by automating questionnaires and demonstrating your security posture with a customer-facing trust center, all powered by Vanta AI. Over 8 ,000 global companies like Langchain, Lila AI, and Factory AI use Vanta to demonstrate AI trust and prove security in real time. Learn more at vanta.com slash NLW. That's vanta.com slash NLW. Today's episode is brought to you as always by Super Intelligent. Have you ever wanted an AI daily brief, but totally focused on how AI relates to your company? Is your company struggling with AI adoption, either because you're getting stalled figuring out what use cases will drive value, or because the AI transformation that is happening is siloed at individual teams, departments, and employees and not able to change the company as a whole?

7:05Superintelligent has developed a new custom internal podcast product that inspires your teams by sharing the best AI use cases from inside and outside your company. Think of it as an AI daily Brief, but just for your company's AI use cases. If you'd like to learn more, go to bsuper.ai slash partner and fill out the information request form. I am really excited about this product, so I will personally get right back to you. Again, that's bsuper.ai slash partner. Welcome back to the AI Daily Brief. Amazon has had a long and interesting journey when it comes to their relationship with foundation models in the AI space.

7:41We've had context to discuss them a couple times in the last week and noted that much of their strategy was formed in reaction to ChatGPT coming out and being, frankly, much better than the version of that that they had planned on releasing themselves. In fact, they took the name that they had been planning to use on their ChatGPT equivalent, which was Bedrock, and turned that into an AWS service to help enterprise customers figure out which models to use. Since then, they've doubled down on their relationship with Anthropic, as well as really focused on their infrastructure play with their Tranium chips.

8:12But it appears that they are not content to not be in the Foundation model game, as at AWS Reinvent this year, the biggest announcement of that event was the unveiling of a new family of proprietary models. Called Nova, the range includes four sizes of LLM, Micro, Lite, Pro, and Premiere. Nova Micro is a text-only model described as being optimized for speed and cost. Nova Lite is a low-cost multimodal model capable of quickly analyzing image, video, and text inputs. Nova Pro is described as a highly capable multimodal model with the best combination of accuracy, speed, and cost for a wide range of tasks.

8:44And Nova Premiere is Amazon's most capable multimodal model designed to excel at, quote, complex reasoning tasks and for use as the best teacher for distilling custom models. The lineup also includes an image generation model, Nova Canvas, and a video generation model, Nova Real. Each is claimed to be state-of-the-art in their respective fields. All of the models other than Nova Premiere are now available within the Bedrock Model Library on AWS, with Premiere expected to arrive sometime early next year. Amazon CEO Andy Jassy said, We've continued to work on our own frontier models, and those frontier models have made a tremendous amount of progress over the last four to five months.

9:18And we figured, if we're finding value out of them, you would probably find value out of them. We don't know how many parameters these models are, but based on their descriptions, they seem to line up with similar latest generation model families from the leading labs. Context windows are also comparable to rival models, but Amazon promised to deliver an ultra-long 2 million token context window for some models next year. The LLMs all support fine-tuning using text images and videos as well as model distillation. Canvas, the image model, looks on par with leading models from rival labs. While real, the video model feels a bit like a teaser version, only supporting six-second videos which take about three minutes to generate.

9:53Amazon says that a version that can generate two-minute videos is coming soon. Quality appears up to standard in the demo video provided by Amazon. However, the user-generated videos that have come out so far range dramatically from good enough to extremely janky. Coming next in the Nova lineup is a speech-to-speech model expected in Q1 of next year and an any-to-any model expected in mid-2025. The benchmarks appear at first glance to be competitive. Amazon is claiming that Real outperformed Runway's Gen 3 Alpha in A-B testing, achieving a 61 % win rate for video quality and a 71 % win rate for video consistency.

10:27For the language models, Nova Pro seems to be at least competitive with Quad 3.5 Sonnet and GBT40, claiming outperformance in some areas. One of the things that I think is important when we talk about benchmarks is that it's probably more valuable to understand these in terms of competitive ranges rather than in terms of exact specifics. And if that's how you take it, Nova Pro is in the class of models that include Claw 3.5 Sonnet, GPT-40, etc. Wale Akin Fadarin, a tech lead at AWS, pointed out an interesting area of outperformance, commenting, one exciting aspect of the newly released Amazon Nova Gen AI models is their impressive performance on agent and multi-agent benchmarks.

11:05Offsetting this point, though, one of the areas that Nova Pro seems to be lacking in is encoding benchmarks. AI entrepreneur Bindu Reddy ran her own testing on LiveBench, finding Nova Pro was further down the leaderboard. She writes, The benefit of having a benchmark that changes every month is that it can't be gamed. On our latest November challenge, Amazon's Nova scores below Llama 70b and is slightly better than Haiku. Net-net, this model doesn't change the leaderboard significantly, though it seems surprisingly fast. Predictably, Sonnet 3.5 and the O1 line remain on top of the list. One of the more important features, though, which was kind of glossed over, frankly, at reInvent, was how competitive Nova is on price.

11:40Nova Micro and Nova Lite are both priced below Gemini 1.5 Flash and GPT-40 Mini, making them the cheapest models available from a major lab. Nova Lite also has the distinction as the cheapest available model with multimodal inputs. Those two models are priced so close together that the differentiation seems to be speed, with Nova Micro about 10 % faster than Gemini 1.5 Flash. Nova Lite runs at a similar pace to GPT-40, which is still faster than the rest of the pack. Nova Pro is available at around a third of the cost of either GPT-40 or Claude 3.5 Sonnet, and also positioned as slightly cheaper than Claude 3.5 Haiku with much higher performance scores.

12:15The net of all of this is that Amazon's in-house LLMs are now the cheapest available for many use cases. Jerry Liu, the CEO of Llama Index, writes, Amazon's Nova should have advertised the cost reduction front and center instead of me having to dig up a hacker news thread. It's a huge value proposition. Nova is an exciting push towards way cheaper models that are comparable to the state-of-the-art in terms of context window, performance, and multimodality. LLMs have come a long way, but a huge issue with using them for repeated loops is cost, especially for multimodal agentic flows. Overall, the biggest takeaway from this announcement, if you're just trying to understand it in a sentence or two, is that Amazon has gone from skipping this generation of LLMs to frankly fully in play with a complete lineup of competitive models.

12:57Warren Professor Ethan Mollick summed it up, and then there were six, or so. Based on the stats, it looks like Amazon's Nova Pro is a competitive frontier model. This rounds out the GPT-4 Gen 1 models. GPT-40, Gemini 1.5, Claude 3.5, Grok 2, Llama 3.2, and maybe three non-US models, Quen, Yi, and Mistral. Another question for me is that given that Jassy is talking about making progress over just the last four to five months, it sort of implies that they've caught up to the other labs since summer. Which, if that's the case, also lends credence to the growing sense that maybe there really aren't particularly strong moats when it comes to models.

13:33The next big group of announcements related to Amazon's Tranium AI chips. Tranium 2 instances are now generally available on AWS for training and inference. The company also announced a new generation of chips, Tranium 3, expected to become available late next year. For the Tranium 2 chips, Amazon is claiming a 4x improvement in speed over the first generation, which frankly saw very little adoption. They boast that Tranium 2 inference can deliver 3x higher token generation throughput on Meta's Llama 405b model compared to offerings available from other cloud providers. In their announcement post, Amazon claims, quote, Tranium 2 offers 30-40 % better price performance than the current generation of GPU-based EC2 instances, presumably referring to NVIDIA's H100s.

14:13AWS CEO Matt Garman claimed that Tranium 3 will be twice as fast as the second generation and deliver 40 % more energy efficiency. Garmin said, Today there's really only one choice on the GPU side and it's just NVIDIA. We think that customers would appreciate having multiple choices. One big surprise regarding Tranium was a special appearance from Apple's Senior Director of Machine Learning and AI, Benoit Dupin. Dupin took the stage to promote Tranium, stating that Apple is currently using the chips to power services like Search and will evaluate the use of the latest generation to pre-train their Apple intelligence models.

14:44It's pretty unheard of for Apple to endorse a supplier, particularly in a segment as competitive as cloud. And so people are taking this as a pretty strong vote of confidence. Amazon also announced the gigantic new AI training supercomputer dubbed Rainier. Designed in collaboration with Anthropic, Amazon says the supercluster will deliver 5x the compute that was used for Anthropic's latest training runs. The design is somewhat unique with the cluster spread out across multiple facilities networked together. Usually training superclusters are housed under one roof as the latency to transport data across the network is a major bottleneck.

15:13Amazon claims to have solved this problem with networking technology that they call the Elastic Fabric Adapter. Amazon said in their press release, when completed, it is expected to be the world's largest AI compute cluster reported to date. The current high-water mark for completed supercomputers is Elon Musk's Colossus facility, which houses 100 ,000 NVIDIA H100s. Gotti Hutt, who works with customers at Amazon's chipmaking unit Annapurna Labs, said that Rainier will contain, quote, significantly more than 100 ,000 Tranium II chips. The cluster is expected to be ready sometime next year. So that is the story.

15:47Lots going on on the Tranium side, but the big news is Nova and Amazon being in the Foundation model game for real. For now, that's going to do it for today's AI Daily Brief. Appreciate you listening or watching as always, and until next time, peace.

16:07Thank you.