147 | OpenAI’s 12 Days of Releases, Google’s Game-Changing Video Generation Leap, and Amazon’s New Models (Nova) and more AI news for the week ending on Dec 7, 2024

0:00Hello and welcome to another weekend news episode of the Leveraging AI podcast, the podcast that shares practical, ethical ways to leverage AI to improve efficiency, grow your business, and advance your career. This is Isar Maitis, your host, and we have a jam-packed week. It seems that there has been more new feature and model releases this past week than any week in history. So while we usually dive into several large topics and then rapid fire, this is going to feel more like rapid fire beginning to end, but the entire first segment of the show is going to talk about new models and new features that have been released or just about to be released by more or less anybody who has anything to do with AI on the planet.

0:46So we have a lot to talk about. So let's get this started.

0:56If you've been listening to the show for a while, you know, there have been huge anticipation to see what OpenAI is going to do towards the end of the year, or more specifically towards the two-year anniversary or birthday of ChatGPT. Well, nothing actually happened on the ChatGPT day itself, but what they have done is they just announced 12 days of OpenAI, which basically are 12 days in which every single day they're going to release a new capability. Some are big and some are small, or how they labeled it, big ones and some stocking stuffers. So the first one that we got is the full O1 model.

1:31So we had access to O1 Preview and O1 Mini. Both were groundbreaking new kind of family of models that can think and analyze things in a much deeper way than before. This took the world by craze and now basically everybody's chasing the same thing. And we saw multiple releases from Chinese companies, which you're going to talk about as well in the show that are trying to do the same thing. Well, now we finally got the full O1 model, and that was the first release in the first days of the 12 days of OpenAI. Just as a quick reminder, we talked about this when this was announced, but O1 is achieving incredible solutions in problem solving, and it scored an 83 % success rate on the International Mathematics Olympiad Qualifying Exam, which is a very big spread than GPT-4-0, who scored 13%.

2:20So that's a huge spike. Also big, huge decline in error, a significant improvement in error reduction and a very high success on anything that has to do that requires deeper reasoning as well as STEM capabilities. Now, in addition, they announced a new ChatGPT Pro subscription that is going to be$200 per month. So that's a pretty big spike from the$20 a month most of us are paying of the$25 or$30 for the Teams versions, depending exactly which plan you pick. And then there's obviously the enterprise versions that are priced very differently. So right now there is going to be one more tier. There's the free version.

3:00There's the plus version, which most of us use if you're just a regular user, which is$20 a month. and then it's going to be$200 a month for the pro version. And I'm reading for their website. It's going to have everything in plus and unlimited access to GPT-40 and O1 and unlimited access to advanced voice and access to O1 Pro mode, which uses more compute for the best answers to the hardest questions, which basically tells us that there is going to be limited access to O1 if you're not on that mode and standard and advanced voice mode accessible, but with some limits if you're just on the 20 bucks a month plan.

3:41Basically what this is, it's targeting people who are heavy users who are going to use this all the time and needs more bandwidth. And for them, it will probably make sense to pay$200 a month. It will be very interesting to see if that actually works because that's a very, very big spike. That being said, people who actually use this at that capacity, probably see the value in it and probably will pay the$200. This will open the door for many other providers to do similar significantly higherly, to follow a similar approach with significantly higher monthly rates to use various advanced services.

4:14On day two, which was on Friday, December 6th, they released the capability to allow people to do their own reinforcement fine-tuning. So you can now apply, and now I'm reading from their website again, apply to the reinforcement fine-tuning research program. We're expending alpha access to reinforcement fine-tuning and inviting researchers, universities, and enterprises with complex tasks to apply. Spots are limited. So that was the announcement of day two. On the day this is released, it's going to be day three and so on. So if you're listening, They're releasing one of those at 10 a.m. Pacific time every single day, and you can chime in and see exactly what the release is.

4:57But there's going to be a lot of exciting stuff. What might be some of that exciting stuff? I really hope and many other people hope that Sora will be released as part of those capabilities, which is the really incredible, presumably, video generation model that they have announced and showed off in February of this year and haven't released to the public yet. and now a lot of the other companies are catching up to it. And so that's a highly anticipated one and they may or may not release that and maybe glimpse or segments or previews of GPT-5 or whatever they're going to end up calling it. We may get that.

5:32Maybe more advanced voice capabilities and stuff like that. So there's a lot that they may release, but we don't really know. And like I said, some of them announced that are going to be big, some are going to be small, but definitely pay attention because every single day in the next few days, there's going to be another announcement from OpenAI. Now, staying on the topic of interesting releases, a Chinese AI firm called O1.ai has achieved a really interesting release. So they released an AI model that is at the level or close to the level of GPT-4. The company is called Yi Lightning, and their model is currently ranked number six on the LMSYS chatbot arena.

6:11and they have trained the model with an investment of$3 million. To put things in perspective, ChatGPT 4.0 was trained at an$80 to$100 million investment. They've done this with only 2 ,000 GPUs, despite the US restrictions. That's what they were able to get their hands on. And they were able to achieve that by just solving for problems and being more innovative than what the big companies that just have the money and the brute force to put into this. And they solved things like reduced computational bottlenecks and multilayer caching implementation, specialized in inference engine that they developed specifically for their needs and optimized memory usage.

6:56So when we hear people like Ilya Saskover saying that it's not about just old scaling laws, but we need to think out of the box, this is a very good example in my eyes. So this model is now available. It's open source. You can use it. And it's significantly cheaper on inference as well than the big models, while it's generating results that, as I said, place it six out of hundreds of models in the world right now. Another big announcement came from Google this week. So they're rolling out a major update to Gemini on Pixel phones and Pixel watches. And what it allows you to do is that allows you to use multiple extensions to connect and control and work with multiple apps on your phone.

7:37The biggest ones that they just released are Spotify, Messaging, Calling, and Smart Home. The biggest gripe that people had with the new Gemini that they're trying to replace just a Google Assistant is that it didn't control most of the important apps on your phone. And now this feature has been released that allows these extensions to talk to the other apps. As I mentioned, a few has already been released, and I'm sure we're going to see more and more of them, which will most likely make Gemini the new tool for most people use Android phones like myself to engage with your phone and with different applications.

8:10But that's not the biggest announcement from Google. Google has launched Vio, which is its AI video generation model that was in private release for a very long time and now is available for preview via Vertex AI platform, which anybody can have access into if you're just creating an account and logging into that. And it is incredible. So it generates 1080p resolution videos from either text or images. It is very good at following content in various visual cinematic styles. So you can pick a specific style and it sticks with that style through the entire video generation. It can produce videos beyond a minute per them.

8:51I haven't tested it yet. And it includes built-in safeguards against harmful content so people cannot generate violence and otherwise problematic content. The other additional interesting thing is that it also includes DeepMind's Synth ID watermarking technology, meaning it's nothing you can see in the video, but it allows Google to detect that these videos were generated. Synth ID was released as open source. Technically, now anybody will be able to find out which videos are created with Vio. This should be a very serious catalyst for OpenAI to release Sora because this is now Sora domain capabilities.

9:281080p, one minute long is not something we have from any other supplier. It's not surprising to me that Google are able to make that jump because they can train on every piece of video from YouTube, which is the largest video repository on the planet. And they obviously have the compute and the human resources to develop this kind of tool. But I'm personally very curious to start playing with this and see what it can produce. Now, in addition to that, they are planning to also release a new version of Imagine 3, their text to image generator to all Google Cloud customers. And the combination obviously will allow you to create very specific images, high resolution and detail to present exactly what you want, and then use that image as a feed into the new VO engine to create videos.

10:14Again, very exciting to any creator who wants an additional tool at their fingertips. I want to share with you some exciting news from Multiply, my company. We just opened the registration for our January AI Business Transformation course. The current course was sold out. The current course that started in November and is ending this week was sold out. We're not going to launch one in December because of the holidays, but there's another cohort that's opening on January 20th of 2025. This course has transformed hundreds of companies and business leaders that have taken the course since its inception in April of 2023.

10:54So if you're looking for ways to start 2025 with the right foot forward, whether for your personal career or your knowledge, or for the sake of the success of your company, business, team, organization, et cetera, come join us. There will be a link in the show notes so you can open your phone right now or your computer and go and see exactly all the details over there. And now back to the episode. Now, staying on the same topic, but from a different company, Runway, which is one of the leading providers of video capabilities, just launched Frames. So Frames is an image generator built into Runway.

11:29They had one before, but this is a completely new model that they've developed from the ground up, and it prioritizes stylistic control and visual fidelity. What it basically means, it means that you can generate multiple images with consistent artistic styles which can be used as the starting point to create videos, which is really important when you're trying to create a consistent video that is generated from multiple images. So right now, the videos that Runway can generate are significantly shorter and they are a few seconds each. So in order to create a longer video, you need to start with multiple images and you need these images to look consistent.

12:07Otherwise, your video will not look consistent. And this is exactly the capability they have just provided. Now, it's already rolling out gradually to anybody who has access to Gen 3 Alpha, basically any paying user of Runway, and it will be available through the API as well. Staying on the topic of video generation, Kling has released Motion Brush. So it allows you to select up to six elements in a single image, paint them with a little brush and point an arrow where you want their motion to go. They also provided camera movements with six types of cameras with camera motion, horizontal pan, vertical pan, zoom, tilt, and roll, which means you can now control both the motion of the camera as well as motion of elements within the scene.

12:56And they released two modes, standard mode with 720p video iteration that is faster and more cost effective and professional mode that allows you to release 1080p HD capabilities. capabilities. Now, I've got to go back to a prediction I made in Q1 this year, and you can go back and check those episodes. But I was saying that by the end of 2025, we will, by the end of 2024, we will be able to create videos that will be indistinguishable between professionally generated videos of real life or cartoons or any other style we would want. And by 2025, we will have full control over the cameras, the scenes, the view, the story, and everything else in the video.

13:34And the first half of the prediction is already correct. Now, these tools are not perfect yet, and they still have some morphing, and there's still some issues in consistency, but it's getting better and better every day. And now with the release of Vio and potentially Sora as well before the end of this year, we will have even more advanced capabilities with more consistency. And I think the consistency issues are going to be mostly resolved in the near future. And then the full control over camera and scene and action will happen in 2025. The other thing that I think will happen in 2025 is AI editing, meaning you'll be able to go to an existing video, whether a real one that was shot with a camera or an AI generated one, and go back and request edits.

14:15Like take this section out, change the camera pan from this to this, and do whatever you want in order to actually edit the video that you already have created to have a lot more control. And if I have to bet, we will have those capabilities before next year is over, which will allow and we'll probably start seeing full videos created completely with AI. Right now, people are kind of like playing with it, creating short commercials and 30 seconds videos. But I assume and I bet that by the end of next year, there's going to be a full featured film created with AI or at least an episode of like 30 minutes of something that would be like a series.

14:52What I just said obviously have profound implications on TV and Hollywood, and it will be very interesting to see where that goes. Another company that made some big announcements in the past few days is Microsoft. Microsoft launched Magentic One, which is an open source multi-agent system that is featuring five specialized agents working in concerts. So the idea is to create a multi-layer approach where you have an orchestrator, which is the lead agent that's coordinating all the other operations, a web surfer that handles browsing to navigate the web and get information from the web, file surfer that manages document and file operations, coder, which writes and analyzes code solutions, and computer terminal, which executes the codes and provides system operations.

15:38So with all of these working together, you can create magical, really advanced, complex processes that are auto run and executed through the orchestrator agent that is going to run everything. Now, they're not the first ones to release something like this. AWS has already released something like this. IBM released B agent. OpenAI has Swarm. So this multi-layer agent control system is something that we're going to see more and more of, and it will allow real advanced development of sophisticated processes that will become easier and easier as these systems will learn our needs. And with simple prompts and instructions, we'll be able to complete very complex tasks.

16:19Now, Microsoft also is launching its screen reading AI that they're calling Copilot Vision. It's something that they demoed before and it can analyze text and images on web pages in real time. And then you can ask it to provide summaries and translations and it can help in product discovery in online catalogs and it offers gaming assistant if you're in the middle of playing a game while you're on a browser, et cetera, et cetera. Basically, it's going to be AI layer integrated into the Edge browser that can see everything in the browser and provide information and assistant to everything that's on that web page.

16:55Right now, it's going to be released in the US only, and it requires you to be a paid member of Copilot with a$20 pro subscription. Now, the good news is that it only works with pre-approved popular websites, and it cannot do the same thing with like your bank account or stuff like that. So there's some guardrails that has been put in place out of the box for us to use it. The other safety guard is that it deletes all the data that it captured at the end of each session, and there is no storage or model training using the process content that was generated in each of those sessions. So in theory, you can use it safely to get assistance on every web page you're on if you are using the Edge browser.

17:35Google has already shared that they're releasing something like this in the very near future, probably in the beginning of Q1 of next year for Chrome. This will become a standard thing, regardless of which browser you're using, to be able to ask AI about what's on the screen right now. The next big and interesting feature comes from Anthropic. They just added Google Docs support, so you can connect multiple documents in a single chat. But the coolest feature in this is that there's an automatic sync, meaning if you are changing the version of the document, Claude will already know that. That, to me, was one of the biggest benefits of Gemini.

18:10So Gemini allows you to basically look into a folder and use that as the data set for a Gemini conversation without having to actually connect the files or upload them into the chat. this is still the case, meaning it's still a unique feature of Gemini that allows you to look into a folder, but this new functionality by Claude closes the gap somewhat that allows you to at least see updates when there's an update to a document and re-ask questions and get updates without having to re-upload the file into Claude. The disadvantage of this tool is that it cannot process images or comments in Google Docs, so it doesn't know how to process that.

18:48I would say it doesn't know how to process that yet. Another big news that came from Anthropic this week is a game-changing MCP protocol. And MCP stands for Model Context Protocol, which is a technical standard that enables seamless connection between AI assistants and data sources and tools. So if you think about to create successful working agents, which is the future of everything AI, you need three things. You need the model itself, the thinking entity, you need access to data, so whatever the sources the data are, and you need tools, you need access to the internet, a browser, an application, and so on to actually execute the things you need to execute in order to complete the task.

19:30And this MCP protocol that Anthropic just open source is supposed to be an infrastructure that will enable all of that. And because it's now open source, anybody can use it and more and more people can contribute. And I actually see that as a very solid step in the right direction of sharing this kind of information across different companies and organizations. What it allows you to do is it allows you to connect AI applications with databases and knowledge bases and business intelligence graph sources and then link chatbots to development tools and development environments and so on. So it basically allows you to connect the three main components seamlessly through this interface.

20:09Now, they already have several connectors built into it, like Google Drive and Brave Search and Slack, and they're planning to connect this to any other known organization data sources. The idea is obviously to break the silos of information. So far, to do stuff like that, you needed to deploy data lakes and data lake houses, which is in many cases a very complex and expensive process. And this may or may not replace DataLakes, but it definitely provides a smaller footprint, significantly cheaper and less complicated solution for at least some of the company needs that can be deployed very fast using now an open source architecture.

20:48Now, we both know that Anthropic just recently got another$4 billion investment from Amazon and AWS for a total of$8 billion so far. But Amazon are not just counting on Anthropic. They're actually developing their own models. And they just released a whole family of new models called Nova. And that series of models is going to be available on AWS Bedrock, which is their infrastructure for everything. And now they currently release three models. Nova Micro, which is fast, cost-effective text model. Then there's Nova Lite, which is a low-cost multimodal model for image, video, and text. And then there's Nova Pro, which is advanced multimodal model.

21:26And in 2025, they're planning to release Nova Premiere, which is advanced reasoning model, basically like GPT-01, Nova Canvas, which is image generation tool, and Nova Real, which will be video generation model with watermarking. And they're planning speech-to-speech and native multimodal models in 2025 as well. So as if we didn't have enough models in the mix, now we have a whole set of new models available directly from Amazon that are available on their Bedrock platform. I assume they will make it highly competitive rates for anybody on their platform to give some benefit to people to use their models over other models on Bedrock.

22:06Now, staying on Amazon and AWS, they released three big tools in the AWS environment. One is automated reasoning check. So it's a new tool that allows you to check the data to reduce or maybe eliminate hallucinations. It can cross-reference both public data as well as customer supplied information that resides in AWS. And it's similar to such offerings that already exist from Microsoft and Google. They also added model distillation, which basically allows to take information that was put into one large model and move it into smaller models. The idea is obviously to get higher efficiency and lower cost and higher speed on tasks that do not require the large language model.

22:52there's a lot of discussions about this and you heard me talk about this. That's the future where everything is going, right? And we talked about this before with delegation and multi-agent collaboration, which is the third thing that they have released. So they released a multi-agent collaboration tool that enables AI tasks to be distributed with supervisor agents and allows parallel processing of complex tasks by multiple smaller agents. So this idea of multi-layered, multi-tiered approach is something that's going to become the norm across all the different platforms and will allow the main tool to understand what the task and what the needs are and then orchestrate it across multiple smaller agents that will work in parallel to complete the task faster, but also with more specifically oriented models that will be tailored for specific tasks.

23:37Another company that we talked about last week, which is Cohere. Cohere is focusing on enterprise solutions. They released Rerank 3.5, which sets new benchmarks on enterprise search capabilities, its accuracy and speed. So it sees 23.4 % improvement over existing hybrid search systems. It's doing 30.8 % better than traditional search algorithms on financial data sets, etc. And it supports 100 languages on the data sets, including Arabic, Japanese, and Korean, which were considered to be more complex to query through. So significant benchmark achieved by Cohere in this particular solution, Cohere has selected to not compete in the crazy race for the most advanced models and instead are developing tools that are geared and tailored for enterprise benefits.

24:27And this new release is just going to push them even to be more competitive in that particular field. Another company that has made a release, and I told you lots of releases this past few days, Eleven Labs, which is a company that has been around for a while doing really advanced voice models, has made two releases. One of them is a customizable conversational AI agent builder. So what they released is a complete conversational agent building platform. It has multiple LLM options behind the scenes. So you can use Gemini, GPT, and Claude. And you can customize the voice parameters and response characteristics.

25:03And you connect it to multiple databases that they already have integrations with. So it comes with a full SDK for Python and JavaScript and React and Swift. So whichever platform you're using, you can connect it to that. And it has the capability to have multiple customizable variables for tone and response length. So you can adjust parameters such as the language selection, the response temperature, the token usage limit. So you can limit the length of each answer, the latency and stability of the model. So you can control how fast it's going to respond and take that into consideration when creating the answers, conversation length, et cetera, et cetera.

25:41So multiple controllable parameters in there. And they're obviously coming to compete with OpenAI real-time conversation API that has been released in the past few months, basically jumping into their field. So they've been the leader in the voice field for a while. And now both OpenAI and Google has stepped into their field. So this is just them upping their game and providing a complete SDK and controllable capability for agent development. From an availability perspective, this is awesome for anyone who wants to develop any kind of voice agents, whether for internal or external usage, whether for customer service, employee support, whatever you want, these tools will become more and more capable.