141 | Clear signs of AI slowing down, and yet AGI is expected in the 2 two years, OpenAI Agent controls applications on your computer, Claude 3.5 Opus is coming soon, and more important AI news for the week ending on November 15, 2024

0:00Hello, and welcome to a weekend news episode of the Leveraging AI podcast, the podcast that shares practical ethical ways to leverage AI, to improve efficiency, grow your business, grow your business, and advance your career. This is Isar Maitis, your host. And just like in previous week, we're going to focus this episode on one or two big topics. And then we're going to do a rapid fire list of other things you need to know. The two big topics this week is going to be, is AI development slowing down or not? And what that means to the future of AI and to the companies that are developing it. The other is going to be a web controlling app from OpenAI.

0:33So a version of Chachapiti that can control multiple aspects of your computer and run things for you. And lots of other important news from this week. So let's get started.

0:49As I mentioned, the first topic we're going to dive into is, is AI development slowing down? There are multiple resources that are hinting that this is what's happening. And it's been coming from multiple different directions. There's obviously people who think the other way around. So let's try to understand what's going on. So first of all, Anderson Horowitz, both founders of the company, had interviews in the past week claiming that there is a clear slowdown in AI development pace. They're giving several different examples. One of them is that they're saying that the growth from GPT-3 to GPT-4 was significantly bigger than the growth between GPT-4 to Orion, which is supposed to come next.

1:31And they are claiming that in addition to that, while ChatGPT had a huge lead in the beginning, that lead has been diminished and that most companies have caught up. And most companies are roughly at the same level right now, which suggests that that's some kind of a glass ceiling for the current technology. They're stating multiple potential reasons for that that all compound one on top of the other. So the first thing that they're talking about is the shortage of data and specifically high quality data. So they're stating that between April 2023 and April of 2024, many websites has blocked access to data for training for these models.

2:10And they're saying that restricts 5 % of overall data, but more importantly, 25 % of high quality data is now not accessible for these models training. The other thing that they're mentioning is obviously GPU shortage. So all these companies are fighting on GPUs. And the biggest benefit of that is obviously NVIDIA. that has became the largest valued company in the world again this week. And it's probably not going to leave that position for a while. But there is definitely a GPU shortage right now when it comes to training. But even if the GPU shortage gets resolved, Anderson Horowitz's partner are saying that then there's going to be power constraints to power those new GPUs.

2:50And even if that gets solved, there's a cooling capacity issue. So across all different aspects of the ecosystem of training models, there are bottlenecks that are not easy to resolve. Now, if we go back to what Ilya Satskova mentioned when he left OpenAI and founded his company SSI, he mentioned exactly that problem. He said that he thinks that the current scaling tools and processes have plateaued and we need to look for something else. And the exact quote is, 2010s were the age of scaling, now we're back at the age of wonder. Which means he thinks that new breakthroughs and discoveries are going to make the difference versus just pouring more compute and more data into these models.

3:35And this is true for many companies. In an article by The Information on November 13th, they're claiming that Google is encountering serious performance issues in the training of their next Gemini model. They're saying that the traditional scaling laws have now significant diminishing returns. And they're also saying that AI-generated, also-called synthetic data, that has been used more and more to compensate for the fact that we have consumed all the human-generated data or that it's blocked from training. They're saying it's not delivering the expected improvements as the scientists expected initially.

4:15Google's way to address that is that they formed a new team led by Norm Shazir, who we spoke about several times in the past. He's a top scientist that Google has brought back in the cost of billions of dollars. And together with another scientist called Jack Correa, who is going to focus in a new department that will be shifting to manual improvements and parameter optimization, meaning they are not going to just be relying on more compute and more data, but actually looking for better ways to run their algorithms in order to get these results. And the approach that they're going to take is going to be somewhat similar to reasoning models similar to OpenAI's approach with ChatGPT-01.

4:57So both these companies, both Google and OpenAI, are focusing more on the capability to reason and think in real time versus to push the boundaries with more and more training power. Another scientist that has leaned into this topic is Jan Peleg, who is the founder and chairman of Deep Trading. He's a top scientist in the AI world running within the financial AI space. And he said in a tweet this week, and I'm quoting, I think it is safe to assume that all major players have reached the limits of training longer and collective more data already. It is about data quality now, which takes time. Basically, what he's saying is that the rate of change and improvement is going to slow down.

5:43So what does this mean? It means that we are going to see changes in technologies and changes in approaches in order to break this new ceiling that seems to be impacting everybody for all the reasons I mentioned earlier. And the main focus, as I mentioned earlier, is more thinking models, basically moving the emphasis from bigger training to better performance in runtime, on inference time, which is the time that the models actually perform their actions. Noam Brown, who is one of the leading scientists in OpenAI, who has developed the poker tool that has beat humans right and left, and also other tools within games, he's claiming that 20 seconds of thinking time in poker specifically matches the performance boost of 100 ,000 times the scale increase in traditional training.

6:37Basically, allowing these models to think and work through problems in real time provide huge benefits way beyond what you can achieve with a lot more investment in the training side. And I think we're going to see more and more of that across all these different companies. What does that mean? It means that in addition to the change in direction in approach, it's also going to impact the need of compute. So these companies and us, the users, will focus more and more on inference-focused infrastructure and not necessarily on GPUs to train the models. This will shift the need maybe from GPUs right now to inference-focused chips like the one from Grok, as well as NVIDIA themselves is saying that they're seeing a shift from their traditional GPUs to their Blackwell chips, the newest ones that are significantly better at inference.

7:30On the same topic, the information, which has been a very reliable source of information about what's happening within these companies, mentions that internal reports from OpenAI are saying that Orion, which is the model they haven't released yet, that is supposed to be quote unquote GPT-5, has far smaller quality improvements compared to the jump from GPT-3 to GPT-4. So they're strengthening the message that's coming from Anderson Hurwitz. Also, they're claiming that some researchers report that Orion isn't even consistently outperforming GPT-4 itself, which is almost two years old at this point.

8:08Now, on the flip side of that story, there has been two really interesting interviews this week. One with Sam Altman with Y Combinator and the other with Dario Amadei, the CEO of Anthropic with Lex Friedman. So on the interview with Y Combinator, Sam Altman, the CEO of OpenAI, is claiming that OpenAI has a clear path to achieving AGI and that they know what to do in order to get there. And he made it pretty obvious that he thinks it can be achieved in 2025. Now, the flip side, which is very interesting, is that he believes that it is going to have, and I'm quoting, surprisingly little impact on society.

8:49I personally strongly disagree with that assumption, but he obviously thinks that the path to AGI is clear, and that means that maybe the compute limit or the training limit that everybody's talking about is not impacting or not significantly impacting the path that he sees forward. The other interview was Dario Amadei, the CEO of Anthropic, being interviewed on the Lex Friedman show. Now that's a five hour episode. I didn't get a chance to listen to all of it because in this past week and a half, I've done three different company workshops plus speaking at one conference and launching another cohort of the AI business transformation course this coming Monday.

9:35and I'm doing another company training next week. So I've been doing a lot of training myself. And so I did get a chance to listen to the full five hours. By the way, those of you who are interested in joining our AI Business Transformation course, as I mentioned, it is starting this Monday, November 18th. So if you're interested in starting the year of 2025 with significantly more information for yourself and your career, as well as the company and your team, it's your more or less last chance to sign up for this. The course is two-hour sessions every Monday or four weeks in a row, and it has transformed hundreds of businesses with AI implementation through the knowledge that we're providing since the launch of this course on April of 2023.

10:20So if you want to change the trajectory of your career or your business and learn how to really implement AI in a business perspective, it's a perfect opportunity. Come join us this Monday. There's going to be a link in the chat and you can use promo code AI2025 for$100 off so you can be better prepared for AI in the year of 2025. But now to that five-hour interview. In this interview, Dario mentions several different things. Some of them are repeating what we already shared with you a few weeks ago when he published his blog about this topic. But he feels that they are on the path to having what he calls very powerful AI, which everybody else calls AGI, by the year of 2026.

11:06So again, in the very near future, he also expects superintelligence, something between 2026 to 2027. He actually believes, based on what he's seeing in his research and in his team, that the scaling laws are still live and strong. And he thinks we're on the path of training bigger and bigger models that will be better and better just by throwing more and more compute and data into them. And so now I'm quoting, I would guess we are operating in, you know, roughly 1 billion scale plus or minus factor of 3, right? Those are the models that exist now or being trained now. I think next year we're going to go to a few billions.

11:47And then 2026, we may go to, you know, about 10 billion. And probably by 2027, their ambition to build$100 billion clusters. And I think all of that actually will happen. So he is obviously one of the top minds when it comes to AI implementation and scaling. And he believes that the scaling goals are still holding and that path is still a viable path. So who is right and who is wrong in this scenario? I don't really know. Again, much smarter people, more knowledgeable people than me are saying different things. What is obvious to me is that it doesn't really matter. What matters is even if the technology is slowing down, and even if it will take a while to figure out the next breakthrough or method to move forward, the capabilities that exist with AI right now will transform most businesses, all of the aspects of knowledge businesses, and it will happen within the next two to five years, even if we see no improvement moving forward.

12:50And so I think that discussion is a very interesting discussion from a technological perspective. It's a very interesting discussion from the social and economical implications that it's going to have. But from a practical business perspective, it doesn't matter. When I teach my courses, when I do my workshops, which are happening more and more frequently, it is very obvious to me that most business people in the world are very far behind from implementing the current capabilities that are available at their fingertips for either free or almost free. And so the revolution is already here, even if we stop developing for the next two to three years moving forward.

13:28So let's switch gears to the other topic I want to dive into a little more in this episode, which is the fact that OpenAI has launched an operator agent that can run and operate applications on your computer. So a few weeks ago, we have reported that Anthropic has released a similar platform that allows to run on the local computer and control more or less everything, but there are some major differences between these two launches. First of all, OpenAI are planning to release this as a full product, not as a beta, in January of 2025, meaning it's just around the corner. It has already been released as part of their existing application.

14:06So if you are a paid user of ChatGPT and you have the macOS application, you now have the ability to run several different applications on your computer. Right now, it's focused on code development. So it's limited to specific applications that are around that topic. And it is using the OpenAI Swarm framework that we has reported on a few weeks ago. It allows to run applications such as VS Code, Xcode, Terminal, and iTerm2, all that are code related. And in the demo they shared when they released this application, they're showing an amazing example of creating code using multiple environments, while ChatGPT application is connecting and orchestrating the code development process and transferring data between these different applications.

14:52A few big differences between what Anthropic has released and what OpenAI is released. So the first thing, it looks a lot more baked than the Anthropic application, and you don't have to learn how to install it through different ways. It's just there in the application. The second big difference is you can provide or reject access from this tool to specific applications, meaning it doesn't control your computer. It knows how to interact with just specific applications, which presumably, at least in my head, is supposed to make it significantly safer to use. Another big difference is if you look at the way that the Anthropic application is running, it is consistently taking screenshots of your computer to understand what's on the screen and then moving the mouse based on counting pixels to where it needs to do it.

15:40The OpenAI application doesn't seem to do that. It seems to actually just see the screen and be able to interact with the application that's running on it. And it seems to be running significantly more efficiently and faster and smoother than the Anthropic application. Now, it also has some additional functionality that OpenAI added to the application. It has better voice assistance capability. It has screenshot functionality where you can take screenshots and just ask about it and you will understand what to do. It has web search through search GPT now built into the application and document context understanding.

16:16So a lot of improvements to the Mac OS application, but the biggest deal is the ability to control things on your computer. We had a very long conversation about this a few weeks ago when we were discussing the Anthropic tool, but this is the direction everything is going. All the big companies, including Microsoft and Amazon and Salesforce, are all working in that path. And 2025 is going to be the year of these agents that will be able to control and engage with different applications. What is going to make them successful, more or less, and achieve wide success is two things. One is their ability to run consistently and without many mistakes.

16:56And the other is our ability to trust them from a security perspective. Right now, it's very hard to say whether they're in that situation or not, but I have very little doubt that by the end of next year, so a year from now, and most likely sooner than that, we will be in that situation for more than one of these tools, meaning we will be able to let it run free and do specific complex tasks on our behalf. and you will have access to different tools and we'll be able to perform them on our behalf while connecting multiple tools we have access to together just by controlling our computer and having access to the web.

17:31That sounds almost like science fiction, but this is the direction it's going, and it's going there very, very fast. Combining this to the previous point that I talked about, this may not require additional compute and training new models. is just new ways to implement the existing capabilities while providing significantly more value to us, the users, with, again, some risk and other aspects that needs to be resolved. But it will allow a huge jump forward in the efficiency gains that can be gained through multiple aspects of the business without necessarily training GPT-5 or CLAWD-4 or Gemini 2.

18:08And now let's switch to some rapid-fire news. So staying on ChatGPT, Greg Brockman, who has been the president of Chachupiti took a three-month sabbatical a few months ago, and it wasn't clear if he's actually coming back. Well, he came back. So different than other people who left the company, like the CTO, Mira Moratti, and the chief research officer, Rob McGraw, and a few vice presidents. Greg Rappin, when he left, defined it as a three-month sabbatical, and he actually came back. He's a big supporter of Sam Altman, and that's going to give Sam Altman a lot of tailwind in the really difficult process that he has to do of changing the company from a nonprofit to a for-profit company.

18:46Now, something that may slow that process down or potentially prevent it altogether is Elon Musk has just escalated his lawsuit against OpenAI. So a quick recap, Elon Musk sued OpenAI earlier this year, then pulled his lawsuit against OpenAI and then brought it back in July. And now he's expanding that lawsuit, adding Microsoft as a defendant in this battle as well. And so the previous lawsuit was targeting just to switch from OpenAI's initial approach of being a nonprofit to help the world to a for-profit organization. The current expansion of the lawsuit also targets Microsoft saying that they are not allowing the same amount of access to compute to X as they are providing OpenAI.

19:34So in other words, this is anti-competitive practices, which are not allowed by law. If this is true, this is going to obviously impact both Microsoft and OpenAI in this very interesting relationship. So right now, Microsoft controls 49 % of the for-profit arm of OpenAI, and they've done that through investing$13 billion, mostly through compute. And what Elon Musk is saying, that to similar access to compute like OpenAI is not possible right now, even if you're paying the same amount of money. As I mentioned, what's the impact that's going to have on OpenAI's process to becoming a for-profit organization?

20:12I don't know, but it may definitely impact that process and slow it down. And they may not have time for those slowdowns because they have to complete that process in two years in order to keep all the money that they raised just a few weeks ago. And from OpenAI, let's talk quickly about Anthropic, Claude 3.5 Opus is going to be released sometime soon. That was also revealed in that interview with Dario Amadei. He is claiming that this model has taken a long time to train and cost him a lot of money in alignment to make sure that it's safe, but it is coming out in the near future. He also claimed that Claude 3.5 Sonnet is now better than the original Opus 3 model, and even Haiku 3.5 is roughly at the same level as the original Opus 3.

20:56So now Opus 3.5 is coming out and it's supposed to be most likely the most powerful model that exists out there and it's going to compete to the top performing and it's going to be a serious competition to GPT-40, N01 and Gemini 1.5 Pro. Another interesting piece of news from Anthropic is they just hired their first AI welfare scientist. So they hired Kyle Fish, who is going to lead their alignment team, and he's going to evaluate the system consciousness to verify that it's running well and focus on the moral considerations of AI development. I find this fascinating, and it's perfectly aligned with everything we know from Anthropic and their approach to scaling and growing AI.

21:42Something very practical from Anthropic, they just launched a new prompt engineering tool in their developer console. It has several different functionalities, the two most interesting ones. One is automated prompt improvement system, so you can put in your prompt and it will create a much better prompt for you. They're claiming that using these improved prompts will give you 30 % improvement in several different aspects of the output and 100 % word count adherence in summarization tasks. That's huge, obviously, if you're doing a lot of summarizations. I actually think that already Anthropic has the best summarization capabilities, but definitely worth testing this tool out.

22:20The other thing it has is it has a tool that allows you to create examples using synthetic data. So you can tell what you're trying to do and it creates synthetic examples for you that can be used as part of the prompts to get better outcomes. And from Anthropic to Google, I've shared with you several times that I absolutely love Notebook LM, the tool from Google, which allows you to summarize information very, very quickly and very efficiently and even turn it into podcasts. Google just launched a new tool that is similar, but not exactly. And it's called Learn About. And it's another free tool that you can have access to and you can personalize your learning responses based on the information that you're going to give it and based on your knowledge level.

23:03So it has the ability to process questions and images and documents, very similar to Notebook LM, but it's going to focus on helping you learn stuff in your pace based on your current level of knowledge. Now, staying on Google, they've just announced a very controversial new hub that's going to be built in Saudi Arabia. In 2020, Google pledged to stop developing and training algorithms based on oil and gas power resources. They also have a pledge to reduce their emission by 50 % by the year of 2030. That being said, they just announced this partnership with Armaco, which is the state oil company in Saudi Arabia that's going to be financed in a partnership with Saudi Public Investment Fund.

23:47And they are going to focus on helping them develop Saudi-specific algorithms and models. This will obviously increase their carbon footprint because being in Saudi Arabia, A, puts it outside of the US, which means less control of what these models are and who controls them or what can be done with them. But B, definitely is going to increase the carbon footprint of this operation. Connecting this to the election results in the US, I think we're going to see a lot more drilling and oil-based data centers in the US as well. So from that perspective, I don't think going to Saudi is going to make a big difference.

24:21Overall, not good from a global warming perspective. Another small piece of news from Google. they just launched the Gemini AI app for iOS. So they had this application for Android before, but if you're an iOS user, you can now install the Google app on your phone and be able to communicate with Gemini through the app directly. And from Google to Grok, so X is apparently testing a free version of its AI chatbot, Grok. So far, it has only been available to paid users on the X platform. And now they're testing a new version of that platform in New Zealand. It's going to have different limitations on how much usage you can do per hour and per day, but that's being tested right now and may be a sign that they're going to provide free access to more users around the world.

25:11And from the big giant store company we don't talk about a lot, but it's a very interesting company and that is Rider. So Rider, the Rider that's spelled correctly, W-R-I-T-E-R, has just raised$200 million based on a$1.9 billion valuation. Now, a few years ago, Rider was a cool AI writing platform. That was before ChatGPT came out, and it was just a great tool to use to write blog posts and social media posts, et cetera. But they completely revolutionized their company. And now they're more of a well-rounded AI solution for enterprises. I actually had the opportunity to speak to some of their founders a few weeks ago in a conference, And I was really impressed with what they're doing.

25:53They're providing very interesting capabilities to build and train your own models based on their own proprietary technology. So they're not running on one of the large companies tools like Chachapiti or Anthropic and so on. It's their own models that are optimized for business operations. And so definitely a company to keep an eye on and follow what they are doing. Another interesting development from this week is that Perplexity, which is an AI search and research tool, just started testing ads on their platform. So I use Perplexity more and more. I think it's an amazing tool. I still think it's better than both Google's and ChatGPT's Search GPT as far as of right now.

26:36And they just started testing two different formats of ad. One is sponsored follow-up questions. So at the end of every one of your searches, there are follow-up questions and some of them are now going to be sponsored. And the other is going to be paid media positions that are clearly labeled as sponsored content that is going to come as part of the results. Some of the big names that are in their initial launch of ads are indeed Whole Foods, Universal, McCann, and PMG. All of those are participating in their initial program of running ads through their platform. It will be very interesting to see how that works.

27:09Google also starting testing different types of ads in their AI results. And Google obviously is under much bigger pressure than perplexity to figure this very quickly because that's what their business runs on. And if they lose some of that business, their stock will take a very serious nosedive. So they have to quickly figure out how to move from the traditional ads that they're running right now to AI environment ads in order to maintain their current leadership in that market. but it will be very interesting to see how that evolves for perplexity and everybody else in that industry. That's it for this week.