123 | The “Holy Shit” moment of AI, Open AI introduces o1, lessons learned from interviews with Sam Altman, Bill Gates, Reid Hoffman, Mustafa Suleyman, and more AI news from the week ending on September 13

0:00Hello and welcome to a weekend news episode of the Leveraging AI podcast, the podcast that shares practical ethical ways to leverage AI to improve efficiency, grow your business, grow your business, grow your business, But also, OpenAI dropped a bomb, a nuclear bomb, if you want, this week with the release of Strawberry or Q-Star or what ended up being called O1, which is a completely new kind of AI model. And we're going to dive deep into that topic with the background and examples and what exactly it might mean for us in the near future and the longer future. So most of the episode is going to focus on that.

0:55And in the end, if we'll have time, I will go very quickly like a list of news that happened this week. But most of the episode, we're going to dive deep into the release of O1 and my learning from the Macon conference.

1:13On Thursday, September 12th, OpenAI released a new model called O1. 01 and actually what they released is 01 Preview and 01 Mini. And we're going to talk more exactly what that means later on in this episode, but it's a new kind of model called 01. Before we dive into what is 01, what does it do, what's the big deal, and why am I doing a special entire episode about it, I want to go back to some background information. In Q4 of last year, Ilya Satskavar, who was the chief scientist of OpenAI and one of its co-founders, have seen the development of a new project within OpenAI that was called QSTAR.

1:56QSTAR was rumored to be able to have reasoning beyond anything that any model that we have seen before, and we're going to dive into what that means in a minute, but that led him to believe that OpenAI is making progress on things that might be dangerous and that is not reported to the board, which led him to go to the board and get Sam Altman fired from the company in November of that year. And the rumor mill talked a lot about what did Ilya see? What was the big deal of Qstar that made him think that this was a leap forward that might be too dangerous to release without additional oversight as Ilya was in charge.

2:39As you probably know, Ilya was in charge of alignment, meaning safety of the models that they're releasing. And he felt that OpenAI is developing it in a faster speed and not with enough oversight for the capabilities of that model. Later this year, like around mid 2024, we started hearing rumors about Project Strawberry, which became the codename of QSTAR. And what we've learned is we've learned that Strawberry has very powerful reasoning capabilities and that OpenAI is even using Strawberry for reinforcement learning for a new model called Orion. So what does that mean? When these companies are creating these models, all the big models are created in the same way.

3:23They build an initial model and then they use what's called an RLHF, reinforcement learning from human feedback. They basically allow the model to provide several different answers to a question and let people rate those answers, which one is a better answer. And this human feedback is what allows them to fine tune the models before they release it to us to make it as relevant and as powerful to be able to serve our needs as humans. So they were able to now use Strawberry to do part of that process for a new model called Orion. What does this mean? It means that you need a lot less people and you can build, maybe not now, but this is definitely sets up the foundation for a self-learning AI platform, which means you can iterate significantly faster, assuming this produces good enough results.

4:17And from what we're hearing and from the rumors from OpenAI, it does. So Strawberry was now being talked about something that may work itself into NextChat GPT. There were, again, multiple names for this thing during the process. So we mentioned Qstar, we mentioned Strawberry. There was a big presentation by the head of OpenAI Japan that mentioned GPT-NEXT. So nobody knew exactly when it's coming out or exactly what it's going to look like, but they landed up on the name of O1. Now, I must admit that for a company that has some of the brightest minds when it comes to developing AI platforms and with maybe the most powerful AI platform there is out there today, they could potentially think of using AI in order to give better names to their platform.

5:05So Strawberry or Orion, I think would have been better than O1, but it is what it is. It might be Orion 1. Maybe that's what it stands for. But why 1? Because we had GPT-4 before. And so the reason that they're claiming is that this is a completely new type of model with reasoning capabilities that are very different from everything we have seen before. And that's why they decided to restart the count with O1 versus continue to GPT-5. So what's the big deal? What does it do? So now I'm quoting from the release of the model notes by OpenAI, and they say, performs similarly to PhD students on challenging benchmark tasks in physics, chemistry, and biology.

5:50They also mentioned some additional statistics like on the Mathematic Olympics, which is a mathematic competition that happens every year, GPT-4-0, so the previous model, correctly solved only 13 % of problems, while the new reasoning model was able to score 83 on that competition. So a huge jump. This is a 5x improvement on solving math problems. High sophisticated math problem at a global math level scored 83 on that test. And they're also saying that the coding abilities evaluated under coding contest scored in the 89th percentile on CodeForce's competitions. So on another thing that requires more deeper thinking and planning, it scored in the 89th percentile compared to the best competitors in the world.

6:45So if you compare it to the average developer, it probably exceeds their capabilities. Now, I want to connect back for a second to Ilya's concerns. And as Ilya ended up leaving the company eventually after Sam got back, he stayed in the background. Nobody knew exactly what was happening with him. And then he left a few months ago to start his own company that is called SSI, which stands for Safe Super Intelligence. He raised a billion dollars as his first round of raise, which tells you that a lot of people with a lot of money respect him as an important mind in this field. And we're going to get to Ilya as well, but he had obviously big safety concerns.

7:26So one of the things that OpenAI emphasized during the release is that they were testing this model for jailbreaking as well. So on one of the hardest jailbreaking tests, GPT-40, which again is the latest model we had before this one, scored 22 on a scale from a zero to a hundred, meaning it's relatively easy to jailbreak it, while the new model, O1 Preview, the one they released, scored 84, so four times better in protecting itself from jailbreaking. Now, in addition, they shared, as I mentioned to you last week, that they are collaborating with both US and UK AI safety institutes and giving them early access to these models to be able to test them and provide feedback before and after the models are being released.

8:11As I mentioned, they released two models. One is called O1 Preview, which is a less powerful version of O1 that they will probably release sometime in the future after they get feedback from users. And they also released O1 Mini, which is a faster, cheaper reasoning model that is particularly effective at coding. It is 80 % cheaper to use O1 Mini on the API, and it will provide a cost-effective access to developers who wants to use this model, but do not need some of the other capabilities of this model, such as, and I'm quoting, broad world knowledge. So if you're looking to use it just for coding, O1 Mini is probably a better way to go.

8:53As of right now, users do the chat GPT interface. So if you're just using it through the chat, you'll be limited for 30 messages for O1 Preview and 50 messages for O1 Mini. If you're using it on the other hand of the API, you just pay the API token fees and it doesn't really matter. You're not limited. So before we dive into what is this whole reasoning thing, I want to share some things that were said by Noam Brown. So those of you who don't know, Noam Brown moved from Meta to OpenAI in June of 2023 to do exactly this. So what Noam is known for, he was the one that developed the models that were able to beat the top human players in diplomacy and then in open unlimited poker to be the first machines that can reason through very sophisticated open-ended games and still beat the top humans in the world.

9:46So Noam shared a long tweet on X, which is weird to say tweet on X. It should have been either tweet on Twitter or X-ing on X. I'm not exactly what's the right term right now, but putting terminology aside, he shared his thoughts on the day of the release on this model. And he shared several very interesting statistics and his thoughts about the whole thing. He shared a bunch of graphs, which I will share with you in words. So math competition, basically competing on the most sophisticated math problems that exist in the math Olympics. GPT-40, the previous model, scored 13.4 points. O1 Preview, the one that they released to us right now, scored 56.7.

10:25And O1, the full model that is not released yet, scored 83.3. Huge spread on all the different aspects. On code competitions called Code Forces, GPT-40 scored 11 out of 100. 01 Preview, the one we have access to right now, scored 62, so six times better. And 01, the full model we still don't have access to, scored 89. That is an incredible score because that means it surpasses the vast majority of developers in the world on complex coding problems. And now maybe the most amazing statistics is on PhD level science questions, GPT-40, the model that was released before, scored 56.1. The preview and the full 01 models scored 78 and 78.3.

11:18And then human experts, so actual PhDs on these topics, scored 69.7. So these models scored 10 % better than the experts on those specific topics. That is, on one hand, amazing and mind-blowing and represents a path for a future where these tools can generate on their own new scientific discoveries. On the other hand, it's very scary because it has a lot of negative implications as well. Now, the other interesting thing that they shared that this model is not necessarily better than GPT-4O on all tasks. What they shared is a graph that is showing the win rate of the new model, GPT-01 preview versus GPT-4O, which is the model that we were using before the release of this.

12:06And on personal writing, our GPT-4O, the previous model, is actually winning more times. On editing text, it's exactly 50-50. But then on computer programming, data analysis, and mathematical calculations, GPT-01 preview, which is the new model that we just released, is doing significantly better, winning 60 % of the time on programming and data analysis and 70 plus percent of the time on mathematical calculations, which if you tried it on the previous models, it was crap. Like you cannot do math if you give it anything more sophisticated than one plus one. The other thing he shared that is very interesting is a direct correlation between the model's time of thinking to the accuracy of the results.

12:53So the biggest difference, if you're going to start using this model that you're going to see, is when you finish typing your prompt, it doesn't start answering. It literally says thinking, and then it thinks about the problem for a while, and only then it summarizes stuff in its head, and it shares the results with you. And the graph that they were showing is that there is the time that it took to compute on the x-axis and the accuracy of the results on the y-axis. And it starts with a very short amount of time being correct 20%. And it goes all the way up to 80 % accuracy when you give it more time to think.

13:30So the ability to give the model time to think and teaching it the process on how to think, which I'm going to explain in a minute, is the biggest difference between this model and the previous model. So what does it mean time to think? And why the hell does it need time to think? And how is that different than all the GPT models and actually all the large language models we had before? All the large language models were trained on a huge amount of data. Basically, every digital piece of data that these companies could get their hands on, whether legally or illegally, I'm not going to dive into this right now, but a huge amount of data.

14:04And what they know how to do is to pool the right data and summarize it as a coherent answer based on the prompt that we give it. So it doesn't need to think. It needs to summarize existing information. Why is that? It's because it was always given the answers, right? We give it a huge amount of answers to a lot of questions, and then you ask it a question, it looks for an answer across all the data that it has, and it provides the outcome. It does not know how to work the steps through the process in order to get there. And this is exactly the difference to this new model called O1, with its ability to think, reason, and correct itself through the process.

14:50So it can actually work step by step like humans do, evaluating every step, whether that step takes it into the right direction. It can go back and refix and try something else, just like we solve problems. So why does it matter? It matters because so far when we use these models, we were the reasoning engine. And I teach that in my courses and I work with all my clients. If you want an LLM, all of them until a couple of days ago, you had to be the step-by-step process on your own, meaning you had to tell it, let's focus on this first and this would be step one and I'll take the outcome of this and let's do step two and let's reevaluate step two because it wasn't great and go back and do it again and try to fix this and focus on that and don't do this.

15:33and then go to step three and four and so on. And you can get amazing results way faster than you could have done on your own. So the capability of these models is still great, but you had to walk them through a step-by-step process. You want a tangible example. Let's say you want to write a proposal. If you just ask it to write a proposal, you will get a proposal of one page. But if you focus with it step-by-step on, let's evaluate this first step of the proposal and focus on just this initial problem, you will get a page and a half on just that problem. And then you go to step two and write that component and this component, and you're going to end up with a proposal of 20 pages that's going to be very detailed and a lot better than just trying to do it all in one step because you are forcing it to work the step-by-step process and to fix the different steps.

16:19So now I want to give you a couple of examples that people already did and shared about how this new model, O1 Preview, is solving problems differently than existing previous models. Ethan Malik, which if you don't know, you should. He is one of the most amazing thinkers about AI right now. He's a professor and he shares lots and lots of content across every platform that you can think of on AI and giving feedback and his thoughts about it. I follow him on LinkedIn as well as I regularly read his amazing newsletter. And his recent newsletter is called Something New on Open AI's Strawberry and Reasoning.

16:58And I'm quoting one segment out of it, and I will quote a few others. But this is from Ethan Malik's newsletter. So Ethan Malik is saying the following. Let's the AI think through a problem before solving it. This lets it address a very hard problem that require planning and iteration, like novel math or science questions. In fact, it can now beat human PhD experts in solving extremely hard physics problems. But the test that he passed it through, which I found fascinating because I believe it's a great representation of real life, is just solving a complex crossword puzzle. Now, he, first of all, gave that problem to existing advanced models, and they all failed miserably, or like he's saying, they didn't even come close.

17:46Why? Because they cannot do the iterative process of understanding the dependency between the steps. In order to solve a crossword puzzle, you need to solve one step and only then evaluate if that actually fits the following steps. And if it doesn't, you need to go back and change it and try again until it works for the next words that cross with it in order to be able to complete the step. Now, the previous models, as I mentioned, failed miserably. But when we gave it to 01 Preview, the model took 108 seconds to think. That's a very long time. Usually it thinks about 10 to 20 seconds before it gives you an answer.

18:19And you can actually see the thinking process when you do this. So if you're using this model, you can click the drop-down menu button, like the little carrot, and see what the model is actually thinking, what it's doing. And Ethan Malik shared the process that his model was doing. But you could see that it's actually creating and rejecting new ideas that could help it solve this problem. Now, in the beginning, it failed because one of the definitions was very hard, and he actually took it too literally and was not able to solve it. But when Ethan gave it the answer to that first definition, the model was able to solve the rest of the puzzle on its own.

18:58And then in the summary of this, Ethan Malik said something that I find profound. He said, AI does so much thinking and heavy lifting, churning out complete results that my role as a human partner feels diminished. So I want to go back to why this is a big deal. It's a big deal because Ethan is probably one of the most knowledgeable people on AI progress and capabilities in the world today. And if he feels that our role as humans in the process working with AIs is being diminished, not eliminated yet, but diminished, it's a very big deal. It means these models can really reason at a very high level, achieving amazing results in stuff that wasn't possible before.

19:48I want to give you another example. Last month at the 2024 Association of Computational Linguistics Conference, the keynote speaker, which I know just as R-A-O-2-Z from Twitter, that's his handle, was titled, Can LLM's Reason and Plan? And in it, he showed problems that tripped up all of the existing large language models. But when it was given to O-1 Preview two days ago, it solved it on the first attempt. So a problem that was built to trip large language models, this new model handled with no problem without any assistant. You want another example of very simplistic, but very interesting example.

20:32I do my AI Friday Hangouts every single Friday at 1 p.m. Eastern. By the way, it's open to the public. If you want to join us, it's just a bunch of people who come in and geek about AI and current progress and things we learn and solutions and tools and and so on, and it's a lot of fun. One of our regular participants is Sean Bailey, and he shared his experiment with, and he shared his experiment with O1. So something that he likes to show them some of the limitations of the previous models are, he asks the models to tell him how many R's are in the word strawberry, which is obviously a cool pun about the whole strawberry model.

21:09And these models cannot do that. Even if you work them step by step and say, okay, let's break it into syllables. How many are in the first syllable, second syllable, third syllable? And it can do that once you break it into syllables. And then you ask it, okay, so how many R's are in the word strawberry? And it will get it wrong most of the time. O1 solves this every single time correctly. So again, this could be very simplistic, but it shows you the ability of this model to analyze, understand more complex problems, and solve them across things that the previous models were not able to do. Why is that important?

21:45Where does that put us on the bigger picture? A few months ago, a person from OpenAI leaked a document that was OpenAI's roadmap to AGI that has five steps. Level one is called chatbots, AI with natural conversation language abilities. That's what we had until Thursday. Level two is reasoners, AIs with human levels of problem solving across a broad range of topics. That is the step we jumped into that, again, we did not have before. Level three is agents. It's AI systems that can take actions independently from human instructions. Basically, they can act on their own and take action on our behalf without our need to give them exact and specific instructions.

22:34Level four is innovators, AI that can aid in the innovation of new ideas and contribute to human knowledge and make new scientific discoveries on its own. And level five is organizations, AI that is capable of doing all of the work of an organization independently. That could be a committee, that could be a company, that could be potentially governments. So these are the levels that they have defined on the road ahead. We just went from level one to level two. Now, I must say something about my personal thoughts between going from level one to two versus going from level two to three. So going from chatbot to reasoners, which is what just happened, versus going from reasoners to agents.

23:17I have a feeling that from a technological perspective, going from one to two is a huge step forward. Going from two to three, from a technological perspective, should not be a big deal. Because once they can reason, giving them the ability to take action is actually relatively simple. There's several companies are already doing that across multiple very specific use cases. But once they can reason, you can dramatically broaden the use cases. And what's going to be the limitation of going from step two to three is us, our ability to trust these models to actually take the right actions and not put whatever thing at risk.

23:57That thing could be our personal livelihood. It could be personal information. It could be the success of the business. It could be a million other different things. But our ability to trust that these models will actually take the right action is what's going to limit the deployment of agents. But I feel that while we had to wait almost two years from the release of ChatGPT to the release of the second model, I think it's going to be months before we start seeing very sophisticated advanced agents that can take actions on our behalf on specific tasks with specific guardrails to reduce the safety risk.

24:34Talking a little more about the speed things are moving, Noam Brown, which I mentioned before, was the guy that was leading the development of this capability in OpenAI from July of 2023, has said himself that he was shocked on how quickly they were able to achieve these results and that he was thinking that it's going to take them significantly longer. So even the people who are driving this train and that are supposed to know more than all of us on how long it's supposed to take was surprised with how quickly they were able to achieve these results. And now because they are potentially using these models to train the next models, as I told you, they're using strawberry to train or help in training the next models, we might get very quickly to a self-improving loop where AI needs very little human engagement in order to make better and better versions of itself, which has, on one hand, very promising outcomes, on the other hand, very scary outcomes.

25:37Now, this may have very profound implications on everything we know about scaling laws. And I'm going to address it with quotes from two different people. So Sam Altman mentioned that strawberry allows them to scale in ways that were not possible before. The other person that said something completely independent of this, but when I thought about it and connected the dots, I found it to be highly intriguing, is that Ilya Saskover, again, the guy that led to the firing of Sam Altman because of the beginning of this progress, which, as I mentioned, just founded a new company called SSI that is supposed to focus on a safe super intelligence.

26:20He said that SSI is going to take a completely different approach to scaling than everybody else is doing, meaning not going necessarily down the path of adding more compute and more data in order to create better models. And he's saying, and I'm quoting, everyone neglects to ask, what are we scaling? He said. Some people can work really long hours and they'll just go down the same path faster. It's not so much our style, but if you do something different, then it becomes possible for you to do something special. Meaning Ilya has a way to scale the models in a different way with a different approach, not by throwing more hours, more compute and more data into them.

27:10Is that aligned with what he learned in his early steps in that process? Maybe. Now that might be just my personal interpretation, but I have a feeling that this new step unlocks significantly faster development and scale of new models without throwing billions of dollars on them. So the scaling laws so far basically said, throw more money, more compute, and more data on these models, and you get better and better results. And all the people at the realm of the different companies agree that there's no upper limit to that as far as they can see right now. Meaning if GPT-4 was trained with$100 million and now the current model is about a billion dollars, and then in a couple of years they're talking about a$10 billion model, and then three years after that, a$100 billion model, where the$10 billion model is supposed to be better than all human PhDs in the world on every topic of PhD, the$100 billion is supposed to be better than Nobel Prize winners on everything.

28:11But if there's a new way to scale these models, maybe the$10 billion model, which is the next model than the one we're seeing right now that everybody's working right now, could be better than Nobel Prize winners on everything. And that could happen in two to three years. So will this new kind of models and its ability to train itself will dramatically accelerate the timelines that are already extremely fast? Maybe. I don't know for sure, but I have a feeling that it might. Now, to connect the dots between this new model and everything that I shared with you right now about it with my learning from Macon.

28:53So Macon is an AI conference that's been going on from before ChatGPT. So this is now the fifth time it's been run. It's run by the amazing people of the Marketing AI Institute led by Paul Reutzer and Mike Kaput, but they have an amazing team of people that has been doing an amazing work. And it's my second year of attending this conference. And I actually want to start with the end of the conference. At the end of the conference, Paul Reutzer interviewed two fascinating individuals, Adam Brotman and Andy Sack. So Adam Brotman, the executive VP for global retail operations at Starbucks. And before that, He was the chief digital officer for Starbucks, the guy who invented and brought to us the Starbucks digital app that allows you to drive baristas crazy with all your needs and specific requirements.