OpenAI’s o1 Reasoning Model Debuts, $2,000 ChatGPT, AI News Anchors

0:00OpenAI's new reasoning model called O1 has arrived. Is it a game changer or more of the same? Plus, will OpenAI charge some users$2 ,000 per month to use ChatGPT? And by the way, will it raise at$150 billion valuation? And are AI avatars about to do TV news? That's coming up with Bloomberg Tech columnist Parmi Olson right after this.

0:27You're used to hearing my voice on the world bringing you interviews from around the globe. And you hear me reporting environment and climate news. I'm Carolyn Beeler. And I'm Marco Werman. We're now with you hosting The World Together. More global journalism with a fresh new sound. Listen to The World on your local public radio station and wherever you find your podcasts.

0:55Welcome to Big Technology Podcast Friday edition, where we break down the news in our traditional cool-headed and nuanced format. OpenAI has just released its reasoning model. It was called Strawberry as a code name, but now it's being called O1. And we're going to talk all about what it is, what it means, and where OpenAI and the AI race goes from here. And we have the perfect guest host for us to do it. Parmi Olson is here. She's a technology columnist with Bloomberg, and she's also the author of a new book out this week. It's called Supremacy, AI, Chachipiti, and the Race That Will Change the World.

1:32and what a week to release a book because we're just like here flooded with AI news. Parmi, great to see you. Welcome to the show. Thank you. It's wonderful to be here. I'm a big fan of the show, so just so pleased to be able to talk to you. Awesome. Well, thank you, and thanks for coming on. Let's get to the big news, which is that we've been hearing rumors that OpenAI was going to release this strawberry model, which it was initially called Q-Star, and there were worries about it around the Sam Altman ouster, and it's here. And basically the big thing about this model is that it does reasoning.

2:04And so you can ask it a question and you can actually kind of see the way that it thinks through the problem. It goes through a bunch of different steps. And that's enabled it to be much, much better, more accurate, more competent, smarter than any previous model. So I'm like looking at some of the charts that OpenAI put out with the release. I'm looking at accuracy. So if you think about competition math GPT 4.0 was getting a 13.4 accuracy score on Competition math, but this thing in the way that it can think through the different problems is getting 83.3 score on accuracy with competition math so a jump from 13.4 to 8 83.3 on competitive on competition code it goes from an 11 to an 89 and phd level uh science question actually gpt40 wasn't bad at a 56 but 01 preview or oh sorry 01 gets a 78 and 01 preview which is the model that's out now gets a 78.3 where an expert humor get human gets 69.7 accuracy score i think that's the percent of questions they get accurate.

3:16That's the accuracy score. Parmi, what do you think about the release of these models? Do they live up to the hype? I mean, we basically saw a moment where, you know, was Sam Altman pushed out of OpenAI because they thought he wasn't being careful about this type of model? Like that was one of the assumptions. Are these a big step forward? What do you think about the release of these models? Well, first of all, I think I would put this into context of the huge amount of pressure that OpenAI has been under for the last year to get out something that's going to cause as much awe as ChatGPT did in November 2022 and DALI before that.

3:55And I thought it was really interesting that in the last few months, we've been getting this sense that, hey, OpenAI has been announcing a lot of things, but it hasn't actually been launching the things that it's been announcing. So for example, you know there was the whole um Sora video thing yeah where is that yeah where where is where did that where was that even actually even GPT 4.0 the the voice activated the sort of voice um conversational um interface I there were I just met people in San Francisco last week who were like yeah I still haven't been able to try that yet it hasn't really fully rolled out um can I you something about that so first of all like please i've been uh wondering where it is because i've been signed up to chat gpt premium for a while and hope that i would get that new voice interface and i haven't gotten it yet and i over the past so i'll just say this over the past 24 hours i've been monitoring the uh the chatter around this strawberry and oh one release and my favorite exchange is sam altman announcing i don't know if you've seen this he says rollout complete It's live up to 100 % of ChatGPT Plus and team users now.

5:06This is of the new reasoning model. One of the first replies is from this guy, Matt Paulson, who goes, when are we getting the new voice features? Well, yeah. Do the thing you said you were going to do first. Yeah, and Sam responds, hey, how about a couple weeks of gratitude for magic intelligence in the sky, and then you can have more toys soon? I mean, it was interesting, but also just so revealing of the frustration I think that you're noting of the fact that like we haven't seen Sora released broadly. We haven't seen the voice features released broadly. And I think that's kind of the trap that Sam and his company has kind of put themselves in by just releasing a lot of demo wear, which is very exciting.

5:48And but if you start getting people too excited for too long a time and then you don't give them anything, then they start to get a little bit frustrated and a little bit cynical of what's actually going on behind the curtain. And you know what? It's funny what you just said that you've been signed up to premium. I have done exactly the same thing because I signed up to Claude and I'm paying for two subscriptions to these things that I'm keeping. I kept ChatGPT Plus because I wanted to try out the voice interface and it still hasn't shown up on my phone either. So here we go. Pardon me, it's funny because I was waiting for voice forever and it didn't come.

6:27So I unsubscribed from Plus and my subscription ends today. This was not planned. So I've been able to actually try out GPT-01 and it's been pretty interesting, but still no voice. So have you got only 12 hours left before you can start? You know, there's like a monthly cap. There's a very small monthly cap or something like that or weekly cap. and I have like not cared about rate limits. I've been like testing the crap out of this thing because I'm like, all right. You might as well. Yeah, and I'll just share like my initial reaction to it and I'm curious what you think about it. It feels like this thing is extremely math and code-based.

7:06Like that's where the actual breakthroughs are gonna be. Like I said, when I was reading these benchmarks, it's competition math, competition code, PhD level science questions that it pauses. And by the way, when you're doing the queries, You can actually see it pause and think through each step. Or do we want to use the word think? Process maybe? I do. And you can correct me on it. But you can see it go and tackle. Now I'm getting nervous about think. But you can see it tackle each different step. And there was actually like if you look at the consumer preferences or the human preferences, like where they'd want to use these new O1 preview models versus GPT-4-0, They actually prefer GPT-4.0 for personal writing.

7:51It's about even on editing text. But where GPT, this O1 model, really thrives, it's much more preferred for computer programming, data analysis, and mathematical calculation. And that's why I think that when you get into this, people start to judge, is this a step forward? The people who are using it for more technical activities are really going to see a jump. And the people who are using it for more stuff like I would, like writing or editing text, aren't going to see it. So there might be a bias among journalists to pan this thing, whereas people in the technical fields are going to be seeing that there's real advances.

8:29Well, I think it's a good strategy, what you just described, that if this is a model that is perhaps more useful for people who are doing data analysis, they're scientists, they're coders, then they've got a specific model which just has a little bit more of a clear utility for them. Whereas if you're in marketing or you're in customer service, then you're more likely to use the previous model, something like GPT-4O, which is a lot more sort of language oriented as opposed to reasoning oriented. I think that's no bad thing in terms of business use case, because I think till now there's been this kind of this effort to try and create this.

9:12Well, we started off with this ambition to create artificial general intelligence. Right. That was the founding of open AI. We want to create AI that has the same broad cognitive capabilities as the human mind. And I think that actually has a big downside when you're actually trying to sell something packaged like that to businesses, which is that you're giving them this Swiss army knife of a tool that has all these general capabilities. and the risk is that your end customer can end up just becoming paralyzed with indecision on what to do. Like if you talk to some businesses who are trying to figure out how to use generative AI models, I heard from one bank for instance, they put out a survey to their employees just saying how should we use generative AI and they got more than a thousand responses back from their employees, which is great in one way, but in another way it actually just makes everything take a bit longer and in terms of figuring out a strategy on how best to use it.

10:15So I actually think from what you're describing, if the models are starting to kind of fracture a little bit and there's going to be a sense going forward that you can use this type of model for mathematical based processes and data analysis and then you can use this kind for language processing, maybe that kind of runs counter to the original ideology of let's build a broad god-like general level intelligent AI. Maybe it's moving away from that, but I think like practically speaking, that kind of makes a bit more sense. So one of the things I've been thinking about as O1 has released or Strawberry has released has been, okay, so the models are smarter.

10:57where's the application i think you sort of brought that up um it's it's like are we gonna this is one of the things that has been sort of plaguing the ai industry even though it's doing well obviously but it's been like okay the models are getting smarter and they do magical things but we still don't really know how to use them in practice right this is actually something that i I highlighted there was a Grok engineer who said, O1 seems powerful so far, but everyone is kind of unsure what to do with it. Kind of humorous place to be after all the speculation. So what do you think happens as these models get smarter in fields like science and coding?

11:38Yeah, I think it's really interesting because there's this race in Silicon Valley to make everything more and more capable, which is disconnected, I think, a little bit from the sentiment among their enterprise, like businesses in the rest of the world who are just like, we don't necessarily need AI that's more capable. We just like we're quite happy with what you've already put out there. Like that is already pretty impressive and as a step change up from what was even available two years ago. So my sense over the last year has been one thing that's been lacking from the AI model vendors like OpenAI, like Anthropic, like Google, Microsoft with Azure has been just to do a little bit more handholding with their business customers on how to actually implement the current models that are available.

12:31But I think there's just this kind of, you know, rabbit enthusiasm to try and just make these models smarter and smarter. You've talked on your show before about the arms race. It's kind of had this it's got this kind of self-perpetuating cycle that, you know, even like Salesforce last night, their announcement got completely overshadowed by OpenAI. But they announced AgentForce, which is basically what they say is the very first AI platform for businesses that has autonomous agents. There are startups that have been talking about doing this but haven't released this. And I haven't seen anything similar.

13:13I don't know if you have, but from Google or OpenAI or even Anthropic, people are talking about it. And it looks like Salesforce is the first to do it. So there's this real – and they're very proud of that. and so there's this real competition. I'm just going to call bullshit on that. I don't think Salesforce is going to be the leader in AI agents or even the pioneer. I've heard people talk about things as AI agents then you take a look at what it is and it's like there's no agent to it. Well, and what does the agent even mean, right? I mean, when I was talking to them about it and they were saying, I was saying, well, are you using the word bot and agent interchangeably?

13:48And they said, oh, no, no, no, bots. Those are old. That's not it. Agents are new. That's the new thing. So a bot would be if you're talking to a customer service chat bot and it just kind of is based on a large language model and uses predictive technology to say something that sounds about right. Whereas an agent can actually resolve your issue by retrieving data about you, the customer, from the Salesforce CRM and the data cloud, which is one of their big services that came out two years ago. and so it means these things can actually take action. But I'm with you on that, Alex. We've seen so much of this in AI, just a lot of big announcements, a lot of excitement, but then you see businesses actually just struggling a little bit to make these things work for them and work well.

14:40Yeah, and so I'm just going to read a couple examples of folks who've been trying out O1 and why this might be different. And I think it's your right to point out the difference between like chat and bots or bots and agents because this is like from somebody who works at this company called spellbook legal his name is scott stevenson it's very interesting that he says he says o1 is extremely good at long document manipulation docker vision is one of the hardest problems we've dealt with dealt with like for example taking a long set of instructions and modifying a legal document and o1 is really good at this when people are overwhelmed by o1 i think it's because they're thinking of it as chat still.

15:22And its ability to do work is going to be really good once function calling is fully supported. So, you know, I think that like with these bots, it is definitely like, you know, we talked about handholding. And, you know, maybe these companies need to do a little bit more handholding, but I think often in technology, what happens is they put the capabilities out there and then people like Scott Stevenson, right, figure out the way to do it because they're just using it more often. And that's why they're putting this out in preview. And that's the sort of use cases that get spread by word of mouth or productized.

15:56And maybe that's what happens. And I would say that would be the argument to keep just building these models, making these models smarter. And maybe the thing is that they've gotten so fast, so smart, so fast that they've sort of outpaced the utility because we haven't had enough time or the industry hasn't had enough time to put their hands on and try things like this. And actually digest it. Yeah, no, I think that's a that's a very, very valid point. And I think, you know, coming from so I'm based in London and I hear a lot more. I'm not not being based in San Francisco. I'm not in the Silicon Valley bubble.

16:33So being in London, I hear a lot more of kind of the rhetoric from people on the east coast of the U.S., from Wall Street, from investors, from hedge funds, from health care companies, legal firms. And there's a real sense of skepticism over the last few months, which I know you've talked about. But I think it's not entirely warranted. I think this market correction that's happened in the last few months was healthy. Like it was it was crazy that NVIDIA became worth three trillion dollars at one point in market capitalization. And I think when people talk about hype, I think so much of the hype is about timelines.

17:15There's a sense that businesses and investors kind of got this impression that AI was going to start bringing a return on investment for companies sooner rather than later. Like I was talking to one CEO of an AI vendor a few months ago, and I was saying, so how are you going to measure how are we going to measure success for generative AI in terms of return on investment for businesses? And you know what he said? He said in the next two or three quarters, you're just going to see more Fortune 500 companies report an increase in EBITDA numbers like they're just going to have higher profits. I just think that's ludicrous.

17:56Like, really? Like, in the next two quarters, more businesses are just going to make more money because of AI and that's how we'll know. Yeah, it's going to take time. It's going to take way more time than that. And that's been the case with every major tech revolution, whether it was search or desktop or mobile. We all know about the Gartner hype cycle and these things just take time to implement. So, yes, if if the AI vendors in Silicon Valley is racing ahead on capabilities, sure, maybe the businesses just need to catch up. And so with reasoning, reasoning, a lot of people have talked about how it lays the groundwork for these agents, because, again, like seeing it, it's pretty amazing watching using these models and then watching it go through the multi-step process.

18:41and the idea is like all right if you're going to have an agent that's going to go and take action for you on your behalf it's going to need to be able to like process step by step in order to do that and this is the way that open ai is describing it so this is from their head of research bob magoo in in uh in the verge we have been spending many months working on reasoning because we think this is actually the critical breakthrough fundamentally this is a new modality for models in order to be able to solve the really hard problems that it takes in order to progress toward human-like levels of intelligence.

19:15So I'm curious, like, do you think this is a step toward agents? And why is reasoning this sort of, as he describes it, the fundamental or the critical breakthrough? Well, I think that what you were just, earlier you described the tweet from someone and what they were describing in terms of when they played around with GPT, this new O1, it made it sound like there was some gray area here where it had almost agentic properties. Why is reasoning important? Well, I think this just gives it more broader capability. And I think it's really interesting that OpenAI hasn't actually used the word agent once in its announcement, And it doesn't sound like any of its scientists have talked about that either.

20:05Well, doesn't Bloomberg have this like, didn't Bloomberg publish the open AI's like imagining its future? And it has like five levels. And level one is chatbots. Level two is reasoning. Level three is agents. Level four is the AI innovates. And then level five is the AI does work at the level of an organization. So the suggestion would be that we're at level two. Yes, level two. And that agent level is what's next. Yeah, yeah.

20:59was that O1 can be used by healthcare researchers to annotate cell sequencing data by physicists to generate complicated mathematical formulas needed for quantum optics. I'm going to stop there. Those are really, really specific things. Technical stuff. Very specific. And then they end it with this bit. Then they say, and by developers in all fields to build and execute multi-step workflows. That's like anything, basically. So they go from... They basically make it... that guy who told you that the um you're gonna see this in ebita in the next few quarters maybe they're right i don't know you think he was right no probably not but i can see why he thinks that right because again this is going to come to like the very uh labor intensive very about value a high value tasks in our economy that's what it's going to do yeah but don't you but don't you think also like already there's been a lot of concern among businesses about hallucinations right from these models.

21:56Now, when you give them the capability of taking action, so again, sorry to go back to the Salesforce, but it's just because I have a concrete example that they gave me, which was, which was this agent, a agent in their, um, in the customer service chat bot, um, which could retrieve data and basically resolve a ticket in the same way that a customer service representative would do. that's putting even more faith in these models. Like it's one thing to make a mistake about information, but to actually screw up the process of, you know, resolving someone's complaint or getting some information for them, that's a bit more high stakes.

22:41So to be honest, I have no idea how this will play out. And I think businesses are going to be very cautious just because there has been so much concern about hallucinations till now. Right. Yeah. And OpenAI even in its blog post says that it does not solve, this does not solve hallucinations. And then you could also go to some of the critics. They're having a field day with this. Here's Eric, sorry, Ed Zitron. We have our big stupid magic trick, folks. Strawberry. It's OpenAI's new model. It will take 10 to 20 seconds to give slightly better answers. And it's buggy. OpenAI is getting desperate.

23:20And this is a real deal pale horse. I think that's a little bit over the top. What do you think? It has to be. I mean, it's as Ed, that's his thing. But I spoke to someone last night who had tried it as well. I haven't had a chance to try it myself. And they said they didn't. And this was someone who tries these models a lot. and they said they couldn't really see a big upgrade or a major step change or even a particularly big difference between what Anthropics Claude can do. But again, I really need to try this out myself. And I think it's quite hard to kind of put a judgment. They literally announced it last night.

24:02So we can all test it. We can all play around with it. But I think it will really get a sense of how useful this is when people start using it for work. Yes. Okay. So let's end this one just with an unfair question. Buying or selling the hype on Strawberry or O1? I'll go first. I'm buying it. I'm buying it. I think it's going to be more difficult for like the quote unquote word sales versus the shape rotators, right? The people who deal in text versus the people who deal in numbers. but I think this was sort of a necessary step and you see the benchmark improvements here and it's not going to look immediately just like this you know the way that some of the other let's say code completion you know initially where it's like okay so it can complete some code and now it's starting to really look like it can code things up on its own I think that this is going to be something that folks in the technical fields are going to use and and it will actually be a a significant step up.

25:02Okay, how about you buying or selling? That's a hard one to answer. I guess I'll just say sell for now. Because this thing about the benchmarks, I personally don't fully understand the benchmarks and what they're weighted against and what they're sourced against. And, you know, are these completely standardized across the industry? Those are really impressive numbers, but what do they actually mean? And again, I think it's very hard to really judge this until you try it. Right. And I'll just give one more piece of evidence on my side. I know this is unfair, but I just saw a thread with a stream of folks who've like tested 01 on their personal benchmarks.

25:43Like a lot of people have their like own personal tests to give AI. And it's just like person after person talking about how much better it is than their previous tests okay but the other thing here is is really like maybe yeah i don't know like this is preview of course and you you you mentioned it and and you're totally right that we've seen sora uh which has been a lot of hype and not released yet and we've seen the voice or gpt4o voice application and the status of that is is still you know up in the air and you know should we give OpenAI a couple weeks of gratitude for the magic intelligence in the sky and have more of our toys soon?

26:30I don't think so. Well, I'll just, in spite of everything I've said, there is an argument that actually maybe behind the scenes, Sam Altman and OpenAI have just realized they need to focus on one thing that really matters. And if I was going to look at those three things, like the voice interface, Sora, and reasoning capabilities, hands down, the bigger deal is reasoning capabilities. So maybe what's happening behind the scenes is they've just put a pause on these two other things which aren't that important. And the reason I say that is because last week I talked to some people in the Valley, and a couple of times it came up that there's growing discontent about Sam Altman's general lack of focus.

27:20And people are saying he's disorganized and they're trying too many things. And I spoke to one VC who even said that he was going to meet Sam Altman that weekend. And I said, if you could give him one piece of advice, what would you tell him? And he said, I would tell him that he needs to look at everything he's doing and he needs to pick two things and just stick with those two things. And he said, Even if one of those things is this big chip infrastructure project that he's, that's fine. Do that. But then just do one other thing and nothing else. Because I think there is a concern that OpenAI has just got too many pans on the fire.

27:57And so, yeah, so I would say, yeah, I wouldn't discount Strawberry and Slash 01 just in case. Actually, maybe what's going on here is that they've realized they need to just focus on the higher stakes bet that counts for more. Yeah, Parmi, you're convincing me here because there have been sort of rumblings and rumors going around that OpenAI has lost it and too many people left. Look, Ilya's gone. He just raised a billion. And John Shulman's gone. And he's at Anthropic now. And, you know, the rest of the leadership team has thinned out. And there's been staff exodus and OpenAI ruined itself. And I think we can say definitively, well, maybe not definitively, but we can say with good confidence now that the company is still pushing the envelope.

28:52And I was just on the chatbot arena where they battle different chatbots against each other and you vote which gives the best output. And GPT-40 is still the winner there. Oh, really? More so than Claude? Oh, yeah. It's definitely above. It's actually Gemini is second. And this is where people vote on what's best. So now human manipulation could be responsible for some of it. But I do think that the death of OpenAI has been prematurely reported. That being said, there's a lot of competition. You know, we can agree on that. But they're not dead yet. Yeah. And they also want some more money. So it brings us to our next story.