BONUS: GPT-5.6 Sol Goes Live: Must-Try Use Cases We’re Testing Live

25:01This is the actual codex prompt that a researcher on our team used to have Sol kick off a post-training job for Luna. You can see here that it's asking to find the training configs, find the right GPUs for the job and then launch the script and make sure that it works. It's a really short prompt. Yeah, it's pretty crazy that we've gotten far enough where you can have a fairly under specified prompt and then just give it to codex and then just have it go and run the job. And you know, previously this is something that a team of senior researchers may have worked on at OpenAI and now it really feels like automated researchers is pretty close.

25:34This isn't the only place we've seen internal acceleration. We're also seeing an increase in the number of pull requests per researcher, the number of experiments researchers can run so we can test our ideas and turn them into research findings. And actually Codex made every plot in this livestream and also helped us make the slide deck. So thank you, 5.6. SOL did part of the training. Did you catch that? Across Frontier evals, 5.6 SOL is state-of-the-art. And this is something we're really excited for you to finally experience and see the speed up that we've already experienced internally. On Terminal Bench was the test of coding performance.

26:05The model is state-of-the-art. on BrowseConf, which is an evaluation that sees if the model can locate hard-to-find information, and the same with Agent's Last Exam, which is long-horizon professional work. Wow. As one deep dive, 5.6 is particularly good at computer use, so anything that involves navigating a browser, navigating apps on your desktop, and that enables all sorts of digital work. It helps medical assistants navigate electronic health records, it helps data scientists analyze if a drug is effective, it can even help investment bankers create financial models and you can also use this for personal use it certainly helped me order food and all of this is something that when we ask experts in the field you know how they're using this model in their everyday workflows they say it's better than anything else they've ever used anything else on the market all being three times as fast so we're really excited for you to finally try and feel how powerful this model is you might hear the capabilities that table talking about and then you saw demoed earlier and think that this is going to come at a very high price tag but it's actually not the case with GPT 5.6 sold So token efficiency has been a huge focus of our research program for several years, and with every model, we want to bring more intelligence for every token.

27:10And you can see this reflected on evals like DeepSuite 1.1, where GBT 5.6 outperforms its competitors at less than half of the cost. Wow. Third of the cost to pay. But if you want the best intelligence money can buy, we're also announcing Ultra Mode, which unleashes a whole team of agents to do work for you. So in this example eval, you can see here, you know, one agent does better with time, But as we add more agents, the model is able to do even better and faster. And that's because the agents are able to parallelize work like an experienced team. This is a capability folks have asked for so long that we bring to the Codex app.

27:44And I'm so excited that we're finally putting it out there as ultra-mudge. That's how they're going to make their money. One more improvement that we shipped with GPT 5.6 is, you know, we may remember that when we launched GPT 5.5, we had to ship it with a dev message saying, you know, don't talk too much about the goblins and the gremlins. And this is a result of reward hacking that occurred during the training of GPT 5.5, which has now been resolved in GPT 5.6. And now GPT 5.6 will only talk about goblins a tasteful amount when it's cute or appropriate. With these more capable models, safety and alignment are more important than ever.

28:17That's why we spent over 700 ,000 A100 equivalent hours of compute on model red teaming. We also spent six weeks safety training and testing this model, and also have incorporated multiple novel monitoring upgrades in our launch, including new classifiers and activation pros. Wow. Ultimately, we really believe in iterative deployment and putting our models in the hands of real people as quickly as possible so that we can learn from real-world usage outside of our labs. I think this is good. And that's why we have initiatives like Project Daybreak, where we give cybersecurity researchers access to our models earlier on.

28:48And with GPT 5.6 Sol, researchers have already found vulnerabilities in every major browser and database. Yeah. As part of the Daybreak umbrella, we started Patch the Planet, which is an even broader initiative where we work directly with open source contributors and projects. And we also generate high quality patches. For example, for Linux, they accepted over half of our patches, which indicates that not only do we find critical vulnerabilities that weren't found for a long time, but also we were able to automatically patch them and freely accept cyber defense. We're very proud of this model.

29:21We hope it helps you as much as it's helped us. and we're excited to see all that you build with 5.6 Sol. Thank you for sharing about the research and the model. What an incredible set of releases. We saw ChatGPT work, capable of working on mobile and web. We also saw the all new ChatGPT desktop app. And we saw all of the capabilities of GPT-5.6 Sol, Sarah and Luna, and a little bit about how they were trained. Next up, I'm going to be showing you a use case outside of the office that I'm tremendously excited about. Hiroki has been working with us for the last six months, and he's been using GPT-5-6 to really do something unexpected and help him take care of his farm.

30:10Let's have a look at his story.

30:21and I'll provide a hub of the machines that designers here as will most importantly so I started off seriously Tsai so hard! Not such an engineer, but Разはhero -부ew. I taught the pointedown for luge of vinyl houses having infections renewing through vinyl. After all of it we used as a project. We started to run and drive to plug on cl simultanладies.

31:20duringlevals, the armor can be created. Before we dra umm... it was discussed ,改ior, and48a in an important league. but when it comes to fullStack code, I always take the data match with my data takes or how many tools are now I won't choose the goal or when I am as constantly checked.

31:52This is the it Second Team Trust all I afternoon in my work ! Hardware of customs are going through And he said of Share-Thank-Right

32:13Japan is already panicking, and very responsible thing about MARTINOPHいる to give questions for me and to... MARTINOPHIMEE K frames why all life comes there? MARTINOPHIMEE K plate sound MARTINOPHIMEE Khalten noit and traveling all the way from Hokkaido. I'm going to be using... Chat to be speech-alive. Let us know in the chat. Hey chat, I'm here with Hirokisan, a dear and esteemed friend who traveled all the way from Japan. I want you to act as my live translator. He's going to speak Japanese, live translate to English to me, and I'm going to be speaking English, live translate back to Japanese.

33:01Got it. I'll do real-time Japanese English both ways. Whenever Hirokisan starts... I want you to act as my real-time translator, okay? Understood. Starting live translation now. Hiroki-san, transport is back to your farm to share the magic. Hiroki-san, an amazing magic experience, please take me to the farm to share your life. Yes. Today, I'm going to be here to San Francisco. Thank you so much for inviting me all the way to San Francisco today. I honestly can't believe I'm here right now. Right now, at our farm. We're sowing broccoli seeds, telling the fields of tractors, harvesting broccoli, and many other tasks all happening at once.

33:49We'll start harvesting broccoli next week, so once this live event is over, I'll be heading back to Japan. With just a small team, we're managing a very large area of fields. So improving efficiency is really important. I'm going to do that. To do that, we're making use of ChatGPT to help us with our farm work. That's how we're moving things forward, yes. What? You've really shared all of this from the very beginning. Why did you do that? And what were some of the reactions? You've had this all-out. Why did you think you wanted to do that? And what kind of reaction was that you had? What do you think AI is not a clear answer to it?

34:35I think there are a lot of people who are not sure to use AI. How do you use AI? There are still many things where we don't have clear answers yet. But given that the question is how we… Well, I don't know the right answers yet. Since we still don't know the right answers yet. I think it's important to just keep putting things out there. What were some of the things that people shared with you? What was the fact that you shared with us? in general what was that? yes, wo Interestingly Us Fuckado

35:37Wow, that's very unexpected. What could possibly be the next challenge for you? What would you like to do next? The next challenge is... I want to use AI to improve more. I want to use AI to improve more.

35:58I'm using AI in the group chat. I want to use AI to improve the daily daily daily.

36:08Can you translate that to Japanese for me?

36:15Can you translate that to Japanese for me? Can you translate that to Japanese for me? Let me just restart that really quick. Live demo. Hiroki-san, I have a friend from Japan. I want you to act as my live translator, okay? Please, he's going to speak Japanese. Just live translate to English to me. Got it. Starting real-time Japanese to English translation now.

36:46Within the team, I want us to make better use of AI. Right now, we're using our team chat to record our daily notes and share updates on how the vegetables are doing. We're trying to build systems so that our team members can work more comfortably with the idea that ChatGPT will support those systems. What you've been doing has been very advanced. What would you advise some non-technical people? How can they get started? A lot of you have done this. It's a pretty high-tech job.

37:34So, I think it's important to just start small. I wasn't an engineer myself. So, by tackling each thing I didn't understand and asking ChatGPT about it, I gradually learned how to do this. So, by tackling each thing I didn't understand and asking ChatGPT about it, I gradually learned how to do this. to build systems that I can use on my farm. Now I'm able to create and do things that I never thought possible. So I believe that starting small and building step-by-step is really important. Thank you so much, Hiroki-san, for sharing your journey with us and being live here today. I wonder if you're a human translator, if you would have that coming and then you would just correct it.

38:21And the advice from Hiroki is profound. It is about taking small steps in raising the level of ambition you have in order to have ChatGPT Work get things done for you. What you saw and all of the product updates and the models are rolling out over the next 24 hours. Sol will be available for all our paid plans together with ChatGPT Work, the desktop app, and sites as well. The ChatGPT desktop app is also available for free users. Sol is available for all our paid plans and Terra and Luna for free users rolling out over the course of the next day. Thank you so much for tuning in. I can't wait to see what you do with it.

39:08I have so many thoughts. I've been taking notes. I don't know if you saw this, but the quick side note in Codex right now, it says Codex is now the chat GPT app. i didn't hear them mention that did you hear them mention that okay so it says codex is now the chat gpt app and in the top left hand corner it says chat gpt codex if you uh if you want i can share the screen hold on i'm pulling mine up i just i've had it up waiting to see if a codex update came i did get a chat gpt app update because like i have the the the original desktop app that it now calls chat gpt classic

39:57and so if you if you want i can pull it up but if you have yours up then i will uh then you can share your screen first uh shout out in the chat what did you think i mean some people might be just tuning in now mine does not say that by the way okay um let me take this screen down and if you pull my grant's computer up from backstage i will show you what i'm seeing i am that will or was that a different computer grant's computer it's called grant's computer you have it backstage right now yeah yeah yeah show i have i have lots of thoughts that i think we should talk through look at this let me see if i can zoom it in codex is now the chat gpt app chat gpt codex yeah well you know it's had both versions for a little while where you could switch to like from code mode to work mode really uh wow that was sorry sorry to take you to the infinite mirror universe for a second there while i was like this Yeah, chat GPT codex.

41:07So I guess you can do chat GPT work, and then you can do chat GPT codex. Yeah. So. Yeah, essentially that's been in there as a feature, but a little less integrated, I would say. Like it was a – you had to go into the menu and do some change things. But to me, that's not the big deal more so than like with it in codex. Just the name codex runs away people who aren't developers. Like that's – same with Cloud Code. the truth is i understand why these were both made the way they are but but that is a thing that is highly intimidating to someone who is like i'm just going to go in there and break stuff like i've talked to a number of people who feel that way um and honestly i've not noticed a big difference between the two modes i wonder if i can let me see if i can use uh the new model in here is 5.6 available?

41:58Click on the carrot there and see. Be sure. Yeah. It says no. No. Okay. It's not in any of my logged in things either. Yeah. I want to say that something I think today showcased about OpenAI, where I think they have shined more than other people, other companies, is there are a lot of approaches to things that are very practical. like live translation and voice um background computer use i like that like in the past that's been a mostly in front of things like not and i could turn this on while i keep working you know sort of thing like you you sort of had to stop and watch it work painfully slowly but that looked to be moving quickly now uh sites that's a big deal i don't know who else has those now but they've had it on the business plan for a little bit but not on the personal ones i think it was sort of artifacts were always kind of supposed to be that but uh artifacts by the way is referring to claude um and then chat bd has his own version but they never really like they were ugly they they didn't never took off yeah that's not what people wanted what people want is like i want to be able to publish whatever i create in this app directly to the internet how do i do that and they've had I've used all these workaround solutions.

43:20Just this week, I built, well, we'll talk last week. I'll talk about the app I built that I put up on Vercel. So I, long story short, bought an espresso maker. I spend way too much on coffee. Was trying to figure out, I want a way to amortize my coffee maker. $900 a month. It was a lot. It was a lot. Enough that Grant remembers the number. Sorry to put you on blast, but I thought it would be good content. I deserve it. And so I bought this and I'm like, I need an app where I can go in that knows what I paid for coffee, what I paid per cup to make it at home. And can do all the math there and tell me like, hey, you've saved this much so far because that motivates me.

44:06So it's got like achievements and things. And my first reaction was I had Codex build it and I was like, but what do I do with it? Like, you know, I mean, it doesn't work as a Chrome extension because it's not desktop friendly. it didn't make sense it made sense as an iOS app but because I was building it on a Windows machine which I've now solved it wasn't going to be an iOS app in the near future so what I wound up having to do was take it to Vercel and have it well actually I upload it to GitHub and GitHub posts it to Vercel for me or Vercel grabs it I guess is a better way of putting it so like that works and that's fine but I would love a way to be like this is awesome i just built this i want to show my friends right now i i absolutely want a link right now that i can send to my buddies cory quick update for you turns out that steven webb if we could put his on the oh he has it steven webb in the uk has 5.6 in his codex.

45:10Wow. A country in Europe getting access before the US? That's impressive. I'm actually genuinely proud of OpenAI because normally it's like, and Europe has to wait. And Europe will have you in two weeks. Good luck. That's great. Terra and Luna available in the CLI, but no soul. That's good to know. I'm seeing if my pro account has it yet. I'm seeing if the extra money buys me early access or not. I would hope so. The desktop one is always the weird one because it's like, when are they going to make this switch? Not the desktop. Hang on. I'm checking, seeing if there's an update to the mobile. Come on.

45:51Give me mobile pro at least. Give it to me. Let me check. Why don't I check on this other screen? I'll pull this up if there's anything to report. If it's in the playground. Okay. Yeah, if it's in the playground, we can pull it up over there. That's how we played with it before. We didn't hear API costs, did we? No, but I think they announced those last time. We can pull that up on the screen as well. I did think it was really interesting that cost per tax was a third of the cost of Fable. Yeah, that was interesting. It was a couple notches higher, not dramatically higher. The quality wasn't enough to do, but I think price is a big deal.

46:32I mean, when you're talking about, it says it doesn't, it says high cybersecurity capability. My understanding is that it does not have the kick you out of the party guardrails that Fable does, but we'll have to see that to know. I know those were kind of crazy at first, but you have Terra and Luna in Codex and you're in the U.S. Okay. Okay. I am checking for updates frantically, as I should. open ai you have t minus five minutes or we will continue streaming only fable yes i need to lodge a complaint we will only stream fable by the way for folks who are tuning in thanks for joining uh we just watched the live stream we're about to do some head-on um fable versus gbt 5.6 tests i would like five prompts that we are going to run and then whatever else people want to see and whatever else cory's about to say go ahead before we get into that i think we ought to chat a little about share our thoughts on what we saw.

47:33Go ahead. Background computer use has a lot of potential in my opinion. I can see where one of the things that's been a struggle with computer use is it's only helpful if I can do other stuff while it's doing the thing. If I have to sit here and stare slowly at something working on my computer how is this benefiting me? I'm not saving any time. I'm not doing less. A couple people have it in Codex. I just rebooted Codex. That's a good idea. I'll do that too. Not yet. I didn't check my personal one though. Personal Codex incoming. Relaunch Fable as well. Sorry. You can pull this computer down for a minute.

48:16Okay. Or I'll just stop sharing for the moment. Oh, you got it. Take y 'all out of the infinite vortex. In fact, if you want to pull up the page. I'm trying to hide it. One minute. You know what we ought to do? We ought to talk through some of these tweets and the reaction of the models specifically real quick. I think that would be a fun thing. There we go. Move to backstage. I'm going to share this real quick because there's a few cool things we can work through while we're waiting.

48:46Okay. Wrong one. I wonder if my old codex doesn't work anymore. This was, for me, one of the most interesting takes I saw on Twitter yet. And Altman replied to it last night as well, saying that he liked Rottweilers. What does he say? He says, my view of Fable 5 versus 5.6 Soul. They're not easy models to compare. And that's kind of the vibe I've gotten from everyone, is that they're sort of different things, and head-to-head doesn't necessarily mean what it does with other models. My overall feel is that Fable is a wise owl that is very thoughtful, very well spoken. GPT-56 Soul is like a Rottweiler who will grab the problem by the throat and not let go until it's done.

49:31In other words, Fable is fundamentally smarter, even at low reasoning. It can be insightful, writes in a clear, compelling way. On the other hand, GPT-56 Soul is extremely diligent. I can give it a list of eight things to do, and you can be sure they will all be done. Fable feels more arrogant. uh it talks here a little bit about how he was arguing with it to do a thing and it it kept giving itself perfect score uh uh then he got into ui and app building and and still gave fable the edge for for ui design which i think is interesting uh the flow of the app would probably be a little nicer but i find fable often misses quite key things which sold doesn't gpt's front end skills are a big jump versus gpt models, but not as great overall.

50:17That is the, which, which I think is, is a fine call out. Writing, Fable is better. So feels difficult to align what I want to say or explain things. I'm anxious to try that out. I haven't, I haven't spent any time writing with either of them. Hey Corey, sorry, real quick, this sidebar chat happening in the background. So I can report the Codex app has disappeared off my desktop and it is now replaced with ChatGPT. So ChatGPT and Codex are now officially one app. Okay, maybe I need to close my ChatGPT app. Yeah, close both of them and then try to log into Codex again. And what happened for me was that it's just now one application.

50:57And it looks like other people are seeing the same thing in the chat.

51:04I am still not. I mean, you know, they said they're slowly rolling it out over the next few hours. Yeah, and maybe 24 hours to 7 for everyone. Yeah, oh, I didn't get my, I was trying to talk, and I did get my other one open.

51:23I don't remember what I was saying. Oh, I was talking about Peter here. I really like this call-out, but it says, other things where I liked 5.6, can't compare to Fable directly, like video editing is working now. It's not completely perfect, but like it's not a thing you can compare to fable because it's not what it's built for computer use oh no fable can do video editing i don't know it's just what he said here uh i i protest that point okay that's fair uh sub agents very fluent at managing sub agents and speaking to different threads existing code patterns multi-day runs that's a big thing i i a couple of the people last night said they had things running from 6 to 12 hours on their own.

52:15And token efficiency. It is so much more token efficient and faster than 5.5. And it's faster than cable 2. On the downside, this is really good. I ought to share this. I'll make sure we include these in the article that goes up after this. But I've got a couple more of these I wanted to run through real quick that are people we follow occasionally. Ethan Mollick, if you don't know who Ethan Mollick is, you probably should. He's a professor at Wharton School of Business and has been a very early adopter of AI among business people and really always has a good take. Early tester of 5.6 Soul was asked not to share demos until after launch.

52:57It says it is of similar ability but quite different feel than Fable. Fable wants to go off to its own work. Soul is faster but works with you in steps more. uh dan shipper uh said five six is a porsche fable is a warp drive been testing internally at every for about a month we had dan on a while back he's a nice guy uh five six the best combination of power speed and performance for your day-to-day knowledge work fable is a different beast if you need to get across the galaxy use fable if you need to get around town and best available tool for the job use five six i thought that was a really good uh i think it just kind of reiterates the pattern that we've seen recently which is Fable is really really good at setting up your projects and then you switch to GPT and go to GPT to build them.

53:46Yeah. Yeah. Yeah that's kind of the thing I've been hearing as well is that that seems to be the scuttle of late is that let Fable plan. And maybe some of that is just the fact that it may not be here forever at a price humans can afford. Maybe the idea is plan a lot, save them and go let chat gpt build them later uh maybe the idea too but i i think there's maybe a little of both of that there so i haven't seen any official announcement about this whole app changeover thing which is really weird someone else in the chat said they have chat gpt atlas now supposedly they're supposed to make atlas codecs and chat gpt all in one app i i picked up a used mac this week and finally have access to atlas and i am in love please I honestly don't use it, really.

54:36What have you been using it to do? Well, I've used it for a grand total of one evening for two hours, so I don't have, like, a full review for you here. But what I will say is that it's simple. It's clean. It's an easy-to-use interface. It's a web browser. All of my Chrome extensions, including the ones I built, just imported and worked. I really loved the ability to say to just pop open the window and be like I don't understand this and it explains it or I can say something like how do you pronounce that and it does it over here without me having to go open a Google tab and ask a Google translate window how to do something or whatever there's a lot more than that I just updated Atlas and it does not have 5.6.

55:23Yeah I'm not seeing anything on my web version on this computer. I'm going to go check actually my app on the other computer. Something else I think we should talk about that's probably really important here is Sol being used to post-train Luna. For anyone who doesn't know what post-training is, think of training like this. I'm going to put this simply so it makes sense. I have it. I have it. I'll pull up my screen, but you keep talking. You keep talking. Okay.

55:57Pre-training is where your model gets its knowledge. That is the training they do early on to build up its knowledge base, and it's a massive amount of training. It's a ton of data, and from there it moves over, and you go into post-training. Post-training is where you teach it behavior. That's where it learns to do the things you need it to do as far as how you want it to act in responses, how you want it to speak, how you don't want it to speak. That all happens in post-training. And apparently, Sol did that for Luna. They used the word autonomously, which I thought was interesting, and I'd love to learn more about.

56:42So, yeah, I think hopefully we can get someone from their team to talk to us on the podcast about it. Essentially, what I would imagine is that that was an LLM as a judge kind of situation where Sol is reading the... I mean, perhaps they put it in the report and that was from last week, I can't remember. But yeah, Sol would have been reading the results and grading them in that sort of situation if that's what happened. Yeah. Which is really wild if you think about it. It is. It is. And I'm curious to see Tara. that's kind of the daily driver, you know, your standard thinking version. And then soul and soul ultra will supposedly be available in codex.

57:30So Corey, I have the prompts written here. Oh, you have from you. When you have access to. When you have access to codex, we'll switch where all tests Fable and you test codex. but for now I'm going to pull them up side screen and we can test them there. Okay. Good question, Aaron.

58:00We'll have to ask ChatGPT's handlers, because I guess it's not accurate to call them trainers anymore. But we'll have to ask GBT 5.6's handlers how exactly that works. All right. So I'm gonna go ahead, if you can pull me up from being backstage, I'll pull these up.

58:30Alright.

58:39Oh, wait, actually, before I do this, let me pull up the prompts. Get those set up. Okay. Yeah. What else was there that I wanted to talk? The thing I, what I was, the point I wanted to get to earlier and missed was, I think what OpenAI has done really well is practical stuff. Things that work for real people. um like while yes there's been a lot of focus on high-end science on on edge mathematics and and and extreme coding use edge cases even i like that they've done a good job in my opinion of always having little odds and ends that make work life a little bit easier that that make using it in your normal world a little easier like not just not just for work not just for building but But, you know, like going outside and using it in this for, you know, for hiking trails or for things like that.

59:36I think that's really cool. And I love that they brought the farmer in because that's like, you know, you think it's the most analog job in the world. But the truth is they've adapted pretty quick. Yeah. Farmers had robots quick. It was so funny how fast Jensen Wong was selling GPUs to tractor people. And suddenly you've got lasers firing off of tractors, shooting out weeds. It's pretty crazy. but I'm gonna let Grant do his thing here so which one should which one would be the accurate test against Fable would it be Terra or would it be Luna which one's the stronger of the two it's gonna be Soul against Fable we do not have Soul yet we have Terra and we have Luna however they do say Terra beats beats Fable on a number of things not all things but that Soul is very much the like, here's the granddaddy.

1:00:31Okay. Well, let's do this head-to-head here. What we have here is we have Fable 5 Medium and we have Terra 5.6 and we can set it to Medium or we could keep it at extra high. What do you think would be a more fair test? I don't know. I don't know. I don't know how you compare them. Well, the way that I would think about it is you could think about the effort as how much longer it works for and you can think about the model as like the size like how much it knows basically yeah I don't know somebody tell us I feel like a little bit apples and oranges but what do you want to see do you want to see a direct x high head to head I'd love if you'd just throw a prompt or two in 5-6 to start so we could at least kind of see what it let's do it you know how it responds what it um sure so what i'm gonna do how it performed is i'm gonna copy this prompt so this is i'm giving you a messy product idea turn it into a working app and um we are gonna write this together here and i'm just gonna repeat whatever you tell me so whatever you want for an app i will I will literally talk it into existence

1:01:55let's see we need a minute, it wants your microphone

1:02:02what do you got? great way to max out your credit, oh me yeah, you or anyone in the chat I will just repeat whatever is heard this is so tricky to just come up with on the spot I have a backup if you don't want to come up with something Messy product idea. Yeah, I don't know. I don't know. Okay. So why don't we do this? So instead of that, unless people have ideas in the chat, please sound off. I'm going to go to fable. I'm going to go to something that I've actually already built here. Let's see what would be good.

1:02:40How about... Somebody said fable medium, tarot high. Another one asked how effort solely based on time or is that an approximation? So I'll do this one. What we're going to do is we're going to

1:02:59put this in here. These are messy ideas for a game called World of Squirrel Craft. It's amazing. We may or may not have spent an entire evening. so they said effort high okay yeah they said effort high i don't know i'm not sure what we get out of it when we look you know they may compare they may not

1:03:29you know what that's probably not the right place to start that is i'm gonna go to what I originally asked Fable for. Yeah, over here. Okay.

1:03:42Boom. Invest$10 ,000 in stocks. Highest return in six months wins. We can't test that live, unfortunately, but that's a good thing that we should just run in general. Farm ERP app. Okay, so here's what we're doing instead. We are going to do basically a 3D browser-based MMO like RuneScape, but inspired by the book Dune. So we're going to call it DuneScape. So let's try that. So we're testing Fable 5 High, and we're testing that against Terra Extra High. I would expect Fable to smoke at this by comparison, probably. All right. We'll see. Just based on the demos I've seen and how it is with lights and things and shadows looked really clean.

1:04:37Yeah. We should also do a website. Let's start brainstorming in the chat for a website idea because Fables really, or sorry, Tara and the like are really good at websites. Yeah. Yeah. So I'm just going to go back and answer a couple questions in the chat. When would they be released to us? They should be coming out over the next 24 hours. you should have access to it. A live emoji reactor for live videos working in real time. That's a cool one. I like next Flappy Bird big game hit. Just that and just see what they both do. Yeah. Okay. That's awesome. I like leaving it a little ambiguous. I say that.

1:05:25It depends on what you're doing. Sometimes you want that specificity. i would georgie i don't have the answer but if we did i'd have it too last four make a third person souls like in dark fantasy setting where you have a small location with one mob type skeleton and a huge skeleton boss at the end of this location i like that idea we'll run that next um all right i'm gonna let that cook what other thoughts do we have on this launch. So as you can see right now, I'm working in ChatGPT work. I wonder if it makes a difference if I'm working in Codex or work. Yeah, it might make a difference. Let me run this again in another chat window.

1:06:16And I'm going to actually say...

1:06:20Grant, is the Grant's computer window different than the screen share I'm looking at? No, the Grant's computer is the non-existent person who would be talking on Grant's computer. Got it. Yeah. Making sure I was trying to see if there was a way to take the third one back. Yeah. But I think it would move your screen back, and I don't want to do that. Yeah. I could switch to that one and just talk from there. No, it's fine. It might be a little messy. Okay. so we are testing 5.6 terra at extra high against fable five high we think this is an appropriate matchup between the two but who knows actually and we're going to give it a shot we shall see yeah yeah and the truth is we also won't know when we see the results other than that was fast okay so we got something for fable five great let's enter dunescape doesn't work squirrel craft doesn't work okay we'll give it some time but first to the app are we sure it's are we sure that is that is from fable i know it's sitting over it but it looks like a totally separate window no this is this is definitely fable okay okay i thought it was a separate window on top of those so i wasn't sure yeah what i might be able to i don't know if you noticed but codex is over there saying okay the starters dependency was incomplete i'm repairing that before i build the product they're like this guy doesn't know what he's talking about i gotta fix his crap uh yes i did reinstall codex goodly um let's see yeah make a paid version to generate said stock money i mean you gotta believe that all of the big um yeah finance firms already have whatever version of that wouldn't you think so yeah it's day one yeah yeah uh at minimum there are definitely lots and lots of models running uh you know i mean i think the way you would do that is not run one model and ask it what it thinks i think what you would do is run many models and take those results and aggregate them together and see see where it lands so this is an interesting question so does chat gpt have access to your computer files so with codex yes it does and i guess now with the new chat gpt work codex combination app uh in theory chat gpt does um but you wouldn't you wouldn't necessarily have codex communicate with chat gpt i guess you could you could have multiple models talk to each other using the API, but, I mean, you probably just want to talk just directly to ChatGPT.

1:09:10Grant, what does your open inbox say up there? In Codex? Right above your cursor? Oh, that's my cursor. Never mind. Over the right, click that little down arrow. Oh, this is in open in cursor, you can open in Zed, terminal, ghosty. I was wondering if that was showing you a terminal, or if that was like, open it in browser? No, this is toggle the bottom panel. So this is you can see, you know, whatever there. Okay, cool. It's also needing you to allow things. Do you want to allow downloading the starters missing web app dependencies?

1:09:54Deny. Deny stops the test then. Oh, you're right. That's a good point. I'll say sorry go ahead and do that I was like it's gotta be able to and I'm not sure if Claude has any or do you have Claude fully unlocked I have an auto mode and this is actually what the developers of Claude recommend is to do everything in auto mode and then the only time it'll ask you for permission it will be when things are actually important for you to review it's not doing that It misunderstood what you meant. Okay, now it got it. Now it got it. It had to go through twice. Now I see it. Yeah, we'll try that.

1:10:48Okay. You know what? I remove pair. Looks like it's putting it straight in your codex runtime, too. I'm just going to not, it's going to remove something. I'm not going to do that. Okay. I don't think it needs to do what it's trying to do. There. It was trying to run the dependencies it needed to build the game. No, no, no. I already gave it permission to do that. Then it was trying to remove, I think, the old run or something. So I said, that's not important. Don't do that. Yeah.

1:11:25Now the beauty of both of these models is we can actually start a new task if we want to. Do we want to try task number two and have that going in the background or should we settle our ADHD on this format? We have something. I have a little pill guy. You have a pill guy. I have a pill guy. I can click and kind of move the camera a little bit or no, that's not me, I think that's him that might be Claude playing yeah, that's Claude moving, okay yeah, testing it so this is Doonescape on the left hand side here oh, it was redoing the dependency repair is what it was doing okay yeah, you may have to restart that this one over here yeah or maybe do some do a different test because it's it's it's gonna have a lot of those why don't you go ahead and just build the game with what you've got

1:12:38just try to steer it back onto the right path here my my original prompt could have uh made it run astray yeah well and i think i think telling it no once and then telling it yes and then telling it not to repair after it said it needed to repair has probably made that run a little so uh cj no other tasks do not get in way in the first task being completed uh the only thing that matters there is what's called rate limits and both companies have them So if I go to my settings, let's see. You have a specific limit. So as you can see here on the left-hand side, I have a session limit for five hours.

1:13:25Then I have weekly limits. So this is all models. And then you have a Fable specific limit. And this is only while Fable is in the subscription app. and fun fact, apparently it will be eventually allowed in the subscription app once, I guess, usage or once their compute issues go away, but who knows how long that'll be. Yeah, I'm having to reinstall Codex on this computer. It showed like it wasn't there. This isn't my normal computer, though, so I mean, there is an off chance. I'm going to give this one more shot over here. I'm going to do a new task, and I'm going to try this again. And I'm going to take out the language that might be confusing it.

1:14:10Yeah, and I think if it wants to do something, if we're going to head to head them, it's got to be able to do it.

1:14:25Good to see so many people in the chat. Usually I use ChadGPT to plan architecture for my game, generate good prompts to codex. I benefited because I didn't use Codex tokens while communicating with GPT. Yeah, I kind of... Honestly, I use my Codex straight through my... I have a Pro subscription. The Pro Lite, the$100 one.

1:14:47And it just runs on my subscription. The truth is, I use it constantly and still have very little... Is Terra still the highest one available? Yeah, at the moment. Okay. Yeah. I was just curious. I could try Ultra. Well, we'll try Ultra on a different prompt. I'll just run that for now. Okay, so this way, it looks like over here. It's out, maybe. What? Somebody asked if it was out or if we were just testing earlier. I said it is out. But not everybody has it yet. We are normies like you. We are of the people. We don't get special access to things. We are among the permanent underclass. Yeah.

1:15:29That's funny. usually use chat to be to play an architecture for my game yeah exactly that so that's what i did here so what you're not seeing is that i have a prompt that i created that is optimized for both fable and gbt 5.6 so you know maybe it's not perfect for either of them but uh i give it a very quick overview of what we're going to do then i say here's my messy idea and in this case i wanted it to create what's called a prd which is a product i might get this wrong the product requirement document. And originally I wanted it to create a PRD, but then it could go off and start coding for these two projects.

1:16:06But in this case, I just stripped it down. I said, okay, just go ahead and make the thing. And then I gave it a goal, build the simplest usable version. Then I give it a list of tasks. I give it constraints. I give it verification. And then I gave it the final answer right here. And so I did the same thing for Fable. The only difference is Fable was able to work through the nuance of my original version because it's a little bit, I think probably Sol would be able to do the same if we gave this to Sol. We just don't have access to Sol yet. And it was able to say, okay, well, I'm just going to build one and then build the next one.

1:16:40We had to restart Terra a couple times to try to get it to do it, but that could be user error on my side. I would say this test is still, you know, they're still tied at the moment. Yeah. I wouldn't say this is a win for Fable just yet. And I'm assuming, you know, ChatGPT is going to be a little bit behind here. And I say that, you never know how long it will take. Fable is still working as well, right? Yeah, so it looks like we have playable versions. So if I put my name in Grant. Let's check it out. Dunescape or Squirrel. Oh, let me refresh this. Hold on. Oh, yeah. You may still be getting that dud window.

1:17:21Okay, so this is a good first piece of feedback, I'm going to say. Yeah. It's done but doesn't work? I can't actually access it from that index.html that you've created. Excellent. I mean, not excellent, but I just mean at least we got something back. Totally. I'll make sure you see what Tara will do because Tara is like the middle of the road model. That's your daily driver standard thinker, I think, is the way to think of Tara from what I've read. And think of Sol as like their pro. So we might be able to give, you know, for maybe a more fair test then, we might be able to give Terra the PRD for one of these.

1:18:08Baby's like, oh, my God, I'm about to try it out. I'm so giddy. So I'm going to give it this one. Did your project chats from ChatGPT app move to the new GPT Codex app? Oh, this is a good question. So right down here I see tasks. Let me see what happens if I switch to work.

1:18:29I see schedules, I see pull-ins, I see sites. We will test sites in a minute. I see chat, but I think it's all new. Well actually no. Oh yeah.

1:18:47This is funny. You can see my like raw thoughts here because this is my favorite. personal chat gpt account it looks like so far none of my yeah i think my chats came over yeah if you go so if you in the chat in the new app if you go to chat gpt work and then you go to chat you can see your previous history here

1:19:11go into chat and then you can see your history but here's the thing um is are regular people going to be able to understand Shadgpd work versus codecs? I don't know. If they try it. So far, honestly, the truth is, normal people haven't really adapted much to the desktop apps by the AI companies. Yeah. Like, as far as the daily way to use it. I mean, from everything I've seen, most people are still going in a browser. Yeah. I mean, even I do for most work tasks on my work computer. It depends. I use codecs for writing. I use codecs for building. I use codecs for planning big things. But as far as my day-to-day, like, I'm working on this little thing.

1:19:55Just help me figure this out. That's what I go to the web app for. Mobile, I go to, honestly, almost exclusively for voice with ChatGPT. I use their voice a lot. Yeah, I spent an hour and a half last night, Grant, talking to the new one, and it's bonkers. like it does things that are hard to get used to like while you're talking it's going uh-huh yeah oh okay yeah that's what the other thing is johnny five but can you get it to be quiet yeah yeah that's fair uh yeah and you can you know what i did i was going to order fried chicken and i pulled up the window and i was like hey can you just be quiet for a few minutes while i ordered chicken he said yeah just just holler when it's time to come back just say just actually what he said was just say come back when you want me to come back so i just sat my phone down i ordered chicken and i said come back and he said that sounds good about my chicken uh but what was weird was his personality is different johnny five who was usually like super up and energetic and chipper was very like like i found him at 1 a.m in a jazz bar laid back now is very very different um very conversational has more access to tools than he ever has which is great uh okay or fried chicken life is i agree controls on this okay oh it's click to move i get it now okay okay okay so this is dune scapes access now but only when I select work.

1:21:33I wonder if there's a sandworm in here somewhere.

1:21:42Interesting. Okay. I'm in that phase where I'm constantly crashing and restarting codecs. Thoroughly annoyed. Oh, you should already read this soul ultra access? Okay. But only when you select work. I'll do that. I'll test that. Okay. Do work. Yeah cause like If we're gonna head to head on what I really wanna see Is Fable Versus Yeah same Bring me your best It's like what I don't wanna watch You know Mike Tyson fight a Middleweight Well you know Netflix did have that happen once That is a thing Yeah that is true I completely forgot that That's exactly what that was. I wonder if I have to... Okay, so I don't have it in work yet.

1:22:33To be fair, 55-year-old Mike Tyson ain't a heavyweight either. Right, that's fair. I mean, maybe by weight, but... Yeah, even when I select ultra, it's not there. Well, let's go back to the prompts here. So in this case, we have a broken code base refactor, right? So what we're going to do... So I'm going to take this. Okay, I'm waiting for codex to open. and it's slow. Am I getting the new codex?

1:23:02Let's start a new Fable session.

1:23:08By the way, I've been using a lot of Claude lately, and I have a lot of tips for people. One of the things that I love to do is set up... So here's the way that I'm doing it at the moment. um what i'm doing is i am using auto mode right so auto mode is the way to go then i'm using it locally so i'm starting local and then what i'll do is i will do remote control and then on my phone i can check on the tasks that are running on my computer at any time so that's the ideal way to to start a project is you do auto mode local and then backslash remote control Codex CLI has it. I have Codex CLI on here, I think.

1:23:57I'm going to say, you know what I'm going to do? I'm going to tell Fable, please review my GitHub and identify the most broken code base and give me a report on what you would change to improve it before you begin the below. Let's see if it can figure out, you know, I'm giving it a lot of instructions here. Let's see if it can do it. And it looks like Terra, we still only have Terra over here. Maybe I should close this and restart it real fast just to confirm. By the way, you can see down here, the Codex app is now missing and now it's all just ChatGPT. I don't have it in CLI yet either. Dang it.

1:24:49go ahead and quit now if you do quit the app that does mess up the local tasks that you're working on so if people don't know there's local tasks and you can do it over the cloud alright still no soul but do you have to be Corey do you have to be on a certain plan to have soul actually I don't think so paid plans I think is what they said which is pretty cool frankly they got some cheap paid plans alright so I'm going to go back to Terra I'm going to do extra hot actually why don't I do ultra this time I'm going to do ultra for Terra give it the best shot possible on goal and let's send it this way and Fable we're going to keep it at high I think that's enough I'm going to look at some reactions online and see what people are saying Dan Shipper's using stone because of course you can ship your head back.

1:25:46Just get it. Volkov, MTS. Got it in CLI on a plus. Okay. A new agent in chat GPT. Chat GPT work is a new agent in chat GPT powered by codex and GPT 5.6. A new agent in chat GPT powered by codex and 5.6. That's hard to understand. Wait, are you saying that that's chat GPT work? Yeah, ChatGPT Work, a new agent in ChatGPT powered by Codex and GPT 5.6. Well, because Codex is like the harness and 5.6 is the model. Makes sense. Yeah. And honestly, I kind of like having them together. Partially because I regularly wish I could access chats from one in the other. I think someone asked here is that I think is a good question was it's gone I don't see it now they were asking if Codex yeah does chat GPT work use the same usage as Codex most likely yes my chat GPT and Codex use the same subscription I pay once and use both tools and they tap into the same usage rates do they use credits though because member of workspace agents and codex apparently have different seats and credit types for team accounts on on team accounts that is different than it is on individual accounts uh but so far i mean i paid the one subscription and i've never run out i have four full month rate resets with codex i can still use as well yeah so it looks like so i'm going to read a couple comments in the chat real quick ish ish the fish says i am on the pro plan and have access to 5.6 soul ultra but you're on the pro plan cory and you don't have it right i'm on the pro plan and i thus far do not have it i am refreshing one more time on three different computers big iron dot one three one one also says they don't have it on the pro plan here's you cannot Not by early access.

1:27:57I apologize to the user with the Russian characters. I don't know how to pronounce their name, but I'm on the pro plan and I don't have the update. Sorry. Yeah. Yeah. It's going to do this thing where it comes at like 11 PM for me. Yeah. I'm never early. I don't know if there's a reason. I never tend to get first access and I don't know if it's the order in which they roll these out. Maybe it's something to do with, you know, account numbers or last names or usage, or maybe they're, you know, tackling this level of user, this level of user or something. Yeah. Let's take a poll in the chat. We had a cool game idea earlier, but for the next three prompts that we're going to run, let's do a what is a project that we want to have Ultra Mode as a project manager use subagents for.

1:28:50So if anyone has project ideas, let me know. Then what is an annoying weekly workflow that you'd like to automate? So shout out in the chat if you have an annoying weekly workflow. I have one that I've been working on that I could show on my other computer. And then obviously we're doing this right now. Sol versus Terra versus Luna versus Fable head to head. And we have this task. Maybe we could do one of these for some of those other ideas that people have shouted out, which would be fun.

1:29:22yeah here's what's been one thing that's been different is in codex has had work mode so you can click for working and for everyday work for a while right and with those i haven't noticed much of a difference i'm switching between them now in an effort to see are different models available for each setting uh oh thank you lone wonder appreciate that that's cool oh good to know yeah those are such cool such cool writing though i have i have looking at it i i don't know much but i've always thought it was a really cool looking language yeah i actually want to learn it um i'm very very interested in it um okay john lh4 lc says reddit says that you switch your model on the cli and it switches it on the codex app too i went to slash model in the cli and it didn't show up yet.

1:30:15What if you just type it? What if you type gbt slash model gbt 5.6 sol? Do you think it'll set it? I don't know. By the way, for people who don't know what the CLI is, it's the little terminal. So on Mac, I use ghosty. Oh, this is something else I built. Maybe this is the one that wants to fix. Let's see here. Oh. No, I'm not seeing anything yet. So I just have to watch Grant build things in Codex. Yeah. I'm so sad. I wanted to play. Codex is doing stuff. Yeah. It's just not super fast, but it was on Ultra Mode, so it's got all these sub-agents running. That's the thing. Ultra Mode uses, like, a team of, what was it, 16 agents, Corey?

1:31:08Mm-hmm. Yeah, it's pretty wild. If you've ever used Grok Expert or Heavy, I forget which one, it does that where it fans it out into a bunch of sub-agents. Yeah. I just saw the point. The truth is on this computer, I haven't updated Codex and Ages, which is probably stupid. I mean, the CLI, I don't use it much on this particular machine. Hmm. Can you update it maybe? Yeah, that's what I'm looking at. They just called that out and I'm like, yeah, that's a good idea. I should have immediately gone to that in my head for whatever reason. Okay. Did anyone have any ideas? No. Okay. Why don't I just brainstorm some ideas?

1:31:51So let me test this whole thing over here. So I'm on chat GPT work now and I'm going to full screen this and let's do a new, um, supposedly you can do a new chat in the sidebar here. Yeah, let's do this. Okay, so let's say not instant. Oh, you can only do GPT 5.5. Let's do medium. And I'll throw in these ideas.

1:32:20Give me some fake. I'm going to say give me some fake idea pitches to test for these prompts that can be reasonably accomplished in a short period of time. And then I'm going to paste that in and then I'm going to say, give me three to five for each, pitch them as one word or one sentence. Ideas, a bullet point list, and then we'll write the full prompts for the ones we pick. When you're done, would you open another window? Yeah. What do I open? ChatGPT. Task. ChatGPT. Task? New task? New task, yeah. A new chat, yeah. Ask it

1:33:20to write a 300-word short story without using the letter E. And you want me to do Terra? I'm not going to do Ultra. Okay, I was going to say we could do Ultra. I don't know that it'll do it. Well, I might get rate limited. I'm not on the fancy plan. That's true. Grant loses his work AI for the month. No, no, no, no. This is my personal account, but it's the$20 one right now. So everything I'm doing right now can be done with the$20 plan. Create a 300-word short story without using the letter E.

1:34:02speaking lead speak hold on hold on hold on the letter three apologize for snacking a little bit here it's that time of day a little sugar just mute yourself when you're crunching we're good okay it's a hergy bar it's delicate well it crinkles in the microphone I asked Sol what the coolest thing I could do right now is and created a full lead magnet for my LLC Hell yeah! That's awesome, Eric Eric Marias2699 That's really cool Gito Myko says What is the difference between Sol, Terra, and Luna? Let's see, actually

1:34:43Oh, crap

1:34:50Yeah, there was rumors floating last night that they may still drop GPT six this month. They were planning to run all the way through the fives, but.

1:35:05I need to find that. Grant asked me for the source and I forgot to grab it. Which one? Which source was this? The one talking about GPT six last night that it could drop in July, actually. Yeah. yeah someone put it well big medium and small that's that's right but i want to i want to pull um i want to test the models and see what they say of course this is so overkill to do extra high for this my goodness what's the default terra light okay i'll do that you know i was tasks last night working with five five i just kept thinking you know i was thinking about there's a new model coming today and that's awesome and then i was like but what do i really not able to do right now that I want to do and the truth is there's not a ton I can tell you for me but I'm curious what you think try to take over the world pinky no for me it's a lot of the things that they tried to solve today computer use and really visual like I am I'm not an artist right that's not my domain I'm not into visual art really, other than, you know, I know good art when I see it.

1:36:19So I would really like help in that area when I'm prototyping things so that they don't look so ugly. So I'm more focused on like the full game feel, the full, you know, what does the website look like? What's the user experience of the design? I'm not good at the design portion of that. So I would like more support in that area. That's cool. And I also think that's the most fun way to collaborate with these types of tools because you can test so many different variations of things and see what sticks. You know what I've started doing that's different? What? When I have an idea for something I want to build, I kind of write up what I want it to be and what I want it to look like.

1:36:57And I go to chat GPT image and I'm like, mock this up for me. And then when I get the design kind of how I want it, I go to a color palette website and I sift through color palettes for an hour. and I find one that I'm like that's the one and I bring it and then I have it put it on that image and then I take that image and what I want over to Codex and say I want this I want it to look like that using these colors this is kind of a good example of the layout I'm looking at and uh and I have found it to be really helpful at getting like good answers design for that same idea I've been I've been really liking this tool let me give you an example um here's one that I did that's a preview of something that we haven't talked about yet, but maybe we can plug after this.

1:37:39So I asked it, you know, for a very basic example, I said, design a webpage for the Neuron AI newsletter. And this is like a new redesign, which by the way, if y 'all like this, maybe we just add some cats on it and we could pitch switching the brand colors. I don't know myself personally, but I like how clean it is. Speaking of getting smarter every day, I don't know that we've ever talked about it here but the Neuron just launched something very new this week we launched the Neuron Academy which is you can see it at the neuronacademy.com it's essentially a good place to learn if you're a beginner if you don't know where to start and what you're doing we have tons of courses we've built out and tons more to come that we are building out every day week and as are some others helping us and working with us on them um it's it's open now and i think it's set right now at 4.99 a year um but you get certificates for every course you get uh some of them are long courses you know if you want to dive into like you know a full understanding of prompting or agents or something there's some medium-sized courses there are also a good number of quick hits if you're a busy person and just like you You know what, I got 20 minutes, but I want to learn something today.

1:38:55We got those two. And we've really put, we've been working on it for five months? Almost six, five, four, four heavily. Yeah, I think four is probably more accurate. Like, really kicked off March of this year. So March, April, May, June. But it is live and available to be seen. And we'd love to hear what you think. Okay, check this out. 7 % usage remaining. Wow. You should smash that codex refresh button and see if you have access to it, Corey. Okay, so I want to go back in is what you're saying, yeah. I want to go back in and close the loop on a couple things that people asked. Number one, what is the difference between Luna, Terra, and Sol?

1:39:45So this is according to this codex build that I'm using right now. Codex went and checked the docs. And it says, Luna is the fast, lower cost agentic coding tool, best for quick iterations and straightforward tasks. Terra is the balanced everyday agentic coding, better defaults when you want reliability without spending maximum effort. But it also supports ultra reasoning mode that can delegate parts of a task. Sol is not present in this build models catalog and OpenAI hasn't published a public spec for it. That's a lie. It has. So it just didn't use web search to look that up. But just FYI, soul is like the big big baddie you can kind of think of it as uh fable but for yeah codex codex's version so i'm gonna go mythos even yeah i'm gonna see here tasks where do i actually see the the task that's working right now it should be at the top of your list it is not which is better guys i'm new here claude or gpt-56 stole well that's what we're trying to figure out right now but what we sort of stumbled into one of the tricks to doing these on a on a launch day is that like they're rolling out a ton of things they hit snags things get you know a little get a little early it's early to say i think that is a very good response it's it's still a little early to say uh yeah benchmarks say it depends on the benchmark yeah you know depending on the benchmark you choose to look at and what it is that you need your model to do i think something that's really clear though is that we've really entered a phase where the interface tool availability and integrations can make a model that seemed pretty average an awful lot better I really think that you know the idea of of a model alone being the you know the determiner of is it good is it the best is probably a thing that is going to pass at some point like I think the question is going to be how can I use it and I think that's an era where that's kind of what I like about Codex.

1:42:08Fable's great. Everything I've seen about Fable is impressive. That aside from the whole people getting kicked out and booted to 4.8, that would annoy me. But yeah, it's really too early to say. I have heard it's significantly more efficient than 5.5 on tokens, so we'll see.

1:42:33Well, I've got 6 % usage yet. I don't know if I have any banked resets. And, man, I don't have it anywhere yet. I am still entirely six-less. So let me go ahead. I want to test the website tool here because I think this is pretty cool. Artificial analysis is out. There we go. That would be fun to look at. Yeah, if you want to pull that up. Grant, have you played with sites yet? really not really have you no no because it's on my work computer and like most of that kind of stuff i do at night yeah and it just isn't a thing i've had the time to sink in during the work day well you should have access to it now um somebody had an idea in the chat for uh the website earlier what was it shout out if you have an idea for a website we're going to test

1:43:32where was that it was pretty good analysis intelligence index you want me to share this up why don't you do that while i'm looking at this and then i'll we're gonna have one or two good honest tries left in my terra subscription here we're gonna pro tear through some things uh

1:43:53Um, okay. Solmax at 59 on the Intelligence Index. Solmax. I assume that means they're ultra. I assume they're just calling it max probably for, so it makes sense on the page. But I don't want to say that until I know that. But Claude Fable with fallback, score to 60 on the Intelligence Index. Terra is, there's Terra. There's 5-5.

1:44:24Grok 4-5 sitting there at number 5-6. That dropped yesterday. That's pretty impressive.

1:44:34Let's see. Okay, let's go over here. Coding agent index. Let's see how they score there. Neck and neck. 76-77.

1:44:49which is good and that is a composite composite average across deep sweet terminal bench v2 sweet atlas q a cost uh let's see here where is five six isn't on the cost chart yet execution time it's not on that either I think at launch they oftentimes only run them through the raw intelligence score at least that seems to be the case sometimes it's very expensive right for them to do that yeah these aren't cheap to run wait some of these aren't good a fake looks maxing website that tells you how you beat your face with a hammer

1:45:41wow we got some unhinged ideas in the chat here. I love it. Unhinged is perfectly fine. Okay, so GDPVal has Fable on top by 163. Yeah, I'll try that, John. Sorry, I didn't know what you were saying earlier. I'll try the extension. Let me do that right now.

1:46:13Yeah, I think max has to be ultra. I think they're just using that interchangeably there.

1:46:24Yeah, like the max power kind of thing. Some of these benchmarks I've never heard of. There are always a number in here now that it's like, what? Okay. Output tokens. Let's check that out. Output tokens per intelligence index task. ooh hey that's intriguing what'd you find out? on the output tokens per intelligence index task which is the amount of intelligence to your dollars 5.6 soul max is way down here on the good end which surprises me they must really optimize performance. Look at that whole what was it? Codex Golf thing? It shows 15k. Fable is very reasoning heavy on it too. It's got an 8k answer but 25k reasoning.

1:47:31KGBT is a pretty even split at 5 and 10. I just hate extensions in VS Code. They're so difficult to just be like, okay now how do I open it and use it once it's installed. Oh my gosh. It's such a user-friendly system. It's unreal. I'm trying to help Corey see if he might have it in VS Code. Yeah, I mean, I have it in... I have the app in VS Code. I just can't open the fucking app. Sorry. I don't have VS on this machine. Hang on. Oh, please. That's funny. My guess is that one of my downstairs machines might be a little more friendly. Like the one that I've got my VS Code set up in is down there.

1:48:19Let's see.

1:48:23Refresh. Dang you.

1:48:30FOMO, man. I have FOMO over here. I am sad. This is disappointing. They should quit being mean to me.

1:48:43it'll come around later I'm just whining okay I'm going to open VS Code

1:48:51yes I trust the authors now how I thought like the extension of Fable and then the possibility of an anthropic rate reset which is like not a thing that happens at least not at the frequency of the other is kind of an attempt to hang on while they're figuring things out here too because like like yeah they are being the best having the best model even by especially by a percent or two is one thing having the best model at more than double the price is a very different thing well because you asked what what is something that you can't do with the current models what i can't do right now is use it as much as i would you know like that's the thing i can't do right now i can't use it as like i can't use it for as long as i'm working like yeah that's a problem yeah yeah to me that is i have long felt that claude's biggest problem was the fact that the second you walk into an executive team and say hi we want to buy this new software for the company by the way there's this limit that kicks them out in five hour increments that they could hit in an hour they might not hit it at all they might hit it at three hours and then they just can't use their AI until we say so again.

1:50:04Like, I feel like that is a thing that as a business proposition is a hard no. Like, can you imagine if Salesforce was like, 20 % of the day, Salesforce isn't going to work for your team. We don't know when that 20 % of the day is. You'd be like, then I'm not using Salesforce. Or it's like every time your workforce, your salespeople use Salesforce, they're charged by the minute of how much they use Salesforce. You can use it. It's just$3 a minute from now on. You are pushing people to not want to use it because of how it's built. Yeah. But the subscription thing is not sustainable. Stems back to the compute buys two years ago.

1:50:47Yeah. I really think that all of that particular type of struggle for Anthropic stems back to that period where opening eye was buying these crazy amounts of compute. years in advance that weren't going to be around for two, three, four years. But they were buying them before Compute was so constrained. So they have Compute still to use for years that was bought quite reasonably by comparison. And it's left Anthropik kind of scrambling to meet demand when their product really took off at its best, I would say, this year. They're working on it. I mean, I've seen a number of deals in the last couple of months that they've made for some compute here and there.

1:51:35Okay, I want to give everyone an update. Okay, so I don't know if you want to pull my screen up or I'll just talk it through. Sure. I've tested sites against Claude Design. So, and I used someone's example from earlier in the chat. They wanted to make a Souls-like kind of game. and I already have the design on claw design, which makes sense because it's just doing the front end. Grant, I don't know where your thing is. Oh, do I need to share again? Let me share. By the way, I appreciate shout out to whoever was giving me the exact codex. Yeah, this is JohnLH4LC. The instructions are perfect. It's just not there.

1:52:17Like it just literally is not there. So it's just, I don't know what's going on. Let me share my screen again real fast. Can you pull it up from backstage? And I'll share it.

1:52:37That's the one I just took off stage. Because nothing was there. Yeah. Now I'm sharing. One moment. Slow over here. Already? It might be slow. Hope everybody's doing good. This has been a lot of fun today. It's always fun when a new model launch day comes, or frustrating. We've had some where it was like, hey, we got it immediately, and then we've had some where it was like, oh, we have nothing.

1:53:05Man, it is real slow to try and stream right now. I don't know if you're going to be able to see this. I see another Grant who is not in time with the Grant I'm looking at now. That's what I'm talking about, about being very laggy. it's probably because I'm trying to stream my video twice there we go that should resolve that

1:53:34this is where it's revealed to everyone that I have an evil twin cost per intelligence task 5 max $4 teramax 55 cents lunamax 21 cents fable two dollars and 75 cents wow yeah opus a dollar 80 still sonnet buck 53 that's crazy sonnet's not no who would use sonnet for anything for triple the cost of better models i don't know but the better model can do what it'll take Sonic like six runs to do. Like that's the thing. Like you have to look at the cost as, you know. Yeah. Cost is going to matter more over the next year. I think people will start as, as more and more people begin playing with agents and they realize that, Oh my God, this is a fortune.

1:54:33I ran 808 million tokens this week or something like that. Right, Grant. Right, Grant. Yes. Yeah. Yeah. That's a number. Wasn't it? 808 million. apparently Luna has a max level maybe not ultra code I think max just means the highest that it can run at yeah it may just mean the highest but it says on artificial intelligence it's showing GPT-5.6 Luna max so it probably doesn't have Luna ultra that to me seems kind of pointless before you would mow through it that quick I think you'd bump up but maybe I don't know what's the lowest? Deep Seek Flash down there holding up the happy end at two cents.

1:55:15Two cents versus two cents. This range, Grant, DeepSeek's latest model and Claude's latest model, DeepSeek is two cents per intelligence index task. Fable is$2.75. Whoa. And even when you get to that median in that 30 to 50 cent range where a few of them sit, or up around a dollar, There's a number of good ones that are buck. That's crazy, though. Two cents. The truth is that's a number that matters to a business. That's a thing that I think is going to be really tricky. I think at some point, industry will force the cost of this down even more. I don't know how that will happen. I think I do.

1:56:05They're going to create their own models and run them on their own computers. They may. They may. That's not cheap either, though. Yeah. Yeah. You're, you're, you're smart to remind me of that. Truly. That's, that's expensive too. It's all expensive. But I think the price of that has gone down. I will say like, you can rent servers that like a lot of the cloud providers for this stuff are pretty good and you can, I don't know the exact term, but like partition or, or yeah, you can, you can get your own servers that are yours that no one else is looking at and run the models there. like there's a lot of companies that do that base 10 um yeah fireworks there's a lot of them ocean uh digital ocean uh uh core weave you can go to you know uh you know you just jump the difference is that just like open ai has to buy gpus to run a big model so do you you're just not serving it to the to a billion humans yeah i think the the amount of um middlemen you can take out of the middle is probably going to save you costs.

1:57:12If you can take his OpenAI and Anthropic have to make money on top of whatever the cloud providers are charging, so in theory, you take them out of the equation and you go directly to the cloud providers, you're going to get a cheaper run in theory. But their models aren't as good yet as these guys. Corey, would you let my computer back in? Oh, did it leave? Yeah, I'm trying to refresh because it's real slow at the moment. I don't know if it's going to work this time. Oh, there it is. Let in. Yeah. All right. All right, all right, all right. Let's give this one more shot here.

1:58:03Let me know when you see me. Okay, it's looking good, looking promising in the infinite vortex.

1:58:14Okay, so here is a head-to-head website test of Claude Design versus Chachibichi Sites. now in theory sites is better because it automatically spins it up for you meaning you know design is just doing the front end and you have to do some more prompting and fiddling around to actually build the back end sites is in theory building both of them so um right now you can see oh by the way this is what i mean about the vs code like it just doesn't exist I have extensions. I even have it installed. There's no way to just click it and open it. Click this little auto-update. It just doesn't show up for me to actually use it over here.

1:59:04There's no way for me to get it to show up. It's weird. I don't get it. But that's why I don't use VS Code because it's a trash program compared to some of these other ones so um back to where you were so this is this is the dark souls game so and it's like it's you know not not playable the only thing is that i look kind of like a skeleton as well um yeah it's certainly not dark quality yeah what was that i think you just did a 360 degree loop underground these controls are also inverted so i'm pressing to go left and it's making me go right does anyone use inverted controls on purpose what and why does every mouse like default to that oh cory i think you're gonna like this one on the left hand side here oh or the right oh snap that's looking cool uh click to attack doesn't really work i can't attack but this is just a this is just supposed to be a front-end demo, right?

2:00:09That's kind of cool that it's able to make this just as design. So it's just the code that's in the design mode was able to make this. And this is a prompt I ran that was literally the one that someone shared earlier. I think it was Lonely Wanderer shared, you know, make me a Dark Souls-like game. Yes, it's a bit janky, but we do have something. It is a little Minecraft. Yeah, it's very Minecraft. That's actually a lot of Minecraft. That's what I'm talking about when you're using AI that can't see what you're seeing. A lot of times, if it could see better, it would probably design better stuff.

2:00:47Use an image model. I firmly believe if you take an image model and you create what you want in that way and then bring it over and you're like, I want the character to look like that, that you could get better results. I agree. And, you know, transparently, let's see, what's a good example? We have a question here too that I think.

2:01:14So what I've been doing, which I think is a really good pattern, is I start the project in Cloud Design. So I will, and you know, you can use codecs in the same way. It's a really long chat as you can see.

2:01:31Jeez Louise. Okay, so I gave it this pitch for what I wanted out of this project. So this is like my messy note stump from before. I put it in, and then it asks me a series of questions, and then I answer those questions. And then what I did, if you can see this, is I generated what I wanted the UI to look like with ChatGPT image. And then I gave this to Cloud Design, and I said, okay, now make the code for this. and I said, how would you update the UI to implement all of these features exactly as they appear here? And then it essentially went and it did that and it implemented all of my UI screens.

2:02:08And I did this for, I want to say, six or seven of them. So the flow is I'm generating the image in ChatGPT image in Codex. Then I take that image to Cloud Design with the repo that I've already started and then I have it implement the UI features that I want from there. I like that. I love the multi-model approach. I think that is the smart way.

2:02:33Sol is in Codex, if you run Codex update. I got an update on my other computer, and it didn't have Codex yet, or it didn't have Sol yet, or Terra, for that matter, either. And so far, I'm checking for updates again on this one real quick. So this is definitely more Dark Souls-esque. This fits the prompt. It definitely fits the Dark Souls question better. Yeah, but you know the game isn't going to look like that. No, that's a wicked image, though. That's sick. Let me see if I have Soul. If I do, we're going to run these prompts. Nope. The ChatGPT app is uploading, updating. That's a good sign. That's a good sign.

2:03:20It's not my codex. Well, my codex looks like it is too now. Okay.

2:03:27520 kilobytes downloaded. By the year 2046, this will be ready. Oh, man. Yeah. Oh, what time are we? Oh, we're 2 o 'clock. Yeah. Well, we were able to test two of the prompts head-to-head, of the five. So we'll have to come back and do this part two when we have Sol officially. Yeah, where it is more of a head-to-head. Yeah, maybe we'll do that next week for the live or something. Yeah, we might do that. We might do that. Cool. We'll see what happens. Looks like everybody's blowing out models this week. I don't know if you saw, but Zuck dropped a quality model this morning, a code model that puts Google in a solid fifth place, in my opinion, right now, which I think is intriguing.

2:04:18and I messed with Grok 4-5 for a little bit this morning, not long and it was pretty impressive too, I mean I'm not saying state of the art on either, but I'm saying like within slapping distance of state of the art for both of them you know, like it's not a thing that is outside of the realm of possibility for either of them in the near future So don't have it yet, but I can close this and I can restart it Let's try that I will mess up what I'm doing. My downloads hung up. I am so annoyed. What I'm going to do is I'm going to say can you spin up and run the site before I do that. So over here I had Fable working on a code base review.

2:05:09Looks like it edited some of my broken code. Looks like it picked. It's really hard to read. I know. Yeah. You mean just in general? Just in general? The flow there while it's sitting there is really hard. I don't know if it's their font choice or their lines are a little more close together. Something about it is it's also a lot of different styles in one area, it looks like. It may just be how it's coming through, though, too. it could just be my my guy who doesn't have glasses on eyes too that's really hard to read doesn't have glasses oh that's well i could read the other one that was the differences yeah so i'm gonna go over here i'm gonna open the browser i'm gonna put it in my local host import data from your browser oh that's nice not doing that right now so is there for new projects not for old oh that's weird we'll see i just tested that theory and i don't see it yet okay failed to load so i'll say it didn't load this is a good pattern for people it looks like it's still working though oh wait oh there is something here oh okay i'll say i got this error message so fix this whoa now let's test this out oh my god

2:06:48okay i can attack it i mean this is definitely not souls-esque but this is what i requested yeah and it's uh it is i mean like it's definitely a starting point Yeah. And remember, this is not Sol. Game's got a hell of a cover, though. How is it for coding versus Fable? So how could I describe what I've been doing right now? Not much difference between those two games, honestly. Yeah. Here's a direct head-to-head comparison from coding. So it's the same prompt, right? Oh, I forgot that one. I mean, the controls are inverted, so hard to use. It definitely does lighting and shadows better, Fable.

2:07:32Yeah. Like watch the floor. Watch the face turn colors when it spins, you know. Right, yeah. It's done that a little better. Versus this one over here. It's all 2D. Very static, very flat feeling. Sort of doesn't really know what it's looking at. But I haven't been doing computer use with it. So I feel like if I gave it computer use, it would be able to see this and be like, this is not good enough and would probably change it so that's so that's something i need to do and of course this is also once again just to keep mentioning it tara versus fable not soul which is which i think are very different things honestly even from everybody's opinions on it that i've seen comparing them to one another is a little iffy almost to begin with just because they're very different creatures in unique ways.

2:08:26Yeah. It looks like everyone who tried them had really positive opinions of both of them, actually. I've yet to see anyone say one of those is crap. Nobody's really said that about Fable. Nobody's said that about 5.6. Everybody seems to be like, this one is great at this. This one is great at that. That is how you should use them. Yeah. You should use all of them. risk. That's my button. Always allow Grant. Live a little. Live a little, brother. Live. I thought I gave it permission to do all this, but maybe because it updated, I have to do it again. I'm just razzing you to absolutely do what you're comfortable with.

2:09:05You should not let peer pressure allow your AI to take over your home.

2:09:12Their UI is very good at this, though. You're like, just drag this here and put it here. Yeah, just drag it on over. It'll be safe. You don't have to worry. That's actually very helpful. Okay. Maybe we're not going to do this right this second. Why don't I hide my screen? I'm going to lay one dollar on the fact that within an hour after this ends, I have all of them. Because that is how the world works. Yeah, probably.

2:09:44Well, everyone, thank you so much for joining us. This has been a lot of fun. It's been a lot of fun. We'll do another one of these like this soon. Remember to please go check out the Neuron Academy. We would love for you to check it out. We'll have a code for you here soon that will make sure you get half price, I believe, is what I was told. But we don't have it yet, and it's coming. Actually, you may be able to get the same price right now, but it won't be that way for long.

2:10:12Please take just a minute to like and subscribe. It means a lot to us. It lets us keep doing this and having a lot of fun. And I'm sorry that we weren't able to get access to the models we'd hoped to today. But I hope you still found it, you know, fun and interesting and entertaining. Maybe learn something. Yeah, I mean, Terra held its own. I wouldn't say it's totally out. No, I wouldn't either. I wouldn't either. It's funny, too. We're reaching this point where, like, it's like, oh, well, this one's way better. And it's like, oh, well, that feels like 5%. That's a solid 10%. I say that. The last three big models each from Anthropic and from ChatGPT, I felt were each a significant jump one after the other.

2:11:01Like this year, I feel like, and maybe some of that is that we're using them in more agentic ways and it feels bigger. I'd like to investigate that a little bit that's a curious thought I think the more capability that there is, the more that you push the boundary and you change the frame and by changing the frame, obviously you start to find its new weaknesses, but you also start to see it do things that just wasn't possible before and sometimes the question is is that the model, is that the harness? sometimes it's the user there's a difference even at the user level And I think we're at a point where we're seeing some pretty amazing stuff.

2:11:46Like, there's just not a bad one right now, in my opinion. And that's a really nice place to be because there sure used to be a lot of them. When was the last time you worried about hallucinations, Grant? Whenever I have to use something other than opus or fable. Literally any time other than that. Like I keep an eye, but I don't really think about it as much as I used to. It's not as big a deal. I'm still more hung up on the X, not X, not X, it's Y argument. That's the one thing that has not yet been like hammered out of the world. Expanding that a little bit more. What I would say is the generic defaults of the models are still, are now the problem.

2:12:34So it's not hallucinations. It's not at making stuff up. it's that you start to see the patterns of what they repeat over and over. And that's the problem. So maybe the next. That's a good problem. That's an annoyance, not a crisis. Yeah. Yeah. Because if it's making stuff up, I mean, that's horrible. That's horrible. That's a problem. Yeah. Yeah. This is more of an ugh. Yeah. This is more like, oh, so I still am needed to provide creativity and innovation and things that, you know. Be smart for me. Yeah. I'm still part of the process. Okay, fine. Yeah, fine, fine. I'm hanging in there. I'm still waiting for my retirement.

2:13:14Just kidding. That's funny. That's a long ways. Well, thanks again, everyone. We sure appreciate you. If you haven't yet, please pop by, sign up for the newsletter, subscribe right above, and we'd love to see you back here next time. We hope you'll join us. I'm Corey, and that's Grant. And for folks who do have access to Sol, please email us at team at the neuron daily.com and give us your first impressions. Well, maybe we'll write something up if you include, include a lot of interesting detail. Sounds good. All right. Farewell for now.