We Tested OpenAI's GPT 5.6 for a Month

0:00GPT 5.6 is here and it's a big deal. My friend Dan Shipper has been testing it for weeks and in this episode he shows you how you can use codex with GPT 5.6 to run your personal life and run your business. His setup is really, really cool and by the end of this episode you're going to learn where 5.6 works, where it doesn't work. You're also going to learn how to use codecs in a really, really, really cool, interesting way so that you can make money, be more productive and just automate a lot of that boring stuff. I loved having Dan on this episode. He's a fountain of knowledge and he shows everything.

0:37So enjoy the episode and I'll see you at the end.

0:48Really excited about this podcast, Dan Shipper. I begged him to come on. He came on. Dan, by the end of the episode, what are we going to learn? We're going to learn how to use Codex to run your entire life and how to use it to build a compelling software business. You have my attention.

1:12Should we get started? Let's do it. Okay, so if you haven't used Codex, it is the OpenAI answer to CloudCode, CloudDesktop, CloudCowork. If you've used it before, you may have used Codex CLI, which I do not recommend. It is like, Codex CLI is trash. The Codex desktop app is what the Cloud desktop app would have been if they'd just thrown it out and they were like, what is the ideal way to build this? Because right now, like the Cloud Desktop app, it's like you've got chat, you've got code, you've got coworking. And I'm always like, which one do I use? And Codex, well, now they have two tabs, but really it's a much simpler implementation, much cleaner and much more powerful because it was built after OpenAI got to see, okay, like this new paradigm of using an agent on your computer to do your knowledge work it had already started to happen because of cloud code and cloud co-work and i feel like opening i just like fast forwarded through all the like messy stuff and they're like this is the ideal implementation so and and i i feel like anthropic has the has a little bit of the mandate of heaven right now and open ai is quite underrated and uh and i think codex is the reason why.

2:36So Codex with 5.6, which I find to be the most usable, most powerful, fastest model for specifically for knowledge work and for coding, but like specifically for knowledge work, it's, it's not Fable level, but Fable is like, it's like a tactical nuke, you know, it's like, it's, it's so powerful. It's like actually illegal. Um, and 5.6 is, it's more like, I don't want to use a weapons analogy so I'm going to move away from the weapons analogy because it's not a weapon. It's more like having a Porsche. It's great to drive around town. You can do anything in it. It goes fast. It handles really well.

3:16It's really good for collaborating. I think of Fable as being something that you have to have skill or a lot of skill to use it well. 5.6 it just works for everything you want it to do. So anyway, that's all a big way of saying five six and codex is where i spend all my time it is my operating system for work i do everything from my emails to writing to i'm actually starting to train models now which i think is a new frontier um i don't even know what that means i need i want to get into that later let's get into it let's get into it um i would say like just a just a real like a small preview is training your own model is the next step after making your own skill if you've made a skill for something and it's not quite working, fine-tuning a model is the next step.

4:08Until recently, it was completely out of reach and not at all practical. But 5.6 and Fable to some extent, if you can get around the guardrails, makes it easy enough to make a machine learning pipeline and run it. That's everything from grabbing the data to making more synthetic data to actually running experiments and makes it easy enough to do that. I think it's now possible for non machine learning engineers to start doing it. All right. Well, I'm, I'm intrigued. I I'd love to see some, you know, what you got. You want to start sharing your screen? Let's, let's go. So this is how I do my email.

4:52Um, so this is a little app that I built. Uh, by the time this comes out, you'll probably be able to get get this app um it's called tend but you'll probably be able to get it as an open source library i'll release but honestly you can literally just feed this video into codex and just be like make something like this and it will do that so you know do whatever you want um but basically what this does is it goes through my inbox every day and it just finds every single unread email and then for each email it makes a card and the card summarizes what was in the email and then it has a little draft reply that it that it drafts for me and then what I can do is I can say can you well I didn't deal with this yet can you find some more times like later this week and show them to me in a draft and then it'll go do that and now we're on to the next email and I just go through my emails like this let's see we want to keep writing.new it's on it's on Namecheap currently I can get you the Cloudflare credentials and now we're on to the next one I'm going to just this reminds me of like how I picture like in the 1960s really wealthy business executives like Mad Men style would answer their letters.

6:22Dude, it's literally the same thing. It's just now everyone can do it. And what's cool about this app is it works for your inbox, but it works for, and I will say, this is an open source experiment that I've built. We have an email app called Quora, which is a full inbox experience that is designed to be used in this way. So if you want to make your own as a builder, you can do this. This is also coming as a full app. But what's really cool about this is I have different feeds for different things that I do every day. And in my codex, basically, my codex is built so that I have a bunch of pinned threads for each part of my life that I care about.

7:12So this is like my company thread. And in this thread, this is all of the stuff that's going on in my company. so basically what it'll do is instead of sweeping through my inbox it will find all the slacks that i haven't seen and all the meeting notes that i haven't seen and turn it into cards um and uh and it gives me an idea of like the issues that are happening in the company and i can be like i can decide is this an issue that i care about or not and if i do care about it which i don't um if i do care about it i can say like oh i'm gonna go ping brandon or ping someone else and be like, hey, what's what's going on with this thing?

7:49In this case, I would just say archive. But what's interesting is over time, it starts to learn what I what I care about, what it should bring to me. And then also what my next what the next best action is usually like what I would normally do. So after I go through a feed, whether it's my company feed or my inbox, It then compounds the learnings of stuff that I liked or didn't like archived or didn't archive replies that I did. And then it turns it into revised prompts. And you can see all the prompts here. So it has a feed policy for here's what Dan cares about. Here's what's on his mind. Here's how to get specific sources, all that kind of stuff.

8:39And yeah, it just sort of changes how I do work every day because everything can be turned into a feed with cards and a next best action. And that's pretty sick. And with 5.6 specifically, you're just seeing better results? Yeah, I think that 5.5 was fine, but not quite good enough. I would say generally the email responses here, So like this email draft, for example, is like it's it's as if I wrote it like it's maybe there's something that I would slightly change and I might. But for 90 percent of the of emails, which are like, you know, scheduling stuff and just more generic, it's it's really good.

9:26So it's I think it's a very good writer and it is it makes far fewer mistakes. It doesn't send emails that I probably wouldn't want it to send. It's a little bit more careful. And it's very resourceful. It easily connects to Slack and it easily browses the web. I got a new apartment and I literally have it searching Facebook marketplace and rendering what the guest room is going to look like and shit. And that's fucking crazy, man. I think that people are sleeping on how flexible and powerful of a tool this is. Because, okay, so it's got the image model in it. And then it also has the ability to change other threads.

10:19So I can have this thread message another Codex thread. And suddenly you're orchestrating all these different agents doing different things. And anyway. Well, that's sort of the aha moment I had at least with Codex. was once I was starting to reference other threads and remembering things. And I was like, okay, this is, that to me was like, okay, I understand what, you know, you said Codex is an operating system. I had seen that for such a long time, but it was only until I started really referencing things that I was like, okay, I can see why this is, this plus the browser, it feels like an operating system.

10:57That's the thing is like, I do all, the reason I can do all my work in here is that it has this in-app browser so this is proof this is a markdown editor that I made and and for example this is like my my day today um and like I'll just be in proof and I'll like plan out my day and I'll like click things and then I can say like okay um add an agenda for my startup ideas pod um interview um and codex will do it and while it's doing it I'll be like down here writing like you know uh uh a few ideas on my mind you know codex as an operating system blah blah um and it's just gonna like go it'll go do a bunch of research it'll it knows what we're building it knows what ideas I may have and that's that's one of the really cool things about it is it's so it has all the context on who you are and what you care about that it can bring to any app that you're using now I can see now it's already in here it's you know looking at stuff and i it just it makes a big difference because yeah the the model being powerful is really important but the model's only as powerful as the context you can give it and this thing is just like connected to your whole computer and the whole internet and so i'm just like constantly in here i like any kind of sass app i think the sass apocalypse is so dumb um you know here we've already got like a nice little agenda i think the sass apocalypse is so dumb because um a yeah i'm like vibe coding this inbox app but 99 of the world is going to want to use something that's maintained by someone else yes you can one shot everything but the um i think the the hard rare thing is moving to can you build and maintain something over time and uh and that's still expensive and hard and in general i think of software as being if you can bring your agent to software it's really powerful it's like giving it a structure to work inside of you know for us we have a um we have a pretty big consulting business now and of course the first thing we tried to do is vibe code around crm and have it be like a combination of agent and you know google sheets and stuff like that and it's you can kind of do it but it becomes a mess really quickly and we've just moved to a crm vendor that has a cli and it's like they've thought through all the little corner cases and they're going to continue doing that and we get to actually do the work that we want to do instead of like vibe coding you know customer records and like thinking about thinking about that kind of stuff so um yeah i'm very bullish on SaaS, especially SaaS that's designed to be used inside of the codex in-app browser.

13:46Right. Yeah, I think, yes, you can build anything, but people will pay for maintenance. That's sort of like the insight. Totally. If this thing works, I'm fine to pay for it. and then I think that again another aha moment I had with Codex was just looking at this, seeing my Sass we don't really think about it as a Sass on the right hand side but it is a Sass and how it interacts with my threads back and forth and how you can ping pong it just feels like, I don't know about you but you can literally, it feels like you can spend 99 % of your time in Codex or in what other super app that you're going to use at this point.

14:35I definitely agree. I'm kind of back and forth on whether the super app thing is going to work, but I absolutely... More of an operating system. We say super app, but it's really an operating system. This episode, you're showing me ways how you operate your business and your life with Codex and 5.6. Super app, I don't know. It's just a cooler way of saying operating system. Should we just make something? Yeah, let's do it. Okay, because one of the, and I've been playing around with this already, so there's probably some of it on my computer already, but I think this would be kind of fun. So what we're talking about, I think you framed it exactly right, which is maintenance is the thing that people pay for because you can one-shot anything now.

15:27And so I want to make a little SaaS that is designed to help people display that their app is being maintained. And there's already stuff like that on GitHub, for example, where you can kind of see how many commits have happened over the last period of time. But I think we need that for any SaaS app that you encounter on the internet to be like, someone is maintaining this. And not only that, but whenever a bug is reported or a feature is requested, here's how much time it took to uh to fix it or to add the feature um because that's the like sign that there's someone who's actually working on it and it's not just like a vibe-coded one shot do you think that would be fun to build 100 let's see if we can do it all right let's see if we can do it um all right let's see um all right i want to build a little sass app called turnaround There may be some demos and stuff on this computer already, but I want you to just ignore them because I want to do it from scratch and see what else comes out of this.

16:33But the basic idea of turnaround is the value of SaaS is not in building the first version. It's in doing the maintenance. and we need a way to help people who are running SaaS apps display the maintenance that they're doing, which is basically the turnaround time from when someone requests a bug fix and the bug is being fixed or just overall the number of commits. You want to see that commits are happening all the time on this app and I want a little beautiful badge that gets displayed that you can put on your about page or something like that that validates that this app is being actively maintained and worked on.

17:16Sort of like what GitHub does, but it should look good. It should look cool. It should look sort of AI-y and futuristic. And we should have a way where anyone can sign up for it. You can put the badge on your website, but our job is to be the people that are sort of validating on the back end how many issues in GitHub you're taking down, stuff like that. And it should work with any sort of data source. So if you use linear, use Notion, whatever. Can you use CEIDA and CE Brainstorm to help me figure out how this should work? And the sound effect is from Monologue, which is our speech-to-text app that we've made.

17:56It's operating on my other monitor, so you can't see it, but you can see the output and hear the sound. What should we talk about while that's running? I mean from your just experience building with 5.6 you said it wasn't exactly Fable like pretty close when you say close are we saying like 80 % 90 % like a Fable's at 100 % just trying to get a sense of like where we're at I think Fable's like an S tier S plus tier model like it's so good it's illegal and this is like an A tier model you know so yeah you're not gonna hack the NSA with 5.6 but or like you're not gonna you might I mean I'm not

18:45you might not successfully hack the NSA like with Fable I was we were like literally our bug backlog bug was emptied out you could just set it off and it was emptied out as long as you're willing to spend the money and with five six it's not really like that like you have to do some thinking to fix bugs for example yeah but it's close so it's like 60 70 ish but fable is this other thing it's because it's so it's also so slow and it's so um uh expensive that it's really only meant for essentially like people who are doing machine learning or are orchestrating like fleets of agents which is just a skill that most people don't have yet cool all right first product fork what should the badge promise prospective customers alive responsive or dependable um hmm i will say dependable like i like that there's a recommended option it's pretty good That was sort of a miss in prior models.

19:59It wouldn't have a taste. I don't want to say the word taste because I shouldn't use that word. But it didn't have a point of view. An opinion, yeah. And part of this is I'm using our compound engineering plugin. So part of this is the compound engineering plugin. but I just think that these are good options. Like 5.6 just has good, I'm going to say it again, taste. And I like the different vectors that it's, is it alive, responsive, or dependable? I like alive. For it to be credible, what happens when activity stops? Yeah, state changes automatically. Actually, yeah, state changes automatically.

20:46Let's say that. But it's not judgmental. It's just like one commit in the last 30 days or whatever we get to, something like that. From a speed perspective, 5.6, what do you think? I think it's great. It's so fast and it's very to the point. It doesn't write too much. I like 2 because it has more concrete details, but I also don't like the idea of necessarily saying it's alive versus like dead versus it's not like it should just use the concrete details to tell you what's going on so you can see it in a second I'm just going to like set it off and see what it does all right I want you to make the best possible decisions and just use LFG to um and a goal to just make this thing.

21:41And show me what it would look like for, let's say, Quora and Monologue. So show me on a page. So what is LFG and what is Goal? So LFG is, again, this is our compound engineering plugin. It's a plugin by Kieran Klassen, who runs Quora, which is just, he's like an insanely good AI engineer and he's just taken all of the stuff that he knows and put it into a plugin. and LFG it basically just loops through the whole cycle of plan like execute work, review the work compound the learnings and then just keep going until it's made and then goal is Codex's version of like give it, like give the model a high level long term goal that it just like keeps going until it's it's achieved like for example one of the things i'm doing i told you i was doing model training um i'm training a model to do copy edits and this goal is four it's been going for four hours and i just like let it go but four hours is honestly very short like i over the weekend i had one or two that went for like 20 hours each and so it's just a really nice thing to be able to say here's where I want you to get to here's verifiable criteria that would tell me that you are actually done with this I'm going to go do something else and or theoretically I'm going to go do something else what I've actually found I don't know if you have this but I've actually found that I'm incredibly addicted to just like watching the stream which I hate but I just I just am like that for whatever reason.

23:26So anyway.

23:30This is another nice thing about 5.6 that's pretty new is it does this like design checkpoint. So it's really, they're trying to make it better at design. And I think it's not perfect. Like it always wants to do warm paper backgrounds for some reason. I don't know why. So it's like better, but it's not perfect. I give it like a B, you know, where previous models were like a C or D. So, yeah. Well, at least it asks, right? At least it asks. At least it's thinking about warm paper and I can be like, no, I want cool paper. Yeah, I think in the past it just jumps to conclusions and that's when you have the same looking website as everyone else.

24:18So I think what they're probably doing is they're giving you an opportunity to add more context before they continue. So while this is going, I think one of the things I wanted to say about SaaS and all that kind of stuff and building, I think there's a huge opportunity for Codex native SaaS apps. So think of any SaaS tool that you use. So we made proof, which is document editing, but there's email, there's CRMs, there's any of that stuff. Any software that you use every day would be better if it was built to be used inside of something like this, inside of something like Codex, where you can bring your agent to it and your agent has all the context on you, on who you are and what you do.

25:09And you're not designing it so that it's a CLI and the agent just like you're just delegating it and the agent is in using the CLI without you. It's designed for you and the agent to be working on it together inside of Codex. You want to be able to bring your agent to the software that you use and have you and the agent collaborating on the same surface. And this is what programmers already know about the best way to build apps. Like the reason the in-app browser is a thing is when you were building an app, you want to be able to use the local app that you're making in the same place that your agent uses it, so the feedback loop is shorter.

25:45Same thing is true for any kind of knowledge work. And I think there's a massive opportunity for anyone that understands that to just make apps that are built that way, because most apps are not designed for that. And do you think that customers are going to be looking for these agent-native apps? If they're smart. So I think that there is a crew of early adopters that is starting to figure it out. and it will take a while for it to be totally mainstream. But to me, that is the opportunity is to start establishing yourself as making an app that's codex native for a particular category so that when it is really mainstream, you're the place people go.

26:33I would start trying to rank for that and having landing pages around using it inside of codex and all that kind of stuff because having a lens into all these companies, I think a lot of the model companies and just general big AI companies see this as a trajectory that things are starting to go. And yeah, it's going to be a big deal. And one of the cool things about it is most SaaS companies, now they're installing agents in their products. so they have to pay token costs. In this world, you're just a SaaS provider. You're just hosting software, so there's no token costs. So your margins are back to being a regular software business instead of being an AI business, which I think is great.

27:26Yeah, it's also great for the people who want to take the non-venture-backed route. Totally. I think there's a lot of us who are kind of like, hey, I'd love to build a business that makes$10 million a year and the costs are low and stuff like that. I think a lot of people were looking at SaaS and these agent-native SaaS's and they're like, okay, wow, great, this company's making$80 million a year in revenue but they're spending$60 million a year in tokens. And it's just like, for some people that's interesting but some others not. So I do agree that these codec-specific apps are probably going to do well.

28:10The only thing I'm thinking about is from a distribution perspective, it's not similar to the mobile era when you had the app store. I remember I built an app in 2013, and we were featured on the app store, and we were getting 25 ,000 downloads a day just from the features on the app store. I think what would be really cool is if the codexes and the clods of the world start promoting these apps that are specific to making the experience better. I bet that they will. OpenAI has launched already three or four different app stores, literally. Every year they have a new app store play because I think they realize that playbook works really well.

28:55It's incredible to have an app store because then you have an ecosystem and you have people building an ecosystem. And it's not just about stuff you can build. It's about the value you unlock for everyone else building on your platform. And I bet you within a year, the opening eyes in the Anthropics of the World will have app stores for apps that are built specifically to be used inside of these services, maybe even cursor too. I don't have any insight on any of this. It's just too good of a business opportunity. And it's just a logical thing if you grew up in the app era and saw how good the app store was to want to bring that to the agent era.

29:33All right. See? Warm paper background. Turn around. Let's go. Interesting. Your inbox. What are we looking at? So this is its take on. This is the thing. is they've they've they have improved the design here but it's a certain kind of design that this sort of flat aesthetic i don't think is quite right for this but anyway so what we're looking at is a mocked up version of the embed that we were talking about where you can see okay on the core about page 26 commits last 30 days um and you can see like how active it is um i don't know if commits is really the right word. And especially if we're going to call it turnaround, that's kind of an interesting one.

30:25But it's essentially like a, it's like a, it's a status page for the vibe code era to show you, okay, yeah, there's a lot of activity happening in this, in this project or not. And I don't quite like the, the initial visual implementation, but you can kind of see what it's getting at. Totally. Yeah, I think it's like 75 % of the way there. Yeah, it's enough that I can kind of see it and I might be able to, like I could probably publish a landing page that's like sort of like this. But to be honest with you, at this point, I would probably, I would go to Claude for the actual design and then port it back in here.

31:09because I still think that Claude has a better sense, especially Fable RIP. But even 4.8 would be better at this. And I think for a product like this, being able to see it at a glance and understand what it is is the most important thing. And I still think that Opus and just Claude in general is ahead on that one. And while we're waiting for this, what are your thoughts on using, for example, cursor or codex and having local models as well play a role. There's this whole new wave to local models. Are you like a token maxi where you're just like, I use OpenAI and I use Anthropic and that's it? Or how do you think about it?

32:00I have a feeling that open source models are going to be very important to the way that I work. Currently they are not. um one is i just as a as a person and as like what we cover it's like we're very i just want to be at the frontier like i just want to be like fable maxing all the time uh but we're starting to get to a point where even for me model advances in raw intelligence are to some degree hard to consume. You know, like I can't use Fable for everything. There's not, I don't have a big enough problem to use Fable for. I do use it, but not for everything. And so I think as we start to get toward that ceiling where I can't consume extra advances in model power, then speed and cost and efficiency is going to be more important.

32:56And at that point I will be interested in incorporating more open source models into my workflow and like i said i've been doing some some model fine tuning in that case like you have to use open source models because none of the none of the major model companies or at least open and anthropic don't really do fine tuning anymore um and i think that's a huge area of potential opportunity um because there's there are still you know for us these models can't do copy edits they can't do copy edits like like a copy editor can and it's weird because copy editing shouldn't be that hard um at least copy editing to like 70 or something like that and and so for us we have all this internal data of we've made tens of thousands if not more hundreds of thousands of copy edits over the last five or six years we should be able to turn that into a trained model and then sell that and i think that there's a lot of organizations that have that same underlying data source that they can turn into like a product or a capability.

34:05All right, what did we get here? Neutral badge, core monologue previews, blah, blah, blah. Okay, but like, okay, but how would it be implemented? Like, how would I install this so that it would be updated? See if it's trying to get around the hard problem. Let's see. Open the readme. I see. This version validates the badge, public record, visual identity, and core measurement language, accounts, authentication, persistence, billing, blah, blah, blah, GitHub ingestion, and generated embed code. Okay. I probably should have spent a little bit more time pushing it to get the best out of it. But I would say we're 70 % there.