ChatGPT Codex Voice + browser + Sites: an expert’s AI workflow | Nick Baumann (OpenAI)

0:00Nick Baumann:If we go back to 2022, 2023, around the release of ChatGPT, I think a lot of people felt the AI then. A chatbot that I say things to and intelligently says things back to me. I think with the advent of Kodi Agents and the Kodi Agent Harness in particular, now this AI is going out and reading files and creating files and running commands. This is crazy. I sell one thing and then it figures out all the other things on its own. I haven't gone deep with voice. I want to see it from the pro. How should we be using it? What's cool about it? So I'm just going to trigger voice. I've got a hotkey for it.

0:30I'm listening. Go ahead.

0:32Nick Baumann:There should be some Amazon receipt for a mic I just bought recently. It's for DX uses. I'm hoping you can help me expense it. Could you spend another thread to handle taking care of that expense report for me? The expense task is running. It will find the Amazon receipt and stage the Navon report, but not submitted until you confirm. I'll let you know when it's ready to review. If anybody has been lucky enough to have an assistant, this is how it gets done. And it is just like such a nice delegation experience to be like, hey, can you take care of this? Hey, can you fix this? Tell me what's going on.

1:02Nick Baumann:Typically voice is great. You can like talk back and forth, but it's usually with a less strong model that is less capable. Whereas this, it's fully able to delegate and manage these fully, you know, five, six sole threads on its own, which is great. Because then you can have it do things on your behalf. Hot take question. What do you think matters more on voice experience, latency or intelligence?

1:25Welcome back to How I AI. I'm Claire Vo, product leader and AI obsessive here on a mission to help you build better with these new tools. Today, we have Nick Bauman at OpenAI, and he's going to show us some of the advanced use cases of ChatGPT codecs and ChatGPT work, including how you can talk to your computer to book your flights, use ChatGPT sites to build websites you can share with anybody or nobody and use my favorite workflow, which is editing creator content in codex. Let's get to it. This episode is brought to you by Bolt.new, the AI app builder for people who have ideas and want to ship them.

2:05Most AI tools spit out code that looks great in a demo and falls apart the second you try to do anything real with it, or they lock you into their own platform with no real way out. Bolt is different. You describe what you want to build. A startup MVP, a landing page, an internal tool, a side project. And Bolt generates production-ready code in minutes. Connect Stripe or any other MCP, hook up your domain, and deploy it live. Founders are using Bolt to build businesses doing real revenue. Product managers are shipping prototypes their teams actually use. Designers and marketers are launching campaigns without waiting in line.

2:44Anyone can build, engineering can ship, everyone wins. You just need an idea and a weekend. Check it out at bolt.new slash howiai. Nick, welcome to How I AI. Hello, Claire.

2:59Nick Baumann:Thanks for having me. I am going to make you laugh because I might be the number one Codex fan, fanboy, tech bro, fanboy in the world. I am like constantly telling people, Wait, but have you tried Codex yet? It is just my daily driver. So I'm so excited to have you on because even though I use it all the time and for some things that you and I think both think are valuable, I have not touched most of the new features that have come out in the last week. It's really hard to keep up. And so I'm really psyched that you're here to just show us some of the new stuff and maybe how you use Codex both for work and life.

3:38So tell us, what are the things in Codex you feel like people don't really know about or are brand new that you think are completely going to change how we all work?

3:47Nick Baumann:Yeah, I feel like, you know, with yesterday's release of Voice, the ChatGPT Codex app, it's kind of like the amalgamation of all these primitives we've been, like, putting together kind of quietly. And now we're super useful. So a lot of people don't know that the ChatGPT app, you can ask it to create threads. You could ask it to message your existing threads. and these threads actually talk to each other too. And it's like not super kind of forward in the app, but these are all capabilities. And now we've added this voice orchestration layer really where now you can, you know, with the hotkey, you can trigger this orb that like pops in your screen and no matter where you are on your computer, you can talk to it, it'll talk back to you and it can see your screen, it can see what you're working on.

4:30Nick Baumann:And because we have those primitives of see your existing threads, create new ones, talk to them, it can like basically manage your entire chat gpt app for you yeah one of the things that i i don't want people to miss before we even get into like the fancy voice stuff is i really do feel like most people do not know the meta capabilities of codex and even like we won't say but like the the other one in that like its ability to not only spin off sub agents but actually start completely new threads fork threads look through old data look through its own memory um it's it's a really good mechanism to let the AI manage context and decide when forking off different tasks makes a lot of sense.

5:14And so I've been doing that a lot, but I actually had to discover that capability sort of organically when one day I was sitting there and it was probably five, six was like, I'm going to kick off a new thread. And I was like, you're going to do what? Like what now? And so I do feel one of the challenges with these core kind of platforms like Codex's in ChatGPT is people just don't know how to discover the features and they're so rich. And so do you have any, you know, like we'll talk about voice, but are there any other like sort of edge features of Codex or ChatGPT that you feel like people underuse, not just this thread one?

5:51Nick Baumann:The ChatGPT work experience, you know, people are talking about, you know, should we use ChatGPT work or Codex? I think ChatGPT work on the web, specifically mobile, and that's where I use it mostly, is so insanely powerful. it's one it's it's essentially like the codex harness which has this incredible compaction you have all your plugins so if you've like already connected gmail and gcal like whatever you use it's already there and then you have it all accessible on your phone so you can like create these automations um also it's heartbeats um and it's already there it's like i mean a few times a day i'll just get like a notification saying hey like i got this email about this thing you bought are you sure you bought that or like hey this package is arriving make make sure you grab that.

6:33Nick Baumann:And I'm not watching my email or my finances, but that's just kind of being monitored for me. And that's, that's really magical experience. Okay. I love it. All right. So let's get into chat, GPT voice and what you, what you use it for. Cause I, it came out yesterday. I've been super busy and around humans. We're going to talk about that a little bit later. So I haven't gone deep with voice. I want to see it from the pro. How should we be using it? What's cool about it. Yeah, I think it's really magical. So I'm just going to trigger voice. I've got a hotkey for it. I'm listening. Go ahead. I'm hoping you can help me handle a few logistical things right now.

7:12Nick Baumann:Can you see the offsite? I've got pull up in Slack. I'll take a look. Let me check that Slack view. Yep, it's the DX team offsite in Paris, August 26th through 28th. You're due in Tuesday with the program starting Wednesday morning and wrapping Friday afternoon. Flights and hotel are supposed to be booked in Navon this week. All right, so I muted myself so that Codex doesn't hear me right now. But what happened there is it can already see my screen. You can see that I've got Slack opened up. And what it did is it took an app shot. And so an app shot is something that you can do or whatever window you have open, you just hit command, command.

7:53Nick Baumann:and it essentially takes like a smart screenshot loaded with context. I can use computer use or browse use on it instantly. But, you know, kind of a meta feature, Codex Voice already has that built in. So I'm hoping you can help me book my flights and hotel for the Paris offsite. I'm also thinking about sticking around for the weekend and maybe a couple days after. Could you maybe like check my calendar, see what my availability is like, and see if it's feasible for me to stick around? I'll check your calendar against the off-site dates and then look at travel options. Is this using the connectors?

8:27Nick Baumann:Good question. Yeah, so this is using the plugins underneath the hood right now. So it's not having to actually go to... You're clear after the off-site through Monday, August 31st. A Tuesday, September 1st return would conflict with a late afternoon meeting. The simplest plan is returning Monday, August 31st. Do you want me to plan around that? You know what? I'm thinking I might just take PTO on that Monday and Tuesday. I'm flying out of SFO, so why don't you just go ahead and use Nivan, spin up another task to figure out the travel, pick a hotel that's reasonable and within our company policy, and then when it's ready let me know and I'll click book now.

9:06Sure, I'll sort that out. I'll find options for flying out of SFO, arriving Tuesday before the Wednesday start, and returning after Tuesday, September 1st. Then bring you choices that fit policy for you to book. So is this chat going to be in? Oh, sorry. I'm talking to his codex. Invite Claire to the Paris offsite. Put her up in the Ritz. Get her in business class.

9:34Nick Baumann:And honestly, we don't really have a Paris offsite scheduled yet, but so much of our DX team is French that I'm hoping that I can start seeding the ideas here. That task is running now. Comparing flights from SFO in a practical hotel for August 25th through September 2nd. I'll share the suggested itinerary when it's ready. Cool, so we've got this task already running. And make sure it uses my existing Chrome browser for this one. I'll check that. I've updated the travel task to use your existing Chrome session and keep it to preparing options, not booking. Awesome, so right know it just it just sets a message back to that task it's already running but I think while we've got that going I'm going to ask it to do another thing for me.

10:19While I've got you I've got an email

10:22Nick Baumann:I think there should be some Amazon receipt for a mic I just bought recently. It's for DX uses. I'm hoping you can help me expense it. Could you spin up another thread to handle taking care of that expense report for me? Sure I'll take a look. I've started that expense task and it'll use your existing Chrome session. The expense task is running. It will find the Amazon receipt and stage the Navon report, but not submitted until you confirm. I'll let you know when it's ready to review. Awesome. So those are both running in the background. I've muted our voice so we can talk without being interrupted.

10:59Nick Baumann:But we can already see that in Chrome, this task is using the browser. It's figuring out our flight options. It'll figure out the hotel. Frankly, this is something I just enjoy not having to do. It's nice to just have this handle in the background for me. What I like about voice is I do think sometimes people with AI really struggle still with like blank chat window syndrome, which is like they look at and they're like, what am I going to ask? What am I going to ask this brilliant AI to do? And so what I love about first, just like voice transcript to text, which a lot of people have been using is it is what our beloved multi-time guest, Hillary calls the yappers API, which is like the best highest bandwidth way to communicate with an LLM is just to yap out loud and just context them.

11:47But what I love about this like voice to voice interaction is if anybody has been lucky enough to have an assistant, like this is how it gets, it gets done. And it is just like such a nice delegation experience to be like, hey, can you take care of this? Hey, can you fix this? Tell me what's going on, blah, blah, blah, blah, blah, blah. And so I'm, I'm guessing the hypothesis here with this voice kind of experience is there's going to be like better discovery of use cases and sort of like more sprawling use cases because people can kind of delegate in a more natural way, as opposed to having to like sit there and think through how to instruct kind of in text the models.

12:27Have you found that there are specific things that you reach for voice with that you don't, that you don't like type with your human, human fingers? Are you a full voice pill? Like, where are we?

12:36Nick Baumann:Yeah, a few thoughts there. One, I think it's like this iteration itself is like kind of a new primitive in that typically voice is great. You can like talk back and forth, but it's usually with a less strong model that is less capable. Whereas this, it's fully able to delegate and manage these, you know, fully, you know, five, six sole threads on its own, which is great. Because then you can have it do things on your behalf. Other thoughts there? I find the thinking process in voice just a lot better. When it comes to, I could dictate for two minutes, send off a long paragraph, get back a couple long paragraphs and read those.

13:20Nick Baumann:And it's really hard for me to get in flow. I think of planning in voice to be really, really effective. Hot take question. What do you think matters more on voice experience, latency, or intelligence? If you could only pick one, I'm going to make you pick one. I would say when I use ChatGPT, I put it on high intelligence for voice. I think there's like a middle ground. I think if there are good tools you can delegate to, then I care more about latency.

13:50Nick Baumann:So I guess I'll say latency. Yeah, I think it's what we're going to be talking. And this is the Clairvaux prediction. I think we're going to talk more and more about latency in the second half of the year. because I just think it's like almost the thing to differentiate on right now is like how real time can these experiences really be. I see a lot of people abandon great AI workflows because of a spinner or because of a delay. And so like the more you can close that gap, the more I think people can discover cool things and actually pull through and get some stuff done. Yeah, I feel like so many product assumptions in design and AI tooling is like built around this like there's going to be latency.

14:29Nick Baumann:There's going to be delay. Yep. And the more we shrink that, we can kind of drop those assumptions and new products will, you know, will emerge. amazing okay so so what we've seen here just to reiterate for people who are maybe not watching or listening is you can hotkey spin up voice it can manage these like very intelligent five six threads you can delegate tasks it can be hooked up to plugins it can use your browser it can use app shots it can use all these things and it's just like a very high bandwidth low friction way to talk to superintelligence that can get nice stuff done. You mentioned that a little bit earlier that you're also loving mobile and this combination of dictation and managing through mobile.

15:16Can you tell me how being able to do some of this on your phone has changed your workflow? Maybe give me an example of that.

15:22Nick Baumann:So I put together this site, and so this is just an aggregation of all these really cool tweets that both people in the Codex and ChatGPT community and also people who work at OpenAI have shared with how they're using Codex and ChatGPT. This was done just with ChatGPT, like the ChatGPT app where I had it use its own in-app browser, go through Twitter for a couple hours, and then embed these links and kind of add a little bit of description to them. And now what I do is I've got a thread on my work phone that is essentially managing this site. And when I see something I like, I just drop in that link.

16:02Nick Baumann:I say add it. And it does. We also have this prompts tab. And I do the same thing there. If I have an idea for how somebody could be, maybe he's newer to ChatGPT work, I'll just dictate my phone, explain like, hey, there's this prompt for having to manage your inbox and your finances. Can you add this to the site as well? And in terms of actually editing the site, deploying it, that can all happen. from ChatGPT work on mobile. And I think what people are not seeing here, which again, so many new features coming out that I think people aren't taking advantage of is the domain here is chatGPT.site.

16:42So you can now deploy these like artifacts, these sites live. So you just walk us through a little bit how that works or maybe we can do one just so people can understand the power of these sites and how you might use them or what their limitations are even.

16:58Nick Baumann:Yeah, I think for a 10-second description for the technical audience, it supports a SQL database. It has S3 storage or files. And it even has environmental variables you can put there. For the non-technical audience, it's essentially a website that you can store things on, which is great. And you can also control who sees it. You can filter by email. So it can be private. It can be fully public. That's up to you. But what we could do, if you're interested, is we could try building a site live. Let's do it. And see how that goes. Love to build live. Yeah. All right. So let's jump into it. So I've got an idea for a site.

17:36Nick Baumann:I'm here right now with Clara Vo. She hosts the How I AI podcast. And we're hoping that we can build a site that takes all the best tips from the videos on her YouTube channel and makes it so you can easily filter through them and have quick links to different timestamps on our videos. Make it match the How I AA branding. We use good old AI blurple, purple, and black, and white. And let's have it, if possible, categorized by tool. So like what AI tool is being used in the workflow and also function. So if it's for designers, if it's for engineers, it's for personal productivity, any of those things.

18:23Otherwise, I'm excited to see what you come up with.

18:26Nick Baumann:Awesome. That sounds great. And go ahead and deploy it privately, and then we'll take a look at it when it's ready, and then you can deploy it publicly from there if it looks good. While this is running, let me make you laugh, because this two-human chat-to-chat GPT, I did recently with my mother and she was like stuck on an administrative task and she was like getting all frazzled about she's like oh I'm gonna have to look up all this stuff and blah blah blah blah blah blah blah and I was on the phone with her and I was like mom just hold on one second and I opened up chat GPT on my desktop and I turned on voice and then I was like can you just please tell me what you need to do and I turned the speakerphone off and I put it up to the computer and had her like babble kind of like we did.

19:12And I asked her a couple of questions and then I was like, thanks, pressed enter. And I was like, I did it for you. I did it for you. So I really do feel like people are underusing like two person voice chat. You can do it on your phone. You can use speakerphone. Like it is this really nice, again, like high bandwidth way to get requirements into the system. And it doesn't just have to be you alone.

19:35Nick Baumann:Yeah, I've had times where, you know, I need contacts like somebody I work with. And I'm like, please just like ramble, dictate into Slack and send me like garbage. I don't care. And they're like, no, let's just like set up a meeting. And I'm like, no, don't, don't do that. I don't need that. I don't need this to be like pretty or anything. Just, just tell me that's fine. So are we at this meeting could have been a voice note like level? Yeah, I think so. I mean, it's more like give me information that unstructured, however it is that my agent can decipher. And that's that's good enough for me.

20:12This episode is brought to you by HyperAgent, the platform for deploying always on agents that actually run your business. With HyperAgent, you build agents in the cloud and deploy them where your work already happens, like Slack, Telegram or email. An agent will scan your inbox and draft replies to vendor follow ups. Another monitors competitors and spins up rich ad kits and landing pages. A third notices a deal going cold in Salesforce and writes the save email with full account context. These aren't chatbots waiting for a perfect prompt. They're proactive, learning your preferences, retaining your playbooks, and getting better with every run.

20:51One user built four agents to run an outbound sales pipeline, prospecting, outreach, follow-ups, CRM updates, all in a single afternoon. No local setup, no VPS bills, no fragile permissions on your laptop, just powerful agents with full control over skills, tools and guardrails. HyperAgent was built by the team behind Airtable and How IAI listeners get$1 ,000 in free inference to start building. Claim yours at hyperagent.com slash How IAI. Okay, so this is going to go spin up. We're going to let it kind of like whirl a little bit. maybe we'll come back to what what we see but there's one other use case that both you and I really like in codex um that I thought you could show in particularly how how you prompt it which is codex for video editing and for those that don't know it I do a lot of this for um for the podcast I also do a lot of this just for like general work stuff I'm cutting a lot of videos So tell me like how you came to this use case, why you feel it's so useful, and then how easy is it to prompt kind of like video editing?