⚡ [AIE CODE Preview] Inside Google Labs: Building The Gemini Coding Agent — Jed Borovik, Jules

0:04Okay, Jed Borovic, welcome to Lanespace. Yeah, thanks for having me. So we're sitting here at F.Ink's beautiful podcast studios, and we're actually meeting at GitHub Universe. How's it been so far? It's been great. I mean, yeah, the keynote today was awesome. It was fun to see Jules up there a little bit. You know, we have a lot of folks from our team here. Jules is, yeah, partnering with GitHub for the new Aging HQ stuff, which we're excited about. And also, this is an incredible podcast space. So yeah, I'm excited to do this here. I'm glad for them to loan us this space. You are also an emcee for AI Engineer Code.

0:37uh that's exciting in new york where you you went to college but you don't live there anymore yeah no i spent a bunch of time in new york you know it's funny um being part of the new york tech scene i actually think it's great having big major conferences there like i think uh so there's a lot obviously that happens on the west coast um but being someone in tech on the east coast it's yeah it's just awesome to have have stuff there so yeah you mentioned you fly over to sf a lot and like like what's it what's the scene like in the east coast like uh like obviously we are pretty new where like this is our first year coming to new york um what else it happens in the new york like what are the highlights for you in the new york tech scene yeah i mean there's so much there's you know obviously a ton of great companies um i think the thing that's interesting about new york is it's such a big city with so much going on right and so there's like you know tech is a huge part of it but there's also you know so many major issues there whether it's fashion media session finance like it's uh um and somebody's like i think that helps push the tech um and do do all kinds of stuff.

1:34But yeah, no, the East Coast is a great city. The great, you know, the, the, all the schools, there's, you know, all across the East Coast, a ton of great schools and great students doing all kinds of stuff. So yeah, you know, I went to school there. The hackathon scene there was amazing. Really fell in love with tech and, and programming there. So. Is there a big NYU hackathon, like, like in Stafford, where like Cal hacks and stuff? Yeah. So there's a. A tree hacks. Yeah. There's one that was put on by, you know, this was a while ago, but we put on by NYU and Columbia. we do a hack and why uh so there's there's uh there's a bunch of events kind of that we did together it would bring you know people across new york city students across new york city and those are super fun yeah so it'd be at the columbia one one time then nyu the next and recycle back and forth so yeah a lot of cool stuff was made there nice uh so you've been at google for a while nine years uh you worked on a bunch of things including with malta which which is also another guest that i'm interviewing today uh how do you get into jewels like what's the what's the Yeah.

2:31So, you know, this is going to sound really cheesy, but I've told the story a couple of times to folks when they're like, Oh, how'd you end up, you know, doing this? Um, but it is actually very true. So I worked on search for a long time, uh, and specifically kind of like news and freshness. Um, and then, uh, you know, when stable diffusion came out, that to me was the first like Gen AI mode. I know some people talk about like chat GPT is like the first thing, but for me, stable diffusion, uh, you know, it was a couple of months before chat GPT came out. It was a huge thing i was i was following it a ton online and there were two groups of creators having reactions to it you know there was one group that was um you know this is stealing my art this is stealing everything that's near and dear to me i hate this this is ruining my life and there was another group of artists and creators who were like oh this is a tool to create better art and so i was watching it's a new brush yeah exactly and right around then um i was having conversations with a couple people who would say things like you know if i had a kid in college i wouldn't recommend then they studied computer science.

3:27I was like, what, what? And this was long before like Jensen Huang and people like had been saying this kind of stuff. I was like, well, why? And it was like, oh, AI, like software engineering is going to change. It's going to be so, you know, who knows if there are going to be jobs. And I was like, I love being a software engineer. I love programming. And I was like, wait, this is my stable diffusion moment. This is either, it's going to take my art, my craft. This is a tool to create better art. And I was like, I definitely know which path I'm taking. So I got, you know, very into building coding.

3:53So I was still working on search. but I spent a bunch of time making stuff for my own time and playing with things and ultimately tried to find a role that would the most exciting role I could find to do this stuff and that was to join Google Labs and Jules where we were right around then we were starting to build these kind of coding agents at Google and yeah the timing worked out well and I joined and yeah it's been awesome. Can you since we're talking about Google Labs I am actually unclear about where Google Apps starts and the rest of, and then DeepMind and the rest of Google, like what, what is the org chart layer?

4:28Yeah, yeah, yeah. That's a great question. Um, so Labs' mission is to build kind of new, like innovative products that the rest of Google isn't well positioned for. Yeah. Which we've had like a rise of an Opal Glam. Exactly. Exactly. So Opal Glam is maybe the most wide, the most. Yeah. And then it's called Nano Banana. I don't know if it's. Yeah, so the thing that's really exciting about Labs is we work incredibly closely with DeepMind. So all the stuff in terms of the, you know, we're building a product, but we work so closely for the model. And, you know, one of the nice things about being at Google is you have this opportunity to really build an end-to-end AI product, right?

5:04From like pixels on the page, through the infrastructure, through, you know, the model and the training and all of that loop. So Labs is here to build new products and we're really like a product org, but a true AI product org where we work incredibly closely with DeepMind, but also other parts of Google, as it makes sense. Yeah. Just on the history of AI coding, I had heard that actually Google had an internal version of Copilot or something like that that was never released. Is that true? What can we say about it? Yeah, so I think there are, Google has published papers in this space for a while.

5:42And so, yeah, we have, in Google, we built a lot of our own tools. And CIDR, which folks maybe have heard of, is our internal IDE. And we've had all kinds of capabilities and tools there for a while. So, yes, we certainly have had pretty good tools for a while, but they were for internal use. Yeah. Yeah. I think, I think it was interesting because like, I think the, uh, one of the hype moments when Google started getting into the sort of like LM game, like basically when everything rebranded to become Gemini and like starting to push out Gemini where people were like, oh, like, did you know that Google probably like Google's entire repo is probably about the same size as GitHub.

6:22And like, you know, there must be some interesting data in there. Oh yeah. I mean, And that's one of the things in building a lot of these internal systems is the data is incredible. Especially when it's not only as the model and the training in-house, but all the data around the usage and whatever. So we could build really kind of sophisticated things there. Yeah. Okay, so let's introduce people to Jules. On your website it says Jules Autonomous Coding Agents. We've seen lots of these. They're not octopuses. They're not purple. So you got that going for you. But what really is the core thing you're trying to nail in a very crowded coding agents landscape?

6:58Yeah, so what we think about and what we set out to do back when I joined was like, where are coding agents going to go? And as these models get more and more powerful and sophisticated, what is that experience going to be? And let's build for that future. And so when you think of a really powerful agent that can run for a really long time, doing really complicated things, that's when the products start to take shape for us. So for example, autonomous, you know, means like it has its own computer, right? So for Jules, it's, you know, the end of the, exactly. So tons of agents that run, you know, locally or in your workspace with you while you're coding.

7:31But if you want something that's going to run for hours or let's say days, you know, you might want it to have its own environment where it can, can do its own work. So that's just one of the pieces that's important for kind of this autonomous coding agent. But it's really like, you know, think about this future where they're incredibly powerful. You can spin up tons of them, right? They're autonomous, but also we're thinking about what does it mean for it to be ambient? When it has its own infrastructure, its own computer, its own ways to interact with it, how does that start to change what it can do?

8:04For example, we have an API. So people are using it for all kinds of things, triggering it from when something happens. We saw an example where someone has, they're triggering Jules to do all kinds of updates to their site. and then they have a GitHub action that is going to automatically merge Joule's pull requests. So it's just like all kinds of stuff is flowing, really kind of changing how people are able to do stuff. And it's a CLI related just to close that loop. Yeah, CLI. So we also have a CLI, which is, you know, we want to meet developers where they are. And so part of the, you know, an API is like you can trigger from anywhere, but also, you know, when you're working locally, like you want to be able to trigger stuff.

8:38So we, yep. So we have the Joule CLI we launched a couple of weeks ago, which lets you interact with it. By the time this podcast comes out, we'll be integrated with a Gemini CLI. Yeah, so I was thinking, like, you have a number of CLIs, I'm not sure. Exactly. So Gemini CLI, all kinds of places where we're going to kind of mix and be able to harness this power, right? Because developers work in all kinds of spots. And so making it easy to, you know, have this autonomous ambient agent that can really do all kinds of work for you. Yeah. What is your journey? Like when you started out, like, did you find any assumptions that were quickly challenged, you know, when working with Gen.EI and coding agents in general?

9:23Like, I guess you're maybe not too unfamiliar with it because search uses a lot of like machine learned, like black boxy type things, including BERT, which, you know, was a major update a few years ago. Yes. I mean, just fill us in. Like, what is your AI engineering journey? Yeah, totally. So I think one of the things that keeps coming up is like the model makes such a difference. I mean, maybe it sounds obvious, but it's like the quality of the model really changes what you're able to do and how you engineer around it. So for example, when we started, this was, you know, with relatively early models of Gemini, we had the agent scaffolding around it was incredibly complex.

10:03I think one of the things we've seen is scaffolds get simpler and simpler over time as the models get better. And in some ways, this scaffolding is almost a crutch for things the model struggles with. For example, like, you know, really complicated sub-agent systems. You know, we've played with that. We've experimented with that. Can you give an example of a kind of subagent that you had to abandon? Yeah, we're just basically like you have, you know, you give Jules a coding task to do. And it's going to have different agents for whether it's, you know, making a code edit or handling a sub problem or, you know, doing any kind of action with an integration or, you know, having like full subagents for different parts of it.

10:46like a reviewer agent or even like people, sometimes people do these like different personas where you're like, you know, one of the things that cracked me up is like, you know, you're the product manager agent and then you have the code reviewer agent, you know, we didn't go, you know, that far. But I think a lot of these things aren't as in favor. I mean, certainly people do, you know, like I don't want to say the agent harness isn't sophisticated, it certainly is. But, you know, as the models get better, like less is more, especially as it comes to like being able to improve through whether it's machine learning or um just you know regular maintenance um i think certainly we found that you know we're finding that less is more um i think that you know that we were talking about a little bit before we started recording like the um like rag right and and like you know co-based and next thing and all that stuff and you know it seems like you know not just for jewels but kind of across across the industry um that like agent-based search right like it's you know maintaining embeddings is hard but getting the chunking right is hard like in terms of like the the black box aspect you mentioned um a lot of that is is hard to improve upon and and uh yeah i would even say it's maybe not even hard so much as though it will never be good yeah well tell me more why do you say that never because like a chunk that happens to capture the thing you're looking for um you know will will fail to capture something else and so if you only retrieve based on like your embeddings of a chunk like it's it uses very arbitrary boundaries that are drawn like with like some hope of like some semantics being captured but you could just throw attention at it yeah and you can scale probably much better using grab like totally so i think that's you know that's an example of of you know in these these harnesses how like they're simplifying you know yeah well i haven't abandoned it completely because one of the things that we were doing i don't know if you saw the cognition sweet grep uh work was basically using semantic semantic surgery and chunks in and with embeddings as a tool uh but on the same level as the other tools like to wrap and file access and and glop and whatever else uh other variants you have uh so i think like that yeah i mean that that makes sense like don't abandon it just just don't reify it into like the only way to do things exactly and to be clear like you know this is an area of research we're doing tons of work on um and you know i actually expect you know uh in the coming months we'll we'll be talking about some stuff we're doing here too but it's um it's yeah it's it's not the i feel like when we started it was like rag it was like embedding based rag yeah it was like the thing everyone did and it's interesting to see how it's people ask me for like where are the code embedding models and you know i pointed them to like a few like Chinese ones.

13:38There are some like Nomic was working on one. And then like we found that we didn't need them. Yeah, exactly. Very bitter lesson. So, you know, I think like these are good things. I think like when Jules came out, it was kind of a preview. I mean, like the trusted testers group. So I got to see a little bit. But now it feels like more of a real product. What's that transition like? Is there a process in Google Labs to promote things when you feel like there's some traction? Yeah, absolutely. So I think the Google Labs is not, you know, about just experiments, right? So like, you know, notebook elements, we talked about it.

14:12It was not very serious. It was an incredibly successful product. It was a real money. It's real, yeah. And for us, IL was kind of a little bit of a turning point. So in May, when we announced Jules, you know, it was like great reception following IL. And that was a real moment of us to like turn this into, you know, very much a real thing. I mean, it's something that we were, you know, we always intended to, it wasn't ever intended. You know, I didn't, you know, talk about my journey, like, I mean, it was always a goal to build a real product here. And, but for us, that, that was kind of a very key moment, very key milestone for us.

14:49And so, yeah, now it's, you know, it's very much a real thing. You know, as mentioned in talking before, like, you know, Jules and the, you know, being talked about on the, in the GitHub keynote, it's, yeah, it's certainly here to stay. We're, we're, we're, we're excited to kind of keep building and expanding. awesome let's talk about just like coding and just in general you're uh you're coming to mcd aie code summit uh it's gonna be your first time at aie and and the mce uh what do you want to know yeah yeah yeah well tell me why why would someone want to yeah this is yeah let's turn it around oh boy this is embarrassing um so i mean you know fortunately we're in our third year fourth year now and we have a bunch of you know prior art we can just point people to and say look at our YouTube.

15:35You'll like that, you'll like this. There's some great talks. You know, I haven't been before, but I've watched the talks. There's a lot of good stuff. Yeah, and I'm proud that it features content from all labs. And basically, we are like the, this is a pattern I've seen across my career in terms of like every industry needs its focal gathering points to just like trade tips and stuff. So I've seen that in JavaScript, I've seen that in Cloud Native, I've seen that in data engineering, and I was like, probably AI engineering will need something like this and then i also the the concurrent thread to this was i went to a bunch of the academic ml conferences new ribs i see ml i clear and a lot of them like europe is 40 years old and that hasn't really changed and is very focused on academics and phd students whereas i think really you know the the transition in ai going from research to industry is that if you you gradually see a shift unfortunately less open source less papers and more products and more startups and and closed models and what have you but these people still want to share people still want to hire they want to promote their work so they need a place to to do that you can always do that at your company conferences obviously io and like github is github and microsoft is build and ignite but like there usually is one place where it's like the industry neutral thing where everyone is on the same playing field and me the best person with yeah and like honestly some people like that uh you know it's not like you're not going to be treated as like the vip and like you know you can't have to like earn your spot but like when you earn your spot i think like people give that uh that requisite level of attention better uh because you had to yeah you know let's say i've watched i've watched the videos online i kind of get a sense for this but what's happening between that for someone who hasn't been before like what goes on other than the talks like oh yeah uh a lot of uh well just logistical stuff of like invoicing and like vendor selection and venue selection and like did you know we have like five different pieces of software to like coordinate speaker logistics and booth logistics and borders and av and attendees somewhere ago i'm gonna sit oh yeah sorry but uh yeah what am i gonna what am i gonna yeah yeah so actually uh It's really weird because I'm the content guy for AIE, right?

17:51I curate the speakers. I invite them. But I actually know that the content is the least important part because all of it's filmed and we're going to edit it and post it for free on YouTube anyway. But the reason you come is because, one, you can talk to the speakers, but also you can talk to each other. And so I always say the hallway track is the most important track. Yeah. How do you get the most out of the hallway track? What's your guide? Begin of the hallway track? I don't have as collected of thoughts as I should. one i think if you have some prior history of like what you're interested in and work on so basically like the best intro to somebody is if they've seen you online before so they can skip the whole like who the hell are you right part and just get into like well yeah i saw you wrote that thing like let me talk to you in person about it since they're both here um that's way better than like who are you what do you do and and that's and that's like a very cold interaction ideally people come warm or they can come with some clear idea of like here's here's why i'm here here's here's what i'm looking to get out of this uh because if i think if you show up with like no uh real intention or if you're like in and out for uh for your thing and nothing else then you don't have the space and the mental energy for the unstructured serendipitous connections and the thing about eie at least in at least in our scale our size right now especially for the summit which is the one that you're going to um everyone had to apply to get in yeah uh so usually uh you know our first um summit we had like something like a 10 to 1 applicant to invite ratio invited spots ratio this one's going to be when it went up to like 10 to 16 to 20 something this one's gonna be 23 uh one of the 23 people who yeah yeah so so like uh yeah it's it's it's a lot But I think like, and really was trying to filter for people who would be speakers at any other conference.

19:41But like they are the top of the field. They are either founders or honestly enterprise buyers of the best companies you can find in New York. And that's another reason for our New York conference, which is we're bringing kind of the best of San Francisco or tech to the finance sector, really. uh there there is a little bit of media but mostly finance and like yeah that's that's great like i mean i i think so what i'm trying to say i guess is you're there to meet the other people so make time to meet them have a calling card like who are you like like a quick like what who are you what do you do what what can you help with what are you looking for help for that kind of intro stuff is really good going with friends is really good obviously like we actually offer we uh for the waltzware we offer bundle discounts this one i don't think we do uh but just reach out if you need something uh but yeah i mean like uh i think like the idea of getting immersed in the code agent community is really important uh we and i think maybe the last part i'll bring up is that we themed it for the first time right so you used to be these are just generalists here's the state of ai the best because we can get at any point in time but now we're really trying to push ourselves the theme everything so we have the best people in code the best people in data sets the best people in rl i want to do a mac interp one that'll be fun cool uh that one that one i'm thinking it will be in london because um the people i want to target are in london but yeah i think like when you do a summit it should be focused everyone there should have an agenda of like trying to learn what's the what's the state of the art trying to have off the record conversations with their peers doing the same thing at the other companies and who knows what could happen like that's the that's the weirdest thing like i organize the thing and i don't even know half the things that go on just because my job is to provide the nexus of people to just connect last time we were in new york there were 13 maybe 15 side events organized by people just like dinners meetups whatever around this around the summit and we encourage it we we posted and we just want people to meet up yeah i was going to ask is there a whole like off menu set of events happening?

21:51How do people know? They organize it. Honestly, if you're not scared of strangers, you should organize your own. A little dinner. We leave all the evenings open. So just organize a dinner or a meetup. Focus on your thing. We have people doing only voice. So if you want to do voice, great. If you want to do code review agents as a small subset of generalist coding agents, do that. And I think you'll find it. Or you can do AI in finance, AI in bio, whatever the particular sector might be. And I think that is honestly the highest signal way to get a bunch of people who really resonate with your thing to meet and have high bandwidth conversations.

22:36Yeah, yeah. Are you and me going to do the autonomous coding agent dinner? Well, no. My job is to float. Yeah, my job is to handshake, ask how everyone's doing, fight fires. so i i tend to just leave myself open open until you know the end but yeah it will be it'll be a sprint it's it's it's always a mad rush because then i have to do my own talk uh and uh i don't know yet i think um so far so like the last time i did this summit i was talking about how this year had to like develop as the year of agents and like it's really played out a lot obviously now the trendy thing is to say it's no it's not just a year it's a decade of agents but like this year I think agents really took off and most people got it right like the consensus was correct you don't have to be too spicy or counter consensus to say like if you worked on an agent you're probably a lot better off you probably made a lot of progress this year and maybe you can tell me how it feels on the Jules point of things I didn't see myself at the start as you're joining an agent company and I ended up doing that and but like I've gone so agent pill to the point where like people come to me with startup ideas for infra companies.

23:47They're like, what if we made an agent framework so that other people couldn't build agents? And I'm like, well, don't you just build agents yourself, bro? Like - There are a lot of these frameworks, yeah. Frameworks and infra companies. And all of these guys are just like, they're good developers with no conviction whatsoever in what they want to build. They don't know what customer they want. They're just like, we want to build developer tools so that's where we feel comfortable. But honestly, it's not that hard to actually take a stand and be full stack and verticalize in some particular agent field that you want because guess what they like the the business and the economics are are you know aligned that way and i'm not saying that you cannot make it as an infra company there's some fantastic infra companies that are sponsors and like that i admire and you know i would invest in myself it's just that comparatively those are a lot harder and like agent companies seem like they're shooting fish in a barrel they seem like they're ramping up in AR a lot faster and it seem like their margins are better so why not yeah so i mean i think for us it's certainly been the agent like as the models you know what was you talking about what is let's build jewels for where things are going and as the models get better i think it just becomes clearer and clearer that agents are super powerful you know like we have um uh you were talking about like before high context and management so all that stuff's important like we have people we had this is a funny story we we store some data for a session but it only lasts we only store for 30 days and so after 30 days uh your your session becomes locked and when the first user starts first started hitting that they were upset we're like there's no way anyone's gonna be using a single session for 30 days like maybe we're doing a single track of work for 30 days uh but just like how powerful that could be so yeah how do you compress context when you run into it yeah so we have i mean i can't talk too much about it but we you know we do a lot of the standard things and And it's also, you know, we're developing a bunch of stuff.

25:41It's an active area of research for us. Yeah. I think like, you know, just to, I'm not asking you for how exactly Jules does it. There's just a number of approaches, right? And you just have to pick one because you can't just use up your 2 million token context window. Is it 2 million? It is up to 2 million. Especially for coding agents. Because like, you know, like you're reading files. Like it's so, you're running commands with huge outputs. Like, you know, I think coding agents are a really interesting area, both product-wise and the impact they're having, but also for research. They really push the limits of, you know, what other domains are you running an agent for 30 days?

26:12What other domains are you accumulating so much context in so many turns? And so it's, yeah, coding agents are, I think, kind of a special spot of like super interesting product, impact, research. Yeah. I see the AMP folks drop the auto compaction for a handoff mechanic, which was pioneered by the agents SDK, which is basically the subagents pattern where like you spin up a subject and do a thing you don't need all their context that a subject is doing and then you can sort of come back to the main thread totally yep so yeah it's a good pattern it also has this challenge is like how do you make sure enough stuff information is going back and forth but that's the part you know the summarization is a pattern you know like kind of externalizing some of that context whether it's like writing it to you know like a note kind of thing is a common pattern so yeah there's tons of things tons of things to try and do yeah yeah I mean and one thing I do want to get more consensus about is what is the best because I don't think I've read any papers about which methods compare better it's also interesting as the models change the answers change a little bit too yeah yeah you probably know Claude externalizes too much yeah yeah how much does your work actually I feel like I switched back to Jules mode yeah yeah how much does your work inform the model creation, right?

27:36Like at the end of the day, like you obviously are a very big consumer of Gemini models, but also you are not the only consumer and they have other priorities than you. Yeah, totally, totally. I mean, I think we're lucky in kind of how we're positioned. We have very close relationships with DeepMind. So we have, and you know, coding agents are an important area. Like let's be honest, right? Like for any kind of company building models, like you can see it in all the labs, like coding agents are important. Coding capabilities are really important. Yeah, my OG image of the AIE code, I wrote something obnoxious, like code is the first spark of AGI, which is like probably true.

28:15Totally. Yeah. It's important from kind of AGI perspective. It's important from a dollars perspective. It's important for all of it. So it's, I think we're in a really lucky position. Yeah, we have, we're able to have a lot of kind of good collaboration. And yeah, but both ways, you know, like all kinds of capabilities that are being developed. and you know it's interesting it's it's a whole host of things right because you know in terms of like agi and the capability of things it's also like computer use models and browser use models and so it's it's a you know models that output code but it's also the whole suite of you know things that you want an intelligent agent to be able to do um so it's uh you know multimodal you know it's all kinds of stuff um that goes into it so it's yeah what would you want to find out from your peers at other coding agent companies because you're going to meet all them basically yeah so i think one thing and you know i don't think of this as a zero something i think this is like really like uh there's this tide that's gonna lift all of our boats and um it's we're inventing a new way to do our art right you know um and how to create good art as a software engineer and so what does that look like and how does that feel what is that you know what is the experience we want to create i think as as people working in the eye sometimes we don't do a good enough job describing this beautiful future we're creating i mean i know like you know like the ceos and heads of these labs that have started like you know writing their their think pieces on this but right um you know for software engineers like what is this beautiful future we're creating and like you know i think there's like one it's it's inspiring it makes it you know maybe less scary for for people who are who are thinking about these tools but also like you know if we can't articulate it and think about it it's less likely we'll get there right so like what is this you know great place we want to create like writing software is so hard like it's so many companies it's such a especially like big companies it becomes so challenging to manage a code base and create um and and what can we do to make you know being a software engineer absolutely incredible experience what are these you know how do you want to interact with your model how do you how are you doing things locally versus in the cloud and how does that interop and um so i think like as as an industry we're trying to like you know which is changing like we're inviting in in some ways inventing and there's this movement to you know change how we do our art um and yeah the more you know the better we can create this experience like we all we all win to some degree um so uh yeah i think that'd be one thing where it's like yeah yeah the local to cloud sync is um the most contentious or important i guess topic for a lot of people i wonder if we'll ever get like some kind of interop thing probably not but uh mac and dream tell me more about what was your dream flow here?

30:54I don't know. Start with Juicy LA and end up in Devon. I don't know. Oh, you're not between ages? It's probably meaningless. No, but like I'm not actually serious about it. Traffic to me all centric. So I think Codex or is it Cloud Code? Cloud Code Web can do this teleport where they just basically dump the entire history and you can pick it up in Cloud Code on your desktop. And probably that's the right move. Yeah. Maybe there's some more sort of elegant things, but they were first, so why not? Yeah. And actually, maybe the real thing is, maybe it's not the conversation. Maybe you don't need to teleport if the unit of, if the artifact that you pass back and forth is the linear ticket or the GitHub PR, right?

31:43So you don't need the full JSON. You don't need the full chat history. You just need to pick up where other people left off because that's how humans do it. Right, right, right. I don't transfer my brain state to you. I just tell you what I did. And then, you know, if I forgot to say something, you find out eventually. Right, right. You say like the cloud agent dumps some kind of summary onto the ticket or whatever kind of it needs to pass on to the next. In Slack or linear and whatever. Yeah, that's interesting. I think we have, there are some patterns emerging that are like IDE, CLI, cloud, right?

32:14Like these are the pieces. VS Code extension. VS Code, yeah. Like whether it's, you know, have it like there's a like the surface area is like standardizing it feels a little bit um and um yeah how these things interop how you can kind of make this like great experience with all of those yeah i think it's really interesting yeah yeah i think like and then the other point i just want to backtrack a little bit to something else you said which is like what the the thick pieces that the yeah these ceos and stuff do uh i think there's a lot of question about the impact that coding has on the software engineer industry in general, the humans?

32:51Do we end up, do we stop hiring juniors altogether? Do we, is it actually increasing productivity or do you just feel like you're increasing productivity? I don't know if you have any take on that stuff. Yeah, it's only so. I mean, I, it's something we spent a lot, I spent a lot of time talking and thinking about with folks and, you know, I also spend time talking to people at companies and, you know i think sometimes working on these tools it's interesting to see uh it's not as like diffused this technology isn't as diffused across software engineers as i sometimes expect right there's plenty of places that i think are are not really using ai a ton uh a lot of companies a lot of software engineers aren't um that being said i'm very kind of excited about what this what the future of software engineers are like could you imagine going back to not having these tools no that sounds horrible right like that um and so so that's one aspect of it i also think um you know i don't really buy this like you know that we're not gonna hire more software engineers story i think like for a few reasons um i mean this is an example that often comes up but is it like kind of the elasticity of the demand for software okay yeah jevin's paradox exactly and you know like a lot of the cases sometimes come up as you look at like farming right and so you know there was a time in america where like the vast vast majority of americans were farmers right and then technology happens and today it's like less than one percent yeah and that's one example but the flip side of that is you know electricity which like as that gets cheaper and cheaper people just consume more and more and more electricity um and you know with food there's only so much food we're going to eat right there's there's a kind of a uh there's an inelastic demand for that whereas of just a very elastic demand it seems like software you know software keeps getting better and better.

34:34Like the ability, like we're creating more and more software from like, you know, obviously like punch cards through to where we are today is like remarkably different in terms of how you're able to create software. Um, so much more software is being made and software just keeps becoming more and more of our GDP, right? Like it's, it's a, um, so I'm, I'm, I'm bullish on kind of the, the amount of software we'll be able to create, how it'll be created. I think there's also something here about, you know, as, as an engineer, being able to be more productive, like encourages more investment in people building software, right?

35:05If it's, you know, the job of a software engineer can now, you know, they can do 50 % more, 100 % more, 10x more, like justifying investment dollars into projects, like dramatically changes, right? And so, yeah, I'm bullish on this idea that it's actually going to be great for software engineers, both for our ability to kind of do our craft or art, but also just what it means for the number of companies and the amount that's made and the quality of it and what we're able to do with it. So yeah, that's what Rose-Called Glasses take. Rose-Called Glasses, indeed. Yeah, I have this take on the different kinds of work.

35:46Like we were splitting up the different kinds of software work and there's a lot of commoditized work that we used to spend a lot of time on and now we can basically entirely delegate to agents. and then that leaves us ideally for more strategic important novel high risk whatever uh work deep focus work that uh you know is is something i i posted here on the uh semi-async value of death where basically you kind of need to on the in the extreme end you can delegate to async agents which jewels uh you know uh code whatever but then over here you kind of need the sort of deep involvement in understanding the code base and like not vibe coding whatever the opposite of it is.

36:26Actually, that's my talk, which is I've been thinking about this. So I tweeted out like this phrase because I think I feel it's in the air that like the term vibe coding was obviously coined by Andre and he's super influential in February. And like people have just come to kind of use it as a blank check to just YOLO on prompts and stuff and create the worst code imaginable and leave other people to clean it up. So I think people are kind of at their limits with this. It's probably maxed out in terms of popularity. But we don't have yet what's next. So my talk is really challenging every attendee, every speaker to come up with what is the aspirational good version of Vibe Coding that we can actually trust.

37:12Yeah, what is it? Well, the punchline right now, what is it? I mean, the current leading candidate is agentic coding. which is what dharmesh shah who's like i don't know if you know who dharmesh is he's he's pretty good track history when he's naming things uh it's just too many syllables i don't think it just has the it doesn't have the joy that that vibe coding invokes which i think people want but then people also want care and craft and like reliability and all that stuff that but if we don't have the term i could describe it maybe we don't have to catch this phrase for it but what is what What does it look like, even if we don't have the phrase?

37:48Yeah, that's a great question. Well, we have some speakers who are going to be pitching spectrum and development, that you have to really be thoughtful and effectively write a PRD. I think that is obviously correct in terms of, basically, it's just a glorified prompt, but a very, very, very good one. And models are tuned to follow your prompt, for good and for worse. If you prompt sloppily, you're going to get slop. so a spec sounds good i think uh i don't know how often it'll be followed in practice because effectively what that transitions us to is a waterfall development approach where you spend three days writing a 50-page document and then you kick off the agent that doesn't seem right uh so like you know obviously i i have some bias here because uh cognition has from the start believed in interactive planning where like you kick off a thing you get some feedback then you're like you're like oh that's not what i meant let me correct myself because i don't know what i wanted when i when i started so you you work with the machine to discover what you wanted and the machine works with you to either get you what you wanted or show you the errors of your ways and and then you correct it from there yeah i mean one thing we talk about which very line is what you're thinking is like they're kind of like two problems as these things happen one is like how do you specify what you want and the other one is how do you verify that what you got is what you're yeah yeah and so um yeah whether it's you know specifying through a spec or this like you know interactive plan or whatever it is but then yeah and then on the flip side with the vibe coding thing is you might specify but you never come back and verify right you're like you're it's more hands off the wheel like maybe i'll click around the app a little bit and see how it works but it's i'm not really engaged with the code so how do you yeah how are you verifying and making sure that it's you know to my knowledge you guys don't emphasize tests that much right it's not like you volunteer to write my tests yeah i mean it depends like we um if there are tests in your code base um it's right it's right out of the picture here and jules will run your test feed exactly exactly so um but it's not like it's not like you know after everything everything must have a matching test to the prompt that was mentioned you know that would be the extreme of what we mentioned people always want i mean maybe it'd be helpful to do that to kind of show that it was right but let's say i don't write tests in my code base like i want to merge that pull request that is introducing tests just for this one thing like you know i think in some ways the the the engineer should be able to control what what kind of outputs they want yeah if it helps and they want it you know absolutely um and then do you think there's other innovations on specifying apart from just chat oh totally totally um i mean agents md yeah i mean uh the spectrum development i think is in this this category i think um one of these is like multimodal right like you know if i'm going to show you a bug on our website like do i want to come and like type it with words to describe it or am i going to point yeah the picture yeah um and so you know with jewels you can upload images now um but you know kind of more you know we have certain ways we communicate as humans that are easier in certain situations yeah but let's bring that to to our engagement with with the agents so of all people i expect you guys to be best at this because Gemini has video understanding.