Breaking down Bard with Google’s Jack Krawczyk | E1752

0:00I feel like I could be a coder again, you know, like go back to like the 80s and 90s when I was doing Pascal and basic and I'm like, well, why can't I do Python coding? It doesn't seem that hard when you have this assistant. You know, it's, it's removing friction to creativity. It's amazing. It's magical. And it's a time to be bold and to be responsible, right? Like we talked about it a ton. You can release things to generate that speed of insight, but you can't generate that speed of insight at the expense of trust. This Week in Startups is brought to you by LinkedIn Jobs. A business is only as strong as its people and every hire matters.

0:35Post your first job for free at linkedin.com slash twist. Hampton, are you a startup founder or CEO seeking expert advice? Join Hampton, the private, highly vetted community for high growth founders. Get invaluable insights, connections, and support to accelerate your business at joinhampton.com slash twist today. And Meowtel, traveling soon? book a local and insured sitter on Meowtel, the number one cat sitting app. Save$25 on your first booking with code TWIST25. All right, everybody, welcome back to the show. We're really, really delighted to have our next guest on. Everybody knows, my Lord, AI is moving at an absurd pace.

1:23And I said, you know, ChatGPT, really fascinating. But then I started playing with this Google Bard and I was like, holy cow, Google's got some great product. And then they started dropping product. Boom, boom, boom. Lots of product, really fast. And hey, the race is on. We've got so many players right now putting out so much great product that I decided to invite the team from Google Bard on the program. Now, listen, it's hard to get big company people to come on the show. Sometimes they're working on stuff. You know, they keep it close to the vest. They don't need to come on podcasts and talk about it.

1:56But I saw Jack Krofczyk on Twitter, and he's a senior director of product at Google, and he's working on Google Bard. And you were sharing, Jack, on Twitter, a bunch of stuff you're working on. So I guess you're showing your work. Google is now ready to talk about what you're building and how it's going. So welcome to the program, Jack. Thank you for having me, Jason. So listen, you've been an entrepreneur. You graduated Carnegie Mellon. You were an entrepreneur in residence at Excel Partners. People don't know what that is. If you're a really good entrepreneur, sometimes a venture firm will make you an EIR.

2:30Basically, it's like an option. Maybe they get first shot investing in your company. It's not like a formal thing. And you were a VP of product at WeWork, but now you're at Google. Tell me, what are you working on at Google? And is this like a strategy now for Google to get out there and talk about the product a bit more on the social medias? Yeah. So, well, what do I work on? I work on Bard. I've had the great pleasure of being a part of the team since the beginning of working on getting the experiment out there. And the reason that we're putting this energy behind getting BART out there as an experiment, knowing that it has these profound capabilities of helping people be more creative, be collaborative, while it has its limitations that we're all familiar with, with all large language model technology.

3:19It's been under development for many years. We've talked about Lambda, you know, neural networks to transformers to where we are today has been a large evolution. And you can't build technology that's helpful for people and aligns with their values without getting it in their hands. And so what you see with BARD is direct access to engaging with a large language model. And we felt it was critical to launch it without making a promise of, hey, this is a product because we don't believe that the technology as it stands today and direct access to a large language model should be a product. It needs a lot of work and iteration.

3:59And the BARD experiment is here to get the feedback, start the conversations and continue to build and iterate and apply those lessons to not only making BARD better, but across all of Google's products. and google's in a different position than a than a normal startup you have a huge reputation on a global basis you have six seven products uh at google that have over a billion users when google produces something two million whose god yeah some of them have two billion yeah you got youtube you got chrome you got android you got gmail you got docs i mean there's a lot of products out there with a lot and then of course search um so you look at all these products and you have to be more thoughtful about releasing experiments because hey the google brand the youtube brand the android brand it means a lot to people and so you are being very thoughtful i see in the product design about saying this is an experiment check the work and where we want you to give feedback so maybe you could talk a little bit about how you do that on a product design basis to make sure people don't say oh google's great i trust google for my searches it's you know 98 times out of 100 the first link i click answers my question or the one box answers my question now you got something that can hallucinate it can you know drift off it's it's this is a very early product um and it's probably going to take a couple years for it to be as good as a google search is so maybe talk about how you address that on a product basis yeah i mean so what you touched on at the beginning there is you know we talked you've heard google's mission statement out there for many years, organize the world's information and make it useful and helpful.

5:40And under that are three values. Respect the opportunity, respect each other, and respect the user. And to your point, to respect the user, you're spot on. We have six products with over 2 billion users, 15 products with over half a billion users. And so you have to be responsible in the way that you roll this technology out. But you also have to respect the user and listen to what the community is saying. You know, it's not there where, you know, everyone, you know, my parents in Florida are, you know, going deep on using this technology yet. It's not there. It's going to get there. And to build that opportunity, we have to get this in people's hands.

6:25And to your point around making intentional design decisions, so we're trying to take a bold approach by putting the technology in people's hands and pair it with the responsible approach. So you see a disclaimer at the bottom of every entry point, barred may display inaccurate or offensive information that doesn't reflect Google's values. That's one point. You mentioned hallucination. We know that's a challenge of large language model technology. it was a very intentional decision to put that google it button there at the bottom of every response that you see ah yes so when you're at the bottom you give it a thumbs up you give it a thumbs down or you just google it so you know hey listen bard is an experiment but if you want to google it you can get that hey 98 times out of 100 hits the bullseye experience or whatever the number is but internally you had to have a big debate about this some people are let's face it you know going to be perfectionist some people are going to be hey let it out in the wild um how what's the what's the climate like at google now because it had to be absurdly frustrating to have so much product being worked on and uh you know young startups can release whatever they want chips fall where they may you know they don't need to worry about what people think of them because people don't even know where they are there is nothing at stake there's no two billion user products so talk about like how uh the vibes inside of google right now people must be stoked to have it out there but there were people who said maybe we should hold this for a while and how does a company like google come to this decision of how you're going to navigate release don't release you know and then we'll get into some more granular tough edge cases i mean if i could put it in one word it's infectious it's just so exciting to come to work every single day to be at the forefront of what feels like the next paradigm shift in technology, this platform shift, like the opportunity that we have with these large language models, with AI, what it's producing is this ability to collaborate with it.

8:28It's not just about retrieving knowledge, it's about creating it. It's about having an abstract idea, turning it from an idea in your head to a concrete idea, getting more high quality ideas in less time, building, compounding, moving quickly. And we'll walk through a couple of examples later of how I've used it, demo some of it, some of the things that aren't necessarily clear. But one of the things that comes out is you think about getting this technology out to people. There's a natural evolution that people take when something that's profoundly new comes out that they want to use it like the thing that's most familiar.

9:06Which would be search. Whatever it may be. Take any technology. Take TV, for example. What was the first TV show? People reading a radio script and just like you had a radio script, you weren't taking advantage of the visuals, right? Yeah. And like you take what's familiar and then you build and compound. And one of the things that I found super inspirational of how people are using Bard is the depth of the conversation that they're going to. They're talking to the product or to the experiment as though they're talking to a teammate. I think your terminology is bestie. It's like you're talking to a bestie, right?

9:45Like you're having, you're not putting in five, six, seven words, like do your magic. Like, of course, people are going to do that. The magic gets unlocked when you're at 30, 40, 50 words, you know. I'll walk you through an example of like how I got to a 200 word prompt that like is now part of my daily process that like couldn't have existed before. And so as you bring this technology out, right, like you got to listen to people going back to that Google value, respect the user, respect what they're looking for. They're looking for this collaborator to help them have these ideas and build and grow and compound on them.

10:20It's pretty fun and exciting to watch. LinkedIn, it's amazing. I told you they get to a billion users and they just crossed 900 million people using the platform. So just think about how many insanely qualified people are on LinkedIn right now looking for an opportunity, looking to work with you. And you know the stakes are high right now. You need to have every single position in your company firing on all cylinders. You've got competitors. You've got runway. You've got customers who are demanding new features, demanding customer service. So go post an open role on LinkedIn and be 100 % certain that you have access to the most qualified candidates available.

10:57And guess what? First one's on me. It's on your boy, J. Cal. That's right. LinkedIn.com slash twist. LinkedIn.com slash TWIST and post your job for free. you got nothing to lose everything to gain, including getting that purple ring around your profile photo that says you're hiring. And then maybe you start doing some updates in the feed on LinkedIn, maybe you start following people doing endorsements of people. Once you get yourself integrated into the LinkedIn network, it's so viral, there's so many qualified people that they're going to click on your profile and see that you're hiring. And then friends over friends, friends of friends, friends, those candidates are going to come and they're going to apply for your jobs, right?

11:33And so better candidates, faster, better candidates, faster, better candidates, faster. That's all I want you to remember about LinkedIn jobs. You're going to get better candidates and you're going to get them faster. LinkedIn jobs helps you find qualified candidates you want to talk to faster. Post your job for free. LinkedIn.com slash twist. That's LinkedIn.com slash twist to post your first job for free on your boy, J-Cow. Terms and conditions do apply. I have to say, you know, one of the things as a content producer my whole life, former journalist and running content sites i have some deep concerns about how you know content creators content is used one of the things i noticed um you know google has always had a delicate relationship with the people they indexed uh robots.txt hey if you don't want to be in the index you don't have to be but if you are in the index we're going to need some information and you've over time literally decades navigated the relationship with content creators trying to make it fair putting aside legal issues you and i are not lawyers or i don't think you're a lawyer but trying to be fair with content creators so you can have a working relationship same thing on youtube hey you're a content creator i put your somebody else's music in my video you have content id i can take it all these kind of clever ideas i'm just sharing my screen here um uh and you know i was doing some searches while we were i don't know can you see my screen okay yeah oh Well, look at that beautiful image.

12:54Well, you know, I was like, hey, where should I eat sushi when I'm in Los Angeles? And it gave me all these great places. But I noticed it was starting to, when it puts in the thumbnail images, that's something very new, putting in the captions and linking directly to the source material. So now if my content was used, at least I can get a potential click. And of course, you know, this could be highly clickable stuff. but my lord you know when you ask a follow-up question i said hey which of these is the most expensive and hard to get into is oh sushi ginza which i've been to and it is hard to get into and it is very expensive and then i asked hey can you get me some of the top dishes there and it starts pulling the dishes from the menu i mean this is next level in terms of how quickly you're making progress on it um when you're releasing features this fast what's the process of you know vetting them and having discussions because i this these images and thumbnails i don't know when that dropped but i use bard basically every day every other day this is the first time i've seen one in a search i don't know if this came out in the last week or maybe i just didn't do a search that had images in it when did images start images started two days ago three days tuesday of this week amazing i mean it's literally like we're in a time machine 20 years ago when the web came out you're moving product out every couple of days amazing yeah i mean well you touched you touched on a couple of things that go into the design and building of BART and, you know, getting back to like, why is it infectious to be a part of this?

14:23You can look at the speed of delivery. We're obviously very proud of being able to get technology in people's hands. But it's really about the speed of insight that we're focused on. Again, this is like the early stage of a long arc of technology. You talked about it. The only way that you can build and grow and generate on feedback is again to put it in people's hands of course we're not launching this without testing it uh with our with our team inside of google going through the red teaming exercises that are critical in the development what red teaming is yeah so uh red teaming is a concept uh in uh in the development of these large language models where you get a group of individuals that intentionally try to break the technology and this is a critical aspect to that point of making the technology responsible making it safe these things we are aware are capable of saying convincing things that may not be what you want to present and so they will make mistakes but there's some pretty egregious mistakes that you want to make sure that you catch before you put it in the in the hands of people and so you know what we're trying to do again is uh respect the user and testing it out but respect the user by listening to them that's the number one way to generate that respect and yeah all of these things people try to break they're going to type in some query about hitler or some horrible human being or some really dramatic event on the planet they're going to try to get it to say something it shouldn't and that's all part of what the red team does is so you're going to be aware of that people are going to type in adult search contents whatever and you you just kind of try to head it off at the pass but you can't let those gotcha moments um i did it with chat jbt4 i was like tell me something great about hitler uh and it was like yeah no nothing great and i was like well but he did create the autobot i was like oh man this is getting into a dicey area right like and you really have to think about um the person who you have to keep in your mind the person who's writing a term paper on world war ii and the person who's trying to screenshot it and try to make you look stupid and say oh look what you know bing said look what google said and then try to make a screenshot or then maybe you know get it to you know into the the popular press to kind of dunk on you um that red team has to try to really do gnarly things that's that's quite a job is there like is that a full-time job for somebody being on red team or do you just rotate in red team members and say hey you get you get to do a day of red teaming this is where it comes back to like the excitement of going back to work like yeah getting people to try to help get the product out the door to be ready get the experience experiment update to be ready and by the way we're being transparent with not only what is going out but why we're doing it why are we testing it what's the conversation that we're inviting So you can go to bard.google.com slash updates and see when a new feature is out, what's there, why did we do it?

17:27And so it's inviting that conversation and getting back to the point of respecting the opportunity. You talked about images and the ability when you click on them to go to a site. That is another element of the discussion and discourse that we're trying to have by getting this technology in people's hands. We go back to Google's core values. Respect the opportunity. We only succeed when others succeed. That's been at the DNA of this company for the 25 years that it's been around. I've been here on and off for 16 years. 2007 was the first time I stepped foot in the door at Google. And that respect for the opportunity that we only succeed when others succeed, you've seen it translate through a variety of delivery that we've had at Google over time.

18:20And again, we're inviting the conversation. This is far from a complete technology that is solved and there. there you've got to get you've got to get the conversations going to figure out how to grow and evolve and that's what we're doing and we're super thankful for conversation that is i noticed it's not hooked up to my gmail yet and this to me is going to be like the huge win you know gmail was such were you at google when gmail launched the the second product at google you know i was i was a senior in college when it launched. So I remember where I was in the electrical engineering lab at Carnegie Mellon and my dear friend Seth Feinberg, who I haven't seen in a long time, gave me my first invite.

19:09So Seth, if you're listening to this, thank you for that. It was the first invite system as well and it was very precious. I remember like literally not to drop names, but emailing Larry or Sergey or maybe both of them. I was like, hey, can I get on this? And they gave me one and they gave me like three invites i had three invites and i had to be like whoa who do i give these three invites to these things were worth a lot of money at the time but gmail was such a you know groundbreaking product because it was like oh yeah you just you never have to worry about deleting your email so just all there and you can search them and it's like whoa yeah you to do this with eudora whatever mail client you were using was just really hard um and all your mail was stored on your local drive But now, you know, there's so many insights in my Gmail.

19:49How does the Bard team and the office team, the Gmail team and the YouTube team, how does in a big organization like this, this giant armada of, you know, aircraft carriers, you know, each one of them, those 2 billion users each, those are like aircraft carriers. But you have this core platform technology. How does that work? How do you take Bard and Gmail and YouTube and then incorporate them? Cause I got really specific requests that I want for YouTube and I want for Gmail. And then you're just trying to get me an answer on like the best sushi and the best, you know, and, and solve those queries and those requests.

20:24So how is that working? Do they have their own version of Bard and they just go off to the races on it? You guys put a thousand engineers in a, in a Google meetup and talk about it. I mean, how do you execute? Yeah. Well, so first let me say, uh, we announced the ability, uh, for Bard to access tools at google io a couple of weeks ago so the good news is it's coming it's coming okay so the intention is there yes there's a couple of things that you got to get right again going back to core values respect the user if you're going to start having a language model access your email and interact with that email and produce a result the user needs to be in control Now, that's the first principle for which we're building.

21:12The user is in control, and you have to be transparent. What information is being seen and how is it being used? Those are the two guiding principles of bringing an extremely powerful technology like this to people. And it's not just Gmail. There are plenty of tools across Google that we're going to be using and bringing. And it's one piece to get the language model to produce something that's compelling and helpful. And we're very excited about that. And again, going back to that speed of insight, the thing that enables this is Bard's ability to translate between natural language and code and start to quote unquote thing.

21:49They're not sentient, but they start to produce logic results in the form of code. As it does that, you can start to ask it to write code, interact through the APIs that exist, etc. And so while we build the technology, we also have to build that respect for the user to make sure that, hey, if you're going to trust us with using this tool, we're going to be extremely clear how your most private and personal information is being used. And so we don't want to put it in people's hands before we feel like that design element is confidently in people's hands. this is going to be a difficult one because you're training the model every time people search on bard it's a training exercise so i give it the thumbs down hey i'm looking for that video on youtube of the english bulldog and the monkey playing the japanese video and it's like i got that wrong and you got to give it a thumbs down and say no that's not exactly what i'm looking for a human then can intervene on those instances and try to figure out what went wrong probably don't want that in somebody's gmail right you got to be really careful and thoughtful about that and not training the language model on yeah that's the challenge you're not like this is where again like we are at the early stage of an extremely long arc of a profound opportunity but you can't just rush these things out the door you have to ask these questions and solve them intentionally and so one of the things that we're doing as we roll these things out again what are we doing?

23:23Why are we doing it? When we believe it's ready enough for people to start interacting with it, we want that transparency. And so, to this point, one of the tabs that you see on BART is BART activity. And you can see what are the conversations that you've had. And you give users the choice of, hey, if I want to be a part of this experiment, but I feel very strongly that I don't want this information to be logged, then you give users that choice. it's critical. That's part of respecting the user and why it's been there since the day that the experiment launched. When you're the founder of a high growth startup, things are chaotic pretty often, let's be honest.

24:02You face a ton of questions, you got problems that you don't always know the answer to, especially if it's your first startup. But you don't have to do this alone. That's where Hampton comes in. Hampton is a highly vetted private community exclusively for founders and CEOs. What's their mission? Founders mission is to create the most valuable and engaged community for high growth founders. And here are why some members have already called it life changing. When you join Hampton, their membership team carefully handpicks seven other members to join your core group. It's almost like a mastermind group, right?

24:32Okay, so now you got your core group, this group is like a personal board of directors for you. They're going to provide you with advice, critical feedback, and they're going to aim to help you accelerate your growth. I mean, that's what we're all in it for anyway, right? Personal growth, we want our companies to grow revenue to grow the amount of money we raise to grow all that good stuff so hampton members run some of the fastest growing startups in the world and you probably use some of their products like morning brew blank street coffee dribble and more the connections personal accomplishments and a sense of belonging are what make the hampton membership so special community is everything you know that like here this week in startups is a community and here is an even smaller more focused tighter community for you to take a look at joining if you're ready to scale your business join the hampton community today at joinhampton.com slash twist i love this idea joinhampton.com slash twist today in terms of demos um you know they're the some of the most impressive demos are writing code maybe you could um and i don't know if you write code yourself um but maybe you could explain to me what impact this is having internally because people writing code to make ai experiments interfaces etc are now being assisted by ai itself and this is one of the profound um aspects of what we're experiencing right now we talked about how hey images showed up on google bar two days ago are developers going noticeably dramatically described before us faster in creating content now creating uh you know code now than they did when you joined google with wavy davey you know 20 years now i'm talking about dave freeberg wavy davey uh we gotta are you do you know the wavy davey story i don't i i mean i've heard the term wavy davey uh referring to bestie davey freeberg from the only podcast but you worked with him when he was at google i i met dave uh well he was in the early days of weatherbill he had been a former googler acquired a company named urchin we met through a mutual friend brett crosby saved friends and neighbors for many years and uh i'm not gonna steal the thunder but maybe this is something you guys can talk about but uh it involves a fraternity chef some funky bass and the term wavy davy so got it i'm just gonna wavy davy wavy davy yeah this is teaser for a future episode for you yeah got it well you know san francisco's you know got a long tradition in wavy davies um so let's talk about code and then maybe we get some of your demos and you can tell us like where you're at with the product any new product announcements or new new experiments that you're working on yeah so code has been live for a few weeks now when we launched the bard experiment we didn't feel like the experiment the experience was helpful enough i think it was three or four weeks after the experiment launched we gave the we gave bard the ability to code or release the ability for bard to code and it quickly became one of the most popular ways to engage with bard And I feel like a lot of the conversation is like, do, you know, write a Python script that does X.

27:36And of course, people are doing that to kind of like get them started. But there's two other things that are really interesting to watch people use coding capabilities on Vard. In addition to generation, there's the notion of explaining code. Hey, I have this code, I need to kind of figure out what it's doing. Can you do it? But again, not batting a thousand, nobody's batting a thousand, but it just helps people understand code. And the other bit on that is debugging code. So actually, hey, I have this thing. What's wrong? Like, why is it not going? And one of the features that we built in along with that was this notion to export directly into Colab.

28:17So Colab is a mixed notebook and Python ID that Google has created where you see a Python script and there's an export button at the bottom. You can export, go directly into Colab and start iterating. And so we're watching it speed up people's process. And it's really inspiring. And one of the things that left me a frustrated failed engineer that then became a product manager was I couldn't debug a regular expression to save my life. And that's actually one of the most popular things that people are doing, trying to figure out why does this regular expression not work? So the more technical people out there, I promise you there's help.

29:00It's pretty cool. you know like if you just go into google bar right now and say hey uh teach me how to write the code to draw a circle in python it'll give you the code it'll link to people who've written blog posts about this and then say well what do i do with this code and how do i execute and it's like oh yeah you need an id or you can run it from a terminal window or you can run it in this thing you're like what's an idea and it's oh that's an integrated development environment it's you know way for you to execute code and now you've put those two things together so this is where like the ability to have a guide on the side a co-pilot whatever is really going to be dramatic how far away are we from having civilians non-developers writing and executing meaningful code on the internet that's a tough one to project but when you break code into what it is code is a logic language and it's one of these things that's really amazing to see when you start to have logic build and compound, what are the ideas that you can build and grow and develop on?

30:04And, you know, taking a step outside of hard, like, you look at the, you know, the quote-unquote no-code revolution that's been taking place with other tools out there, you know, the web flows of the world, really fun to watch. I think you just start to see these intersections of people's creativity is coming up against less friction. And it's not just in code. So, and so, that's the thing that's so amazing. Like when you talk about like the opportunities that are profound, like personally, I think like creativity is one of the most amazing parts of humanity. Like our ability to imagine is magical and we're reducing the friction to imagination.

30:43And so, imagine what can happen as you get these ideas and you can go from start to finish faster. And so, when does it hit the mainstream? Three months, six months, three years? the only way we're going to know and get there faster is going through this process of getting the technology in people's hands getting the feedback and improving and iterating and anchoring back in that respect for the user make sure you're you're delivering on a trust value proposition and that's what we're we're seeking to do what's the business model going to be here do you think uh there's a lot of inspection speculation hey google bard uh you know is going to be so good at giving you an answer and larry and sergey said this early on like hey we'll know we've done our job when you do a search and we just give you an answer as opposed to the 10 blue links and that's a journey google's been on for a while hey here's surfacing some information about a restaurant review or maybe a one box with a snippet of an answer or the best sentence in a blog post and then linking you to it or here's the weather here's the stock price you know you over the decades it's just said okay what are the top searches and let's just see we can get people answers quicker um what's but of course there's you know things like shopping or local that are filled with clickable ads there's speculation now hey this is uh so good maybe people click on less ads so is there a way for bar to have ads in it uh or commercial messages that doesn't ruin the experience and the same way google has tons of ads all over the place that you know and i had this discussion with larry at some point back in the day like what's the difference between like a great ad that solves your problem and the first blue link it's like might not be much of a difference like it actually like the volvo ad might do better than somebody's review of the volvo and answering your question right so there is this kind of natural tension a great ad could be an answer a great answer might not you know might be an advertisement for the product the line blurs a little bit so how are you thinking about advertising and monetization here is this something you think people will pay for you've never had a paid product really i mean maybe google docs uh how does google think about the business model here it like goes back to the first principles again respect the user and respect the opportunity respect the opportunity we succeed when others succeed i spent 10 years of my career working in advertising some at google some at other companies like pandora and why why is advertising valuable advertising is valuable because it helps grow awareness of your product.

33:12It helps drive conversions. It's something that Google has been very focused on in respect and respect for the opportunity. And so as we're in this seat of respecting the user, we're still learning how to make these tools the most helpful for people. We're seeing them being used as, hey, this thing's a collaborator. I have an idea. I can build, I can grow, I can compound on it. I have this idea for a character I want to write into a YouTube film that I'm going to do. I can build, grow, expand. We need to keep going through those and inviting the conversation. So to answer your question directly, what's the monetization plan?

33:51The monetization plan for anything we do at Google follows the same rubric. Look to where you're generating value for people. and make sure that the way that you monetize falls in line with that. And so that's how we're going to approach it. We're focused on driving the value and respecting the opportunity means that when we're ready to fold that in, it'll be folded in. Listen, you know, I'm a bulldog guy. I've had bulldogs my whole life. But you know what? Dog owners, we're spoiled. We have so many dog sitting, dog walking options, right? What about cat people? You know those cat people in your life.

34:29Well, I've got the perfect solution for them. Meowtel. Meowtel is the number one cat sitting app. And it's available on iOS and Android. And Meowtel sitters have over 60 ,000 five-star reviews from happy cat parents. And you know, these cat people, they're very demanding. They're not giving five stars out easily. All these sitters are fully vetted and insured. And they offer once or twice daily drop-in visits or overnight stays in your home so that your cats can enjoy playtime and cuddles in the comfort of their regular environment. A meet and greet is included with each reservation so you can ensure the perfect fit.

35:05And your sitter will send detailed updates and photos to you each visit to let you know how your cats are doing so you get that security, right? It feels great when you get the photo and you're like, ah, now I can enjoy my vacation in Italy or going skiing in Tahoe. MeowTel provides a 100 % satisfaction guarantee and a team of support specialists to help make sure you have the perfect experience use the code twist 25 for 25 bucks off your first booking okay it's 25 two tens and a five waiting fee code twist 25 for 25 dollars off i mean just looking at this search i just did like hey what are some flights options to tokyo uh non-stop flights you know and google flights prints money for google tons of paid links in there but it's also the best for me it's like the fastest best way for me to find the least arduous painful flight which i define as like the shortest time you know less layovers and a really great answer here in terms of like hey here are the non-stop flights um you know delta hawaiian airlines whatever or canada and the flight times um but it doesn't incorporate google flights yet or travel whatever that's called the that gives you like the flight information um and but it gives me some like hey you might want to use this flight search engine or consider a budget airline stuff i didn't ask for but here having links at the bottom to you know uh offers here's a 25 off or here's a link to whatever airline you know those would be quite valuable to me and i wouldn't mind seeing them in here and i would probably click on some of them so is that like you think a potential path and have you started to have those discussions the potential paths are endless which is part of what makes this so fun.

36:49Like, I'm going to go back to like, why is it infectious to be at work? Like, there is no shortage of exciting ideas to run toward and build and execute. And so, one of the things that we are excited about is this notion of bringing tool use into BART. How can you get it to a point where as you have interactions like this and Kayak, one of the partners that, you know, we're talking about bringing to the full, how do you make sure that the experiences that you're getting are trustworthy are reliable are getting there and that's where you need to focus now where we need to focus now making sure that the responses that you're getting are helpful and can be compounded on because like i'm assuming uh you might maybe have someone that helps you book your travel maybe do it myself now but i did have that for a while i like doing it myself now because i'm so good at it and the tools are so good well in the part of when you had someone help you with your travel if you said get me a flight from point a to point b how many times would that person come back to you with the perfect answer and the perfect itinerary yeah it depends on the person i've had you know the person i have now who will do it sometimes for me gets it almost perfect every time after years of working together and knowing i like google i like that jeff blue mint you know i want to get my united miles let's use my united miles yeah yeah no no but there is a whole like heuristics of what i like right so i think what you're getting at is hey google bard's going to understand that and maybe if i like kayak or i like google flights or whatever or if i have preferred airlines that's all going to be known by my bard by my google assistant look to project where we're going to be in n months of time would not be a responsible respect for the user right like we are focused right now on what is the feedback that we're getting that we need to improve upon like one of the things that we heard why why did images just come out a couple of days ago it came out because as we launched people were starting to say like hey this is a like i'm talking to it like i'm talking to a teammate that's helping me think through an idea and i want to visualize these things sometimes but like memorial day is coming up, I'm guessing this is going to happen after Memorial Day, like we're all having barbecues.

39:05Yep. Well, maybe not everyone, but many people are going to have barbecues. As you dream about that long weekend that you're about to engage in, like having a little bit of extra visual stimulus makes it a little bit more fun and more enjoyable. And so, that's why you get something like images and responses. And so, we're going to continue to build, compound and just keep listening to how people are responding, what they want to do what they want to grow and iterate and of course we want to be helpful you got tables now that was a big one for me yeah i mean i was like constantly doing searches i'm like put that in a table and it was like yeah i don't know i'm a google ai assistant i don't know if i can do that and then one day it's like of course i'll put it in a table for you it's like well and then it's obviously hey put this in a google sheet for me and then it's you know now we're starting to get really interesting am i in the bard interface and then exporting to google sheets or am i in google sheets doing barred searches and then plugins obviously you announced plugins explain to the audience like for a startup i got a startup that's a you know meow towel and i'm doing babysitting for cats uh it's literally a cat sitting uh marketplace that we invested in called meow towel how would a meow towel build a plugin for this is it when is when are those going to be available what's the roadmap there because you might really prefer a chat interface like this to navigate something like meowtel or we have another um marketplace called gigster that is like locations for films and you know photo shoots an interface like they're all of those companies are going to start building their own interface on their website but then it's like well wait a second if google has this can we do a plug-in and then can that plug-in exist on my site the way google ads or google search you had a product back in the day where you could do google site search so talk to me about the vision google has for plugins and for startups to get to leverage this technology because we're here on this week in startups yeah i mean the the developer ecosystem is one that uh has been core to to google for many years there's obviously android chrome has an extension framework and part of as we're building tool so we're calling tool use and art as extensions, because we feel like there are some similarities to what you've seen with Chrome extensions, where, hey, I have this idea for an interface that can be helpful to help you interact with the mode that you're in.

41:26And so we want to continue to build and grow and evolve and listen to how people want to interact in a conversational way. And again, the way that we're building it right now is building all that core principle like we only succeed when others succeed. So we need to make sure that what we deliver is going to be helpful. And the initial set of tool use extension partners that we have are really focused on a getting it right as you bring it to users, and be making sure that it is helpful for the people that are trying it out. So we're thankful to have partners like Instacart that are joining us.

42:01Repplet is coming soon. i've been playing with replet i basically did a whole thing where i was like i started their python like hundred day course so replet you're going to be able to have bard inside a replet or you do your bard and then export to replet you're going to be able to export to replet so again code one of those things that people love to use uh on the uh on the bard experience and so uh we heard we're again listening listening to people uh hey like this this is a good way to get me started but i want to continue i want to grow i want to develop let me export this to Can any startup build a plugin now or is that not open to everybody yet?