How to use Microsoft’s AI Copilot, laptop microphones, and Amazon Sidewalk

26:08But matter will mean that you can easily switch between them. So just like you have to choose a cell phone carrier, you have to choose an ecosystem, but it should be pretty easy to switch. Yeah. And I think the Belkin thing is so funny to me because I read that whole announcement And it just comes off like the most epic self-own of all time, where it's just Belkin being like, well, we can't figure out how to make good, interesting products that are better than anybody else. So we're just going to stop and we're going to force you to buy our products because we have pretty good distribution and we sell our stuff in Apple stores.

26:43So we're just going to con you into buying our crap that doesn't work with anybody else. Because if we did the thing everybody else is doing, you would just buy cheaper versions of it on Amazon. I still think it's mostly a cell phone, but I do see at least a little bit of Belkin's point here. I mean, the initial reaction was, oh, bad Belkin, but it was kind of brave and kind of like, OK, yeah, we're kind of in a rock and a hard place and we don't really know what to do. So we're just going to stop. These companies in the smart home are scared of commodification. That was always going to be an issue with matter, like from the beginning.

27:16But commodification is actually what we as consumers want. Yeah, it's great news. It's good. Yes, we want I would like Belkin to produce Wemo products that work with everyone and only cost$5. Yes. But just turn on and off and just give me the basic features that I need. I mean, the smart home is a natural evolution of the home. So eventually, when you go into a Home Depot or you go to Amazon to buy your light bulb, you don't need to see whether it works with one voice assistant or this protocol. You just buy it like you would any light bulb and it will be connected to your home. That's where we need to go.

27:51One undercurrent of the stories you've been writing about this stuff is that Apple seems to be, if not the holdout, then at least like one of the more sort of complicated parts of this strategy that Apple obviously has this big focus on security that has made it harder for companies to work with other stuff and also HomeKit over time. And Apple also, I think, has a long history and vested interest in things not being super interoperable. What is Apple's role in kind of this moment as we're making this transition? Like, is it the stick in the mud? Is it moving stuff along? How is Apple doing here?

28:27I think it's been very interesting. Apple is probably the most vocal supporter of matter of all of the ecosystems. But vocal in Apple's way in that they never actually say anything. Right. Right. So that's why it's been kind of hard to pass. But from talking to people who sort of are in the discussions and sort of know what's going on behind the scenes, basically HomeKit was not really a success for Apple. It was complicated. It was hard. So to them, Matter is the solution for the smart home for them. That's why they're sort of phasing out HomeKit. They've rebranded themselves. They're now Apple Home.

29:04And they want everyone to work with Matter so that they can work with Apple. They don't want to have to deal with this separate ecosystem anymore. But they're a stick in the mud in that they keep their cards close to their chest and matter requires openness so that developers can figure out how to connect. And this is, from what I understand, one of the holdups with the Google Home and the Amazon Alexa apps coming to iOS. Apple has changed things every time they release a new iOS update. And then there's also some struggles, as we've seen with 16.4 that just came out, which is the first time they've brought back the new HomeKit architecture.

29:42There was the new architecture and then the old architecture, and now it's the new architecture. Now we're back to the new one because the rollout of the first one was kind of a mess. That's a holdup. So I don't think Apple is stonewalling anything here, certainly not on purpose. I think it's a complicated process for them to kind of untangle themselves from HomeKit and reopen their platform to Matter. And that's what they're trying to do. And again, this is why all of this is taking so long. Crucial parts of the infrastructure just have not connected. Again, six months, they launched in November Matter.

30:16They probably should have launched it this summer or next November. It is going to be a long time until I come back on the Vergecast and say, hey, Matter just works. It's going to be a great day. That will be a great, great day on the Vergecast. All right. We got to move on. Jen, thank you. Appreciate it as always. We'll see you again to Yellowbat Matter in six months. Sounds good. We're going to take a really quick break and then we are going to come back and we're going to go into New York City and test some laptop mics with Monica Jen. We'll be right back.

30:56Support for this show comes from Shopify. One of the most exciting parts of starting a business is when the sales start coming in. If your business is looking to start selling online, Shopify has everything you need to create your online store. Shopify is the commerce platform behind millions of businesses around the world and 10 % of all e-commerce in the U.S., from household names like Mattel and Gymshark to brands just getting started. With hundreds of ready-to-use templates, Shopify helps you build a beautiful online store to match your brand's style. And Shopify is packed with helpful AI tools that write product descriptions, page headlines, and even enhance your product photography.

31:36And best yet, Shopify is your commerce expert with world-class expertise in everything from managing inventory to international shipping to processing, returns, and beyond. Turn those dreams in two and give them the best shot at success with Shopify. Sign up for your$1 per month trial and start selling today at shopify.com slash vergecast. Go to shopify.com slash vergecast. Shopify.com slash vergecast. Support for this show comes from MongoDB. Innovation is the key to success, but innovation also comes with its own set of challenges. And if you're a developer who wants to find a new way in and think outside of the box and think beyond legacy code, it's time to check out MongoDB.

32:24MongoDB is a flexible, unified platform that's built for developers by developers. MongoDB is ACID compliant, enterprise ready, with the capabilities you need to ship AI apps fast. That's why so many of the Fortune 500 trust MongoDB with their most critical workloads. Ready to think outside rows and columns? Start building at mongodb.com slash build.

32:53Welcome back. Every now and then on this show, we like to do a microphone test. Because we spend a lot of time worrying about our headphones and how things sound to us, but not enough time testing our microphones and how we sound to others. Last year, we stress tested the microphones on a bunch of flagship wireless earbuds to see which sounded the best for voice calls. You should go listen. It's all over the Vergecast last year. But the short version is this. The Sony LinkBuds, Apple AirPods, and Google Pixel Buds Pro are all pretty good. The$20 Apple EarPods, the ones that actually plug in, are amazing.

33:27Everything else we've tested is less good. But the thing is, not everyone uses earbuds for Zoom and Meet and Teams or whatever else you use for calls and video chats. So for this test, we're checking out how the microphones on a bunch of laptops sound to figure out which is the best for taking calls, meetings, and even just recording audio with the built-in mic. You might remember that when we tested earbud microphones with Chris Welch last year, we sampled what the laptop mic sounded like too, and it, well, it sounded like trash. You hear everything. The whole world is present. There's a truck.

34:05But things changed fast. So we had to see, can laptops handle a voice call? Can you get away with no extra gear whatsoever the next time you get on Zoom or Meet? Monica Chin is The Verge's laptop reviewer, and she helped us stress test a few of these laptop microphones. So while I sit comfortably at my desk at home in Virginia, we sent Monica with our producer Andrew Marino to the single most chaotic spot we could think of in New York City. So we are walking through the middle of Times Square, which is probably the loudest place in New York City that you can go besides a concert. What you're hearing there is the 2021 MacBook Pro.

34:42It's the 16-inch one, the one with the M Pro chip. This is like the laptop that our video team really loves to record stuff on. The mics are pretty good, I would say. How does it sound? I cannot believe how good you sound. This is blowing my mind right now. Really? Well, I'm used to these microphones being like really good, but they catch everything, right? So you get tons of background noise. You get tons of everything. We could just have a video call with you doing this. We could, except that I can barely hear you. There's that. An important detail I should mention here is that Monica did not wear headphones for this test.

35:17We found that a lot of people don't have wired headphones with them when they're out of their house. So you're going to hear a lot of my voice being picked up in the laptop's microphone. Which, spoiler alert, is actually one of the biggest takeaways here. Mics are getting better, but echo cancellation mostly still sucks and causes a lot of problems. For the mics themselves, though, the biggest problem we came across with basically all laptop microphones is when the wind picks up. Really no surprise there. Okay, so we're getting to the most iconic part, which is the big TKTS booth where everyone hangs out.

35:51Look, this is probably the least realistic test we've ever done. Not many of you are walking through Times Square holding your laptop in front of you. If you are, I have a lot of follow-up questions for you and feel like we would be very good friends. but I am sure that you've tried to take a call in a crowded coffee shop or at a park in a city or in your backyard with airplanes flying over. Not that long ago, I would have told you that doing that and trying to talk into your laptop mic was a horrible idea and you're going to hurt the ears of whoever you're talking to. But I kind of couldn't believe how easy it was for me to hear Monica so clearly.

36:29It's not amazing, but it works. Okay, now tell me what you had for breakfast this morning? This morning, I had a cake pop from Starbucks on the way to work. Part of this balanced breakfast. Yeah, exactly. I'm not a breakfast eater really usually. So that was like a breakfast like my dessert from the night before basically. Okay, so that's the MacBook. Next up, Monica pulled out the Dell XPS 13, which has long been our go to Windows laptop recommendation. And it held up pretty well here too. This is one of the best Windows laptops you can get. but it's obviously a good bit smaller than the MacBook Pro 16.

37:06We got an Intel processor inside. We got a teensy little webcam on the top. We got downward firing speakers on the bottom. It's a much smaller, much smaller little deal. Yeah, so I'm hearing more and more detailed background noise. Like I can sort of pick out individual people talking instead of just like the whooshing background sound that was in the MacBook. But even still, your voice is coming through pretty well. I can pick you up fine. It's definitely more processed than the MacBook. Interesting. But even so, I would rank this in mid-range headphones to be using in a crowd of people. I'm blown away by how good this is, in all cases.

37:43One of the big things with Intel's 12th generation, which is what's in this, and the 13th generation as well, which just came out, is they make a big deal out of AI noise processing. So it makes sense that it would sound more processed, I think. Yeah, it's definitely working hard to do it, but it's doing a pretty good job. Okay, prepare yourself for this one. The next laptop Monica pulled out, she didn't tell me about before we sent her to Times Square. And when she joined our video call, it just had a sketchy name like Surprise Laptop. It definitely surprised me. Okay, that was literally just caused physical pain in my headphones when you did that.

38:22It's awful. I hear everything. okay so this is the gateway 14 is a 400 laptop that i picked up at walmart because i was curious if it would work at all that was the first one in all of these tests that we've done that was like actually physically painful as soon as monica entered our chat here and i started hearing sound i like literally did the thing where you have to go like ah and just pull your headphones off because that's what happened. That was awful. Before we go any further, I should say that the way we recorded this was the same as our other tests. We use an app called Riverside FM, and we like it because it records your local audio and then uploads it.

39:05So you don't get any of that internet artifacting or echoes. It does mean these tests are going to sound slightly different from what you'll sound like on Zoom or Meet or Teams, because those apps all apply their own kind of filtering and noise canceling, this is much closer to like the pure audio coming out of the microphones. That said, that gateway laptop was worse than just the microphones. In addition to sounding like a metal band playing inside a scratchy tin can while a lawnmower goes full blast right next to my head, the computer itself could barely even keep up with having that one Chrome tab open right out of the box, and it initially lost the recording because the computer literally couldn't keep up.

39:45The point is, it's not a very good computer. I wouldn't use it for this or anything if you have another option. Okay, let's get back to better sounding things. The next laptop we tested out was a Chromebook, the Lenovo IdeaPad 5i gaming Chromebook to be specific. Lots of people use Chromebooks, especially in education, and we live in a remote world and lots of people take classes online. And this actually seems like something you'd hope Chromebooks specifically would be good at. All right, we are here. This is so much better. We are on a 15-inch gaming Chromebook. Where does this fall in, like, the range of Chromebooks?

40:19I mean, it's not terrible. It's got a little bit of a plasticky build. But, you know, it's got a big screen. It's got a nice little numpad here on the right side of the keyboard. The audio is way too quiet to be using in Times Square. But, you know, if you're going to be using headphones all the time, then maybe that doesn't matter so much to you. This is a good middle ground of, like, the good ones that we heard at the beginning and the horrible garbage one that we heard before. This one is, like, I can hear a bunch of background noise. there's definitely more wind and stuff coming into the mic than before but i can still hear you and if we had a call like this i would not hate your guts whereas like three minutes of that last one and like i don't want to be friends with you anymore could you hear there was a man rapping in front of me just now but it sounds like it wasn't that wasn't quite coming through uh it does seem like i'm picking up a lot more behind you than behind the computer which is interesting i guess yeah the microphones are facing that way so this is actually a good example because as i'm looking at you, a crowd of people just walked up behind you and I could hear all of them talking.

41:16But then when somebody walked right in front of you behind the computer, I didn't hear that at all. That's very interesting. Interesting. So I guess if I'm doing a call in Times Square, I want to be sitting at the back of Times Square. Yeah, like face your back to a building or something. Yeah. Laptop number five also falls into the probably ought to be good at this category. It's the Lenovo ThinkPad X13. It's kind of a classic business laptop, but this one is actually built with mobility in mind, which is all the more reason that the mic should be good. This is a Lenovo ThinkPad. It's super, super light.

41:48And it actually has a Snapdragon chip inside. It's powered by the Snapdragon 8CX Gen 3. This is one of those computers that's like trying to kind of be a tablet and it's like very mobile on purpose, right? So it should be good at this. Super, super thin. It's like, it feels like it weighs nothing, but I don't know how good the speakers are. It's very, very thin. So there's not a ton of room for great, great microphones necessarily. It's doing less processing than most so like your voice actually sounds really good. I'm also picking up a lot more background noise. Can you hear? There's like a party happening down and a bunch of people cheering.

42:24I think oh, people are screaming at Mickey Mouse down the block. Can you hear that? No. Spin me around. Let's see if we can hear. I think Mickey Mouse. There he is. Yeah, Mickey Mouse and Sonic are walking into the Levee store and people are very excited about it. But I can hear you much better as well which is a big relief. Okay, let's wrap this up. One more test subject. Let's go back to Apple. That first one we tested, we kind of knew was going to be the best mic of the group. But this one's probably the most common laptop you're going to see out there. This is the newest MacBook Air with the M2 chip.

42:58It's the laptop I just tell everyone to buy because it's great and you'll like it fine. But it turns out having a better mic array in the Pro really does make a difference. We are here with the M2 MacBook Air. tiny little tiny little guy and it is about to get a lot louder because we're going to walk to the subway station how are we sounding on this one this is i would put like right in line with the thinkpad and dell like that first macbook is clearly better than everything else oh interesting because that's one of the things that you know sometimes you wonder like how much of the microphone performance is the physical hardware and how much of it is the software and presumably these are using the same software.

43:38So it sounds like the hardware does make a difference in this case. It must because that one is like a one order of magnitude better than this. Like this is pretty good. It's kind of like the ThinkPad we were just on in the sense that it's like not a ton of processing, which means I'm getting a little more background than usual. Yeah, this one is definitely harder to hear than the 16 as well. The 16, the MacBook Pro 16 was the one where I really didn't have to lean in at all. It is impressive to me being in an environment that's so loud and knowing that these tiny things are filtering out all that noise around me like i would not expect it's technology man we should say as a general rule don't do conference calls from the middle of times square but i guess it's encouraging to know that this is a thing you can pull off if you absolutely need to it sounds like if you're on the 16 you can probably do it just don't do them on don't do them on uh something that only costs 400 that you picked up at walmart and is blue.

44:34And is called Gateway, which is a brand I legitimately did not know still existed. Definitely don't do them when you're walking through Times Square to the subway like I am, because Andrew has had to shepherd me across the crosswalk several times. Alright, well you guys go. Don't die. Don't become the new spectacle in Times Square. But thank you for doing this. This was great. Thank you. This was really fun. Okay, three thoughts on this before we go. First, I'm still kind of amazed at how good these sound. I've typically told people to avoid their built-in mics except in really quiet spaces, because even when they're decent mics, they're made to pick up audio from a much larger area, so they tend to get much more background noise than your average headphones, because those can just point their mics right at your mouth because they're always situated in the same place.

45:25That's still true. A good pair of headphones will sound better than any of the laptops we just tested, but it's much less true than it used to be. And most of these at least fall into the totally usable in a pinch category. Second thing, my rule is that you should always, always, always, always, always wear headphones on a call. And that rule still holds. Without echo cancellation, if you're not wearing headphones, you're going to hear your own voice a ton. With it, the computer has to do a lot more processing, so audio gets worse, and you get that thing where you can only talk one at a time. It's just a lot worse.

45:59So even if you're using the built-in mic, you should still use headphones. Also, for like basic human etiquette reasons, you don't want the world to listen to your phone calls. Another reason to wear headphones, by the way, is that one of the most interesting things about our testing was that Monica could barely ever hear me. I could hear her okay through the mic, but even with the volume all the way up, she was always having to hold her head right down by the speakers, and even then, she could barely hear me. So again, for so many reasons, wear headphones whenever humanly possible. And third, the hardware really does matter here.

46:34It's not an accident that the Macs, which have three mics and are set up to do noise cancellation, sounded better than the other ones, which mostly have two or in some cases even just one mic. But there is also a fair amount of software work here, which you can hear in some of the audio processing, especially on the Dell and the Lenovo. And that's really good news, actually. Over the last couple of years, it looks like laptop manufacturers started to care about the quality of their microphones for basically the first time ever. So things have gotten a lot better really quickly. Maybe someday I'll ditch this fancy mic and just make a podcast by yelling at my computer.

47:07But here's what that sounds like right now. So yeah, maybe not yet. Anyway, we got to take a break and then we're going to come back and talk about Microsoft's many adventures in AI. We'll be right back.

47:23Support for this show comes from Figma. Figma is the collaborative design platform where teams come together to turn ideas into the world's best digital apps, websites, and experiences. And now there's a faster way to bring those ideas to life. Introducing Figma Make, an AI-powered prompt-to-code tool that helps you turn ideas into something real, fast. Figma Make is a different kind of vibe coding tool. With just a few prompts, you can turn any idea into a code-backed prototype or app that anyone can explore, iterate on, and share. And it's all connected to Figma. So you can start from scratch or from your design file.

48:01Figma Make is made for designers, product managers, and builders who want to move fast and explore more ideas. Try it now and see how quickly you can make your ideas real with Figma Make. Just go to Figma.com slash Vergecast. That's F-I-G-M-A dot com slash Vergecast.

48:46rideshare and food delivery platforms in the world. That's what Crucible Moments is all about. Deep diving into make or break moments that set the course for some of the most important tech companies of our time with interviews from some of the key players that made these companies a success. Hosted by Sequoia Capital's managing partner, Rule Off Botha, Crucible Moments is back for a new season with stories of companies as they navigated the most consequential crossroads and their journeys. Hear conversations with leaders at Zipline, Stripe, Palo Alto Networks, Klarna, Supercell, and more. Subscribe to Season 3 of Crucible Moments and catch up on Seasons 1 and 2 at cruciblemoments.com on YouTube or wherever you get your podcasts.

49:31Listen to Crucible Moments today.

49:39Welcome back. One of the questions I'm personally most interested in in tech right now is, what is AI actually good for? We've talked a ton about GPT-4 and ChatGPT and Bing and BARD, but a lot of the things that we talk about are neat novelties, or there are glimpses at a far-out future for when these products get a lot better. But the truth is, AI is much more than chatbots, and Microsoft is being much more aggressive than most in figuring out how exactly to put AI into all of its products. The Verge's Tom Warren has been writing about and testing as much of it as he can get his hands on over the last several months.

50:18So he's here to tell us how AI is going to change the way we do everything, at least the way Microsoft sees it. Hi, Tom. Hello. Welcome back. It's been a while. Yeah, it has. There's a lot happening right now. This is why I wanted to do this, because it feels like Bing has gotten tons of coverage, and we've talked about it a lot, and it's really interesting. But if the question is, what is AI actually good for? I feel like it's all the other stuff that Microsoft has done, which we have not talked enough about. Yeah, that's the promising stuff. The search engine stuff is cool, but having it everywhere is cooler.

50:49Okay, so let's start with Copilot, which I think is probably the sort of biggest version of what Microsoft is thinking about here. And I was reading a bunch of stories and realized I have a hard trouble figuring out exactly what Copilot is. It's kind of a chatbot, but it's not totally a chatbot. It's different things in different apps, but it's kind of one thing. Like, Frame Copilot for me, what is this thing? Yeah, so it's essentially, think of like Clippy or like all the promises of Cortana. It's like basically an assistant, so a virtual assistant. So yes, it can spring up and be a chatbot and you can, you know, converse with it and stuff.

51:24But it can also just sort of like magically appear almost with AI magic. in documents and like say you highlight a paragraph and you're like i want to rewrite that or reword it or whatever then it will just sort of like hint in a way that you know you know when you goof up and you get a typo and word comes along and underlines it that sort of thing it will just sort of gently appear and be like i can rewrite this for you so that's kind of the interesting way that copilot works so you can say you know create a document create a powerpoint create an excel based on a bunch of prompts or you can drag a file into a powerpoint or a word file and then based on that all that text it will generate a bunch of slides so you can do all that sort of stuff but i think the interesting part is where it just shows up or like in excel let's say it does something like you say here's a bunch of data here's you know financial earnings for this company the past six months put it into a table or pivot table or whatever or something you don't really know how to do it will then tell you how it did it and show you how it did it so it's like training you as well so it's like it's an assistant but it's also a tutor i guess as well so it's like there's a bunch to it really that's actually really clever because i think one of the things i was looking at as i was reading your story is like the sort of famous office problem over the years is that office is this like hideous mishmash of menus because there's so much that all these apps can do that it's impossible to figure out how they work and microsoft is like we made the ribbon better and everybody's like all my buttons are gone and like it's just it's an impossible problem.

52:53And it feels like if they can get this right, like there's somebody at Microsoft being like, we can finally get rid of the ribbon, right? Like that's the has to be what's happening here. It's the complexity of the interface, right? And just Excel's the worst for this because it's just like, you can open it up and it looks relatively simple, but the stuff that you can do with it, like the difference between me using Excel and like some power user is like, you know, massive. And it's just reducing that complexity and making it more available to people and just easier to use i think that's kind of like half the story behind copilot and the other half is like yeah you can chat to this thing have a back and forth get it to do stuff and you can say to it like you would in chat gpt really and bing and stuff you can say like write this and you know you can prompt it afterwards to tweak it and make it better or make it into a list and all that sort of basic stuff but it's really interesting where it takes that complexity out of office apps and then you know just surfaces it in really interesting ways and i think they're really at the beginning of that so there's not like a ton of examples yet so i was trying to like kind of get that out of them like you know where is this heading but i think for me it's like it sounds like it's just going to be the virtual assistant they've always promised um all the beginnings of it at least because they tried they said that cortana would do this stuff right well that's part of what i think is so interesting here because there's i mean you go back and like you mentioned clippy and there's cortana and there's like one of two things is happening here either microsoft has been right all along and finally the technology is starting to catch up to kind of this thing that they want to build or microsoft has been wrong all along and just refuses to acknowledge that like this sucks and nobody wants it and clippy was either a great idea that was too early cortana was either a great idea that was too early or a total disaster of an idea and this feels like the same like why can't microsoft let go of this idea they just love an assistant don't they i guess So it's kind of interesting because obviously Clippy was, it was interesting at the time, but then it got annoying because it would pop up when you didn't want it to.

54:49And that was obviously the meme. And then Cortana was like supposed to be this thing that would do everything for you. And it just failed over. And I think Satya Nadella recently said it was dumb as a rock, right? And all of them, like, you know, Alexa and the rest. So they obviously, and they went down the route of going for bots as well. Chatbots initially a few years ago, obviously before open AI models were ready. so there's always been this promise and they've always been obsessed with it but it does kind of feel like okay now's the time where it might actually work it's just they have to be careful they don't go down the sort of clippy route where it pops up annoyingly um it just kind of gets in the way and it's more like actually a useful assistant um that you call upon when you need stuff but it's also like what's the interaction model there like when do you call on the assistant or the assistant appears rather than you just click a button to bold something or you know like the user interface is still there like how do they lean back from that basic user interface into this into the complexity so there's a bunch of different challenges and they like customize it per app so it doesn't do everything you'd expect in each app so that's another kind of like learning curve to it almost is that you might think it's going to do something in this app but it's obviously been customized heavily for like powerpoint and because powerpoint's completely different to word so yeah so they have to manage a bunch of a bunch of that as well but from what they showed me and I have played around with it on my own PC in Word, but it's very basic now.

56:14But from what I can see, what they've shown is it does, it is pretty impressive. I was kind of impressed with it. And I'm old now. And, you know, I don't get excited about things. So like this stuff is like, oh, okay, this is actually interesting. So it's not like, it's not one of the silly future vision things. It's like, oh, it's actually working. Yeah. How does it give me a sense of kind of what those demos are like? Because I think one of the things you mentioned that jumps out to me the most is the thing where it's like it pops up when you write a paragraph and it's like you did this thing wrong and i think honestly even tools like grammarly go too hard in a space like that where it's like i'm writing an email that i'm like deliberately writing like a person because i'm a person and it's like you should potentially use more formal language in this email and i'm like no grammarly like shut up leave me alone and i can imagine with all of this stuff tuning that thing between like how do you sort of get involved and be helpful and how do you avoid being clippy just popping up every two seconds being like look at this thing that you're doing can i help that's a super hard line to walk and microsoft is going ham doing all of this stuff i can imagine a world in which they would kind of way overshoot and have to pull it back over time but what have you seen so far yeah so the basic stuff i've seen so far is that like you highlight a paragraph and then it kind of like it does like a purple sort of outline to the paragraph and then you you kind of go to that purple outline and then it's like co-pilot sort of like gradually appears almost okay it appears like as like a prompt that you can click and then it obviously comes into the sidebar from there and you know you interact with it and tell it how you want to rewrite the paragraph or it might even actually i think in that particular instance it comes up and says like here's 12 examples or whatever i've rewritten this paragraph for you and you can just flick through them you can just see them you know the text so that's like okay because you have to take action on it it's there like it's always kind of in the background so it's not like trying to jump out it's not trying to suggest you know text to you without you you know prompting it to do that so i think that's probably the way they will go about it that that's the the main one that i saw of that sort of use case that they showed there's some other stuff like powerpoint where you can add a file and then it will say you know do you want to match the colors across the slides and stuff like that which is kind of useful but i think right now it's like it's less trying to jump out at you and trying to interact with you and more like you call upon it you know for example if you wanted to in powerpoint you want to create a file based on that word document you were working earlier it will know what you had recently so it'll kind of suggest like oh do you want to pick this file and you go yeah okay and then it will take a few seconds to do that and then you can and then it'll sort of have you know like in bing where it says suggested responses after you've you started talking it does that it says oh do you want to you know customize this based on that and the way that they're doing it right now seems to be very you know baby steps step by step being careful with it so i think they're obviously cautious of the whole clippy stuff so yeah because did they mention clippy on the on the stream i don't think they did but i definitely asked them about that and they said you know they're proud of the fact that they've tried this stuff of like helping people in the past.

59:20But yeah, to me, it looks like they're being cautious. Yeah. And it does seem like if I'm Microsoft, this is a tough spot to be in. Because on the one hand, I did get the sense reading your stories that they think this is the moment, right? That like all the stuff we've been trying to do this idea of like, how do we bake in these sort of helpful additional tools? Like they're not shy about how sort of big a moment they feel like this is. I mean, obviously, you don't invest$10 billion in open AI if you don't think this is the moment, right? But then on the flip side, like the long history of Microsoft says most people don't want to change their workflows.

59:51People who do these things all day, every day don't want help. And what they want is for you to like leave them alone so they can do your job. And that is making it really hard to, you know, change the interface, to change how things work, to even add things like collaborative features has been tough because it just screws with the way that people have been doing things for 30 years. is microsoft wary of that or is this just kind of a like we're just going to push through the pain because this is the future moment i think they are because the way that it shows up right now is it's literally a button in the ribbon so it's not like trying to replace that ribbon it's just another command that you kind of summon almost for now anyway and it comes up in a sidebar and it's been a sidebar in office for years so so none of that is completely different to what you'd expect in office whereas if they'd gone you know pulled out the ribbon and said this is how you're going to use it like they do like the xbox one with connect they're like you've got to talk to your xbox and then they had to like walk that all back and make it work with a controller yeah so like they haven't gone that drastic with it so i think they're definitely aware of that but i think like you said it is the moment for them you get a sense that they feel like they're ahead of google right ahead of competitors that are trying to do similar stuff in this in this space and that they're poised to sort of you know take on that across everywhere they've done a co-pilot in they've it in GitHub for like a year or so now, which obviously wasn't quite as powerful as what they showed.

1:01:12But then they, you know, a week later, they were like, here's GitHub Kipilot X or whatever. So they're going fast everywhere. And I'm sure they announced a security copilot as well. Well, so this is what I'm trying to figure out. There's the security copilot, there's copilot on all the apps, copilot exists in teams. There's like business chat, which is kind of copilot, kind of not. Is the big vision that there's this like one assistant that you sort of interact with everywhere? Or is it like lots of different little AI things across your different apps? Like how does Microsoft see it? Yeah, I did ask that.

1:01:45And I think that the basic sort of gist of the co-pilots is that they're tailored to each sort of instance. Like in Word, for example, it's obviously customized for writing and, you know, in certain imagery and stuff. And PowerPoint's obviously a bunch of commands to make your slide decks look nicer. And you can pull in sort of, they're going to pull in like dolly images and stuff into there and excel is obviously very command driven so it's very different per app but then obviously github you don't want like they don't have like the gpt4 chatbot sort of interface where you're in line in your code because that's slow you know the latency there would be too slow for them to do those automatic suggestions that they do with the github copilot there so like they're tweaking it and customizing it where it makes sense which which i think does make sense right now because we're still in the early phases this stuff is still a little bit slow for you know that rapid response stuff that you might want in certain scenarios but who knows long term will it become this sort of cortina sort of assistant i think it probably will but i think that's probably a ways off from the initial stuff that they're adding over the next couple of years but yeah it makes sense because like the like you said the business chat stuff that is copilot but it's only in teams so you go to that and it sort of scours across your whole m365 stuff so all your documents so it kind of can take actions or those individual copilot actions in that interface like say you said oh i want to find this word document can we make it into a powerpoint you could do that there whereas if you said that in the word copilot it doesn't have that broad view because it's very tailored that's kind of a hint of where they will go and i think there probably will be you know 10 years there'll be a co-pilot that can do all your personal stuff and help you out all that sort of stuff and it just switches over to work mode and does all that sort of stuff so i mean that's they haven't quite laid out as a vision but it seems like that would be the way right yeah i wonder even if that's why it's called co-pilot and not like you know john the co-pilot like it seems very obvious to me that microsoft has been like deeply burned by having too many sort of silly characters that everybody grows to hate and it's whether it was cortana or clippy or like tay it's like microsoft has done this thing where it's like interact with the machine like it's a person and everybody's like this person sucks and i hate them get them out of my face and copilot is just much more of like a it's a it's a set of tools rather than your new best friend and i feel like that's probably a much saner way to go about the future yeah and it's also then it doesn't also feel like it's replacing you right true yeah as a cortana So I think that's definitely part of it.

1:04:18And the Copilot name kind of lends to what is the way they've designed it. So that it's there alongside you to help you rather than, again, replace you. So, yeah, I think they've been very careful with that. And apparently Satya loves the name Copilot. So obviously GitHub kind of started using initially. Yeah. So you mentioned business chat, which I think if I were to pick like one thing that feels like the sort of big picture, like if i'm if i rewind you know a few years and it's like this is when microsoft really got interested in what open ai is up to like business chat feels like that thing right where it's like plug all of your company data in and use this model to essentially like query and talk to your company's information and for microsoft it's like it's been it's experimented with you know a million different ways of storing company information sharepoint one drive and yammer and there's just a billion ways to communicate and share stuff and never quite gotten it right.

1:05:14But it feels like business chat, especially with coming with this like redesign of Teams that's coming out now. And it seems like Teams is very clearly working. Like is that combination as central to the future of Microsoft as it seems like it is to me? I don't know if it's necessarily like essential to the future, but definitely with Teams, they've always viewed that as like the work hub, like where you sort of start your day essentially and then you go into your office. So it kind of makes sense why they've done business chat there. But yeah, the redesigned teams, they're definitely like, they've gone head on into like fully being just web on that.

1:05:49Like it's just a web. It was Electron anyway. But now like they could have gone to a native app. You know, they haven't. So you can tell that they're going to fill that with loads of AI experiments. And there already is a bunch in there anyway. So they're definitely prepping that as the platform that perhaps they'll push more of this stuff too. Because if everyone's spending their time more in that and then jumping in and out of these Office apps, that would probably be the hub where they can experiment more with this AR stuff. Yeah, that makes sense. That definitely feels like it. And probably the safest place to do that just because fewer people, like we were talking about before, nobody has a 20-year history of how to use Teams in the way that people are really married to how Excel works.

1:06:26So you can have your cake and eat it too a little bit there. Yeah, you can play around with it there. And they have done a bunch of that. They've done a bunch of features in Teams over the years that they've added in pretty rapidly, especially when since the pandemic kicked off they really started adding stuff to teams then they were like zoom can't beat us right yeah they just went crazy um with that sort of stuff so yeah like i if they move stuff around there like moving people's cheese i think that's less of an issue with teams because i think it's that interface is pretty pretty basic you know you can move the buttons around yeah it feels like that app has changed every few months for like three years now yeah it seems to be fine yeah it does people are over it yeah exactly okay what do you make of the the security stuff around all of this because obviously i think one of the interesting trends of the last few years for microsoft has been it's doing like the the work analytics and it's just showing people sort of more of the information it has about them and all of their information and making use of all their documents and all this stuff and now to say you know pour all of your most sensitive data into this large language model that will spit information back to you that is like sometimes true and sometimes a lie and i think the idea of like telling chat GPT to make sense of my company's financials is like a great idea in theory, but it's going to be wrong half the time.

1:07:44And it's like, are we just going to tank the stock market by letting chat GPT look at people's Excel spreadsheets? I know Microsoft's answer to this seems to be like, oh, we're taking it slow, whatever. But what is your sense of how they're thinking this stuff through right now? Yeah. Like my personal opinion on this is that I think the Bing stuff is more dangerous because you go to Bing, you go to a search engine, you want to find answers that accurate right like you you don't have that feedback loop there available for it to go wrong like it needs to be right most of the time right if that's wrong you're putting misinformation out into the world right whereas if i generate a document at work i use this copilot thing if it's wrong that's on me you know that's not like i'm putting the essentially the misinformation and it's my name on that document when i submit it to my boss or whatever so it's up to me to fact check it and it's less it's less of an issue i think in that context but financial data if it screws that up there's some dangerous scenarios where this could go very wrong so they have to be obviously very careful there and i did ask them like is excel like a case where you're you know treading carefully considering it powers the world's economy but they did they gave the general you know we're being careful it's in preview that sort of thing uh which is like yeah but then they must be right they must be going careful there with with the features that they go in certain apps.

1:09:00And they're doing the thing that Google is doing too, right? Which is just like, have constant banners that are like, this thing will lie to you. Do not trust anything this thing says. Exactly. Yeah. And it's like, yeah, this is your confidential information going into this. Make sure you fact check it. And those warnings only go so far, right? Like you see them and then you ignore them after a while. So they're kind of, those warnings are kind of irrelevant, but it's definitely dangerous. I think, I think the speed, because now we've got microsoft going up against google those two biting heads over this stuff it just keeps pushing so until there will be like some disaster scenario that will happen where it screws up like some big bangs data or something like that and then you know that will be a lesson to be learned sort of yeah um where they need to slow down but i think it's very different to the search stuff like the search stuff i think i'm personally not happy that they're going so fast with that stuff because that stuff is can do real world harm outside of your organization yeah like it's it's really down to people using the co-pilot stuff alongside there it's like it should be a tool right you shouldn't rely on it to write your whole document for you because that's that ain't gonna be accurate right yeah and i think the question of kind of where one ends and the other begins is for microsoft and for everybody going to be really interesting right because i think like take this document and turn it into a powerpoint or like you use the example in one of your articles of like I have blue titles on every slide in my deck and I would like them all to be orange is like, that's the kind of thing that takes a long time.

1:10:29And it's like a lot of people essentially do that for a living. And then you can make that a, just a simple command that I think is like universal victory. Like let's, let's solve the interface by asking questions of technology, right? Like into that, love that sold. I don't know that it's great at all that yet, but I'm very excited about the path that all that is on. But then this question of like, how much information can it give me? How much should I rely on it? And how much should I allow it to make stuff for me is just a mess. There's the lowest stakes stuff there is all, I think, fine. The highest stakes stuff there, like if anyone ever reports AI generated earnings from their company, like we're hosed.

1:11:06But somewhere in the middle is that cutoff. And I wonder if there's gonna have to be things that it's like Microsoft is sort of artificially, you know, dumbing down some of the stuff these systems can do. until they're much, much better than they currently are. Or if we're just going to get into a place where it's like, the data lies sometimes, do with it what you will. And in that case, I think you're right that we're in trouble. Yeah, I think it's also whether we get to the point where it like, makes so little mistakes, where it's almost like making human mistakes, like when it gets to that level, which is far away.

1:11:38But then do we accept that? Because you can't expect the machine to be 100 % like you wouldn't expect a human to be. so it's like when does it get to that level as well which i think is some interesting questions around it but it's definitely like that blurred line between what is acceptable and in what context and what app and wherever you're using it and how you're using that data as well because some of this stuff could be a bit freaky as well where it starts like you know like all the ai tools we have right now sometimes they're a bit like alarming you can look yourself up on bing and it's like it pulls out so much data you can't kind of forget that's out there right bing thinks i'm four different people which i enjoy very much this is the delightful thing about having an incredibly common name is it thinks i'm i'm the texas the university of texas baseball coach uh i'm an actor who was on fraser i think i'm like an attorney in phoenix uh and i'm the editor at large at the verge like a marvel superhero or something different backstories yeah that's that's what i don't like about this bing stuff it's just it pulls out so much information like i i put up my girlfriend's name on it and it's like it just lifts off so much stuff and even like gets to the point where it's about to give out her like mobile number oh wow because obviously she's had her cv out and sure yeah like caches that stuff but it's just like i think for normal people like she's not you know in the public eye for having normal people's data out on it is yeah it's gonna get strange so yeah agreed so last thing before i let you go here is in the stuff you've been writing over the last few months there have been like little sort of glimmers of what all of this might look like on windows and there's like there's the new bing button in the taskbar but i had this moment of reading the story of like is the future of windows that you just hit the windows key and just up pops a chatbot and that's like how you use your computer from now on like is that is that where we're headed what's your sense yeah i think so because i think like they have the search stuff right like you just mentioned but it's just basically a link that just throws you into edge and into bing yeah like opens edge yeah that's not that yeah so it's not like even really integrate into the search experience i think obviously the logical next step is to fully integrate that into the search experience and into file explorer and all those sort of like touch points that you use daily the start menu um which has some ai stuff already but yeah like i wouldn't be surprised me if like if they did something like alfred for windows but it was just like the chat bot and you know pulled out to everywhere oh man that's the dream that just made me very excited that would be awesome yeah that would be awesome like an actual search interface that was universal what a world and could just yeah you don't have to worry like where your favorites are or like your app links any of that stuff all that user interface stuff just goes out the window you just have this search prompt that's another one where i would assume microsoft is gonna have to tread pretty carefully although i don't know they've fully redesigned the start menu like 16 times in the last decade so like what's one more yeah they could do a windows 8 couldn't there again just have a full screen chat in that went so great the last time like well what could go wrong as long as they make it mouse and keyboard friendly it could work this is what i'm saying microsoft you heard it here first just forget windows just one it just boots to a chat bot it's like going back in time like a command prompt exactly yeah this is it's everything from the 1970s is is new again it's just like the screens are better now that's all that's changed uh awesome tom thank you i appreciate it this is not over anytime soon so we'll have you back shortly.

1:14:58Okay. Thanks for having me. All right. That's it for the Vergecast today. Thanks to Monica, Tom, and Jen for being on the show. And thank you as always for listening. There's a whole lot more from all of these conversations at theverge.com. There are a bunch of links in the show notes, but also lots more on the website. It's a good website. And if you have thoughts, feedback, feelings, or Taylor Swift tickets, you want to sell me, you can always email us at vergecastattheverge.com or call the hotline 866-VERGE11. Send us all of your questions, all of your thoughts. We love hearing from you. This show is produced by Andrew Moreno and Liam James.

1:15:30Brooke Minters is our editorial director of audio. The Vergecast is a Verge production and part of the Vox Media Podcast Network. We'll be back on Friday with Alex and Neelai to talk about Elon Musk, Twitter's chaos, and everything else happening in tech. See you then. Rock and roll.

1:15:54Support for this show comes from SC Johnson. We've all been there. Choosing not to wear your new white shoes because there's a 10 % chance of rain. Bending awkwardly over the tiny coffee table to enjoy a sip of your latte. Not ordering the red sauce. Those feelings of dread are what we call stainsiety. But now you can break free from your Stainxiety with Shout's Triple Acting Spray that has stain-fighting ingredients to remove a huge variety of stains so you can live in the moment and clean up later. Just breathe and shout with Shout Triple Acting Spray. Learn more at ShoutItOut.com.

1:16:36Support for this show comes from Amazon Ads. Every business owner has been there. You put a significant amount of money into an ad buy and then wonder, did those ads actually have an effect? Luckily, there's Omnichannel Metrics from Amazon Ads. Omnichannel Metrics helps advertisers understand how their Amazon Ads campaigns drive sales both on and beyond Amazon while campaigns are still mid-flight. OCM measures performance across streaming TV, video, audio, and display, helping you understand what's driving results across the full funnel. Using Amazon Shopper panel data plus third-party signals, you'll be able to see beyond Amazon product sales, units sold, and ROAS.

1:17:16Whether customers buy on Amazon or at a brick-and-mortar store, you'll understand the full impact of your campaigns. Measure comprehensive sales impact to better understand purchase behavior and drive greater efficiency, effectiveness, and ROI. Tired of guessing where your ads are actually driving sales? Capture the full impact of your media spend with Amazon Ads Omnichannel Metrics. Head to advertising.amazon.com to learn more. That's advertising.amazon.com.