How AI really works, and how the smart home broke

In short

The Vergecast Episode Summary: How AI Really Works, and How the Smart Home Broke

Podcast Details

Title: The Vergecast
Hosts: Nilay Patel and David Pierce
Release: Weekly on Fridays
Focus: Discussing technology news, gadgets, and software's impact on daily life.

Episode Overview In this episode, David Pierce engages with various guests to discuss the intricate relationship between AI development and human involvement, as well as the evolving concept of smart homes.

Segment 1

The Role of Humans in AI Development

Guest

Josh Dzieza

Topic: Insights from Dzieza’s investigation into the human element in AI.
Key Points:
Common perception: AI operates autonomously after training on large datasets.
Reality: Humans play a critical role in data annotation and training.
Historical reference: The creation of ImageNet, which required massive human input to categorize images.
Ongoing need for human involvement in refining AI models.
Tasks include data labeling, interpreting ambiguous data, and retraining models.
Challenges: AI programs often struggle with tasks outside their training data, necessitating human oversight.

Industry Insights

A billion-dollar industry focused on data training exists, comprised mainly of outsourced labor.
Example: Companies like Scale AI employ thousands of annotators working under stringent conditions.
The hidden and sometimes exploitative nature of this industry raises ethical questions.

Segment 2

The Smart Home Misconception

Guest

Jennifer Pattison Tuohy

Topic: Reevaluating the idea of the "smart home."
Key Points:
The term "smart home" can be misleading; it implies complexity and a commitment to a fully integrated system.
Reality: Smart technology should be viewed as a gradual evolution of home devices, not an all-or-nothing adoption.
Recommendations:
Introduce smart devices gradually as needs arise, rather than overhauling the entire home.
Focus on practical benefits, such as energy efficiency and convenience.
Societal trends: Emerging use cases for smart homes include energy management and aiding aging populations through technology.

Segment 3

Smart Home Platforms

Key Platforms:
Apple Home
Google Home
Samsung SmartThings
Amazon Alexa
Considerations for Choice:
Assess existing devices and ecosystems for compatibility.
Each platform offers unique features and strengths.
Example: Google Home has advanced voice recognition capabilities; Apple Home emphasizes privacy and local processing.
Future Prospects:
The introduction of the Matter protocol aims to simplify cross-platform device integration.
Users potentially benefit from a reduced "lock-in" effect, allowing for more flexibility in device choices.

Key Discussions

The importance of human skills in AI development underscores the misconception that AI operates independently.
Smart home technology is evolving, and while it can seem daunting, gradual integration and focusing on solving specific problems can enhance user experience.
The ongoing development of Matter could revolutionize how smart devices interact across different platforms, making it easier for users to adopt new technologies without fear.

Conclusion This episode of The Vergecast provides a deep dive into the human aspect of AI and the complexities of creating a functional smart home, encouraging listeners to adopt a more pragmatic approach to technology integration in their lives.

---

*For further inquiries or feedback, listeners are encouraged to reach out via email or telephone as indicated in the podcast.*

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.

Get VO free

Transcript

Automatic transcript. May contain errors.

0:02Welcome to the Vergecast, the flagship podcast of open source lightbulbs. I'm your friend David Pierce and I am coming to you from my laundry room slash furnace room slash I don't know what you call it. It's just that weird room in your house where you keep all your tools and toilet paper and laundry detergent and stuff. Anyway, I'm in here because, well, last week was Smart Home Week on TheVerge.com and I was inspired to try and connect all of my stuff a little better together. So I'm trying to get a water sensor rigged up to tell me if my hot water heater is leaking, which sounds simple and is frankly exactly as boring as it sounds.

0:36but it will make me sleep better at night. Anyway, we have much better things to talk about on the show today than water sensors. Josh Jezza is going to come on and tell us a very different kind of AI story than the ones we've been hearing over the last few months. And then Jen Pattison-Tui is going to come on and we're going to talk about smart stuff and whether we've been approaching the whole idea of the smart home all wrong. Actually, come to think of it, we might end up talking about water sensors after all. So sorry in advance if that happens. All that is coming in just a sec, but first I have to see if I can shimmy behind the water heater over here to get this thing installed.

1:12So if you don't hear from me again, I'm probably stuck back here. This is The Verge Cast. See you in a sec. Support for the show comes from Charles Schwab. At Schwab, how you invest is your choice, not theirs. That's why when it comes to managing your wealth, Schwab gives you more choices. You can invest and trade on your own. Plus, get advice and more comprehensive wealth solutions to help meet your unique needs. With award-winning service, low costs, and transparent advice, you can manage your wealth your way at Schwab. Visit schwab.com to learn more.

1:52How does F1 turn data into insights at 200 miles per hour? AWS is how fans get inside the strategy. aws powers next level innovation for millions of businesses welcome back i made it everything's okay obviously ai is the biggest story in tech right now both in the chat gpt chatbot sense and just in the sense that ai tech is suddenly showing up in every product and service we use to help us write help us find things help us make things and lots more And we kind of understand how all the underlying tech there works, right? You feed a bunch of data into a large language model, it automatically, black boxy, trains itself to understand all that data, and you're done.

2:40But of course, it's not that simple. And it turns out it's not that automated either. The Verge's Josh Jezza has spent the last few months looking into one particularly misunderstood part of the process, where humans matter far more to AI development than you might think. He even got some firsthand experience in the AI process. So we brought him in to talk about it. Hey, Josh. Hey. So there's a lot to get to in this story. And I want to spend a lot of time talking about the part where you became a data annotator, because I find that deeply fascinating. But I think just to set the scene a little bit, we need to do a little bit of, I think, term defining here to make some of this make sense.

3:19We go back to 2007, which is kind of the earliest date in your story at Princeton with a researcher named Fei-Fei Li. Tell me what's going on kind of there and then and why this is sort of the moment that starts what leads to your story. Yeah, so this is really the moment where deep neural networks and machine learning really comes back. It was sort of a technique of machine learning that had been languishing a little bit. And then you have these fast processors and then Fei-Fei Li just realizes that you need way more data than anyone's been training anything on to get image recognition, which is really sort of the test bed of this at the time.

3:55So using millions of images instead of tens of thousands, but it was really impossible to label or categorize that many images. You need, you know, there's this unprecedented corpus of imagery now of the internet, but you no longer have had grad students kind of like page through encyclopedias or whatever and scan images. But you still need someone to go through and say, you know, cat, airplane, automobile, whatever, and sort these images into something that, you know, this neural network can suss out the patterns in and begin to identify stuff. So she creates this database called ImageNet using Mechanical Turk, which was sort of recent at the time, the Amazon work platform where you can have people from around the world do micro tasks for pennies usually.

4:39And, you know, it worked and the machine learning revolution kicked off. People realize that this is actually super effective if you have enough data and enough annotated data to train these models on. The phrase annotated data is like the one that kept jumping out to me because I think the way that we understand machine learning so much is basically like develop a model, just throw the Internet's data at it, whatever you can find, like scrape Google images, scrape Reddits, whatever you can find, throw it at it. Magical step three, sort of black box nonsense. And out of it comes AI. And I think the thing that really jumped out to me through your story is not just this realization that what we need is a huge amount of data, right?

5:22Which I think now, 16 years later, is relatively well known, right? Like, to make these things great, you need outrageous amounts of training data. What I was surprised by is the amount of human interaction that there kind of was and still is in this process. That not only is it like a handful of people at the beginning saying, this is what a cat looks like, this is what a dog looks like, this is what an airplane looks like, that this is like a massive, ongoing, never-ending process. And it seems like that was kind of the insight that sent you down this road to some extent, that this is a bigger and more human task than anyone realizes even now.

5:59Yeah. I think there's two levels to that. So there's the sort of model creation training aspect that's super manual. And there is sort of a first round where you just kind of throw all your data in there and it finds its own patterns, the sort of unsupervised learning process. But then you have to go through and, you know, there's different methods, but you need humans to either sort of pick the best examples of what the model does and then sort of retrain it on that. Or, you know, write your own sort of like, here's the gold standard essay or, you know, Python program or something and then train it on that.

6:37and then kind of do more rounds of feedback. And so that's sort of state of the art for the language models you see now. But then you have this other element, which is like, all these programs are super brittle, you know, just because they can do one thing well doesn't mean they can do something else that seems quite similar well at all. You know, whenever they encounter something that's not in their data set, they struggle. And so you have kind of this other side that I had not really realized existed of people who are just kind of, you know, minding automated programs that are out in the world, you know, they're like sorting through credit card data and saying like this came from 7-eleven or whatever their you know self-driving car gets stuck and they're going to go yeah we really need to you know get some more bird annotations or something like the sort of the world is changing and things are always encountering data that's not in their training data and so they they fall apart and you always need kind of a human there to catch them okay so this thing this this sort of insight from 2007 birthed this gigantic industry.

7:33And I want to talk about the industry at a higher level and then the specific job of what the people who work for these companies do. But the industry is totally fascinating to me because you uncovered this gigantic, billion-dollar, sketchy, undercover world of data training. I don't know. There were moments where it was like, oh, this is a big, booming industry. And then there were moments where it was like, This sounds like the way that sketchy billionaires try to hide their money by setting up shell corporations all over the world. This industry does not want you to know about it, but you learned a lot.

8:06Sort of paint me the picture of what this training industry looks like now. Yeah, it's really weird. So there's a bunch of different super opaque elements to it. One component, it looks sort of like an outsourcing company, like a call center. You go to some vendor, usually in the global south, and they have a lot of people in an office labeling data. But then increasingly you have these things that are kind of, I think of them as like the sort of next generation of Mechanical Turk. Like Mechanical Turk is sort of a lot of work to get anything out of. Like you need to go in there and like post a task and sort through a lot of garbage.

8:40You have these other companies and sort of one of the ones I focus on is called Scale AI. And their whole thing is like we have well-trained people. They're really experts in this stuff. and they are also being assessed by automated systems that are testing, you're doing sort of quality assurance and things like that, and they're trained. And it's also a platform like Mechanical Turk that anyone can sign up for, but it's a job. You'd spend hours sort of learning each new task and you can get fired or banned or whatever the algorithmic platform equivalent of fire is. And that's sort of what you have now.

9:18It's a huge numbers of people, like hundreds of thousands of people who are doing this work. And because you don't want your in-development AI system leaking out, there are sort of these nested layers of opacity. You know, the Codenames, Scales platform is called Remo Tasks. There's no sort of public disclosure that they're related. And then every sort of project on Remo Tasks is also codenamed with sort of weird stuff like Pillbox, Bratwurst or whatever. So no one knows what they're working on. It's all quite murky. We're going to come back to Remo Tasks, by the way. I'm very excited to talk about your life in Remo Tasks.

9:57But the thing that kept jumping out to me is that it seems odd to me that this industry is so secretive. Because on the one hand, it sounds a lot like kind of the way content moderation works, right? Where the job is a lot of sort of menial drudgery in a lot of ways. It is sort of repetitive tasks. And folks outsource that. But I think that's in part because content moderation is viewed by a lot of these companies as not that important. It's like a thing you have to do, but they're happy to have it done sort of serviceably at the cheapest way possible. But in this moment of AI, like this is the game now.

10:31These companies are raising gigantic amounts of money. They're spending a ton to train these things. You would think that they would want to have as much quality control, as much privacy and secretive control over all of it as possible. So I guess it's weird to me that it's this big secretive industry that is so mission critical to what everybody says is the future. Like, why is it this third party subcontracting world instead of like Google hiring a team of 10 ,000 people to do it for Google and Microsoft for Microsoft? I think this is changing a little bit, but I do think there still is this perception that it's not that important that like, well, we just, you know, AI is improving so quickly where we just need to get a bunch of data and then it's going to be able to do this.

11:13and we can automate it and we won't need this anymore. So why put a bunch of people in your payroll permanently for this? So it's viewed as like the human element is like a stop along the way where like, yeah, we won't. Yeah, I think that's wrong. And I want to talk more about that in a minute. But the job of these annotators, you talk to a bunch of people who actually like sit down at a computer and do this kind of training work for a living. Like, what does the job look like? What do you do all day when you do this job? There's a lot of variation. So the image recognition stuff is very like this.

11:42I think the thing people are most familiar with, it's like the self-driving car stuff. You get like a bunch of capture looking things and you outline the fire hydrants or whatever, or like a bunch of LiDAR laser points. And you sort of say like, this is the human and that's a human there. That is kind of the classic stuff. And then the chatbot stuff is more, you're in this like kind of weird sort of text edit kind of interface a lot of the time. And you have a bunch of things you're supposed to be reading for or something and you're given, honestly, it feels like a standardized test. It's like you have like an essay and you have like two attempts to summarize it and you like critique which one is better or write your own version of it.

12:25Or like, here's some marketing copy. Did this include all the information in the product or did it make up a bunch of stuff? Things like that. It's pretty repetitive, but it is also really hard. Yeah, what is hard about it? Like, I understand why it's repetitive. You tell a story of a guy who has to go basically like you're talking about the self driving example frame by frame, which just it just sounds brutal. And it's like the fire hydrant was here in the last frame, and it's here in this frame. But I have to do it because that's how you train the thing. And it has to learn all the edge cases and so on.

12:55But I did get the sense that this is a like, genuinely sort of difficult job. What is it about it that people find difficult? You have to be really precise. So like that kind of LiDAR task, you have to go through every frame from sort of every perception mode for a car and label it and say, this is a vehicle. It's moving. It's this type of vehicle and sort of do that for each thing. And so you have like a few seconds of footage that ends up taking an eight hour day to fully annotate. I mean, you have to be really precise. Like I kept failing a task where you have to outline pallets for like an autonomous forklift or something.

13:31And it was just like the quality you're supposed to get was down to the pixel level. So it's like if I got one pixel off, I would fail. And it's this blurry photo in a dark warehouse. And it's like I got I flunked it. Wow. The other element is just that the things that you're doing are really alien. Like the way these machines process information is not human. And so you're having to do these things like categorize clothing, but also label all the clothing in mirrors. because to an AI system, it's the same thing. And then what is clothing? You know, are rollerblades, shoes, like these sort of weird questions that you get into.

14:08And like the engineers, they don't even know that they need this to make these distinctions, but then they, you know, put the data in and their model doesn't work. And they come back and say, don't label rollerblades, but do label flippers or something like that. So you just get these kind of incredible, you know, 40 page instruction manuals that you have to follow. Right. So yeah, and this is where we get to your specific adventures in this world. You, if I'm remembering correctly from your story, decided at one point in the reporting process to sign up for Remo Tasks, right? Tell me about the process.

14:40How did you get into this world? Yeah, so I was kind of beating my head against the wall. It's a hard world to get into because it's so opaque. You know, you can't go to an office, you can't sort of figure out, people don't put it on their resume on LinkedIn very often. And so it was kind of the only way to figure out who was doing the work and what the work was like was to join the platform and it's totally open. So this seemed like the obvious way to go. and one of the quirks of the system is you're only brought into a group on remote tasks of workers if you pass like a training program like you need to pass the test and then you're brought into a Slack channel with people and so it was like this video game where I kept having to pass these training programs and I kept failing because they're really like I don't know you see a thing and it's like leave all the clothes and the social media posts and it seems self-explanatory and so I would sort of click through the instructions, fail completely, have to go back and be like, there's like dozens of pages of distinctions that you need to follow.

15:39You need to keep it up on like a second monitor and always be checking it because it's just incredibly precise and these sorts of weird. Give me an example. So like I'm imagining I can think of a social media photo and it would say circle all the clothes. I would see somebody's wearing pants and a shirt and a pair of shoes. I I would circle all those things as exactly as I could and then just like move on with my life. That doesn't sound that hard. Like what what were the problems you kept running into here? So first off was what is the definition of real here? Like the instructions are like only circle the real clothes that can be worn by real people.

16:13And one of the first things I got was just like a magazine on it, like a photo of a magazine with some clothes on it. I was like, well, that's not real. That's a magazine. Not correct. Like those are real images of real people in a magazine. And so you circle them. And the same goes for mirrors or whatever else, because like the AI, it's all just pixels. It doesn't know the difference between a mirror and a magazine. And so that was kind of the first thing was sort of that you need to be looking at, you know, someone's in the background and you don't think it's the thing that's relevant, but, you know, they're wearing a hat and you need to circle that.

16:44And then it was weird stuff where it would be like, don't label clothes on action figures, but do label them on mannequins or label costumes, but don't label armor. And so what is, you know, someone is dressed up in like a stormtrooper armor. Is that a costume or is that armor? Like these sort of weird things where people do these distinctions and you can't keep them all in your head. And so you just sort of look at through it and you're like, I have no idea sort of what side of the line that falls on. And then you guess and then you fail out of it. So and I guess the assumption then would be that this AI is trying to learn to make the same distinctions that you're learning how to make.

17:26And essentially, you're just trying to train it over and over. You're doing all of the thinking so that the machine can just learn what the answers are, basically. Yeah, I never really figured out what that one was about. But, you know, I could guess. And there was a lot of e-commerce stuff on there. And it all sort of seemed geared around the same thing, to be able to, you know, comb Instagram and say this kind of like this shirt is spiking in popularity or something like that. Or like this person is posting about this sort of item of clothing, like maybe we should market this to them. Like that sort of, it seemed like it was geared toward being able to make those kinds of calls.

18:00And so you had to teach it, you know, how to recognize a shirt in a sort of a social media environment, which turns out to be incredibly difficult. Yeah. So you eventually passed clothes. Congratulations. Thank you. What did you then get thrust into? What was your first big project after that? So, yeah, I did. I did a little bit of it. And then I, you know, failed pretty quickly. And I think it was I started getting some people in mirrors or like in windows, you know, reflected in building windows or something like that. And that was something you were supposed to label, but not if it was too blurry.

18:33and so I flunked out of it and then it kicked me to something I think it was called crab generation it was like it's quite humiliating you know you just get this pop-up and there's like a cartoon crab that's like your low quality has has booted you from this but now you can train for this new task and that was more of a chatbot thing and that was sort of just like seeing whether it was paraphrasing news articles without making things up basically but you know it's quite eclectic There's all kinds of stuff on there. There was one where it was just sort of videos of people talking about their jobs.

19:05And I was supposed to rate whether they seem like emotionally stable, I think, was the criteria. One of the criteria and sort of how outgoing they were. No idea what that was about. It's just all over the map. So, okay, this makes me think two things. One, this is an enormously hard job to get good at. And the kind of job that maybe isn't going to last forever. Like you train on this closed thing, you do it for a minute, and then all of that knowledge is useless to you for the rest of your life, which seems like a wild way to use that kind of tool and time. Memorize these, you know, 40 pages of rules just to look at pictures for an hour and then just kidding, we're going to fail you anyway.

19:46And then on the other side, that this stuff is so we're trying to teach computers how to think like humans when maybe that's just impossible. That like the idea of quantifying what outgoing is like, I can't do that. You I don't think you can do that if you figure out how to quantify outgoingness. Congratulations. But so it just makes me think there's this like fundamental disconnect between what people are being asked to do, which is basically teach computers how to think like humans. But then in order to do that, you as a human have to think like a robot thinking like a human. And it's just I don't know how anybody would do this.

20:21Well, it seems insane. It is really kind of crazy making. And you start to question, you know, how how do I know things? I've been making all these calls about, you know, what clothes are all my life. But do I really know? And you do have to kind of think like a robot. Like the trick is sort of being so literal, just sort of, you know, way more literal than any human could be and operate in the world. But in terms of the unpredictability and sort of the stop and go nature of it, that was the big complaint I heard from people. You know, there were complaints about the pay, like the pay can be quite low.

20:52But even more than that, people would complain about spending a day learning a task, doing a dozen of them and then having it end and have to go learn a new task. Or there's a bunch of work and it pays pretty well and you start doing it. And then like a week later, it's gone and there's nothing for weeks and you don't know, should I go find another job or what's happening here? It's quite stop and go, which, you know, talking to engineers and vendors, it has to do with the way machine learning development works. Like you need a ton of data to train your model the first time and you want it really fast.

21:25And so you sort of get as many people working on it as you can, and then it's done for a little bit. And then you need to find, you know, something comes up, you need to sort of retrain it on something. You need like a smaller, more specialized group of people. it's very kind of peaky. And that means people are, you know, you always have to be on the lookout for like a new task to drop, and then you have to learn it. And then maybe that's going to be totally wasted because, you know, work for a day and then the AI gets it and then it's on to whatever else. Yeah, that's tough. What was the general sense you got from the folks you talked to about whether this is a good job?

21:57Obviously, like you're talking about the variability is tough, but did the folks you talked to, did they like the work? Was it better than other things you could do kind of sitting in a computer doing these kinds of tasks all day? Like what was the vibe you got? It varied a lot. You know, I talked to a lot of people where a lot of this work gets done and also where I happened to just sort of get placed in my remote task experience. And when remote tasks started out there in like 2018, 2019, it was a good job. Like it paid pretty well for the area that people were making about 10 bucks an hour.

22:33But then, you know, more people joined the platform. They started adjusting pay based on regional cost of living. And then it dropped to like a dollar to$3 an hour. And even then, you know, people stopped thinking it was a career, but it was kind of a backup. It was my sense of it. Like this is better than nothing until I can find something else to do, but it was still so stop and go that they felt jerked around by it. It's a full-time job for a little bit, but like you can't depend on it and then it's gone or then and banned for mysterious reasons. And then I did talk to people who, especially doing sort of the newer language model stuff, who enjoyed it even.

23:08Like a lot of this work is coming to the US because you need English fluent people and people with certain cultural references and what have you, a certain expertise. And, you know, they're making 20, 30, 40,$40 an hour. And a lot of them, it's like one of these, you know, work from home things. They let, they enjoy it while they have it. No one knows how long it will last. I think the people who are getting paid the best They wanted it to be a full-time job or a dependable job, but they still don't know who they work for or when they're going to be done collecting data for it. So they're kind of just like, as long as the tasks are running, I'm going to do them.

23:43All right, we need to take a break and we will be back with more from Josh on humans and AI.

23:52Support for this show comes from LinkedIn. When you're a small business owner, your business is on your mind 24-7. So when you're hiring, you need a partner that works just as hard as you do. That hiring partner is LinkedIn Jobs. When you clock out, LinkedIn clocks in. LinkedIn makes it easy to post your job for free, share it with your network, and get qualified candidates that you can manage all in one place. LinkedIn's new features can help you write job descriptions and then quickly get your job in front of the right people with deep candidate insights. Either post your job for free or pay to promote.

Read the full transcript

24:26Promoted jobs get three times more qualified applicants. At the end of the day, the most important thing to your small business is the quality of candidates. And with LinkedIn, you can feel confident that you're getting the best. Based on LinkedIn data, 72 % of SMBs using LinkedIn say that it helps them find high-quality candidates. Find out why more than 2.5 million small businesses use LinkedIn for hiring today. Find your next great hire on LinkedIn. Post your job for free at linkedin.com slash track. That's linkedin.com slash track to post your job for free. Terms and conditions apply. Support for this show comes from MongoDB.

25:06Innovation is the key to success, but innovation also comes with its own set of challenges. And if you're a developer who wants to find a new way in and think outside of the box and think beyond legacy code, it's time to check out MongoDB. MongoDB is a flexible, unified platform that's built for developers by developers. MongoDB is ACID compliant, enterprise ready, with the capabilities you need to ship AI apps fast. That's why so many of the Fortune 500 trust MongoDB with their most critical workloads. Ready to think outside rows and columns? Start building at mongodb.com slash build.

25:50We're back. the expert level stuff was one of the things in your story that I really jumped out at me as something I don't really know anything about. Walk me through kind of what that looks like, because obviously, like you're saying, there's this huge amount of work that is teach a self driving car what a fire hydrant looks like in every imaginable situation. Or, you know, does this chatbot response make any sense? That's the kind of thing that is hard work, but I think a lot of people could learn how to do it. But there's also it seems like this other side of the industry that is super specific and expert driven and really different?

26:22How does that side of it work? Yeah, this is sort of where I think things are headed. You know, I would love you to see it just sort of how many people think ChatGBT is giving them good information, you know, submit legal attempt to submit like things before the court or something like that. But like, It's just very fluent at sounding accurate in a bunch of different fields. And so you need an actual lawyer to go through it and see if it is making sense. Or you need an actual physicist to see if it's just bullshitting you or whatever, or if it's actually being accurate. And so that has increased the demand for people who have specific backgrounds in whatever area you're trying to fine-tune your system to do.

27:08And that can get quite technical. Like there's a lot of, you know, you need a lot of programming if you want it to program well. And so that is kind of the trajectory that you see sort of as these things get better. You can foresee something similar happening to what happened with self-driving cars. Like you have a high stakes area like driving a deadly vehicle around. You need to get everything exactly right. And so you have these fields where people are, you know, saying it's going to be great in medicine and health. You need that to stop hallucinating. And so you need medical professionals or people with some kind of medical training to go through and make sure that it's actually recognizing the cases it's supposed to be recognizing and not doing something weird.

27:47The thing that's the most that really jumps out to me about that is this is kind of a random aside, but it comes back around. I promise I was doing a bunch of research on. Do you remember Ask Jeeves, the search engine from like the early 2000s? Ask Jeeves was like the sort of natural language question answering service 20 years ago. We're kind of back to the same idea now. And everybody thought Ask Jeeves was this sort of magic of technology. But what it actually was, was a bunch of human editors who worked at the company, going in and seeing the most popular queries, and like literally just answering them.

28:21And so because a lot of people tend to search the same things, they were getting these very good answers. But what that doesn't do at all is scale, right? And so it was like, if you wanted to do one of the, let's say 200 things that everybody else on Ask Jeeves wanted to do, you were in great shape, you're going to get incredibly helpful. information. It was going to feel like magic because there was a human on the other side. But if you went down this kind of long tail of new stuff, and as we've seen on the internet, that long tail is infinite. It's just going to get worse and worse over time.

28:48And part of me wonders, especially with the kind of stuff you're talking about, if we're going to land in the same place with a lot of these AI tools, because we're going to get to the point where like, if this question that you ask your chatbot has been seen and answered by a person who has actual expertise and knowledge in this space, you're likely to get a good answer because they've picked the right answer or responded or even potentially written some of the text that you might see. But even as you start to get one degree away from that, it seems like it's going to start to get worse. And it seems like there's this bet that if I just show you enough things, eventually we'll have the right answer for everything.

29:25And I'm just not sure that that reasoning holds up. Is this just like a forever problem that these companies are going to be chasing with folks like you're talking about forever? I ask people that and I could never get, obviously they hope it isn't a forever problem. People working on these systems hope that at some point, you know, you add enough handwritten accurate examples and it sort of starts to learn something about accuracy or something like that. But, you know, I think I sort of as an outsider from what I can see about how this stuff works, it seems like that's not going to happen right now.

29:59And it's kind of in the whack-a-mole ask Jeeves phase where it's like, oh, someone asked it this thing and it did something totally weird. We need to have some people, you know, handwrite some examples to retrain it on so that now when you ask it this, it says something that is, you know, not dangerous or not crazy or whatever. And even if you do like a lot and a lot of that sort of work, because it still doesn't really reason in any kind of human way, there's always the risk that it's going to get stuff really accurate, seeming really authoritative, sort of get all the formatting right and the language right, but substitute, you know, wrong nouns or something.

30:37Do something that just means it's like a little it's it's off in some crucial way. Yeah, ChatGPT is very good at confidently lying to you, which is somewhere between delightfully fun and super dangerous, depending on the thing that you're asking at any given time. That was just the thing I came away from your story with is this feeling like what everybody wants to happen is that we're just going to train, train, train, use humans, use humans, use humans. And then one day this switch will get flipped and it will work. And what I came out of this thinking is that actually maybe this is going to be a more human process for longer than potentially anybody wants or has planned for.

31:16But that what's going to happen is just that the people are going to push it up ever so slightly every day as they keep working on this stuff, rather than feeding an engine that will eventually, as if by magic, figure out all the right answers. Yeah, that's my sense too. And kind of reinforced by seeing how much stuff that I had already assumed was super heavily automated is still relying on people saying, you know, this TikTok video is heartwarming or this ad is too sexually provocative or something like it's not stuff that I had sort of assumed was sort of so we had so much data on already is still, you know, hand tooled because norms change, language is confusing and ambiguous.

31:58and programs just haven't been able to sort of keep up with it. You always need kind of a human there, giving it new examples, making sure that it's adjusted. And so that's sort of what it looks like with language models. You know, it could be really high-end stuff. You could have, I mean, you already have math PhDs and stuff working on annotations, but you're always going to need some of that because it doesn't, it just doesn't think in a human way. There's always going to be a little bit of an area that's not overlapping. Yeah, one of the questions you reckon with a bunch in the story is this debate over is AI going to replace everybody's jobs or is AI going to create a bunch of new kinds of jobs or some combination or something else entirely?

32:42Where do you land on? Obviously, that's a huge question that we could spend many hours talking about. But do you have any kind of new thoughts on where you land after reporting this story? I love feeling like it's going to both automate and create new jobs in sort of the same area that, you know, one of the data vendor CEOs that I spoke with had a very convincing comparison to industrialization. You have this task that used to be all done by one person, and then you break it down. Machines are quite good at one sort of whatever it is, sort of tool punching component of it. You automate that, but that means you need to change what everyone else is doing, and everyone becomes more specialized at their little task.

33:24And that's what you could see happening. I mean, it's what's already happened with remote tasks. Like an automated system tries to identify something. It can't. It goes to a human. They annotate the scene. That scene is then broken up into three things that go to other humans. They annotate it. It's sort of this global assembly line with all of these little, little tasks that add up to doing some larger process, but it's hard to say what it is. Yeah, well, and I wonder how long that chaotic opacity stays useful to everybody. Because on the one hand, people are going to get more sophisticated about how these kinds of systems work, right?

33:58But on the other hand, we're still in this era where I think a lot of people feel like the best strategy is just to get as much of this data as you can, throw it at the machine, and then kind of see what happens. And, you know, with the people sort of endlessly refining the rules about clothes is such an interesting one to me, because what that means is they're still learning how their system works. And what is happening is it is just this constant back and forth of trying something, learning new mistakes, and then trying something else and then learning new mistakes. And at some point, we're going to hit a point where the stakes get high for this stuff.

34:34Right now, it's like people don't really expect chat GPT to be right. And when it's wrong and you, you know, use it in court, like you were saying, it's like a hilarious joke. But at some point, this stuff is going to be the kind of thing that people rely on for really important high stakes stuff. And I wonder if that's going to be the moment where all of this stuff at every level of the process is going to have to start to be a lot more sort of transparent and understandable on all levels, because otherwise we're just running like it's just black boxes all the way down. Right. I think that's right.

35:08I think especially, you know, you could see in healthcare, you know, there's already heavily regulated some sort of, you know, supply chain transparency mandate. Like you need to say who is annotating what in this system or something like that. Like that would be one way you could see it unfolding and maybe having that sort of with other, I mean, it's a supply chain, right? So it's like you need to sort of say, what is your impact at these various places? What are people getting paid? What are they doing? What kind of like quality assurance do you have? I think something like that would probably be better for everybody.

35:39You would have to move more slowly. But the number of tasks I saw in my brief time on there where it was clear that the engineers had no idea what was going on and there was this sort of international game of telephone happening. It happened kind of every time. I remember this task where I was supposed to be labeling sort of rare obstacles in the street for self-driving cars. And it would be like traffic cones, traffic control directors, wires, cables, like potholes of a certain size, things like that. And sort of every couple of minutes I would be doing it and then like log back in and it would be like, oh no, don't label traffic control directors if they're not directing traffic.

36:16Cause I had just labeled someone, you know, on a sidewalk having lunch, um, or like, okay, like a truck that has a bunch of traffic cones in it. And it's like, you'd label it and then come back and be like, no, only label traffic cones. If they're blocking an obstacle in the street or something. You know, it's just sort of like they don't realize sort of how their instructions are being interpreted and then what that does to the model. It's kind of this sort of horrible game of telephone that keeps happening that you would think would be cleared up if like the people working on the model were doing the annotating or were in the same room and communicating so that people knew kind of what the intention was.

36:50What is the status of your remote tasks career? Did you ever figure it out? Are you a good annotator? How are we doing now? I'm a terrible annotator. And then I got suspended, I think, because I was using a VPN. And now it just gives me this sort of sad face. I signed into my account. So my career is over for now. Which it sounds like is probably for the best on all parties, including those of us who want to use good AI. It's probably good that you're not annotating it. Yes. I think it's good for me and good for the AI. Fair enough. All right, Josh, thank you. I really appreciate it. It was a really great story.

37:22Everybody should go read it. Thanks. We need to take a break and then we're going to call up Jen Toohey and get deep in the weeds on how to build a smart home and frankly, what that even means. We'll be right back.

37:53makes it easy to post your job for free, share it with your network and get qualified candidates that you can manage all in one place. LinkedIn's new features can help you write job descriptions and then quickly get your job in front of the right people with deep candidate insights. Either post your job for free or pay to promote. Promoted jobs get three times more qualified applicants. At the end of the day, the most important thing to your small business is the quality of candidates. And with LinkedIn, you can feel confident that you're getting the best. Based on LinkedIn data, 72 % of SMBs using LinkedIn say that it helps them find high-quality candidates.

38:30Find out why more than 2.5 million small businesses use LinkedIn for hiring today. Find your next great hire on LinkedIn. Post your job for free at linkedin.com slash track. That's linkedin.com slash track to post your job for free. Terms and conditions apply. Support for the show comes from Udacity. AI seems to be everywhere these days. and if you want to prepare yourself to be able to use it, Udacity says they can help, no matter what field you're in. Udacity is an online learning platform with courses in AI and tech, including generative AI, Gentic AI, Python, data science, and much more. When you learn with Udacity, you're not just passively watching videos or reading articles.

39:12You're doing practical exercises and projects that prepare you for the job you want. That's why 90 % of Udacity graduates surveyed say that they achieved their enrollment goal. Udacity just launched a master's degree in AI. You heard that right. And when you have a certification from Udacity, recruiters and employers take notice. So for a better job, better salary, and better skills, you can check out Udacity today. The tech field is always evolving, and you should too. You could try Udacity risk-free for seven days. You can head to udacity.com slash verge and use code VERGE for 40 % off your order.

39:51Once again, that's Udacity.com backslash VERGE for 40 % off and make sure you use the promo code VERGE so they know we sent you.

40:05Welcome back. Like I mentioned at the top of the show, last week was Smart Home Week on TheVerge.com and we spent the week doing lots of stories about how to make your smart home work and the interesting stuff that people are doing with their smart homes and much more. It turns out, by the way, that The Verge staff has some wild ideas about smart home. There's a lot of good stuff on the site. Go check it out. But I haven't been able to stop thinking about this one story that The Verge's Jen Pattison-Tui wrote in which she basically tried to start at the beginning and answer a simple question. What is a smart home and why do you need one?

40:37The story is great. We'll link it in the show notes. But it really got me thinking. Isn't it weird that that's a question you have to answer and such a complicated one? How did we get to the point where a smart home is even a thing and should it be a thing? So after a few days strangely deep in my feelings about all of this, I figured I'd just grab Jen and we'd talk it all through. Hi, Jen. Hey, David. Always a pleasure to be here. So I was reading your stories and I had this like weird existential crisis I did not expect to have in the middle of reading your story. which is that I've never really thought about the question, do you need a smart home?

41:16And what is a smart home and what makes a home, a smart home, a thing. But the more I think about it, the more I kind of have come to the conclusion that we all talk about this exactly the wrong way that we think about it as sort of this like big holistic system that is like, you have to have a smart home and it's very complicated and involves buying a bunch of stuff and you have to pick a platform. And I kind of get to the end of this and I'm like, Maybe the answer is just buy some things that connect to the internet, let chaos reign and try to sort of figure it out over time. So I almost wonder if like, do we talk about the smart home in sort of the wrong way at this moment?

41:55Like, should the smart home even be a thing as we think about it now? Or should you just have stuff in your house that's on the internet? So you are completely correct. We should not be calling it the smart home and we should not be talking about it like that. However, we are and you kind of can't go back. It's like the cat's out of the bag. In one of my pieces that we published last week, we had a big Verge smart home week in case you missed it. Go check it all out. I talked about this a little bit that really the smart home is a misnomer because it's just the natural evolution of our home. Right.

42:28Our homes are just getting better. Just like, you know, we went from candlelight to electric light. We're going from non-connected to connected. And it doesn't mean that you need to go out and make a smart home. You're right. Like, it's great if you can. I would love to. Maybe not everyone. But if I bought a brand new house that had just been built and it was all connected and all wired and, you know, wonderful. But ultimately, for most people, you know, that's not going to happen. So, no, you don't need to go out one day and buy every single gadget and make sure you have everything working in your house.

43:01I think the most sort of natural approach to the smart home is adding devices as you need them to solve problems. And then once you get to a sort of a point where you have something like critical mass, you may want to start making them all work together. And that could be after you've bought two light bulbs or that could be after you've bought seven gadgets. And suddenly you're like, oh, hang on here. I can do, you know, I have a smart garage door opener and I have smart lights. I can have my lights turn on when my garage door opens. So, yeah, I think the problem with saying we all need a smart home is that that's a bit terrifying to people.

43:38It's like, oh, what am I doing wrong? What do I have to go out and do? No, you consider this natural. Something breaks. You need something. Consider a smart replacement. See if it's going to fit in your home. And I talk to a lot of companies about this and they all quite often say, yeah, I kind of wish we didn't call it the smart home because that's causing us problems now. and it's not always about the internet too. You've got to remember the smart home doesn't mean you're necessarily connecting your home to the internet. There are ways that you can run a smart home locally. So yeah, but we're stuck with the label.

44:12It's a label that people recognize and we don't want to rebrand like what happened to Coca-Cola or Pepsi. True, I do appreciate that the same companies who five years ago were ramming the phrase smart home down everybody's throats at every available opportunity are now the ones being like, Maybe we overshot and panicked customers about all of this. But that transition you're talking about, I think, is super interesting because it makes me think of if years ago when refrigerators suddenly got built in freezers, if everybody was like, I have a freezer home now. It just doesn't make any sense. But there does come a time where you go from, I have one thing that connects to the internet.

44:49Like, I have a garage door opener that I can control with my phone. two i have a home that can sort of be more than the sum of its parts and put some of this stuff together like you're talking about and i feel like the key question for a lot of people is kind of where that moment happens and where you go from like like my own experience is actually a good example of this right i have a couple of phillips hue lights around that i just control with an app on my phone i have a honeywell thermostat that's connected to the internet that i control with a different app on my phone. That's all fine. I don't actually need more system than that, but I feel like I'm probably one or two things away from needing more system than that.

45:27And also being able to do a bunch more stuff if I put all those things together. How do you identify that moment where it's like, okay, you go from just having a couple of things to like, you should build a system because it will actually make it better for you. So there are two answers to that. And in your case, I would say you're maybe one or two children away from wanting to do more because convenience is one of the big things and for me that was a tipping point having children once my children were older and you know we had more need for routines in our home the smart home is brilliant for routines and I think that's where that can really help parenting or aging in place or elder care or any kind of specific life use case where you could really benefit from something being automated rather than you having to do things yourself, saving you time.

46:17Working from home is another great example. You know, setting up a home office and adding smart sort of automations to your home office. I know one of our editors, Dan Seifert, recently set up an ingenious automation that turns on a do not disturb light outside his door when he's on a Zoom call. You know, things like those kind of solutions to problems that come up in your home. That's sort of one use case. And it's probably the primary reason to start looking at routines and automations, because that's when you need the platform is when you want things to work together, when you want more than just one light to turn on, when you want your light to turn on, your thermostat to adjust, your shades to open, your door to unlock.

46:56That's when you start needing these platforms and ecosystems and more of a cohesive smart home. The second answer would be, and this was a piece, This is one of the pieces that we had last week was looking at smart homes with energy, smart energy management. So basically building homes that are better. Like as we've gone from building simple homes without a lot of insulation or, you know, electrification, you know, now we've started to move towards better built homes that are better for the environment. They cost us lots of money to run. And now the next level is adding smart to that. We talked to people who lived in new smart community in California that it has been built from the ground up to use smart energy.

47:39It has like a smart electrical panel, has solar panels, but it's also been built to the point that insulation is so good that you don't actually need to use an awful lot of energy. So there's this kind of combination of looking after the environment, saving money and convenience kind of all comes together. and that's where I think the smart home sort of of the future has a lot of potential and if you go back to the beginning of the current smart home the diy smart home I know there have been smart homes for a long time but you know the start of where we kind of think of it today the nest thermostat was kind of the darling of the beginning of this current smart home and that was all about saving energy saving money you could feel good because you're helping the planet and feel really good because you were saving money.

48:25So I feel like that kind of a use case is one where we're really going to start to see not only individual people wanting to make their home smart, you know, retrofitting with a smart electrical panel, but we're also going to see society as a whole pushing us towards making our homes better. And, you know, there's government subsidies, there's all sorts of things going on in this space that are really interesting. And then the other use case there, I think I mentioned, touched on it earlier, is aging in place. And again, this is another huge sort of societal shift as, you know, the baby boomers reach their golden years.

49:02And, you know, we've got a huge influx of elder care that's either going to fall on the shoulders of the next generation or is going to fall on society, you know, the state. And aging in place is a really compelling use case for the smart home. If you can keep someone in their home for longer and not have to move into a nursing home or have to go to assisted living, you're really, you know, helping everyone. It's a win-win for everyone. So there's these kind of larger societal pictures that I think are often left out of the conversation about the latest smart doorbell or the best smart light. I mean, those gadgets are great and fun, but there is a much bigger picture here.

49:41Totally. And I think one of the challenges that I know you deal with in covering a lot of this stuff is figuring out kind of what is true now, what's going to be true in a year and what's going to be true in sort of 10 years. Right. Because I think that idea of having all of this stuff set up in a way that just kind of magically works feels to me like it's just not where we are right now. Right. Like it's we're still in the kind of like you have to build all the blocks yourself to make some of this stuff work. And it's getting a little easier and the platforms have made it easier to kind of build routines and stuff.

50:14But like you still have to say, when I get home, I want the thermostat to change. I want the garage door to open automatically. I want this light to turn on. I want this light to turn off and I want the oven to start preheating. The right way for all of this to happen is that that should just happen automatically, right? Or that some process of that should like learn how you live and adapt around you. And I feel like we're just at the beginning of that. And it's like starting to happen in just enough really cool ways that it's like I can see why that's going to be great. but it's not quite there yet.

50:41And so that's where I keep coming back to this idea of like, forget the idea of a smart home. Like, let's worry about a smart home in 2035 when AI makes all of this really great. For now, just like buy the coolest, smartest stuff you can find that works for you. And don't worry about like the big 30 ,000 foot system of it all until you absolutely have to. And I think that's probably bad advice because I think the thing that happens is you then kind of get out over your skis with too much stuff and then you have to like retrofit a system to it. And that doesn't work that well either. but I don't know we're in this like limbo position especially with things like matter which will solve some of this stuff that it feels like the less you worry about building a quote unquote smart home for the moment the the saner your life will be yes exactly I think you're right I think there are great use cases for individual or a few devices and if you have an issue you want to solve you want to save money on your HVAC bill you buy a smart thermostat you keep losing your front door key, you get a smart door lock, you know, and that doesn't mean you need a smart home just because you bought a smart door lock.

51:42I think you're right. AI could provide a huge shift in terms of that learning that our homes actually starting to sort of think for themselves and learning what we want and how our home should operate. I think ultimately our homes are going to become, you know, like our cars have become, they are going to be computers. But I think we're a long way from that still. I feel like, you know, for individual people today, you're right, don't necessarily need to think the big picture. But if you have one of those use cases we discussed before, if you're scared of the whole process, because it is daunting, it is confusing.

52:19And one of the worst parts about it is once you get it all set up, and it's all working great, and your routine is, you know, kicking off exactly as you wanted, and then it just stops. and then you're like what happened like now you've become like a network sys admin for your home and you spend your weekends troubleshooting why your smart sprinkler didn't go off and all your plants are dead or why yours you can't open your smart garage door right now you know and that's frustrating and that's where the smart home really starts to break apart when it causes more problems than it was designed to solve i'm sure that's a purely hypothetical example for you and not something you deal with every single day, all the time, every hour.

53:01Yes, yes. My house is, yeah. What's the most recent smart home chaos you've had personally? Oh, well, my husband not being able to open the front door because I put a new smart door lock on and he couldn't figure it out. And he was not happy. Yeah, this is why my wife just straight up doesn't allow most of these things. She's like, I'm not worrying about this. And when it breaks, it's not going to be my problem. Keys are great. It's going to be fine. And it's like, I'm like, yeah, you're right. But also, what if I put a cool door lock on? But on that point, though, and I should mention this, and it's not something we cover a lot, but there are integrators out there that will do all of this for you.

53:35And they have kind of mastered a lot of these problems and solved them for you, if you're willing to pay the money, because, you know, it costs money to have these people come out and do them, although it's gotten a lot less expensive. They've obviously facing a lot of competition from the DIY smart home. So you can have some of these experiences without having to deal with all the troubleshooting yourself, it's just going to cost you a bit more money. So I think, you know, it doesn't have to be frustrating. And as I said, you know, one of the things that I really, I think my biggest piece of advice for people when they're looking to embark on the smart home is, as you say, start small.

54:09But when you scale, don't scale all at once. Scale small. Because if you're like, oh, I love this, I'm going to buy 50 smart switches and 50 smart bulbs and wire them all up and get it all set up. I'm like, make sure you realize that smart bulbs don't work with smart switches because otherwise, you know, so add slowly, build slowly, you're less likely to run into some of those sort of more frustrating scenarios that we've talked about. Yeah, that's very good advice. So let's talk about the platforms though, because if, let's say, you've either hit the point where you kind of need to build a system or you just are the type of person who wants to kind of solve the system bit from the very beginning, you basically land on one of four platforms, right?

54:51There's Apple Home, Google Home, Samsung SmartThings, and Amazon Alexa. Are those kind of the four? Is there any other one that even sort of belongs in that same category with those four? Well, there's Home Assistant is another one that is a very popular option for more advanced users. It's the open source one that like, it's the one all the nerds really like. It's just true. Yes. Okay. And, you know, and it also, So it is more complicated. So, you know, I would generally recommend starting with one of the big four that you mentioned. I mean, there is a big sort of push and pull, you know, is it a platform?

55:25Is it an ecosystem? Is it a controller? But, you know, ultimately, those four are platforms that you can use to control your smart home. You can also tie those in to some of the other options out there, like Home Assistant or like Habitat or security systems like Abode or Vivint. You know, there are other sort of platforms out there, but they can also work with these four, one of or maybe more than one of these four big platforms. So these four big platforms do kind of that they're going to be present in every conversation about any kind of smart home setup. So especially if you want voice control.

55:59True. And it seems like maybe the simplest kind of heuristic about which one to choose is just to sort of look at the devices you already own. Right. That if you're a person who owns a lot of Apple stuff, there are good reasons to kind of go all in on Apple Home. And Sam, if you have a bunch of, you know, echoes around your house, is it kind of that simple that like the first question you should ask is what other devices do you already have? Yeah, I think it's what other devices you have on the smart home side and also just which phone you use, because these platforms are all pretty tightly tied to their smartphone.

56:30Although, you know, some of them are cross-platform, but you generally are going to have a better experience with Samsung SmartThings on a Samsung Galaxy device, with Google Home on a Google Pixel, and definitely with Apple Home on an iPhone because you can't use Apple Home on anything else. Amazon obviously does not have a smartphone, but that also makes it the broadest. So it works really well with any of the smartphone options. Plus, if you have any kind of Fire TV or any kind of Fire tablets, there's really good integration there. So yeah, I mean, ultimately, if you're like, I'm interested in the smart home, where do I start?

57:04If that's kind of your first question, then look at your phone and look at what you already have. If you already have an Echo smart speaker, if you already have a Google Chromecast, even if it's just one or two devices, if you have a Nest thermostat, that's a good place to start. You're not limited in most cases by that. So if you've already bought a device, it doesn't mean it rules out the other platforms. it's just a good place to start because you're already probably familiar with that platform to some extent. Totally. But okay, so let's throw all of that out then, right? Let's say I don't own any devices.

57:37I have never in my life bought a gadget. And I don't know why you're listening to the Vergecast, but welcome. You're a hermit and you've just moved in from... And you're trying to decide between these four platforms, are there sort of meaningful differences between them? It seems to me there are a lot of devices that work across two, three, even four of them. So the idea of like, I have to, you know, use this platform to use this light is less true than even it used to be. And it seems like in most ways, they're very similar, these four platforms in terms of like what they let you do and how they work.

58:07Are there big differentiators between them at this point? So originally, the biggest differentiator was which devices will work with which platform. And that has been a big issue. But that is what Matter was designed to solve, because it is meant to bring in cross-platform compatibility. So the idea being any device that you buy, if it has the Matter logo, it will work with any of these platforms. We're not quite there yet, but that is the promise. So I would still recommend buy Matter devices if you're not sure which platform you want to stick with. There just aren't a lot to choose from yet.

58:41So that was a big differentiator. That was an issue, especially for people who chose Apple Home because there weren't a lot of devices that worked with Apple Home and the ones that were were more expensive. They had, you know, the Apple tax. So whereas everything worked with Amazon Alexa, like everything, although how it actually worked with Alexa would vary. So there was a bit more Wild West with Alexa. Then Apple Home was very tightly curated. But in terms of platforms like features, each platform does have different features and different things that you might consider choosing it for. For example, Apple Home is very well known as its devices are.

59:19for privacy. It keeps all your data local. It doesn't use it to sell ads to you. It also works locally if you have an Apple Home hub. All processing will happen locally for things like video. So even though there is a cloud component to Apple Home, it does all work locally in your house, which a lot of people feel more comfortable with. It also means it's a little faster. Local control means that when you ask a light to turn on, it turns on almost as quickly as flipping a light switch. And, you know, that's the ultimate sort of goal of the smart home. Let's make turning a light switch on just as fast as flipping the switch, but way cooler.

59:55And then, you know, Amazon has a great integration if you want to buy stuff, surprisingly enough. Very easy to buy yourself toilet paper when you're, you know, stuck in a crunch, assuming you have a smart speaker in your bathroom. Alexa is actually one of the more innovative platforms. There's a lot of really kind of fun, unique features that it offers. One in particular is hunches, which is an AI generated kind of, it takes what it knows about how you use your smart home and makes suggestions. So, for example, if every night you normally lock your door at 9 p.m. and then one night you forget to lock your door and you say good night to your smart speaker and it will say, oh, by the way, did you realize you didn't lock your door?

1:00:35And that's its hunch. And that, you know, that can be useful. But obviously there are privacy implications there. you are basically letting Amazon know everything you're doing in your house. So if you don't feel comfortable with that. Right. But that is, as we were talking about with that kind of broader, your house learns how you work thing. If that is the goal, then Amazon is kind of one full step closer to that than just about anybody else. Very much so. And this is the real kind of push and pull with the smart home right now is privacy and data. You know, we want privacy in our homes. Completely everyone wants privacy in their homes.

1:01:07But if you want your home to be smarter, it needs data. It needs that context. That's the only way it's going to get smarter. So it's trying to figure out how we can provide data without providing intimate details of our life. And that's a real push and pull. And that's something that Matter is looking at trying to help fix. But I think it's also something that AI could be really instrumental to. Also machine learning on the edge. So devices being able to process that kind of contextual data on device rather than having to send it to the cloud. So you're not sharing your data in the cloud. And then with Google Home and Samsung SmartThings, they both have really interesting sort of, I think one of the things with SmartThings, if you're interested in energy management, that's a really good platform right now.

1:01:53It's working really hard and innovating really well in that space. Also, if you have SmartThings, Samsung Appliances, that's at the moment the only platform that really works well with integrating those appliances. It's we're still a long way from smart appliances being super useful, though. But, you know, that's the future. I think that we'll get more of that as we as the smart home expands and develops. And then Google Home, there's been a lot of really good innovation recently there. It's taken a while. I think the biggest selling point of Google Home right now is its Nest products. The Nest hardware is good.

1:02:28It's some of the best in class. And it also has a lot of local component, too. So it does the machine learning on the edge, a lot of AI processing locally rather than sending it to the cloud. But there is also, you know, Google is a data giant. And, you know, you do have to weigh that if you feel comfortable with supplying Google with your data. And Google Home is also probably the better of the voice assistants. That's what I was going to say. That's what it's been for me is the one that is most likely to hear me say turn on the office light correctly is Google Assistant by a mile. Yeah. I don't know if it's just me, but Siri is, I mean, Siri is a dumpster fire.

1:03:06Alexa's good, but Google Assistant, at least for me in most of these cases, is sort of at least one order of magnitude better than everybody else. I think it depends also what your main use case is, because the biggest problem I find with Google Assistant for smart home voice control is the delay. It doesn't have a local processing that Siri and actually Alexa does have on some of its devices. So I can find it can be quite slow. But Google does have a really neat, I love its presence sensing feature. I think its presence sensing is probably its best feature because it has any device in any Google Nest device in your home to determine if your home, not just your phone.

1:03:46Most of the other platforms rely on phone geolocation, which does not work for me because I live in a cell phone dead zone. I once was lying in bed. I had Apple Home geolocation turned on, turn the lights on when I arrive home. The lights just kept turning on and off, on and off. And I was just sitting there with my phone and it's like, Jenny arrived home. Jenny left home. Jenny arrived home. I'm like, nope, nope, still here. That was not fun. So whereas Google Nest can use like your Nest Protect smoke detectors, your Nest hubs, your Nest smart speakers, your Nest thermostat to tell if there's anyone in the home, not just you.

1:04:23That's the other thing. Like if you have kids who don't have smartphones and you leave them at home alone because they're old enough, they may turn the heating off because it thinks you're gone. But, you know, so I like that present sensing. And that's one of the things that Google does a good job for. And we have actually had, we had four or five writers on The Verge go and write about their personal experiences with each of these platforms. So if you kind of want to dive more and find out what they like about them, we've got lots of great content on TheVerge.com. Totally. Yeah, it's all very good.

1:04:52And I do think it's interesting to hear they've all very much sort of converged on how they think a lot of this stuff should work and like the apps all look increasingly the same which i think is really interesting but they do each still kind of have their own strengths and weaknesses they do and one of the nice things about matter now if you know is that we do have this multi-admin feature which means that you can use any of the platforms with any of your devices so if you have a matter a device, it will work with any of these platforms. So you don't have to commit. If you do choose a platform and you decide you don't like it, you should be able to move to a different platform relatively easily.

1:05:29As I mentioned, Matter is still in its infancy, but it is coming. We are going to have more devices that work with Matter and hopefully work out some of the bugs that we've had to this point. But yes, you can chop and change. You can try out different platforms. And if you get really good and really enjoy some of these platforms and you start to hit some of the limitations of the platforms, which are largely based on more kind of complicated automations. Like if you want your lights to dim only if it's after sunset and it's 30 degrees outside and your wife isn't at home, you know, if you want to add all these conditions and do some like really cool, we've got some crazy neat examples of people's fun automations.

1:06:08We had a piece that our staff wrote about their favorite smart home setups. There's some really neat, fun stuff you can do, like real geek out and you might want to move to a different platform you know from one of these four platforms sort of graduate to the next level so to speak um and you can do that like there's the beauty of the smart home is you really aren't completely locked in there is ecosystem lock in to an extent but the walled gardens are beginning to sort of crumble and we're beginning to be able to choose what we want and control it the way we want so that's what makes me excited about the smart home space that, you know, we're getting to the point where you don't have to choose your Apple Home device to work with Apple Home.

1:06:48You can choose what you want. Yeah, no, it's good. And I appreciate that, you know, you come on The Vergecast at least once a quarter and remind everybody that matter is great. It's going to be great. If everybody could just ship stuff with matter in it, it would make everybody's life better. Can we all please make this happen immediately? Love The Vergecast. I do feel that way. I am not, I am, there are a lot of people and I'm with them that think that matter has not done its job yet. I completely agree, but I don't believe that that means it never will. I think it's been hampered very largely by the platforms and some of the manufacturers starting to kind of shrink back a little bit and get a little nervous.

1:07:27And a lot of people kind of doing it. Well, we're just going to wait and see if this is going to be good for our customers. Whereas the customers are going, we want matter. Yeah, right. The answer is yes, everybody. That's it. But is it really, is it good for our customers or is it good for the bottom line? You know, we're entering the political stage of matter and I'm just hoping that we're going to get through it. Because, you know, otherwise we can all enjoy the alternative. Which is pure chaos. Exactly. What is the alternative? Like, what else are we going to do? Are we just going to throw this in the bin and then what?

1:08:01Go back to these walled gardens? So, yeah, we don't want to do that. Agreed. Awesome. All right. Well, we got to take a break. But Jen, thank you as always. We'll be back here to name and shame all of the people doing this wrong next time. Jen, thank you as always. We'll be back here to name and shame all of the people doing this wrong next time. All right. It's always a pleasure. Thanks so much, David.

1:08:24All right, before we go, we have a hotline question. This one is less of a question, as you'll see, but just something I think is interesting. And I have some thoughts. So let's just play it back. This comes from Didi. It's Didi from Maplewood, New Jersey. Love your Verge cast. I was just listening to your assessment of the Apple Vision Pro. You were speculating about what to do with eventually augmented reality glasses, of not being able to completely occlude your entire environment as the current Vision Pro can. So I just wanted to throw out there, well, what if the end game is simply contact lenses?

1:08:58Presumably, they would cover your entire retina, and then you could also presumably reproduce the entire VR experience. So maybe they'll do all-encompassing goggles and make them smaller and smaller, and then eventually go directly to contact lenses. Just putting it out there to erase your thoughts. Thanks again. So the funny thing about this is this has come up for me a couple of times in talking to people. And I think it is kind of the long term trajectory of this that people talk about, right? You go from headsets to glasses and then, OK, what's the next step after glasses? Obviously, it's contact lenses.

1:09:31And then you can have a debate about is the thing after that brain implants or, you know, what Neuralink is up to and stuff like that. But I think contact lenses is a really interesting kind of far end point for a lot of this tech that would solve a lot of problems. And I guess I have two thoughts. One is that I do think there are a lot of people in the tech industry who believe that. I think there are, you know, whiteboards that have the word contacts circled on it as the long-term goal. I truly cannot overstate how far away that is from a technical perspective. Obviously, things change quickly.

1:10:04Who knows what will have been invented a decade from now. But the number of essentially technical miracles that have to happen between now and then in power consumption, in how these things are produced, in the size of chips to make a contact lens possible is just staggering. Like we are several orders of magnitude of technical invention away from something like that being possible. Not to say we'll never get there. I just am not spending any time holding my breath. But I do think that is an interesting end point of this. And I think as a product becomes really fascinating. It also raises really complicated questions because they get harder to get off of your eye.

1:10:44That brings up different questions about like, are you more of a cyborg if you're wearing contact lenses versus if you're just wearing glasses that you can more easily take off? How do we power these things? How do they get software updates? How do they connect to the Internet? A million open questions. And I think one of the things we're going to start to see is if you, to pardon a pun, use contacts as a lens through which to look at a lot of technical advancement, you can start to see, are we getting closer to that as things get smaller, as things get more power efficient? Imagine a chip that gets hot that is sitting on your eye.

1:11:15It's just totally, incredibly implausible right now that that is going to happen anytime soon. But I do think there are a lot of people who are going to look at a pair of glasses and say, OK, we've done it. They look like Ray-Bans. What's next? So, DeeDee, I think you're right. I think that's a way we might be headed. I am unbelievably skeptical that we are even remotely close to being decades away from that. I think that is a really cool thing that still exists and pretty much only in science fiction.

1:11:47all right that is it for the verge cast today thanks to jen and josh for being here and thank you so much for listening there's lots more on this conversation a big story that we did in collaboration with new york mag that josh is talking about all of smart home week on the verge.com was amazing we'll put some links in the show notes but as always read the verge.com very good website one of my favorites if you have thoughts questions feelings or crazy smart home hacks you want to send my way. You can always email us at vergecast at the verge.com or call the hotline 866 verge one one. We love hearing from you.

1:12:20Send us all your thoughts and questions and ideas for what you want us to do on the show. We're going to have more hotline stuff every episode. Obviously keep it coming. This show is produced by Andrew Marino and Liam James. Brooke Minters is our editorial director of audio. The verge cast is verge production and part of the Vox Media Podcast Network. Neil, I, Alex and I will be back on Friday to talk about pixel tablets, Spotify, Sous Vides maybe, Reddit, and lots more. We'll see you then. Rock and roll.

1:12:55Support for this show comes from SC Johnson. We've all been there. Choosing not to wear your new white shoes because there's a 10 % chance of rain. Bending awkwardly over the tiny coffee table to enjoy a sip of your latte. Not ordering the red sauce. Those feelings of dread are what we call stain-ziety. But now you can break free from your stain-ziety with Shout's Triple Acting Spray that has stain-fighting ingredients to remove a huge variety of stains so you can live in the moment and clean up later. Just breathe and shout with Shout Triple Acting Spray. Learn more at shoutitout.com.

1:13:36Support for this show comes from Nordstrom. oh what fun holiday invites are arriving and nordstrom has your party fits covered you'll find head-to-toe looks for every occasion including styles under a hundred dollars dresses sets heels and accessories from bardo princess poly dolce vita naked wardrobe coach and more free styling help free shipping and quick order pickup make it easy in stores or online it's time to go shopping at nordstrom

From the publisher

Today on the flagship podcast of open-source lightbulbs:
David Pierce chats with Verge investigations editor Josh Dzieza about his story detailing how humans matter far more to AI development than we may have thought.
Inside the AI Factory: the humans that make tech seem human

Later, smart home reviewer Jennifer Pattison Tuohy explains why we're probably getting the idea of a “smart home” all wrong.

Smart homes for smart people

How microgrids and smart homes are shaping our energy-independent future

Every device that works with Matter

What is a smart home, and do you need one?

How to pick a smart home platform

From brilliant to basic, here are our smart home setups

Email us at vergecast@theverge.com or call us at 866-VERGE11, we love hearing from you.
Learn more about your ad choices. Visit podcastchoices.com/adchoices

Ask about this episode

In short

Segment 1

Guest

Segment 2

Guest

Segment 3

Transcript

More from The Vergecast