The smells and tastes of a great video game

0:00Support for this show comes from MongoDB. You're a developer who wants to innovate. Instead, you're stuck fixing bottlenecks and fighting legacy code. MongoDB can help. It's a flexible, unified platform that's built for developers by developers. MongoDB is ACID compliant, enterprise ready, with the capabilities you need to ship AI apps fast. That's why so many of the Fortune 500 trust MongoDB with their most critical workloads. Ready to think outside rows and columns? Start building at mongodb.com slash build.

1:01We talked about the N64 Joystick Crisis, both super fun episodes. If you haven't listened to them, go do so. But I would be lying to you if I said this is not the episode I was the most excited about. We gave our producer, Andrew Marino, I would say the roughly impossible task of taking the two senses that make the least sense in the context of this series and trying to figure them both out. Those two senses are taste and smell. How do you do that in games? Let's find out. Andrew Marino, welcome. Hi. Hi, David. You knew I was going to give you this episode, didn't you? I would have fought for it.

1:36I'm glad. So where do you want to start? I feel like we just sort of sent you on a like, God only knows where this is going to go adventure. Let's see. Let's see what you can come up with. Yeah. Where do you want to start? Okay. So let's just bring up the idea of where smell would even go in this stuff, right? Okay. So you may remember the term smell-o-vision. Oh, yeah. Yes. I honestly could not tell you if that's a real thing that ever existed or not, but I have heard of Smell-O-Vision before. Yeah, people just use it as a term for like broadcasting smells, you know, but it was actually a product at one point.

2:10There was a movie in 1960 called The Scent of Mystery.

2:20And it was a movie that used this technology called Spell-O-Vision. They had this whole machine built where there was kind of like how a soundtrack in a film strip would be. There's just triggers for when scents would come. Throughout the film, there was 15 scents that were deployed into the audience. so there was a villain in the movie who smoked a pipe so whenever that villain was on screen you could smell the tobacco from his pipe and it was supposed to be like the future of cinema they were just like we got sound we got visuals now we gotta add another thing and that's smell the other important part of movies now like you still kind of see that i don't know if you've ever been to one of those four dx theater experiences those are the ones with like the seats that rumble when there's action sequences and stuff, right?

3:12To that extent, I think 40X is super fun. But every time there's one of these, I remember this thing from, it was the Honey, I Shrunk the Kids thing at Disney World, which to me is like the canonical version of all of this to me. And there's a moment in it where a bunch of rats are scurrying around and they actually had the rats scurry underneath you. And it freaked me out, like genuinely, earnestly scared me. I am still scarred from that. It's so funny you brought that up. I remember the tails. There's tails on the bottom of the chair. It's awful. I don't like it at all. And so that forever has ruined the idea of 40X for me, even though I think the rumble when you're in the car thing is actually pretty cool.

3:50Yeah. A lot of them don't do the scents. Apparently, there's a theater in Japan that has like a thousand different scents you can load into a movie. But we're talking about video games here. So how do we do this in video games? Yeah, that seems like a really different problem to me. You can't just like load the track with the smells, I guess. Right. Because it's so interactive. Once we got into like the desktop era of computers, there was this company called iSmell. Amazing. Which is just an amazing name for a person. And this is in the 90s. So they were ahead of their time on iSmell. Yeah. In 1999, there was like a USB device you'd plug into your computer.

4:28It would work with websites and programmers to install these smell activation files, where it would release a scent when you click on something in a game or a website. You can imagine Tomb Raider, you're going through different spots and smelling mold and mildew and animals and tapestries. And the shoot-em-up game, of course, that can get pretty gross. When you say this game stinks, it might be a compliment. That never took off. In 2006, PC World magazine listed iSmell as one of the worst tech products of all time. Oof. That's tough. So there's a lot of challenges there. I don't know if people were ready for scent in video games.

5:10We're still getting the visuals and sound right at that point. But now, technology has progressed. David. Guys, guys, guys, guys. Smell-o-vision. It's real. There's been lots of prototypes I've seen. Linus Tech Tips did a video with something called the Cilia, which worked with a game called Adventure Climb VR. Oh my God. I can totally... I can totally smell the water. This fan will blow scents on you if you're in like the wooded area or like near water. Okay. There's some innovative stuff there. but I actually found one product that is shipping does add scent to your game and they promise to work with any video game console and any streaming platform it's called game scent have you heard of this I'm guessing this is the sort of thing that we would have seen and been like well that'll never amount to anything and then just all sort of forgot about in this long history of smell products yeah but you're telling me it's real it's real we're here smell-o-vision is back in games.

6:16It's made by a company called Elevated Perceptions. You can buy it right now on Amazon. You can hook it up to your Xbox. You can hook it up to your Switch or whatever. It's like this hexagon box that you hook up to your console. It has six spray nozzles, each with a different fragrance. So when you're playing a game, it will just eject some... It's like an oil diffuser. Okay, sure. Sure. And it's powered by AI. Oh, God. Andrew. Well, that's the end of the episode. No, David, hear me out. AI is going to solve smell of vision. Okay. We're finally at a point where this is actually going to work. Because, you know, previously in all these examples I talked about, you have to map it to the game, to the movie before you play it, you know?

7:04Like the track you're talking about, it actually doesn't work if you don't have the like cue points for the sense. Exactly. So you'd have to work with the video game developer. You'd have to work with the movie distributor to sync everything. But what GameScent is doing instead is you're actually just running an audio signal to the GameScent from your game console through this little adapter, which syncs to that hexagon box called the atomizer. And so the adapter is listening to the sound in the game, sending it to a cloud server, and then some AI software is figuring out what those sounds are.

7:39And then it tells the atomizer to deploy those scents. So it's related to what you're hearing in the game. I can't tell if you just described something really clever or that is like the most hacky hack in the history of hacks. And I think it's actually probably both as I'm thinking about this now. I think so. Yeah, it's super clever because, yeah, you don't have to map it to any game. You can use literally any game. You can watch literally any movie with it. It's a lot of smells though, Andrew. What are the scents in this thing? It's a lot of smells. The starter pack that comes with it is gunfire, racing cars, explosions, forest, storm, and then a neutralizing scent or called clean air scent.

8:24So those are the six that come in the box. I love you can just tell what kind of game and movie they're designing this for that three of them are just action movie things yeah so like the game scent hears gunshot noises it will deploy the gunfire smell if you hear leaves rustling and birds chirping you might smell the forest scent from the device like if you played this podcast through the game scent i'll just play some thunder and rain sounds right now

8:57the game scent should hear that and then it will deploy the storm scent so they want people to use this for movies and whatever too okay so you mentioned a few minutes ago that this thing is available to buy on amazon i'm begging you to tell me that was foreshadowing and that you have one of these things you can tell me how it works oh yeah i got i got the game scent so after this little break we'll have, I'll get into my experience with using the game scent and see how it actually does with Smell-O-Vision in 2024. Oh my God, the suspense is killing me. All right, we'll be right back.

9:34Support for this show comes from Shopify. One of the most exciting parts of starting a business is when the sales start coming in. If your business is looking to start selling online, Shopify has everything you need to create your online store. Shopify is the commerce platform behind millions of businesses around the world and 10 % of all e-commerce in the US, from household names like Mattel and Gymshark to brands just getting started. With hundreds of ready-to-use templates, Shopify helps you build a beautiful online store to match your brand style. And Shopify is packed with helpful AI tools that write product descriptions, page headlines, and even enhance your product photography.

10:13And best yet, Shopify is your commerce expert with world-class expertise in everything from managing inventory to international shipping to processing, returns, and beyond. Turn those dreams in two and give them the best shot at success with Shopify. Sign up for your$1 per month trial and start selling today at shopify.com slash vergecast. Go to shopify.com slash vergecast. Shopify.com slash vergecast.

10:47All right, we're back. Andrew Marino, you have a GameScent in your life right now and you have not told me about it. I am desperate to know about your experience with the GameScent. How'd it go? Yeah, I was so excited to play with this. I reached out to GameScent and the company sent me a unit to try out for the Vergecast. When I got the package, immediately it had a very strong scent. Yeah, I think some oil leaked or something because it's very strong. So as I was opening the glass vials of fragrance, I noticed that one of them was actually shattered. Oh no. The clean air scent. It's broken. I wonder if that's why it was all oily.

11:33What I was smelling was clean air, apparently. Better that than gunfire or explosions, though, I guess. Yeah, yeah. So all these little vials of oil, they probably should have used different canisters. Like they're just generic glass containers. You can screw the nozzle spray thing on it. But yeah, so I didn't have the neutralizer scent anymore. But I felt that was OK because my entire apartment smelled like clean air for a few days. So you just had infinite neutralizer smell for a while. Yeah, exactly. But it was actually pretty easy to set up in my living room. Usually, as you know, devices that are connected to the internet and Bluetooth can be a pain in the butt.

12:13So I was actually surprised how easy it was to set up. I hooked it up to my Nintendo Switch, plug it into the audio adapter. I put the atomizer on my coffee table, and I was sitting on the couch, plugged it in the wall. And the first game I wanted to try it out with was Mario Kart.

12:34For the first lap or two, it wasn't really doing anything. I couldn't smell anything. But then suddenly something came through outside of that clean air scent. Oh, I'm getting a new smell.

12:51Oh, I don't know. I don't know what it is. So it was working, but some of the scents were just like too hard to figure out. Like storm scent versus forest scent. Can you see the scents coming out, by the way? Is there a steam thing coming out? You can see it working. There's like a cloud of puff coming out, and you will smell it for sure. Even if you can't tell exactly what it is. Yeah. Eventually, it started to make sense. Oh, I'm getting a very burnt kind of smell. I think that's the racing car smell. I'm on the third lap, and it just triggered that. So it kind of smells like burning tire. I got the racing car smell from playing Mario Kart.

13:37As you can expect, that just kept going off for a bit. Right? Yeah, it's just racing cars the whole time. Right, because you're always hearing turbo boosts and engines revving and all that. Some other things I would activate the game scent during Mario Kart was Grumble Volcano, that course. There's like an erupting volcano in the beginning, and that deployed the explosion scent. bullet bill that big bullet that whizzes past you or hits you depending where you are in the race that got the gunfire scent oh this is a new smell it's very smoky so uh so overall it does work i also played this with animal crossing i know these are probably not the best case scenarios but they said you could use it with anything yeah i got the forest scent immediately when i was in Animal Crossing on my island.

14:29I have a lot of trees. So overall, it does kind of work. Does it add anything though? Do you feel like it made Mario Kart more exciting or immersive to have this working? Yeah, I don't think so. I think maybe if I'm using it longer over time and I'm not noticing it, like if I crashed horribly in Mario Kart, I smell that burning tire scent. It might be a little bit more powerful. Also, you kind of have to have the atomizer fairly close to you to like immediately smell it when it happens. If you put it a little too far away from you, it might take a bit for the scent to come up. But if it's more experiential environment thing, you may not change scents very often.

15:14Like if you're in a forest in Zelda or something, you're probably there for a long time. But I'm not sure if it's all there yet. Like I'm not more immersed in Mario Kart. Well, I have sort of two theories about that. One is that maybe the problem is just that smell doesn't work like sound does. It sort of sounds like you're saying, what if the soundtrack of the movie just played constantly for an hour? And I was like, well, they wouldn't do anything, right? The fact that you can play music at specific points that does specific things is actually really powerful in a way that just diffusing oil into the air, you just don't have that kind of control.

15:51right but also then i'm like what if there were just way more smells what if instead of six it was like six thousand and it could be like small explosion big explosion you're dying in the explosion like maybe maybe we just instead of doing less of this we need to do much much much more of it that's what i think i think six cents are not enough and technically it's five because there's like the neutralizing odor right so i don't think it's there's enough yet for a full immersive experience but i talked to the people at game scent they were experimenting with daisy chaining a bunch of these so you can buy a bunch and maybe put you know multiples of six cents in your gaming setup so like forest plus gunshot equals war yeah oh totally yeah uh and they do plan on adding more cents to swap out uh right now on the website it says there will be a ocean scent, a sports arena scent, fresh cut grass.

16:52There's a zombie smell that's going to be coming. That's what they told me. This is my new favorite job in the tech industry that I am desperate to know much more about is the smell designers for video games. That's a life goal right there. Yeah. They work with a third party company to design these custom scents for the game sense. So it's not just these stock ones that they just buy. But I would love for them to make like a bundle for certain games. If there was like a Mario Kart bundle, I could have a racing car scent and like a banana scent for the banana peels in the game. And then an ink smell for when you get bloopered.

17:29That would be cool. And you could, you know, have different packs for different games. That's a really fun idea. You buy the$70 game and for 15 more dollars, you get all the game smells. Yeah. I like weirdly feel like that would work. I think so too. And I talked to the president of the company and the VP of marketing at the company about this. And they like want to work with companies and movie production houses to make custom sense for games and movies. They said they would love to partner with like Dune and figure out what the spice smells like in Dune, you know? Oh, that's, that's very compelling.

18:04Yeah. So David, would you use something like this for movies or games? I think games, I actually find more interesting than movies. And I think actually your spice example is sort of the perfect one where to me the idea of having like lots of different smells kind of coming and going as you play a game or watch a movie is probably just too hard to pull off like just because what you're talking about like it has to diffuse oil into the air at a particular time but if i could sit down to play dune and it just kind of smelled a little like spice it's like you know how when the room smells like popcorn like every movie's a little more enjoyable like it's just that little thing that if i could just kind of or the thing people do where they like sync up the lights to the movie.

18:45Yeah. Just having that one little additional thing where instead of having everything on screen have a smell, you can just sort of have the game have a smell or like the world. Like you mentioned the fresh cut grass thing. And like I play a lot of FIFA and the idea of having that where I can sort of feel like I'm on a pitch even though I can't smell the sweaty dude next to me. It just kind of smells like I'm on a soccer field. That kind of does something for me and feels maybe slightly more achievable. I'm just not convinced that the like soundtrack a game with smells thing is even technically possible but I hope I'm wrong like I I love the idea I think it's great that people have been working on this for like what 50 years now I think it'd be awesome if it worked I'm just not convinced it will like even in that movie I mentioned the scent of mystery there was like a point where it will deploy a perfume scent of one of the characters that was off screen and so you knew this character was about to come in kind of thing.

19:41So there's like a storytelling aspect to it that you could totally take advantage of. I love that. You can smell the bad guy before the bad guy appears on screen. Oh, totally. Yeah. This is great foreshadowing. It really is. Yeah. I want one. I'm like, this thing is on Amazon, which is a real problem because I'm now tempted to buy it just because of the sheer sort of delightful silliness of the whole thing. Are you going to keep using yours now that you don't have to for the Vergecast anymore? Do you think it'll still stay hooked up to the Switch? Probably not until they get new scents, I think.

20:11It's pretty strong overall. And after a while, it's like, I actually don't want to smell this forest smell. A lot of it's very perfumey so far. Has it lingered in your apartment? Do you come home and you're like, smells like burnt tires? Well, the box does. The box with all the oil still inside it. That's not great. That's not great. It's okay. It neutralizes a lot of other orders in my apartment it's perfect i love it but david we're not done today we still have one more sense to talk about oh right oh man which is i would say if anything i would assume much weirder than this particular journey that you've been on oh yeah possibly uh we'll see uh we're gonna tackle the question what if you could taste your games oh man so after this break we'll get into that i'm so excited we'll be right back

21:06Oh, hey, welcome to gift wrapping. Whoa, Zoe Saldana. Hey, can you wrap these, please? Wow, iPhone 17s. You splurged. At T-Mobile, you can get four iPhone 17s on them. It's the perfect gift for everyone. I'm the worst. I only got my mom a robe. Well, it's better than socks. So I have to trade in my old phone, right? No, at T-Mobile, there's no trade-ins needed when you switch. Keep your old phone or give it as a gift. Incredible. In fact, wrap up my old phone, too, for my Aunt Rosa. Forget that. Aunt Liz will be jealous. Sounds like my family drama. Oh, I got it. I'll give it to my abuela. I'll take reindeer paper with...

21:39Hey, where are you going? To T-Mobile! Thanks, Zoe! The holidays are better at T-Mobile. Get four iPhone 17s on us. No trade in needed when you switch. Plus four lines for just$25 a line. With 24 monthly bill credits and four eligible portings on essentials. For well-qualified customers, auto pay plus taxes, fees, and$35 device connection chart. Credits end in balance due if you pay off earlier. Cancel. Contact us. Finance agreement. 256 gigabytes. $830 required. visit T-Mobile.com. Welcome back. It's time for maybe the weirdest sense in the five senses of video games, taste. Andrew, it turns out smell has a decades long history in movies and video games.

22:20Does taste? Have there been people trying to make you taste video games since the 70s? Well, so the first time we started talking about this, I remembered this 2017 podcast we did, Our colleague Addie Robertson interviewed Jaron Lanier, who is a pioneer in that first phase of VR from the 80s and 90s. At the time, he was promoting a book. And in the book, he mentioned this VR taste simulator he tried from a Japanese company. And then he talked about it on the podcast. The next time I was in Japan, I tried this crazy thing. And it was really hard to sterilize between users and really hard to initialize and really crazy.

22:59but it was like this weird little morphing robot with little chemical squirters and you would stick it in your mouth and start chewing on it. Wait, did it just look like a blob, like a little blob robot? It was like this rubber thing with all these little robotic effectors inside it that could collapse or move. And then it had like these little nozzles that would squirt flavors out of little holes in it. I know, that sounds really disgusting. It's absolutely the most repulsive user interface item I have ever seen. A, heck of a sales pitch. B, I agree that does sound sort of disgusting. Yeah, that is bizarre.

23:33I would love to have seen what that looks like. I would love to try, honestly. But you can imagine something like that is very hard to emulate. Taste, putting something in your mouth. I haven't seen anything remotely close to what he was talking about there. But I did find someone who is working on something in the world of taste simulation. I'm Nimesha Ranasinghe. I'm an assistant professor at University of Maine in the School of Computing and Information Science. So I talked to Namesha. He was previously in this research group at the University of Singapore, focusing on human interaction and virtual reality.

24:14And he specifically was interested in incorporating taste, sensation and simulation into this. So the best way that he was figuring it out was through controlling electrical currents and electrical pulses onto your tongue to give different taste sensations. So like, you know, if you lick a battery, if you've ever done that. It's kind of like a soury kind of sharp taste. So he built on that idea. So I thought like maybe we can focus on that and we can fine tune the electrical stimuli, like reduce it to very, very, very low electric current around like 100 microampere. And then start studying like when we change the electric current and the frequency as well as the different areas of the tongue when we stimulate, how we can sort of relate those electric stimuli, like perception, into like actual basic taste perceptions like salty, sour, sweet, and bitter sensations mainly.

25:16You know, different currents attacking your tongue, maybe changing the temperature to make it hot or cold for like a spicy or minty feeling, you know? I had no idea this was like understood science that you could essentially push a taste onto someone's tongue with electrical currents. I guess I shouldn't be surprised by this, but that's very cool to have just learned about just now. Yeah, it's pretty wild. Apparently it works. There was an early prototype they worked on that was like a 3D printed spoon with electronics inside it and little electrodes underneath the spoon. So when you put it on your tongue, it would like zap these sensations.

25:55Whoa. So you bite the spoon and it's like cinnamon toast crunch. And then you bite the spoon again and it's like spaghetti i don't know if it was that advanced okay but it was like oh maybe this tastes salty now this is super sweet you know they also had like metal chopsticks they worked on that was like a plus and minus kind of circuit they tried a soup bowl all these things they wanted to make sure it all represented stuff that people were already used to putting in their mouth. Sure. We wanted to see like a range of different food items and like a semi-liquid kind of food items like porridge and soups and how we can sort of augment the flavors based on the electrical stimulation.

26:40So that kind of thing only manipulates primary taste sensations like you mentioned salty sour. They also realize that individual senses influence each other so that can be a powerful tool they started adding like smell to give a more immersive experience in this drinking machine much like the game scent you could engage food and drink smells in which totally influences the way you taste and then also they want to incorporate sight so they're experimenting with color if you go to a bar they have these like beautiful like range of colors cocktails and each and every color sort of gives us like some kind of like a pre sort of conceived or like an idea like this is going to be sweeter, this is going to be sour, this is going to be sweet and sour and bitter, kind of like some idea, right, based on the look and feel or based on our previous experiences.