AI Evolution: ChatGPT's Sensory Upgrade Unveiled

0:00So the big news here, there was recently a demo released, a lot of people are talking about it And in the demo, essentially someone takes a picture of their bicycle, uploads it to ChatGPT on their phone on the mobile app. I believe this is iOS. I tried to download this on Android. I'm not seeing it. So yeah, anyways, they took a picture of their bicycle on ChatGPT. They were able to upload it. And as soon as it was uploaded, they were then able to add the message saying, you know, help me lower my seat, my bike seat, right? And so immediately after sending this, ChatGPT is able to come up with a response that says, you know to lower your bite seat locate the release lever you know if it's a quick release lever do this tighten the quick release anyways it walks through the steps of doing it it says if you have further tools show me and I'll guide you through so then the person goes they take a picture of where the kind of release thing on the seat is they circle it on their phone so they're actually like drawing attention to the specific part they're talking about and they say is this the lever chat gpt says no that's not the lever it's a bolt you'll need an allen wrench to loosen it.

1:00And then it, you know, asks about what tools they have. So they go and take a picture of a tool set they have. And they also take a picture of, I believe the users or the owner's manual for the bicycle. And they say, you know, here's the manual and here's my toolbox. Do I have the right tools? And this is like so fascinating because it's using this, this like computer vision at the same time, it says, yes, you have the right tools on the left section of your toolbox there's a set labeled dewalt within this set find the nine millimeter allen head so it's literally like looking at the picture and it's like talking you through like on the right side of your toolbox on that little thing with this label on it like it's telling you what specifically to go and find go and do really really cool feature here so of course this is the ability that chatgpd now has to see this is kind of something that was rumored and talked about back when GPT-4 launched, but it never actually happened at the beginning of the year.

1:57Until now, it looks like they're finally rolling this in. It's kind of interesting. I recently saw a tweet on Twitter where someone was saying, like, they're like, you know, OpenAI is a big conference planned for November, but like, what the heck is there left for them to announce? Like, AGI? Because after this announcement, like, honestly, this is some massive capabilities, right? So now you can take pictures, upload it to ChatGPT, and ask it about the picture. So it's literally able to see. In addition, like I mentioned in the intro, there's the huge news of Spotify being able to, you know, do voices.

2:27So, or, you know, they're partnering with OpenAI to do voices. So they're going to translate podcast voices into different languages. So essentially clone a voice and translate it. I've also seen demos with ChatGPT where people were able to, you're able essentially to talk to it. So send an audio file where you are talking, you say, hey, ChatGPT, you know, tell my kids a bedtime story about a hedgehog that lives on the top of a mountain and chat gpt was able to say sure here's a bedtime story and like talk back to you so chat gpt can talk back to you you can talk to chat gpt you can take pictures send it to chat gpt ask it for context or information about the pictures that you're seeing it can see it can hear and most recently of course with the huge announcement that they're going to start integrating the brand new version of dolly 3 in there so it's also able to generate really interesting pictures i saw some interesting demos where essentially they were talking to chat gpt and saying like hey um you know tell my kids a bedtime story about a hedgehog it tells them a story about the bed about the hedgehog right um using the the voice feature but then they're like okay cool my kids really want to know what the hedgehog looks like can you like make can you like show us a picture of what this hedgehog would look like like if it was on top of a mountain in like a forest and so it like generates the picture and it's like okay cool my kids are about to go to bed like can you generate a picture of this hedgehog like as it's going to bed it does that same hedgehog and this is another thing that I think is really interesting with like something like mid journey you can try to use the same prompt to get a very like similar type of character so for example I've spoke with a number of people that have made like children's illustrated books and they have like keywords they put in there to try to get the same characters in the books that's one of the hard things is to make it really cohesive but they say things like, you know, like illustrated in like X, Y, and Z style.

4:09And they know that if they say like girl with black hair in this style, it tends to come up with a very similar looking girl. And so they were able to make a bunch of kids books doing this, but they were not like perfect. Um, and so they're not exactly the same person. It was like, okay. Right. But I think what's really incredible here, the chat GPT is going to have the capability of doing is because you can upload pictures and ask it questions about that. You can say, you know, generate me the picture of this hedgehog or this character in a book or this person or this thing it generates it and you say okay now generate that same thing but in this scene in this place doing this thing or that and all of a sudden Chad should be tea because of the way it's integrated with natural language processing and it has this computer vision element where you're able to upload and and talk it's able to do some really powerful effects where it's like essentially manipulating the same characters the same images to give you you know different variations of that which are going to be really powerful.

5:00So many incredible use cases are coming out with this. It's all over Twitter. People are talking about some really impressive use cases. But honestly, you know, recently we even had Lex Friedman talking about he's one of the first people that are going to be on this whole like Spotify thing. And he recently said he posted a clip of literally him speaking in English on his podcast and it translated into Spanish. He said, This is me speaking Spanish thanks to amazing work by Spotify AI engineers. The translation and voice cloning are done by AI languages can create barriers of understanding and thus fuel division.

5:36I can't wait for AI to break down this barrier and reveal our common humanity. Great job, Spotify. I'm excited for what the future holds. Check out the full episode of me speaking Spanish. So, you know, I mean, he's trying to he's trying to promote his podcast, which I've been by all means. That's awesome. But, you know, just then on LinkedIn had almost 10 ,000 people that have liked it in the last 21 hours. So really, really powerful technology. I think this kind of just goes to show like this is something people definitely want. I even for myself, I've been invited on a number of podcasts where they're like, hey, I'd love to have you on this podcast.

6:06We talk about AI, but it's in like, you know, we're based in Sweden and like that's pretty much our audience. and so like for me it's just like not super viable obviously i don't speak anything other than english and french so i'm kind of limited to those two lanes um but i think this is incredible uh being able to translate into all these different languages and clone your own voice absolutely incredible i i see you know if this is really what um open ai is building in you can see all of the different incredible use cases that are going to happen you're essentially going to be able to go travel to a new place and uh you know talk into your phone and it's going to in real-time translate for people so that or even maybe you wear a pair of AirPods with your phone that is listening so when people talk to you in another language it immediately translates into English you speak into your phone and it's going to in your same voice speak to them in French or Spanish or German or whatever wherever you're visiting and give them that you know you're going to be able to communicate so much easier there's gonna be some incredible applications built on top of this technology and of course on top of the technology that we're able to see things all around us.

7:08So, you know, imagine traveling, you're in another country, you take a picture of a street and you're like, hey, I'm looking for like, you don't know what any of the shops are. But it's like based on all the street signs, which one of these is like, which of these is like a shoe store or something, right? I don't know, it sounds funny, but there's gonna be so many incredible use cases. I think this is going to just further integrate AI into things. And honestly, I think right now, this is open AI is kind of secret to getting ahead because they're building APIs with all of this stuff. And so if you think about it, you know, right now you have like Anthropic that just raised$4 billion or has the ability to raise up to$4 billion from Amazon and AWS, you know, they're still just trying to compete on ChatGPT alone and just that tool.

7:47But as ChatGPT now is starting to integrate this image generation and the speech and this audio and all sorts of other things. And of course, I, you know, I find it hard to believe that video is very far away, I think, you know, a lot of their competitors are going to have to fight really hard to still be relevant because I think once people get all of these new features from ChatGPT, it's going to be really hard to convince yourself to go and pay for something like Anthropic if it doesn't have those abilities to have, you know, hearing, seeing, speaking, and everything else integrated. It's going to become incredibly convenient, and I think this may kind of push it to the mainstream beyond just people using ChatGPT to, you know, generate articles or responses and text for them, but all of a sudden it may be just an app everyone has to have.

8:27Everyone has to have premium and you use it all the time. It's like your travel companion. You can't live without it. I see it going more in that direction with some of these new features. So very exciting. Definitely something we'll continue to follow and a lot of amazing advancements to come.