Melodies of the Mind: Google AI Researchers Translate Thoughts into Music

0:00Parler tout français. Hablas espanol. Parler l 'italiano. If you've used Babbel, you would. Babbel's conversation-based techniques teaches you useful words and phrases to get you speaking quickly about the things you actually talk about in the real world. With lessons handcrafted by over 200 language experts and voiced by real native speakers, Babbel is like having a private tutor in your pocket. Start speaking with Babbel today. Get up to 55 % off your Babbel subscription right now at babbel.com slash wandery. spelled B-A-B-B-E-L dot com slash Wondery. Rules and restrictions may apply. What can 160 years of experience teach you about the future?

0:46When it comes to protecting what matters, Pacific Life provides life insurance, retirement income, and employee benefits for people and businesses building a more confident tomorrow. Strategies rooted in strength and backed by experience. Ask a financial professional how Pacific Life can help you today. Pacific Life Insurance Company, Omaha, Nebraska, and in New York. Pacific Life and Annuity, Phoenix, Arizona. Google researchers have just released a new AI model that can turn your thoughts into music. So this is absolutely amazing. And this comes on the back of an earlier release that Google has done called Music LM, which is a language model or a machine learning model to help turn stuff into music, to help generate music essentially.

1:30And typically this was more text generated was the concept, right? You describe what a piece of music looks like and is able to use AI to generate that. But now they're starting to integrate this with a bunch of other very interesting spaces. And one of them specifically is our mind. So this isn't super new to see a technology like this come into play when it comes to like our minds and whatnot. There's a bunch of research that has happened earlier this year where essentially they were able to train AI models to, you know, quote unquote, read our minds. But it's not perfect. It's pretty good. Definitely cause for concern if that is a concern of yours.

2:05Essentially how this works is they scan your brain while you're under an fMRI scanner. They have you read a script. They then take what essentially happened to your brainwaves, your brainwave responses, and they feed them through an AI at the same time as that script and that fMRI scan. And that script are then paired together and the AI learns what your brainwaves do when it reads a certain word. It does a certain thing. after that they're able to scan your brain and just have you think about something and the AI is able to tell you what your thoughts were and it's not perfect but it gives a very good idea of what you're thinking it's you know fairly accurate as far as the concept goes sometimes it would go as far as quoting something out of someone's mind not always word for word though there's a lot of really interesting implications if that's going to be used in TSA security or you know some countries like china are using some sort of ai mind readers to detect how loyal different people are to the communist party so there's all sorts of implications that are happening here with this ai and now i feel like this is a very interesting next step where google essentially is taking this to a new level and are kind of applying this to music in a very interesting way and essentially what happened in this experiment or in this piece of research was that they had they had their study group listened to different pieces of music.

3:26And while they were listening to those different pieces of music, they were looking at their brain patterns. They fed those through an AI and eventually, essentially were able to detect what the music looked like. They were almost able to reconstruct the music that you were thinking. And the reason this is interesting is because the implications for this are that if you, for example, when this technology gets better, if you could think of a piece of music, right? You're a Mozart, but you don't know anything about musical theory or how to get it down. You could just imagine a piece of music and this thing theoretically could do it.

3:57Now, one other really interesting implication that I've, that's just on the top of mind because I've heard this talked about a while ago and it sounded really far fetched, you know, earlier this year, maybe like three or four months ago, but there was an episode of the My First Million podcast. If you listen to it, where essentially they were talking about new AI startups and things that were coming out and they talked about a concept that I think was really interesting um sean and sam over there we're talking about the fact that at some point um you were going to be able to pick an artist that you liked let's say you love johnny cash and you were going to be able to essentially remix uh just his music using ai so essentially it's going to generate a whole bunch of new johnny cash ai songs now i think this is really interesting because now that we're starting to get into our own brainwaves it's like taking it a step further.

4:45And it's like, we almost could be able to actually create that music in a sense. You could also maybe imagine a song, feed it to an AI and have the AI like kind of like on mid journey, how they have an image upscaler where it, the way that it essentially makes it more high quality. You can do that with music because it's kind of interesting the way that like image AI works. I see it being kind of similar and I'm going to show you some really cool demos. If you're watching on Spotify, you can see what I'm actually clicking on. But if you're on Apple Podcasts, which I think like 70 % of you are, you're still going to be able to hear everything and I'll explain what's going on.

5:25But this is really interesting because the way that an image generator works, an image AI generator, is it's called the diffusion. And essentially what it does is it's like it's trained off of all this trained off all these images and it it generates like a bunch of like fuzzy screen like you imagine tv screen when it's like all fuzzy and white like that's what its first generation of the image is and then it like from that it slowly gets clearer and clearer less and less pixelated more and more color and it just like morphs from a fuzzy screen into whatever image you're kind of looking and that's how like the diffusion models work like that's very crude, but basic explanation of how those AI models work for image generation.

6:07And I see some very interesting parallels that could potentially be taken from diffusion image generators and applied to music generators, where essentially you get these kind of like fuzzy songs, and then you use some sort of musical diffusion to like increase the clarity on those songs until all of a sudden you have a new Johnny Cash hit that maybe was like based off of something that you kind of conceptualize in your mind. So that's just a whole new aspect to this beyond just the AI using its data set. Essentially your mind becomes the data set. And there's interesting implications too I think for text and image and video that could also come from this as well.

6:49But without further ado let's listen to some of the samples that they were able to get in this recent study. Let's listen to some of the musical samples. So if you're following along on Spotify, you should be able to see the screen here. This is the Brain to Music, Reconstructing Music for Human Brain Activity. So what we're going to play here is they have a couple highlights. So essentially the main concept here is that this is able to, obviously it's not going to recreate the song you'll see but what it is able to do is recreate recreate the mood the genre and some other you know vibes and this is obviously a very kind of preliminary early piece and um it'll be interesting to see how this progresses but this was the song that they gave them

7:48okay that was this the stimulus or whatever they were then able to you know reconstruct that with music lm this is what it sounded like reconstructed with music lm

8:07it's funny there's like words in there and of course they don't actually make sense but like it is able to reconstruct that kind of same feel

8:26This is literally reading from their brainwaves to get these.

8:40Okay, so that was the first song. This is the second song.

8:57All right, that was the song they gave them, and this is what they were able to reconstruct.

9:10Oof, that one wasn't that great, but I mean, it captured a female singer, it captured a beat that was similar, and it captured the same mood. I didn't say these were going to be masterpieces, but this is pretty cool. Okay, here's the next one.

9:32All right, and this is the third.

9:41Super, super interesting. All right, so for the final and third one that I will show you, because actually if you go to the Google Research GitHub.io and find, they've done a ton of different ones, and they have a ton of really great explanations over this whole thing. But I will show you one last one that they were able to do. This was a stimulus.

10:11And this is what was able to be generated after the fMRI scanned the brain.

10:27Second one.

10:38Interesting that one actually had words that were in it, although of course they didn't actually mean anything. And this is the last one.

10:51So this is really interesting how this works is essentially they give them the music stimulus, they get the fMRI response, they put it through linear regression, they then do music embedding, put it through music LM and then they have the music reconstruction that is generated. This is absolutely incredible. They've done this with a ton of different people, a ton of different genres that are super, super interesting. And it's just really impressive that this AI essentially is able to, you're able to listen to something and it's able to detect the genre of what you're actually listening to. And I'm not sure like what a benefit or what all of the applications of this would be, But you can imagine a world, for example, where, I don't know, perhaps you're going into like a store and they want to know some sort of detailed bit of information about you.

11:37So they try to scan your brain as you walk in to listen to what's on your AirPods, reconstruct and use that for highly targeted, you know, product placements around the store. I don't know. Like there's some crazy, there's some crazy ways that this can be used, this potentially will be used. So it's going to be very interesting to follow this, but definitely a very interesting piece of innovation out of Google. And we'll definitely be following to see where this goes in the future with this AI mind reading to detect, you know, what is in your brain and essentially create music straight out of your thoughts.

12:09Thanks so much for listening to the AI chat podcast. Tune in next week or tomorrow, as I usually post every day. Make sure to subscribe to our newsletter on AIbox.ai. Join the wait list for our new AIbox.ai software that is about to launch. And we will see you next time.