Meta's Open Source Music Generator: A Challenge Making Google Sweat

0:00Today on the podcast, we are talking about some big news as a new player has come out with a very powerful AI powered music generator. So we're going to talk about the implications, who it is, and as someone that personally has experience creating music, and I've made a lot of my net worth overall from, you know, royalty payments from Apple Music and Spotify, this is something that really interests me and I'll give some interesting insights into. But without further ado, let's dive into it. So the big company is Meta, who has now officially open sourced an AI powered music generator. Now, a lot of people are saying, you know, what's the big deal?

0:35Google recently has created music generator as well. But the difference is this is open source, meaning anyone can use this music generator without having to pay licenses or royalties to meta. And the Google tool, obviously, at some point will probably be monetized. So this is really interesting. And a lot of people I can see implementing this into their own projects in really powerful ways. Now, what is another interesting part about this whole scenario is the fact that, you know, when we're seeing big companies like Google and Meta come out with these tools, you know, one of the first, I guess, pushbacks we get is the fact that a lot of people are saying, you know, like, hey, we don't like it when you train music on musicians music because, you know, that's, you know, stealing from them, copyright, yada, yada.

1:20But Meta has said that all of the content that they have trained their AI model on, they had legal licenses and rights to use and train their AI model on. So this is really interesting. What they're doing is calling it Music Gen. And essentially, Felix Kruik, who made the announcement on Twitter from over at the Meta team, he said, We present Music Gen, a simple and controllable music generation model. Music Gen can be prompted by both text and a melody. we release the code MIT and models CC BYCN for open research, reproductability and for the music community. So honestly, really, really cool.

2:02You use it similar to how you would use chat GPT or something else, you can prompt it with text to create a song. So you could say something like an 80s driving pop song with heavy drums and a synth pad in the background. And it will create around 12 seconds of audio, give or take. And you can kind of steer it with, you know, references to audio or existing songs. And it's going to try to follow your description and your melody and whatever kind of input you give it. Now, of course, this thing isn't generating full on songs. But honestly, I think this is a really cool concept, even for an artist that, you know, like you're interested in some sort of style or some sort of concept and you come up with prompts and get it to generate, you know, a 12 second clip of a song and you're like, oh, dang, like, that's a catchy hook.

2:45I want to use this for X, Y, and Z. And you can incorporate that into whatever project you're working on. So I think this is a really cool idea, inspiration kind of project. And I'm actually going to play some samples on the podcast. And I'm going to compare this new, it's called Music Gen by Meta. I'm going to compare that to Google's Music LM. My personal opinion, you know, spoiler earlier is I think that this open source one by Meta is slightly better. But we'll go into what exactly, you know, I'll let you guys listen to it. And you guys can make up your own opinion. The one thing I do want to say is that it was trained on around 20 ,000 hours of music, including around 10 ,000 high quality licensed music tracks, and around 390 ,000 instrumental only track so a much larger you know instrumental only kind of thing has been added or data set has been added to what it was trained on and it's interesting because they actually got this from shutterstock and pond 5 so sources that like literally anyone could go and use to train which I actually think this is really cool using something like shutterstock or pond 5 or I'd be curious to see what the rules over like audio jungle would be but like for example if you had enough money, you don't have to go and kind of like schmooze and convince some sort of audio base company to, you know, sell you rights to their data to train an AI model on, you can literally just go to these marketplaces and buy a lot of this content if they have the right licensing set up and use it, which I think is really interesting.

4:18So the company has not provided the code it used to train the model. But it has made available pre trained models that anyone with the right hardware. So pretty much just a GPU with around 16 gigabytes of memory can run. So obviously meta isn't just giving you, you know, the access to how they did this, but they're saying, Hey, here's a model we've trained it and you can use it. Um, you know, open source free for whatever you'd like. So how does it do? I think that is the question that everyone's asking right now. Um, I think it's pretty good. It's probably not going to win any awards, but I will give you a sample.

4:53So this is a prompt that a TechCrunch journalist asked it for to output a jazzy elevator music. So this first one I'm going to play you is by Meta, and then the next one will be by Google's Music LM.

5:18Okay, so that's jazzy elevator music. And then this one is by Google's Music LM.

5:35All right, I'll let you decide which of those you prefer. So next they actually tried to give it a little bit more of a complicated prompt and the the prompt that they gave it was Lo-fi slow BPM electro chill with organ samples. So this is how music Jen did

6:00Come on, that's pretty sick. I was liking that one. Okay, then this is how Google's music LM did

6:17All right, not my favorite, but everyone's got their own opinion. So they then tried to switch things up a little bit, and they tried to get it to generate a piano song in the style of George Gershwin. So they tried, meaning pretty much Music Gen doesn't have really any filter, so it was able to generate something. but uh google's music lm has some filters that make it so that you pretty much you can't they pretty much block prompts that mention a specific artist and there's a lot of different um there's people that do similar things with images where like they don't want you to say do x y and z in the style of vincent van gogh and well pretty much they won't let you monetize or sell that art because you are like stealing copyrights from vincent van gogh essentially um but so google is trying to do the same thing, I think, with not allowing you to do anything that's mentioning a specific artist's name.

7:09But in any case, here's what music Jen came up with. And you can tell me what you think.

7:27Pretty accurate. So I think that music is coming a long way when we're kind of looking at this generative AI space. You can look at things like OpenAI's jukebox as previous examples of, in my opinion, a terrible job done on AI music generation. I think there's still a lot of different legal and ethical issues that they have to iron out and figure out. You know, people are complaining that it is learning from the existing music produced by people. And even if those people are, you know, selling their music on Shutterstock or something, I'm not sure exactly what the legalities or the licensing deals will have to be.

8:04So there's a lot of this, a lot of questions that are happening. A lot of different labels have been asking or litigating concerns around intellectual property concerns for deep fake music violations. There's been a whole bunch of viral tracks lately where it's in the style of Drake or in the style of other famous artists. So I think it's going to be a long time before we really get some concrete laws and precedent set on those matters a lot of different lawsuits making their way in the courts right now are going to impact how this music generating ai evolves into the future um you know right now you can generate 12 second clips that are kind of okay decent whatever workable you could use them to get some inspiration in the future um you know if we really worked on this ai stuff and this ai music stuff like you could see a world where you just say generate me a new hit drake song you know based off of these four songs I like and it will just come up with a perfect one and so I think that's what people are worried about um and I think my opinion on it is that I actually like what Grimes has done where she's offering to split 50 royalties with anyone that creates a song using a deepfaker for voice um and she's created some you know software to kind of do that royalty payment um I think stuff like that would be really cool personally you know sometimes I find a band or an artist and I'm like, oh man, you know, this is awesome.

9:25How come they only have like three albums out? And I just listened to their three albums, wish there was more. It would be super cool if I could just like auto generate like seven more albums by the artists and I'd be thrilled to give them all the royalties for, you know, uh, all the music that, uh, I, I generated, but I just want to hear more songs, you know, like that or like what they've created. And, uh, I mean, I don't really care if the AI is as good as a human. I don't really care who made the song. That's just me personally, right? Like, it sounds funny, but I absolutely love Johnny Cash.

9:55And if I could just generate another five Johnny Cash albums based off of a few of his songs I like, I think that'd be pretty cool. So it'll be interesting to see what happens into the future following this and kind of how the courts litigate what exactly is legal and not legal when training AI models to generate music.