Amplifying Success: Riffusion Secures $4M Funding from Viral Phenomenon

0:00The wait is over. Dive into Audible's most anticipated collection, The Best of 2025. featuring top audiobooks, podcasts, and originals across all genres. Our editors have carefully curated this year's must-listens from brilliant hidden gems to the buzziest new releases. Every title in this collection has earned its spot. This is your go-to for the absolute best in 2025 audio entertainment. Whether you love thrillers, romance, or nonfiction, your next favorite listen awaits. Discover why there's more to imagine when you listen at audible.com slash best of the year. So a year ago, there was Seth Forgren and Haig Martinos.

0:44They were both developers and they unveiled Refusion. So this was a really unique project that essentially kind of harnessed the power of images to generate music. Now, this might sound counterintuitive, but it was a huge revelation. So as millions of, you know, curious users discovered this whole thing, Refusion not only piqued the interest of music enthusiasts, but also it was really popular. A lot of, you know, big tech companies were looking at it, specifically Meta, Google and, you know, TikTok. So they all kind of cited it in recent research papers. So I think this really intriguing venture got a lot of investors also attracted to it.

1:21So today, Ferguson and Martiro have decided to take Refusion to the next level. And they have just closed a seed round of$4 million. And they have, you know, Graycroft, which is leading that round, which of course is a massive, massive VC firm top, you know, top 10 VC firm in the US. And I think it's also interesting, they also have some, you know, big celebrities, they have, you know, the chain smokers, I believe invested in there. And I think the chain smokers have their own kind of VC fund that they have started. So that's probably why they're interested in getting into this space, particularly.

1:54and then I think they also have South Park Commons and Sky9 that are investing. So in any case, Refusion right now is launching an upgraded version of its platform, which is of course a free-to-use app that lets people, you know, you describe lyrics and a musical style and then it can help generate shareable riffs. So as Foregrin explains, the new Refusion empowers anyone to create original music via short shareable audio clips users simply describe the lyrics and musical style and our model generates riffs complete with singing and custom artwork in a few seconds from inspiring musicians to wishing your mom uh good morning riffs are a new form of expression and communication that dramatically reduces the barrier to music creation i think this is really interesting like there's a whole bunch of um people that are interested kind of in the space but this is definitely one that's been there from the beginning.

2:48And it's, you know, it's been there quite early. It's obviously very popular. It's cited in a lot of big papers from a lot of big companies. But I think this is such a fascinating area where really music has been democratized. And I know there's going to be a lot of people that complain about this type of technology. The same thing with like Mid Journey, a lot of artists complained that, hey, you know, if anyone can create art on Mid Journey, you know, what's the point of our skill? There's, of course, a whole reckoning that happens there. I think it's going to happen in music. I think it's going to happen in audio podcasts like this one you're listening to right maybe i'll be an ai someday i don't know um and then of course video will be a really big one too that's coming down the pipe so forest grin and martyrios go way back they both actually met at princeton during their undergrad and they spent the last i think decade just like jamming together in an amateur band so um they kind of bring their experience from founding two venture-backed tech companies hardline and yodel and materios worked as one of the early employees at a drone startup called skydio and so i think the pandemic which kind of brought you know more time at home inspired forrest grin to learn to play the piano and as he says you know he he reflected on the power of music during times of isolation and sees potential for generative ai tools to connect people through creativity so it's kind of interesting he specifically said quote generative ai is a new and rapidly changing space and Refusion aims to harness this technology to deliver a fun new instrument, one that empowers everyone to actively create music throughout their lives.

4:18I think the enhanced Refusion like now that they have is powered by a custom trained audio model. So like the one they had before, it's fine tuned on spectrograms, which visually, which are essentially visual representations of audio that depict the amplitude of different frequencies over time. So So Fourskrin and Martyrus created spectrograms of various like music genres and tagged these images with relevant terms like, you know, blues guitar or jazz piano. And this approach essentially helped the model learn what different sounds look like and how to recreate or combine them based on a text prompt.

4:57This is so interesting, right? It's really, when they kind of explained how they've done this is one of the first times I've actually really looked into how music generating AI works. And it's so interesting that like this, at the end of the day, this is like pretty much you can do this with stable diffusion essentially, right? Like this is the same way you generate images. Like it's trained on images of what the audio looks like. And it's not necessarily trained on, and I'm sure there's, you know, there's other things that go into it. It's like sounds of instruments and stuff, but It's literally just trained on the images of audiograms, which is just, or spectrograms, sorry, which is just so fascinating to me.

5:36And that's how they're able to recreate this. So essentially, right, users describe musical qualities using natural language. They can also even record their own voice to prompt the model to generate some unique outputs. There's a ton of different possibilities here. Essentially, it's enabling music producers and audio engineers, I think, to explore new ideas and find inspiration in entirely new ways. now a lot of people are going to tell you hey don't worry this is just you know helping you get inspiration i'm honestly probably the original refusion team is going to tell you stuff like that um i believe this technology goes to a place where essentially anyone can recreate any song in the style of any person with any voice and maybe even you like record yourself talking or singing um and it can put your own voice right like we we can create ai versions of our voice so we can clone our voice on something like Eleven Labs.

6:25Wouldn't it be interesting if you could say, I want a song about X, Y, and Z. Here's a couple of clips of me singing, throw it in this style, edit it to sound nice, and it makes anyone into a singer. Very, very interesting concept. A lot of people, though, are asking about copyright concerns. Lately, homemade tracks using, you know, generative AI have gone viral, causing a lot of people to worry about potential copyright infringements. I personally have heard a bunch of really funny Johnny Cash songs on YouTube that Johnny Cash did not sing. And so, yeah, people are kind of concerned about that.

6:56Forescreen is kind of quick to clarify that the improved revision isn't trained to recognize famous artists or songs and can't replicate them. Now, the caveat there is he said the improved revision, so obviously, or refusion. So obviously the first version they came out with was doing that, but they've fixed that since. He said, quote, the production isn't built to produce deep fakes and doesn't recognize famous artists names and its prompts instead it lets users craft personal messages and catchy hooks using the app now a lot of people probably are like oh come on i wanted to like you know make stuff just for myself of you know artists and whatnot i think it's essentially they're just avoiding lawsuits at this point um if you if you go look it up i believe there was anthropic that just got served a bunch of lawsuits from a bunch of different music production um studios and whatnot because if you ask Anthropic for the lyrics to songs, it'll give you the lyrics to songs.

7:49And that's all under copyright and licensing agreements. And so essentially they're getting sued for that right now, which I mean, in my opinion, that's kind of silly. Like who cares? You can Google it and just find the content on Google anyways. But yeah, whatever. Yada, yada. A bunch of licensing stuff. It's just the lyrics. But anyways, Anthropic's getting sued. And so I feel like the music industry is pretty sue happy to be honest and uh yeah personally not a fan of that but that's just me um because it kind of makes it like a bummer when like i would love to make a bunch of johnny cash songs just for myself about random stuff um or whatever artist and i know i don't know maybe there's a way to license that to give the artist like some sort of credit for it nah i don't want to pay them so that's just me i'm sure i'll get a lot of criticism for that i would like to I'd like to just be able to do that.

8:37I know it's possible. And honestly, I could probably just go and like get a clip of them singing or talking on YouTube and upload it to something like, um, there's going to be platforms where it says like, upload your own voice to use for the sample. And you just upload a famous person's voice. And I don't know if they'll be able to like detect, like this is, you know, Katy Perry, you can't use this voice or whatever. So it'll be interesting to see where that goes, but I'm sure there's going to be a, that's going to be like one of the big areas where if the music industry really does push back hard on it, I think all they're going to do is force people to just go open source and there's going to be like, quote unquote, bootlegs, like music app, AI music apps, where you can like download them on your computer.

9:17It's just not going to be like a web app. It's just going to be like an open source, download it on your computer. Then you can make deep fakes of anyone's voice. I think there's already a bunch of tech like that. It'll just get more common. In any case, monetization strategies for refusion, I think are still in the works. They're still kind of focusing on expanding and developing new generative AI products. However, there are, I think, some hints of collaborations with a bunch of artists. The Chainsmokers obviously invested, so they were interested in collaborating to essentially explore how this technology could enhance their creative process.

9:47As Forgreen points out, it's very early days for generative music, and Refusion stands out as one of the first to enable users to generate lyrics to their music via a fun and accessible website. very interesting to follow this and to see where it goes in the future. Hopefully, you know, we get a lot of great products and it doesn't get too litigated. And I don't know, it'll be interesting to see where this goes. But I'm very interested in music and AI. I'll keep you updated on it in the future.