943: Creative Machines: AI in Music and Art, with Prof. Maya Ackerman

23:32Jon Krohn:Excellent. And yeah, it looks like there's a free tier, but then even the paid tiers, I mean, they're quite reasonable for a SaaS product ranging from$3 US a month up to$10 a month in the US, depending on which package you choose.

23:51Jon Krohn:There's this gold mode where you have better context detection and advanced metaphors at a slightly higher monthly rate. So that is really cool. And then tell us about Melody Studio as well. It sounded like you started off when you were singing the song that you sang to us earlier about meeting me, that well-known hit, it sounded like you were actually, it sounded like you had the lyrics in mind already. And then, so maybe that's where Melody Studio comes in and it allows you to have melodies generated? Exactly. It's sort of the other two big components of songwriting in addition to lyrics is melody creation and then also chords.

24:32So it gives you ideas for chord progressions and then it gives you different ideas for melodies. And just like Lyric Studio, artists use it differently. Some of them rely on the product really heavily for every single line. Other people would get one little riff, one melody, and then they can go off and write the whole thing depending on where they are in their creative journey.

24:53Jon Krohn:Nice. I like it. And so then with Melody Studio, is it generating, it's generating kind of sheet music like we were talking about that we've had from the 80s with the Bach and the Vivaldi. Now you can have kind of original music in the style that you desire. And can you literally pick basically any style? You can be like, I want jazz. I want this to be rock. I want this to be hip hop. How does that, what kinds of constraints are there around what you can do? It's pretty wide. We cover most common genres. It's very, very flexible. It was fascinating throughout the development of this system to figure out which musical components are most representative of the genre and where genres overlap.

25:38It's just a random fun fact. When you automate something, you get to learn to discover new aspects about it.

25:46Jon Krohn:Really interesting. And have you ever performed music that you have maybe, you know, to a large extent has had the lyrics automatically generated, the melody automatically generated? Have you tested that with audiences or do you have feedback on how people perceive this music? Of course. I mean, first of all, people at Universal Music use our systems. There you go. So there is popular music that uses our stuff. There is a guy named Curtis King who had a number one hit on iTunes, whole hit album where he used Lyric Studio. One of my favorite stories is actually super early on. my good friend James Morgan always wanted to write an Italian aria.

26:30The problem is that he didn't play any musical instruments and doesn't speak Italian. So we built a little system for him, trained on public domain works of Jacomo Puccini. Anyways, after a couple of months, with something that would just spit out tiny little vocal melodies, given Italian lyrics, the guy wrote this amazing piece called Arito Tavrajo. about this World of Warcraft character riding a dragon. It's a beautiful Puccini-like piece. That really blew my mind about how far AI can help people go. And of course, I performed a ton of songs myself that I made with this over the years.

27:09Jon Krohn:Nice, all right, that's cool. So that's amazing. Universal music using it, these hits coming out. I've got to try this myself because before we started recording, I was griping to you about how, it sounds like maybe you were like this as well. I, you know, I've been trained to play classical piano, to play guitar, you know, classical vocal performance as well. And almost entirely, that has been about just regurgitating, you know, compositions that have existed for centuries. and I never was taught how to be creative, how to create my own music. I've done a tiny bit of it, but I feel like it's a real weak point for me musically.

27:55Jon Krohn:And these tools, Lyric Studio, Melody Studio, they sound perfect, especially the way that it sounds like you've calibrated these tools in a way to work where it doesn't feel like I'm being dictated at, but I'm actually working alongside these tools to create lyrics and melodies. Exactly. John, I think there's a future famous composer inside of you. We'll see. We'll see. We'll have lots of great hits about all of your favorite data scientists from history. Have you missed them? Performed by me. Yes. Starting with Jan LeCun. He's going to be our first. I don't know. I'm making noise up. Um, so let's now kind of bridge both your, you know, your, your, your startup as well as your book.

28:45Jon Krohn:So you've previously argued that the sci-fi dream of AI as yeah. I mean, this is, we talked about earlier as an infallible answer machine is, is disempowering and it's misguided. And in, in chapter 10 of your book, you note that disempowerment doesn't scale, but human flourishing does. And so how can investors and companies reframe their perception here? So reframe engagement metrics away from stickiness and towards sustained creative empowerment, kind of like you and David have with Wave AI. Yeah. It's interesting that this addiction model persists even when the biggest problem with Gen AI products is this kind of one hit wonder phenomenon.

29:33When a person feels really like they're really unnecessary, let's say there's like a little app where I upload my photos and then it does something fun with them. I might enjoy it once or twice. But if I feel like I'm not really doing anything, I don't have any control over what comes out, even if it's kind of cool, I'm very unlikely to come back. So we had a whole bunch of one hit wonder sort of Gen AI products. people want to feel that they're doing something people want to express themselves they want to realize their own ideas and industry and investors in particular have been very very very very slow to realize that i believe that the reason chat gpt is successful is because it for those who want to give of themselves, for those who want to really collaborate, it makes that possible.

30:28And that's why it's so successful. So it's sort of like, we need to sort of snap out of these old, outdated, exploitative practices that a lot of us sort of believe in them as gospel, and really explore what's meaningful and appropriate with this technology, which I believe, from my experience in business and otherwise, that the real potential is to really elevate humans in our capacity.

30:56Jon Krohn:Yeah, it's a great idea. And I agree with it wholeheartedly. It does seem like it's going to be hard to convince product managers. You know, do you have any sense? I realize this is a really tricky question. I don't have a good answer to this, but you're a lot more creative than I am. How do we convince, you know, big enterprises, product managers to move away from just stickiness to empowering people. I think we need to explain to them why something like, why the products that are successful, why they are successful. Chachapiti has not been beat yet. It's the number one. It's one of the most successful products in history.

31:37And that's the case, not because you press a button and it replaces you, but because of power users. You always have to look at power users because those are the ones who are... The customers that love you show you where the product needs to go. And those are the users that go deep, that become better as a result of using Chachapiti. I've heard people say that their vocabulary expanded, their writing style got more diverse as a result of using Chachapiti when you use it with a certain kind of intention. And also to show examples of how this sort of replacive paradigm has often failed in a lot of smaller products, but also how it ultimately fails collectively.

32:17This whole like everyone optimizes for themselves really fails when we want to apply AI to replace human workers. Because if we don't have any more human workers, the entire economy collapses. And so it's really time to wake up from these incredibly greedy principles that are guiding our economy and to think more holistically in this moment, not assume that the old principles are just going to keep working and somehow magically everything is going to work out.

32:47Jon Krohn:Yeah, perhaps Gen AI will kind of force this shift that you're hoping for in product managers and in enterprises. You mentioned in your most recent response, this idea of diverse responses. And so in interviews, you have previously contrasted convergent systems that gravitate towards safe, average outputs with divergent co-creative tools, presumably like your Lyric Studio and Melody Studio, that widen the search space. And like you just said, you know, broaden people's vocabularies, you know, increase the diversity of their writing styles. So as AI scales to creating music, to creating video, how do we keep models from nudging creators toward averaged risk averse aesthetics where, you know, you kind of you hear another pop song that sounds the same.

33:44Jon Krohn:time. Yeah. How do we instead get diversity? Yeah. It's really impressive what industry has done to Gen. AI. Gen. AI has always been about expanding possibilities. That's how it was born, going to new spaces. We think of creativity as this massive search of possibilities. The next line of lyrics, there are billions of options. How do we search, right? You're about to play the piano, John, you were talking about playing other people's pieces, like we're all told. Where do we even start? What could be my first note? What could be my second note? It's a search space problem. And machines are phenomenal at just exploring different unlikely possibilities.

34:30And even in trying to take you to places that are pretty good, but not very common. The problem is that science fiction has convinced investors and some entrepreneurs that it's better to create an all-knowing oracle that gives you the most expected, most safe, most you've seen it a million times already answer, which is maybe good for a fact lookup table, but it really limits what Gen AI can do. And it's never actually going to be very good at being an all-knowing oracle, regardless of what we do. And so when you look at ChatGPT and you're like, oh, AI is not creative, that's because ChatGPT is optimized for the opposite.

35:11Inherently, from a technical standpoint, you have to take creativity into account from the beginning in the way that you construct this machine brain.

35:19Jon Krohn:But in the grand scheme of things, in the grand scheme of things, it's not that hard. If we figured it out as a team of three, granted we have David, but still a team of three people originally at Wave AI, then I'm sure somebody like OpenAI, Microsoft and Google can figure it out. It's just not their goal. Their goal is not to open our minds. Their goal is to replace search. And I think that goal needs to be modified. I like that perspective. Yeah, hopefully we can get there. Hopefully we can. Yeah, we can we can iterate towards these divergent, creative, you know, supportive systems that are allowing human creativity to flourish and allow humans to flourish as well more broadly.

36:10let's uh let's talk now a bit about your academic work so you're a professor at santa clara university

36:21Jon Krohn:does you know how how much is the overlap in what you research how much is there an overlap between your research and what we've already been discussing in this episode is your research a lot about creativity i see that a lot of your most uh cited papers for example are about clustering. And so, yeah, so I'd love to hear if, yeah, if, you know, what the overlap is between, you know, your interest in creativity and what you're doing academically. So my PhD work and a little bit afterwards focused on foundations of cluster analysis, which was a much larger research community. And that's why you see more citations.

36:58It's really, really simple. And then I entered the niche world, which my department chair was not pleased with me for switching into gen AI back in 2015, because that's not what I was hired for. And so that's why, you know, it doesn't bubble up to the top quite as quickly. I've done all types of stuff in my academic work. I did a lot of work on bias in these models, and sort of also a bit of a Jungian lens on that. One in particular is called brilliance bias. So brilliance bias is this belief that we all have to some degree, which is that exceptional intellectual brilliance is a male trait. So basically, when people think genius, they think male and they mostly think Einstein.

37:50And we were able to show that generative image models have the same bias. And so what does that mean? We're essentially perpetuating biases. through the AI. We're just mirroring what people already believe and often amplifying it. But it's interesting, you know, we can rush to kind of get mad at the companies. Oh no, how dare you be biased? What were you thinking? But it's also reflecting our world. And I think in that sense, it's very interesting. It's an opportunity to learn about ourselves a little more. Yeah.

38:22Jon Krohn:So I've got, I think I found that paper here. It looks like it was presented at IEEE at the Global Humanitarian Tech Conference, GHTC in 2022. And so I'll have the paper for people to read in full in the show notes. It is kind of amusing to me, a little bit meta that, you know, so you're testing at that time, what were the cutting edge models in 2022, GPT-3 and the specific variant of GPT-3 that was most common and most sophisticated was called da Vinci. And so it's funny that you're studying this brilliance bias in a male trait and the models are literally named after people like Leonardo da Vinci.

39:05You know, the world can't help it because women did not have educational opportunities and whatnot, right? This idea of only men being true geniuses is easy for that to sustain. There is also a conference paper at the International Conference on Computational Creativity about this phenomenon in image models. And we're doing a journal paper right now with my PhD student, Juliana Shehade.

39:30Jon Krohn:Really cool. We'll look out for those and I'll include in the show notes all of these papers for sure. What else is really exciting you academically? What's the terra incognita that you're hoping to traverse into next? Oh my goodness. where do I even start? You touched a little bit on the Jungian stuff, but I didn't properly zoom into that. Yeah, let's do that. Because there are papers here that I'm seeing, recent papers. For example, there's a 2024 paper called The Collective Mind, Exploring Our Shared Unconscious via AI. That's really interesting. And it looks like, if I'm understanding this correctly in this Google Scholar Citation, it looks like it's in a journal called Religion.

40:17Oh, Religion. Okay, we'd have to verify that briefly.

40:22Jon Krohn:I don't know how that happened. It must be, I think it could be a Google Scholar issue. No, because it looks like it's actually conference proceedings. Yeah, it's a conference proceeding, yeah. Wow, that is weird. What a weird thing for a Google Scholar to do. It got creative on us, huh? It did indeed. Can't trust anything. So Carl Jung, I find, would be incredibly inspiring. Really, really deep thinker. I went back to his original writings and I felt like I got to know him a little bit. So he wasn't really the first, but he really expanded on the idea of a collective consciousness. So especially in our society, we like to think of ourselves as independent, just a whole bunch of people with their own unique brains and their own unique ideas.

41:04Now, of course, that's complete nonsense. We tend to share fundamental beliefs about life with each other to an incredibly high degree. And each culture has sort of its own belief, its own belief system, that its members subscribe to it to a very, very high degree. So what we think of as objective reality, as the right way to understand the world, is often just being part of this collective consciousness. So I talked about this a lot and how it impacts us and what that means. It's super fascinating, and I encourage everybody to dig more into Carl Jung's work. But what's really amazing, And I think Carl Jung would have been so excited to live today because it's this collective consciousness come to life.

41:48We took all this Western data and we created a brain out of it. So now if we want to know what Western consciousness thinks, modular the alignment that all these companies are doing, we just talk to the LLM or go to something like MidJourney, which is not as aligned, the text to image model. And it just reveals Western biases like there is no tomorrow. For example, I typed Hanukkah into Mid Journey, and I got like a German Christmas tree. They fixed it since I published it. Kind of revealing that people in the West tend to associate Hanukkah with Christmas. It has nothing to do with it whatsoever.

42:26It's not even a very important Jewish holiday. I asked for Tzuvgan Yot, which is a Hanukkah food, and I got bagels, right? kind of associating Jewish food with bagels too much. It's so beautifully honest about Western beliefs, more so than any person would ever be. So I think there's just so much to study in there beyond just yelling at the companies and we should get the companies to fix it. But in the meantime, we get to understand ourselves better.

42:58Jon Krohn:That is interesting. And I have, it's some years ago now, but I went down a bit of a Carl Jung and grab it whole myself. And he did have a lot of interesting ideas. So I can see how you got so into that. So we've covered in this episode already how you're an accomplished CEO, book author, university professor, AI researcher. I've mentioned how you're a pianist and how there's lots of videos that I've seen of you improvising on LinkedIn. You record improvising at the piano and have published that kind of regularly on LinkedIn. So I've seen that. But our research also dug up that you're an opera singer.

43:35Yeah, that's right. I had a very unusual journey with music. I started playing piano when I was eight years old. And when I was 12 and my family moved to Canada, they were not able to take my piano with them. And so there is this massive 15-year gap in my music education. I got back into music as an adult. When I was 27 years old, I started taking voice lessons. which was actually a callback to my childhood in Israel. I used to perform. I was recorded on national television as a kid, but then there was this massive gap. And so I learned how to sing opera in my late twenties. And I started performing.

44:14My love of creativity is very, very real. I like to make music. I don't care if other things and machines can also make music. I want to sing I want to play piano I make art I like to take classes that challenge me in traditional art forms and music um yeah I miss I miss opera performing I should really do more

44:39Jon Krohn:yeah I was gonna ask if there's any way that we can check you out uh performing in opera it sounds like maybe nothing is lined up at this time but we can but there's a lot of a lot of older stuff on YouTube. Cool. All right. We'll try to dig some of that up for the show notes as well. So Maya, now looking forward, what are the kinds of things that our listeners should maybe be able to expect with the Gen AI systems in the future? So obviously today we've become pretty good at music generation. It's been a couple of years now that we've had high quality image generation, you know, that all hands are getting five fingers and that kind of thing.

45:18Jon Krohn:And video generation has come a long way in recent months as well. Is there anything, do you think you have any, maybe, you know, you've been in this industry for decades in a way, you know, studying Gen AI. So do you think you have any particular insights that we might not be expecting about what's coming next with Gen AI? There's a lot of focus on sort of replacing different industries, different types of workers. I believe that's been the main effort of sort of investment funds in the space, pick a career, try to replace it with AI. But people are also learning that most of the time it works much better when you do have a human involved and they're slowly starting to recognize that.

46:04So with things like images, we have in-painting now where you can kind of highlight a section and regenerate it. Suno also learned that its best users are people who actually want to make music. So they started to make that more interactive. So instead of just sort of more and more and better and better, make these machine brains able to interact with others, right? You can think about really, really brilliant people who don't know how to interact with other people. So that's kind of what we have right now in AI to some degree. Now, what I hope we're going to say, and there is a movement in this direction, is that these brilliant AI brains and all these different industries are going to become more social, more able to collaborate with human beings.

46:51Jon Krohn:Fascinating. I like that. Yeah. So kind of, I mean, I guess taking the kind of approach that you've had with your own Wave AI products, where we're more collaborative and it's less, it's not just a simple providing a prompt and getting a perfect output. It's about iterating. and as you mentioned, being able to highlight maybe parts, very specific components of an image or a video or a piece of music and being able to iterate on those particular elements as opposed to regenerating the whole thing from scratch and kind of wondering where you've gone off to. Yeah, exactly. Isn't it frustrating, right?

47:30It's almost there, but it doesn't understand when you say change the color of the hat. It can't.

47:38Jon Krohn:Yeah, for sure. I actually, I recently, uh, I, I made a mistake in my notes for our production team. Uh, I basically, I, I copied the wrong file from an episode from one episode into another episode. And so we had a, a thumbnail, uh, a YouTube thumbnail generated with the created by our, our thumbnail designer with the wrong title. It was a title of another episode. And I thought, huh, you know what? I think I can use Gen.AI tools. I tried a couple. I tried Gemini. I tried ChatGPT to upload my image and change the text to the correct title without changing anything else in the image.

48:26Jon Krohn:and the one that I happened to use first was Gemini because I'd heard that it was quite good at this kind of thing. And there was a spelling mistake in the generated text. It looked perfect. Everything was perfect. It retained the imagery, but there was a spelling mistake. I had this word accelerated and one of the E's in accelerated was just missing. It was correct in the prompt that I gave it. It was missing and it could not fix it. I went through half a dozen times of saying, it kept saying, here you go, I fixed it. And it just keeps printing out the exact same image with accelerating misspelled in the exact same way.

49:04Jon Krohn:And so that was a really interesting experience. And then I was, I just moved over to chat GPT and happened to get it right, right away. And so, yeah, so it's an interesting, this, I think there's a lot of opportunity in having all of these generative tools work better with us. Still lots, years of progress hopefully to be made. Yeah. It's also really interesting how it struggles with this sort of task, right? Which we consider fairly straightforward. How it still struggles with some basic math, although it used to be even worse, because these are fundamentally imagination engines. And these companies had to put a lot of layers on top of it to try to make it behave, try to get it to go in a right, reasonable direction while it keeps wanting to do its own thing, kind of like a kid.

49:52Jon Krohn:Yeah. So for the math thing, for example, instead of using model weights to approximate numbers based on similar kinds of math, it's memorized from the internet to actually generate some Python code to do the calculation instead or something like that. Yeah. These layers, the layers on top, that's become a big part of the secret sauce of any of the frontier labs. For sure. For sure. Which makes sense. I think it's helpful for you to be able to ask it to add some straightforward text without mistakes. This is all valuable for sure. Yep, yep. Well, Maya, it's been a fascinating episode. Thank you for your rich perspective.

50:27Jon Krohn:I don't think we've had an episode like this on creativity before at all. And I don't think we've ever talked about music so much in an episode either. So thank you very much, Maya. Before I let my guests go, I always ask for a book recommendation. And so that's how I found out about you in the first place was by asking David, your husband, for a book recommendation. Maya, do you have a book recommendation for us? I'd like to recommend the work of Anil Seth, who is a neuroscientist and professor. And he just does a phenomenal job explaining why we hallucinate all the time, developing a little bit of empathy.

51:07It could help us develop a little bit of empathy for the way that machines work as well. And not to imagine that they hallucinate and we don't, but also kind of reframe imagination in a more positive light. There are some wonderful YouTube videos that he did, but in particular, there's also a book called Being You, A New Science of Consciousness.

51:28Jon Krohn:Yes, yes. I found that as you were speaking about it, looks like a great recommendation. It seems like it ties together a number of themes from today's episode as well. So thank you so much, Maya. For people who want to get more of your insights or maybe watch you do outstanding improvised piano from your beautiful view at your office there on LinkedIn. Where should people be following you? LinkedIn is a main platform that I use. You can easily find me, Maya Ackerman on LinkedIn. Yeah, great to connect there. Wonderful. All right, thank you so much for taking the time out of your no doubt very busy schedule.

52:07Jon Krohn:And yeah, maybe we'll have to check in again in a few years and see how the Gen.AI music generation in particular industry is coming along. That sounds great. Thank you so much for having me, John. What an interesting episode. In it, Professor Maya Ackerman covered how we should assess AI creativity by outcomes rather than process, whether machines are involved is beside the point. She talked about how her company Wave AI built Lyric Studio and Melody Studio as co-creative tools that help millions of users write songs. She talked about how large language models function as manifestations of Carl Jung's concept of collective consciousness.

52:49Jon Krohn:And she provided her vision for the future of Gen.ai, wherein machines become more collaborative and better at understanding our creative requests. As always, you can get all the show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Maya's social media profiles, as well as my own at superdatascience.com slash 943. Thanks to everyone on the Super Data Science podcast team, our podcast manager, Sonja Brejevich, media editor, Mario Pombo, partnerships manager, Natalie Zajewski, researcher, Serge Masise, writer, Dr. Zara Karshay, and our founder, Kirill Arimenko.

53:23Jon Krohn:Thanks to all of them for producing another stellar episode for us today. For enabling that super team to create this free podcast for you, we're oh so deeply grateful to our sponsors. You can support this show by checking out our sponsors links, which you can find in the show notes. And if you'd ever like to sponsor an episode yourself, head to johnkrone.com slash podcast to learn how to do that. Otherwise, support us by sharing this episode with folks that are looking for creative inspiration from machines. Review the episode on your favorite podcasting app or on YouTube, wherever you consume the show.

53:58Jon Krohn:Subscribe if you're not a subscriber, but most importantly, just keep on tuning in. I'm so grateful to have you listening and hope I can continue to make episodes you love for years and years to come. Till next time, keep on rocking it out there. And I'm looking forward to enjoying another round of the Super Data Science Podcast with you very soon.