Possible: Artful Intelligence, part 3, featuring Paul Trillo

27:26The records store data such as greetings in 55 languages, music from different cultures like Bach, Beethoven, and Blind Willie Johnson. It had whale songs, bird songs, and mathematical diagrams, including the location of Earth. It's a very sci-fi kind of strange idea to create this message in a bottle intended for aliens. That's what it was. It was designed with these kind of hieroglyphics that were instructions. that if an alien were to ever discover this record, they would be able to play it, which probably, I don't even know if Carl Sagan fully believed that. I don't know, but it was like, just in case, you know, here's what humans are and could potentially be like, you know, whatever in the far, far distant future, it may be the only record we do have that's out in the universe.

28:25Fast forward to adult. Around the time Voyager 1 was leaving our solar system. Five, just six years ago, I started to look into, like, all right, what was that exactly? It felt like a good storytelling device to kind of go off on these tangents and explore different aspects of humanity and history would make for a good anthology. Paul and other innovative filmmakers started working on an anthology project about Carl Sagan's golden record. Beginning with the invention of the wheel and far off into the future into like almost a post-human universe. What does the technology do when man is gone? The golden record almost becomes like a holy grail for like some sort of AI descendants of us searching for like a ground truth.

29:19When Paul got his hands on Sora, he knew it could be a breakthrough for this old idea. So he used it to create a film trailer for the project. It starts out with this meteor that's falling down onto an early Pangaea, carrying gold metals down to Earth for the first time. The gold coalescing over time, being mined and collected. Rocks breaking apart, smashing together. gold chunks breaking off of it, going through this boiling process, melting, converging, until they form this golden record. Sort of magically launches through rock particles and smoke and dust and soars around through the air. And then we kind of cut back to reality to see it resting on the Voyager spacecraft.

30:17the main purpose of creating the trailer was to tinker with this new tool so that he could learn how to push it to the edge of what's possible. And so with that golden record experiment I did, it was to see, can you take these AI video models into a more kinetic, dynamic, cinematic place? And there's this kind of strange, surreal alien quality to it, but also to do something that It felt old and vintage at the same time and kind of mashing together what looks like modern day visual effects, but through the filter of like 16 millimeter film. And that was really what was exciting, too, about A.I.

30:58is to be able to mash up different concepts together, trying to break it, throw as many words at it as possible and see how fast I can make that camera move. So it was it's definitely like a little nauseating to look at, but I think that's a little bit by design. Paul's golden record video was released by OpenAI in a blog post in March of 2024. It became one of the primary ways that people could see what was possible with Sora. In the blog post, they quote Paul saying, Sora is at its most powerful when you're not replicating the old, but bringing to life new and impossible ideas we would have otherwise never had the opportunity to see.

31:39Now that Paul knew how to get the results he wanted from Sora, it was time to start mapping out the washed-out video. The song is called The Hardest Part. Essentially, it means the hardest part is kind of moving on and not being able to go back. The lyrics are very much about kind of mourning a relationship.

32:13That made me think back to this idea. Back to the surprisingly fertile land of the graveyard of ideas.

32:32This one came about a decade earlier, when Paul was experimenting with using photographs to make an infinite zoom effect. Paul and his team scheduled a shoot to see what was possible. Waking up at like 2 a.m., we started shooting around 3.30 a.m. And we shot 41 photos, 41 blocks of New York down Fifth Avenue as sun rose. They used each of those 41 images to create an experimental film where the viewer is continually zooming through the streets of Manhattan. It was kind of mind-bending. It was like a time-lapse and an infinite zoom combined together. You know in Google Maps when you click forward, forward, forward?

33:15I love that. Did they steal that from you is what I was wondering. No, I think I must have been influenced by Google Maps to some degree. Yeah, for sure. But I thought, how can I create more meaning here? How could I give this some story? Paul wanted to use this technique to tell a story about the passage of time, clearly a reoccurring theme in Paul's work. Going back to that idea of like the sort of melancholy of time passing, barreling forward without being able to look back.

33:53Paul kept experimenting with the infinite zoom effect, this time to tell a story. Infinitely zooming through a couple's relationship, beginning with kids and to their 80s. We keep kind of zooming through the back of a bus. You see two people like kind of glancing at each other from different parts of the bus. That's pretty much as far as I got.

34:20Like all my ideas, they start to escalate and get too ambitious. And then I kind of write myself out of being able to pull them off sometimes. And so I kind of just like put that back in the drawer. Back in the present. Paul needed a new idea for the washed out video and remembered that idea in the back of the drawer. Sora might allow him to dust it off and try again. I was like, ah, that's it. that's the idea that I put back in the drawer and I kind of resurrected it from the dead. Paul knows the ins and outs of working with Sora and knows the concept for the video, an infinite zoom through a couple's relationship.

35:01Now it's time to start prompting. But first, Reid, I want to ask you a question about AI solving specific problems versus creating an entire paradigm shift in how we make things. what areas do you see the paradigm already shifting? So typically with technology, it actually starts with reproducing things from previous technology. Like in the earliest days of the internet, classified ads became online classified ads. And sure, they had new features. The new feature was, you could search it or it had a kind of a freshness to the database that you wouldn't have with a newspaper. And I'd say a lot of the things that are happening so far tend to be in that still same.

35:50It's kind of like what it was before with a feature or two versus a complete paradigm shift. You know, like, for example, you could even say like classifieds becoming marketplaces on eBay and so forth. Now, I think we see lots of glimmer of areas where the paradigm shifts, where this enables new patterns of interactivity. Like in games, maybe you have NPCs that are kind of like Turing agents, can be fully linguistic. You see the beginnings of what is scripting a computer mean when you have the AI as the scripting front end and has access to all of the interface of the computer to do stuff. Now, that's been productized at the developer level.

36:35That hasn't been productized at the consumer level yet. And then similarly, in terms of creative work, we tend to have a broader set of people that can create kind of amazing images, much cheaper to create video, but still kind of like I produce something that I could put on YouTube. Right. Kind of is equivalent. And even like, for example, in things like Read AI, we're doing stuff like that. And just for the audience, I highly recommend that they visit your LinkedIn page where they can watch your read AI videos. Essentially, it's a fully AI version of yourself trained on video and audio of you and your writings and podcasts.

37:13Did I get that right? Excellent. Sure, it was generated by ReadGBT for the script and sure it was AI image, AI audio. but it's still like okay it's delivered in a edited video right as a consumption as opposed to hey it's a read ai presence real-time interactive most of the various things so far seem to be of the version of it's cool that you could do that with ai but not oh my god i've never seen that before. Right. So like a friend of mine, Bennett Miller, had a very early access dolly and created an art exhibit in the Gagosian. And yeah, it's Bennett without AI wouldn't have been able to create an art exhibit.

38:01It was cool that the first art exhibit of AI dolly tools, but also it was an art exhibit at the Gagosian. Right. So like the the conception is, oh, it's this image that's on the wall framed this way. I have complete conviction that some paradigm shifting will happen, probably major paradigm shifting but that kind of comes down to a marketplace consumption and is it just like consumption that will create paradigm shifts or how it's productized or what do you what do you think leads to that well it's it's it's a it's a kind of a iterative loop between them so it's kind of like someone's created finally something that's paradigm shifting and then that paradigm shifting thing is like engaged in some way you know these are one of the reasons why it kind of like tends to start in feature, small step, feature, small step.

38:52And then eventually, oh, like sometimes paradigm shift is those five small steps together, together become a paradigm shift. Or it's like, oh, yeah, I recognize this is entirely different.

39:11Meet Nicole Nicholas, Capital One business customer and co-owner of Ansett Uncles, a plant-based restaurant and community space in Brooklyn, New York, that got its start from a need for unity. The inspiration, it was born from the desire to create a space that felt like home, where we can connect community culture, good food, and come together with family and friends. That's how we birthed aunts and uncles. Nicole and her husband, Mike, were fulfilling their dream of bringing people together out of their home kitchen. But they soon learned that the demand for community was greater than they knew.

39:42It became overwhelming and we were like, we need home, but not in our actual home. We realized that there was also a need in our community for something bigger in our neighborhood. So we had to find a place. Moving from a home operation into a storefront was a huge next step. But Nicole and Mike were able to take it on with the help of Capital One Business. It's not for the weak. As a small business, finding resources is super important because that's the way you'll be able to manage and scale. We would have never done that without having Capital One to be able to help us along the way. The cashback rewards are very helpful.

40:18You know, it just gave us that runway to be able to breathe a little bit. Then you get to focus on the cooking of the food and making the experience great. To learn more, go to CapitalOne.com slash business cards.

40:32Part three. To create a paradigm shift, it requires excavating right at the edge of what's possible today.

40:44Paul's objective was clear Create the first ever completely AI-generated music video He knew how to lean into the constraints And knew what he wanted to achieve An organic and emotionally true Infinite zoom of a couple's relationship But creating art doesn't always go according to plan I had like a rough beat sheet Of what I wanted to happen across the video But as I was making it, I was sort of reacting to the results, kind of understanding what the tool does and what it doesn't do. And something that I started to really love is when the machine gets confused. Sora, just like any AI tool, started hallucinating.

41:27A car would morph into a road. A road would morph into a building. To many filmmakers, this would pose a problem, but not to Paul. What is this affording us? What is this giving us that we couldn't shoot with the camera? And let's embrace that. Remember, the Washed Out song is about examining the past and not being able to change it. And when humans remember the past, we tend to hallucinate too. When we close our eyes and try to think back to a time, we really narrow in on very specific details of the moment. But we don't remember what kind of happens on the phrase, on the edges. That's what AI is doing.

42:10It's sort of misremembering history. That's not something you could capture with a camera or even ask an animator to add nonsensical details to the frame. Once Paul discovered that the hallucinations were actually the most interesting part of the video, he decided to lean in hard. I started to embrace those hallucinations and the things that made the tool unique. There became this kind of marriage between conceptually what AI is and the story that we were telling, which is this database of false memories. He found that the best way to allow Sora to hallucinate while staying true to his direction was to write some insanely long prompts.

42:59The prompt itself is one continuous zoom through the scene. It's a tunnel of time following behind a couple walking through a bedroom into another room into an infinite apartment. A one point perspective. The couple are slow dancing in a hallway appearing, showing the straight view of the infinite hallway. There's a lot of repeated words here with sheets of fabric floating in the air. we zoom through the foggy hallway the floor is made of astroturf lit from above the shot is captured on 35 millimeter film fuji film stock anamorphic 24 millimeter lens halation chromatic aberration soft color palette subdued that's like the first paragraph and then there's like nine more paragraphs after that oh my god says a couple is curly redhead chick with eye shadow nose ring and a good looking Mexican guy with slick black hair and shade shower curtains from the 80s fake florals budding leaves then the flowers grow out of the astroturf there's no roof over here the curly redhead woman are in a classic convertible surrounded by floating white bed sheets a massive bonfire and then a huge sheet of fabric surreal hallway filled with cars, distorted memories.

44:19Soon the sheets take over, blah, blah, blah. It kind of keeps going. So it's not like written, like you would write a script ever. You're right. You're definitely writing a bunch of jargon and, and sort of hashtag speak. So it sort of becomes this alchemy of words. It's, it's almost like chemistry where you like put in a word, you get this reaction, you take out the word and you get a different results and sort of experimenting with that right equation of words, but you're still trying to evoke a feeling at the end of the day. After more than 700 video generations and 320 total minutes of footage, Paul saw his vision of an infinite zoom through a couple's relationship realized in a way that only half-remembered memories and AI hallucinations could achieve.

45:10It was time to release the video to the world. But first, Reid, what kind of impossible ideas do you imagine will someday be possible with AI? Well, there's already some stuff that I'm doing. Like I made custom one-off editions of my book Impromptu for people, like, you know, has a chapter for my friend, And Eric Schmidt or Gayle King or whoever, that's like a reimagining of the book. And Impromptu itself, obviously, is the first book on AI co-written with AI to kind of show that that's possible to do. And, you know, I imagine that rapidly no books or other things will be written without use of these tools in various ways.

46:00Not just, you know, writing, but also research assistant, critiquing, et cetera. Now, in terms of impossible ideas, so one of the obvious ones is there's an enormous number of people who otherwise, like myself, who couldn't create an image. But now, because most of us know how to use language, and those of us who are good with language and have visual imagination can prompt it, you know, a little bit like Paul's like, oh, I have this idea from a different kind of music video, and I can do that. And it's one of the reasons why I kind of tend to call it prompt directing versus prompt engineering, because to some degree, what we're all becoming much more is like directors in kind of the creative context of these things.

46:42And so then the kind of the limits of what might be possible that was impossible is, well, all right, so there's a bunch of range of people doing things. And then you begin to go, OK, well, how would I tie AI to crypto or how would I tie AI to a museum exhibit or how would I tie AI to the creation of a virtual space? How would I tie AI? This is part of my doing read AI. I, people have obviously thought about this as things like I could put it up on LinkedIn and it could be my agent proxy for talking to people. But one of the other things that it's good for was part of thinking is like, well, I'll do some talking to myself.

47:20You know, you could imagine taking Paul's music video and you could say, well, what we're going to do is we're going to reprompt and recode a music video based on your genetics. Maybe you get it from 23andMe or other, and you have a genetic screen and you put it in And then it gives you your own endless music video. Like, well, that's not doable before. That's impossible to possible. And that's just the beginning. So the landscape of the collapse of the impossible. Then the question is, of course, which of these impossible things are really interesting to make possible? And that's the work that creators, artists, innovators, technologists, many people with the broader tools of AI can start using their creativity and imagination.

48:09Part 4 When creating a paradigm shift, dealing with trolls not only comes with the territory, but means you might be doing something right.

48:23It was time to release the video to the world. Paul remembered what it was like to see Sora videos for the first time. It was pretty earth-shattering. It was unsettling, honestly. it was not like a great feeling. And when people watched the washed out video, Paul expected many of them to have a similar reaction. But if anyone was going to be the first to dive into these controversial waters, it would be him. I knew someone was going to do this. And so I knew I had to like, I just wanted to be the first one that did it. I was interested to see what would happen if I put myself on the tip of the spear there.

49:03And there could have been other versions of this video that could have been maybe lazier or kind of phoned it in. And the fact is, like, you know, I did spend five weeks on the video trying to hone this and perfect it. And there was still a ton of time and thought put into the video. The release date arrived, and the comments started rolling in. It was a sort of a nuclear bomb in a way on the music video world and production. And I was bracing for that backlash to see people like upset at what the video represents versus what the video actually was. A lot of the comments, yeah, weren't reacting necessarily to the content of the video.

49:53it seemed like a lot of people were just watching you know the first five to ten seconds of the video and not the whole thing and then just kind of retweeting it or commenting on it as they do yeah people saying oh it's like looks like shit is the kind of comment uh the typical anti-ai comment or looks like ai slop or it's soulless which is ironic to be chastising something for originality when they're all parodying the same talking points.

50:29Most of the negative comments and write-ups were not about the content of the song or the video, not about memory or past relationships. The backlash came simply because the video was made with AI. And I get it. The vast majority of AI content on the internet is actual slop. Looking at you, lobster Jesus. So I understand why this would be the gut reaction. But Paul is trying to change that. And it's working. The music video was included in the best of Vimeo staff picks. It also won the Golden Nika Award for AI and art at the Ars Electronica Festival. It ultimately was a positive thing. but you know i just know that this technology is not going away and you know we might as well try to make the best things we can with it

51:31that's really good man i'll say like the first time i watched the video you know obviously i'm making a podcast about ai so i'm into ai stuff but uh first time i watched i was like oh my god looks incredible how'd they do that with ai i wonder what the prompts were i wonder what you You know, blah, blah, blah. The second time I watched the video, I was like purely just transported to the world of like, I sort of forgot it was AI. And this was just like this character. Oh, that's great. But no, I mean, I think that's the highest compliment is for people to forget how something was made. That's really what I want to do with technology is to enable new ideas or explore things that we couldn't explore before.

52:07but to not have that be the only thing that makes it interesting, that at a certain point, the work actually transcends the process.

52:28Okay, Reid. Final question of the series. There's a lot of AI slop on the internet. There are also artists doing incredible work with AI. In each of the stories in the series, we've heard frustration from these incredible artists for often getting thrown in with the slop just because they use AI. And each of them wants to change this paradigm. As an AI creator yourself, what advice would you give to these creators? well you know it's it's a story as old as humanity and art itself that when you start doing something like people start doing photography and that's not painting painting is real art that's just copying and it's terrible and blah blah and it's the typical reaction of the status quo because status quo says no no my thing is much more valuable and important than this new thing this new thing is awful and terrible and it's going to cause the destruction of humanity and society and culture and the respect of the art form, blah, blah, blah, blah.

53:26And by the way, to some degree, that's a necessary, maybe not sufficient condition to know that you're on to something and it's good. And so that gets to, A, you have to do the work and have to persuade some of the audience to engage with it. But B, you also, you know, part of what I think, you know, human expression, art, et cetera, also does, is you have to pitch it. You know, it's like pitching an entrepreneurial thing or pitching a thing else. You have to kind of like, what's the frame? What's the way to think about it? What is the lens by which we get a perception or an experience or a awareness of something about the human condition?

54:02You know, as a mirror on me, as a theory of the world, as me mediating to the world. You know, like one of my favorite historical paintings is, you know, the drawing of a pipe and saying, this is not a pipe. So you're like, ooh, it's kind of causing you to reflect on what it is you're looking at and what it is that's been created and what that process is and a bunch of other things. And that's an instance of what art's about. And so for the creator, you have to both do the work and you have to pitch it. You can't do the classic entrepreneurial failure of build it and they will come. It's like, I just built the thing and everyone should go, you know, should have a hallelujah.

54:39It's like, what? And it's like, that's not the way it works. doesn't work that way for products, doesn't work that way for companies, and doesn't work that way for art. Now, how you pitch it and so forth is kind of relevant, and who you pitch it to is relevant. And, you know, as people begin to see it, then they begin to go, ooh, that's new and different. And, you know, more or less, one, there'll be tons of critics. There'll be tons of critics about AI. They'll say, it's just AI. Screw it. I've heard a bunch of this stuff myself. That's fine. That's usual. and then what you're trying to do is you're trying to find you're not necessarily trying to overwhelm the critics you're trying to find the people go oh this is really interesting i have a growing audience or interest in this you know there will be a ton of ai slop it's very very easy to do so you have to have a theory of how you're changing that and then that's also part of your pitch you know it's like when i give people custom copies of impromptu that has, you know, their own chapter in it.

55:38I don't go, and this is the next, this is what all of art means for the next thing. Cause I'm like, oh, that seems to be some interesting pictures and some interesting facts and it's personalized. I don't know if I would go to, it should be in, you know, MoMA, right. As an example. So you pitch it and you go to market with it, you know, a la classic entrepreneurs. And then have some resilience because for sure, one of the necessary conditions of doing something interesting is you will have some people going, this is crap, terrible. You know, same thing when you write a book. If you don't have some people who are like, this is totally wrong, then you're not really striking something that's really interesting.

56:23Love it. All right, Reid, any last words about AI, the future, humanity? Do you have any final words to leave us with?

56:38Well, I mean, I think the short answer is everyone should be trying it. They should be trying it for things that are just kind of fun, you know, sonnet for your kid for the birthday, whatever. They should be trying it for things that are like work. So it's like, how do you make your work better? And just about everybody should be able to do something to make their work better with it. And then, as per the series, try to use it as a way of expressing your creativity. Like expressing humanity in different ways and use that to learn. And it doesn't have to be that you become the Picasso of prompting.

57:19because there will be some of those people. But the fact is, is it gives you an ability to express your humanity and to be creative. And that opportunity to be creative is, I think, a super important part of how humanity runs. And that would be my closing, for the miniseries, my closing comment. Awesome. Reed, thank you so much. This has been unbelievable. So thank you. Chris, always a pleasure. Yep. You too, Reed.

58:09Artful Intelligence is produced by me, Chris McLeod. Massive thanks to Reid Hoffman. Special thanks to Paul Trillo, Aria Finger, Sean Young, Saida Sapieva, Ben Relis, Ian Alice, Parth Patil, and the team at Wonder Media Network. Mixing and Mastering by John Delore. This has been a Blue Elevator production.