Real-time AI-powered design with Krea CEO Victor Perez | E1850

0:00We're getting very close to going from somebody's mind and doodling to a finished product. Is that what's happening here? That's 100 % what we are trying to make. I mean, we were trying to make that happen. And so filmmakers are playing with it. Other folks are playing with it. People have had their minds blown by this. You've got hundreds of people paying 30 bucks a month for this, dozens. Where are you at as a company? So we have a waiting list of more than 200 ,000 people right now. Wow. This Week in Startups is brought to you by Squarespace. Turn your idea into a new website. Go to squarespace.com slash twist for a free trial.

0:39When you're ready to launch, use offer code twist to save 10 % off your first purchase of a website or domain. Masterworks is the first company allowing investors exposure into the blue chip artwork asset class. Twist listeners can skip the waitlist by going to masterworks.com slash twist. And FitBod. Tired of doing the same workouts at the gym? FitBod will build you personalized workouts that help you progress with every set. Get 25 % off your subscription or try out the app for free when you sign up now at fitbod.me slash twist. All right, everybody. Welcome back to This Week in Startups. There's a ton of AI-powered creative tools that are coming to market.

1:24You may have seen Canva, OpenAI, Adobe, MidJourney, obviously Stable Diffusion, Runway, so many different products and tools to help creatives make more interesting output faster and better and even allow maybe people who aren't that creative to get creative. All these platforms run in a similar way. You enter a prompt and then the model goes to work. You wait a couple of seconds, sometimes a little bit longer, and then it spits out an image or maybe a collection of images. And then you refine those images further with more prompts and thus restarting the cycle. It's not ideal. And it doesn't actually work that well unless you have a lot of creative skill and you can refine those images.

2:06It's very rare that the images come out fully baked and ready to go. But a new startup called Crea.ai, that's with a K, K-R-E-A.ai, is working on a real-time AI-powered creative tool set that works in the browser. It went viral on X this week. So we decided to have the CEO on. X is the website formerly known as Twitter. Victor Perez is here today, and he's going to show us how it works. Victor, I don't think you've done any interviews about CREA yet. So this is This Week in Startups Exclusive, I think. Welcome to the show. Yeah, thank you so much, Jason. Super happy to be here. So let's get right to it.

2:41Show me what you've built and why people are losing their minds over it. Let's go. What we have here, this is a tool, by the way, this was built in one week, right? So there was this new technology that got released. we were here at right now where I'm recording in a party where we decided to make like a fun interaction with this new technology so everybody that was in front of the webcam would be turned into something else in real time that's how this tool started and after the day of the party we realized like holy shit this is actually very very interesting should we build an interactive tool on top of this and in a few days we already hacked the whole thing together and very like we divided the team some of us were working on the infrastructure on making a um an infrastructure that can scale because this is kind of crazy to have thousands of people being generating images in real time and some others we were working on the design all right so we're in a web browser essentially it looks like yeah and on the left hand side there is a pink circle that is layered on top of a blue rectangle.

3:48And on the right, we see what looks like a mushroom with a pink frog on it that mimics the very simple circle and square. So explain to us what the prompt is here and how this all works. So the prompt here, it juts blue mushroom in top of a pink frog. Yeah, like that's exactly like what the AI is doing is getting this initial image that we have on the left side and it's turning it into something that looks very much realistic on the right. This is nothing new, right? Like we've been able to do this with models like stable diffusion before. This technique is called image to image. So this is nothing new.

4:23What is new is that if I start moving this pink circle, you will see that the frog starts moving around in real time, right? So this is right now giving me full control. It's giving me a whole new dimension for prompting, which is like right now I can prompt this AI model visually. And so what I could do right now is, for example, let's say that I want a, I don't know, instead of a pink frog, I want a blue bird on top of this blue mushroom. So what I could do is get the same shape, change the color, turn it into blue. And what we should be able to see is a blue bird on top of a blue mushroom that if I start, and that's the thing, like only through prompting.

5:09Right now with disabled diffusion, I would have gotten something like this that doesn't look at all like a blue mushroom in top of a bluebird. Actually, that's kind of a hard one. Let's do a bluebird on top of a blue mushroom. That would be an easier one. So if I change the color here, like as you see what I'm doing - So you're changing the color palette and then AI is automatically reacting to that. So by using the color palette, it's sending a prompt, I guess, saying, hey, use this color to the AI in real time. Exactly. Yeah. It's just like, you can think of it as a visual co-pilot, where we are used to co-piloting in text, that it kind of autocompletes what you want.

5:55Here, the input image is on the left side, and it's kind of autocompleting it with this final result that looks very realistic. And so what's the back end here? What are you using? What's the language model or the image model? The image model is a version of a stable diffusion that is distilled with a new technology that is called consistency. And with these models, we're able to run stable diffusion. Yeah, we're able to run a single image in 40 milliseconds, which is insane. So it will do a single image in 40 milliseconds. And so it's almost to the point where it is when you drag and drop this and move it around.

6:30So if you were to move the bluebird to the other side of the mushroom, it kind of does it, you know, whatever, five frames a second, four frames a second, something. Yeah, that's mainly because of all the network delays. And so how much compute power is this using or have the stable diffusion models gotten so good that they don't require as much compute? You can run this model really on like 3090 or like some GPU that you could have on your computer right now to make it work at these speeds we are using A100s. Got it. And so if you have an A100, which costs, I think 20 grand still, something in that range, how many people could be doing this in real time on one of those units right now i think that we should be able to support from four to ten people so in other words you need about uh let's call it three thousand dollars in compute to be doing this in real time if you were to divide the the gpu by the number of people using it so that's what desktop computers cost today it's not that big of a deal where is this going what else can it do oh yeah i think that we should think of this demo as the worst it will ever be this is the the dumbest like the ai will ever be this is about to get way more efficient and it's like the quality is about to get way better so we can expect that in the who knows when because like breakthroughs are very hard to to predict but i can i can think that in a few months we'll be able to see these demos running on your own computer maybe on your m1 m2 i wouldn't be surprised if that happens and we're kind of like setting up everything towards that and apple just launched the m3 and so right their chipset is obviously getting powerful enough to do this and maybe next year you'll be able to do this on a on a desktop computer or a laptop you think yeah yeah like i don't really know when it's gonna happen but uh it's 100 gonna happen yeah if your landing page looks terrible i'm out we all know that you see and ugly website, you skedaddle.

8:31You leave. You're done. So you need to stop selling for okay or good and start using Squarespace so you can be excellent and extraordinary. It's an out-of-the-box business solution to build beautiful websites, engage your audience, and sell anything you want. You know Squarespace's amazing features, gorgeous templates that are always optimized for mobile, drag-and-drop web design with their Fluid Engine, advanced analytics, marketing analysis, sales data, and more. And with Squarespace, you can create an online store or start a blog at the click of a button, create a subscription business for members only content and so much more.

9:05And you can do this all simultaneously. It's the simplest, most effective and best looking way to start a business online. So here's your call to action squarespace.com slash twist for a free trial. And when you're ready to launch, go to squarespace.com slash twist for 10 % off your first purchase of a website or domain. You guys are in a hacker house there. I see like three or four people writing code behind you where the heck are you is this your startup here yeah yeah yeah we all live and work in here are you in the bay area where are you yeah yeah we're in san francisco this is more like the the kreia house right like we have founders founding engineer uh is also living here in a room there are like some friends in the city that we just like set up in here so they can sleep how long do you guys how late you guys stay up coding what's the this few this few weeks has been a madness with all this growth and like trying to scale up all the gpus and onboarding users we stayed here until very late normally diego my co-founder stays here until 3 4 a.m normally i normally go to sleep at one so people are grinding it out you guys are super motivated and so is this a company now that you've built have you raised money for it and then who are the customers that you're trying to get on board and what are they using it for totally yeah this is a company we raised our seat round last year and the customers are mainly creatives we are right now focusing in kind of a consumer product so the spectrum of people that get interested in this is pretty uh wide like we have from professional film directors that they are creating shots for a mood board that they want to show to their art directors for example we have graphic designers that are using this tool to make uh letters like kind of typographies or to make all sorts of of illustrations we have 3d artists also making like kind of brainstorming or making like very low or yeah like images that they can show to their clients to confirm if they can proceed to go and to the to a professional tool and actually like spend time uh doing it uh high resolution and you charge 30 bucks a month for this tool which allows you and people don't know this but this is how like um ridley scott works ridley scott the film director aliens blade runner gladiator etc he makes these ridley scott like tiles basically little drawings on the set and then they work with the cinematographers and everybody make it happen i'll share my screen here because i have a couple of the demos that were shared on twitter yeah so here is a demo somebody did just drawing the moon and we see the trees behind it i'm not i can't see the prompt there it's a little bit too small but they put in some prompt obviously to make a spooky nighttime foggy place i'm assuming but they start drawing and when they're drawing you know very rough sketches of waves or whatever it just makes this incredible uh evocative image and so this feels like we're getting very close to going from somebody's mind and doodling to a finished product.

12:06Is that what's happening here? That's 100 % what we are trying to make. I mean, we were trying to make that happen, like getting a great interaction, getting a great communication with AI so you can use it end to end from pre-idea until a product that it's 4K resolution, extremely detailed, and that you can use as a final result for whatever client you have. Yeah. And so filmmakers are playing with it. Other folks are playing with it. People have had their minds blown by this. You've got hundreds of people paying 30 bucks a month for this, dozens. Where are you at as a company? So we have a waiting list of more than 200 ,000 people right now.

12:42Wow. We have people paying, but right now that's actually an issue because we are still working on scaling this up. We are doing it gradually. We started rolling out invites during these past days and we are doing it very slow while we monitor how all the GPU cluster handles all the requests. And yeah, we had issues because people were paying thinking that they would get access right away. So only yesterday we got like more than$7 ,000 or$8 ,000. of people paying. And yeah, so all that people will get access, of course, like either today or tomorrow. But yeah, like we're still not letting them pay because we want to monitor all the cluster and we want to make sure that everybody can have a good experience.

13:21Yeah, and so I could see this working for graphic designers, people building web pages, people doing illustrative work. And now you could have, you know, if you think about just journalism, Victor, in journalism, there was an art department that would do illustrations. Now you can have the journalists themselves say, hey, I'm doing this, you know, editorial about, I don't know, US-China relations. I want to do an image of the US as an American eagle and China as a dragon and, you know, just start riffing and, you know, show the Pacific Ocean, whatever, put Taiwan in the middle and, you know, I'm just riffing here like a journalist might.

14:00And it could make an incredible image that would normally take an illustrator, you know, a couple of days and cost what would probably be low thousands of dollars. And it could just be done by the journalist in 10 minutes. It seems like you're pretty close to that, huh? Yep. Yep. We are almost there. And for people who spend a little bit of time on learning how to use these tools, I would say that we are already there. Like people are already making things that they can publish in like a blog post or like in the news, et cetera, where you can already get very nice quality. Do you have any news outlets using this for illustrations or cartoons or that kind of stuff?

14:33Because I was just thinking like, the Dilbert guy, you know, he does everything digitally, Scott Adams. But now he could just talk to prompts, and you could make a Dilbert AI. And with ChatGPT4, it could be making jokes too, or at least giving you ideas for jokes. You could make a verticalized version of this, and everybody could make Dilbert paneled cartoons for their organization or to do marketing. I mean, I guess if you allowed it and licensed it, It would be crazy. Yeah, yeah, totally. Like right now we've seen it in a couple of places. I just talked with some guys that they were working on a project.

15:06It was not exactly journalism, but they were doing like kind of a documentary of people that were tortured by the police in Barcelona, which it was kind of a, yeah, a little bit crazy. And they were telling me how this tool, it was pretty interesting because they want images, right? Of these people that were tortured. They wanted them to create the exact places where that happened. I think that the main difference of what we are doing is that you have full control, like way more control over the final image. Like you can really decide this object needs to be here. This person needs to be there.

15:39It needs to be in this position. And you can make this with very, very simple doodles. Then refine it with the prompt and the AI will 100 understand what you mean. And you will be able to, yeah, to get it like with way higher quality and with a much better composition. This is a classic documentary technique. if you can't get you know to recreate what happened they'll do illustrations uh moving illustrations and there was a really great film the kid stays in the picture about robert evans i don't know have you ever seen it kid stays no i haven't you should watch it just an incredible documentary but in this documentary um they will show you know moments in time but they take pictures and they kind of layer animation behind them and and kind of bring things back to life that you know they don't have documentary footage of and so this is a classic technique but you it's expensive to create animation right you can even if you're outsourcing it to china or korea has a big animation uh outsourcing business here you could just make incredible illustrations yourself and and yeah fill in the blanks for people when they're trying to get a visual and that would typically be on a documentary budget that could be like half the budget a third of the budget could be making those, maybe a third of the budget could be making those illustrations.

16:56If you're making a three or$4 million documentary, you might spend a half million dollars, a million dollars making that art and that art direction. Now, who knows? I think it could be done for close to zero. Yeah, totally. And this same idea is going to be applied everywhere from product photography to advertisement, gaming. I think that we will start seeing AI everywhere because it just makes sense. You're going to be able to use this technology end to end and get the same, if not better results than the ones that you're getting right now okay this is doing static images is there an ability to do moving images yet or loop videos yeah so we've seen people recording their screen while they while they um move sure the shapes which was very very interesting uh so we've seen like people doing these kinds of kind of even psychedelic animations because you see like all these things changing all the time but there is one model that got open source a few weeks ago It's called AnimateDiff.

17:52And we are already thinking on ways how we could apply this technique to this video model. And once that happens, we should be able to make a pipeline that works from video to video. And you're able to get extremely consistent frames out of your like very simple shapes. So yeah, that should be possible very soon. So how did you get into AI? I know you went to Cornell University. I know you published a paper on animation, but how did you get into this? and how long have you been doing it? So everything started around 2017, so six years ago. And my story is before going to university, I was very interested in artistic things, especially on music.

18:32I see that you have a guitar in there. I've been playing since I was eight years old and I had a music band through all my teenage years. And I was just like, yeah, recording music, making photos, making like all sorts of creative things, painting graffiti, all of this. At some point, I decided to study computer science in university. And in my third year, I got introduced to image processing. And with image processing, I started to learn a little bit more about neural networks and artificial intelligence for image processing. Very shortly, I discovered that you can not just process images and detect objects, but you can also generate.

19:05So I got introduced to things like DCGAN or like StyleGAN, that these are these early generative AI models for generating images, very realistically. And at that moment, something clicked in my head and I went crazy to understand everything about how they work. I started to read a lot of papers, make a lot of implementations and use this technology creatively. And I guess that what I saw at that moment, it was a new kind of creative medium. Like if I wanted to record a, I don't know, like a hip hop beat, I needed to learn how to use this program. If I wanted to do like a 3D shape for a video, I had to learn how to use Cinema 4D.

19:46And I was in this loop over and over and over. And I think that with AI, what I saw is something that can execute your ideas for you, which I think that is the most important thing on any artistic or, yeah, on the artistic process. I think that the really important thing is the idea that is behind the process and the execution is just something that is in between and that enables you to get these ideas out there. So I think that since I started messing around with AI, I really saw these. And that's what I've been. StyleGAN was done inside of NVIDIA, I think. And it was NVIDIA. I remember there was a famous Uber engineer who did this person does not exist, which was a website that would make a photo of a person that doesn't exist.

20:31And that kind of blew people's minds because then all of a sudden this concept of stock photography, where you have stock photography models do things like, I don't know, pour a bowl of cereal, but you have to find a person and bring them to a studio and have them, you know, spend a day doing inane things like pouring a bowl of cereal or, you know, eating cereal from a bowl. Now the whole concept of stock photos is just, you don't have to do that. You can just make one and style GAN is still going. Yeah. Or is it stable diffusion has just leapfrogged it so fast? Yeah. I think GANs are not a thing anymore.

21:04I think that stable diffusion shots, it was better. Like the main difference is that it's able to do everything. Whereas GANs are only good at doing a single thing. Like you can get GANs making faces or making cars, for example, but they will only be able to do that. And there's some new research that tries to fix that, but it's not working too well. So Stable Diffusion is an open source product. There's a company called Stability AI. That is the company by some of the people who worked on the project or created the project, and they've raised billions of dollars and they're going for the gold there.

21:34But anybody can fork Stable Diffusion and do what they want with it, correct? Yes, totally. Yes. It's open source. Yeah. So there are a ton of startups coming out of that space. How many people are working on that project? And explain to the audience who are unfamiliar with open source what the activity is within Stable Diffusion and then how code gets committed and what that's like right now. Because this is a very unique moment in time where there's a gold rush, there's so much creativity, there's so many ideas. What's happening to the open source project of Stable Diffusion? because I don't know if they've ever had a project that's had so many people who want to participate.

22:11You tell me. Yeah, I think that what is crazy about stable diffusion is that they are making accessible this thing that costs so many millions to create, right? Like Stability AI goes ahead and shares the weights of this model that they spend millions of dollars on GPUs in training. And now people can use it freely to do whatever they want. Once that happens, what you see is like a crazy amount of improvements like a year and a half i guess it was august last year is when it got the first version got released and i remember like at that moment the best thing that you could try it was dally and we were all mind blown by dally and when stable diffusion appeared it was not really that good and it just took the open source community a few weeks to get this model to a point where the results were even better than dally and now of course it is like so much better than at least the previous version of Tally.

23:02So what I think that is interesting about open source is like it shows you the power of people, of the collaborative work for making something work. Like people really, it's so crazy how before, like all the breakthroughs that came in the AI space, they were all coming from research institutes or they were all coming from companies like NVIDIA. And now it's crazy how a lot of the techniques that we are using in Korea to make the quality of the images better are coming from random people that who knows where they are. They are, sometimes they just learn how to code because of this technology, but they are so passionate, they're so driven to make this thing work that they go ahead and make it and they make it public to the community so everybody has access.

23:46So this compounding of, yeah, like this community effort, it's what it is. How many people are contributing meaningfully to the code base at this point, do you think? Well, there are many code bases in the end, like you have on the one side hugging face that I don't really know how many. Tens of thousands of people probably. Tens of thousands. Tens of thousands. Yeah. So explain to folks who are not familiar with what weights are and why who understands and knows the weights is important. Because we do hear about open AI, which some people refer to as closed AI now, like who has the weights for their models is important.

24:21Microsoft has access to them. Nobody else does is, I think, the public positioning here. So why is it so important that the weights are shared? And then what are weights? How can people think about those if they don't understand that concept? Sure. So in the end, these AI systems that are so successful, like stable diffusion, they are at their core neural networks. And the way how neural networks work is they get an input, like an image. They pass this image to a set of weights or to a set of neurons. Like it's the same idea, right? We call weights what, because in the end they are numbers from, they are like float numbers that they weigh the features that need to be generated, for example, or comprehended.

25:04And then you get the final result. So these weights in the end is like when you train these AI systems, what you do is you, for example, in the case of image recognition, you have an image as an input, you pass it through all these weights and these weights give you an output. and if there's a car in the image and the weights end up deciding that there's, I don't know, a dog in the image, you will be able to update all these weights so the next time they don't predict a dog and they are able to effectively predict a car. But essentially, this is everything that you're doing when you're training these kinds of models.

25:40You're just like making sure that you find that combination of weights that have knowledge and that can understand the input that you have and that they can give you the right output. So when someone release the weights, you can use all this knowledge that it was extracted with the neural network and you can generate images or generate text, do text understanding, et cetera. Listen, public markets can be volatile. Don't I know it? And if you're looking for a unique asset class to diversify with, let me tell you about blue chip art. Blue chip art has historically been uncorrelated with the stock market.

26:17And Bloomberg reported that as equities dipped in 2022, blue chip art had its best year on record. Last year, the big three auction houses posted record high revenues of a combined$17.7 billion. But here's the problem. Blue chip art has always been an exclusive asset class until Masterworks. With Masterworks, anyone can invest in fine art without needing millions of dollars. This is because Masterworks securitizes blue chip pieces and sells the shares to investors. And Masterworks provides liquidity. To date, Masterworks has sold over$45 million worth of art. And net proceeds have been paid out to everyday investors, not billionaires.

26:58Masterworks has more than 840 ,000 users and north of$800 million in assets under management, AUM. And Twist listeners get special access to skip the waitlist. Just go to masterworks.com slash twist. That's masterworks.com slash twist to skip the waitlist. Past performance doesn't guarantee future results. See important disclosures at masterworks.com slash CD. So one of the big debates has been on copyrighted images or just the training data. What is the state of the training data inside of Stable Diffusion today? It was trained on what? And then how is it being trained as it grows? And then how do copyrights, if at all, play into that?

27:42So Stable Diffusion was trained with this data set called Lion 5P. This is a dataset that comes from pairs of images and image caption that it was scraped from the internet. And it's like these kind of crazy scrapers that they go all through the internet and they get these pairs. So there's definitely copyrighted images in the datasets that it was used to train stable diffusion. And right now the situation is, well, there are like several cases going on with mid-journey, stability AI, etc. there's still not a verdict of where this is going to go. But yeah, my sense is that the really important thing is how you use this technology, not how you train it.

28:21And it's equivalent with what happened a few years ago with Google Images, that they were trying to sue Google because of showing the copyrighted images when you search for something on Google Images. And in the end, you need to answer the question of, is this a transformative technology or is this a derivative one? If it's derivative, like you are making a copyright infringement, you're making money in the same way that the creator of that asset is making money and you cannot do that. But if you realize that you can do this, that without this technology, it's impossible from a text generating an image.

28:56In this case, you could say there is something transformative. It was not possible before. So it's not making any copyright infringement and it should be allowed to exist. So we'll see how that case works out. In the case of the stock images, if i had a stock image library and it was used to train it i would say well i should be able to train my own model and i should have that opportunity to then make stock images based on the library of stock images that i spent decades acquiring there's some sympathy to that stock image library holder yes in the community or is the community just like hey we can do it it's too late we can't unpack this what is what do people in the community think yeah i think that the power that this technology gained out of being trained of all this data is superior than like all these stock photography companies that they had all these images and copyright.

29:47For me, the really important thing is only people actually spending a lot of time on creating certain things. Like for example, imagine that you are an artist like Beeple, the famous, like we all know Beeple. Right now, the AI got so good thanks to having seen so many images of Beeple and that right now everybody is just able to say this thing about in the style of people or this other thing in the style of people and they are able to make money just out of that i don't think that that's something positive but i think that there's something very very special on being able to say okay do this image in the style of people that mix it with the style of i don't know like michelangelo and suddenly you have something new that you were not able to have before and that it can be 100 consider something artistic and something new for me it's like way more important to protect like the copyright of artists than to protect like yeah so if you you if you evoked people or you use people in the training set it could just say hey you know you have to use the people version of this just like you know there might be other versions uh made eventually star wars characters etc if you want to make star wars characters and you want to evoke star wars and darth vader you just pay the license and so the industry's got to figure that out at some point and i think it's going to be a negotiation because what a mess like what would happen if it had to be retrained is that even possible or you'd be starting over if it if it had to take out all the beaples if it had to take out getty images or whatever it was scraped on with that lion uh five what would happen would it just set the whole thing back no that's kind of what uh firefly is doing for example like adobe it's training uh their model using adobe stock which is everything it's non-copyrighted images and you know like it's gonna have its use case like if you just need like a regular image of a woman drinking a coffee next to the beach, like very basic stock image.

31:37For that, it's going to give you a great result. But if you are an artist and you want to give it certain styles and you want to play with things that come from copyrighted images, you will realize that this is a very, very boring model and that most artists are not happy with it. Fascinating. Well, listen, continued success on this is super exciting. I want to let the dev house get back to work. I can see everybody's out there having a great time. This is the spirit that built Silicon Valley and, you know, amazing for America. But you're from Spain, yeah? Right, yeah, I'm from Barcelona. You're from Barcelona.