Adobe Acquires Invoke.ai + Launches Custom Model Service

0:28All right, welcome back to Denoised. Joey and I text each other all day long. Hey, did you see this? Did you see that? And then over the last few days, I just haven't been getting any texts from Joey. Joey, we got an episode shoot. Let's go. And, you know, I've done the same. I get into projects and then Joey's texting me. So there's been some big announcements. So Adobe with two prong announcement. One is they have this foundry where you can build custom models. We're going to get into that. Second, Adobe acquired an AI startup called Invoke.ai. We'll get into what the implications of that is.

1:04Runway now has full-weight training for their models and what that really means and what's the strategy behind it. So we're going to speculate on that. And then finally, this was Tech Week in LA last week. And Joey and I attended one of the events together. Joey went to another really interesting event on Monday that we're going to get into. So yeah, here we come.

1:29All right, let's start. You want to talk about Adobe's Foundry service? Yeah, so let's take a step back. This is 2023. We're all kind of scratching our head as we are looking at MidJourney and Firefly, thinking, could this be the future of a lot of the things that we do in computer graphics today, or manually today? three years ago generative AI was certainly a promise but it would certainly didn't sort of unfold into the ecosystem that we have today and I think two years from now we'll be saying the same thing it'll just further proliferate so Firefly was one of the earliest models to kind of hit the scene and the reason it was a big deal is because it was commercially safe Adobe trained on licensed data only.

2:19And this was around the time when, you know, frankly, you had stable diffusion and mid-journey, which trained on publicly available data, which loosely stands for anything you can find on the internet. Scrape it, scrape it, the vacuum cleaner. So it's, yeah, exactly. The problem with, now there's pros and cons to both. We're just going to put the ethical stuff aside. I'm going to just kind of pin that right here because that could be a whole conversation and a whole episode. So, of course, ethically, it's not cool to grab other people's data. A lot of the AI companies will argue that if it's publicly available, then it's for us to take.

2:58And so there's like a murky line there. The pros is when you have a ton of data, and I'm talking about think of every images off of Google image search, like millions and millions and billions and billions of images. is every little article that Google ever searched for, right? And I keep going back to Google because a lot of AI companies in the past may have used Google Search to extract a lot of the data. And what that entails is then you get into a model that has a really good understanding of world context, right? It could differentiate between a ginger ale and a Coca-Cola, right? Like it could be very granular, very good at real-world sort of problems.

3:42versus one that is just trained on licensed data. Now, if you think about just licensed data, think of Shutterstock, think of any stock photo website, you're talking maybe a million images, perhaps 10 million images at tops. So I think when you don't train on enough images, you don't have a good understanding of the world. And so even though you're ethically safe, commercially safe, your model is just not as good. These are the things that it's a trade-off, you know. And then if you have a model that trained on publicly available data and it's not commercially safe, then how do you monetize it?

4:20How do you make money from it? Because no true company will want to use that to make movies or whatnot with. So this was the Adobe Firefly backdrop that I wanted to present. And now in today's world, Adobe is, and I'm purely speculating here, I don't work for Adobe. I don't actually, yeah, I don't have any friends on the Firefly team. so I don't have any insider knowledge here. But I'm speculating that Firefly models are just not good enough when it comes to quality and fidelity. A lot of times when big companies, like Coca-Cola or Pepsi, will use AI for a commercial, for a printed ad, or what have you, you won't even know that it's AI-generated because the quality bar is just set that high.

5:03And on top of that, manual human intervention, like Photoshopping goes into it to make that AI generation as believable and lifelike as possible. The problem is Firefly's custom model training, like LoRa training, and all of the stuff that Firefly still offers, it still wasn't getting you to that quality bar. So in the past couple of months, and I think we covered this on one of our previous episodes, is Adobe opened up their ecosystem to any models, right? So now I think Google image models, ChachiPT image models could be found on the - I'm now way more familiar with this because I've been knee deep in Firefly and Firefly boards.

5:42They added a lot of models, but not every model. So like the ByteDance models are not there. Kling is not there. Wand is not there. But the big ones that Luma, Runway, ChachiPT, Google, they're all there. They're image models and they're video models. But yeah, we did talk about this. this was sure sounds like some of the like the open weight models the open source models are not there no obviously yeah not one 2.2 and not um yeah not uh c dream and c dance which are not open weight as we clarified in a previous episode they are by dances models yes which are rather good but they're not there by dance be dance c dream i will say also having messed around a bit more with firefly and using firefly probably a year ago and it came out and be like what is this and then trying it now with the firefly 4 it is way better it's not on par yeah it is way better i mean it's probably better especially if you're using it in the applications that they built it into like photoshop with the harmonized feature where it kind of blends the objects together and matches the lighting and then or like in painting and stuff like that uh it's a lot better than when i messed with it i don't know a year ago um but still i would just keep going back to banana banana just all the time i i don't blame you for that i mean just nano banana is just on such a league of its own right and then you have the open source models in in this small other league and then you have the commercially safe firefly thing here um they're all you know i think in a couple years they're all going to play on an even field but kudos to adobe for kind of course correcting and bringing that product into the folds of what modern performance, modern quality should be.

7:26Yeah, they turn Firefly into a platform and not just the model. So it's like Firefly is their AI platform to do whatever you want with whatever models they connect to, not just their models. Right. So now that they've brought on third-party models into their Firefly ecosystem and they have their own model, what do you do with it? How do you actually monetize it? How do big brands use it? And all of the stuff that I'm talking about, really, honestly, it doesn't have anything to do with film and TV. Because the requirements for our industry, even down to image format, right? Like you're talking 16-bit EXRs and TIFFs and Rec.

8:022020 color space. Our requirements are so high up here that AI models in general just can't sort of compete in this space. So all of the use cases that I'm talking about is primarily like e-commerce, product photography, some type of social media campaign, vertical ads and things like that. And there's a huge sort of market for that. Like AI is already being used for a lot of that now. And so Adobe wants to kind of recapture that market because they had it initially. And then Firefly wasn't good enough. So a lot of people migrated away from Firefly. And now they're trying to recapture it with third-party models and a solid ecosystem.

8:43Yeah. So, okay. So the big news with this is, like, they launched a service where they can train their own custom model for your company, for enterprise clients. This actually, I guess, I don't know, this was, like, an announcement or something. But I remember at the TIFF panel, Hannah Elsacker, who's the vice president of Gen.AI, she was on a TIFF panel that was run by AI on the lot. And Moon Valley was there as well. And I remember she mentioned, like, we can train custom models for you. It's not cheap, but we train custom models for customers. So hearing this, I guess that was like, if you had a lot of money, they've been doing this for a while, but now it's announced as an actual service if you have a lot of money.

9:21Yeah, exactly. So I think in the past, how they dialed in the custom models was it went into a bunch of AI engineers' desk, and then they manually fine-tuned and weight-trained the model Firefly in that instance. So they were finding that maybe that cost was too high on a use case by use case basis. So what they're doing now is just turning this entire thing into an automated platform. So I haven't used it. My guess is you upload a set of your images. It builds a LoRa or it does a weight training. And then you can then do inference based on your quote unquote custom model. Did you see something that made this sound self-serve?

10:01Because it just sounds like it's a product called Foundry, but it's for enterprise only, which means mega money. like it still sounds like it's a hit it's a purely speculating yeah like this still sounds like a very the hands on hands involved like service where they're gonna you know be very involved in training a model for you and also did you know anything did you see anything else about that they are bringing other models into this training or is it because it also seems like it's being still being built off firefly i am also speculating on that i'm just connecting the dot from their previous announcement and the next article that I'm going to get to.

10:37The acquisition. So I'm just like extrapolating a clear line. Yeah, exactly. All right. Cool. We'll go with that. So invoke.ai is an AI startup. We covered it on the podcast in the past. It's a node-based workflow that is very model agnostic. In the same way, we talk about FreePick, right? You go on FreePick, you can use Kling, you can use Vuo, you can use Nano Banana. and it's a seamless experience. Like you could just pick your models and go do your thing. Invoke takes it a step further and is more technical. You can start to build node-based arrangements, sequential workflows. So if you have, for example, image references that go into a inference node, those image references can then be trained and generated themselves.

11:27And then they could sort of kind of sequentially fall into a nice little node diagram. So Invoke's strength is not just their spectacular node workflow. The demos that I've seen is really clean. It's also the fact that they support all the models the same way FreePick does. And so if Adobe acquires Invoke, it automatically gives them a leg up on providing support through all of the third-party models that Invoke already serves. Well, it's not an if. The news is Adobe did acquire Invoke. And their front page now splashed with Invoke is joining Adobe. All right. Yeah. So I Googled it this morning, and I was like, Adobe acquires Invoke.

12:10And Google Gemini came back with, that is inaccurate. This is a false rumor. And I was like, really? Let me just go directly to the source. So I had to go into it. Their website. Yeah. So it's official. Invoke is acquired by Adobe. This is a big shift in Adobe's direction. And I think the course correction here is something that is definitely going to benefit them if they pull it off correctly. There's a lot to be said about integrating a completely new company and their technology into a big, giant ecosystem that Firefly already is. I mean, yes. And we've had this debate before of, you know, do we ever see Dolby going down the more technical product rabbit hole of like a node based system like a Nuke or like a Fusion or Comfy UI?

13:00And, you know, so just like, do you feel like they're just going to merge the invoke product into Firefly? Is it more of an acquihire to just bring the AI talent on board? Like, what do you think about this? I mean, the strength of Invoke is that it's all web browser based. So imagine running Comfy on a browser, I guess, Comfy Cloud type thing. And it's way less technically jarring than Comfy. Comfy tends to be, even for the two of us who use it time to time, like even for us, we're learning new nodes. And we're like, wait, that node doesn't connect to this. Wait, there's a dependency. Anyway, Invoke is way cleaner.

13:37It's for the average creative who is just technical enough to get away with using Lightroom, Photoshop, After Effects, and all the Adobe products. I think that level of proficiency will be enough for them to then jump into Invoke. I see a really nice integration path into all of the web-based stuff they have already. Adobe Express is a good example, right? That's a native web browser-based platform they already have. And so Firefly, the board thing that you're going to do a live stream, that's another one. Yeah, there have been times on the board where it's like, man, you know, just connecting a couple of dots right here would be like a lot easier then.

14:15Yeah, that's it. Yeah, so this is that missing link. And I think it's a really smart acquisition. We have just yet to see how it'll come into fruition. Yeah, for sure. But yeah, I mean, excited to see where this goes. And yeah, even on Envoke's website that says they'll be shutting down their existing online service at the end of this month. So like next week. So if you are a current user, that does kind of suck because you're gonna have to find something else until the product relaunches again and then maybe bounce back to it. Yeah, if you're using Invoke or Firefly, hit us in the comments. Let us know if we're accurate here, how much off we are.

14:47We'd love to hear from you. Yeah, I would say I've not used Invoke. I mean, the other thing too is you could run it, oh, you could host it locally too, right? Was that a thing? That was another, was that a unique feature? Like you could download the code? I'm not sure. Yeah, not sure. a creative engine for locally hosted generative media models media models so like i also wonder if that type of stuff will stick around or if that's that's uh adios if you're an adobe competitor listening to this let's say you're a google or whoever good news is there is a couple of invoked competitors that are still out there and i'm sure they'll be happy to be acquired by large companies.

15:25So the other two that come to mind is Weavey, W-E-A-V-Y, and LTX Studio. So those two, I used to put Invoke, Weavey, and LTX into the same kind of category as really friendly user experience that is node-based and is powerful enough to get a lot of extra creative juice. LTX is also node-based. Oh, I'm sorry. Or are you thinking of Flora Fauna? Flora. Flora. Flora. Flora. Yeah, Flora Fauna. Oh, it is Flora Fauna. I thought you were making a joke, but their website. The product's called Flora, but their website's called FloraFun. I think Flora.ai is taken, just like vpland.com is taken, man. It's a race.

16:03It's the dot-com race again. So I'm confusing LTX with FreePick. I think LTX and FreePick compete one-to-one. Right. And I remember LTX has some of their own models. And I remember when they first launched, I think they were trying to be kind of more of a media and entertainment, like take a script, turn it into a storyboard, beat sheet kind of thing. and I think they've evolved into more of just like, we'll be a platform with a couple of their own models that they made. Exactly. The AI ecosystem is so rich and so full of these little companies. You know, I tell you this all the time, I totally missed the dot-com era of like small startups in San Jose and all the little like 20, 30 people companies.

16:42I was too young then and I didn't live in the area, but I feel like I'm living through this now, like right through the mix. But too old to be like, I'm going to sleep on a couch and eat pizza and run a startup. Yeah. Like, dude, I'm not sleeping on a single couch ever again. I'll go get a hotel. I need my eight hours. I'll see you later. Exactly. I can't podcast like this. Yeah. So, yeah, there's some good updates from Adobe. And I'm curious to see where Invoke and a node-based editor goes. because, yeah, being using Firefly boards a lot this past week, I was like, I want to connect some stuff.

17:21It'd be nice. Yeah. And I'm curious how much of this new stuff will be included with a Creative Cloud subscription because that's when I'll jump into it. Some credits, yeah, but it's like, it is a separate subscription, which I've also discovered and found out. But there's like, there's Adobe Creative Cloud and I think they give you some amount of credits, but then there's a separate, whatever they call it, Firefly AI or something subscription. That's like another add-on that gives you additional credits, but it's another thing you've got to pay for. Yeah, it's like an add-on to the Creative Cloud subscription.

17:54You don't have to have a Creative Cloud to do that. If you're just like, I like Firefly and Firefly Boards and I just want to do the AI stuff, you can, I think, just get a subscription to the AI stuff separate from Creative Cloud. But yeah, it's a different system. I will at one point, my goal is to do a mapped diagram because the other thing with like, with, with Firefly boards, with FreePick, with like all of these AI platforms is like they have a credit system and then the credit system converts into how that company figures out what like running a VO3 generation costs versus a Ray 3 generation.

18:34One of my side projects is to just build out a board to like price compare like but you know converting all these uh credits into dollars to be like okay who's actually the most cost effective using their platform it's a side project get an ai agent to do that yeah i still i i have tried that and they're not always accurate so it needs a human to actually verify that the information is not bs well if you ever figure it out it'll be a giant poster behind me yeah we'll put it up we'll be have a ranking of who has the the most cost-effective credit system, the best bang for your buck. Yeah. Yeah, we'll figure it out.

19:10Yeah. So another episode, another runway announcement. So on the last episode, we covered runway's presets, which I think is just a fancy name for Laura training that's sort of automated. I think it's overthinking. I think it's literally just some nice prompts under the hood that are just giving you what you're looking for. Yeah, yeah, sure. I think you're right. Yeah, it's just maybe like a frictionless experience to get to the thing you need to get to. It is what it is. It's presets. It's what it is. So this is something that's slightly more different. There is a big, massive, labor-intensive way to take a big foundational model like an image model.

19:56So if you take Midjourney's image model, and if it has enough hooks into it, then you can actually make it output something completely different that it wasn't trained on. So, for example, if MidJourney just trained on cartoon data, right? Like, it's just really good at outputting 2D animation, cartoons, and stuff. You could actually take a million images of, maybe not a million, maybe 100 ,000 images of live-action footage. It's called full-weight training. You can full-weight train the model, and then now it'll output live-action imagery. But the good news is you don't have to build the model from scratch.

20:37So full-weight training is maybe a day worth of work, whereas building a foundational model, you know, it could take months. It could take millions of GPUs and so on. So it's still a really clever way to kind of squeeze more lemon juice out of the lemon, if you will. Just get more out of this foundation model and have it pivot and do something it completely wasn't intended to do. So a lot of proprietary closed models don't exactly have this feature versus open source models like WAN 2.1, Kling, Stable Diffusion. These guys are quote unquote open weight. So then you can actually take their full model and then retrain it at your own whim.

21:20Runway is introducing something called model fine tuning. although we historically have thought of runway as a closed model because it's only available on their API and it's sort of behind closed doors, they're introducing hooks into it where you can actually do full-way training. And the reason for that is now you can apply runway's video generation, which Joey and I go back and forth on this. We can argue that it is film and TV oriented, right? Like all of the demos that runway has shown is adjacent to our world. But what if it is really just good at doing robotics training and replicating the real world?

21:59What if it's just really good at doing architecture, archviz? So you can take the runway foundational model, give it enough imagery of architecture and buildings and interior spaces, and then now it'll just output that in its own. Yeah. I mean, again, this doesn't sound like any self-service type of product. if this is another enterprise money, money, expensive service. It sounds like a service version of what the runway Lionsgate deal was or what they were hoping to do, where they were like, we'll train the model on the Lionsgate or fine tune a model on the Lionsgate media. This sounds like that thing as a service to other enterprise customers.

22:40That doesn't have to be M &E. It's interesting because in their headline is, you know, it's robotics, education, life sciences, and beyond. So anything where you're like, I need to roll. I absolutely agree with you. A world model as a basis to modify, to train, to do for some applications that I'm trying to do. Yeah, this is when I say squeeze more lemon juice out of the lemon, this is exactly that. So you spent millions of dollars training this model and you have this amazing model. And the industry that you're catering to is just not making the money for you, right? In this case, most likely film and TV, media and entertainment.

23:13I don't think we're going to ever see another runway Lionsgate partnership. like there's no money in there and it was cool while it lasted they high-fived each other but nothing really came out of it but you have this amazing model so what do you do with it you can totally pivot and turn into other industries uh accommodate where anywhere generative ai is utilized and you go the nvidia route you go the google route you go into industrial you go into robotics you go into autonomous driving you go into architecture those are major lanes where there's a lot of money on the table. So this is a smart move on Runway's part, and I think they're diversifying to increase revenue here.

23:53Yeah, I think it's a smart move. Like you said, they already have a world model, like why just restrict it to one industry. And as we have joked, all roads from world models lead to robots. They're all just trying to build a world model so the robots understand what the world is like. So the Unitree robots, the most popular robots that you see on the Internet, They're now$16 ,000 and I believe you can buy them online. Are you buying one? Look, you guys want to donate some money to us, humble podcasters, we'll have a third host. Is this in your Amazon shopping cart right now? You're just waiting to click buy?

24:31Is it on Prime? Can you get two-day, one-day delivery? I'm going to wait. I'm going to wait until one of my neighbors buy it and then just kind of get some feedback. You just need to buy one so you have it with your lawnmower out. side and so it sees your neighbors and then like waves or something you know what i i don't know what i'll make the robot do i mean cleaning the bathroom that's like top of the list yeah what did i just see something of uh that they're they were paying people to send videos of um them folding laundry to like train robots to fold laundry yeah so like for so to do that you need a video model that the robot can train on and that video model just has to output pictures of clothes and folding laundry all day long for hundreds and thousands of hours yeah you could just strap a gopro to like for that runway would be perfect if it fully trained on folding laundry yeah yeah good good use of that yeah and then you could take that fine fine tune and then sell it on a marketplace so if anybody has a unit tree robot they buy that fine tune for you know 100 bucks and now their robot can instantly do laundry folding.

25:41That reminded me of, now I don't remember the company name, but this is also probably a good segue into LA Tech Week. But yeah, I met a company where their service was basically sourcing those types of video clips. It's like someone's trying to train a model on something. They need very specific training data that they have a gap in, like robots folding, like someone folding laundry or someone, like whatever, setting up a microphone. And they'll source those videos for that company that's trying to train the model on whatever specific need they have. It's sort of like a stock footage library, but for, like, AI trading.

26:19Yes, exactly. Well, you know, we're so familiar with, like, Unreal Engine Marketplace. Like, you can grab anything. You can build any world. You can add any animation. We're going to see the same thing with physical AI. eye like uh you're gonna see a robotics uh secondhand market like used cars kind of thing you're gonna see um modified arms and modified legs and extra like it's just like the camera gear ecosystem right like i can go to tilta and get something that red doesn't make and so on yeah yeah all right so yeah sticking on that tangent and merging into la tech week so last week was la tech week there were a lot of events going on um i only went to two addy and i only went to one together but they were really good so the first one i went to was uh last monday it was um all right this was kind of weird because there was a amazon event aws at stage 15 their awesome virtual production stage uh but the it was called the culver cup aws event it was weird because last year was the first culver cup and it was actually a film short film competition uh i was one of the finalists so that was my little film right there i remember that yeah and so it made sense it was called the clover cup uh they basically scrapped that entire thing but kept the name of the event so it's called the clover cup again but there was like absolutely no there was no clover cup trophy there was no competition there was no competition it was just two panels that aside the panels were interesting yeah so the main one was a panel with the house of david creator uh john irwin uh has a david show on amazon prime produced by with wonder project and yeah it's a biblical retelling of the biblical story uh through a series sort of you know game of thrones bible-ish version limit i don't know what the budget is but limited budget and season one they did admit you know not admit but they did explain that they use gen ai for like a full sequence this season they upped the game and used gen ai for 253 shots in some fashion that's a lot of shots yeah and they did basically it was the opening battle scene one of the big examples was an opening battle scene of i forgot which battle there's like one of the opening battles and one of the episodes in season two and it was just a really good example of a lot of the stuff that we've talked about put into practice so it was a combo of uh some shots were just completely fully Gen AI.

28:43Some shots, they had a small LED wall set up. Actually, I believe Vue set the stage up where they're filming out in Europe. I don't remember where, but they set the wall up and generated backgrounds and then shot Gen AI backgrounds with live actors. And then just a couple other hybrid stuff with real life shots augmented with some or composited with some other ai elements like i think they were able to do some ai horses uh and some other effects and so it's just like all of those elements thrown in together edited together where like all these shots created different in different fashions but cut together it looked pretty seamless and they played the clip and it's one of those things where it's like knowing beforehand some stuff's ai generated then you're like oh yeah i could see it but if you didn't have the preface of that and you just watched it it would just look like a regular battle scene no it was not like oh that looks it weird no it's just a regular i have so many questions joey i'll try to answer them as best as i can from the panel like which which technologies were used which models who were the artists and yeah they didn't get that nitty-gritty which ai models were used i mean i'm gonna guess because it was a aws amazon hosted event that it's going to be any of the models that are on Amazon Bedrock, their AI platform.

30:03That's pretty much all of them. Yeah, more or less every model exists on AWS. The reason I was asking is not for image quality per se. Clearly the quality was good enough and they probably massaged it a lot after the generation. I do have the mention. Erwin mentioned Midway, Runway, Kling, Magnific, and Topaz for upresing. Oh, that's okay. I'm getting to my second question, which you already have answered here. My second question was going to be, what about the ethical and legal hurdles? And it looks like I called every lawyer at Amazon and just bludgeoned them until they said yes, said John Irwin on getting approval for AI-generated shots.