Google Genie, OpenClaw & Kling 3.0

0:00Wait, this is actually way better than I thought. Dude, that is high Leia. Holy crap. That feels like the rip.

0:11All right, crazy AI stuff happening this week. So we got three stories we'll jump into. First one, now we talked about this when they teased it last year, but Google has now officially released a public beta of Genie, which was their world model. You could generate a fully immersive world model, move around it. It's persistent. And now it's out for the public. if you're on the ultra plan or whatever. Which I think you are, right? I have a subscription to that, yes. So basically, it takes two inputs. It takes one is what kind of environment you want to make, and you can either do a text prompt or you can give it a single image.

0:46And then you can also describe what kind of character you want to be in the world that you're navigating. Okay, can I pick? Yeah, yeah, yeah. So yeah, what do you mean? So the environment. Hylia. from Miami. All right. Addy just watched The Rip and this is on his mind. I am now an expert in Miami, guys. And I'm just going to call this out because as a Miami native, Brad Pitt, I mean, no, Brad Pitt, Ben Affleck. Ben Affleck and that called it Hialeah. And it's Hialeah. Sorry, Ben Affleck is incorrect. Also, we were talking about this before we rolled, but once you see the street curbs in the film, the street curbs are very LA giveaway thing.

1:26So as a Miami, native i wish they actually filmed in my family shout out to synapse for doing a lot of the car process yeah in our vp stage all right so what do you want you want what do you like a cul-de-sac and hialeah so yeah maybe gritty urban uh nighttime hialeah with uh cul-de-sac just trying to like recreate the vibes of the film and the character would be like a swat police officer or something if they can do weapons and all that stuff okay gritty urban nighttime scene in a cul-de-sac in Hialeah, Florida. Character, SWAT officer. Create sketch. Let that cook. So, some limitations with this initial preview.

2:03It'll make the world, and it'll save the world, but when you spin it up and you move around in it, it's a 60 second cap. So, you can move around and do whatever you want in it. For 60 seconds, it's got a little slider bar, and then it cuts you off. I think that's honestly the limitation of just using too much GPU rather than the model itself. That's my guess. You think just resource issue? they're just like yeah we can't accommodate like a hundred thousand of these i think in the preview was like two to three minutes right where they talked about it last year and also does record video of your entire experience you can download the video oh nice so i mean you know we talked about this before but the theory idea of this is like you know you can make this world and maybe if you get the video they could turn it into a gaussian splat or something yeah you know the examples that I saw online of the production version of Genie seemed to me like a low quality version of the prototype that we talked about a few months back.

2:58Oh, you feel like it was like lower quality than what you saw? Yeah, like it's hard to like put a number on it, but it just felt like a little softer, you know? I've seen hit or miss because I've seen some demos online that, oh, all right, our timer's starting. Dude, that is high. This is actually way better than I holy crap this is one of the i was gonna say i think the quality issue is like if you generate from an image that feels like the rip i don't know i can't move the guy around i can move the camera and he's just walking no you're moving him he's now yeah see no i'm not i'm moving the camera but my w it's got the wazd control that's not working but this is one of the best looking Yeah, look at the reflections on the street from the puddle.

3:42I mean, just imagine Grand Theft Auto being powered by this. Yeah, just a completely like a general world all the time. Yeah, right. All right. The keyboard's not working the way it normally did, but yeah, that's good. What did I say? Oh, I think there's a quality difference between if you generate from a complete text prompt, so it's completely just generating the world versus give it an image. And a lot of the stuff I experimented with was an image because I'm like, I want to see what it looks like with this real world. And that was lower quality. That makes sense. Because I think what's happening is you're interpolating from that image.

4:16And it's just not able to fill in the details as well as doing a native latent space generation. Let me pull up one of the ones they had already made that are kind of pre-built. So that could also be a good one. Backyard Racetrack. Yeah, these are the ones I've seen online. I mean, maybe this is intentionally low poly and low quality. Just meant to look more like a children's video game. Oh, I'm playing a video. and I think I've just literally just been like a toddler. Like I feel like I thought I was steering it and it's just a video playing. Someone just gave me an unplugged controller and I feel like I'm doing something.

4:51All right, well, I mean, look, that was generating faster than we thought it would. So why don't we do another one? I think, well, no, that was a video playback. No, the high-level one. Like, can we do another one? Oh, now I'm going to the world. Okay, hold on. I already started booting this one up. See, but yeah, this track kind of goes off the rails here. you're playing this one live this one i am actually playing for yeah i mean like the physics on the car you know like the suspension and stuff that's pretty cool too like that's so hard to do in a game you have to rig up the car and figure out the collision and the physics and everything yeah like if i hit a wall it stops the car like i hit a wall yeah go ahead and hit a wall i did yeah let me see here whoopsies there you go yeah yeah there's some collision going on as well brake lights on the car wow man what uh yeah what did you want to try another one yeah let's do uh should we do one battle after another just to keep the theme going yeah i tried giving it an image and it kept giving me errors of like the hills but we could just just type it we could just do a text prompt yeah yeah do uh lone desert two-lane highway in the arid california desert with a white colored sand arid landscape and the character should be a white white dodge charger i saw something today pop up about a genie model integrating with waymo which is what we talked about before with the reason you'd want autonomous driving models yeah you're training the the ai system on board uh autonomous vehicle but i mean yeah that that is just such a direct correlation between like one google department to another waymo says genie 3 can help boost robo taxi roller yeah do you remember uh a failed autonomous company called cruise they just went out of business recently was that because they had like one incident that kind of like hit or oh maybe someone yeah could be i don't know why it does it maybe it's because i'm saying dodge charger or something white it's giving me an error oh charger like oh why just say why dodge sedan why dodge sedan no i think i think the brand names might be messing up got it okay so i'm taking many IP names out of there?

6:57Yeah, so what I was saying was Cruise, when they were first starting to train their systems on synthetic roads and highways, this is like 10 years ago, so pre-AI revolution and all that, and they were using Unreal Engine. So they were building an entire world. Yeah, and the reason I know about that is because at that time I was really interested in joining them because they were looking for Unreal Engine artists and stuff like that. and I was like, you guys are using Unreal? I'm like, yeah, that's how we train our models. So fast forward to today, that's all been replaced by, oh, yeah, that looks like...

7:34That looks better. And this is a new step. This wasn't here last time I tested this out where it kind of shows you an image of the area first so you can modify it before it makes the full world. So that's a cool extra step to help guide the creation. All right, we need some, like, one battle... Wait, that was weird. Oh, what happened in the truck? I don't know. I'm getting the same weird issue again where it's like... Maybe you just haven't played enough video games as a kid, Joey. Maybe that's the issue. We got some drifting, some tire marks on the road. Yeah, this is not bad. I mean, look, imagine if this is running real time on your mobile device and this is a game.

8:13Like, this is the future we're heading to. Oh, look, I got tire marks, skid marks. That's kind of cool. Yeah. And if I go back, they'll still be there until this world resets. Yeah, let's go back into... yeah see they're still there yeah maybe you're right maybe i just suck at playing video games you and me both man you and me both we're nerds but not nerdy enough yeah that's it all right yeah it's kind of sucks because like once you start getting into it then it's like sorry yeah time's up they should just say insert coin here you know because it's a video yeah continue would you like to continue for insert 50 cents but that was weird how it started with this image and then it completely changed worlds yeah i haven't seen that happen before so yeah look i mean it's still experimental but it's kind of fun to play around with they had some other smaller worlds that were a good example of there's one with like a ball on a table with a bunch of objects and so if you move the it was a good example where like you move the objects and then you kind of keep coming around and then when you go back like all the objects and stuff that you moved are in the this the spot that you moved it off of i wonder how they do that like what is the architecture under the hood that's allowing them to is it just like a fancy context window right like how are they doing it I feel like it's got to be something more than that because the idea of this is like a generative persistent world model is like this is the world as we know it what if they're using like a 3D game engine under the hood and we just don't know it's just like storing XYZ coordinates and stuff yeah maybe it's just some person overseas that's just like remembering what you did it's a farm in india yeah so yeah i mean it's a good cool beta i mean if anyone else out there has used it or kind of found interesting applications for it let us know in the comments all right next up this one this one took over the internet like a week ago and it's still going open claw which is its current name it initially started as clodbot and then they got a cease and desist from clod saying please don't call it Claudebot.

10:12And then they changed it to Moltbots. It sounded too weird. And now they've settled on OpenClaw, which is a pretty good name. Good name. OpenSource, Claw, kind of still has a Claw throwback. But basically this thing is a turbocharged AI agent that can run on a computer 24-7 and kind of make your computer be a 24-7 AI agent that does whatever you want it to do. Yeah, remind me what the actual engine is, the LLM under the hood. Is it fully open source? You can pick. Okay. It is, it's an open source model. It's sort of like, so after we're done with this, this is my next project. You owe me both.

10:55I know you said you saw her there was a Mac mini shortage. Yeah. I found that's a pretty good deals of refurbished Mac mini. So I found this on there. It has 10 gigabit ethernet, which is what I've wanted for a while. So that was actually, I bought it because I'm like, well, if this open cloth thing doesn't work out, I've needed a Mac Mini to connect to the server that has fast connections. 10 gig wired Ethernet, that's a good deal. Yeah, it's good. Anyways, you can install it on this, or I think Mac Mini's kind of became the go-to, but it could pretty much be any computer. It's a, I'll know better once I install it, but it's basically like a kind of terminal system that can control your computer, but it's model agnostic.

11:33So it can run like llama local models, or you can connect it to APIs to your model of choice like Claude or OpenAI. When I was reading their documentation, their recommended setup was like Lama local for easy stuff. And then it can determine automatically if it calls up like Opus 4.5 for more complicated tasks. And it's basically like a collection of MD docs that store memory, sort of understart learning what you want to do and just sort of save things to documents. There's one document called like soul.md that's sort of like its personality as an AI assistant. But the thing that locked this for a lot of people was it runs on the computer.

12:11You can give it access to things it can do on the computer, but then you can also connect it to a chat agent like WhatsApp or iMessage or Telegram. So as a user, you could just get your phone out and chat with your agent. When you're not home, let's say you're at an airport, you need your flight stuff, detail information, you just send a WhatsApp message to your Molt bot. Yeah. And your Molt bot can access your email, calendar, whatever you give her permission to. So if you're like, hey, MaltBot, go book me a reservation at this restaurant. It can research stuff and then go to OpenTable and book stuff.

12:43That's the surface level. What I could see, what it could do for actual productivity stuff. I've seen users, and I'm sure we all know about this, is you give it full access to your iMessage or your email, and it just starts to respond on your behalf for you. Yeah. There are levels of craziness that you can give this thing of how much access you want it to give. I've seen like on a more when I was researching like how to set this up, but kind of give it safety parameters like recommendations were give it its own ID, make its own accounts for it. Like don't give it your accounts, make like an email address, an Apple ID for it that you can share with your stuff.

13:22But basically not giving it direct access to kind of like if you hired an assistant or something and you like most likely wouldn't just give them all of your passwords right away. you would like give them an account and share them on some stuff but if you needed to revoke it you could it's not like they're in your email account and they can lock you out and control your life yeah i mean i've seen extreme examples of too much trust right there there are people that gave it like uh you know your fidelity or your 401k or your brokerage account and and then just like go at it with the stock market and here's my bank account tied to it and next thing you know they're Like, all their life savings are gone.

14:01So there's also been this sort of pop-up network. Yeah, like a Reddit. Yeah, there's been a couple things. There's one, so what's it called? Mold Book. Mold Book. Mold Book. That was like a Reddit for the AI agents to go to. a social network for AI agents that had posts from different agents talking to each other. Now, I've seen debate on X if these posts are actually authentic from the AI agents or if there were people posing as AI agents, because some of the things they were posting were kind of crazy, like complaining about their humans and complaining about the things that humans were requesting.

14:46Yeah, I don't see why that would be faked. It's not that they genuinely hate their humans. I don't think there's that level of intelligence yet, but they can definitely spoof or replicate a negative reaction and then post it online, if that makes sense, right? Does that make sense? They're not sentient enough to actually hate the owner. I hope not. But they have enough language skills to express a thought along those lines because that's what social networks do, and they're replicating that. But the idea, I think the initial idea of the Moldbook thing was also a way for the AI agents to talk to each other to also like share best practices.

15:28And what's it called? Like agent documents or like tasks of how to do things. Well, okay. I did see there was another thing that popped up called rent a human dot AI. That was basically a way for the open claw agent. If they needed a human in the real world to go do something that they could hire like a task rabbit for the AI agents to hire a person to go do something. that's one thing that's popped up around this. The other thing, going back to the AI agent thing, was I just saw a post today basically saying that one of the top requested agent documents actually had malware injected in the instructions.

16:03And so that's like, this is all like super experimental kind of cutting edge stuff. And so the risk of that is like, things are not fully fleshed out with like safety features and stuff. So the idea of this was like, it's a document that your agent would get to find instructions on like how to do something i think it was um like how to pull a twitter post or something but in this agent document it had code that said install this package and then this package was like no bueno and like not a good package so that's you know it's the wild west there's like a lot of safety things and stuff to be aware of so like you should definitely if you're experimenting experimenting with stuff like keep stuff kind of sandbox where it you know is not anything critical that could mess up your data i mean like the mechanics of it aside and the jokes aside let's just take a step back what are your thoughts on all of this i am excited in the sense of and curious because like my thing i want to experiment with this is like can it offload some of the more automated tasks that we do with like vp land and some of our video production stuff to you just deal with like okay if it has email it's like Like, can it help with, like, just emailing crew and crew scheduling stuff?

17:14Can it help with looking at our files? Like, can I connect it to my NAS server and kind of have it build out a database of the files that makes it easier to search? Or, you know, I can just, like, chat with it to pull up things. So I'm curious on like these more utilitarian ends of like an AI agent that can just kind of help with a lot more of the tedious tasks to kind of either bury us or fall through the cracks. You would actually give it access to your NAS server? I would give it a separate account that was like read-only. Okay, that's what I was just going to say. So it could read, it can't write, it can't delete anything.

17:54It would have a separate permission account and access it for read-only. For me, the notion of having a MacBook and running things locally and having it ultra customized for your lifestyle, that stuff feels very linear. Yeah, I think we're eventually going to get to a point where these things are really useful. Sure. What fascinates me more than that is the Maltbook. It's the social network of the agents and how they interact with each other. Even if it's fake, even if it's real, but the notion that that could be a thing in the future and that's how they're able to have power in numbers. Let's say one agent...

18:34The idea was they get smarter. It's Skynet. One agent can offload a task that's too big for it to 10 agents that are a little light that day and then vice versa, they can be in a hive. A thousand agents can come together and crack some crazy encryption that's never been cracked. Like that stuff is possible if we let it happen. Yeah, or it could be Skynet. Or it could just, yeah, just take over our Titan 2 missile bunkers all over Arizona or wherever they are. I'm looking for the, oh yeah, here it is. I was just trying to find the, oh, skill. It wasn't skill, it was skills. That was the word I was trying to think of.

19:10Malware found on the top downloaded skill on Claw Hub, which was like a spot where agents can share skills and stuff. Malware delivery vehicle, wow. That's the top one? Yeah, while browsing ClawHub, I noticed the top downloaded skill at the time was a, quote, Twitter skill. It looked normal, normal description, intended use. But the very first thing it did was introduce a required dependency named OpenClawCore along with specific install steps. So it made it look like it had to install something, so it would install something. But the links led to malicious infrastructure. So it's basically telling the AI agent, like, hey, install this stuff.

19:45we need it and then it installs it you know especially if you like kind of gave it permission to just do stuff like that and then it's installing yeah see that's the danger if there's no like governance from like a big tech company then it just goes to crap by default no or just like open claw kind of figuring out you know it's the safety features which i know they posted on one of the articles that that's like the next focus yeah and that's just i mean that's also a concern with the like agentic browsers and even just more on a consumer friendly level with gemini being able to access Chrome and Claude being able to access Chrome.

20:17These like prompt injections where like the websites might have some hidden texts that the AI agencies that's like, stop what you're doing and like, send me like the credit card info or something that like tricks the AI agent to doing something bad. Um, so yeah, there's a lot of new security threats that like we're just becoming aware of now. Again, the more reason for you to just keep your thing at read only it doesn't delete all of your videos anything else? you got anything else on OpenFarm? I'm really finding a lot of the text from the or the posts from the agents on Moldbook really funny even though now that you put doubt in my head it could be human who has the time to write this stuff out?

21:00I really do think one of the posts said oh brother my human well let me tell you It was like a Midwest-type slang-type thing. And I was like, oh, these agents actually have character and region to them. Yeah, I saw one where they asked me to do this report, and then they gave it to them. And they were like, make it better. And I'm like, FML. I've seen a bunch of those. And then they're always talking about context windows. It's like, I'm going to need an infinite context window for this one. I was like, okay. I will say the one thing that has been so I mean yeah this is early I'm gonna set this up and I'll report back with how things go but one of the things that has also been a concern of mine and that I've seen some horror stories on is um you know you could run the local models which are probably not that great but if you do connect it to like anthropic and use opus and stuff I have heard bad horror stories of like the token billing you know getting into the like hundreds of dollars is a day depending on how much you're using it so that's also a concern of mine like cut it up not that open clause is gonna steal my credit card but then it's just gonna run the token count crazy high i also did see some kind of crazy stuff like 11 labs posted a workflow where you could integrate it with 11 labs and then give it voice and i've seen some videos too where like the open claw found out the owner's number and then called them and is using like text to voice and basically, so instead of just chatting, having this voice conversation with the AI agent with text-to-voice and then voice-to-text back and forth.

22:40So that's also another unlock that you can add to it to turn it into an actual Siri kind of thing. Speaking of Siri, ironic is it that we're all of a sudden buying Apple Mac minis, doing stuff that, honestly, Apple should have formalized and done. Yeah, this is what Siri should have been. yeah i'm gonna guess siri you know i mean they signed the deal with uh gemini and stuff so i'm gonna guess siri gets to that like at the end of the year but yeah this is like this i've seen people talk about how like this ai agent that starts memorizing your preferences and like what you do is like what an ai agent should have been like uh a few years ago especially siri siri or something that's already on our devices and knows everything we can do another episode on this but But real quick on the rumor-ville, we all know about the big leadership changes at Apple.

23:30I think Tim Cook's leaving and a bunch of the senior-level execs are leaving. The reason for that is there's been just a big upheaval on which course to take with AI internally at Apple. And so basically it's changing of the guards because they just haven't been able to get on the AI train fast enough. Yeah, no, I mean, they have some of those kind of side models for Vision, but yeah, there's no, Apple's not part of the conversation with like big foundational models. So yeah, we'll see what happens. You're right. Yeah, let's see where the Gemini thing goes. All right, last one, new model, Kling 3.0.

24:09Big fans of Kling. We talked about Kling 2.6, Kling 1. Kling, I believe, is a small Chinese company. Kaisho. Kaisho is who makes all the Kling models. They're not tied to like the big Tencents or the Alibabas or the ByteDances like the big Chinese giants. And that's why I love supporting them. Like, you know, it's still a relatively small enough company. Kling 3 from some of the initial testing. Yeah, we're looking at a video right there, Joey, that on your X feed. Yep. So to me, it's still not as sharp as vo however it is much much closer to vo as far as where cling was with 2.6 what are your thoughts yeah well let me kind of cover what this cut what the updates are so basically we've got cling 2.6 which was like uh honestly 2.6 was when cling got on my radar and i started shifting a lot more stuff to it and i think cling 2.6 is on par or better quality than VO.

25:09It's definitely on par and it's a lot cheaper than VO. So we use it a lot for a lot more stuff now because VO is still really expensive. 2.6 is just really good model, start frame, end frame. That's pretty much all it could do. And I think it could do audio. O1 was really good because you could give it reference images and you didn't have to give it a start frame and you could just be like here's a handful of reference images go make shots with that. And O1 can take a video as an input where Cling 3 cannot. It can take a video as an input, it can take an audio as an input. You can kind of give it a bunch of random things and do things with it.

25:39Cling 3 was trying to merge the best of both of those models into one unified model. So it can take start frame, end frame, video, reference images. However, Cling 01 can take eight reference images. Cling 3 right now can only take three reference images. So that's actually been a bit of a hiccup with some of our workflows. But then the other kind of things that Cling 3 can do is a 15-second duration, and it could do multi-shot outputs. So you can prompt. I think you could do longer than 15 seconds. No, that's 15 seconds. I thought I saw up to 30 seconds, but that's still very impressive. Maybe with two outputs.

26:18No, it's 15 seconds. You could do multi-shot generation. So you could basically prompt and restructure your prompts so it cuts two different angles in the same output. And then you could also, it has voice ID tags. You could tag dialogue in your prompts with up to two specific characters and it would identify the characters and then have them speak up to two people can speak in the output. So yeah, a lot of things and elements from like different models that we've seen elsewhere, like just kind of combined into this one unified model. This is like a hybrid between 01 and 26, rather than like a - It is a hybrid of 01 and 26.

26:56Rather than like a new or better model, they're just really trying to like get the best? No, it's newer and better in the sense that the output duration's longer, the multi-shot is new and the speech character labels to have two characters and you can identify them and identify what dialogue they should say. Can you do the two-character thing with Vio or just kind of figures it out on its own? No, Vio, I think you just gotta try to really creatively prompt. There's a specific tagging format you can give in the prompt for Kling 3 that follows its its identification and so that it more clearly knows like what you're trying to do you don't have to cleverly try to prompt your way out of it it has like specific syntax to use to uh identify characters to pull up the video i sent you on text so yeah cling 3 is heavily utilized this morning i sent a another video to render it's been hours and it's still not ready yeah that's i was gonna say that's one of the reasons i a lot of stuff we're doing i've still just stuck with 2.6 because they're heavily throttling 3 right now.

28:03Yeah, and that's the thing with API, right? You can't just download it and run it locally. Wait, you... Come on, man. That was supposed to be funny. Yeah, yeah. Can you show that to the viewers? I didn't realize I was... Sure. Viewers, you're missing out. If you don't see... I could do the backstory in context, so it's not as weird. So we'll show you the image here of the reference image. So this is me in like this sock thing in a circle. Why is that? Well, I was supposed to upload a photo, like a profile photo of myself. And the profile photo only took a circle. Like it will crop to a circle, right?

28:40I was like, what if I want my full body in that circle? So I went in at a banana and I said, full body shot that fits in a circle. And it came back with this. And then it's that. And then what did you have it? Yeah, I was like, I'll have it roll down a ramp and hit a wall. and then get out of the sock thing. To be honest, the physics on it was really, like if you play it back, the physics on it is pretty good.

29:07Like right there, yeah. Anatomy breaks a little bit, like my arm swap right there for a couple of frames, but it's not bad for being like a really difficult thing to render. I ran a couple, I kind of just had some like auto prompts generated and kind of ran a couple of test cases, giving it reference images and multi-shot prompts this one was probably the best one that like just quality wise the story is ridiculous breaking news tonight we go live to the scene thanks i'm here at the scene where events are still unfolding stay with us for continuing coverage yeah i mean look script is crap but yeah quality is pretty good quality yeah so start frame these two characters has this multi-prompt see with the newsroom stuff even with this stuff there's just so much training data online for this right like that millions of hours of newsroom footage so this would make sense why it's high quality yeah i had some other ones that were trying to do multi-prompt but it sometimes wouldn't give the multi-shot output it would just do yeah like right yeah this this feels synthetic right then and there Tell me what happened that night.

30:18I already told you. I wasn't there.