XGRIDS' $5K Scanner, Krea Realtime Video, Acer's AI Workstation, + More AI News

0:28All right, welcome back to the noise. I did it with Geosystems about how LiDAR can play into film production. The problem with LiDAR is these scanners have been really, really expensive. And they come from a background of surveying, construction. They're looking to get very accurate data, but not the most photogenic data. Here comes X-Grid. It's X-Grid that has some other scanners in the past, but now they've built a new scanner that is specifically targeted for film production called PortalCam. And also its price point is excellent. Scanners traditionally cost anywhere from$15 ,000 on the low end to$60 ,000 to$100 ,000.

1:03Portal cam is$5 ,000. That's insane. Yeah, that's awesome. How's the quality? Yeah, the quality is excellent. It is a LiDAR scanner combined with a variety of cameras on the front and on the side, which, let's get the specs of these up, but these are much higher resolution than is normally built in. and so I had a briefing from them a few weeks ago and they showed some scans of like an apartment and basically however long you sort of hover on a space is you get more detail. So if you kind of walk around the space and quickly you'll get like a decent amount of detail but if you kind of walk around and like go in crevices or go towards walls or details or closer to objects you'll get a higher resolution.

1:44And it had like some pillowcases and some like very kind of fuzzy texture items in this apartment they scanned. And if you went in and zoomed in really close, you could see the texture of the objects. And so just the combination of the cameras that is capturing the actual video or physical data, and then the LiDAR scan that's capturing the actual geometric space. And then their software just processes it all for you. And then you get these really beautiful, really complex, high-quality scans. Yeah, so it sounds like it's doing slam tracking on device, similar to what Leica or any of those higher-end scanners would do.

2:22I mean, it's funny. We're covering this on AI Roundup. As much as we'd like to think that a lot of this production is AI-powered, the reality is 99.999 % of productions that are using VFX are using traditional stuff like this. And this is still very much a work in progress, right? Like there's still room for improvement with photogrammetry and 4D photogrammetry, videography. And when you are recreating a world for VFX, one of the first things that you do is scan the real world. And you bring that in even as a reference object. And then you start to augment it with 3D objects. So this is like a bread and butter tool, I would say, for any sort of VFX team that's combining digital elements with practical elements.

3:11And, yeah, I mean, there's a variety of use cases with this. In film production, I mean, anything from location scouting. I mean, so you can go out to a location. You know, usually you're like, oh, let me take a couple photos and stuff. And then you're like, oh, I missed that angle. Or like, oh, what's the measurement of that door space or that window space? With the LiDAR and the scanner, you just walk around and scan the space. And then later on, you can just navigate around the space, move around it, take measurements. And the measurements are really accurate. And also, even if you're trying to scout somewhere, you know, super distant, you could send a person or have someone with a scanner and somewhere else scan the space.

3:43And then you could just navigate it on a computer and a headset, virtual location scouting, shot planning, same idea. You have this space scanned and then you could just figure out your shots in that virtual space. Yeah, it's really important even on pre-production as well as principal photography. Exactly. I mean, you could load it up on a wall. You could do virtual production with this and this is your environment. What do you think? I mean, we haven't seen the actual scans here, but what do you think? What would the level of quality be for what you would need for final pixel for loading on a virtual production on a LED wall?

4:13Yeah, so it's hard to say from the outside of the device looking in because you don't know what type of lasers it's using. It obviously looks like using some RGB cameras, multiple of them. So either there is a really advanced algorithm in there that's doing the slam tracking or there is just really high quality capture elements in there. The reason why the Leicas and some of those what I would call industrial grade scanners cost so much is because the actual hardware inside is inherently expensive, right? Like, you know, if you want a high powered laser that can, you know, you can shoot up to like half a kilometer or whatever.

4:53That's going to be expensive no matter what. So they're relying more on RGB sensors here, which tend to be cheaper. But then you still need a pretty high-powered compute on that device to crunch those polygons in real time and make sense of them. And that's what the SLAM tracking does. Yeah, I mean, they had told me, too. It's like the slower you go, the closer you get to the surface, the space, the higher detail your scan would be. Yeah, if you're in a big, large open field or something where you need to capture a larger area. Yeah, it looks like it's a class 1 laser, and it gives you the nanometer range there.

5:28that's beyond visible light. Here's the camera specs. Four camera ray, two fisheye, two front. Right. So it's got enough hardware in it to be very, very useful. And at$5 ,000 price point, that's insane. That's like one-tenth of what you would pay for a traditional scanner. Yeah, yeah. I mean, this price, I mean, even just for a regular scanner, I don't think I've seen a price point like this. Yeah, if you're a professional VFX artist, you probably own like a Canon 5D for capturing HDR and things like that, I would imagine at the same price range, you would probably want to own one of these and do both photogrammetry and light capture at the same time.

6:08And also the nice thing is their software does make it easy to load and process this. Cause like, I think, I think the gap too, is like, if you're just a regular filmmaker, not a VFX art or so not, someone who works is super familiar with like 3d tools or 3d space, like how, you know, the, the process from scanning to getting a photogrammetry 3D model scan that you can move around in and making that easy to use. I think their software handle is the bulk of that. But then if you want more, if you want to take that model out or load it into Unreal or Blender, you can do that as well. So it's sort of got the levels of user complexity.

6:43For sure. I would say the game changer here would be if they integrate directly into RealityCapture, which is the standard go-to software for anything photogrammetry has been for a long time. And Epic Games acquired it and now it integrates really well with Unreal Engine. I'm guessing they probably know about it, if not already integrated into it. And also going back to AI, the one thing we mentioned, and I'm 99 % sure their software does do it, is a gaugness plat. So you could also take your scan, turn it into a gaugness plat. Nice. Something that's lightweight, easy to navigate, and have that as well as an option versus a standard 3D model.

7:18Do you know if it does 4D, like if it actually does anything moving? I don't think it does movement. I did see, they don't have it here, but they did show me some demos where, I mean, we were talking about scanning 3D spaces, but like it is also designed to scan objects and the cameras are high enough quality that like, and you just can kind of go up to, you know, your object and walk around and scan it. So you can turn it into a 3D model. And that's what demoed scanning a person where there's a person standstill and they could really walk around and scan, but they were static. I don't think anything was video.

7:49I don't think there was anything with movement in it. The only thing is it has the opposite of movement feature where like if you're scanning like a sidewalk and people are walking by, the software can remove people from the scan. So it's clean or cars or stuff like that. So, yeah, that was the first one I'm excited to see once that's out in the world. And more and more people and filmmakers are using LiDAR scans as part of their production process. Yep. All right. Next story. And this is firmly in the AI space. Korea has teased real time video. Wow. It's almost here, man. I've been talking about this for months.

8:25Yeah, we've talked about Crayon before. And I mean, they're always the one that had the sort of best real-time AI image generator where you would kind of draw your shapes. And as you move the shapes around, it would update the image in real time. But it would keep making like different versions of the image. In this demo that they're showing here, it's the sort of same shape idea, but the video is consistent. There's a lot of temporal consistency, which is really difficult. Yeah, and this is, I mean, the most obvious thing that comes to mind is ideation and previs and pre-production, right? Like when you just need to quickly iterate on a bunch of visuals, craft your shot, block your character.

9:00I mean, this stuff would, because visual quality is maybe not the most important thing then. But my mind also goes to games, right? Like how far are we from real-time AI rendering? For games and spaces like that. Yeah, you're right. Even like a 2D mobile game that doesn't rely on any sort of 3D mechanics per se. A real-time 2D renderer from Krea, the way you're moving vectors and shapes around, that could be enough of a building block for you to deliver a game on that. That's interesting. Yeah, I'm just thinking too, is it just going to be like a camera record and just everything you're doing, it records it and then you stop recording?

9:39Like puppeteering something? Yeah. Or yeah, just like what's the, you know, because now the video is going in real time. can you just like record everything you do on screen for sure i i mean like what is this is also getting into the world of performance art and any sort of like immersive art where you know there is a spatial and a temporal element to it and you could do you can do some really cool things by driving uh like a puppeteer or an artist driving some basic shapes and lights and then have that transform via ai into something completely else so yeah i'm wild to see this out They had a waitlist before, but I think I just saw today that they're starting to let people into the beta program that were on the waitlist.

10:20So, yeah, I'm curious to see. Yeah, also I'm curious to know if this can run locally on a local machine or this is just real time on the cloud. I think everything they've done is on the cloud. Right. Except I think they had that partnership with Flux with a model, an image model, but most of their stuff is the cloud. And I would imagine this is the cloud. But, yeah, I'd be curious if there's a… Yeah, can you imagine running this on Comfy locally? A Comfy version. Yeah. Yeah. That would be cool. And if you could connect it also, I mean, they didn't show it here and I don't know if they'll enable this feature.

10:46But the other thing that Kria's image generator would do is it was known for the little shapes and you could draw and move around. But you could also use an input, your webcam as an input and you could, you know, use your phone as the webcam. And so there were ways to sort of like find a shot and use that as the framing for the image generator. I'm curious if they're going to do the same thing with their video generator, because then that would also unlock, you know, like, what if you could just film the person, but you change the person into the image. to an instant dinosaur or creature or something.

11:15Maybe you're able to augment it, right, add a different costume or something in the background, something in the foreground. Kind of like what Luma does with Luma Modify. Right, but you're doing this in real time. And you would actually have a device you could move and frame your shot instead of playing around with shapes and stuff. Exactly. Actually, I guess that's probably answered my question because in this demo here is a guy that looks like a webcam and he's messing around with real objects. So, yes, I think I just answered my own question. You can do that. Yeah, in real time, opens up a whole new world of possibilities.

11:48So this is super exciting. And I think this is like a little bit of a fork in the road. We're still going to have traditional AI rendering, if you will, that's high quality, that's trying to achieve photorealism. And then we're going to have an offshoot that's this, that's like a real time version of it that's meant to do cool things that don't necessarily translate back to quality. Yeah, yeah. It's like, oh, what if you could just rapid the ID8, like super fast or just make some, you know, new genre or some experimental film kind of thing. Absolutely. All right. Next one. Our old favorite, Juan.

12:17Juan. 2.2. There was another model. They have another model. They're not done. And another one. So this one is Juan 2.2 S2V. I believe the S is speech to video. So this is a voice-powered avatar animating model where you can give it the image, you can give it your audio file, and then it will animate the character to, say, beak with what your audio file was. And like Juan, you could run it locally on your computer. So yeah, so just sort of like another, you know, just in that same bucket of things you'd like your AI to do. And we're showing people talking, given the character input and the audio input.

12:53Yeah, I got to say, listening to it, it sounded pretty good to me. like much better than what uh vo3 originally dropped and then um i think they're well you are giving it you have to give it the audio so it's not generating sure it's not generating the audio so presumably your audio you either recorded it from an actual source or you know 11 labs is generating the lip sync and the timing and everything the lip sync yeah which is still and the performance yeah i yeah is that what 11 labs does 11 labs is text-to-speech they don't really do character sync. This is more like what Hedra does. Hedra does.

13:28Or HeyGen, maybe. Or HeyGen. Yeah, this is more in that category of the animated avatar with a speech input. But you could run this locally. And also Comfy already has native integration with a built-in workflow. So if you update your Comfy UI, you can load up their workflow that they already built and download the models and run this on your computer. Very cool. And I'm guessing it's all free. Or there are a variety of cloud options you can do. Yeah, it is free. as is with most Chinese models as we've talked about the Chinese models they are free, they want to get you hooked on them you are the product I had also another audio update from Hunyun I got it this time I got it they had another open source model called Hunyun Video Foley and so it is I believe it analyzes the video and then it generates what it thinks are the appropriate Foley sounds for it you heard it it's alright, it's okay, it's getting there I think, look, they're all trying to close in on the missing components of an entire movie pipeline.

14:29And obviously Foley is missing up until now. Sound and speech is something that still leaves a lot to be desired. So, yeah, these guys are closing in on every single bit of movie production. And I think it's a step in the right direction. Obviously, as they version up, it'll probably get much, much better. But for now, I don't think it's quite usable yet for any sort of feature films. Yeah, I wouldn't say for feature films. Now I'm trying to remember because this conversation feels semi-familiar and I don't remember if it was this specific model and we're repeating it or there was another model that came out that also focused on Foley background sounds.

15:02Because I remember saying in this last time we did this. Adobe has something built into Premiere, I believe. Oh, no, but that one was cool. That one was like you could make the sound effects yourself and it makes it sound something good. No, it was something that was a similar video input and tries to figure out the Foley. So I don't know if we're repeating this or if it's a different model, but the gist of what we were talking about with that one, too, was even if it's just AI generated stuff, like sound is so important. Sound also just helps sell the effect and just background sound. And that was just like that audio texture that helps sell the effect of the fake world that you're building.

15:39So it's the fact that someone's focusing on it. Right. Exactly. Yeah. And so the fact that there's models and focus and attention onto that, you know, is important and it'll only improve. Yeah. Yeah, the audio models tend to have one advantage over video models is that they are much lighter and they can run much faster. So a lot of times they can actually run on your device pretty easily. Yeah, yeah. Speaking of on device, Apple, Apple AI update, which we don't normally talk about. Nice transition. I love it. Yeah, Apple threw me for a loop, man. I kind of gave up on them after the whole liquid glass announcement and everything else that they missed on the AI front.

16:15But it turns out they're doing a ton of AI research in-house. Yeah, I mean, I'm not surprised they did a bunch of AI research. But it's interesting, too, because they basically released some models directly on Hugging Face, which is only if you're like an AI nerd are you going to know about this, or Hugging Face. I love Hugging Face. I think it's not a very publicly facing thing. Yeah, and, you know, shout out to Clement Delangue. Delangue? Sorry if I'm slaughtering your name. But he posted out that he found this alert of this new model that Apple posted on Hugging Face that is called FastVLM.

16:47And Mobile Clip 2. And Mobile Clip 2. Yeah. And it is analyzing video in real time and doing live captioning. Yeah, I think the Mobile Clip 2 works in conjunction with the FastVLM to generate those live captions. To look at the video and then live captioning. Yeah, the caption generation is actually probably from Mobile Clip. And then the VLM is analyzing the frame and the video to figure out what's going on. Yeah, so it says, so that's pretty much doing it in real time as the video is playing. Yeah, that's really fast. Really fast. Yeah. And so, yeah, it could even do live video captioning 100 % locally in your browser.

17:23Now, I think a lot of people are wondering, like, why this is exciting? Why do we need live captions? Yeah. Well, let me just tell you. Imagine you have an iPhone, and that iPhone has a really nice camera, which it does. And now that camera can look at the world, analyze the world, know which cafe you're looking at, know which street you're crossing, know which person you're talking to, and then become almost an AI assistant, give you advice on what to order at the restaurant, which direction to go for that destination you're trying to reach, or what agenda you have with this person you're meeting.

17:56So it could become a very capable interpreter of your day-to-day life. Yeah, to understand what's happening in the world around you. Yes. And do this in real time and do it locally. And so, yeah, I think Apple has sort of missed the ball with large language models and building out these massive models. And they're talking to Google and other companies to partner for that. But I think where they can win is if they can get models that are small and light and specifically functional that run on device. Yeah, and they have a really amazing device that this stuff can actually run on. Exactly. So imagine if Siri was an actual assistant that can see and process the world, not just a voice assistant.

18:37And you don't have to wait, have an interconnection or wait for it to process or figure stuff out. It just does. It'll even work on an airplane where you'd have no reception, right? Because it's all running locally. Yeah. So I think that's where Apple still can win in the AI game. Especially because their devices are in so many places. And they have the custom built hardware. And they can build the same models that are fine-tuned for this custom hardware and processors. For sure. I think in our world, the stuff that gets the most attention are image generation, video generation models. We're very visual oriented in film and TV.

19:08And so we tend to focus on that. We went bananas for nano bananas, right? But I think what will really change the world for everybody else outside of our industry, just like 99 % of the public, is stuff like this. either it's going to be an LLM that's super capable or a VLM tied to an LLM that then gives you complete AI agent capability on device. Yeah. And I mean, like go talking about image generation, like the one thing that Apple has had released with AI is Memoji or whatever their image generator thing is. And we've joked about how, you know, playground, yeah. How, um, not really useful that is.

19:45Yeah. What's like actual useful AI tools come out from Apple, like a Siri that works and is functional and actually helps you do stuff. Can I tell you a funny story? So I have a family friend who's, I guess, not as tech proficient. And he just discovered AI. So he'll occasionally send me playground, Apple Playground generated avatars to make fun of me and mock me. And to retaliate, what I do is... As it was meant to be used for. I go into Nano Banana and send a highly photoreal, highly complex thing back just to one-up him every time. I will say the most useful function that has happened in Apple or iPhone, the iPhone recently is where you can, the AI that can isolate the subject in your photo, but then you can turn that into a sticker.

20:34Yes. And so in our group chats, we have like taken the like most unflattering photos of all of us and turn them into stickers and now use that as like reaction posts on comments. How do you delete? yeah so um you know that's using that's using ai for really important things joey yeah exactly next one ai hardware update nice so you know we got the d uh the a the nvidia spark g uh g they renamed it their little mid the nvidia mini computer that was talked about at cs and we haven't heard about it again or something like that i know they changed the name of something it was like there was like the spark and now it's the dgx or gx it's something it's that little gold box yeah it looks like a little apple mini but it's built from nvidia and it's like a little personal ai computer acer now has announced they have their own ai mini workstation built on the nvidia gb10 super chip so this one is called the veriton gn100 ai mini workstation oh no very very catchy very it's good that's that's a really good naming veriton sounds like voltron uh so it's many workstation it is selling or retail for four thousand dollars us what would you use an ai workstation for yeah so four thousand is actually not bad for a high performance computer like it's you know a gpu alone is four thousand dollars and i'm guessing you are probably getting as much juice as like a 40 90 card perhaps even a 50 90 card.

22:07So again, it's what would you need it for? If you are a developer that's using comfy UI day in and day out, I could absolutely see high usage of you. But there's not that many of those people out there. I would imagine this goes into maybe not now, but in the near future, when you have a company that relies on AI tooling, and a lot of this local inference based, you give your employees these computers to run on their desktop so like this replaces the computer you have but everything on it is agentic and ai based and running and running locally right so yeah i mean i guess i could see that out for yeah like large enterprises where if they want to have i mean an example from our world is like vfx artists so maybe not now we are still very much you know transitioning in a transitionary period but a year or two from now i i strongly believe a lot of the VFX tools and capabilities will be AI based, right?

23:07And it'll be very complex, comfy workflow type things where you're doing multiple things with multiple models. And all that stuff will most likely run locally with maybe a couple of API nodes that are connecting to the cloud. So for those instances, when we typically relied on VFX hardware to be, you know, GPU heavy and incredible RAM and capable memory, now we're going to transition to something that's really AI efficient and ready for local inference. And yeah, I hear from the press release too. Developers, researchers, data scientists, students can leverage common frameworks and tools to prototype, fine-tune, test, and deploy large language models locally or seamlessly scale out to any accelerated cloud or data center infrastructure.

23:47You could also link to Veritons to scale up and work with AI models reaching up to 405 billion parameters. Pretty big, pretty big. Yeah, that's awesome. So yeah, I also have your training, your own models as well. Yeah, totally. I can see that. A lot of the AI training, I mean, you have resources like Replicate and, you know, like cloud-based resources where you can pool a bunch of GPUs together. Exactly. And not have to buy a bunch of GPUs. NVIDIA is really doing this to accelerate AI development. I don't think they expect to make a lot of money from this. If anything, I think they're losing money making these boxes, but it's like a loss leader to have more AI models out there.

24:28yeah because if there's more ai models out there they got to run on something you gotta give people the hardware to make the models that then make them money yeah there's a pitch for jensen's shovel three chess moves metaphor somewhere in there the more you buy the more you save yes all right back to apps yeah not ai but interesting to the film space adobe premiere they now have an iphone app they brought it up well that's pretty cool i i thought they had given up on this whole mobile effort with premiere on the mobile thing right because it's not the first time premiere or not premiere but adobe has done mobile apps they just never called the premiere it was like adobe rush and it was sort of like a different branding that was sort of trying to position it as like yeah and they had a i think they still have photoshop on mobile as well yeah definitely for the ipad because that's like a bigger use case because you know you got the ipad's cool yeah so i mean so it's cool that they kept the premiere name, the premiere, premiere branding and brought it to the iPhone.

25:29Uh, I think this is more to compete with like CapCut and these other mobile first video editors that are really popular. They're cutting vertical videos together and it's, it's all like three second, 10 second clips, you know, very, very social media oriented. Yeah. Which makes sense for working on an iPhone. All the creators that, you know, make their living off of vertical content. And they're doing so much of it on the phone, on the fly, right? Yeah, that's their main device, and that's where they started on. You know, I'm wondering, too, I'm thinking, too, like, probably, like, also by keeping the Premiere name, this is probably a good entry point, too.

26:04You're a creator. You're someone young. You, like, start on your phone. You start making stuff on your phone. Oh, hey, by the way, there's, like, a much more powerful unlock of the software. Goes by the same name. Same exact thing. Probably similar, you know, UI. Like, similar features. and then it's not that big of a leap to be someone who starts working and editing on their phone and then switches to the desktop version of the computer. It's a great play on Adobe's part. And I think the other side of it, if they pull this off, and I'm not sure if they already have, is if they have Creative Cloud integration on this, and then you can upload your project file, your video, all of the proxies up to the cloud, and then pull down on the desktop and continue your work.