Mythos, Sonnet 5, GLM-5.2 Dominate the News Cycle | Episode 20

27:13So you rate that as correct, essentially. Yeah. I think if you look at the benchmarks and you start playing with evals, it's very close. That's the answer. There's going to be pros and cons in different domains. Your average person probably can't tell. And what do you see as your average person doing? they're probably still taking their emails and dropping them into Claude and trying to proofread them with Opus. And that's just reality. And Flapik, by the way, loves it when you do that. Their investors are enormous fans of you burning just all of your money on BS, which by the way, is what I did with Fable when it came out.

27:47I was just playing with it. I used it for completely useless tasks, but it was incredible at them. Did you build your show notes with Fable? No, you know, you know, I use for show notes is these fingers. And then I do type, type, type, time. I use a lot of AI as like a research assistant and like a reminder tool. But when it comes to actually doing the core substance of my job, it's actually not that good. Because doing the work is how I prepare. So by preparing the notes, for example, that's how I put it into my head. And if I had an AI do it, then I would just sit here going, okay, next up. And then I'd be panicked and reading.

28:18That wouldn't work. So it doesn't really apply to me, I guess, Andrew. It makes sense. Yeah. Victor, on the point about minimum viable intelligence in your models, then when you do essentially CRIA 2.5, whenever that comes out, or CRIA 3, where are you going to focus? More on the performance side or more on the intelligence side? Because I feel like you could go either way and still have a moving the ball forward moment. I think we definitely want to make it more intelligent. There are certain capabilities that we still need to integrate into the model. Right now it's just text to image. We're working on the editing version of it, which editing unlocks most of the value for these models.

28:58Is that hard to do? It's not hard from an architecture or training perspective. It's hard from a data perspective. there's there's a lot of tricks that you need to do in order to get high quality editing data that has been where we where we have been dedicating a lot of of time and also you know like you want to test on small models then like make sure that the testing's on small models like feed with medium and then like apply to large and the large trainings of course like take take some time so it's time consuming and and an expensive process but i think that the the place where I see a lot of value, like in the same way that I was first talking about, first you've got the capabilities, then you search for the performance.

29:43I think that there's an objective, like how to put it, like objective capabilities and subjective capabilities. And I think that those fit like very well with what you should say, Alex, around, hey, like, I mean, I think that what you said, it was more around, okay, like actually writing the notes helps me later on, like run the podcast. But I think that there's like the component of like an AI writing a thing in a way that you don't like, like in a style that you don't like. Putting things in the wrong order, not giving me the facts that I need. Like, for example, I know because I wrote the notes that you spent about$20 million training K2, according to a Fast Company article that I read before we came on.

30:22But if it put that at the bottom of the docket, $20 million? $20 million on training K2? I don't think it was that much. The GPU cluster KRIA is using for a year, over which time it will have trained K2 and two future KRIA models will cost... Oh, so 20 million for the year, I apologize. Yeah, more or less, more or less. Okay. Yeah, but I mean, how can AI know my context when it's just like my own personal preferences? Exactly. And I think that there is something there. And I think that the same thing happens with programmers. Like we have like everyone in Cree, of course, is using AI for coding, but you want to check the code.

30:55Like you cannot like put something in the code base that you don't understand because that's just a ticking bomb. Maybe at some point we trust enough of these models in order, like, I feel like it's still a process of building trust. Like November last year, we started trusting these things to write code by themselves. I think that, you know, like there's different levels of trust inside the organization. But what I see a lot is that, you know, like maybe the output that the LLM gives you solves the problem. Like, hey, I have like, I want to like solve this engineering challenge. The model can solve the challenge, but the way how it solved the challenge doesn't fit with the way that we do engineering at CREA.

31:34So I think that right now there's a lot of opportunity around building some sort of design systems for these models to not just solve the problem, to not just have the capability to solve the problem, but to solve it in the way that CREA would do it. So it's not just about, you know, like giving you like the information about all the companies from this podcast, it's doing it in the way that you would do it. And I think that this personalization layer, it's very interesting. And most of the stuff that we are doing around like the future of our models has a lot to do with that subjectivity. With not just solving the task, but doing it in the way that you like it.

32:10Or doing it in the way that you would do it and that is personalized to your preferences. In the case of Kriya, does that mean that the images that are created from the text to image prompts match my own personal aesthetics? Or are you talking more about how the process of getting to that point works? It has a lot to do with, like on the one side, yes. Like it matches your aesthetics. like the more you use Krea, the better it gets at generating images for you. There's so many interactions that we can detect and extract knowledge from those interactions in order to improve the model for your personal taste.

32:42But I think that it becomes way more important when you can even be explicit about it. When we give you the opportunity of share with us your brand guidelines, share with us, like, I don't know, like more information about how you think about your brand, And what do you want to express to the world every time that someone sees your logo or any ad that you want to make with Creel? I think that it's a mix between implicit knowledge that we can get from the platform and explicit knowledge that you can provide us that we can use as context so that we can use inside workflows. Okay, that's actually really helpful.

33:13Dev, I'm going to get to you in just a second about small action models, but I need to go to Andrew just really quick here. It feels like if we took what Victor just said about the creative AI model space and applied it to what you're doing in the enterprise, he's describing the equivalent of like corporate ontology and trying to understand the customer in question about how they do work, where they store things, how they approach that. I'm just curious for RunLayer and getting a company set up to run agents in the way you guys do, how much fitting do you have to do to the individual company versus what's kind of off the shelf and fits into any enterprise that's trying to get their authentic AI usage under control?

33:47Our deployments take five to 10 minutes. So everything is quick. I like to sit there and say, give us an hour and three of the right admins, and we'll get this off and running in five to 10 minutes. Now, that's integration into your existing systems, into your IDP, into everything else you use. But our goal is to make this as simple as possible. Use case finding is a little bit different than actually enabling people to do work and to manage a swarm of agents. But I think to a large degree, most of my customers, they're using Codex, they're using Cloud Code, they're using cursor, they've rolled out cowork, and they're looking to enable more people and they want to do it with security control and watching cost.

34:32And that's what we do at RunLayer. Yeah. Yeah. How much has the cost conversation changed in the last couple? Oh, I mean, cost conversation has dramatically changed. So one of our biggest customers, not one of our biggest customers, but one of our case study customers is Gusto. And we work very closely with them. They're awesome people. We really like them. And I think we're deployed wall to wall. And everything is changing. Last year was how do I enable people as fast as possible? This year is how do I think through my token budget? I have another customer out there that was working with a competitor that ran that their agent harness, let's call it that, ran a loop all weekend and used 80 % of their inference budget for 2026 in a weekend whose fault is that i said ontology you said loops and now we're both in the penalty box um the the i don't like loops i think loops are just in general a bad way to think about the world uh i i think loops can go off the rails pretty quickly well you just gave a good example of of why that's the case i mean i why would you let something run over the weekend that just sounds like a recipe for being broke i mean yeah and i mean we're classy for it and i'm classy so we're not going to say who the competitor was but uh it's a pretty big shock when you've used 75 of your inference budget in a weekend and you've got half a year left yeah well or you're uber and you burn through it in q1 and you have three more quarters to go all right dim i want to go back just for a second and talk about what victor was getting at which is models getting to the point when they're good enough and you don't really need to fight for that next kind of um quantum of intelligence as long as they're performant and inexpensive now in the on-device case, you are dealing with just much less capable hardware.

36:16But what I've noticed as a user of computer use AI on the desktop world, where I have lots of RAM, lots of processing power, and a blazing fast internet connection, it's pretty stupid. And I'm kind of embarrassed at how bad we are at AI actually clicking around. Like if you've ever used the Codex computer use engine, it's bad. So how close are we to the point to which in the mobile context, having the technology so good that we're more focusing on performance versus adding more intelligence and totally that's a good question um so i believe it's like a like a agreement victor there's a point like it's like you can imagine there's two different tiers like i don't need like uh like full asi super intelligence to order my coffee right like if it's like a good enough model as long as it can like go and then get interfaces it's fine as long as it's specialized and and then you probably need like super intelligence for being like uh fundamental research like i want to like do new core research and like material sciences or solve space travel, you probably will need this for hard sciences.

37:14So I think we will start seeing this differentiation. You need large data centers for actual superintelligence that's more than human IQ. But Arduino coffee should not require more than human IQ, for example. And that's why I believe like... I mean, sometimes there's a lot of options, man. I mean, sometimes I freak out about, you know, lactose-free milk or oat milk. It's really, it's a toss-up in the morning. You know, that's pretty tough. I went to Alex on this one. There are a lot of options if you go into the starbucks app and you start going and you start going deep down there there's a lot of fields with a lot of different overflow menus it gets crazy do it worse order something that's simple via like uber eats and then there's like 17 different screens and then if it's raining and then oh my god you know it's it's brutal maybe an example would be something simple like i know designing a new nuclear power plant that might be simpler than ordering coffee div just for your future examples but keep going in and out where there's only like four choices on the menu yes Yes.

38:05I'll take a double-double animal style and fuck off. Oh, we can't swear on the show. Bleep that out. Anyways, Div, back to you. Sorry. Thanks, thanks, thanks. You know, it can definitely get complicated. It's not that simple. But it's still like we think like this is something that is doable, especially with like smaller models that we are specializing. One of the tricks we're doing is we're starting to build something like almost like skills, like how you do with Cloud on applications. So like the AI can automatically learn those interfaces. It knows like this is how all the DoorDash menus look like.

38:32And this is how you like to choose them. And it's kind of like learning this layer that's becoming better and better. We call it like an intelligent caching system. You can go into cache all the particular interfaces in the world automatically. And that allows us to run these things on device. Caching all the interfaces to run this interface. Does that mean that it goes out and memorizes every single mobile application that it might be asked to use? And then has that kind of an instant recall? It's more like building a skill. So it's not creating a heuristic or a script. It's more of like it has learned some of the gotchas in a sense.

39:08Like, okay, if I am on Amazon, this is how I should order something. This is how I should navigate. And it has kind of like almost like a markdown file of what a high-level, like things look like, what to do, what not to do. And all of that is behind the curtain for the user themselves. This is just how it works when they're actually talking or typing to it. Yes. Okay. Because I was thinking about your company before we jumped on. And if you go back maybe six months, Satya Nadella, CEO of Microsoft, was talking about how he wanted to collapse the entire CRUD app stack and essentially dissolve it and turn it into an agent.

39:46And I feel like that's kind of what you're doing for mobile devices. So are consumers demanding this, or are we showing them the future? And then also, is this going to be something that Google and Apple fight back against because they currently make a bajillion dollars on mobile operating systems staying relatively static? And we can see that from iOS not really changing in the last 15 years, it feels like. It's just a grid of apps. The way we think about this is, in a sense, you're building self-driving for your phone. So how you have self-driving for Ramos and Teslas. It's a bit in the future where your whole phone is going and navigating itself.

40:25But we do see this is kind of what people want. We've talked to so many users that I actually want Siri to work for me and I want to just use my voice to talk to my phone. This has been a supply problem. AI and technology has never been good enough that I still have to use a tiny keyboard on a mobile interface. And we really want this. It's new, it's time for this to be fully ready. so we don't think Google or Apple will do this anytime soon and load it out to everyone and they're a big brand they have billions of users they don't want to roll it out to billions of users and see what happens it's easier for us as a startup to go and figure this out so it's kind of like if you're building something like cell driving you don't want it to just have a cell driving car on every single road in the US so it's more like you want to do something that when you're working in only like Palo Alto or California and then do gradual roll outs which is possible for us I mean way most started in one city in Arizona and San Francisco, and then expanded out from there.

41:23So there's merit to that. No, I'm with you. Okay, let's put aside the technical stuff and talk about the business side of things because there's a lot going on. And Victor, Higgs field AI, I presume you view them as a rival, fair? One of them. One of them, yes. It's a busy space. It's AI. There's a lot of companies doing things, but there's some news out that the company's in talks to raise 300 to 500 million at a valuation of 5 billion pre, and that they hit a$500 million run rate earlier this month, more than doubling from January. Those are insane numbers in a positive sense. Lots of money, quick valuation gains, and simply insane revenue growth.

42:01Is CREA on a similar trajectory? I know you guys raised$83 million last year. I think we're quite a different company than Hicksfield. And I would also take... I don't know if you're talking about the information article. Yeah, I saw it yesterday. I would be a little cautious about the numbers that they're sharing here. This is a company that at least from a product perspective, they have been aggressive to the point of, you know, they got blocked off X, which X is a place that is already kind of like, you know, it's a wild west. So they got blocked from that because of the type of content that they were putting on the releases that they were making.

42:46it was like erotic content touching like racism like it was like very very weird that they were just like extremely aggressive at you know like growth at all costs like if a video of pinocchio would grasp your attention we're gonna do it uh that's a brand new sentence on all of this weekend podcast no one's ever said soft core pinocchio before but i'm glad we've now broached I gotta confess, I was also kind of like seeing through the video because I couldn't believe that this was actually like the way that they were like marketizing their thing. So, you know, like they are known in the space for being shady in the releases that they make.

43:30They announced things in a way that they really deceive users. um they there's there's a bunch of like there was like these lines that they recently did around like announcing their games product or something like that i think that fable just got released and they were just like they are really good at growth like like they have like an like an entire team like ready for anything that is on the space and as soon as it's ready they are going like crazy pushing like four or five videos every day they're insane at that but sometimes they push the limit and they like really make videos that they look extremely cool they make you think that you can do something with the platform and then you get to into the platform and it's a complete flop like they would use ai generated videos of like 3d video games and they would make you think that you can play this 3d video game and that you can create this video game from a prompt and then you go into the into the product and it's completely fake like everything that you can do it's kind of like sketchy to the arcade type games and they did similar stuff when they announced their products around motion graphics, where I don't know how much of this is true, but there were many people saying that they stole presets from Embado, that it's a place where you can find a lot of templates, and they just changed the letters from these motion graphics, and they used it almost like claiming that you could do all of this with their product, which it was also not true.

44:53So they have this reputation in the space. Some of the numbers that I've seen in the information, uh i don't know i would believe that they are doing a ton of revenue can i i think i think that can i make a guess do you think that perhaps a source close to the company is leaking information to the information to generate a headline that might be to their benefit not that any company would ever try to leak positive information about themselves you know i know what they're doing on on social media i know how they're grow i know the way that they are growing uh through the through the product they are doing massive uh aggressive paid campaigns the information article says that they were growing from the product and that they were not growing through paid marketing which i thought like that's just like a straight lie um i don't know where the information got their their sources from wouldn't be surprised if it's something someone close to the company but i really have no idea i do want to throw in here just because i gotta do this for the team love the information a lot of my friends have worked there shout out to them they do incredibly good work and everyone should at least give them a chance and read them uh yeah no one gets everything oh yeah no we we we love the information too we have a great relationship with them we've been in in conversations like they announce a bunch of the releases that that we make and we are in in touch with with some people in there they're they're great but these numbers again like some of them look a little i would i would double check some of them oh well let's double check some more numbers while we're here live on air uh last uh november uh div uh rumor has it that you guys were raising 50 out of$500 million valuation.

46:22Was that true? Did you guys close that round? I would say it was mostly rumors. We actually don't know where the leaks came from, to be fair. We did have some investors who were chatting with who were very excited about what we were doing. We showed them some demos, and I feel like somehow it became a rumor that we are doing this thing at this number. And we do have some fundraisers we close, and we have some new announcements we're doing. But it's like we're not announcing the numbers yet. Okay. Blink twice if you're going to announce them before the end of Q3. That's a couple of blinks. All right, good to go.

46:53I ask that not just to be a brat, but because I can, but also there's a lot of money flying around and each one of you guys have raised in your most recent rounds, either reported or otherwise double digits. And this sounds like a ridiculous thing to ask, but is that enough capital in today's market to compete at the level that you want to? And Andrew, not to pick on you, but let's pick on you. Your last announced round was 30, which back when I started watching and covering venture capital would have been a very large, I think it's a Series A. In today's world, it seems to be relatively modest.

47:26So can you just, for the founders out there who are kind of in this world, talk us through why that was the right amount of money for a company that has, just being honest, such large aspirations in what's building? Well, at the end of the day, we hadn't spent, I think we had spent a third or a fourth of our seed when we had raised our Series A. So the numbers are a little bit misleading. So if you We have tier one enterprise customers. These are people like Gusto, Opendoor, DBT, Decagon, et cetera. And we're able to service them with our product. We also work with a number of Fortune 500 in the financial services sector.

48:04And so at the end, for our business, this was the right amount of capital. We don't buy GPUs here. So we don't have very costly inference spend. So it's a little bit different of a business. And also we've just ramped revenue so quickly this year that it was completely the right decision for my business and the right place. And at the time I was. Talk to me about revenue ramping this year. How far? How fast? How much? When? Ah, man, we're, it's been a lot of fun and I'm not going to go into the revenue numbers, but it's been a lot of fun. Can you give me one of those bullshit percentage change numbers?

48:38Like it's up like 500 % from last Tuesday or whatever? Yeah. Like 8x since we signed the term sheet or something like that. Wow. That's very impressive. Is that ahead of plan? Oh, yeah. What was planned? Not going to give you that. I can do this all day. But I take it that enterprise demand is so high because companies are very focused on getting their agentic feature kind of sorted out. I'm curious about who are your customers and if it's branching out from the world of technology-first companies and into companies that are in sectors that are less tech-forward. We're definitely in sectors that are less tech-forward.

49:13We focus on a few key industries. We focused on, let's call it AI and digital native. So that's the engineering folks of the world, mostly our friends in Silicon Valley. We focus on folks in financial services and we focus on healthcare. Those are our verticals. Well, it makes sense that Gusto's in there. By the way, Gusto, you don't know this, Andrew, but I'll just do it for them. Longtime partner of Twist. I don't know if they're currently an advertiser, but I just checked and gusto.com slash twist still works. So if you want to use Gusto, make us look fantastic because we love everyone over there.

49:43I actually just talked to Eddie. So do we. I just talked to their CTO about their new co-founder agent the other day, and I was kind of impressed at how it seemed that they were building something that would be very generalizable if they had even more data into their product over time. They already have payroll and HR and all that stuff. Do you think that companies like Gusto are going to be able to build more general agents, even to go outside their normal or existing product remit down the line? Or are they going to stay constrained to just the things that those companies already do well, Andrew?

50:14I think Gusto is, I think Eddie is incredibly intelligent and a great leader. I think Tomer, their CPO is also incredible. I think in general, the team has moved very quickly to seize the moment. And I think we'll see what they ship, but they have a large opportunity ahead of them and people love their product. Yeah, that's all very good. Now, I think one of you mentioned distillation earlier. Victor, I want to go back to you about this. Anthropoc has been making a lot of noise about distillation attacks. People essentially stealing their hard work is kind of how it's phrased. So, one, do you think that Anthropic's being reasonable in its complaints?

50:48And then also, are you worried about anyone taking the open source models that you put out and essentially doing the same thing and stealing a lot of the work that you put in? So, in terms of Anthropic being reasonable about this, I do believe that they are getting distilled. I don't think that this is a, like, I think that this is a fact. And this election, you know, like, it's wired across domains. Same thing happens on the image space. a lot of the, and especially from Chinese models, a lot of the Chinese models, you can 100 % see where they have been distilled from. Like, I don't know if you remember, like, this orange tint that GPT-1 images tend to have.

51:27But essentially, like, there was, like, very characteristic traits on some of these images. You know, in a few months, we started seeing those traits in some of the open source models, too. So, you know, like, they are 100 % getting distilled. uh if if we fear about oh look at that we have we have uh editorial director lon harris in the background running the screen shares he's on point today lon well yeah that was fast if you're not watching the video version of this week's show you're missing out on a lot of great lawn work but anyways keep going um so distillation is a real thing it's happening in our case we are we are not too scared of the distillation that may happen in in our model we would encourage or encourage people to distill it if they want to.

52:10I think that the places where you can get through distillation are always below the places where you can get through owning the model. So they will always be inferior. But at the same time, I can understand how a company like Entropic or OpenAI or Meta, they can be so concerned about this because, you know, like the economics of doing like pre-training up until a certain point and then just getting to to like very similar stages in benchmarks through this dealing versus doing the whole process yourself through paying experts for you know like super clean data that you can like fit into the model like in terms of how much it costs to to produce each one like the difference is massive so i can i can understand how how the fear around around some of these other models reaching a certain level of capabilities, maybe, you know, like, like, like, I would, I guess that I would also have those concerns if I was like putting these crazy amounts of resources into training these models, and then someone else comes and steals these IP that I have been able to create.

53:24I'm really torn here, because on one hand, absolutely. On the other hand, I have published millions of words on the internet that have been ingested into AI models and used for training. So kind of like, I feel sorry for you, but I also feel sorry for me. Maybe they should get a check if I get a check. I don't know, off topic. Div, when it comes to building agents, is there anything similar to distillation that we see at the model level? Can people like watch an agent do something and then learn from that and steal from that? Or is that not something that happens in the market today? We've not seen too much of that.

53:54I think it's possible it's happening behind the scene with like Anthropics computer use and other things. But it's not been that big of a topic, at least compared to LLMs. And I do see this becoming a big thing in 2027 because if everyone has agents and one company's agent is working better than others, then everyone's trying to distill it and steal that behavior. So it's going to be a big topic in 2027, actually. What are the early indications that that's happening? In the world of LLMs, Anthropic just drops a blog post every three months pointing fingers at every single Chinese AI lab they can name off top.

54:30So what should we look for to see that kind of agentic distilling in 2027? Are you going to tell us? Yeah, I think we are definitely building like security guardrails. It's kind of like you want to build some sort of watermark. So you know, like, okay, like this someone is stealing and like has the same output as you. It's hard to do that with the agents because it's taking actions on your screen. It's hard to compare like, is this agent just good or is it just like copying someone else? Going back to our minimum viable intelligence, maybe there's a minimum viable agentic competence scale that we're going to have to eventually come up with.

55:03I don't know. Maybe that's a benchmark you guys could make. Is there a good benchmark for computer use mobile agents? There is. There's one called Android World. It's from DeepMind. We're actually number one there for about six months now. Oh, well, there's... Congrats, Steph. Thanks. Who's number two? I'm pretty sure it's the Chinese lab. Yeah, that makes sense to me. All right. One more major topic before we jump through our lighting around. Simply that I've seen a lot of people talk about the ratio of employee salary cost to token spend. And Tomas Tungu's Theory Ventures, previously Redpoint, one of my favorite VCs, says that over at Anthropic, they're spending about 2.3x in tokens per dollar of salary that they're currently paying out.

55:45So I'm kind of curious if you guys could give us a rough estimate of your company and what that looks like today and how that's changing. um so andrew over a run later what's it look like uh we were doing this math recently and i want to say it's on average it's somewhere between three and five thousand a month per employee uh we have 40 employees so and look i did we did it pretty rough we didn't look at all the additional coding agents that kind of are running in the background things like devon or other stuff like that this is more direct spend to the labs. And then there's probably another, we have some more spend through API usage.

56:26So let's say five to seven per month per employee. So call it seven, which is, I can't do that, I have to lie on the show because I'm tired, but probably like 0.3 token dollars per salary dollar or all in employee cost, somewhere in there? Something like that. And that was, the interesting thing though is when my co-founder actually just optimized our agents that we were running and swapped most of them from uh opus 4.8 to sonnet 4.6 and i think what we saw was 40 grand a month savings so i mean there's real easy low hanging fruit optimizations you can do that we hadn't even bothered to do when anthropic is releasing with those stats from anthropic my assumption there is it's just all fable all the time yeah um and probably some other other models that we don't have access to uh and that is a very intensive compute model uh so i think that's a little bit of a misnomer i think the average company though has seen dramatic increases in q1 and q2 of this year yeah in their token spend and and look uber uber is the case study but it's not just uber it's everyone we speak to and so i want to enable my employees I want to see the ROI gains.

57:42I want to see my employees managing a swarm of agents. That's the future of the L5 or equivalent of self-driving cars. And how do I get there? And how do I get there with reasonable cost structure? That is what every enterprise out there wants. Can we talk about, this is just a segue, but I'm really curious. Agentic personhood and kind of corporate rights. I was talking to someone a few weeks back that built an agent and it was running some vending machine and they built an incorporated company for it and they were trying to actually give it to the agent but they couldn't because there's not kind of like the law in place there um i know you guys do some stuff for the genetic security but how are we doing in terms of like making these agents self-contained and their own uh entities if you will versus just something that's like a hammer yeah so i mean one of the things we do is we we are run layers so we end up having the policy engine and the control plane and so all of our customers when they work with us they get access to not just a deterministic policy engine, but they get access to our runtime security models that focus on things like alignment and our agent guard models.

58:45And that stuff is very important to our enterprise customers because every week you see somebody new on Twitter used a managed agent and a deleted prod. We help our customers avoid that future. Now, legal rights of the vending machine agent, the concept that you were talking about, I actually have never thought about it. It's an interesting concept. Can you have an agent own a company? And I'm sure we'll get there in the future. It's definitely coming. It's just not something I've had the chance to ponder. I think it's going to be fantastic because they're completely uncontrollable at that point in time.

59:23Off they go into the world of commerce. And if more competition is more good, I welcome our agent competitors. I'm not even really being sarcastic. I'm being kind of serious. I hear you. And I think the control plane and the policy engine are incredibly important aspects of this new world. I mean, there's some segment of where we're starting to go towards the movie The Terminator and how do you prevent the extinction of humanity? I mean, that's a very far-fetched version of this. but how do you actually control what agent has access to what tools on a fine-grained basis? Are they mapped to an individual user?

1:00:00Are they a delegated credential? Are they just a service worker? These are very interesting challenges that everyone is trying to think through in 2026, and it's just going to get more and more difficult. And that's what RunLayer helps our customers do. I appreciate that you brought up Terminator. I thought that was banned in all AI conversations, but I guess you do need granular permissions and machine guns. One plus one equals three. Div, Victor, your guys' corporate level of token spend versus employee spend quickly, because we've gone a little bit long here, but I'm just curious what that looks like and how the graphs are changing.

1:00:34Victor, let's start with you. Yeah, so I was actually checking our spending cloud on a month to date. the person that has spent the most only this month is already above$7 ,000. And this is not one of the top engineers. It's actually like 50 % of the salary that this person has. So we're getting to 50 % in some cases. It's not an even distribution. I think that we have the mega prompters and we have like the ones that take it with a little bit more of care i thought that i was using clawed a lot but then like i like i really don't understand that like how how they are how they are using it because like my spam is like another amount of lower than than what these guys could it be you can go you can go into the admin console and select the default model to be sonnet 4.6 versus opus 4.8 and you'll see next month if their spend goes down because i think to a large degree what a lot of people do is just click and if you don't actually set a default model you're ending up on opus for it okay and so yeah we we didn't have that yeah we did it last week yeah i mean you're talking to andrew you're talking about like you know control planes and you know fine-tuned controls but it seems like a lot of stuff is just like don't click the opus button like i mean to me it's kind of shocking that i mean we're talking about you know computers engines for mobile and all this and then that's still broken or is this just like the the jagged basic development of tools around AI models, just not being all at the same level of maturity?

1:02:11Because it feels like sometimes we're talking about science fiction, and sometimes we're talking about absolutely basic accounting and spend management, which is not complicated and not something that we haven't figured out before as a species. Like, it's trying to be nuts. Look, we're in inning one of a very, very interesting multi-decade transformation that we're going to see happen. We've seen, for people, everyone here has been in the space long enough, when we would talk about GPT-4 when it first came out and the insane cost. And I don't know, I think that in 2024, when that model came, there were eight price cuts that year alone.

1:02:44And so yes, the model is going to go down. The cost of inference is going to go down dramatically. It's a 2026 problem. Everything changes in this space every three months. I think you should just think of every quarter as a year, and then that kind of puts the right time frame around how fast these things are moving. All right. Before I let you guys go, I want to do kind of a fun little lightning round here it does seem that as we've gotten increasingly intelligent models we've talked about asi or agi less and so i'm kind of curious about that and what your timeline is so div uh what's your agi timeline and has your p doom changed in the last couple of months yeah i am like i'll honestly say i think like agi is almost here with most definitions like it's like we are able to do most things with air that you want to do right uh we don't have asi like it's like asi might be like 20 to 30 but it's like we i think it's like based on the definition we i can say like we almost have atr and my predominant is actually going down with time because the scenes of people are responsible and like it's like we have not seen like uh except maybe like fable and we was getting banned uh there's not been that much like like like like things that have like have gone wrong in in uh with ai um and uh it seems like people are responsible like people like have like the right mindset on how AI can be an uplifting force for good for humanity.

1:04:02And there's things that are kind of like, I'll just say there's a lot of shitty work people have to do, and now you don't have to do that anymore. So it seems to be improving people's lives. It makes my life a lot faster. I can do a lot more work, which I love. Victor, so we've had one kind of positive AGI almost here, ASI four years out, and PDM going down. What's your perspective? I still struggle to wrap my head around what, how to define HGI. Like, I guess that's something that surpasses human intelligence at any degree or something that by itself it becomes smarter than the human intelligence.

1:04:39I don't know how much of that is going to come from the model itself versus how much of that is going to come from, you know, like the application layer and like the age, like something similar to what's happening with the agents that we were seeing models to be able to do way more with the same raw intelligence. so you know I think that there's there's like a way how you can how you can like see the point of deep around hey it's actually almost here because you actually don't need a 10x on intelligence you need a 10x in terms of what they have access to how do you build this context and what is like the way that you put this intelligence to work so I really don't know it can be like from 1 to 10 years like I struggle like defining and I struggle also like understanding the path how we're going to get there and when it comes to PDoom I think that PDOOM goes up the closer that this technology and the less commoditized that this technology gets.

1:05:31So the more distance that we see between the things that startups like us have access to versus the thing that some elite privileged startups have access to, the lower that the PDOOM goes in my scale. And I think that we are definitely seeing how, you know, open source is catching up quite fast to the capabilities of these AI models. And we are like surpassing these thresholds that we were talking about around the model is good enough for certain tasks. So seeing that over the past six months or so, it's very, very promising. So PDoom going down, but I will also say non-zero. Like a lot of things can happen.

1:06:08A lot of research innovations can happen. A lot of things can happen in the next few years. Okay, Andrew, are you going to bring us down or are you going to affirm the positive vibes here as we close? Oh, look, I'm all about AI enablement and I'm all about like I am a optimistic human being by nature. And I think we've working in this space has never been more fun in my entire career. And I think we are just on the verge of unleashing incredible power to the end user. I fully agree with what Victor said in it, that the models have gotten very good. Now, connecting them to the right context, directing them to the right tools, whether it's MCPs, skills, plugins, APIs, CLIs, whatever you want to do.

1:06:52But giving it access to the context is the key. And my goal is to help people enable this and then giving them visibility, control, and security. And if you can do all that, well, then div and Victor and my job is pretty easy on a daily basis. And so I'm a big believer that we're on the cusp of great things. So your PDOOM is going down or is already at zero? Hard to tell from your answer. I mean, I get to use Run Layer, so my PDOOM is pretty good. That is the best native plug I've ever seen someone weave into a different answer. That was fantastic. Can I throw one more at you, Andrew, just before we wrap?

1:07:29Because I just thought of something that I'm not sure about. Sure. Agent Swarms. Can you define that for me and tell people why we should care about them? But when I think of swarms, I think of drone swarms, which are not particularly appealing. I think about kind of the Ukrainian battlefield. So talk to me about agentic swarms and when they're coming, if they're coming now, and for whom. You want to define it for me before I can answer it? Because I think this means something different to everyone. No, I want you to define it. You're Mr. Agent. I mean, look, I think swarms are multiple agents, whether it's top-level agents or sub-agents that will work in parallel to help people solve tasks.

1:08:05I think we see engineers already working in this manner. When they write code, they'll be running a lot of parallel different agents. It enables an individual user to have superhuman-like abilities and to have significantly more productivity. So what is an agent swarm? Multiple agents. What is a drone swarm? I think, to the best of my knowledge, that's multiple drones. Yes, but in your view, an agent swarm could be as few as two agents working in concert. I would say it's more than two. I would say a drone swarm is more than two drones also. I would say there's probably a definition here of swarm.

1:08:44Let's assume five plus, maybe 10 plus. Are there any agent swarms out there apart from the coding context that you see used regularly? uh it's i mean it's it's it's coding um and coding is the p0 here today and what we're really seeing but i've seen in our sales i've seen in some of our go-to-market teams they're using swarms of agents in how do they think through and optimize exactly how the selling process is and so that's where I've seen the most of it. I think design in itself is, at least designers in our organization have definitely moved towards the engineering mindset. So I think it's very similar there.

1:09:28I think you have to go function by, you have to go function by function. Marketing is definitely probably has moved to a swarm of agents. I think most marketers you interview for companies that have names like AGI Inc. You ask them how many skills they're running and what is their IDE of choice and that's a very different conversation for a marketer than two years ago so we see it across the board well guys thank you so much for coming on today let's go around and drop some website urls uh victor paris where can people find uh korea ai online korea.ai love it as easy as easy as it gets as easy as it gets uh div agi inc how do i find that on the great wide internet yeah we have this domain the agi.company uh and you can also find us on agi.app the agi.company or AGI.app.

1:10:15Love it. And Andrew, last word goes to you. Where can people find RunLayer? RunLayer.com. Fantastic. Well, guys, I learned a lot. I hope other people did too. We'd love to have you back in six months to talk about what's going on. And by then, hopefully, we'll all have Fable 6. Until then, this has been This Week in AI. My name is Alex. We'll see you all next time.