Aaron Levie, Steven Sinofsky & Martin Casado: How Do You Secure a World of AI Agents?

23:44Like, by this time on the internet, we've taken out, like, tens of billions of dollars of, like, economic... Well, we've caused tens of billions of dollars of economic damage. We've had words that took out 10 % of the infrastructure. Yeah. The internet infrastructure, which was running critical infrastructure. We had hospitals go down. We really should have blocked the internet in like 97. How do we? Yeah, okay. Okay, okay, okay. So we had, I mean, I remember a time, like I remember when you would buy your CD of Windows 95, and by the time it was done installing, you would have a worm potentially.

24:21Way to go, Steven. No, no. No, this was the reality. No, totally. The reality of the PC until 2001 was you could not install a PC connected to the network without getting infected. Oh, viruses were everywhere. And there were two level, two rounds of congressional hearings. You had all of the same. That is actually, I'll grant you, that's a very interesting point. This type of zeitgeist in 94 would have, we would probably not have the internet. Listen, listen, listen. I mean, the automotive, the airline industry, you always have these periods of kind of like teeth cutting where like you learn about the dangers and you learn about the technology and the internet.

24:58I mean, we were on the ground floor of this. I mean, things were down all the time. There's economic damage all the time. Like, there was, like, novel, there was new viruses, there were new worms, they were all over the place. But Y2K was going to destroy the world. But here's the interesting thing about this. But I just want to say, like, so when we created policy, and we did create a lot of policy, it was kind of like with a bunch of very specific data points on what you're trying to do. And so you know that the policy actually fits the fact pattern. And in this case, I mean, even when you were talking, you're like, well, what if, you know, yada, yada, yada.

25:27And it's very hard to predict a policy when we don't know. Now, one area we can talk about is there seems to be novel cybersecurity risk. Great. And what is nice about the discourse is it's actually starting to evolve around that. Like you actually hear people from the lab saying, novel cybersecurity risk, this is an engineering problem. We're talking about that. So the more this becomes concrete around real identified risks, I think we can all fall in line. But that's not been the discussion to date. That's a very recent thing. Perfect, perfect point. Because if you look in 1986, the Computer Crime and Fraud Act got signed.

26:00It was out of a very, very specific scenario, which was two groups of hackers broke into GTE TeleMail. One of them lived in my dorm. Go figure. Sure. No, it wasn't you. He worked at Cisco later. And the problem was that was 1983, and there was no crime. And it took two and a half years for the bill to make it through. That made it very, very specific. And that's the law that says you can't access unauthorized computer systems. So I think we'd all agree that we probably have like 90 % of laws already in the applied layer. Like you can't hack systems, etc. Do you think there's anything that should be from a liability standpoint in the model layer?

26:46Which of course then creates... Wait, and now are you making a technical legal talk? Well, that's a very... Legal. No, the legal, everything, my read of Hugging Face, OpenAI, Ford, they're all absolutely blatant computer crime acts, except for the fact that they carved out in amendments later that if you're a white hat, it's not illegal anymore. And that was because people kept making mistakes, and they didn't want to go around arresting everybody who was actually trying to make the system better because they messed something up. And so the Justice Department wrote a memo that said we're not going to prosecute for this.

27:20And we're also not going to prosecute if you just, like, violate the terms of use of a system versus actually try to breach it. And so I think the law is ample for this scenario. And it was all written, this GT telemedl was used by NASA and Livermore and all the labs. And so that's why it was, it caught the attention of DC because it was federal systems that were being broken into. And the problem we have now is this rift between the labs and the security community that keeps looking at all their postmortems and coming to two conclusions. Sloppy, and you're not complete in what you're telling has happened.

27:57And so, of course, the CVE process came about in the 1980s, the computer virus and vulnerability reporting stuff out of CMU. And for years, they worked on very structured reporting with obligations and how to, and for some reason, they're not using any of that to do this reporting. And so there's this very basic stuff that I look at and say, well, until they're doing that, they really should stop talking. Like they shouldn't do a postmortem on a breach that looks like an intern wrote it and it's not a postmortem. It's this selective memory. It looks like exactly the kind of postmortem you do when you hire outside lawyers to investigate some random thing.

28:34and you only give them certain stuff. Because you don't have to give the lawyers you hire all the information about what happened. Like, where's all the Slack messages? Where's the actual details of what happened? Can I just interject an annoying aside? So, which is... Which is more annoying than what I just did. No, no, this is good. Which is very in line with this, which is I've been in the security, like the actual cybersecurity community for a long time. And it is, you know, it's always been one of these things where like they just don't like practical solutions. So even if you build, like, you know, a secure system, like, well, what if somebody, you know, shows up?

29:07Oh, yeah, yeah. You know, like, you know, can Russell Crowe, can, like, break the encryption or something like that? It's Mission Impossible. Yeah, I know. There's always, like, these kind of, like, whatever. So my favorite thing that happened recently was, like, Nome Brown. Yeah, that was good. We've all said stupid stuff on podcasts. I've already said stupid stuff. Oh, you didn't like this one? No, it was great. No, no. I thought it was fun.

29:24Erik Torenberg:No, it was fantastic. No, no, no. I'm setting it up. Okay, okay, okay. So, so, so, no, Brown was like, listen, you know, uh, uh, you don't know what a super intelligence could do. It could maybe use like the heat of a, uh, CPU to exfiltrate itself to another computer, which this brought me back to my, I'm like, I'm very, now I'm very comfortable. Great. Now I can have endless pointless discussions on this. But what is interesting is you basically have the X-Risk people saying something you felt was plausible. And then you have like the security people having this kind of endless discussion. And so I think at some level now these communities are being bridged.

29:57which was like, you know, I actually think that was a very reasonable thing for Rome Brown to say. I think you can pick holes in it, but we all say weird stuff on podcasts. I actually think actual risk is real. I mean, I worked in secure computing environments that were highly classified. The covert channels were unbelievable. Oh, yeah, yeah. So these are very real comments, but, like, we actually have a real discourse. And it was the first time I saw really hardcore systems people having pretty, like, I would say, constructive. Well, they were calculating the bit rate. Yeah, it was great, but it was a constructive discussion.

30:27And you had the typical extras people were engaging. Of course, it was Twitter, so there was a lot of name-calling, this and that. But it was actually, I felt like a real discussion for the first time. So I hope we see more of this. I hope we see more cyber-related things. I think the lab should talk more about it. I think it'll engage the community. And I think once that happens, we can actually... Well, yeah, Greg's been out there a lot more on this topic, which has been good. But I think the takeaway is Noam Brown should be doing more brainstorms on podcasts. I thought it was great. It was just, it opened people's eyes to the fact that actually there's a lot of risks in security that most people don't understand.

31:03And so that means that there's more stuff that, like you can't make baseline risk, you know, like how is your authentication layer work? You can't just wave your hand and say that should go away and then bring up, oh, but, you know, space aliens can invade. And that's a little bit of what was going on that I felt uncomfortable with. But like, what do you mean? Like, it was sort of like, but you know, there's also this risk. is how I view that. Of the heat thing. But at least we're now in a domain where comfortable, like I can talk about entropy, and we can actually have a concrete discussion that's not, oh, well, it's super powerful.

31:35At least we've reduced it to the laws of physics and the laws of systems. And most people did, I would say most people had no idea that that kind of risk was real. I mean, like when I'm walking around the Pershing Missiles, I actually had to test the graphics cards and PCs because a DoD requirement is that the screen memory not be sustained when you pull the power. But not for zero time. Like, the minute that the power went off, the memory image had to go. And if there was, like, a three-second delay, that could be read. Oh, see, we had people that came into our offices and would literally measure the distance of the monitors to each other because of tempest attacks.

32:10Oh, yeah, yeah. Which is 100 % a way to use electromagnetic radiation to leak information. I've seen the same thing with spread spectrum from the BIOS. I've seen it from audio speakers. Like, we had to remove the speakers out because it's a very high-batter channel. Our building had just Muzak speakers aimed at the windows. Yeah. Just to produce interference. You need to go run safety at one of these labs. This is like, I've never seen you more excited than heat-based communication. Can I tell you the craziest comfort channel I've ever seen? So it turns, remember the old CRT? I know, this is like one of my favorite.

32:42I'm glad that someone's older than me. Not really, but acting it. Remember the old CRT? So it turns out, let's say it's night, and you're using a CRT terminal in your room. the lightest thing in the room is actually the pixel that the raster beam is on. So most people think it's like the glow of the monitor, but it's actually that given pixel. So somebody figured out that, like, let's say you're in, like, you know, a hotel room and you're on your computer. If you have something that can sample the color of the window, you can reconstruct the screen. Oh, that's crazy. You just do it at the same hertz that the raster beam is moving.

33:13And somebody else figured out if you can subvert three pixels, you can use those, you know, because it just looks like bad pixels, you can use those to basically send a message. So you could literally sit out in whatever, like a program. Do your own SOS. You sample the message, and it is a relatively high bandwidth one-way communication. And so what Nobrand was saying, like maybe heat is not the way to do it, but that level of sophistication is actually real. That's a thing. When I was at the missile factory, I had to lock my keyboard up. That's really an impolite word, but that's what we call it.

33:42I had to lock my keyboard up at night because they didn't want the custodians who didn't have clearance walking by and just noticing which keys were dirtier or cleaner. Oh, yeah. And I once left it out, and there's like a note from security, you know, telling me to report to security and pick up my keyboard. Like the guard who walked the hallways just took the keyboard that night off my machine. And that's like baseline. I was not clear. I was just that sensitive. You know, I was like nothing. I was an intern. The big problem with this conversation is now this is all in the training data of every AI model in the future.

34:13But that's also the threat. The threat model for these things is always you've got a trusted side and an untrusted side. NIST has 500-page manuals. Okay, I got it. This is already out there. And the untrusted side, you assume basically an Oracle that can do and know everything. And then the question is, can you get information off the trusted side? Which, by the way, is what Noam was saying, which, again, is actually quite such a question. Which I do think brings up a super interesting point, which I'm going to bridge, which is just that I think the thing that people really aren't wrapping their heads around and are using the language that's really confusing is just that what AI can do is it can try all of those things in a very short time.

34:51And it doesn't get tired. It doesn't get bored. And, you know, and so, but the interesting thing about it is there's a whole layer of security that you now have to go look at every single API, every single service you're running internally on your network as like, you know, like nobody thinks that their internal GitHub or their internal Slack or internal finance expense tool is vulnerable to a denial-of-service attack. But swarms literally look like a denial-of-service attack. And so now we need a whole layer internally that just is tracking way more about what authentications are being done, what APIs are being done.

35:25But that's just like, now it's just going to be basic. And all the off-sec people that are old are like mailing me like, why are we explaining this to everybody? This is so basic because nobody did it internally. Well, you didn't have to worry about it for your people. And that's the like. But now your person is just a piece of software. Yeah, or like unlimited. Has a credit card. Yeah, I mean, you kind of got by with information security, like to some extent on the fact that most people will do the right thing 95 to 99 % of the time. Wait, the malicious employee is like one in 10 ,000. Yeah, yeah, yeah.

36:02So like, so all of these systems are basically open to whoever wants to access them or like one tap on the shoulder and then you have access. And agent swarms completely flip that because these are just roaming drones. Yes, but like, yeah, time is 10 ,000 and they will easily mistake like a good task for a bad one and vice versa. So the data security in our systems, this is going to be a huge upgrade moment. I actually, Martina, like I think that actually we're going to need a different access and security model going forward. Like, where are we going to go on that? Because the model we have is not granular enough, and it's not performance enough to handle this stuff.

36:44So I want to step back. I want to say, like, a meta point, which is I think this is how these conversations should go. We've identified a novel risk, which is, like, cybersecurity, which we actually have proof points, and now we're talking about solutions. I think the entire discourse around AI can be of that form, and the biggest mistake is that's not what it's been. Like, I think the entire industry and community is very happy to engage exactly like this. And I have something to say about exactly what you're asking, But I'm saying, like, I think this is where the conversation should be. We're known for having healthy conversations that everyone should learn from.

37:14So that's what we do. So here's the thing. I don't think there's a technical limitation here. Like, these threat models are very well understood and have been in the literature for a long time. I mean, like, the operating systems research, the MLS, the multilayer security research, has considered these sorts of things from an academic lens. The reason it hasn't been adopted is just tended to be a usability issue. It's just really hard to maintain and you didn't have to. So you could argue that AI solves the usability issue because it's AI is using it. So maybe now it's going to be a renaissance in operating systems and network and computer languages.

37:51And we should like go back to the old research and we should kind of start rebuilding systems that are secure by design. And oh, by the way, if we don't think that, you know, these things are safe to put out, We don't put them out until we have these systems built. And no, by the way, the AI is very smart, so they can help us build it. And so I think, again, like, computer always evolved. Like, you know how much of the stack we had to change for the internet? Everything. Right, tell me about it. And you know how vulnerable everything was? Like, we could be in one of those moments like, oh, shit, we got to rethink everything.

38:22And that's fine, we've done that before. But I think that's a conversation we should have. So I agree, it's time to think about evolving these things. Well, like, look at, like, you mentioned earlier, like booting a PC and getting a virus in 30 seconds or whatever. So if you look at how you take an iPhone out of the box, which a lot of people are doing this week, what happens is the whole network is basically shut down except for getting the latest version of the operating system. Because even though it was pressed six weeks ago, some zero-day thing has been discovered since. And so actually the whole out-of-box process now involves first step, doing an update where the device can't do anything else and it can never do anything else until it's updated.

39:04That's like, there are all these benign, things that are completely benign that we turned off in all of this desktop software of the era that used to be good things. Like it was having a macro for Word so you could build automatic citations or something. It was like the super cool thing until it became a virus. We had a thing where you could put a CD in and it would just arbitrarily run a program. And so then what somebody did was like, oh, well, I'm going to burn a clone of that CD and replace the program with my virus, but it's going to look like the thing that's supposed to run and it's just collecting all this stuff and being evil.

39:34Yeah, I don't. Then we disabled that. And so one of the things that's happening right now is there's a whole bunch of stuff that happens on your own boxcorp network that actually is going to have to just change the standard procedure. Two-factor-of, five years ago, was not standard in most places. Your whole SaaS world, I remember in 2015 or so when you would talk to a new company about their, oh, we're going to do enterprise pricing or whatever. And then they realized the first thing they had to do would go do Okta integration or Google Auth because they could not have their own directory of how to manage it.

40:13And then that just became a thing. And there's not a SaaS program anywhere that doesn't just launch with managed authentication. And so there's just so many things that need to happen before you're even software now. Yeah, well, yeah, we're, I mean, there's probably every layer of the stack has to evolve a bit in this. Like even the like lack of granular nature of like, you know, you have these modes of like the agent will either ask you every single time, you know, if you want to, you know, give it permission to do something or the exact opposite of like it can just like delete your entire computer.

40:47And like our OS probably wasn't built for the right level of granular set of tools you want to give the agent. we've kind of done a lot of work in this space because obviously like, you know do you want to give an agent your entire file system? Probably not. Maybe in some cases you do but oftentimes you want to have granular controls of like in this folder you can do read-write and in that folder you can only do read and so how do you kind of make this all intuitive for the user? So it's very difficult and so there's going to like What he just said is a very deep comment I know, exactly, yeah which is basically the conclusion of 40 years Right.

41:23No, 100%. It's like you actually can't make it. But maybe with AI, you actually can. Like, maybe there is like... Well, I actually think this is, if you ask me, like, walking in, like, what I wrote down on my note, like, was my biggest fear is that Europe decides that GDPR was the best thing ever. Uh-huh. And they're going to just GDPR AI. Yeah. And AI will be fine. It'll have one prompt for when text gets emitted that just says, this vendor is emitting text and it's probably wrong. Yes or no. That's going to be because you can't really, at least in North America, you're not going to send your speech.

41:56In Europe, they still will. And they'll have filters and keywords and blog lists. But then on any verb, anytime that an agent or, you know, a background agent or a frontline agent touches a third party product, I'm really worried that they're just going to say, we need a GDPR prompt on that. And it's going to be every single write or every single non-lookup becomes like a safety warning, like the airbag thing in your car. And the regulators love that because it's a liability assignment. And so it has this sort of legal precedent. And I really worry that that's actually the middle ground where we're going to end up.

42:40And unfortunately, because the U.S. about 15 years ago stopped leading in tech antitrust, the problem is that Europe is going to lead with that because they have nothing to lose. And so I don't want to be sad and down about it, but I just can't get out of my head that they love prompts. I mean, look, I have to put in that browser - You're being on prompts. No, they, because it's the same way. Look, get into a new car, which I haven't done in years. but you know like you're putting stickers off you know you have all of these things yes and and someone thinks that that was success yeah and it's like has anybody ever read like what is this if you're if there's a baby in this seat and it's like well that's relevant for some people some of the time but it's this it's the entire fabric it's attached to the seat and and they love that that is a very particular thing that they just love and so i would i would say if i were an ai now The one thing I would be trying to avoid, and look, we added it.

43:40Okay, FINRA for AI, we'll create that standard. I mean, look, we had to put this, when the internet was new, the big thing was to download a program and run it. And of course, if your machine is running in administrator mode, it was download a virus and take all your problems forever. And so with Windows XP, which was in 2000, we added this thing that prompted you and stopped. Like literally, your machine stopped, user account control. and like it was absolute assault. And we also put it in Word. The stupid little macro to help you write your thesis also came with a warning. Every time you opened your thesis, say this has my...

44:14Everybody would just click. And so you end up in this world where just like with GDPR, everybody is numb. Yeah. And so then they're like, well, it needs to be bigger. Although Mac, I mean, Mac kind of has... Nobody downloads software. That's the thing. It's a very usage pattern on Mac. I wish... Sorry. I don't have like 10 applications on my Mac, but... Well, that's 10 and then you're done. Yeah, yeah, yeah. But a lot of people still do all this stuff. Imagine if instead it was every time you go to a new website. Yes. Which you do now. No, that would be bad. I mean, to start from here, but to bring it kind of back to the macro point, like I wish this was the discussion we were having, which is like, I feel like we've dealt with a lot of these problems.

44:55I feel like when we talk about philosophical X-risk, we're not solving these very pragmatic problems. I actually think this is actually a constructive conversation to have. Maybe prompts would help. I don't know. And I think part of the problem is, people don't remember how unfettered access was and how bad it was and just how relatively benign that ended up being. I even remember, this is at Stanford during our PhD, I remember the oscilloscope was kind of janky. I'm like, what's going on with this oscilloscope? It's a little slow. And I was measuring the network traffic, and it was like, more network traffic than you would expect.

45:27And I'm like, why is there network traffic? I didn't even know the thing had a TCP stack. And somebody broke in, because there's an old version of Windows CE, and was running a porn server. You know, and so like... I had no idea that. It used to be... Just for the record, nothing. It used to be the case that like anytime you turned over a stone, somebody had broke into something. And like, was it this like worst case malicious whatever? It was actually very rarely, even though the capability was there. And so if we could just somehow tone down the rhetoric and put it in context and you still have computer systems, and yes, there's very serious stuff, and people have definitely died because networks have gone down.

46:04There's been real issues. But like, and then can we just quibble about GDPR? That would be amazing. But the problem is like, that's not the discussion. It's not about GDPR and prompts. It's about species extinction. And philosophy. And philosophy and irrefutable things. And it's just not perfect. It's also very soon. Like, I think that that's such a great point. And I think you hear people now talk about we regulate airplanes and we regulate cars. and you forget, like, well, the first cars were at the turn of the century. And unsafe at any speed was in the mid-1960s. Yeah, totally. And, you know, people had been doing, selling pharmaceuticals during the gold rush and thalidomide happened, you know, 50 or 75 years later and not even in the U.S.

46:46And then there was the FDA. And, you know, people, the first pilots were flying, obviously, at the beginning of the 20th century. And it wasn't until the 1920s that you had to get a license to be a pilot and you literally showed up with your own plane and you got a certificate. It was literally no different than driver's ed is today. And then it was 20 more years until they had anything to do with airworthiness and looking at your plane, but it was minimal. And then it wasn't until way after World War II that they got involved in what you think of as the modern FAA. And so you're looking at 40 years of innovation and they were not moving slow.

47:23I mean, have you ever seen that video in black and white of all the different planes that crashed and everything? I mean, that was 20 years after the Wright brothers. And the FAA is incredibly effective. And, right, it's incredibly effective. It's the safest form of transportation, like, you know, and so I do think that this process... It's also the slowest, most difficult form of innovation. And so if you had started the FAA in 1910... Yeah. You'd never have... Well, I was listening to a Nick Bostrom podcast just a couple days ago. No, no, I... It was interesting, it was interesting. But he actually makes his point.

47:57If you regulate AI too early, you actually basically don't solve anything, and you still just kind of have the same risk ultimately, but you don't understand the systems well enough. You will the thing into being, but you haven't figured out how to control it. We don't have to figure out what it actually is yet. There are new things coming out all the time, and casting AI completely differently than we thought of just six or nine months ago. And I think that all the innovation that's going to happen at the application layer is going to cause things to move in and out of the models in different ways.

48:30And we thought up until last week, I think, that text prompts and text coming back was going to be the best way to interact with it. And then Jeff Shaff. And then Jeff Shaff. It's so good, too. So talk about that. Why you find it so remarkable, yeah. Well, okay, so the way I think about it is, so, okay, so LLMs were kind of text in, text out, right? They generate text. And they came from chat, right? It was to communicate with a human. And we spent the last few years trying to take this thing that spits out text and cram it into a traditional program, right? But traditional programs don't really speak text, right?

49:05And so then you end up doing this janky thing where you're like, in the prompt, you're like, here's the schema. Here's the schema. But the thing is generating text and it kind of ignores it. And it's just been super janky. And so what Jeff basically said is, listen, you know, generating the text on the outside is a very expensive thing. but it's also kind of, you know, it's more complicated than you need. So why don't we, we'll read text and we'll have all of that kind of knowledge to read the text. But then rather than generating text, which is very expensive, we will just, if you give us a set of options, we'll choose the best option.

49:36We can do it incredibly, we can do it incredibly fast, incredibly cheaply, but also we can do it with much more accuracy because we can train just for this. And so for all of the use cases that are not talking to like a chat bot, but are actually trying to put it in traditional software, this is a great fit. And so this has probably been the fastest adoption of an AI model since chat GPT. It's just been remarkable because we're all primed for this. I just want to pile on this one because I can't tell you how much I love seeing this exact form of innovation. And because what it does is it does the thing that's bugged me from the very beginning, which is there's been no user study ever that shows like interacting with the computer using full natural language is efficient.

50:19It's like literally always the least efficient way. And it's very simple. And it's just like, ask yourself, how many people are really, really good at asking questions? And immediately, that's like less than half the people can ask a good question in a meeting. And then how often do you look at the answer and get really frustrated before it's finished, but you have to pay all this money to watch the seven paragraphs come out and then apologize that it's only a little bit. And so that's one, like having a different model. And then the other, of course, is my favorite, which is the output of it is designed for probabilistic programming.

50:49And so instead of saying, like, is this a customer service question, then route to customer service, otherwise route to general help desk or whatever. It's like, well, this is 80 % customer service. And that's exactly simulation. And it turns out there's like 50 years of computer science research in literally like probabilistic if statements. And so suddenly the coolest place to be in computer science is going to be in probabilistic programming, which was like all of computer science in the 1960s and 70s. So it was basically how do we, because all the computers started with doing math and it was all simulation.

51:25So it was like, let's launch the missile and hit that target. But it's windy. But wind isn't constant. So like, let's model the wind and decide where to put the thrusters in order to do the arc. And so most programming through like, say, 1970, before it got to accounting, was probably. And then we ruined everything. No, accounting, there's no probability in accounting. Right. But most programming was basically this modeling kind of thing. And so most programming language design was trying to figure out how to put probability into if statements or into while loops. Like, do this until something happens maybe most of the time.

52:03And so my first CS class, like the very second assignment or so, was a simulation about waiting online at a store. And I didn't know it at the time. I actually looked all this up when I was reading about Jeb, which was like the whole thing was my professor wrote the book. called the theory of simulation in like 1960. And I just didn't know that because it was kind of died by the 80s because it was all replaced by hyper. And so this notion of probabilistic and that slide deck you shared about the future, what's different about language models and stuff that you said was super good. Oh, Halper Fleck's one.

52:37It was phenomenal. It was a great deck. But the part that I felt was missing was that like, oh, wait, this is not all new. Like all of computer science was this probabilistic stuff. And so it's going to be very interesting to dust off all of that work, because it's exactly what's going on. Like, it's not an if statement now is if X percent, not if always. And so the way that Jeff worked is just to, like, you basically, it's a custom programming language almost, which is here's the prompt, come back with a percentage, and then you put that in the if statement. But the consequential thing is, like, finally we have a way to integrate these language models into a traditional software.

53:15And I think it's kind of funny because it, like, you ask the question, like, why haven't the labs done this, right? And it's kind of like a, like, not an indictment, but a reflection on how they think. Like, they're trying to create beings and beings speak. If you're trying to create God, God speaks to natural languages or whatever, where this is really about something that's for traditional software, which is kind of not the direction that they've been taken. But one of the reasons the uptick has been so dramatic is because a lot of us software people have been trying to integrate these models into software.

53:43It just hasn't. And so even before you get to the probabilistic, like if I want a language model to drive an if statement, like it's really hard today. Yeah, yeah. With this model, it makes it much, much, much easier. And then, of course, this could change the nature of software fundamentally to make it more stochastic. Well, I think, but absolutely. And I think that what's so cool is that it is happening outside the models because that's what I think is just going to happen, which is the center of innovation has just moved. And it's just, and now people need to, like, it turns out that the. The platform providers basically reach a point of critical mass where the innovation stops happening at the platform layer.

54:22And then, you know, Apple, there's this famous expression in the Apple community called Sherlocking, where Apple looks around and the things from the outside world become features and people complain. But that's sort of how the innovation works, because once you're a platform, you're overwhelmed. No matter how many people you add, you're overwhelmed with just keeping the thing running and compatibility and stuff like that. And so I think that this is the signal that now people have figured out that there's innovation to be done to the model, but outside the model.

54:53Erik Torenberg:Guys, thanks for coming. This is a great episode. Okay, great. Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts, and Spotify. Follow us on X at A16Z and subscribe to our Substack at a16z.substack.com. Thanks again for listening, and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. It should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.

55:37Erik Torenberg:Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com forward slash disclosures.