BONUS: Apple’s AI Era Starts Now? Meta’s Free AI Agent + OpenAI’s $1M Math Fight

35:48now levant was not working in his capacity at anthropic at the time this worked he is an researcher but this was a side project as a person uh now here's here's the drama as it kind of unfolded today where are i had all of these tweets open yeah so open ai just to put a little more context there open ai for their side said that it heard rumors of breakthroughs and then it pointed thousands of agents at the problem. Yes. In order to solve it. I think it was something on the order of 10 ,000 agents at once. And then they used tools, they used code execution, and they were able to, I think they did, they used 2.7 million agent messages and 130 billion tokens, just output tokens, to solve it.

36:38And then they found the proposed solution showing that a fluid can develop a singularity in finite time. Yes. Which means very little to me, just to be clear. It means shockingly little to me. Yeah. What we should know is this is a million-dollar math prize, basically. Yeah, like it is a – they compared it yesterday. When I say they, I don't mean like open AI, but compared this to a big blue Kasparov or deep blue Kasparov moment where like a human mathematician – I mean a human chess – the greatest human chess player lost to a supercomputer. I mean, tale as old as time, right? It's John Henry versus the train.

37:21Yep, exactly. And, uh, and here is, uh, I want to share this. This is another researcher at Anthropic. This is Shilto Douglas. And he's talking about Levent here. And he says that Levent is a really sweet guy with great intentions. I'm so happy for him that his year long collaboration worked out, but very sad that they didn't get to finish it in the way they want. Now, back here, he also goes on to say that for what it's worth, I think it's extremely unlikely that user data had any influence here. There is no way OpenAI would pull user transcripts for this or knowingly train on it in a way that would have influenced this.

38:00I think it's pretty important people don't run away with your user data isn't safe in codex because it surely is based on everything I can assume from the outside. that's an anthropic researcher saying yes pretty high up at anthropic too yeah pretty high up at anthropic and and i thought it shows a lot of character to to say that because he doesn't have to say that no no um but i think also the important thing there is that he's pointing out it's like he didn't say this but the subtext is that what incentive would they have to steal the data in order to solve it their incentive is to solve it with as little data as possible to prove how good their AI is.

38:37So that would totally defeat the purpose if they died. Yeah, it defeats the purpose entirely to cheat. Yeah, it's like, oh, then people are just going to say, well, like, AI is useless, and it's going to prove them right. It's like, that would be so dumb for them to do. Okay. Now, here's Noam Brown, who, if you don't know him, is a researcher at OpenAI. Noam Brown gave us the research model. Noam Brown gave us reasoning models with O1. He led the O1 team. Very sad to see him double down on the plagiarism application, accusation, and commends the employee from Anthropic for having stepped out and said, hey, it's not that.

39:21But we're going to go back over here because he shared some interesting points that I think are worth seeing in this story. La, la, la, la.

39:33Where's it at? uh yeah here yes this result costs millions of dollars but remember when openai announced oh three it cost a half million dollars to score 87.5 on arc agi one today astra does that for about 20 bucks uh so in 2025 last year it took us and gdm an enormous amount of compute to achieve IMO Gold for$26. Anyone with$20 a month chat GPT subscription could do it. So IMO Gold, for people who don't know, is International Math Olympics. Yes. We did a great interview with the guy who led the team that won it last year. His name is Ahmed Okishki. You should absolutely watch it. It didn't get near the attention we hoped it would.

40:19And it was such a cool story to listen to him talk about the passion for this event and what they put into it. Okay. Yeah, great interview. I'll link it in the chat. So from there, this is Sebastian Brubeck. Sebastian Brubeck is the guy at OpenAI. I won't say the guy. Let's say this. Works on AI at OpenAI, former VP of AI and distinguished scientist at Microsoft. He is the guy that when he understood these specific people were working on this thing, reached out and shares his text conversations here with Levent, which is really interesting. But his response to, and I should tell you that Tristan posted their results and made some lofty accusations on Monday that are absolutely his to-do, but that's where the question about training data and where it was coming from.

41:18Now, opening, I did say, like, we can never guarantee that, you know, that certain training data doesn't influence the quality of our models. But as far as going and cherry picking something for a specific product, project, no. You know, like, as far as training data goes, that's a little bit of a different animal. But here, Sebastian calls out, this screenshot is me reaching out to Levant to coordinate our releases. I hope it's clear the message that we came with. I'm going to start there. Here is the text message that he sent. Sorry to ask you on a Sunday. I wouldn't if it wasn't important. Could we talk?

41:57He's reaching out to Levant from Anthropic here. I'd like to be maximally open with you. I want to talk to you about coordinating the release of our concurrent discoveries. To be very clear, an internal model produced a proof of NS with very little human input. At this point, they think that these guys solved an S, which is the Navier's guess. But it is of the utmost importance to us to recognize your priority and that all academic accolades for this historic result go to you and Tristan. We are ready to share everything we have with you, including the human prompts and everything. Just let me know what you want to do.

42:36I think a call would be productive. And he responds by wanting to know what is the precise theorem you were claiming. and he says existence of forced blow-up in math language. Art of the third, T to the third, it's what it looks like. Yeah, it's a little small. So he reiterates about this here. He says, I never asked for Levant to be removed from authorship of his own work. I clearly said that. I was surprised to learn during the call with Tristan that they had only solved Euler and not Navier Stokes. Navier, I said it. Navier Stokes. After learning this, we brainstormed possible paths forward.

43:15One option was discussed that Tristan could be the lead author on a rewrite of OpenAI's proof. It is in that context that I said it would be simpler if Levant was not an Anthropic employee, because I felt it would be inappropriate for an Anthropic employee to author OpenAI's work. Importantly, though, it was admitted that internal Anthropic models had been used in their proof of Euler blow-up. so I therefore felt I couldn't consider Levant to be an independent academic. Another option I tried to propose is to offer access to our internal models so they could try to finish the proof and bridge the gap between Euler and NS.

43:53Again, I didn't know how to navigate this, giving OpenAIIP access to an Anthropic employee. Three, to reiterate plainly, as my text indicates, and as I said during our call, Opening Eye's intention was to do everything possible to celebrate their mathematical achievements and their heroic efforts that they made on Euler. In the call, I was met with a litany of slander, including direct threats, that if we were to announce Navier Stokes, he would immediately go to the press with a barrage of unfounded accusations. I refuted these accusations, but he replied, there's nothing you can do. I simply don't trust you.

44:28So I was confused why one would turn an incredible source of celebration of their own achievements into such bickering, which is when I said I didn't understand why one would risk their career. This is what came out as the comment from Tristan's paper. And genuinely at that moment, I was trying to care for him and do a last-ditch attempt to get a chance to give them all the credit they deserve. I deeply apologize for my extremely poor choice of words. It is the opposite of what I was trying to convey. I should say that I also retracted them on the spot right away. Overall, on a personal level, it was difficult to have these conversations.

45:04Levent refused to attend any of the meetings despite my repeated asking. As Schultel-Douglas said, there will need to be coordination between Anthropic and OpenAI in the future. I felt I was doing a proxy negotiation with Anthropic while the Anthropic employee refused to directly participate. Okay, so here's where I want to give you time to react to that, but I just want to provide important context. Please do. I get everything that they're saying. I think where this comes across as disingenuous to me, like from my opinion, is that they heard that OpenAI or Anthropic was working on this or some researcher was working on this with Anthropic.

45:43I know it wasn't in their official capacity. And so they're like, bet, let me try and see if I can beat it first. And then they come back to the table and say, hey, we beat it. Ha ha ha. Oh, you want to join us? Like, it just feels very, like, that's where I don't like it. is like, dude, you guys weren't even thinking about this until you heard they were doing it. Then you went off and did it. And then you tried to be like, oh, we can team up. Like, it just feels, that part feels scummy. And that's fair. And to be fair, at the time, OpenAI thought Anthropic had already done it. Not that we're working on it.

46:18Like what they said is, the rumor was that Anthropic's internal models had solved the Millennium problem. And they were like, I bet we could do that. and I think went after a similar, I don't know if they knew it was Navier Stokes. I don't know what the case was there. I just, I thought it was really interesting kind of reading through it. And the truth here is, you know, as the dust is settling, it's looking less and less like they went the same route. Like one went one way, one went the other, and it looks like it's probably not an issue. But it does create a really interesting problem, problem and that is that uh credit's getting muddy and and to be clear in the academic space credit getting muddy is a problem as old as dirt this is not a new thing like like academic wars over credit are nothing new uh however machines roll in the miz uh and nobody has said you know we're not looking to claim the prize we don't want the money you know um which is which is cool uh i don't even know that they were sure it would work right like uh you know but but they did hint that that it'll do many many more of these was was set too um so i think it's a really unfortunate situation of terrible timing i think two individuals were doing work and the rumor became that it was anthropic because one of the people was from anthropic and and then opening i was like we can do that and uh i i think it's an unfortunate collision of very smart people on on every side um i don't know enough about the math to be able to speak to the quality of the proof the value of each one uh you know so i don't want to i don't want to talk out my tail i'd rather uh leave the math to the to the math people but uh story and i think uh i think it's very right that there need to be long deliberate and unhurried i think is the phrase schulte used discussions about how to navigate some of this moving forward and it may be that they ought to work together on some of these is the truth of the matter and probably should you know uh different models different technologies different trainings i mean you know imagine what the two of those could do together tackling a problem from simultaneous angles talking to each other like i feel like there's a lot to be yeah totally like if you use a mixture of experts kind of approach where you have, you know, or, or, uh, uh, you know, uh, uh, drafter and a reviewer.

49:04Um, you know, there, there's a lot of ways in which these companies have combined these techniques to have multiple models, either like work together to solve something like multiple different models from different providers or critique each other's work. Yeah. I mean, I think that would be really cool. I guess the question is where does the business begin and end and where does the research and the, uh, the collaboration. Yeah, because these businesses are full of academic researchers. Yeah. That's the truth of the matter. We're dealing with businesses who are made up of half of the academic research minds in the country in this field.

49:39And so many people are leaving academia to go to these labs, as they're called, and do important work in this space, like either contributing their expertise to help train models or actually working with the models to solve some of these problems. So it's, I mean, yeah, it's interesting. I guess it would be like it would be the equivalent of a joint like a what do you call those a joint operation joint structure. Yeah. Yeah. Yeah. Like I think I think there needs to be an area where the companies join together for science more for science for math for the part that betters humanity. I would like to see them play well together.

50:23I mean absolutely do the competitive thing. If the only thing that comes out of competition is that we solve a Fields Medal problem, here's the problem with the drama end of this, is that it completely hides the fact that a Fields Medal problem was solved by an AI model. That's a big deal. That's the biggest deal that has happened in AI and math. And like a generational caliber thing. and all it is is a sign that it's going to happen more that's true and that it's going to continue happening and I think I think that's important I think that's amazing I hope they settle their differences I have a lot of respect for the researchers at both of these companies and the individual mathematicians like Tristan who don't necessarily work for one of these labs.

51:26And I feel like there's a really unfortunate thing when the problem you've spent your life's work on, you make a discovery that is Fields Metal caliber and immediately an AI surpasses you before the ink is dry on your paper. That's really unfortunate, but that is the nature of technology and it is a thing that happens. actually this is something that we have to deal with because this is going to happen more and more and more you come up with a really great business idea or or you know artists have been dealing with this for years where their style is getting stolen by um ai models or people using models to imitate them like or other artists yeah i mean yeah i mean it goes it goes further back for sure but um i guess the question is you know when you think of this concept of getting scooped what do we what sections of society or culture do we carve out for humans only and what sections of you know science and technology do we allow these AI agents to come in and get involved in because I can totally see a world where you know every time you have a good idea within you know a couple minutes it's imitated and someone someone with more token budget than you can come out and imitate it and roll it out at scale, and then all of a sudden you're sunk because you don't have the same billions of dollars that they do.

52:51This is just the science version of that. But in any scenario, that's not going to feel good. What's the role of your ego and what you care about versus a huge issue like this, like solving cancer? Obviously, everyone who's working on the problem wants to be the one to solve it. Otherwise, they wouldn't be working on it. But who wants to be the one to solve it is irrelevant. if someone solves it sooner. I feel like it's unfortunate. I feel like the work they've done along the way and published should absolutely go into that final paper. Like, you know, I mean, the fact is, that's why there's an index in these things, is it's going through all of the studies whose discoveries led to this moment.

53:33And I think they get credit in that way. But, I mean, I don't think anything that benefits the whole of humanity in some way should be held off in order for someone to get their due. I agree with that. You know, when it comes to things like art, I think they just live separately. I think art is in the mind. I think art is what you think it is. And just like with music, I mean, there's a reason there are a gajillion genres and musicians. We don't all like the same thing. And I think if there's something that you find is cool and that brings you joy and happiness in your life and makes you feel inspired, that's art.

54:14Whether it's, you know, there's plenty of abstract art you would almost argue is borderline art in some cases too. It's just tough when art, because you could say the researcher, their art is doing this research. Yeah, you could. That's their passion, that's their craft. What is and isn't art? Yeah, right? Right. But it's really hard when, I find this with anything, when art and commerce are intertwined. Because think about it, if this researcher got the credit that they feel that they deserve by solving the part of the problem that they solved, that would enable so much more opportunities for them in their career as well.

54:55It would. And then now who knows what's going to happen. Maybe the reputation will be a bit tarnished from this because there was accusations and lack of trust and maybe other people aren't going to want to work with them. And the sad part is, because they took a different direction, what happened is they went different directions. Opening eye went farther. This is still relevant and still has more headway ahead of it. But would you put your life's work into it now that someone found the way into the maze? And the problem is, like, a cabinet maker is doing art. Have you ever seen beautiful cabinets?

55:30you ever seen a beautiful home that guys these people put all this work into like art everything is art in some way and and i don't know somewhere you'd have to draw a line like is are we talking about the difference in in in a painting that brings about some sort of emotion in you music or are we talking about a teacher who really knows how to work with difficult children kids who are struggling you know i mean in that case i mean i mean a teacher's doing something that's beautiful there too. And I mean, art is beauty. And that comes in a lot of forms and a lot of shapes. And I'm really hesitant to draw lines in the sand around it is what I'm thinking.

56:10That's fair. But would you say then that technology and scientific breakthroughs in research is completely different? Because the one good thing about scientific research when it's open is that everybody in the research community can benefit from it and apply those learnings. It's like a multiplier on intelligence where everybody learns the same information once that research is published, and we all benefit from it in the ways that we apply it to solve important problems. So would you say that in that category in particular, we're going to take our egos aside and put them – or we should agree collectively that we're going to do that.

56:48We're going to put ego aside, and instead we're just going to focus on solving the problems. Or do you wind up with a group of people who decide, oh, that's AI science. I don't believe it. Oh, sure. Yeah, that's a whole other category, right? It's a can of worms I don't want to dive into. But it's also not beyond the realm of possibility. But what I would say is I think it's an unfortunate time if you've been working on a problem that is probably AI solvable. I think that's unfortunate for you. I think I don't know what that means. I think you should be the one leading the way to get it across the board with AI and get your name on it, is what I would argue.

57:32Like, if that's the thing, I think we can't stop scientific, biological problems like that. for reasons better than disagreements over credit, uh, and accusations flying in directions and things. I feel like that has to take precedence if it's a thing that is good for humanity and the understanding of our universe and other things. But I don't know. You raise a good question, Grant. It's an incentive problem. You've got to make sure that these incentives are aligned because we collaborate and we work with each other because it's mutually beneficial. So how then does the researcher mutually benefit from solving these problems?

58:24right? They spend their life solving 20 of them now maybe. What do you mean by that? Where did you get 20 from? I got 20 is a random number but what I'm thinking is because every problem unlocks new horizon math and new horizon science you could push so much farther than spending your life on one little niche problem. Perhaps there are people that that will be a really attractive possibility to. The idea that I could follow this stream for 20 years and bang out a problem every eight months or something or every Thursday, maybe not every Thursday, but I mean, it could be that, you know, you pick a pathway and your goal instead of solving one problems is following that path as far as you can take it over the course of the next 30 years of your career, you know.

59:19And I guess that just requires reframing your ego and your sense of self-worth from, you know, I'm going to do this one great thing and be known for it, to I am going to follow the ends of science. Like, my research, I'm going to follow my research to the ends of the earth kind of thing. So it's more about the work than it is about the prestige. Yeah, you're more Indiana Jones and a little less, you know... I don't know. I don't know. I'm trying to come up with something. Nothing's coming to me. This has been a fun segment. Probably a good time to switch topics then. Probably. This is a good time to switch topics.

59:56What's up next? What else is cool? The biggest thing that we have not talked about yet is Meta. So Meta Muse. Yeah. I downloaded it while we were getting in here. I haven't played with it yet. Do you have it? Can you pull it up? It's on my mobile phone. It's an app. Got it. Got it. Which is kind of extra sick. And I've just brought it up and started playing with it. I named my muse, my agent. I don't know what we call them yet. I'm going to figure that out. My personal agent's name is Frumpkin because why not? That's why. And the thing that's really cool about this is that they've really taken the personal agent approach here.

1:00:38Like, for starters, it's app first. Like, I think that this is immediately on a mobile surface like other meta products is really cool. it's very much designed for things like managing your inbox, drafting replies planning a trip, booking a restaurant watch for a price drop buy some cards better than Siri sorry I was cold

1:01:09a new kind of AI this week has been hectic help me stay off on top of school emails yeah so this is this is the demo video of it right this came out yesterday so wow this is muse working through things bam you're giving access to your credit card back it was looking at your email and it's like hey you need this list of school supplies here they are in a cart sorry you're basically it's just like on it in commerce yeah well and and this stuff on your phone like it looks like it's taking control of your phone for you which might answer that question we had just a few minutes ago about about chat gpt voice mode on mobile your flight is delayed that's a bum maybe i wonder if this is possible now that because of apple's um release the only question is i mean you know open ai and apple don't like each other meta and apple really don't like each other yeah yeah and you know what i know very little about meta and open ai's relationship if there is one i know zuck and musk are kind of tight at least somewhat they seem to text one another and talk occasionally uh but i don't know how the meta dynamic is with open ai with anthropic i mean the truth is there may not be one yet i don't know about anthropic anthropic is kind of in its own world when it comes to this type of stuff because they're basically competing with um you know enterprise companies but i would say that you know as far as open ai and and metago they're probably each other's number one threat yeah and um or no anthropic is open ai's number one threat but open ai is meta's number one threat yeah i think that's fair and i think that's so this is sort of their their version of grok bot as is grok pot which i would say that's what i was gonna say yeah i would say grok pot is is is very similar to this this was more more like an open claw a grok bot style system as you said kind of yeah yeah um what's interesting is it runs on muse secure vm which is a dedicated secure computer with its own browser They designed this like WhatsApp.

1:03:35Meta owns WhatsApp. WhatsApp is a fully encrypted messaging system. And this is designed the same way. It's intended to be extremely private. Secure and private. Corey, you're kind of quiet over there. Am I? I'm sorry. Let me speak up a little. Can you hear me now? Much better. Thank you. Okay. Yeah, I'm a little quiet today. I apologize. Yeah, so Meta owns WhatsApp. And the idea here is that this is the same idea, that it is secure and private. At least that's the story. To me, the story is it's a genuine competitor at a price you can't argue with. 100 million tokens per week free. They're a totally free account.

1:04:23But that actually comes with a huge thing to be aware of in the terms of... Big asterisk. Yes, yes, yes, yes. Let's see. Meta TOS. Let's see. Now you're a little bit too close. Their paid tiers aren't bad either. They've got a$20 and$100. Yeah. Oh, yeah. I was looking at this. Let's see. I couldn't find usage amounts for the$20 and$100, which is par for the course in the AI space. This one is 5x that one. Like, well, what is that one? Well, it's 3x the other one.

1:05:02Yeah. Where do they list that here? Let me pull it up. Okay. Muse checks. Private savings. We don't have the pricing stuff on here. Oh, hang on. I've got... Do you have that pulled up? I've got, like, three links. I added the pricing to my article, like, where I wrote about this this morning. You want me to bring it up? Yeah, why don't you bring that up, please? Thank you. Okay, one moment.

1:05:34incoming so here's the article I published this morning which was how to get started with metamuse from setup pricing down here it gets into one moment I went through kind of how to get set up I pulled stuff in from a bunch of other articles and kind of into one place where it would be easy enough to to work from let me pull this over yeah here we go uh free is zero dollars per month which is supposedly coming with uh up to 100 billion tokens per week on the free tier which is pretty crazy and makes me assume oh gosh that was wild sorry uh makes me assume that 20 significantly more than that and maximum more than that even though they haven't done it tokens or the units models process while working allowances translate neatly into a fixed number of tasks of course you know so there is an element of that now the early reaction here we go there's a really good interview if you haven't seen it journalist Alex Heath from sources interviewed Musk and he spent a little time with me as first and called it open claw for normies which I thought was really cool but that interview is really interesting it really kind of got into Meta's approach to data centers their approach to this agentic system and to models moving forward it's a good watch there's there's a lot of other resources in here from various other outlets but I do not have ah here it is read the privacy promises carefully credentials are stored separately from the main agent purchases require approval which is good their security documentation describes a separate system called Sentinel that controls permissions and communication outside of the agent so the idea being that that's not all inside the agent itself those protections matter though they also come with an important distinction launch version does not prevent meta from accessing data when necessary to operate support or secure the service um so that's what i think has some people kind of up in arms is there something else you saw grant

1:08:03you're muted okay all right sorry this is your side slow so um yeah what what i saw i'm having trouble finding again, but it had to do with what happens when you have the free version. And basically, it had to do with that basically they do, they can do anything they want with your data. That's what I was trying to find. I wonder if this is tied specifically to the money around the model because they do have what they call developer pricing around Muse 1.3 Spark, or Muse Spark 1.3. where for like 10 cents per million input, 20 cents per million output, if you did let them train on your data. So like they were making it dirt, dirt cheap.

1:08:54If you're working on things you don't care about. I mean, if you're not working on IP. And not everyone is working on something where that matters. Plenty are. But that's a personal decision if the cost outweighs to you. Ryan McBride. That's basically exactly what I wanted to bring up. Yeah. Is it, it's tied to that thing from the, from the model then that, that happened. It's a, which is fair. Yeah. And, but you know, I mean, it's, it's very much, Hey, you can decide, you know, the truth is the term sounded a lot like what you get when you have an account on Facebook and Instagram, like they can use your images, your posts in any marketing and all of that.

1:09:37They can do anything they want with them. And, uh, You know, I think that's why you do pay attention to the track record of certain companies when you decide to work with them. Ryan McBride here calls out that Muse has been accused of bench-maxing and the actual intelligence, like 30 % in real-world testing. What I have used of them has been reasonably good, not terrible. Now you're a little bit too close to the mic. It's peaking a little bit. Now I'm too loud. Sorry. Right. The joys of doing the live stream. That's right. Hey, that's what happens, man. That's what happens. That's a perfect distance.

1:10:13You're good there. OK, so I'm good here. The key is I've got I have too many monitors. And if I'm looking at a screen share, I'm looking over here and not talking directly. Yeah, that happens to me, too. That's a fair accusation about the bench maxing. The truth is, I don't know a lot about that. But I kind of assume they all do. And I really try to pay less attention to benchmarks. It's hard. I've reached a point where, for me, what I really want to know is if I plug a model into the things I do every day, can I do more? Can I do it better? Is my performance better? Is my speed faster? Am I doing it cheaper than I was yesterday?

1:10:57And if the answer is no to none of those three things, I don't care at all what it says on a benchmark chart anymore. I mean, it's a cool way. Yeah. From like a commercial perspective, I get it, where, you know, you do need a way to be like, okay, which model is right for this task? I mean, there are hundreds of models. What do we want to use? I mean, I can see where when you're making high-dollar decisions where, like, the fact is an extra two pennies is a million dollars a year or something. like you know in those instances i i see why you want to know and i do think benchmarks are good for showing trends like that they're all going in a certain direction and the pace with which it's generally going but as far as what a number says about one specific model i read them i just i don't let them determine how i feel about a model till i'm hands-on if i can help it yeah i agree with that i mean i think it's good for um let's let's call it well i guess setting the floor right so the floor is the best models score this well on these benchmarks so we know that if you are competitive with those models you have to be at least within the same range on those benchmarks otherwise you know it's not it's not competitive in the same way and that's assuming that all models, you know, respond the same way to the same tasks across these benchmarks.

1:12:27It's useful in terms of setting floor, but as far as your own work goes, yeah, you got to just try it yourself. Grant, I want to skip one in our notes that I'm not as familiar with. Let's go for it because I didn't get the chance to do the research on the way I wanted, but unless you are familiar. here wait is this anthropic says there's a 10 chance that ai could kill all humans yes i let's just look at it the guy who left and uh and it's getting some play today but i have not read enough that i i felt comfortable okay let's let's look at this translate to help with that we're gonna do it buddy bring it up no no very very quickly very quickly we'll all read it together we'll all learn together so jacob coxson i don't know if i'm pronouncing that correctly but That's the guy.

1:13:16He left Anthropic a day or two ago. And he said, he gave this whole thread here. He said, I resigned from Anthropic today. I spent the last three years doing pre-training research at both OpenAI and Anthropic. Neither company is acting responsibly. Which is funny because Anthropic's main line is usually like, we're the responsible ones, OpenAI. Well, they never say OpenAI by name, but they say other people are not responsible. And currently only one of them is paused. that's right or frontier rl go ahead yeah they are racing straight to self-improving super intelligence and gambling with our lives more thoughts below and then he goes into it do not underestimate the power of this technology these will soon be superhuman systems that can hack anything revolution revolutionize any field overnight and acquire real power and resources we have all witnessed the progress in each of these domains and progress is not slowing the people building it i earnestly believe that it could kill us all by the end of the decade This is not a marketing stunt.

1:14:13If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible, but I hear the same people express fear privately. No other human activity possesses this level of danger. The common response is, if they truly believe this, then why are they still building it? At OpenAI, many have not deeply internalized the civilizational stakes. Ananthropic, the stakes are well understood, but they are locked in a race to get there first. They believe no one else will act responsibly, so they must do it themselves despite the risk. classic mistake. One doesn't believe it, the other one believes it and doesn't care.

1:14:45Pretty much. Or they think that they can be the responsible stewards, which I think that's like a classic, you know, if literature has taught us anything, you know, that it doesn't work that way, sorry. But, you know, it's a tactic. Accepting this race and entering the end game is a hubristic gamble that should not be launched from a private company, Slack. Attempting to speed run alignment should require extraordinary confidence that there are no better trajectories available. I am optimistic about the potential for coordination. Warning shots like the hugging face attack have made pacing agreements between US labs more viable.

1:15:19I do not feel like we're on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities. If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a super intelligent RL run without a rigorous understanding of its mind should you put your head down because it's happening anyway or take this moment to call for different conditions and this is one of the most liked posts i've seen on x in a while in ai land right so a lot of people agree with this it's fascinating uh it is uh and and i can't say whether he's right or wrong uh evan's comment here is interesting that we do earnestly believe AI could kill all humans.

1:16:10I personally think it's greater than 10 % within the next decade. I believe Anthropik is trying its best. But we do solve alignment for superintelligence and are not clearly on track to. Here's what I struggle with. My gut says I should trust the people building this to know. On the flip side, a number of people building this have gone a little, a little off the deep end a time or two. Like, we've had a number of researchers who came out, and, I mean, even three years ago, a researcher from Google, if you'll recall, who pretty quickly in the race was, like, the first guy to be like, whoa, we're going to kill us all.

1:16:54And I never know what to make of it. Like, I do believe you should read it. You should know that there are people who have this opinion and believe this. I would not like it does not change what I think that I still believe that the risk is probably the reward is probably worth the risk and that I have real concerns that we as a society are on a bit of a collision path already in a variety of ways I don't even mean that as a political statement just just everything geopolitical is hotter than it's ever been And it just keeps getting hotter, it seems like. And the people are more divisive. So I have really serious concerns about where we're headed regardless of this as a global society.

1:17:44Again, not meaning that as a political statement. Just a global, I'm a human, I live on this planet, and it worries me kind of statement. For sure, yeah. That makes sense. And I don't know. like the other part of the population believes it's just autocorrect yeah i think people are waking up to it i think the the the uh rage against the data center machines uh people were the autocorrect people like six months ago or three months ago and then they realized like oh this is more than autocorrect and now they're like not into it um but i think so i think that we need to be very specific with what we're specifically calling out to stop and my personal take is we should stop scaling large language models.

1:18:33Large language models are not safe by design. Large language models do not have value-based alignment built into the way that they are wired, and they need that before we can roll them out safely into a super intelligent recursive self-improvement loop. This is my personal stance from where I'm at with my level with my level of watching this industry. Like I'm not a researcher. Like I don't know the ins and outs of everything. But from what I've observed, this is the key issue. The key issue is that architecture is wrong. Yes, it can do amazing things, but it's wrong. And we need to take it back to the drawing board and figure out something that we actually do understand and that we actually can roll out safely.

1:19:16That's my opinion. So let's be very specific with what we're calling for. This is not going to shock you. but I have a different opinion go for it let's hear it that's what makes this fun and that we can have different opinions and have a good conversation I I don't believe that ceasing scaling is either the answer or a realistic possibility I genuinely believe that When we say ceasing scaling, in reality, we're talking about OpenAI and Anthropik. We would like to say all American companies, but in reality, we're talking about those two, probably Musk, maybe Meta, others like that. We're not talking about Alibaba.

1:20:09We're not talking about Kimmy. We're not talking about any of those others because the odds that these countries will work together for that is like zero, in my opinion. I just don't see a scenario where, and even if they did, I don't believe either country would trust the other enough to not force it to be happening then behind closed doors instead of out in public. Yeah, I think there's a real prisoner's dilemma to it. And I don't know what the answer to that is. I think it's important. There's an interview just dropped in the last few days with Sam Altman and, again, Heath, the journalist. And one of the things they talked about there is about pausing frontier pre-training on certain models when they recognize problems and the things that they've stopped, the things that they haven't.

1:21:07because like it's not stopping everything you know pausing is is stopping this this element of this you're not not creating new features for your product you're not not doing this you're not not doing that you're not just walking up and locking the doors on a company you know and uh it's a really good interview highly recommend listening to it if you have time later um and uh but one of the things he discusses is that this has happened a number of times the hugging face incident was the most severe the most public, most attention and definitely the most scary he absolutely didn't shy away from that but what he did say was that he felt this has happened a number of times where we've reached a point where it was like we need a safety breakthrough we need some type of we've got a gap that needs to be filled here and the researchers buckle down and make pushes and see what they can do there and maybe we pause this approach and try this approach for a little bit and that's a thing that is constantly in flux um now of course it's one of the companies doing it saying that so you know there is a grain of salt there i just think it's funny that the way that this is framed is recursive self-improvement and ipo like like okay we know what the goal is here actually the goal is just to make big shiny splashy public offering like the goal like we'll we'll deal with the consequences of actual recursive self-improvement later on yeah low key though the other thing that was interesting is is he pinned him down and and and altman said, we will build a humanoid robot, meaning OpenAI will build a humanoid.

1:23:01Wow. Like pinned him down on the question, made him answer it, and he said, yes. And they get into, will everybody have a humanoid? He's like, yeah, but that's a ways away. And it's just a good chat because I want to say he's interviewed another person or two there as well. Yeah, you just interviewed Meta, so we'll share that out. Same guy that did Meta. They're both excellent interviews. Highly recommend. He's a good follow. He's largely open-minded and pretty fair with these things, in my opinion. Paid partnership. I see that. I see that. Well, what I would say is he does have, like, three ads in that video.

1:23:43You see them at the bottom. Oh, yeah. I don't know if that. Yeah. That's fair. Paid partnership with Meta. it may just mean this content contains advertising. I would. Yeah, totally fair. Yeah. Before I went too hard on him. Uh, yeah. No. Yeah. He says the, the sponsors right here. Yeah. Okay. Okay. And, uh, I also liked the way he does that. I don't know if you saw my comment earlier, Grant. Um, no, missed it. It's, it's clean and, and not tacky, but, um, yeah, it's good stuff. He, uh, he's fair. Like, I mean, he comes into it with a critical eye, but I don't feel like he comes into it looking for a gotcha.

1:24:24If that makes sense. Like he's not afraid to admit when something's cool, but at the same time, you know, he's not afraid to ask the hard question. Oh, yeah. Sorry, this reminds me of something. We should add this to the chat about WeChat. Oh, yeah. Did you see this? This is really wild. So researchers built an AI-powered WeChat worm in one week. So California researchers use Advanced AI to create WeWorm, a zero-click exploit that spreads via WeChat calls on iOS and Android, seizing control to read messages, make calls, and infect contacts. They found the memory corruption flaw in WeChat's VOIP system with AI help in two days.

1:25:09And then they built the full worm in under a week, a task that took months. So this was reported to Tencent in July and fixed by late August. The demo underscores AI's double-edged role in speeding up both attacks and defense. So just truly wild, crazy story. Well, that bleeds in to one last story I want to make sure we hit today. Can I bring something up? Go ahead. All right. Do you remember this? This blog post. Yes, I do. I can't remember, a month ago or so. But everybody who's co-signed it, it's on the list over here. And I mean, everybody's there. And Anthropic, everyone. And it was this whole idea that we need collective action on cyber defense.

1:26:02Because things are about to get pretty real in that field. For sure. What's interesting here is that Microsoft just announced its September patch. And I'd like you to see this. This is from Ars Technica. Also great reading. So Microsoft's patch is a doozy. With a record 972 vulnerabilities fixed, 112 of them meeting the high critical severity threshold. That is... What is going on over there? Two months ago, they broke their record at 570 vulnerabilities. I'll tell you what's going on. They're using these latest, these hot AI models. They're using Mythos. They're using Astra and OpenAI cyber models and uncovering crises in their software, which is great.

1:26:59Wow. It's great that this is happening because that it's happening means these things are being fixed. before they're your problem. And I think it's good to know that. I don't know if you remember, Grant, you remember we had Mark on, Mark Stockley from Threat Down, Malwarebytes, last year? We need to get him back on. He's a good dude. But he talked, when I first met him, what sold me on Mark was he explained the patch gap. So this is going to be me explaining why you should be running your updates on your computer daily. previously it took and I'm not a cyber security professional but I did stay at a Holiday Inn Express last night so

1:27:45here's the deal previously a patch comes out hackers get a hold of that patch reverse engineer the vulnerability and then begin scanning the internet for computers that have not run this patch because now they have a way in. Previously, this problem, this process, this full circle, took four or six months. It was very slow. It was very meticulous. It was very manual. And what happened was it wasn't a rush to update your system because you do have a little fudge room in there. That is no longer the case because apparently, AI has closed the patch gap from many months to a couple of days in some instance even hours and what that means is that when a patch comes out a timer is starting on the amount of time before hackers are searching the system for searching the internet for systems that haven't run those updates uh it's not necessarily every update i mean the fact is some some have bigger problems than others.

1:28:59But when something happens, like we're pushing out 972 critical vulnerabilities, you should probably go update your Windows machine. And keep an eye on any other ones. Like, I've since meeting Mark, I pay a lot of attention to updates. I run updates on everything about every day.

1:29:21Yeah, that's wise. Ryan McBride in the chat said that's a super interesting concept. On the flip side, we're also seeing way more supply chain attacks. And I said, for real, one of the first things I do when installing any external libraries or third party packages is asking to check for any known supply chain attacks. And I use web research. I say, go out, make sure there's no, you know, documented supply chain attacks that have impacted this, you know, software package in the last, you know, 90 days, 120 days, like just trying to make sure that at least I can catch it if it's been documented.

1:29:55Who knows what's in there that hasn't been documented yet. I think after this kind of came out, people are a lot more sensitive about, you know, not just accepting any PR that comes in. Yeah, yeah. So at least like... Yeah, and I think increasingly so. And I think with time, we'll have more AI tools that are kind of watchdogging that in real time. Yeah. At a level we've not experienced before either. Yeah, it might make it safer than it's been in a long time as well. It might. It might. That would be awesome, too. That'd be awesome, too. I have two questions. Well, I have one question and one fact I want to bring up.

1:30:33Okay. Go for it, man. The fact that you brought up was that Microsoft has actually fixed over 2 ,760 vulnerabilities just this year. Oh, my God. 2 ,760, like total. That's just wild. When you think of just Windows, when you think this is a piece of software that's existed for 40 years, thousands and thousands of engineers and developers have laid hands on that, have written snippets of code that live inside that. Supposedly, it's a fascinating thing to go through Windows code that, like, you can walk through the history of engineering. That, like, you go through it, you see notes from these guys that were there in the 80s that they still left in the code.

1:31:16And, yeah, so, I mean, the fact is everything changes. The technology changes. It's not even necessarily mean something that was a problem before. It might not have been a problem that now is or is now only a problem because now you can hit it with AI at the speed of light or something. And cracking it actually is a risk where it maybe wasn't when, you know, it was a guy with a password logger. I also it also makes me wonder like how many of those vulnerabilities were created by using coding agents you know like that's like how how vulnerable like is the new code like I would I would love to know that how much of it is code that's been there from 40 years ago how much of it is code that was introduced in the last three years I think that will tell you you know how much we should be relying on coding agents um at least in the short term for you know yeah I guess like introduce they They might introduce more vulnerabilities than they help stitch up in some cases.

1:32:16Well, the hope is that as they do, they'll also invent new fixes and patch new holes as they go. And hopefully, you know, the good news is the good guys and the less ethical guys all got the same new tools. So they can both move faster than ever before. And my hope is that, you know, that balances us out a little. But maybe not. We'll see. The cybersecurity end is intimidating when you think of things like we've had a number of, like, water infrastructure attacks in the U.S. this year during the war with Iran and such. I don't know if there's a tie or not. I'm less familiar with those cases. But like those are the kind of things that I think need to be guarded against a little extra safely these days.

1:33:13Yeah. Good time to brush up on your cybersecurity. Yes, it is. Knowledge. Hey, we got a new podcast episode just dropping this afternoon. It is actually Grant and I. No guests this week. We had a little scheduling kerfuffle and thought we'd do something more like this, honestly. And it's a lot of fun. I hope you enjoy it and you'll check it out. We appreciate everyone that comes in and listens a lot. These are a lot of fun and they're a big help for us and hopefully for you. Final thoughts, Grant. What you got? Great quote. Funny joke. I was just sharing the link to that video that just came out.

1:33:55So you can check the chat. If you don't want the party to end, you want to hang out with us for another two hours, go watch that video. There you go. By the way, big live tomorrow. Yeah. Cassie Williams is coming from GitHub to teach GitHub to people who are brand new. We said, hey, in a world of vibe coders, GitHub is intimidating to people. What if we brought someone in? So we went and chased someone down. And she's gracious enough to come join us and talk all things GitHub and make sure Grant and I haven't demolished our entire profiles yet.

1:34:37Sorry for speaking over you there, Grant. That wasn't intentional. No, no, no. It's all good. It's all good. I was just looking to see if there's anything else big that just came out that we should mention before we wrap here. It looks like Anthropic has something, an alignment assessment of recent cybersecurity incidents. Take a look on that. We might be talking about that in tomorrow's newsletter. But, yeah. That's it for us for today. Yeah. All right. If you haven't yet, please take just a moment to like and subscribe. We really appreciate that you're being here. And subscribe helps and helps ensure we can keep doing these lives.

1:35:12Also, make sure you subscribe to the Neuron newsletter, theneuron.ai. Join 700 ,000 others who read it every morning. We'd love for you to be one of them. And on that note, that's all we have. So farewell for now, humans.