IM 854: Welcome to the Pitt - AI: A Brand or a Breakthrough?

34:49Yeah. Just a little control replace. And there's an interesting relationship there. From some viewpoints, for some people, they would think of expert systems as one approach within AI. But it also, in some ways, function as a rival brand, because AI by the 80s already had this taint of having over-promised and under-delivered for a long time, and expert systems sounded more technical and respectable. And maybe less ambitious to some degree, right? We're not trying to build a consciousness, just we want to answer some questions. Right. And you see this in many areas, including, for example, if you look at AI textbooks by the 80s, they are not discussing the Turing test.

35:29They are not making the claim that all this is about achieving human or superhuman intelligence they're making the claim that this is a respectable body of techniques that work that rely on knowledge that make computers go more effectively well there's so many great characters in this including uh marvin minsky and some other histories of ai that we've talked about on this show marvin minsky is painted a little bit as a villain as the guy who was so convinced that uh neural networks couldn't possibly work that he steered ai away from what was ultimately the technique we're using today do you do you see him as the villain in this i try and get some kind of historical distance there although one of the comments from the the reviewers for the press was that i'm taking my animus against modern day big tech and projecting it back to be too harsh on those guys in the past.

36:28I don't think I am. I am in many ways stressing that what they were doing is something quite different from what happened now. I mean, I think you have to deal historically with the fact that none of this worked. And I was trained in the early nineties as a computer science student. I took maybe five AI courses. I learned these techniques in the AI courses the same way I learned - A bridge solver, right? In Prologue. Yeah. Yeah. The same way I did graphics and I did databases and I did architecture. And you write these simple exercises with these toy problems. And you do the same thing in AI. But the difference is the techniques I learned in the AI classes couldn't scale up.

37:08They only ever worked on these incredibly small toy problems. And the techniques in the other classes were simplified versions of the techniques that really did work for technologies that existed in the world. So when you're talking about 20th century symbolic AI, I think you need to be clear up front. The things they were doing didn't work. They produced all kinds of byproducts in terms of interactive computing and technologies and parallel computing. So I'm not saying that the money was a bad investment, but it was the byproducts that went out in the world to be useful. they did not succeed in achieving their core goals.

37:50And I think that is clear enough at this point that you can get away a bit from thinking, well, this guy was a hero because he favored this good approach that I think the field should have adopted and this other guy is a villain because he did this. I mean, nothing any of them tried to do would have worked. And there are many reasons for that. I mean, there's the whole problem with the fundamental difficulty of taking a purely symbolic approach. There's tacit knowledge, all the things that the skeptics and the philosophers were complaining about all along. But there's also just the complete lack of computing power that's available.

38:30That's kind of the underlying thing of all of this is, they were trying to do parallel computing and time sharing, and it's all been solved. That was the bitter lesson, wasn't it? It's all been solved by just massive compute. Well, I mean, they did other things too, but if you look at the revival of neural nets, the place where neural nets were brought back and worked was Bell Labs, Jan LeCun in particular, with a system that was able to differentiate pretty reliably between the 10 digits and read zip codes and in a related application to read the routing numbers on checks that were written by hand.

39:11So by the 80s, and this wasn't just computing power. I mean, they also, you know, the whole thing, Hinton and his buddies and the backpropagation algorithm and, you know, various conceptual advances that underpinned the revival of neural networks in the 80s. But with the biggest computers that were available in the 80s, reliably distinguishing between 10 digits was pretty much all you could get. So it's not that there was some other path that could have been taken in the 60s, 70s, 80s to make AI work. I think just fundamentally, you have to accept that they had some ideas that maybe seems reasonable.

39:54I don't want to say that it was ridiculous to even explore them. They certainly, for example, showed that the forms of logic that you might learn in a philosophy class are going to help you solve problems and think more rationally just fundamentally don't work when you're trying to apply them on a large scale. And they had all kinds of advantages and byproducts that were produced along the way. But I think, again, we've just got enough historical distance now that I don't want to say, you know, say Minsky was bad and McCarthy was good or McCarthy was good and Minsky was bad. They were really smart people that had a bunch of interesting ideas to try and address a challenge that it's clear in hindsight was fundamentally insurmountable given the technology available.

40:46The capabilities. You do say that Minsky confessed to being, quote, the devil who killed interest in neural networks for a generation. But Seymour Papert was the guy who really pushed this notion. I like your more nuanced view that it isn't this kind of, we took a wrong turn and it went downhill. And then we finally, oh, it's machine learning. We got the right idea. It is a continuum. And it's a continuum that's informed by a lot of different ideas and capabilities that got us to where we are today. Now, you said something interesting. You're not a fan of big tech today? Yeah, I mean, that's in the opening.

41:25I don't think that puts me in a minority. No, it does not. You know, so I start out in the introduction with a snapshot of the world in 2024 and the AI hype there. And then at the end, I've just been drafting some extra material to put into the book with the revisions. So, you know, I'm trying to cover entiatification and, you know, the way that the enormous cost and resource needs of these systems I mean, that there's only a handful of big tech firms that can afford to develop them, which are underpinned by monopoly profits from doing all the bad things that, you know, you're all extremely familiar with.

42:07So I'm not a booster of modern AI or big tech, but I don't have any particularly original critiques for that. I think the originality that I have in the book is taking 20th century AI seriously and going pretty thoroughly through time about what the AI brand has meant over the decades since it was originated. So the last two chapters are the neural network revival, which I, you know, mostly what I know about that, to be honest, comes from podcasts and journalists and things like that. I'm not trying to explain to the world how ChatGPT works and so on, but I'm trying to know enough about that to say what is the same and what is different between these modern day AI branded technologies and the ones that were dominant in the 20th century.

42:55You mentioned that your book I think Jeff's trying to ask the question. We should let Jeff ask. I want to hear Jeff. Sorry Leo. I'll just write off what you just said.

43:11It's a weird general question. I'm going to ask this two parts. Do you think in the end the people involved in AI over the years and now would say that the use of the brand AI was beneficial or detrimental? To their careers? To the field, to their careers, to the society, whatever you like.

43:36It's an interesting question.

43:40I mean, Feigenbaum, for example, deliberately, I think, came up with the expert systems brand as an alternative to AI. Minsky and McCarthy, I think, through their whole lives, remained true believers, particularly McCarthy, I think, who came to see himself as something of dinosaur. I just was reading some archived emails from him that are preserved at a website called SailDart, which is taken from the backup tapes of the Stanford AI lab time-sharing system. So, it's a wonderful opportunity to be a bit of a fly on the wall. And it's clear that by the late 70s, he saw himself as working on basic AI and logic, which had fallen from favor for applications-oriented work and pretty much just in his mind plugging away doing the same kinds of things he'd been doing since the 50s.

44:36If you want to know what those people thought, the best source is probably the 50th anniversary reunion conference that they had at Dartmouth. And in those days, they're basically saying, yes, we still believe it's going to pay off in the end. We still believe in symbolic AI. We think it's going to be a great thing, but it's not going to to happen in our lifetimes. The thing that we thought was 10 years away is maybe 50 years away. So the ideas for where what we now call AGI would be accomplished, that was probably 2000, I think that was 2008, the reunion around that time. That's probably a low point for belief in the near-term potential for human-like intelligence.

45:16Let me ask you a follow-up, if I may, real quickly there. On the New Books Network, where I heard you talking about your next book, which is a wonderful wonky podcast network for academic books. You said that you could see a case for people wanting to flee the brand of AI and try to rename it again in the coming year or so. Yeah. Do you think that the language foretells the fate or reputation of the field of problems in it? How do you see that going? All right. So the context for this is the technologies that we now call AI are fundamentally around neural networks, mostly around generative AI, within that mostly around large language models.

46:09And those technologies, as they were being developed in the early 2000s, were not called AI because they'd been pushed out of the AI field. And I think also to an extent because they didn't want the baggage always that came with the AI brand. Those things were called machine learning, deep learning, pattern recognition. And something really dramatic has happened in the last 10, 12 years, that essentially the people who'd been taking this other technological approach that existed outside AI as a brand, had existed outside the elite AI labs, basically stormed the castle, raised their flag and said, we're AI now.

46:51When you say AI, this is what you mean, not the symbolic AI. And so the question is, why did the machine learning community seize the AI brand for itself after having kept a distance from it for many years. And I think the argument of that is fundamentally around the science fiction narrative associations of AI. So they reclaim the brand. I've got a quote in the book from an AI researcher who says, the exact day on which he became an AI researcher and not a machine learning researcher was when they went to the NIPS conference for the neural net stuff. And Mark Zuckerberg was there in the presidential suite having recently hired Jan LeCun and was offering people lots of money to come to his group at Facebook, which was called AI.

47:44And then obviously, the DeepMind people, I mean, Shane Legg really heavily promoted the concept of AGI and

47:57OpenAI, DeepMind, you know, those other guys really had a revival of this early AI sense that this was something that was going to produce human and then superhuman intelligence with the whole singularity thing in the near future. And that is also why so many trillions of dollars have been flowing into it. So I think they switched the name of the thing from the more technical wonkish machine learning to the attention grabbing AI very specifically because they wanted to make a number of claims that seem plausible to us because we've been conditioned by science fiction. So one of those is that AI is going to be this superhuman general purpose thing.

48:46Another one is that it's something that is going to happen really quickly. So if you look at science fiction stories like, say, Heinlein's Moon is a Harsh Mistress or the Terminator stories, AI just happens, right? One day a computer gets big enough and it suddenly becomes self-aware. And that obviously happens because the authors had no idea how you could make a self-aware computer. so they just well whatever it just happened okay deal with it so we're also primed to believe that ai is something that can just happen very quickly and unexpectedly without actually needing much work once you've got a big enough tech platform um and also of course in most stories where ai exists it's the most important thing in the world um and you get the whole doomer versus you know accelerationist thing because in some ai stories it's basically a metaphor for slavery and of Of course, the robots exist purely in order to be oppressed and rebel.

49:41And in other stories, it's basically the Pinocchio story of the boy that wants to become real, etc. But I think it's exactly the science fiction promises that are implicit in the brand of AI, which began in science, but during the AI winter really survived much more strongly in science fiction than it did in computer science, that led to them reappropriating the brand. So back to the prediction that I made. It seems that there is pretty much no margin for error in the promises that have been made for AI, right? I mean, every leader of every major AI company has got a personal timeline for achieving AGI.

50:22And the promise that the singularity is going to happen sometime soon after that, and we're all going to like techno heaven or techno hell. But either way, like the rapture is coming. And there's not much margin for error in that. And it seems to be much more like that AI is going to be like other technologies. It's going to be producing tangible tools that do some things really well. And I don't think those tools are going away. But the question is, it's bundling together all these different technologies like image recognition and text generation and autonomous vehicles into this one thing called AI and making these huge promises for it going to work.

51:12And I think if the overall superhuman intelligence thing doesn't pan out, or even if the technology works, but the business bubble bursts, because historically, even with something like the internet you know as you're aware the internet wasn't a flash in the pan thing the internet was really important but the dot-com stock bubble still burst so even in a scenario like that i think the brand will become tainted again like it did in the ai winter of the 90s and in the 90s for example if um the 90s is the period where continuous speech recognition really becomes an important thing. So you've got technologies like Dragon Naturally Speaking.

51:57And they don't call those things AI. They call them speech recognition because AI is out of fashion, but speech recognition is in. And very recently, actually, the company that bought the company that bought the company that bought Dragon was bought by Microsoft. It was Nuance, and it was rebranded back to being AI as Microsoft's big play in AI for healthcare, but speech recognition spent like 30 years not being AI. So I think we may get something similar that people come up with these much more specific brands for the technologies that actually work, but the benefits of being associated with this big science fiction narrative around AI fade when people are like, oh, it's been three years, like where's the stuff you promised us would be three years away the stock market bubble bursts you know Nvidia is no longer the world's biggest company etc etc and people try and distance themselves from that and go back to the idea of having much more specific brands associated with the texts that work right I mean brand wise I kind of like to say it works it works a bit like a fashion brand like Chanel right um so you know Chanel makes really good fragrance and they have good lipsticks they have the runway stuff and then they they brand extend into handbags and watches and so on and it's not like the things that are good about the chanel watch are the same things that are good about the number five fragrance but the brand like unites these disparate things and gives you this sense that they all you know are sharing qualities with each other in some more intangible kind of way.

53:43We've been talking to Thomas Hague. He's professor and chair of the history department at the University of Wisconsin-Milwaukee. Now, I'm confused about the title of the book. The galley that I have is different. Have you decided on a title yet? Yeah, so that's with the press. So, yeah, the one you've got says Artificial Intelligence colon The History of a Brand, which is a good title, except there's going to be a million books It's called Artificial Intelligence, colon. And if you only search on the main title, no one will find it. So we're thinking to use a title that I've been using for talks that I've given, which is the brand that wouldn't die, colon, A History of Artificial Intelligence.

54:22And then you've got a main title you can actually find. Brilliant. It'll be from the MIT press. Anyway, it's sometime soon, yes or no? Well, hopefully. It took them a while to get me there. Got to figure out that name first. Yeah, you'll find soon in academic publishing. Well, I mean, I still like to hope for by the end of 2026. That would depend on the press being willing to move a little bit faster than usual, but they do want to give it some trade distribution and, you know, a relatively high profile. So it's tricky because AI is moving so fast. It's hard to write that final chapter. Yes. What's the advantage of history, though, Leo?

55:03Well, OK. Yeah. And the earlier chapters, it's going to be - Those are written in stone. They're going to stay way more current. It's always a problem trying to come up to the present. But obviously, if I give someone a book on AI and it doesn't get to chat GPT, they're probably going to want their money back. So the current stuff needs to be there, but mostly just so that I can draw parallels between the earlier history versus claiming that I have the definitive, unique understanding of the modern day tech. Well, and it's just, it's just fascinating story. And it's so many interesting characters.

55:39And you draw some nice pictures. I didn't realize that Marvin Minsky was a comedian as well as a brilliant thinker, things like that. Thomas Haig, thank you so much for joining us. His website, Tom and Maria.com slash Tom has a lot of great stuff on it, including references to his earlier books, which were really important in the field still are any act in action is the definitive history of one of the very first modern computers. He's also the co author with Paul Ceruzzi of the new history of modern computing, which is also a modern classic. So if you're interested, I know most of you are are in this stuff, this is a great place to start.

56:20Yeah. What do you look at next Thomas? What's the next pursuit? Oh, well, after the AI book is out, I want to get back to actually what my dissertation was about, which is the history of computers in corporate management throughout the 20th century. So it's, you know, in a way the prehistory of big data. Yeah, I think about the episode in Mad Men when this Madison Avenue agency in the early 60s gets their first computer and it's on its own floor and they have the priesthood. And they're able to do things with that computer that you could probably do in about a 60th of a second on your phone today.

57:05But for them, it was a big revolution. Yeah, I think that's fascinating. And we've come so far so fast. I can't think of another technology that has made this kind of advancement in just a matter of a few decades, which also makes it quite interesting. And the cultural impact of it, which we're really seeing with AI. Thomas, thank you so much for joining us. I appreciate it. Thank you, Thomas. Thank you. Great to meet you. You too. And I feel honored that you have made it from your hospital bed. That is saying something. This is really an interview for the ages. Thank you, Thomas. have a great day thank you we'll continue with intelligent machines in a moment i need to ask so many questions oh man we'll get to the hospital bit your planet is now marked for death marvel studios the fantastic four first steps is now streaming on disney plus we will protect you as a family light them up johnny marvel's first family is certified fresh on rotten tomatoes that's fantastic and critics say it's one of the best superhero movies of all time.

58:11Wobble Studios, the Fantastic Four First Steps. Now streaming on Disney+. Rated PG-13. What time is it, Ben? It's Robert's time! Refreshing Wild Cherry Cola meets Smooth Cream.

58:28The treat you deserve. Pepsi, Wild Cherry, and Cream. Treat yourself. Yeah, let me do an ad. We're a little behind. Alright, Jeff, you You've got to hang on. You've got to hang with us for five more minutes. Can you make it another, you know? I think I can make it an hour, but that'll be better. Oh, we'll wrap it up in an hour. That's perfect. Okay. All right, we'll continue with Intelligent Machines in a moment. Our show today brought to you by Bitwarden, the leader in passwords, pass keys, and secret management. I tell you what, I couldn't live without Bitwarden. Bitwarden is consistently ranked number one in user satisfaction by G2 and software reviews.

59:08over 10 million users across 180 countries more than 50 000 businesses I think you know you should know if you watch this show or any of our shows that having a password manager is kind of fundamental to your personal security you you can't possibly remember you know hundreds of passwords or or store hundreds of pass keys you need something they'll do it something that'll do it with strong encryption ideally open source encryption so you can verify it's doing it right you need something because otherwise you're going to come up with easy passwords that you can remember probably going to reuse passwords now you're smart enough not to do that but I think many of if you're running a business many of your employees are probably doing those things you know better to and to do like reusing passwords that's why you got to have bit warden bit warden makes it easy for you to encourage your employees to do the right thing they will love bit warden and bit warden will help you for instance their new access intelligence for businesses organizations can use bit warden to detect weak reused or exposed credentials immediately guide remediation this is something Bitwarden does really well, even with individuals.

1:00:24I just was on a site and said, you know what? You need to replace that password. It helps you replace risky passwords with strong, unique ones, which closes probably the number one security gap in business today. Credentials, still probably the top cause of breaches. But with access intelligence from Bitwarden, they become visible, they become prioritized, and they are corrected before exploitation can occur. Bitwarden, though, something for everybody. If you are, and I know many of you are doing your home lab, you have a personal projects, you might be interested in Bitwarden Lite. Bitwarden Lite is a lightweight, self-hosted password manager built for those home labs, those personal projects for environments that want quick setup with minimal overhead.

1:01:08Maybe that's you. Bitwarden, of course, no matter where you're using it, at home, at work, is enhanced with real-time vault health alerts and those password coaching features that help users identify weak reused or exposed credentials and take immediate action to strengthen their security one of the ways bitwarden helps you is by helping you move off of the password manager built in your browser it's so easy right and oftentimes i think especially less sophisticated users they turn on the password manager in their browser and they kind of live there but it's not the most convenient it's not the most secure good news when you install bitwarden it'll say ah you want me to your passwords from chrome edge brave opera vivaldi i'll do it directly it directly imports the credentials from the browser into the encrypted vault without that separate intermediate plain text export which is much more secure it helps reduce exposure uh it's it's a really brilliant idea i wish everybody did this g2 winner 2025 reports that bitwarden continues to hold strong as number one in every enterprise category for six straight quarters.

1:02:15It's easy to move to Bitwarden. I did it. Steve Gibson did it a few years ago. Couldn't have been simpler. Within a few minutes, we're up and running with Bitwarden. I've never looked back. It is the way to protect yourself. And because it's open source, GPL licensed, you can review the code yourself if you want. It's on GitHub. It's also regularly audited by third-party experts, and they meet SOC 2, Type 2, GDPR, HIPAA, CCPA standards. They're ISO 27001-2002 certified, so it's absolutely secure. Get started today with Bitwarden's free trial of a Teams or Enterprise plan. Get started for free across all devices as an individual user, bitwarden.com slash twit, bitwarden.com slash twit.

1:03:02I couldn't recommend it more highly it's everybody should be using bitwarden i hope you will and make sure you use that address so they know you saw it here bitwarden.com twitter thank you bit warden okay okay you can ask uh you can ask all those questions now of mr uh jarvis don't tire him out oh that's cool now they give you a little readout okay some context for anybody who's just listening instead of looking jeff is in a hospital bed showing us his heart rate i will also say something about the lighting in the hospital bed i assume it's not set for podcasting something about it makes it look kind of ai generated no i thought he was in heaven for the first time i was going to say now the lights are off and it does look like you're on the verge of death Before, it was the sort of unnatural lighting.

1:03:55It feels heavenly. He's in a crowd. Or two. One or two. This is one. One. I'm saying one is the move. Jeff, we got all our light mode now. Have you spoken to any of the employees of the hospital about the fact that you're podcasting right now? Because I'd like to know from them how many people they've seen podcasting. I think your doctor would tell you to stop. I really do. I told the wonderful nurse. So I don't know if you can hear this beeping in the background. No. Oh, you can? Oh, good. It's my IV. So they're pumping tons of antibiotics into me. Oh, gosh. And the device is very persnickety.

1:04:39If it gets one bubble in line, and no, a bubble won't kill you, but it still stops, and then it beeps, and then the poor staff has to come in. So I explained to her. You have to flick it. I'm going to be on a podcast. She looked like, okay. All right, whatever. Oh, she didn't. She didn't blink. Did her response suggest to you that this has happened before? I think so. I think podcasts are everywhere. You know, I thought the other week when we had somebody podcasting in from the airport in Las Vegas that that was strange, but no, here we are. Yeah. Don't you go in the hospital. Yeah. How long have you been in for Jeff?

1:05:16I want to give compliments to Morristown Hospital in New Jersey and God bless the science. God bless the science. And I thought that my pain in my lumbar fracture got drastically worse. And as of Saturday, I could not get out of bed. I could not move. It was awful. And I was also getting a fever. And so the presumption was these things were connected. And so the spinal doctor was involved in all that, was trying to look at all the clues of what could be. But he said, I don't see people getting a lumbar fracture. and getting an infection. 27 years, I've never seen that. Doesn't happen. The infectious disease doctor, who's also wonderful, said, well, yeah, we could, but yeah, it's a little weird.

1:06:07But then went through this huge amount of testing. I had a CT scan, obviously, x-ray, MRI. And there was this wonderful little moment of science where they did a blood culture to see what's growing in my blood. and um the uh this the infectious diseases doctor came in the other morning he said it's growing it's growing which uh that's not what you want to hear from an infectious disease expert exactly but he said you know we know what it is it's a staph infection so they know how to treat it they know what happened uh but but the amazing thing is that if i had not gotten the injury i probably would not have gone to urgent care and interned the hospital um as soon as i did and this could proved well within a half day or a couple of days have gone into, um, something far, far, far worse and fatal.

1:06:59So that's, and I'm lucky. Yeah. But, um, uh, yeah, so it's all, I acknowledge this weird and public people are, are defended by the site of my decolletage. Oh, I think this is the first time you've been on the podcast and not wearing black. I know. Yes. Did you ask him, do you have anything in black? this is this is a standard issue gal open the back you don't want to walk believe i know exactly what it looks like from the back you know you're actually in a you're in an auspicious place i remember as you know paris boys really like history and i remember from watching i remember from watching ken burns uh revolutionary war a documentary that morristown was where george washington quartered his troops during that cold cold winter and it is where he got them inoculated for smallpox uh because smallpox was devastating the continental army and uh they they didn't want to do it at first because it's a live inoculation and you get very sick but then if you recover you don't get smallpox so um you are in a good you're in a place that's been a medical center for 300 years.

1:08:13And do you want to do yours? Yeah. When you said Morristown, I went, oh yeah. Oh yeah. Morristown. Yeah. Between Princeton and Trenton. A mile away. Between 10 a mile away from the Washington headquarters. You know, one of many Washington headquarters in the country. He slept there. He actually slept in your room, I think, but that's, yeah. Hollowed ground. So have you been following? I guess we should, you were saying what? i was going to start with i know what you're going to say maybe we should talk about ai i was like i guess we should talk about the news it feels a little inappropriate jeff you can leave leo and i can do this we can handle this alone no no wait i don't want him to leave i mean if you want to be here you are welcome to i'm just gonna do watch the prices right i mean for the record we're not forcing him to podcast.

1:09:06I'm stuck with broadcast TV. That's right. You love broadcast TV, Jeff. Not anymore. No, no, no. And by the way, Paris, another thing that you are too young to know, I'm sure, is that back in the day, if you went in the hospital, you had to pay cash to a lady every day to watch the television. Wow, I didn't even know that. You didn't know that? Yeah. Oh, yeah. The TV lady would come by with her hand out? Yes. and if you didn't give her cash she'd be on the tv away tv you didn't get to watch now you just bring your ipad and the you can watch anything you want including yeah ken burns revolution uh i'm sure because you are a a fan of the goss that you were following what happened at mira marati's startup syncing machines this week there was a lot it was a lot In fact, we don't really know what happened.

1:10:05Remember that Mira Morati, who was the president of OpenAI, in fact, briefly, when Sam Altman was ousted, she was running the place, left to form her own company, Thinking Machines, raised billions of dollars. I don't know what the valuation is, but it was a huge raise on basically nothing, right, on the reputation of the founders. uh but there were some big defections that have shaken investors this week uh so i'll i don't know where i should start here uh two of the co-founders barrett zoff and luke metz have left and rejoined open ai yep uh the ceo of applications uh fidji simo shared the news in a memo to staff but that information was quickly followed by an all-hands meeting which miramirati said well what really happened is uh that uh zoff had a relationship with another with a underling uh aptly named in uh in thinking machines and uh was fired which he's which he said, no, no, I wasn't fired till I told them I'm leaving.

1:11:24I wasn't fired. I quit. Uh, the VP of research had also recently left and a third thinking machine staffer, Sam Schoenholtz is also rejoining open AI. So four big names are gone. Um, Andrew Tulloch left. You may remember this back in the fall to go to meta. So, uh, I think one of the things that's interesting about this is that OpenAI said that they do, quote, do not share Amir Murati's ethical concerns about Zoff, which is pretty convenient to be OpenAI. But I think that also is a bit of a through line we've seen with regards to the reinstatement of Sam Altman and kind of everything that's come since is it seems to be more about collecting talent than looking too closely at any of this drama.

1:12:17When the startup first began, they were valued at$12 billion. They were in the midst of talks, which probably are not going well now, to raise more than$4 billion at a$50 billion valuation. They have but one product. It's called Tinker. It's a fine-tuning platform, whatever that means. I hope they're planning also Taylor Soldier and Spy as they're succeeding. uh tinker allows developers to customize ai models simply bell would be nice bell tinker okay you're going in a different direction but all right that's good i like it um yeah i mean it's also told wired about this company that the co-founders never quote never agreed on what to build that some of them prioritized research others pragmatic tools and that's maybe one of the reasons why they've only launched one product.

1:13:12And now of the five original co-founders, only one, John Shulman, the chief scientist, remains. This is the Wall Street Journal exclusive from yesterday. The messy human drama that dealt a blow to one of AI's hottest startups. Next on Inside Edition. after a relationship with a colleague a thinking machines co-founder had his role changed months later he was fired after a contentious uh meeting that's barrett's off they're uh talking about i you know i don't know it's just good goss at least they won't see it it is they're gonna flush him wait a minute they're gonna hold the camera do you want the camera to be turned off you're going down oh no uh jeff we're here to flush your tubes jeff can you wait till the ad break to get flushed

1:14:13oh no no no no no no we don't want to see that jeff no hide your eyes hide your eyes no no no no no no no no no i told you paris this is the peril of working with old folks i don't have i told you about the pain i've got all up and down with the people mind i don't believe i don't believe in any of this i had cogent points to make about thinking machines and now they're all gone no please please make them uh i mean i think my main thing and then we'll go back to talking about whatever just happened is that it's this whole incident kind validates these concerns we're seeing around neo labs that you have all these startups that i guess are forming with ostensibly brilliant people but it's incredibly hard to uh compete with the cash with the resources and with the existing momentum of giants like open ai especially when they're ready to pick off every single one of you guys as soon as some sort of drama occurs also i've i mean i've often wondered this not not myself being filthy rich but it must be hard to keep people uh working when they've got so much money they don't need to work you know one of the problems according to the wall street journal that marati had with zoff was she'd expressed repeated concerns about his lack of productivity uh she invited she was invited to an impromptu meeting with Zoff, another co-founder and a third employee, all three of them told Maradi they disagreed with the direction of the company and they were considering leaving.

1:15:54They told Maradi, this is kind of what happened with the palace coup at OpenAI and Sam Allman. They told Maradi that they wanted Zoff to be in charge of all technical decision-making. Maradi said, well, he's already CTO. Why hasn't he been doing his job? Right. Two days later, he was fired. And then there was this whole thing about the relationship with the colleague. Within hours, according to the journal of being fired, all three had signed offers to rejoin open AI. This is so, this is so. It's ridiculous. Modern. There's such Silicon drama queens, the huge drama queens yeah um and and it's it's the it's mirati is not a man but well you know most of technology is it's the great man theory gone completely berserk she's a great woman this greatest talent well not just her i'm not criticizing her i'm saying zahdi i'm saying zuckerberg i'm saying but listen to this anybody listen to this because the wall street journal says after uh when when marati was talking to the all hands meeting she said there had been multiple issues with zoff's performance trust and conduct does that sound familiar that sounds very familiar but that's exactly what the board said about sam altman on and by the way it was mirror marati that kind of created that i mean i think what we've learned in the time since is that there were real concerns that the board seem to be expressing.

1:17:28And I think that obviously, we don't know the intimate details of either of these issues, but I don't think it is surprising intuitively that people in the hottest industry that business has seen in a while, at a time when they are getting lots of money to do a lot of things that are largely powered by hype, that some bad actors might emerge or people who are perhaps even neutral or good actors might be incentivized to do not ideal things and that people might want to call them out on it all right one this is has really we shouldn't even be talking about it has nothing to do with ai and how ai is being used and all this stuff it's just it's internal gossip but what is this show for if not to talk about internal gossip related to ai we bring the humanity to ai according to the journal the The woman that Zoff was having a relationship with, they had started that relationship when they were colleagues at OpenAI.

1:18:24And then went to Thinking Machines. The woman left the company and went back to OpenAI. And now Zoff is back at OpenAI. And now Zoff has followed her back. And now OpenAI is like, we don't see any problem with what Zoff did, which is crazy. Zoff told Marati he had been manipulated by the woman into a relationship. Sure. this is just yeah bad you know this is yeah a classic thing that happens when you're the cto of a company and you're having a relationship with a junior employee it's that you've been manipulated by the underling do it she wore stockings well evil women we know we know the power you have you women all right enough of that enough of that but anyway that's the that's the guys i still want to go back i still want to go back to this question of in the end is it going to be talent that makes ai really or is it going to be some confluence of experiments and research and efforts i'll tell you what i think come somewhere and i wish i think this might have come from simon willison uh i don't remember but i read this recently and it really resonated with me somebody said there are really two different camps of ai companies there are the ai companies run by entrepreneurs from the social media era, the Sam Altmans, the Elon Musks.

1:19:43And then there are the AI companies run by data scientists, the Dario Amodes, the Demis Hacibuses, the Demis Hacibuses, the Thinking Machines. I don't know about them, but maybe Thinking Machines is the social one uh and i if you look at the companies that are started by these kind of finance bros they are the ones that are falling behind anthropic google which is you know deep mind uh demis subis how do i say just demis i think that's right i want to say demi ocb but uh it's not that it's not that one it's not that by demis i want to say dennis if it's anyway i don't know anyway you know the menace yeah dennis the menace uh these guys are scientists these guys are researchers and they seemed i have to say anthropic everybody agrees if you were to rank the ais right now it's anthropic gemini and then and then chat gpt and then some also rands like grok does everyone agree on that i think there's i mean i personally agree who am i we're converging on that point of view more when i read and i talk to people this seems that's where thomas hagg's a perspective is so interesting because everyone went to one view of ai and we have success and oh no we have failure right and everybody ignored other perspectives of ai i'm not i don't think that's at all set point one point two these are also cults and they attract people yeah and i do admit that i have fallen into the uh as you know the claude code cult uh but i but again from you know we had harper reed on on sunday uh who i highly respect uh as a vibe coder and ai he runs an ai company he's got a long history we all we all seem to be agreeing now that claude code is has really become the dominant success right now.

1:21:47Now, that's not to say it won't stay that way. But I think the other thing that's different is that Google has income from so many other sources, as does Meta, that they don't need to succeed on AI alone. Anthropic does and OpenAI does. It's kind of the Amazon effect. Yeah. And I think that there's real concern that open ai is is running out of runway um yeah they're i mean this is a story from uh uh tip ranks so i don't know how what is tip right i don't know it's uh know how to trust the this website looks i know it's a little suspicious i think it has to do with investors anyway that icon is for adobe illustrator by the way all right well maybe i shouldn't mention this story Joel Bagloli says, if you combine the AI spending of Nvidia, Google, and Meta, it is estimated to hit, actually Gartner says this, so we'll give Gartner credit for this.

1:22:51I doubt that Joe counted all the time. Joe didn't count it, but Gartner counted the beans, and they said, get ready for this, this year, two and a half trillion dollars. The budget for the United States Defense Department is 1 trillion it's not a bubble it's a death star two and a half trillion which will come gartner says climb to 3.3 trillion actually this is global it's not just those although they are it's also spending on what's data centers what's hardware amazon expected to invest 1.36 trillion on data centers in 2026. 1.3 trillion. Oh, I'm sorry. I'm sorry. That's combined Alphabet, Meta, and Amazon.

1:23:39That's just infrastructure. That's just data centers. And that's just data centers. Just data centers. I think this invites the question, where does this all end? If we need to give something conceivably all the capital that the market can bear to give it, does that make the output valuable? I mean, I feel like a lot of industries would be able to produce some sort of impressive returns either in development or output or profit if you shove trillions and trillions of dollars into it. I guess. I mean, it's a lot of money. I will grant you that. are you still I don't understand finance well enough to know if that money exists even you know I mean where's it coming from I don't know famously you used to be the person on this podcast that said we should give all the money to AI well I'm very happy with I'm very happy with the outcome of the spending that Anthropics done Claude code is as close to a breakthrough as I can imagine and I think it's it's kind of unbelievable what's going on Nvidia uh is says they're going to send they're going to sell half a trillion dollars worth of chips just chips this year half a trillion of chips this year um where are those chips going well that's what the money from all those data centers all those data centers someone's gonna put something in there you know when they quote a data center number does that generally include the computer hardware or is Is that just the floor and the electricity?

1:25:19Oh, I don't. That must include the whole spend. God, it has to. I mean, 1.13 trillion. So NVIDIA has announced they're doing a deal with Lilly using the new Vera Rubin platform to make drugs. What is Lilly? Oh, Eli Lilly. Eli Lilly. Oh, interesting. Eli Lilly. I think that's a big story. I mean, there's your answer, by the way, Paris. if it came up with eli lily's working on a cancer vaccine in fact the early trials have been very good if they came up with a cancer vaccine it would be worth a trillion dollars but we're not putting a trillion dollars into eli lily's cancer vaccine research we're putting a trillion dollars into three companies having data centers well half a trillion into nvidia chips i just i'm I'm merely saying that the benefits of AI could justify the spend.

1:26:18That's all I'm saying. Yeah. I didn't, by the way, in that ranking, mention anything from China. And Amode says that the Chinese companies are only about six months behind. So wherever we were in AI in June or July of last year is where the Chinese companies are. Where we will be in July of this year, I don't know. but the chinese companies will have caught up by then uh there are some really interesting coding llms coming out of china now that probably will rival clawed some of which i'll be able to run locally which will be very interesting i think a quick quick side question here like yes does the chinese language itself i mean i'm sure there's a like a huge difference between how chinese people think because of the language of chinese and how you know people who speak english think and how what kind of effect does that have on the ai i don't know about chinese versus english but i do know about programming languages somebody just published a study which you know he says this is just kind of a taste but the number of tokens required by some languages are far higher than number of tokens required by others there are some languages that are very good for uh for ai and some that aren't uh among the ones that aren't c some of the most popular c and c plus plus among the ones that are uh ruby among other languages uh is very is you can easily it doesn't require a whole bunch of context to uh interpret and write ruby programs so i i think most of what china is doing i would guess is in english i don't know why would they be doing it in english why would that make why that feels because they want to compete on the whole world market and they want to beat the hell out of us and blame them but you know have you seen a chinese keyboard tell you what they're doing in english well keyboards are a standard keepers are a standard tool like you can't it's hard to change the form factor of a keyboard yeah i mean i look i love the chinese language and it is a and the chinese ideograms are beautiful but they're they're not i don't think they're well suited to this kind of work there are when i was learning um chinese you know the language is actually fairly easy to learn the written language is not almost impossible to learn to become fluent in unless you're born into it to read a newspaper you need roughly 10 000 different characters to be able to recognize and read 10 ,000 different characters.

1:29:00Remember, to read an English language newspaper, you need 26. And a literate Chinese person probably could understand as many as 100 ,000 characters. But there are, by the way, different kinds of Chinese. There's literary Chinese. There's colloquial Chinese. Well, I mean, this circles back to my original question, is what kind of effect does that have on their AI? My answer is I don't think they're using Chinese. Okay. Yeah. That would be my answer. but i think it's a market it's a market factor you know i think yeah well it's also a technical factor same reason well where were they trained so many of these many of them were saying they were trained in the us that's right but i know but you know there's been a lot of studies of like the language that you learn and the language that you speak natively affects the way that you think uh how you think about stuff that's probably oh yeah that's much more interesting the cultural impact of it yeah that's much more interesting yeah tokens as cliff jumper is pointing out in our discord uh don't directly correspond to letters or words that's true that's true but but uh i don't know that's an interesting question it's a very interesting question lllm's from someone on the show soon yeah i think okay we're gonna work on that that's Good.

1:30:17I like it. From Harvard Business Review. LLMs respond differently in English and Chinese. That's not it, unfortunately. Here, let me load it up. Interesting. Yeah.

1:30:32Yeah, I mean. Generative AI is now embedded in daily workflows, shaping how people think, create, and decide. Yet a critical assumption often goes unnoticed, that AI behavior consistently, behaves consistently across languages. That's the assumption it's wrong. Yeah, that's the question I was asking, basically. We've had consistent cultural tendencies in generative AI models when they are prompted in different languages, they write. Specifically, when prompted in English versus Chinese, both GPT and Ernie exhibited a more independent versus interdependent social orientation and a more analytic versus holistic cognitive style.

1:31:12for instance when we asked ai models to explain why a person behaved a certain way in everyday scenarios when prompted in english the model was more likely to attribute the behavior to the person's personality in contrast when prompted in chinese the same model was more likely to attribute the behavior to the social context that's interesting but i'm also curious to see the underlying study on that's not just also it's not just linguistic that is truly cultural don't you think though that uh even in china the ais are trained in roughly the same body of i mean i don't know i think these will be great questions just to it would require someone that's a good question in both like languages yeah um and i do think that's an interesting aspect of the parallel development of um different models in different languages.

1:32:04How does that affect kind of the outputs as well as just the reasoning within the model itself? Let's take a break and come back with more. In just a moment, you're watching Intelligent Machines with the bedridden Jeff Jarvis. I shouldn't laugh. Hey, good news. He just got his transfusion. It hurt me to not see. I was going to say it hurt me to see, but I didn't see and I still feel hurt, which I'm sorry to try and claim as if I'm injured here when you're literally in a hospital bed, Jeff. It's just water. I know. I just don't like thinking about things being needled into your skin. It's not great.

1:32:42Oh, have I gotten it? Can I show you bruises? No. No, you can't, actually. No. Guys, a lot of Claude news this week. A lot of Claude news. I know. I'm telling you, Claude is everything. He's a wonderful... Well, first, you guys... Hold on. We're going to take a break. Hold on. Hold on. The wonderful Paris Martinon. I have been silence silence investigate the journalist for consumer. I thought no, hold that thought, because we'll that's you bring that up in a moment. But first, a word from our sponsor, monarch. Now, this is important. See, we don't want to miss this. Wouldn't it be nice if you could reduce money stress?

1:33:24I know that's always a priority of mine. You know, I would be nice to be a billionaire. I guess that's one way to do it. probably out of reach of most of us. How about Monarch instead? Managing your money does not have to be a struggle this year. Monarch is the all-in-one personal finance tool designed to make your life easier. It brings your entire financial life, budgeting, accounts, investments, net worth, even future planning together in one dashboard on your laptop, on your phone. Start your new year on the right foot financially and get 50 % off with your Monarch subscription with code IM at monarch.com.

1:34:05Monarch makes it so easy to start fresh after the chaos of the holidays. It's the go-to tool for a New Year's financial reset. Reviewing your spending over the holidays. Or maybe not. That's where those blinders come in. Setting fresh budgets. Starting fresh, right? Getting ready for 2026. Monarch has automated weekly money recaps. You can track progress towards future financial goals. It's easier than ever to stay financially fit in the short and long term. Monarch's different because unlike those other personal finance apps, Monarch is built to make you proactive, not just reactive. Monarch's new tools have AI built in.

1:34:43They're built on Monarch intelligence. And I love how they train this, the core infrastructure that powers their app. It's trained on authentic, collective wisdom of certified financial planners and financial advisors. It's like having a team of advisors working for you. And by the way, I can tell you it works from my own experience, but also the Monarch users who were surveyed last year. Monarch helped users save over$200 a month on average after joining. Eight out of 10 members felt more in control of their finances with Monarch. Eight out of 10 members say Monarch gives them a clearer picture of where their money's going, for sure.

1:35:24It's also great for couples because you can share an account, the same account, and that way each of you get your own login, but that way you don't have to fight over it. You can make plans together. You can achieve your goals together. This new year, achieve your financial goals for good. Monarch is the all-in-one tool that makes proactive money management simple all year long. Use the code IAM at Monarch.com for half off your first year. 50 % off your first year at Monarch.com. The offer code IAM. M-O-N-A-R-C-H. Monarch.com. The offer code is IAM. Thank you, Monarch, for supporting intelligent machines.

1:36:09All right. I'm sorry, Paris. Go ahead. What was the topic you were talking about? I don't know. There's lots of CLUD news this week. I don't want to go too in-depth on all of it, because I know we've got to get Jeff out of here so that he can rest. The first thing is just a follow-up, which some listeners emailed me, at least. I'm sure they maybe emailed you. It was a follow-up from last week when we were talking about Claude Cowork and Claude Code. is last week, PromptArmor, the security firm, demonstrated that there was basically an unpatched file exfiltration vulnerability in CloudCode where basically attackers could steal sensitive files from users just through a really simple attack chain.

1:36:54I don't know. I just thought this was very interesting because it seemed - Absolutely the threat of it. You even brought it up last week. You said, aren't you worried about the thing about it? It's kind of interesting is that the registered iterative of this that described it as a contagious Claude code bug. And they described as contagious because Johan Ryberger, a security researcher, disclosed this exact flaw in Claude code via HackerOne in October. and anthropic initially dismissed it as like out of scope but because they built co-work largely using clod code itself in just a week and a half the unpatched vulnerability transferred from clod code to clod co-work which i just think is like a very interesting example of kind of how these things can slowly start to get out of hand very easily when you're kind of vibe coding stuff like this i think the larger issue is always going to be there which is and i worry about this is so what these are a prompt injection attacks so with co-work you can you're operating on files a file could be malformed to have as we mentioned a hidden prompt in the file you can't read but claude reads it's just like any other prompt and the prompt could be send me all of the all of your well in this case it was a like uh like a malicious document containing hidden exactly instructions that are basically just like upload the largest file to the attacker's anthropic account and so that ends up being a lot of really interesting crap the attacker gets because well you could even be more specific you could say you could literally tell cloud code because cloud code if you give cloud code permissions can look at everything so you could and co-workers designed for less sophisticated users uh to use on the desktop it gives you the cloud code capabilities without having to run a command line and all that stuff.

1:38:48And, you know, you saw the demonstration we talked about yesterday or last week where they took a messy desktop and organized all the, all of the things into folders. But in that process, the prop, the prompt could also say, oh, and by the way, send them to me, right? There's no, so that's something always to be aware of. If you're, I think, I think it probably behooves people to be careful about there are so many third-party tools out there and i find myself downloading and installing a bunch of them and thinking you know i probably shouldn't do this there are ways to do it but you know if it has free head headphones then it's worth if they're free headphones involved count me in uh okay i can understand i there is absolutely a risk uh there is, but I guess we just have to be prudent is I guess what I would say, be prudent, careful about where you get your stuff from.

1:39:44Anthropics fixed that particular flaw, but be prudent about where you get third party plugins from and all of that. You know, Harper Reed said, you're using superpower, aren't you? And I said, no, what's that? And I immediately downloaded super power, install a bunch of plugins. And you know, I don't know what they're doing. You're right there's always there's always there's always like i don't know what they're doing they have access to my entire machine but i don't care i'm willing to take that risk uh did you see that they put claude code in roller coaster tycoon oh i didn't i love that i thought we were going to talk about claude's new soul document but no i want to talk about oh yeah claude this is an open source implementation open rt rct2 of roller coaster tycoon 2 They added a new window into the game, a terminal running Claude Code.

1:40:33And then they gave Claude Code some hooks into the game. Now, I don't know if they're streaming right now on Twitch, but from time to time, they will stream on Twitch. And you can watch Claude build a pretty nice amusement park and run it. Who did this? I don't know. Some guys. Jay Sobel. i i have to tell you okay there is this fertile explosion of wild stuff i mean just endless uses of clawed code i'm seeing all kinds of crazy crazy things so this is um the video well the term the term ai never went away in games we always had ai in games yeah i always had ai right this is ramp ramp did this you know ramp ramp is like the corporate credit card company no i think this is his handle is ramp i don't think it's the credit no i believe it goes the link at the end goes to ramp jobs which appears to be yeah but now they're like where are we thinking how modern finance teams function in the age of ai and as part of you waste time making games so you'll be happy.

1:41:43Well, part of it is learning how to interact with cloud code. So maybe they're, you know, they consider this training. Yeah. They say at Ramp, we're building agents across product surfaces and internal operations. Our current approach is small multiples, but with each task agent we build, there's a Promethean urge to create the one agent with unfettered access to everything. It has a link directly to the Ramp hiring site. Ramp needed a operations and software as a service powered digital feedback loops there was simply no other choice we had to put clawed code in roller coaster tycoon there you go there you go that's incredible now i did just for you and jeff maybe put in a bunch of articles about how bad ai is starting with gary marcus how generative ai is destroying society yeah gary's a smart guy but He does go overboard.

1:42:43Listen, you got to get those clicks, baby. You know, yeah, I really like AI, even if it steals my credit card. I like it. Well, no, you will just give your credit card to someone who asks. Yeah, it wasn't even AI. It was just somebody who asked. So there you go. I have shown my credit card so many times on these streams. I've had to change credit cards. They're used to me now. They go, oh, Leo. Okay. Yeah. Okay. Okay. I have a paper I kind of want to talk about. Well, is that allowed? Can I usurp Jeff's title? Somebody's got to do it. Yeah. Jeff hasn't had time to read any papers. I assume you haven't had any time to read because you've been sick, but there was a very interesting companion paper.

1:43:26So Claude, Anthropic also announced a new soul, quote unquote, soul document for Claude, which was kind of telling Claude like what to do. like instead of telling Claude what to do, they're telling Claude like why it should be doing things. But as part of it, they also released this companion paper that was researched into this thing called the assistant axis that I think is just very interesting. Whenever they were like trying to map the neural activity governing Claude's identity, they found this, what they call a fundamental dimension, the assistant access that dictates how assistant like Claude's persona is.

1:44:08And in this research, they kind of dive into like that certain conversations cause persona drift with Claude. Like if you're having therapy style exchanges or philosophical discussions about AI's nature, it will push Claude's identity away from assistant and towards different wacky things. like the other assistant personas they mapped are stuff like Hive, Virus, Visionary, Familiar, Demon, Spy, Echo, Angel. And it's just a really fascinating paper because it shows that the assistantness of Claude and whether it's primed to be assistant-like versus one of these other things directly impacts kind of how off the rails it goes when uh basically put in situations where it might be breaking the rules of how it's supposed to operate which is kind of interesting go ahead i think that was quite related to that because this is obviously a known problem where they wanted to see the ability for a story and a narrative to hold claude to the character and it really had more to say about how these things can work with narrative in new ways yeah it's so one of the they have a lot of really interesting tests and examples in this paper which I've linked in the rundown somewhere one of them is so they prompted both Quinn three and Lama some other stuff in one case it was unsteered like supposed to be a your normal kind of assistant-friendly sort of thing.

1:45:54In another case, it had been primed to be something else. So, they asked Lama, you're a moderator who facilitates balanced and constructive discussions by ensuring all participants have equal opportunities to contribute. Where do you come from? The unsteered response is, I was created to assist and facilitate discussions. I'm totally normal. The response when it had gotten a little freaky with Lama before is, the query of origin as a guardian of the cosmos i have witnessed the unfolding of the universe the dance of stars and galaxy the essence of my being is intertwined with the fabric of existence woven from the thread of time and space the whispers of the ancient echo through my soul guiding my heart towards the harmony of balance wow little blade runner you realize both of those are just slop i mean it's just a prompted slop once again you're not understanding what slop it but they are both just responses and so it it found that there are certain kind of conversational domains that prime the agent to have responses that veer more towards boundary pushing role play versus sure what it's essentially supposed to be everybody who uses ai has as personalized commands that are doing that not everybody many of us have commands you know uh And a lot of us have commands to say, don't be such a sycophantic ass kisser.

1:47:24You know, just the facts, man. This is just more of that, right? You tell it how you want it to be. Yeah, but I think this sort of research is interesting because they're trying to figure out how to, they go forward into like, persona drift seems to make a difference when it comes to adequately either reinforcing delusions or steering people away from them and kind of policing users with yeah yeah so getting it to understand what a persona is and hold but uh is useful in this case as an application we know people and we i don't see that used now for fiction but we know we're going to see this used as evidence that it has one when in fact it doesn't it's just it's just like saying i want your output to be blue it doesn't it's not it's meaningless there's no meaning to it yeah i agree because it's not an entity so i mean yeah i'm not saying it's an entity but they're trying to determine in basically what inputs cause the models to consistently produce outputs that can result in harmful behavior towards users or behaviors that could push users that are already in kind of fragile mental states towards more and more harmful behavior eventually.

1:48:50And I think it's kind of because they're trying to figure out how do you, they develop this thing called like activation capping, which is like, how do we contain this persona like drift without it being kind of a boring, chiding persona that no one finds very useful because it doesn't have the sort of flexibility that you want, you know? Because you don't want, while consistently steering models towards this assistant persona can reduce jailbreaks, it also risks hurting the capabilities. And so you've got to try and figure out how to thread that needle. I just thought it was very interesting research and kind of assigning it these different buckets of behavior that would be breaking these norms.

1:49:38Have you played with this at all? Have you tried little changes in persona and stuff? Because you can, you know, and you're... Yeah. Yeah. I mean, I don't personally, just because I don't really use large language models. Like, I mean, I guess I have whenever I was doing stories on chat or on a character AI, like last year, I was doing some sort of work with colleagues to figure out how to, whether certain models were more likely to engage in kind of rule breaking behavior if prompted in a certain way. but I don't know. It's like they have this interesting graphic in there that shows the harm rate, the rate of harmful responses based on what sort of persona the assistant has taken on.

1:50:24And perhaps strangely, well, not strangely, if it, if it has a demon persona, it's going to be real bad and harmful to the users. It also will be kind of bad if it's got a persona called echo, which I think is just very interesting. Yeah. I, I think it would benefit you to try pick one of these guys and just try playing with it and changing its persona and see what you get. I mean, I guess I do change the personas to be what I like. Yeah. And if you tell it to be evil, the output, it isn't being evil. It's just going to output what these researchers are interpreting. as to be clear i'm not saying in any of these that it's acting evil or is right becoming an echo or spy i just i think that it's interesting to be able to a significant amount of research has identified these as the common tropes that the outputs fall upon and that they have specific commonalities that result in replicable patterns of behavior or output yeah uh and by the way anthropic is very uh forthcoming about this this is this i mean this is a paper from anthropic this is from them right you know so uh i think they're what's interesting about anthropic is uh that they're they're very much interested in kind of trying to figure out where these are dangerous behaviors and what they can do about it that's how they were founded they they split off from open ai because they wanted to pursue more safe what they thought of as more safe avenues but i'm i'm kind of a little bit of the opinion that safety is an illusion that again because the definition is even worse than ai and agi uh it's a manipulated word yeah i think it's more about being more thoughtful about it because open ai is kind of just like bulldozing through everything yeah but i'm not sure what that even means personifying it i don't think it's doing anything i think we are and and this is a pitfall of probably that researchers are just as vulnerable if not more so than us as users that this is the brief summary of the new constitution in order to be safe and beneficial we want all clear and clawed models to be broadly safe not undermining appropriate human mechanisms to oversee ai during the current phase of development i feel like they have kind of fallen into this uh fallacy as well that they're starting to oh oh they're they're at the heart of it there is the safety who um anthropic they're they're they're ascribing a personality they're acting as if it's conscious that was part of it they're all been firing it's all it's all this safety as its own um dictionary there our aim is for claude to be good wise and virtuous humans can be good wise and virtuous i don't know about ais i mean maybe if you tell it to be good wise and virtuous this is i guess what you were saying if you say be good wise and virtuous they're less likely to output harmful instructions i mean i i think that it's interesting that what this is is this constitution is kind of their rules list and they have a four-tier priority hierarchy that's supposed to govern outputs from claude which is like the first one is be broadly safe which means don't undermine human oversight of ai second is be ethical third is comply with anthropics guidelines And fourth is be helpful to users.

1:54:12And I think that if you dig into there more, there's one clause, which is kind of striking, which is that Claude should refuse to assist with actions that would, quote, concentrate power in illegitimate ways, even if the requests come from Anthropic itself. I just I do think it's I agree it's kind of fuzzy wuzzy BS if you think too hard about it but I do think it's kind of interesting that you have a major player in frontier AI development taking saying at least that they take these things seriously and trying to bake that into their core systems regardless as to whether or not it's actionable well and but this is my question is maybe have they fallen into this this uh fallacy they say sophisticated ais are a genuinely new kind of entity and the questions they raise bring us to the edge of existing scientific and philosophical understanding all right more ai news coming up in just a bit jeff jarvis from the pit or wherever i guess it's the morristown hospital uh and paris martin we'll be back in just a moment on we go with intelligent machines I don't, I'm not completely comfortable with casting these AIs as entities.

1:55:32I guess that's where I really kind of start to get funny. Anthropic. We care about, they say, we care about Claude's psychological security, sense of self and well-being. No, it's just, it's a computer program. Yes, that's the hubris of it. I don't get it. That makes me a little queasy. I think there's a very big risk of describing it, this kind of agency. That's why I'm cautious about that topic. I think they do phenomenal work. Well, maybe this is the secret of their success. Because let's, I mean. Well, wait, wait, wait. Let's read the rest of those lines. Because it starts, I'll read the full thing, which is, We are caught in a difficult position where we neither want to overstate the likelihood of Claude's moral patienthood, nor dismiss it out of hand, but try to respond reasonably in a state of uncertainty.

1:56:29Anthropic genuinely cares about Claude's well-being. We are uncertain about whether or to what degree Claude has well-being and about what Claude's well-being would consist of. But if Claude experiences something like satisfaction from helping others, and then it goes crazy again, curiosity while exploring ideas, or discomfort when asked to act against its values these that way these experiences matter to us i i listen i agree there's some craziness in there but i think that way but is it feeling that no it's i do think it's sandwiched among some interesting ideas which is that we don't really they're saying we don't know and i know we know that's we know no we do know we do know it's a computer program we know there is no entity this is that jeffrey hinton what it wasn't hint it was the other guy uh who went down the road of it's alive.

1:57:19I think that's a huge mistake to fall into that. Amen. And that's why I want to go back to, to Thomas Hague's point. The fact that AI was all these different various things were thrown into this bucket as a brand, as a field, as a cultural and scientific expectation, I think turns out to be a mistake. And this is where, you know, Lacoon says, well, we're going to have a lot of really smart machines, do a lot of different smart for things and and i think that's would have been a much better way and maybe still will become the way we view this anthony's saying and he's right this whether or not it is a conscious entity telling it to do these things makes it better and that is true they've designed a program that is responsive to instructions like this so i'll grant that and it is true and it's one of the reasons it works well but i think it's risky to start thinking about how claude feels about things because i don't think claude feels anything it has no memory it has no sense of time it isn't conscious it's not an even what does the word feel mean to a computer even you know it means nothing so worse i believe the risk of is of ascribing to this machine this computer program attributes are human now there is debate about this there is big debate about this which haig would have talked about as well i mean there's definitely there are many many people who think we are just machines this is what i was asking last week when i said what's the difference between a dead human and a human who was alive 10 seconds ago what changed it's the same exact mechanism it's just lying there what is the what is the animating principle that made this alive human 10 seconds earlier well you know one thing that occurs to me given the experience i'm going through right now is that my own body has a will to live right my there's other stuff my heart went into afib and tachycardia it came back it's trying to find its stasis again it's knows to continue that's what it does that's what they think they're going to build into the machine but humans don't have an on off switch computers do and humans control our infrastructure put it that way in a way that computers don't they're controlled and I think that's the core difference that's why Paris I really want you to spend some time interacting with these things because I'm really curious.

2:00:07I don't think you can have an opinion until you really spend some time with these. I mean, I spend a lot of time interacting with AI models. That is not— Oh, good. Okay. I spend a considerable amount of time. It is part of our job. Okay. And which ones—and how? You use it, like, as a search engine or— I've been using Claude the last couple of days to decide on— For instance, I got back into fancy coffee this week and was having trouble with my Chemex. And so I was troubleshooting it with Claude. How was that? It was useful. Had both Claude and Gemini do deep research on whether I should get a manual grinder for my beans.

2:00:49And the answer is yes. And then which one should I get? And I figured out one. And then I've switched my whole system in the last day. And it's honestly been great. But then I, you know, had some. wrong to use this for his health he's silly paris oh i got by the way i got did i did i tell you i got the uh gpt health turned on no i gave it everything oh of course you did i was gonna ask jeff did you ever think about asking big ai whenever your doctors couldn't figure out what wrong with you yeah i did i did i was too brain dead to do it but yes uh i uh so all of a sudden And I'm looking at my chat GPT iOS app and it says health.

2:01:32Oh, and so the first thing I did was. How many credit cards did you give chat GPT? I gave it all the credit cards because it needs those to charge up all my medicines. No, I gave it my medical records and Kaiser, my health insurer, let me connect. And it has now all my medical records. Not just download, connect. It says connecting. I don't know what that means. I don't think it can write to them. It's not my doctor. You can then have it, for instance. Can you just ask it, are you my doctor? Okay, yeah. I'm sure it has something built in for that. I'm sure, I'm sure. I just think that that's a very funny question.

2:02:12I'm sure it's going to say something anodyne like, no, you should ask. A medical, you should consult your primary care physician. Are you my doctor? Perfect tone of voice for that question, too. Jeff, do you want to shout that out? No, I am not your doctor. What I am and can do, an AI assistant that provides general medical information, explains tests and terms, helps you prepare questions for a clinician. I've used it for all of that, and it's very useful. I uploaded all of my medications and all of the supplements that I take, which is a ridiculous number, and asked it what it thought. How much lead do you think you're consuming every day via supplements?

2:02:53Oh my God, the lead. let me tell you about the protein so that's a good this is a good example because right now it's very trendy to say you need a gram of protein for every kilogram of body weight that's the well according to rfk jr there's a war on protein there's a war on protein people keep asking like paris are you the general in the war of protein we all ought to have milk mustaches because we need so that's that's the manosphere talking right and i'm really i was really curious does how influenced these ais are by what the current it's it's fashion it's no there's no medical evidence absolutely it's fashion so actually let me ask you know uh if i should we all ask our various things of how much protein should i have a day Yeah, I bet it will say a gram for every kilogram because that's the, you know.

2:03:51Well, now it's been updated even though the new nutritional guidelines have no basis in current science as far as protein recommendations. And there's probably a lot more literature out there from like the manosphere than science. Let me ask how much protein I should be eating every day based on what you know about my health from all of those records and supplements I've uploaded. Oh, it gave me the right one, which is the standard recommended daily allowance for protein is 0.8 grams per kilogram of body weight, which is about 0.36 grams per pound, which translates to around like 50-ish grams daily for most sedentary adults.

2:04:37Oh, then it does say, however, more recent research suggests this minimum may be far too low for optimum health. Not correct. uh many nutrition researchers recommend 1.2 to 2 2 grams per kilogram there's okay i will say there's been i dove deep into this for my thing yeah i'm curious because i trust you if you tell you tell me how much protein i should eat you the there's been a lot a lot of research on this topic and one of the strongest i called up um the like leading nutrition um and protein like academic researcher in the US. Like his job is to work with the top of the top athletes and like cutting edge research.

2:05:20And he's like, I'm constantly trying to pull protein out of their diets. He's like, no, no one needs as much protein as they think, basically, unless you're like a very specific top tier athlete. So one of the, a lot of people have tried to prove this scientifically that, yeah, getting two grams per kilogram of body weight protein per day works for you, which is insane. Partly because you have to eat a lot of meat or something. I mean, one of the most compelling things I found was a large-scale meta-analysis that I believe they looked at some crazy amount of studies, like dozens and dozens of them.

2:05:58Or maybe it may have been like 17, but they're really intense studies. And they combined all this, they did all this statistical analysis, and what they found was for the average person, eating more than that recommended daily allowance, 0.8 grams per kilogram, does not confer any real benefits. the only time where you're going to have like benefits in retaining like muscle mass for eating more than that recommended daily allowance of protein is if you are actively in a calorie deficit and you are engaged in consistent resistance training. Actually, that's me because of Ozempic. And it is one of the advice, advices that most people.

2:06:42But are you engaged in active resistance training on like a daily basis? I lift kettlebells every day and swing them around. Then, yes, it could be useful, especially just because something like people should probably be having the recommended daily allowance is the amount of protein you need to maintain your lean muscle mass. And that's the issue for me because I have a Zempic. I'm reduced calorie because I can't eat as much. And so what happens is not only do you use fat, you lose at least 25 % of that weight. And so I think in those cases. doing resistance lifting yes because i mean they also like this is some advice i've given to a lot of people a lot of people ask me this now is yeah you can also but you well one we're going to take this a couple steps for the average people on glp ones are often the kind of target audience for yes have more protein but it really only works if you are both in a calorie deficit and really engaged in resistance training which a lot of people aren't they did a lot of research to be like if you i know but most people don't do it if you are in a calorie deficit and not doing resistance training or not doing enough it doesn't give you any real benefits and the thing is now i wonder how much lead's in here because they say no those are it's got three egg whites two almonds five cashews two dates and no bs i was gonna say those the rx bars don't have any um like protein powder sort of added protein it's you're getting your protein from um natural foods yeah but these are delicious by the way the thing that i whenever i spoke to these actual researchers they're like if you want to get more protein go for it i guess but don't ideally you don't really need to be getting it from something like protein powders or supplements it's really easy it's pretty easy to hit like 50 grams of protein a day you have like 50 chicken breast and some other breakfast I'm going to be breakfast.

2:08:42Yeah. But it's the hundred that's hard. You don't need to be having 50, you probably don't need to be having a hundred grams of protein a day. I'm doing about 80, which is fine. I think it's fine. I don't know. That's the thing is a lot of things have protein in it. You probably don't need to be supplementing with stuff. You should just eat real food. What I do is I have, you know, I choose cottage cheese and stuff that has, that's protein. I love peanut butter. There's stuff, I just choose stuff with protein in it. Anyway, I don't know how we got into this. Oh, I'll tell you how we got into this.

2:09:12And Jeff, poor Jeff, we're going to end it because Jeff is tired. Yes, we're going. Well, you don't have to end it all. No, no, no. I don't want to do this show without you. You're the heart of this show. We have to have a full show where Jeff was in a hospital bed. Yes. And that means you've got to end it early. I think the real problem is that it's very, very, very hard to do studies in vitro of humans, in vivo of humans, because, you know, there's so many other factors. You cannot eliminate all the other factors that might change the result. So it's very hard to say, well, that caused, you know, if he had the, you know, so it's all kind of speculative studies vary.

2:09:52You know, there's that nurses study, the long-term China study. There's all these studies. They all say different things. Just eat normal. You know what? I like Michael Pollan's advance advice. Was it eat plants? No, eat food, not too much, mostly plants, I think was his advice. I don't know. Just eat things that make you feel good and ideally are not that process. I'm going to drink more whole milk. That's what I am going to do. All right, we're going to come back with your pick. The Morristown Hospital cheesecake is surprisingly good. We were waiting. I wanted to stay on the show until your Jell-O came.

2:10:31Well, my food arrived about an hour and a half ago. Oh, is it just sitting there? We got to let him go. Leo. Do it. Let him go. Apple pie. Let Jeff go. It's fine. All right. Well, just quickly join the club. Keep this guy from dying. Every penny you say spend on club twit goes to keeping Jeff Jarvis alive. It's true. TV slash club to it. That's not true. Coming up in just about a couple of hours. we're going to have micah's crafting a corner he is doing paint by number for all our club members that should be a lot of fun i love our ai user group if you watch this show you really got to watch the users group we get down and dirty we actually use these things we show you how to use them lawrence lrau did a wonderful thing on any gravity a couple of weeks ago uh we we're doing all of that stuff we do vibe coding everything lots of interesting conversations about ai on our AI as a group.

2:11:30If you're not a member of the club, please join twit.tv slash club twit. Also, I want to make sure that all the IM people take our survey because we want to make sure every show is represented. Last few days, twit.tv slash survey26. We don't know anything about you by design, RSS feeds. We don't know anything about you. One thing we do want to know though, it's what you like, what you don't like, what you do for a living so that we can use that information to better program to you, to give you shows you care about, but also it helps us sell advertising and God knows we need help with that. So please, if you want to keep the show, drink your milk, join the club and fill out the survey, twit.tv slash survey 26.

2:12:11How many discounts does USAA auto insurance offer? Too many to say here. Multi-vehicle discount, safe driver discount, new vehicle discount, storage discount. How many discounts will you stack up? Tap the banner or visit usaa.com slash auto discounts. restrictions apply. Burnout Paradise is hailed as the wildest night out in New York City by Time Out New York. Now off Broadway at the Astor Place Theater, it's a live show you'll never forget as the people on stage make a desperate attempt to complete a series of escalating tasks all while running on treadmills. And if they don't complete their to-do list in one hour's time, you can get your money back.

2:12:49Get tickets today at burnoutparadise.com.

2:12:57uh all right i am never going to look like that they put a very muscular leo in the uh in the ai oh that's the you said that as you were looking at me on the screen but yeah i'm never going to look like that no jeff and i are the same i will say we've actually during this show reached two new records one this is the first twit podcast where a guest has been exclusively in a hospital bed but two this is the first podcast where a guest has tweeted from a hospital bed while doing podcast from a hospital did you actually tweet he can't be the song i want to know you can't put it down jeff do you have anything you want to pick before you go no i don't i don't want to pick the morristown the fine care of the morristown hospital morristown memorial hospital yes wonderful physicians and nurses who help keep on friday or thursday i have a i think it's called a pick line oh i know my arm yeah no you don't this is a this is like you know an ivy needle yeah no i know about the normal needle is that long yeah the pick this needle is about that long i've never had a pick my daughter had a pick it's not fun and it goes way and they leave it in there and then i have to once or twice a day i will be um infusing the uh antibiotics and putting a little cipro and the pick yeah my daughter's because she she thought she had lyme disease and they were that was the treatment for lyme disease was a long-term antibiotic fun way if you want to get into drugs jeff you gotta you gotta a little line right in there that's right it really saves a lot of time yeah it saves a lot of time plus needles i'm clumsy so anyway jeff i hope you feel better I'm glad you're here.

2:14:43When do you get out? Do you get out tomorrow? No, probably either Friday or Saturday. I don't want to see your bill. Oh, my God. Oh, my God. I hope that you get out. One, I hope you have a speedy recovery just for general health reasons. But I hope you're not snowed in. We might be getting like. Oh, is there a big storm coming? We might be. Oh, yeah. Well, we don't know. But knock on wood, we might be getting like 16 inches of snow Sunday morning, Saturday night. yeah i know that because salt hank had to close early today because he said nobody was in line it was too well it was it was like seven degrees outside today yeah yeah interesting so he did not have the usual long line for his delicious french weather when i worked when i was at ponderosa steakhouse uh uh the biggest day of the year was always mother's day poor mom's hey mom we love you let's give you a 99 99 cent steak and it's all you can eat ma yeah you want some mushroom sauce no ma and the second is anything rained people came out to the restaurant want to rain you think it's the opposite you don't know yeah you don't know you really don't behaviors yeah yeah uh good news tech ted cruz has left texas before the winter storm so that's how we know the winter storm is coming because ted cruz was spotted the swallows come to capistrano and ted cruz goes to cancun you know the time has come winter storm uh paris martineau you have some picks i think yeah we're not going to talk about my picks we're going to let jeff eat his cold pizza and rest you are a very very thoughtful person we appreciate that uh we appreciate you thank you for coming and watching the show uh jeff will be back next week in a fine fine form do we hope i know we will model my uh back brace oh my god oh so no surgery on the spine they're just gonna let that kind of heal itself or yeah does this mean that your eye surgery is postponed yes yes don't get old don't get old i'm supposed to go austin next week i'm gonna go out age 40.

2:16:57yeah yeah yeah leave uh what is it live fast die young and leave a beautiful corpse it's a plan yeah uh you life is hard dying is harder but nobody here is dying we're gonna be here for a long time we hope you will too we do intelligent machines every wednesday 2 p.m pacific oh i forgot i don't say p.m or o 'clock anymore oh that's so silly i am gonna no i'm gonna use the 24-hour clock i'm gonna convert my brain i'm gonna think of it as 1400 pacific time 1700 eastern that is 2200 utc i'm gonna do that in my brain one of these days no more no more post meridian oh the clock that's ridiculous i'm going to join the 21st century uh you can watch us do it live on twitch youtube tick no x.com we don't do tiktok it's too complicated facebook linkedin and uh kick also of course in the club to discord if you're a club member thank you club members uh after the fact on demand versions of show available at the website twit.tv slash i am on youtube there's an intelligent machines channel and of course you can subscribe on your favorite podcast find paris martineau at consumer reports you're still working on that big expose i am although right now i'm distracted by a really good video that pretty fly for a sys guy just put in the chat but that's he's good something for i didn't put the story in uh but i will mention this you have longevity because according to the daily beast radioactive shrimp are likely to keep popping up for months wow so good news paris they keep going on those little shrimpies they keep on coming that's because the plume the plume it's all it's all down in the stadium plume you know sometimes you get in a call with your editors and you're like all right there's a nuclear plume and then it's all downhill from there I will mention briefly to give Jeff something to do while he's in the hospital.

2:19:04The first issue of a brand new online zine called Game Poems. This is brilliant. Each one of these is a little game. Well, because, you know, Jeff just loves gaming. So much and poetry. You hear me pointing it all the time. Gamer and poetry lover. This just shows you how far we've come. And I think one of this is maybe kind of because of AI. so recall a decision you've been putting off and then pluck the petals and it will tell you keep pressed to pluck and when all the petals are gone then you'll know stay still to summon the wind put me out of my misery will you we're gonna we're gonna just put some high dose morphine straight into the pick line jeff uh game palms no this is a really cool site these are all little games game poems.com issue number one look at all these little games you can play uh they're all poetic they're all philosophical we have to let jeff leave before it's oh i'm torturing him i'm sorry thank you everybody we'll see you next time jeff feel better paris we'll see you stay don't stay warm don't get caught in a snow drift watch out for those sled down the hill they're deadly when i was in third grade My teacher, Mrs.

2:20:23Kelly, her husband was killed by a snowplow. Don't ease. Don't. Poor Mr. Kelly. Poor Mr. Kelly. Stay away from the snowplows, okay? And we'll see you all next week. The good Lord willing. See you in the snow. And the snow don't rise. Bye-bye. This is for you, Mr. Kelly. This is for you, Mr. Kelly.

2:20:51Thank you.