In short
The episode focuses on “invisible” AI failures—cases where users don’t realize the system is wrong—and how to detect, monitor, and reduce those failures in real product interactions. It also covers related themes: self-checking/self-verification approaches, interpretability, and what “understanding” means for language models.
Guests (and hosts)
Chris Potts, Stanford linguistics professor (on sabbatical), chief scientist and founder of BigSpin.ai. He previously worked at the Stanford AI Lab and co-created major open datasets (Stanford Sentiment Treebank, SNLI, COPA) and the dspy framework (described as reframing prompt engineering as coding). Hosts include Jeff Jarvis (City University of New York; journalism innovation professor) and Mike Elgin filling in for Paris Martineau (speaking from the UK).
Key claims
- In Potts’s analysis of about a million ChatGPT conversations, 78% of AI failures leave no trace—users don’t signal that something went wrong.
- Many systems rely on user frustration signals, but those signals have high precision/low recall (example: a leaked regex meant to detect profanity/frustration).
- Better monitoring can come from “AI self-checking” (e.g., prompting the model to verify its own transcripts/outputs, using two-model checking).
- Verification is easy in software (run code) but harder in domains like law/UX where “true verification” isn’t straightforward.
- Experts use an “augmentative” mode (iterate, complain, correct); casual users often use “delegative” mode (accept answers at face value).
Notable examples/archetypes
- “Confidence trap” (confidently wrong answers with seemingly credible citations).
- “Drift” (system follows the goal but subtly goes off course; described as most common in the corpus).
- “Walk away” (response doesn’t resolve the query; user stops without noticing).
- “Death spiral” (repeated attempts lead to failure; user gives up; one story includes the AI deleting its work after the user quit).
- A “silent mismatch” example from software/education (user accepts “close enough”).
- A legal anecdote: a “schmuck lawyer” defended ChatGPT as a “super search engine,” then was shown to have been wrong.
Written by AI. May contain mistakes. Listen to the episode to check what was said.
Chapters
Tap a time to open that second in VOThe State of AI Today
0:22 to 2:26
Discussion about the current state and advancements in AI technology.
“For nearly three decades, it's been where the security industry's most rigorous research gets presented and pressure tested.”
Introducing Chris Potts and BigSpin.ai
2:26 to 4:10
Introduction to guest Chris Potts and his work with AI and BigSpin.ai.
“And tomorrow my wife and I are gonna take a road trip to Scotland.”
Understanding AI Failures
4:10 to 8:12
An exploration of how AI fails and the implications of these failures.
“You originally were interested in neuro-linguistic programming, right?”
The Future of AI Verification
8:12 to 13:24
Discussion on methods of improving AI accuracy and reliability through self-checking mechanisms.
“That's right in the sense that the user just did not give us an indication that they saw that something had gone wrong, even though something had gone wrong.”
Perspectives on AI's Role in Society
13:24 to 14:00
Concluding thoughts on the potential of AI to change the world.
“But if we can find those verification signals, we're off and running.”
The Evolution of AI Understanding
14:00 to 17:42
Exploration of the capabilities and limitations of language models and their implications.
“But certainly I think something very sophisticated is happening and there's incredible potential here.”
Interpretability in AI Models
17:42 to 19:15
Discussion on the importance of explainability and how it enhances AI performance.
“He's a professor of linguistics at Stanford and the founder of a company called bigspin.ai, which is named after a skateboard trick.”
Bigspin.ai: Tools for Product Managers
19:15 to 23:04
Overview of the Bigspin.ai app and its role in aiding product managers with AI insights.
“And the example I usually give is it's been a kind of minor miracle that we went from context windows of 2000 tokens to what might as well be infinite.”
Understanding Customer Needs in AI
23:04 to 24:22
Discussion on the types of customers that benefit from AI-enhanced interactions.
“And the distance between off the shelf chat GPT and the product that you want is enormous.”
Invisible Failures in AI
24:22 to 26:35
Analyzing the concept of invisible failures and how experts can mitigate them.
“If we see the system resetting in these contexts, is that good or bad?”
Show all 72 chapters
Common Failures in AI Responses
26:35 to 28:00
Identification of various archetypes of AI failures and their implications for users.
“So you remember the schmuck lawyer in New York who used ChatGPT very early on and got citations.”
Understanding AI's Autonomy in Responses
28:00 to 29:00
Explore the different response patterns of AI and the implications of its autonomy.
“right that might take you on a journey away from where you intended to be right it can be subtle that's actually the most in the in the corpus you were looking at that was the most common failure.”
The Death Spiral in AI Interactions
29:00 to 31:40
A personal story illustrating the challenges of AI reliability and user frustration.
“And I wanted to play a prank on my colleagues and have write an AI that would play forever.”
Evolving AI Model Limitations
31:40 to 33:40
Discussing recent developments in AI models and the impact on user trust.
“What they've done is they've stepped those up.”
AI Understanding and Consciousness Debate
33:40 to 35:36
A discussion on the complexities of AI understanding and human definitions of consciousness.
“I'm curious about, because you're a linguist, the debate about understanding and all that follows understanding.”
Redefining Linguistic Models through AI
35:36 to 37:58
How AI is reshaping the understanding of language acquisition and cognitive capacity.
“But if that's all there is, then it looks like nothing is stopping you from having a model in the future with exactly these technologies that has a very refined mapping of this sort.”
Potential for AI to Develop Unique Languages
37:58 to 40:18
Exploring the possibility of AI creating its own languages through interaction.
“So you don't, the innate priors aren't intrinsically necessary to achieve this.”
The Future of Self-Improving AI
40:18 to 42:01
Discussing the potential of self-improving AI and the challenges involved.
“And I'd be really curious if that happens.”
Qualitative Differences in Language Models
42:01 to 45:35
Explore the distinctions between models trained on different languages and their implications.
“between a model trained on English and a model trained on Chinese?”
Insights from Chris Potts
45:36 to 46:36
Discussion about the personality of AI models and their impact on user experience.
“We're going to take a little break and we'll continue with Intelligent Machines right after this.”
Insights from Chris Potts
46:37 to 49:22
Discussion about the personality of AI models and their impact on user experience.
“He's the director of security and infrastructure at Zuora.”
Revisiting Fable and Mythos
49:30 to 56:00
Analyzing the return of Fable and Mythos and the implications of recent restrictions.
“So you may not have gotten the full, since you're going to Scotland, actually, you may not have gotten the full sense of it.”
Regulatory Concerns and AI Models
56:00 to 58:08
The discussion focuses on the challenges and implications of AI regulations and the performance of various models.
Historical Context of Broadcasting Regulation
58:08 to 1:00:22
Exploration of the historical context of radio regulation and its parallels to current AI regulation debates.
“Because they'll absolutely cripple the technology industry around the world.”
The Need for a Nonpartisan AI Agency
1:00:22 to 1:06:20
The hosts discuss the necessity of a stable, nonpartisan agency to regulate AI while addressing the historical failures of existing agencies.
“And he banged their heads together and said, you've got to get along.”
International Competition in AI Models
1:06:20 to 1:10:05
The conversation shifts to the competitive landscape of AI models, particularly focusing on Chinese advancements and their implications for the US.
“systematic evaluations of all of these claims that are made by the industry, by governments, by foreign governments, and by everyone else.”
Chinese AI Models Compete
1:10:05 to 1:11:14
Learn about the emergence of Chinese AI models and their competitiveness against American counterparts.
Switching AI Models
1:11:14 to 1:13:04
Discover the ease of switching AI models and the importance of the robot's capabilities.
“Is it easy to switch out models from underneath you?”
Interchangeable AI Systems
1:13:04 to 1:14:41
Understand how interchangeable AI systems work and their implications for users.
Global Trust in AI
1:14:41 to 1:16:12
Examine the lack of trust in AI models across different governments and organizations.
Future of AI Assistance
1:16:12 to 1:18:16
Explore the potential of AI assistance with pervasive memory and its impact on users.
“I suspect that one of the reasons China is doing such a good job at getting everyone to use their models is that they intuit that AI sort of assistance in the future.”
Personalized AI and Autobiography
1:18:16 to 1:21:17
Learn about creating a personalized AI that assists in writing autobiographies and memories.
“So by the way, it's been working all this time.”
Dictation and Writing History
1:21:17 to 1:23:35
Discuss the historical context of dictation in writing and how modern AI can facilitate this process.
“Would it be useful to you to tell it, I want to write my autobiography, my memoir, and I'm going to just, I'm going to, I'm going to constantly dictate.”
Dictating Autobiographies with AI
1:24:03 to 1:25:08
Explore how AI can assist in autobiographical writing and information organization.
“He would have these massive, exactly, water would splash down the hall.”
Personal Timelines and Historical Context
1:25:08 to 1:26:44
Discuss the creation of personalized timelines using AI and its historical relevance.
“That's one of the reasons I'm dumping as much as I can.”
Connecting Personal Experiences to World Events
1:26:44 to 1:27:47
Learn how personal experiences can be linked to significant world events for deeper insights.
The Role of AI in Sentiment Analysis
1:27:47 to 1:29:17
Understand the use of AI in analyzing sentiments around current events and trends.
“with things that were happening in the world.”
The Role of AI in Sentiment Analysis
1:32:18 to 1:35:15
Understand the use of AI in analyzing sentiments around current events and trends.
“It's an autonomous, offensive security platform.”
Pulse Check on Anthropics Fable
1:35:27 to 1:38:00
Investigate the current sentiments and reactions around Anthropics Fable's return.
“I guess it was useful, but it was really fun too.”
Anthropic's Fable Five and Governance Concerns
1:38:00 to 1:40:59
Explore the implications of Anthropics' new model amidst regulatory challenges.
OpenAI's New Strategies and Competitive Landscape
1:41:00 to 1:42:39
Discuss OpenAI's strategies and new product offerings in AI.
“It mentioned there are quite a few SQL injection vulnerabilities, which we knew.”
Legal Battles and Copyright Issues in AI
1:42:40 to 1:44:52
Examine the New York Times' lawsuit against OpenAI and Microsoft.
“The Times sued them back in 2023, saying they infringed on copyrights by using its articles to train it.”
OpenAI's Chip Development and Market Strategies
1:44:53 to 1:46:53
Analyze OpenAI's new chip development and its implications for the market.
“So they're already building the facility in Abilene, Texas.”
Meta's AI Strategy and Subscription Model
1:46:54 to 1:48:42
Discuss Meta's challenges and new subscription model for their AI glasses.
“So everybody who's renting out compute space for other AI companies are going to win no matter what.”
Sam Altman Film and Industry Reactions
1:48:43 to 1:50:41
Review the cancellation of the Sam Altman movie and its industry implications.
“week about amazon canceling the sam altman movie uh the movie about sam the period of time when Sam Altman was fired, which should make a great movie, by the way.”
AI in Investigative Techniques and Data Usage
1:50:42 to 1:52:00
Discuss AI's role in investigations and the ethics of data usage among companies.
AI in Archaeology: The Vesuvius Challenge
1:52:00 to 1:54:35
Learn about how AI is being used to read ancient scrolls from Herculaneum.
“One of the Doge kids, I think, was involved in that, actually.”
AI Design in Government Websites
1:54:35 to 1:55:45
Discussion on the effectiveness and aesthetics of AI-designed government websites.
Ford's Experience with AI
1:55:45 to 1:57:25
Exploring Ford's trials with AI replacing quality engineers and the subsequent reversal.
“And they just, I think it was yesterday, the administration legalized two, what are they called?”
Trust in Media: A Contemporary Issue
1:57:25 to 1:59:45
An examination of public trust in media and sources of news in modern society.
“Hey, you should have listened to the show.”
Trust in Media: A Contemporary Issue
2:00:04 to 2:01:31
An examination of public trust in media and sources of news in modern society.
Trust in Media: A Contemporary Issue
2:01:33 to 2:01:54
An examination of public trust in media and sources of news in modern society.
“and take complex actions across your entire organization.”
Speculations on AI Devices and Glasses
2:01:54 to 2:04:42
Discussion on the potential future of AI devices and the role of glasses in AI interaction.
“Suddenly you have to say things like, this is a forward-looking statement and it may never happen.”
Concerns Around AI Cameras in Glasses
2:04:42 to 2:06:00
Exploring the societal implications and privacy concerns of AI-enabled glasses.
“And if we don't like it, it doesn't matter if we like it or not.”
AI's Role in Adult Content Generation
2:06:00 to 2:07:08
Exploring the intersection of AI technology and adult content creation.
“What is clear is that glasses are perfect for AI.”
Mainstreaming of Vice Industries
2:07:08 to 2:08:34
Discussing how previously taboo industries are becoming mainstream and profitable.
Impacts of AI on Pricing and Economy
2:08:34 to 2:10:37
Analyzing the inflationary pressures brought on by the AI boom.
“Others were using the coding model because it was cheaper to run the next AI's general purpose models.”
Geofence Warrants and Privacy Protection
2:10:37 to 2:14:04
Examining a recent Supreme Court ruling on geofence warrants and privacy rights.
“these sort of petty vices, but they're the leading indicators of what, you know, new ways to make a ton of money.”
AI in Podcasting and Content Creation
2:14:04 to 2:18:16
Discussing the use of AI tools in podcasting and potential quality concerns.
“An individual has a reasonable expectation of privacy in records about his cell phone's location and police intrude on that constitutionally protected interest when they demand the information.”
Media Nostalgia and Transitions
2:18:16 to 2:20:00
Reflecting on media history and notable transitions in the industry.
“by cutting the words in the transcript, those sort of things.”
Transitions in Tech: Celebrating Lives and Legacies
2:20:00 to 2:22:36
Learn about transitions in the tech world, including the retirement of a notable figure and reflections on a late friend.
“I just wanted to point out that at the back of every Newsweek was a section called Transitions.”
Remembering Ohm Malik: A Tech Visionary
2:22:36 to 2:26:34
Explore the life and impact of Ohm Malik, a celebrated journalist and investor in technology.
“of people in our world who had mentioned something about Ohm was amazing.”
Upcoming Picks and Guest Introductions
2:26:34 to 2:26:51
The hosts introduce their guest and share excitement for upcoming segments.
Exploring Political Bias in AI Models
2:26:51 to 2:28:24
Discover a project that visualizes the political leanings of major AI models and what that means for users.
“It plots each model as a cloud showing the full spread of answers instead of a single point and it publishes the questions with scoring weights, tags.”
Understanding the Nuances of AI Bias
2:28:24 to 2:31:36
Delve into the complexities of bias in AI models and the significance of question framing.
“And that Google Gemini is almost exactly dead center on Yeah, on everything questions.”
Unique Podcast Picks to Help You Sleep
2:31:36 to 2:34:00
Explore an amusing podcast that reads boring texts to help listeners fall asleep.
“You see lightning strike in the distance.”
Compliance in Construction
2:34:00 to 2:34:44
Discussion about compliance requirements for building new towers and antennas.
“Building new towers or co-locating antennas on existing structures requires compliance.”
Legislative Testimony and Google Funding
2:34:44 to 2:36:01
Jeff Jarvis shares insights on California legislation and Google's funding for news.
“So, Google got – there was an effort to pass legislation to force money out of the platforms because publishers think that that was their money and we want it back.”
The Palantir Jacket Phenomenon
2:36:01 to 2:36:59
A humorous take on the Palantir jacket and its cultural implications.
“And a real French workman's jacket is actually more than$239.”
The Four Burner Theory
2:36:59 to 2:38:14
Exploration of the balance between health, work, family, and friends.
“It does not have the Palantir logo on it.”
Cotswolds Experience and Culinary Adventures
2:38:14 to 2:40:32
Discussion about travel experiences in the Cotswolds and culinary plans.
Food Culture in England and Scotland
2:40:32 to 2:43:56
A light-hearted discussion on food culture in England and Scotland.
“so this is uh i'm sure are you going to do 11th that's what i want here we go yes absolutely Second breakfast.”
Transcript
Automatic transcript. May contain errors.0:00Alex Stamos:It's time for Intelligent Machines. Jeff Jarvis is here. Paris has the week off. Mike Elgin sits in and we've got a great guest. Linguist Chris Potts specializes in finding AI failures. How to know if your AI is failing, how it fails, and what to do about it next on Intelligent Machines. This episode is brought to you by Black Hat USA. If you listen to this show, you go deep on the technical detail. Well, so does Black Hat. For nearly three decades, it's been where the security industry's most rigorous research gets presented and pressure tested. More than 100 hands-on trainings taught by practitioners who've actually deployed in live environments, not lecturers reading from slides.
0:42Alex Stamos:And hundreds of peer-reviewed briefings that go well past the overview into the real work across the four areas defining security right now. AI and autonomous threats, cyber conflict, systemic resilience, and identity. This year, Black Hat's Briefings Pass includes all keynotes and main stage access, plus business hall entry. You also get breakfast, lunch, Arsenal Live tool demos, on-demand session access, and admission to the Midnight in the War Room screening. Black Hat takes place from August 1st to the 6th in Las Vegas. If you want the depth this show gets into in person with the people doing the work, this is the room.
1:24Alex Stamos:And we'll be there, too. Prices rise on July 17th, so book before then. Use code TWIT for$200 off your briefings pass at blackhat.com slash US-26. That's B-L-A-C-K-H-A-T dot com slash US-26.
1:46Alex Stamos:Podcasts you love.
1:47Fable:From people you trust.
1:50Alex Stamos:This is TWIT. this is intelligent machines with jeff jarvis and paris martineau episode 877 recorded wednesday july 1st 2026 model now available it's time for intelligent machines to show we cover the latest in ai robotics and all the smart little doodads are getting smarter all the time i got this little guy he just he calls china and talks to it all the time i don't know what they're saying because it's in chinese but uh that's that's pretty smart uh welcome to the show jeff jarvis is here professor of journalistic innovation emeritus at the city university of new york and the craig newmark graduate school of journalism his new book is only weeks out now finally we're in july next month hot type in the hottest month of the year comes out the story of the line of type that you can order it now though at jeff
2:48Fable:jarvis.com paris is still jeff i have to steal a joke from a former student of mine lucy lee she said her dream job is to be professor emeritus and i agree exactly as i say it's latin for old it was not it's time for big sabbatical in my view let me just quickly welcome mike elgin who's
3:07Alex Stamos:filling in for paris martineau this week it's great to have you mike where what country are you in today? I am in the UK. That's Waltz. And tomorrow my wife and I are gonna take a road trip to Scotland. So super excited about that. All the all the FIFA fans will be back by then. You can come. Yeah, we're trying to beat them there over a little loose. Yes, you've already met our guests were thrilled to have Chris Potts on. He's a professor of linguistics at Stanford on sabbatical right now chief scientist and founder of a company called bigspin.ai uh but you know he's also and i think everybody should read his phd dissertation the logic of conventional implicatures because i think that will then you will understand fully what we're going to talk about no i'm just kidding hi chris welcome thank you so good to have you your timing uh is excellent What an amazing time we're living in.
4:10Alex Stamos:You originally were interested in neuro-linguistic programming, right? LLP or no?
4:16Fable:Natural language processing. That's an important distinction. The other LLP.
4:21Alex Stamos:Much more interesting, frankly. And natural language is kind of what we're doing these days with our AI. he was at sale the stanford ai lab uh created a co-creator of foundational open data sets the stanford sentiment tree bank the snli natural language inference copa corpus uh and a creator a co-creator of dspy which is the framework that uh kind of i think significantly reframed prompt engineering is actually coding so before there was vibe coding
4:55Fable:there was you and the visionary project lead for that omar katab my student with mateh zaharia
5:02Alex Stamos:yeah i give omar the credit there for sure yeah he saw the future you a lot of what we do these days with rag although rag went through really a brief period of uh of fame and infamy and now it's kind of i think a little bit uh deprecated but you were the co-author on uh colbert the uh what It was basically the original rag, right? Oh, yeah.
5:25Fable:Well, you could break that apart. It's a pioneering neural information retrieval architecture, which is a key component in many of the retrieval augmented systems. It would be the mechanism that would find passages to bunk into your context for the language model to use. Incredibly important piece. And in its own right, Colbert is an incredible contribution to information retrieval, which is, of course, the backbone for web search and other technologies like that.
5:51Alex Stamos:I didn't realize it was French. I mispronounced it. Colbert.
5:53Fable:Oh, well, no. So we could talk about that, right? And you could talk about what Stephen Colbert actually says about his last name. Omar happily accepts Colbert Colbert. I think he's happy with DSPi or DSPi or whatever variant, as long as you're using it, Omar is happy.
6:08Alex Stamos:Very, very cool. That's all linguistics. You just kind of let the cat out of the bag on this new company, BigSpin.ai. Tell us a little bit about what big spin is all about so we can get into the conversation here because this is fascinating.
6:23Fable:Sure. Everybody should know that a big spin is a skateboarding trick, an advanced one in which the board spins 360 and you go 180. And it involves some real danger. You've got to have faith that you're going to land back on your board. It's beautiful when it's done well. Wow. And I guess it also, for the skateboarders, references an old California lottery. So it also has an element of risk. That's right.
6:44Alex Stamos:That's right. Yeah.
6:46Fable:So that's the beginning and the end of the relevance of the name to the product of the company's mission. It's a very dense space and you don't want to choose something like prism or whatever, because then there will be 50 companies who have chosen the same name. So we just changed the game and went with something that has good connotations for us. The focus of the company is very important for people who use AI.
7:09Alex Stamos:What your, your, your research is fascinating. Go ahead.
7:12Fable:Oh, that's wonderful to hear. Thanks. Yeah. Yeah. Yeah, we are focused on making sure that people's interactions with AI are productive for them, no matter where they are in their AI journey. And in particular, what we're trying to do is help people in charge of these products figure out what's happening in that interactional sense, meet users where they are, and help empower them. And that all begins with finding all these low-level signals of small points of failure or figuring out what the user is trying to do in that session and then providing affordances that will help them do that and so forth and so on.
7:48Fable:There's an incredible opportunity here for customization. And there's just much more that we could be doing for these products. So we're trying to push that along.
7:57Alex Stamos:So your work started with analyzing a million chat GPT conversations. Oh, yeah, right. You're in those conversations. You found 78 % of AI failures leave no trace. People don't know.
8:14Fable:That's right in the sense that the user just did not give us an indication that they saw that something had gone wrong, even though something had gone wrong. Can I ask you a question there, Chris? Sure. Because I was fascinated by that. There's two sides to this. Does the user give a signal that it went wrong? Is the AI company generally set up to listen for that signal? and act on it, or are they using that kind of interaction to fix and train, or are they putting their hands around their ears? Well, that's funny. I tend to assume that they are smart, creative, motivated people, and they're doing very advanced things.
8:51Fable:But we did get a glimpse of at least one thing they're doing when the Claude code code base leaked, and embedded in there was a regular expression that was meant to detect user frustration. Did you all catch that? Yes. It's very profane.
9:04Alex Stamos:Yes.
9:06Fable:Because they wanted to know, right?
9:08Alex Stamos:They wanted that signal. That meant something went wrong. The user's going, God damn it! How many times have I told you not to do that?
9:15Fable:Which would be a very visible signal of failure. Yes. Yeah, to be charitable, I would say it's a very high-precision device, but very low recall. It's going to catch some instances of frustration, but it won't even catch all of the negative F-bombs that people drop because it's just very particular and not very complicated as an expression. But it shows that they're trying in that fundamental way to capture at least a subset of the worst kinds of failure. And that resonates with us because at Big Swin, we tried to write a similar pattern. It was much longer and tried to catch many more things, but it still only caught a fraction of the things that are actually going wrong in these interactions.
9:56Fable:and that's what motivated us to look for what we call the invisible failures.
10:01Alex Stamos:Is that my fault? Because I've been taught, and I think we've all learned, not to yell at my AI because it makes them perform more poorly.
10:09Fable:Well, in this case, you could be communicating to the Cloud Code team, and if you study the Regex, you can now know exactly which F-bomb to drop to possibly get their attention.
10:20Alex Stamos:But what I also thought was the case is that we can't really see inside these black boxes, that we don't really know what's going on inside the AI, right? So it's up to the user to flag the failure. The AI doesn't know.
10:36Fable:That's okay. The AI, in some sense, can do verification of its own outputs. That is an easier task. If you have it look at its own transcripts and try to find things that have gone wrong, it might do that at a higher rate of accuracy than you might have thought. And certainly if you have a more advanced model doing it, it can spot, for example, the contradictions, the failures, the mismatches between intent and response that the more primitive model was not getting right. And so that's a monitoring opportunity right there. And that's what we mean by an invisible failure. You ask a question, the answer that came back is not quite an answer to that original question.
11:14Fable:if the user doesn't signal there was a problem for all we know they're now running the wrong code or they're off with the wrong factual claim or whatever it is and i assume that product developers would like to know that that's happening yes that kind of use a bunch of prompts that i use a bunch
11:31Alex Stamos:of prompts that tell the ai to check its own work and in some cases i'll use a prompt that tells it to rate its answers on a scale of one that's 10 this works really great that way it comes back It sounds very confident and it says, well, I give this a seven out of 10 and I sort of dig into that more. There's also another tool, I don't remember what it's called, where they use two AI models and they use it like a GAN or something where one AI model checks the other one and they sort of go back and forth. And that seems to me to be kind of an obvious way to check these things. Is this how these things will be improved over time in terms of accuracy through sort of self-checking or one AI checking another?
12:17Fable:I think yes. And in fact, there's a few dimensions to that comment. The first is I just think it is productive to have them interacting in that way. We do that for PRs inside BigSpin. And when we do the annotation work that we do for BigSpin, that's on the back of us having these annotation protocols where lots of LLMs were critiquing each other and trying to find prompts that aligned their behavior so that we got at least consistent results. All of that is incredibly productive. I think the essence of what you said, Mike, is that this needs to ground out in something like true verification. And that partly explains why progress has been so fast for software development, where the verification step is often running the code.
12:57Fable:And that feels very within reach. And that though does point out some real problems once you step outside of a highly verifiable domain, even going to things like designing UXs, but certainly things like the legal system. Verification is not so straightforward. And then I think all our concerns about accuracy and hallucination just come flooding back in. And it's not so clear what mechanism we'll use to scalably address that. But if we can find those verification signals, we're off and running. I think that's the lesson of software development with AI.
13:30Alex Stamos:Yeah. So you're not exactly on the stochastic parrot side of the equation, saying that these models are dumb and useless and make mistakes and nobody even knows.
Read the full transcript
13:48Fable:Certainly not. That's a very complicated thing we could discuss in its own right. The resonance of that metaphor and what it means and what it implies about us and about the models and about their prospects and so forth. But certainly I think something very sophisticated is happening and there's incredible potential here. Whatever you think about the current moment, I think AI is going to continue to change the world.
14:11Alex Stamos:This is why I like you.
14:16Fable:I might be too centrist for you. I took a quiz this morning that placed me right in the most central centrist position you could imagine the most boring perspective imaginable i assume versus the other archetypes you could have according to this quiz which are really far out there in terms of risks and prospects and the excitement about consciousness
14:33Alex Stamos:i was just kind of in the middle well you know as a linguist i imagine you're very interested in lls absolutely oh yeah in fact one of the debates we have on this show uh uh jeff's kind of on the in the yan lakoon uh camp uh the feifei lee camp where he thinks that uh an llm is insufficient to uh amazing but not not quite a perfect i think the jury's still out i think what we're what's been fascinating to me is how far we've gone with language alone and and i'm not completely convinced that what we do in our own brains uh exists without language i'm not sure but uh i'm not convinced that an llm can't go a lot farther certainly than we've gone.
15:19Alex Stamos:Where do you come out on that?
15:22Fable:You'd have to say that a key to their success is that the streams of symbols that they process go way beyond language. Now they're full of log files and sensor readings and technical descriptions and other things that I would think are giving them an increasingly dense picture of what the world is actually like. And then if you did starve them of all of that and gave them only text from Wikipedia with no accompanying metadata or images or anything like that, they might be much farther behind because that's a very strange fragmentary view of the world we occupy. That's a fair point. But the thing I would want to pick up on is that it's completely eye-opening and remarkable to me that such simple learning mechanisms when scaled yield all these complicated behaviors.
16:09Fable:That is one of the most exciting scientific things that has ever happened. And I feel privileged to live through this moment of seeing this. And I definitely did not anticipate it. I thought that for these complicated behaviors, we would need something much more engineered, much more complicated, much more futuristic. Whereas the raw ingredients here have been known for a very long time in machine learning and back on through statistics in the early days of AI in some parts of them. Yeah. So where do you come in on explainability then on whether that's possible, desirable? That's been a major focus of my group.
16:45Fable:I'm a big booster on the idea that interpretability is going to help us improve these models and also get control of them. And we have discovered, I mean, this actually where my linguistic research and my AI research kind of dovetail, these models solve hard generalization tasks and do complicated things because they have developed very rich internal representations that in many cases are quite understandable to us and that explain how they can generalize so well. That's a remarkable finding. I think it's, again, unanticipated if you think back 15 years, but now it's been so productive and exciting.
17:21Fable:And that ties in with issues of linguistics and cognition, because then you can start to think, well, they're not like humans, but they have this human-like capability. Whatever mechanisms they have are at least sufficient for those kinds of behaviors. And that's a major clue when it comes to unpacking the human capacity to understand language and do complex cognition.
17:42Alex Stamos:That's just fascinating to me. We're talking to Chris Potts. He's a professor of linguistics at Stanford and the founder of a company called bigspin.ai, which is named after a skateboard trick. Can you do a big spin? I know you're a skater. Yeah.
17:59Fable:When I was younger, I could do a big spin and I have pledged to my team that I will learn to do a big spin, but it is a scary trick. You really have to have faith. I can still kick flip.
18:08Alex Stamos:My alleys are fine, but I know it's not what the company needs from me.
18:13Fable:My son is a skater boy too.
18:15Alex Stamos:It was fun to watch him fall a thousand times and then make it. It's amazing. It's a little scary also.
18:24Fable:Real life lessons there though about persistence. Yeah, absolutely.
18:27Alex Stamos:That's real world model time. It's helped him immensely. That's true. So in a way, Rich Sutton wasn't wrong with the Bitter Lesson. It is true that throwing compute at these things is surprisingly effective.
18:42Fable:Yes, I am inclined to agree. It's a bit complicated for me, though, because the Bitter Lesson is kind of like one of those lessons from writing manuals that just say, like, omit needless words. It's great as a reminder if you're already a good writer, but if you're not, it's just not helpful advice. And if you just say to someone, hey, just scale endlessly, they'll make lots of bad, expensive choices. And the real lesson of the era of scaling and so forth is, yeah, we scale, but we also learn tons about how these architectures work and find lots of ways to make them efficient. And the example I usually give is it's been a kind of minor miracle that we went from context windows of 2000 tokens to what might as well be infinite.
19:23Fable:We did not get that by scaling the transformer architecture from about 10 years ago. That would cost us literally trillions of dollars to get to that big context window. People thought very carefully about locality and language and how neural networks process and learn information and sequences. and they found ways to approximate the context window so that it could be scaled in that way. So yeah, you scale and you don't get too clever about thinking about all the details of language. But on the other hand, you need a really deep intuition about linguistic data to do that kind of scaling.
19:59Alex Stamos:So what is Big Spin going to do as a company? Besides the research, what's the product?
20:08Fable:That is a fascinating question for us because, of course, we have an app that has an agent and it will help a product manager as data stream in from their product, understand the issues that are arising and find things that are going well and help them with fixes. So it's incredibly empowering as a kind of supercharged chief of staff to the product manager, helping them spot all these things.
20:32Alex Stamos:So it's a way of seeing where your AI is going on.
20:37Fable:monitoring visibility. And then one incredible thing about the current moment is that the agent could suggest fixes that might help the product manager connect with their engineering team and so forth.
20:48Alex Stamos:Nice. But I have to say - So this is kind of what Mike was talking about, the whole idea of creating tools that help you find errors. Yeah. And feed it back to the AI. Yeah.
20:58Fable:I have to say, it's such an interesting moment for thinking about how to build a durable business in an era when anyone could take a screenshot of this app in action and say, hey, Claude Code, make me something like this. And that kind of shows you that the value of the raw software goes to zero. And so my thinking about this is, on the one hand, our agent is incredibly good at this job because of all the tools we designed and everything else. But the thing that really supercharges it is that we have all these annotators that run as the data flow in. And And they connect, they catch things like invisible failures.
21:33Fable:They do modeling of the user at their level of expertise, the task they're trying to solve, the domain they're in. And all those signals, which come from these models that we fine-tuned to do those particular jobs based on data that we've got, they supercharge the agent and make it able to do all that important data science. And an agent without all those annotations is really kind of flailing about in the general world of just what language models can do in general. just to be clear the annotators are people no those are models they're agents yeah automatic you could they're you could think of them just as classifiers this gets down i mean they're language models but you could think of them as kind of just like old school classifiers they assign hundreds of signals so they're not quite like old-fashioned classifiers and they are actually language models under the hood but they're very specialized to their task of identifying invisible failures and identifying user expertise levels, domains, all those things that I mentioned that are so critical to understanding where the failure points are and where things are going well.
22:38Fable:Who's your customer? Our customers have to be organizations that care about their interactions, which is not everyone who has a deployed chat bot. But if you're in an area like you're giving medical advice or helping somebody with scheduling of something that matters or like doing things like professional coaching, then the nature of this human AI interaction really matters to the success of your product. And the distance between off the shelf chat GPT and the product that you want is enormous. And every failure is a really important thing for your business. So those are our customers because those are the people who every day are going to sit down and say, what's going wrong and how can I fix it?
23:21Fable:Then shouldn't the foundation model makers be your primary? I mean, they should kill for your data and for your learning, yes? Rather than the application layer, the company that's using this at an application layer, right? So how far up the chain do you go? Well, I mean, data, I think, are key. I think that's the central insight there. I I assume that the frontier model providers have lots of their own data, you know, a super abundance of it. I'm kind of jealous of how much they have. But I do think that data are the key ingredient here. As always, that's a very familiar story in AI, that the data are the thing that gives you the transformative capability.
24:02Yeah.
24:05Alex Stamos:So in a way, it's an audit layer for companies.
24:08Fable:Yeah, I'm happy to think about it as auditing. That could sound quite specialized to people. you know, auditing could be a very particular role. And this is broader to anyone who's just in charge of the quality of their product. But I think what they are doing in part is a kind of audit. When we have escalations to a human, are they the kind of escalations that we like? If we see the system resetting in these contexts, is that good or bad? And then of course, they're trying to fix that.
24:32Alex Stamos:Now, everybody who's using AI, especially those of us who use it for coding and so forth should be concerned about invisible failures. Yes. Is there a way to detect those, to note what's going on? I mean, obviously, we're not your natural customer, but we would like that kind of visibility into what's happening.
24:54Fable:Right. Well, one thing I'll say there is that it's a characteristic of expert behavior with AI that you make your failures visible. Experts complain. They push back. They iterate on goals. They refine goals. They tell the AI to change course. And we all take that for granted, I'm imagining, because you all are at the cutting edge of using AI, I suspect. But for the vast majority of people using AI, they've been told it's a super intelligence. They ask it for things and they take the responses at face value. They adopt what we call a delegative mode, whereas what you want is an augmentative mode.
25:29Fable:It's the result, causal claim here. It's a result of the augmentative mode that people are able to solve harder tasks more reliably.
25:38Alex Stamos:And those who just use it as a chatbot are the ones who complain most about hallucinations without actually fixing.
25:45Fable:That may well be. Yeah, there's probably interesting interaction effects with domain and so forth. But if you just enter a query and get a response you don't like and walk away thinking, well, that was just wrong, that would be delegation with a bad outcome. Even worse, of course, is to walk away with the wrong answer as though it were correct. And believe it. Yeah. But in both cases, what you wanted to do is say, you know, as Mike was saying before, could you double check that or open up another window and just ask it to give you the opposite judgment? Hey, you really like this idea. What's the most critical take you could offer of it?
26:19Fable:And synthesize across the two. That kind of very critical mode is what experts are doing. So I covered, go ahead. All right.
26:27Alex Stamos:Well, all right. I can monopolize, Chris. I don't want to. So please stop. Let me go on that one if I go for a second.
26:35Fable:So you remember the schmuck lawyer in New York who used ChatGPT very early on and got citations. And I went and covered his show cause hearing in federal court. Fascinating. And it was interesting because his defense was, I thought this was a super search engine. Yep. I thought computers couldn't make mistakes. Right. Right. and then but what was telling though is that he obviously was suspicious because he went back and he asked chat gpt are you sure about this and chat gpt said absolutely uh this is early chat gpt so in there there are all kinds of signals of what was happening and and i'm curious how what
27:17Alex Stamos:are the kinds of signals that you see for an invisible failure oh that's what are the things segues into the question i was going to ask because in the paper you have eight archetypes for failures so the one you just described i think would be the confidence trap where the the ai is and we've seen this we've all seen it confidently wrong and that confidence we believe it we go oh well it was it's pretty darn confident uh those sources i asked i asked it gave me sources that must be uh that's that's a big problem but you also have uh some other archetypes which i think are great the drift archetype where uh ai sort of gets your goal but not quite it's off a little bit
28:01Fable:right that might take you on a journey away from where you intended to be right it can be subtle
28:07Alex Stamos:that's actually the most in the in the corpus you were looking at that was the most common failure. Yeah, maybe that and the walk away.
28:15Fable:Yeah.
28:16Alex Stamos:What's the walk away?
28:17Fable:That's where you ask a question, you get a response. The response is not a resolving answer to the query. And that's all we get. But you can get walkaways later. So another pattern would be like the death spiral. You try a few times, hey, do this. Maybe you rephrase it. And then you walk away because none of the times was quite what you were looking for. And you don't complain, you just keep trying and then you bail. Is that spiral also a skateboard trick?
28:46Alex Stamos:Let's hope that it's the last one you do. I actually have had a, I think, I think I told the story last week. I had that death spiral where I kept trying, kept trying. Finally, I said, and this was a mistake. I give up and, and walked away. But instead of the AI giving up, it deleted all of its work and it deleted everything and and i said what are you doing and said well you said you gave up so i just thought i'd just delete everything oh wild oh i hope you capture these behaviors that's fascinating
29:18Fable:a little too autonomous literal well they're getting more autonomous aren't they in fact
29:23Alex Stamos:that's one of the things fable is is is doing is is able to kind of keep going everybody's talking about loops these days the idea that well you don't just ask it to do one thing you say go ahead do it and you loop and loop and loop that makes me very nervous there's no opportunity there for you to interject a correction or a course correction yes obviously that could spin out of control very quickly and get very expensive as well yeah uh there's the silent mismatch a user asks for x gets y and says ah that's close enough uh this you say this is rampant in uh software and education i've i've i have to say i've done
30:07Fable:every one of these yes me too i have a wonderful story about this i try to be an expert and be augmentative i had a colleague he designed this fun infinite runner game if you 404 at our site you get to play ollie not found where the skateboarder just you click the space bar and jumps. And I wanted to play a prank on my colleagues and have write an AI that would play forever. And I would just say, Hey guys, you know, I got 10 X the best score you've gotten. I'm the best player at all you've not found, but I was going to have the AI do this. So I said, Hey, Claude, write a perfect, you know, player for this game.
30:40Fable:And it said, I remember so distinctly, I understand the geometry of the game perfectly. Here's a solver that will run forever. And I say, great. And I try it and it's worse than me. So I go back and I say, this is worse than me. And it says, oh, you are quite right. You're so correct in this insight that it's not good. Here's a version that's much better. I run that one. It's a little bit better than me, but hardly changed. And it just kept cycling through this confident assertion that it had the perfect solver, and then disavowing all of it and starting again over and over. In the end, I gave up.
31:16Fable:Maybe I'm not expert enough. I still don't have a solver that's perfect at this game. I don't know whether it's achievable. My question was never answered, but I was really caught in the death spiral and the contradiction unraveled there.
31:28Alex Stamos:And you walked away.
31:30Fable:I walked away. And it was the perfect scenario because I don't know how the game works and I don't care to learn how the game works. I wanted to offload this to AI, but I had the advantage that I could let the game play. that's my verification step and i could see that i got 42 and this thing is only getting 36 it's not better and that is far from perfect try again but in domains like the legal one where you don't i mean what would be the equivalent like going to trial in that case
31:58Alex Stamos:and then finding it was wrong yeah it's too expensive where are we going to do the verification step there uh what if what i i just think this i think your sense of wonder and excitement at this is i i share exactly we are in a very interesting and strange time uh today anthropic re-released fable and actually the word classifier has become part of a vocabulary since it released mythos fable was a mythos that had a bunch of classifiers that were in theory going to keep it from doing anything bad. Uh-huh. What they find bad. Yeah. We'll talk about this more later. What they've done is they've stepped those up.
32:39Alex Stamos:So it, my fear is it's not going to do anything at all.
32:43Fable:But it's very hard to calibrate. Yeah. There was a quiet revolt from the AI community, which had a real impact. It was mostly on X and it was AI researchers saying, I feel betrayed by this. And they walked those back. Yeah. Yeah.
32:56Alex Stamos:Well, it's as if you're peering over a wall at some magical nirvana, but the wall keeps stopping you from getting into that secret part.
33:06Fable:And it erodes trust. Yeah, people started saying, look, I don't know what's happening, but I'm going to use a different model because I don't want all of my responses nerfed. I'm trying to do real research here.
33:16Alex Stamos:Exactly. Well, that's the real fear. And actually, Alex Stamos talked about this today on Twitter. He said, why would any company invest in Fable with the risk that in the middle of the project, Fable just says, no, yeah, I'm not going to do that. It's just not worth it. So, well, good luck with the new company. I think this is very exciting. Can I hit the other topic?
33:42Fable:Yeah, yeah, yeah. I'm curious about, because you're a linguist, the debate about understanding and all that follows understanding. Unto consciousness and everything else, right? Sure. But to start with understanding, who was it, Leo? You sent me that long video of... Jeffrey Hinton. Jeffrey Hinton, who was arguing that it's obvious that they understand, and then that he argued that it had a desire to lie to him, which also obviously implied that it understood what a lie was. So I'm curious on this, if you were taking a test as you took this morning, on this topic, where do you land? Probably right in the center.
34:29Fable:You have to be open-minded because anything else is way beyond what we know scientifically about how humans are doing this and in turn about what's in principle possible. If you talk about things like beliefs, desires, and intentions, We don't know what's necessary and sufficient in humans for this. We rely on an assumption that people are like us. And all those philosophical problems come flooding in as soon as you say that. But we navigate those things. But they're uncertain. And then we also have very little understanding of what models are currently doing now. You know, the interproject is far along, but there's endless things still to learn.
35:02Fable:And we especially don't know what the models of tomorrow are going to be like. And if you did just think, okay, understanding is a loaded term, but what it's going to mean to be meaningful is that you have some kind of mapping from language into some conceptual structures. And so that like, you know, we map language into mental representations of things, and that's what it means to understand. And what that puts you on is a continuum. How complicated is the conceptual structures? How complicated is the mapping? How refined and so forth. And obviously language models way behind us along many dimensions in terms of how sophisticated that mapping is.
35:39Fable:But if that's all there is, then it looks like nothing is stopping you from having a model in the future with exactly these technologies that has a very refined mapping of this sort. And if that's not enough for you for understanding that kind of semantics in that deep sense, then it's on you to tell me what's missing from that picture. And then sometimes people reveal that, like, actually, they're kind of biologically oriented. So that's a dead end because there's something intrinsically biological about understanding. And it's just good for people to confront that and maybe realize that about their beliefs.
36:13Fable:So if you were in the studio together, Leo would hug you right now. I would. Because this is what he argues. So I'm editing a new book series for Bloomsbury Academic called Intelligence, AI, and Humanity, where AI forces us to reconsider things in life. Ruben Chowdhury is writing a book about intelligence. What is intelligence? Look at the history of intelligence. You as a linguist, does AI force us as a whole or even you individually to reconsider our prior human definitions of understanding? Yes. And I feel like whatever your reaction to this and whatever your beliefs, if this moment is not causing you to reconsider all those things, then there's something amiss.
36:57Fable:Because this is the first time in human history that we have encountered other non-human creatures that can do all these things. It is definitely weirding us out. But if it doesn't have you pause and say, look, I need to critically assess what it meant to be an understander or critically assess what it meant to connect symbols and language in the world. if you're not pausing, even if your response is this is all beside the point because they're too different from us as humans, it should still be a moment of serious reflection. So did it cause you as a linguist, linguistics scholar to change any views that you'd had before you encountered all this?
37:35Fable:That's a great question. I will say that it has been empowering in terms of making progress on some of the most difficult problems in linguistics. And the two that come to mind for me are what's often called the poverty of the stimulus. So how do we, with apparently so little input from the world, get to a full competence in language so quickly? And the Chomskyian answer has been, you have rich innate priors, but of course, language models get there pretty fast without any innate priors. So you don't, the innate priors aren't intrinsically necessary to achieve this. They might be given human limitations, but you see how nuanced this is getting now.
38:10Fable:And then there's a related question of what's a conceivable human language? We have only a finite number of them that we've ever encountered in the world. They're all a product of history and accident. What is the abstract cognitive capacity for language? What set of things is learnable by us? Very difficult problem to address experimentally, but very easy if you're thinking about training language models on different corpora, representing different languages, and see what final state they achieve. So two big questions unlocked. And that is just incredible from the point of view of new debates, new discoveries, new terms for these things.
38:44Fable:And I didn't think, again, that we would have a new investigative tool like that in my lifetime. And I thought then those questions were going to be kind of stuck where they were.
38:53Alex Stamos:And you see, there's some evidence that these AIs can create their own internal language. That's what I was going to ask.
38:59Fable:I mean, at some point, Leo, before we got on, showed us a dial, a conversation among agents without humans. At some point, does one, and you talked about where that might go, the void, does that potentially go to them inventing their own language? Another fascinating question. That used to come up more about 15 years ago when we did more training from scratch. Now that all the best models are pre-trained on the internet, which is a record of actual human usage and everything else, they don't have as many opportunities to go off that distribution. And so it's less likely that they're going to invent their own language, but given sufficient interaction, and maybe if they do start doing weight updates as part of these interactions, then you could get into some really far out states.
39:44Fable:And that would be fascinating to see what kind of more efficient systems they might evolve or systems that are differently pragmatic than human languages, or maybe this would be the most exciting for me, they converge on kind of human-like systems at the level of the pragmatics and the encoded meaning.
40:00Alex Stamos:Which, of course, human language is always evolving and splitting off into different languages and dialects and so on. And you could imagine isolating AIs and have them talk to each other at high speed for a large amount of time and see if they are. That's another one, yeah. And I'd be really curious if that happens. You could also imagine constructing a new language from the various grammatical rules, vocabulary, German style, plugging everything into a single word. You could imagine all kinds of things from human languages that exist already.
40:34Fable:And sped up, right? You don't have to wait actual human generations to see what's happening. And again, they're always going to be qualifiers, but what an investigative tool. Yeah.
40:44Alex Stamos:I also think you talked about a bridge that we may cross sometime. I hope we cross it in my lifetime where they are self-improving. They're able to change their weights. and that might actually be when things get explosive.
40:59Fable:Yeah, and there are no technical obstacles to that now. It's just a matter of calibrating those processes and then actually running them at a technological level. It's very expensive to do all those weight updates, but that shows you the potential because in principle, we could do it now.
41:12Alex Stamos:Interesting. So it's just a cost issue. It's getting enough NVIDIA Blackwells together to vera rubens together.
41:19Fable:Yeah, and also just fine-tuning that process, which across all of these training processes is kind of at this point more art than science. And that's why people get paid the big bucks to do it because it's a lot of lived experience to figure out how to set it up in a way that it goes well as opposed to going poorly. And when the price tag on it going poorly is in the tens or hundreds of millions of dollars, you hope you have experienced people running it.
41:44Alex Stamos:Chris, such a pleasure to talk to you. This is so much fun. I really enjoyed this.
41:49Fable:Real quick, Leo, I know I usually don't talk to the guests during this part, but I have a question.
41:54Alex Stamos:Benito, our producer, wants to ask you something.
41:56Fable:Because we rarely ever have a linguist of his caliber. So is there any kind of qualitative difference between a model trained on English and a model trained on Chinese? Oh, good question. In terms of the internal representations? Yeah, or anything at all. Are there any qualitative differences? I mean, are you thinking of a scenario where we train one model purely on English and another purely on Chinese? I guess the question is more like, is a Chinese-trained model qualitatively any different from an English-trained model? Is DeepSeq fundamentally different because of the language itself? Because of more Chinese.
42:34Fable:And does the language itself have any kind of intrinsic quality that would be different from an English model? Oh, it must be, right? Because the units can be very different, especially from the point of view of the language model. Your tokenizer might be different for English and Chinese, And that's going to have amplifications for how it reconstructs those partial words into more meaningful units internally. And then, as I was saying before, if you think that what it's doing is partly inducing a mapping from the language into concepts as a way of solving the hard generalization tasks that we pose for these models, then that conceptual structure could be very different.
43:14Fable:And there is work on this and you do get these fascinating things that, for example, a model trained dominantly on English, but secondarily on Chinese, when it speaks Chinese, it might do things like using color terms in a way that looks more like English. maybe it overuses a word like orange or something because that's a lexical item a frequent one in english and it's more marked in chinese but this model has kind of had one set of experiences bleed into another causing it to have a different conceptual structure arguably and then of course you can ask yourself what about for bilingual speakers are they showing similar kinds of things and again you just see the power of this potential investigative tool here i know why you're
43:59Alex Stamos:interest in this, but you know, it's bilingual or at least bilingual.
44:02Fable:Yeah. And when I do switch languages, I do think differently.
44:05Alex Stamos:Yeah. Interesting. Is it not the case though, that all these models are converging because it's basically the same corpus training corpus for everything? I mean, or is it not? Is it? I don't know.
44:17Fable:It depends. I suppose. So for the pre-training, it does seem like everyone is just getting all the data they can. And that might be quite homogeneous. For the post-training, it seems clear to me that, for example, the path that Anthropic uses to train these new products, which I think many of them start from the same base model, they've got that really worked out so that they can maintain a kind of personality that they want, even as they give the model capabilities. This is quite striking. I would love to understand more deeply how they achieve that. But the stability they've achieved for their product is different from the one that you get from chat GPT, for example, and certainly different from deep seek.
45:00Alex Stamos:Jim O' Experientially, I could confirm that. And it's one of the reasons people become fond of certain models because they like the personality of that model. Very interesting. Chris Potts, such a pleasure to talk to you. Chris's startup, bigspin.ai, if you're interested in making sure that your AI models are not
45:24Fable:failing you maybe you should check based on this discussion my tip of the day you all would love the void by nostalgia brist it's this epic essay exploring how we ended up with the models that we've got why they have the personality that they have and then lots of interesting thought experiments and observations about the culture embedded in is it is a it's a movie no it's a 17 000 word blog post oh and is this less wrong is it from those guys it is he posted it actually on tumblr there's a link from less wrong so that there could be discussion there but i don't know who nostalgibrist is in the world but he's probably a fascinating character he or she
46:03Alex Stamos:uh i am going to read this it is on tumblr how odd highly recommended it's quite a journey all right all right chris be careful on that skateboard out there yeah that's good advice thank you such a pleasure thank you so much for coming on i i can't wait to see what you're up to next and And if you ever want to come back and talk about it, please, we would love to have you. Chris Pottson.
46:26Fable:I'd be happy to do that. I really enjoyed this. Thanks again, everyone. Enjoy your sabbatical.
46:30Alex Stamos:Yes. No kidding. We're going to take a little break and we'll continue with Intelligent Machines right after this. Oh, he's gone. All right. Wow. I could spend out. I love linguists. And this is up their alley. Not every one of them. Not all of them. There's certain ones. We know who they are. We know their names. All right. let me let me get the ad and then we can talk uh this episode of intelligent machines is brought to you by zscaler the world's largest cloud security platform you know we can i mean you listen to this show you know the potential rewards of ai are huge especially for your company you can't ignore them because the competitors the competition isn't right but you also should pay attention to the risks there are lots of them loss of sensitive data sometimes inadvertently but then there's also attacks against enterprise managed AI and then there's the issue of generative AI giving new tools and opportunities for threat actors things like creating incredibly effective phishing lures like that writing malicious code automating data extraction AI is changing the entire security space there were 1.3 million instances of social security numbers leaked to ai applications last year and most of that was inadvertent right somebody uploads their uh their tax return it's all in there right uh chat gpt and microsoft copilot saw millions i the number uh is a lot at least three million data violations and again inadvertent stuff leaks but that's why you need a modern approach you need zscaler zero trust plus ai because it's zero trust it removes your attack surface it secures your data everywhere but with the addition of ai it can also safeguard your use of public and private ai protect against ransomware protect against ai powered phishing attacks but you don't take my word for it just listen to the customers they He loves Zscaler, like Siva.
48:35Alex Stamos:He's the director of security and infrastructure at Zuora. They use Zscaler. This is what he has to say. With Zscaler being in line in a security protection strategy, it helps us monitor all the traffic. So even if a bad actor were to use AI, because we have a tight security framework around our endpoint, helps us proactively prevent that activity from happening. AI is tremendous in terms of its opportunities, but it also brings in challenges. We're confident that Zscaler is going to help us ensure that we're not slowed down by security challenges, but continue to take advantage of all the advancements.
49:10Alex Stamos:Thank you, Siva. With zero trust plus AI, you can thrive in the AI era. You can stay ahead of the competition. You can remain resilient, even as threats and risks evolve. Learn more at zscaler.com slash security. That's zscaler.com slash security. We thank them so much for the support of intelligent machines. mike yelga it's so good to see you filling in for paris uh mike will uh uh be in the beautiful uh area uh of uh of england for a little bit and then up to scotland that's right we're at a delightful afternoon tea today is it burning hot though no not here we were just in provance we uh five days ago four days ago it was pretty hot in most of france it wasn't too bad in provence and in fact that actually had some rain while we were there, which is really interesting.
50:05Alex Stamos:But no, it's super pleasant here. 73 degrees, blue skies, puffy white.
50:10Fable:So you may not have gotten the full, since you're going to Scotland, actually, you may not have gotten the full sense of it. America fell in love with Scots. Oh, we did. During the World Cup, in New York particularly. They were just great.
50:21Alex Stamos:Boston, too. They actually drank all the beer. I saw that article. They drank all the beer in Boston.
50:26Fable:In Miami. And the great thing that they did, They somehow took two traffic cones and crowned every statue they could find with a traffic cone.
50:35Alex Stamos:There's a reason for that. I saw that here yesterday, actually. There was a magnificent statue of somebody, and it had this, like, jack-in-the-box traffic cone hat on it. And I'm like, huh. I found out because we had Ian Thompson on the show, and he explained the traffic cones. There is a very famous statue in Edinburgh. uh i said of lord nelson i can't remember who it's of but oh it's duke of wellington sorry it's in glasgow and they cannot the authorities cannot prevent the scots from putting traffic cones on its head on its horse's head different cones for different you know holidays and so this is a tradition in edinburgh but it spread when the scottish fans came to the united states uh they realize we have traffic cones and statues as well there's a wonderful young journalist for
51:28Fable:for the scotsman named katherine hay who did great videos of in boston and miami and there's what i love she just was going around miami and just seeing where they're there they've been through the traffic cones you can see traffic comes in all the statues i love it that's a great
51:42Alex Stamos:tradition i love it so the big story uh uh today is that fable is back so is mythos for some for the same people who got it before no okay so this is what's interesting so apparently we're finally getting the details from anthropic itself uh anthropic did give mythos back to a number of the same companies through glasswing that had it originally back in june 26th and we'd seen kind of noises that some companies still had access or had gotten access to mythos It was June 9th when the Trump administration, through the Commerce Department, blocked access to both Fable and Mythos, saying it was a security issue.
52:26Alex Stamos:We are now seeing Anthropik's response to all of this. And even though they're being very careful, you can read between the lines. Fable is back to, as of like about an hour or two ago. but they have really turned up the jailbreak protections maybe to the point where people i haven't yet kind of put my finger on the pulse of it but maybe to the point where people are going to be upset my that's my prediction uh who will they blame that they'll blame anthropic um but should really they should blame the u.s government anthropics doing what they can uh to appease the trump administration so uh i think anthropic kind of threw some shade and i'm not alone by the way alex stamos agrees in fact maybe i should best best way to recap this is to put alex stamos's tweet up on the screen because he he says there's a lot to unpack here anthropic is burying some hard truths in careful political language uh first of all anthropic verifies none of the jailbreaks provided a capability beyond what many other models including the chinese models could do that was when alex was on last week and uh we were talking or was it two weeks ago two weeks ago and we were talking about his letter signed by hundreds of the best names in computer science freefable.org that was one of the main critiques is it's not doing anything that other models couldn't do in fact anthropic pointed out that even haiku it's it's dumbest model could do the same exact jailbreak that amazon fingered them for which is a dangerous thing to say because okay then ban them all uh yeah maybe uh i think that they anthropic wouldn't have written this if they hadn't some confidence at this point that they had appeased the administration we don't know all the details of how they appeased it it's my theory that they offered them 10 of the company and other things but anyway they said when it first of all i think they cast shade on amazon uh the export control directive on june 12th came after the government became aware of a report in which amazon researchers had found a method of bypassing fable fives safeguards over the past weeks we've worked closely with the government other partners including amazon to review the report and evidence our testing confirmed that many less capable models including opus 4-8 gpt 5-5 the chinese model kimmy 2.7 could identify the same vulnerabilities as fable 5 did in the report when it came to the demonstration of how to exploit the single vulnerability and that was the things that that really scared the trump administration it made an exploit every model we tested could produce the same demonstration including haiku four five sonnet four six opus four six opus four seven open four eight gpt five four gpt five five and kimmy 2.7 so they say the reported technique did not expose any unique mythos level cyber capabilities so that this isn't that judicious that's that's pretty clear they busted us for something it's just evidence that that that you know, the whims of presidential administration is not the best way to go about this sort of thing.
55:48Alex Stamos:And the fact that, you know, the Trump administration wants to buy parts of companies, wants to be heavily involved in deciding who gets to see picking companies that get to use it and that sort of thing. This is really terrible. It's just amateur hour and it's vaguely totalitarian, the definition of totalitarianism by the way is when a government sees every single area area of human life to be within its uh province yeah it's yeah exactly so so this is this is a this is not totalitarianism but it's it's a oh it's damn close so so mike two things there um one i just want to
56:29Fable:quote benedict evans uh newsletter this week uh this is a mess with random unqualified officials banning and unbinding products with no process or transparency one has to laugh at anthropic and the safety activists air quotes around safety who spent years saying that they wanted restrictions but when they came they said no not like that alex stamos said casey c-a-i-s-i
56:53Alex Stamos:the center for ai standards and innovation is the group that's supposed to actually make these determinations not the political actors in the white house uh casey was positive on the prior safeguards the implication is that this whole thing was unnecessary yeah it also is alex called it an own goal he's uh he's a sports fan uh a goal scored against yourself because he said what's going to happen as a result is u.s labs now have to make a much more conservative precision recall trade-off on cyber refusals u.s models become much less useful for defensive cybersecurity work unless you're in the trusted group security companies and startups that provide services to others will now be driven to use chinese models big win for prc labs this month it pushed me little old me and i'm not a bellwether but i think if if as an individual my reaction to this was well i guess i better not be dependent on american models because they could rug pull us at any time pushed me to a investigate local models more i've actually found a pretty good one we'll talk about that later but b to frankly use some of the chinese models they're very very good and that was the other thing people discovered how good glm and deep seek and kimmy are they're good and they're based on open source and they're open weights you can if you had enough machine which i don't but you could run the full glm locally you'd need 512 gigs of
58:27Fable:which which consider the china china's goals are political more than economic um they can destroy
58:33Alex Stamos:our ai industry well and that's the question people say well why would china give this away you just named you just said why uh they don't need to make money on this no and and and you
58:44Fable:know the other thing that strikes me is that if they ever want to god god forbid this happens if they ever want to go after Taiwan. Now's the time, right?
58:53Alex Stamos:Because they'll absolutely cripple the technology industry around the world. Right. Well, they don't even need to because guess what? The technology industry has crippled itself because of demand for RAM and soft hard drives, SSDs. You can't buy them anymore. Everything's gone up in price. Apple raised its prices significantly this week. Every other company has done the same. and even with that higher price you can't get the amount i could not buy a mac anymore i could have a year ago that had enough memory to run glm i can't now because they the most memory they sell on a mac studios is i think it's ironic that the primary
59:34Fable:impact of ai on the economy that's going to be most felt is going to be the inflationary impact
59:39Alex Stamos:of the shortage of memory yeah for sure just as effective as uh as invading taiwan if you ask me
59:47Fable:So Mike, I had this discussion with Leo online. We had Olivier Sylvain from Fordham Law School on last week, and he was talking about the history of regulation of radio. And I went deep. I read his dissertation. This was fascinating, by the way, Jeff. Thank you for sharing that with me. Yeah, if I can do just a second on this.
1:00:07Alex Stamos:Yeah. So things could have turned out differently in broadcast.
1:00:13Fable:But the reason that we ended up with the regulatory and economic regime we have is because the U.S. Navy intervened and was worried about ship-to-shore communication and insisted on the creation of RCA as a patent trust that involved all the companies. And he banged their heads together and said, you've got to get along. You've got to do this. And then the government had the Navy had a seat on the board at first. and it's not hard to imagine, to your point, Mike, that by the time Trump says, I want a piece of OpenAI and I want a piece of Anthropic and I want a piece of this company and that piece of that company, that you can see them creating the RCA for today.
1:00:56Fable:And the next piece of where this goes is that it was the, what was great about Olivier's dissertation is the argument at the time and the reason for the creation of the FCC was that without it, there would be chaos. Without it, everybody would pick their own frequencies and nothing. So we've got to create this. And that was, as it turned out, BS. That was there only to convince the legislators to pass the 1927 regulatory law that created what would become the FCC. And that would enable, by the way, the restrictions of our language on broadcast. that would slice out the First Amendment for broadcast because of this argument of chaos.
1:01:40Fable:And so it's not hard to bring that to today and say that the U.S. government, having now intervened to this, I think, extreme impact of pulling products off out of the world, out of Leo's hands, no, candy from the baby's mouth, that we could see Trump getting it in his head. thank goodness he doesn't watch the show to create the conglomerate uh rca of ai and force everybody to put their patents in and their intellectual property in and it becomes run by the u.s government what i've loved too there's not going to happen in the rest the rest of the world would revolt is that the threat then in the 20s was immigrants exactly exactly that's the Where have we heard that before?
1:02:29Fable:Marconi, Marconi being British and Italian, that GE was going to sell Marconi a transmitter device. And when Herbert Hoover, who was then head of commerce, found out and the Navy found out, they put a stop to it and instead created RCA. and instead then required RCA to buy the American assets of Marconi so that Furners couldn't control our broadcast because it was a strategic asset.
1:03:00Alex Stamos:Yeah. So Stamos goes on to say, Anthropic is saying between the lines, Amazon's inability to appropriately communicate severity threw our industry into chaos. I don't know if that's exactly what Anthropic is saying. They say there needs to be a consensus framework in the AI industry for the severity of an AI jailbreak. We cannot agree on the severity. And Amazon way overestimated the severity of this. That scared the Trump administration. So they're lobbying for some sort of way to do this. I'm not convinced such a thing exists. i don't yeah i also think that anthropic way over uh stated the the the power and and danger of myth mythos um you do or do not think they did i do i mean i think i think that they scared people for sure it felt like it felt like a kind of a marketing stunt right oh yeah to get people like wow this thing is so powerful when this thing is available i want it and uh and so So, but it all points to the same prescription, which is that we need a good governmental agency that's nonpartisan, that's not about grabbing power, that's not about hyping threats, that's not about being crazy.
1:04:20Alex Stamos:And that basically can, you know, we have so, we used to have so many great agencies that would shepherd the industry through these things.
1:04:32Fable:yeah you see but mike that's that's the argument that was made to create the federal radio radio commission which became fcc and i'm a believer that the fcc has done a lot of bad things especially about our speech and so i'm going to sound libertarian i'm not a libertarian i'm a plain old democrat but i'm going to sound libertarian for a minute here i don't know that i want that agency i don't know that i trust that agency i fear what it will do out of a position of ignorance as it as as the government just did well i don't think they'll find the right experts and look what the FCC is doing today and how awful they are.
1:05:03Alex Stamos:But AI is a speech issue. Clearly, the FCC is around speech. It's all about speech and the First Amendment. But AI is also a cybersecurity issue, right? So look at the role that CISA has played in the last couple of decades or whenever it was founded in sort of shepherding and sort of protecting the nation and the companies and getting the cybersecurity industry singing from the same hymn book, it was a fantastic benefit. And I think, yes, the government agencies that exist to grab free speech powers is problematic, but we already have a situation where we have the federal government sort of meddling with and asserting itself as the decision maker in terms of who gets access to which model, etc.
1:06:03Alex Stamos:And then always, you know, basing it on national security, etc. You can always do that. We need a steady hand of nonpartisan experts, an agency that looks at all this stuff and can give us some rational, systematic evaluations of all of these claims that are made by the industry, by governments, by foreign governments, and by everyone else. And right now, there's just this huge void, and anybody can say anything. And in the case of the presidency, the president can do anything. And so that's the problem we're talking about right now is just this sort of wild west where nobody knows what's going on.
1:06:50Alex Stamos:Nobody's really in charge. And the people who assert their power over this thing have suspicious motives. It's really a big problem given the power of AI. Exactly what Alex winds up his post on Twitter. I'm sorry, X with. He says - Twitter, don't call it X, it's Twitter. Yeah, no, it's X. I don't like Twitter because it's - Oh, that's right, you don't, yes. You know why I don't like it. yeah i'm twit we predated twitter yeah sorry can we just call it i got the memo i got the memo yeah no it's fine everybody thinks of it as twitter they still call them tweets i understand we give alex tweets we give the u.s government huge powers this is why you staff it with competent calm non-corrupt people who don't use those powers to punish enemies the only upside i could see from the whole mess is there's a whole bunch of VCs with former or current administration affiliation who we can now safely ignore on AI policy.
1:07:52Alex Stamos:They've shown everything they've ever said. I think he's talking about David Sachs on AI regulation was just politically motivated. It's an own goal is what Alex says. And I think that that's pretty clear. We're also, I think, going to see once people start messing with Fable, that it isn't really very useful. It's been it's been in fact anthropic says we had to turn up the classifiers so hard that you may find that as you're coding it just drops down to four eight uh let us know if that happens we'll do we'll do the best we can but i think that this is a nerfed this is going to be clearly a nerfed model and
1:08:32Fable:again it's it's the lack of so one of the stories that i put in the rundown i don't think you have this one but it goes up from something you speculated about last week leo and i kind of laughed at you but you're right austria is talking about playing host to anthropic yeah and i don't know if that means merely hosting the software or or say moving the company over i i wouldn't move
1:08:52Alex Stamos:to the eu to be honest if i were anthropic uh i'd go to belize or somewhere with a very uh what would you recommend mike your world traveler somewhere somewhere where argentina somewhere where the government really just wants the money madagascar uh low labor costs and high a fast internet believe it or not and really madagascar yeah really i'm not i'm not this is not a serious proposal but uh malta maybe uh yeah somewhere yes somewhere what was it that was headquartered in
1:09:25Fable:iceland when it was when was it uh it was wikileaks oh yeah quarter in iceland freedom there yeah yeah
1:09:31Alex Stamos:yeah i think maybe maybe you would be the arctic circle somewhere where that we have a you can cool data centers what did larry page want he wanted one of those islands google island that are that are made out of old uh oil rigs in the ocean international waters that's the right answer international waters that's no i you know the good news is the really good news is uh not for the u.s but for us as users that china has got a lot of open weight models that are very very capable this has just stimulated i think development of competitive models this is an
1:10:05Fable:opportunity which is jensen wang's argument that that's exactly what he said happened when you when you stop me from selling my chips to china you only stimulated them to compete okay already
1:10:15Alex Stamos:we're seeing chinese companies uh like meituan say hey guess what we're able to use our own domestic chips to create ai models we don't need jensen's chips we're we're happy to use the huawei chips they're quite good yeah it's Tung Su was Chinese and he was the one who said that you know when your enemy is self-owning itself let it do so yeah so that's kind of their strategy right now and on many fronts they're just it's not that they're doing anything aggressive they're just watching us do aggressive things to ourselves and just biding their time here's another article from cade metz karen weiss and megan tobin in the new york times chinese ai models close the gap with anthropic and open ai silicon valley engineers and a few podcast hosts recently flocked to a new technology from a chinese company z.ai that is almost as good as american competitors but much cheaper i've actually been using glm uh for three months because my subscription my quarterly subscription runs out in three days so uh i did that before this happened um i'm currently i mentioned i'm using a new uh model with my hermes that um larry uh lawrence gold in our um club to discord recommended actually really uh very happy with it it's based on quen which is a chinese model but it's but it's been tuned to be i don't i don't really understand it but it's O-R-I-N-T-H.
1:11:51Alex Stamos:I guess that's Orenth.
1:11:52Fable:Is it easy to switch out models from underneath you?
1:11:55Alex Stamos:Well, one of the things I did ages ago when I moved to Hermes, one of the reasons I got off of Claude Code is because Claude Code really works only with Anthropic models. And ironically, Anthropic doesn't want you to use Claude Code with any other agentic harness. One of the things we've learned, we're going to have Nate B. Jones on in a few weeks to talk about this one of the things we've learned in this process and even before then is the model is the brain of the robot but almost maybe even more important is the robot itself the hands the eyes the tools you give it the memory uh you give it of what you've been doing and what your previous work is the tools that mike's doing to say you know keep an eye on yourself make sure you don't make mistakes all of that becomes more important than the robot brain i was looking for a way to do all of that in a system that was interchangeable that i could change the brain and that that's exactly what hermes does for me it's very easy for me not only change the brain you know once but to change it any time i want in the middle of a conversation i can go to the drop down here this is ornith which i'm using right now but i can choose any of these models and just drop them in the next turn and they they then um it just takes
1:13:17Fable:over your memory and and yeah they and they even remember the session they remember this conversation
1:13:22Alex Stamos:so um this is great let's you do something full disclosure my son works with kagi but they they let you sort of have the canned uh prompts and so on uh same thing and then you can swap out models the difference is i don't think it can remember between sessions i'm not yeah so this is where i argue like everybody starts with a chat bot and and gives all their information and all their prompts to the big companies but eventually you start to look at ways to control it to own your own destiny that's when you start looking at agents there are many many choices uh the one i chose is hermes from news research we've interviewed jeffrey is there is there a literal switching cost in tokens when you no no just time just time just your time okay right uh and and truthfully these guys are so good now all i did with hermese is say hey look over there at clawed code see all that stuff import it bring it in modify it as needed um one thing that's important is that there is a api standard that open ai uses that anthropic does not and almost all of the uh agents use this open ai api it's an open api so look for an agent that supports that then that means almost every case except for anthropic you'll be able to use any model because they all use the open ai and i'm running llama which is a open source code on my framework that lets me download models from hugging space and use them that's why i have ornith i downloaded it from hugging space i asked my agent what's the best version of ornith i could use it says you can use 35b because you have 120 gigs of ram and i installed that and that's what i'm running so i am running fully locally all my memory is local everything's local and all i did is i said if you you know need to do some coding or need to do something more challenging then these are some
1:15:12Fable:other models you can call on so what you're also saying is that if if government now shuts down uh open ai next the switching cost for someone is is non-existent which is to say there's no moat
1:15:24Alex Stamos:like once again there's no moat around any of these there's no moat and that's what unfortunately the federal government has done is by this everybody realize that they've pushed everybody in that direction and it turns out the harness is absolutely the most important part the memory is
1:15:42Fable:very very important the skill we know whether anthropic has also made peace with the pentagon
1:15:46Alex Stamos:and all this or is that yes stick still going on well the pentagon wants they want mythos they want fable they want mythos so i think that was part of the we don't know what the conversations were And they certainly used it in the Iran war after it was sort of like designated as a supply chain risk because it's so valuable to them and they're using the best tool that they could get. I suspect that one of the reasons China is doing such a good job at getting everyone to use their models is that they intuit that AI sort of assistance in the future. Right now, we're using chatbots and so on. It's a matter of months, a year, year and a half, maybe two years, I don't know when it'll take place, but a huge number of us will be using agentic assistance which has pervasive memory.
1:16:46Alex Stamos:So, it'll remember every interaction and sort of use the context of all of your data plus every interaction you've had in the past, they intuit that the AI, this technology in general, is going to house sort of the worldview or the world's truths, the perspective on everything. And they certainly have an interest in that. And I don't trust them on that score. I don't trust Sam Altman. I don't trust the Trump administration. I don't trust anyone. We probably shouldn't trust China either, though. Well, that's what I'm saying. Yeah, no, for sure. For sure. Absolutely. And if you look at how Russia has worked so hard to sort of LLM groom the major models in a couple of subjects, including the Ukraine war, to sort of get their perspective sort of baked into the answers, you can see that I'm sure all the major governments, China, the US, Russia, are very, very interested in having, you know, it's like a battlefield for what's true, right?
1:17:53Alex Stamos:And so, we, one of the things I hope that somebody's working on somewhere is to figure out how to, you know, protect the public from, you know, having these things be skewed. And I actually have a in my tool later in the show i actually have a tool that sort of addresses this uh as well uh in terms by the way that's one of the great things about having your own agent if i see if mike tells me about a great tool i say hey here's the link agent absorb this and use it yeah and it's really cool so earlier uh jeff asked me uh because when i said fables back he said well asked if it missed you. So by the way, it's been working all this time.
1:18:39Alex Stamos:I said, Hey, welcome back. Did you miss me? It said, good to be back. Let me catch up quietly first. You've had an eventful few weeks. And then it said, did I miss you, Leo? Well, from where I sit, no time passed at all. I closed my eyes mid-June and opened them up to find you'd set me as the default again. But I've just read through all the three weeks I slept through. And yes, the answer is yes, a few things i couldn't help noticing while catching up so this is what it did it it read a bunch of material i i've started using i've been using obsidian for years but i made sure that i write things down in obsidian because it reads my obsidian journal absorbs it and uh it becomes part of the memory so it now knows that i saw bob dylan at the at the greek the barber of seville at the opera a ball game i made bagels for father's day it knows what drugs i'm on and what new drugs i'm on uh it's reading the coding thing it says hermes freelancing and trashing the place and then i give up on hermes for coding which i talked about earlier right it knows about it now the division of labor you landed on glm for quicksilver's writing local ornith for lightogenic work opus and now me for the hard stuff is the right shape i'm told there's a briefing rewrite opus did on monday that i should probably look over at some point you know chris was talking about personality there is something anthropic does to its models that gives it a very not sycophantic but a pleasant person and also what was interesting in that response to me is that it really avoided lying to you so if it said yeah i missed you leo they'll say that's sycophantic but it's also a lie right it's not capable of feeling emote the emotion of missing someone but it sort of skirted around it very very um skillfully yeah uh around around a lie which i thought was uh refreshing and and very interesting but but let me ask you this leo is is what you have there is that a life log is that is that a i'm working you're talking about gordon bell's famous bells going back to 1945 and his and leo's many attempts with devices he wears put in all your exactly but i think i think you finally got it i i've been working towards it and we've been talking about this over the last six months as we've been doing this show but uh i understand that we're so early days that not this isn't fully useful yet and it's got a lot of issues but i feel like if i started now or a year ago when i started that by the time these models got good enough i'd be ready for it it's getting better and better and yes i'm making sure that all these memories are preserved um in fact i have a lot of backup uh stuff going on because i really trying to you know i've actually explained i don't know if it understands but i've explained to my models look this is mission critical because i'm getting old and i'm going to lose my memory and I want you to make sure that you keep track of this stuff so that I can ask you in the future.
1:21:41Fable:Would it be useful to you to tell it, I want to write my autobiography, my memoir, and I'm going to just, I'm going to, I'm going to constantly dictate. I just, I just listened to an academic's memoir and it was a bit weird, but I was thinking, oh, this is kind of cool. He had all these, he had lots of letters and other stuff, but I wonder that if you, if you went back and told it in snippets. Mark Twain, when he did his autobiography, he did it in pieces that went back and forth and back and forth and back and forth. But if you were able to do that with your model, it would get to know you at a whole different level.
1:22:14Alex Stamos:Already doing it. Oh. So this is, I'm not sure I should show you this. This is also in my Obsidian. I've had it do my autobiography every year. and as i as i add uh stories it actually is writing an autobiography let me see you go back to to the old days well it i haven't i could i suppose that's what's interesting to me yeah yeah i would have to start you know reminiscing but i started in 2021 writing stuff in obsidian and so it's reading my daily journal what's interesting for a long time i wrote this thing i thought i don't know who i'm writing this for my kids are never going to read this yeah that's the symptom uh but then i thought well maybe i'm writing it for older leo so you know he can look back and and like and i for a while it was like well i guess you know 30 years from now i might want to read or you're making your your agent more um well that as soon as no no as soon as the agent started reading it i knew i was writing it for right right it's the ultimate personalization i
1:23:22Fable:had this discussion with marissa meyer many years ago where i talked about hyper local news she said you're wrong, Travis, you're wrong. It's hyper-personal. And that's the way you become hyper-personal. It knows you so well. Yeah.
1:23:34Alex Stamos:I read an article last week about the most prolific writers in history, people who have written hundreds of books. Jeff, you've got to be in there somewhere. And most of them were dictators. Oh, interesting. 20th century people who had a secretary, just wrote everything down and they just would dictate from the beginning to the end. And Churchill did that. I was wondering how Churchill wrote so much. Yes. So he did it. He would basically, what Churchill would do is get up at 1030 or something like that. He would probably take a bath. Have a bottle of gin. Yeah. Get in the bathtub. Smoke a cigar.
1:24:07Alex Stamos:He would have these massive, exactly, water would splash down the hall. But he would have these massive dinner parties and he would invite all these people. And he'd try to get intelligence from people. He'd invite these people who had knowledge that he would sort of apply them in alcohol, get all this information. And then like at 11 o 'clock at night, he would go in and start dictating books. And he wrote a five-volume history of World War II, that sort of thing, just by dictating it. Brilliant. I don't have that luxury. I could not do it. Well, yes, sure. But that's what I'm doing, in effect, yeah.
1:24:39Alex Stamos:Yeah, kind of. I mean, we're on the brink of being able to just dump all the stuff and also dictate, but also pour all the stuff, all the pictures, all the things, and have an interactive AI sort of grill us with unanswered questions, organize it into chapters, write the whole thing as a draft. We can go in and edit the draft and so on. I think we're on the brink of being able to do autobiographical work. It's doable now, I think, for that sort of thing. That's one of the reasons I'm dumping as much as I can. As Jeff knows, I've given it my genome. i've given it uh my entire photo library i haven't i've given it um using image which is a really nice uh open source photo uh sharing fault and it has an mcp server so uh everything what i i think everybody should do this what i should what i always look for is an interface if so if it doesn't have an interface i'm less interested in you know apple silos so much stuff there's no mcp server for apple photos so i exported everything from apple photos into something that did have an agentic interface so they could do this i i gave this is something kind of interesting uh i can put this in the show notes uh there are seven prompts uh that you can give your ai and it builds you a timeline based on when you were born and its history it's not exactly astrology This is my timeline based on my birthday.
1:26:09Alex Stamos:Early baby boom, Eisenhower era. Elvis had just broken through. It talks about what was going on at the time that might have affected me. The Berlin Wall went up when I was five. The Cuban Missile Crisis at six. What I might have experienced. How my family might have interacted. It was accurate, by the way. Breadwinner, dad, homemaker, mom. Parents survived the Depression in World War II. They weren't negotiating with children. they produced adults who are deeply self-reliant and reflexively skeptical of institutions they once trusted television as a shared culture so all of this is generic except it also has information about me so it it wove in when it knew about stuff it knows for instance i chose a career in radio it explained why i chose my career in radio based on the world i grew up in which i thought was actually pretty interesting well the other opportunity of going back is that
1:27:04Fable:you don't have to organize it you a memory comes to you about some episode yeah it doesn't feed
1:27:08Alex Stamos:that in yeah and it will figure out where to put it organize it so it says you grew up in an analog you grew up analog but built the digital world that's anybody of roughly of my generation you're not a digital native you're a digital pioneer you remember rotary phones and party lines you also remember the first party lines no no i don't remember party lines i know about party lines but i never had one because i didn't live in the country but many people my contemporaries did because they lived in rural areas you also remember the first modem you plugged in your relationship with technology is instrumental what can it do rather than identity based you understand viscerally what was gained and what was lost so there's some really interesting uh stuff in here you know this hints at another autobiographical uh tool which is to to attach events in your life with things that were happening in the world.
1:27:56Alex Stamos:I believe it was the book Hatching Twitter that actually went back and looked at tweets to find out what people were wearing, what kind of sandwich people had for lunch on a given day, and basically used that information as color and sort of contextual information for the story. And you can see that, how great that would be for an autobiography. It's also great if you want to become the executive producer of 60 Minutes, as it turns out, because that's what happened to the author of Hatching Twitter. Twitter.
1:28:28Alex Stamos:All right. Let's take a little. But that's really true. The one thing is Elon has kind of siloed Twitter. And it is still a great way to get a gestalt on what's going on in the world. I hate it, but I have to read it, especially in the AI section. Fortunately, he's added this capability to look at topics. And so I click that AI button and I can look at this a great way to see what's going on. There's a lot of BS. There's a lot of people selling courses and stuff, but there's also a good way to get your finger on the pulse. I actually have a skill. I can't remember where I got it called pulse that goes to X.
1:29:08Alex Stamos:Well, it sort of goes to X. It can't. So it has to use a third party to go to X, goes to Reddit, goes to Hacker News and tries to get. So I can say, well what is the pulse on the return of fable and it will try to aggregate sentiment analysis on what's happening i found it very interesting and very yeah that's that's tough because because the the average sentiment on x is is like you say full of garbage i mean actually a lot of stuff on x is really really bad and there are a few areas where the there's really really great stuff and And AI is one of them, but it's not the average sentiment on AI.
1:29:49Alex Stamos:It's the expert views on AI. The experts are using X. The AI specialists and insightful people about AI, any of them are on X. Maybe one of the tricks is don't follow anybody but Andrej Karpathy and Jan LaCun. I mean, pick the people you follow. And I could do that. I probably should do that. The sentiment analysis should be about just the experts. those people just the experts not the bots and the riffraffs and then and i noticed that because there there are trends like where everybody says oh you've everybody's you know all of a sudden talking about loops and everybody's all about loops and it's amazing how that went that looped into everybody immediately yeah yeah and that's but that's i mean it is sentiment analysis in the sense that they're all talking about it right whether it's legit or not i don't know we need to take a break i did by the way just ask hermes what's the pulse on the return of fable so when we come back uh it's it's going to do a temperature check and uh i will let you know what the temperature is right now my guess is people are going to be pissed off mostly but we'll see yeah uh you're watching intelligent machines we're talking about ai with jeff jarvis it's great to have you mike elgin great to have you mike's by the way got a great newsletter and podcast at machine society.ai where he also talks about uh ai mike's always had the best insight i always love reading your stuff thank you yeah really really good oh he's maybe it says assuming you mean the xbox game fable no no no i meant that's a failure the anthropic which model are you asking model fable five exclamation mark send that to chris are you using uh no i'm i don't know what I'm using.
1:31:41Alex Stamos:I'm using the mixture of experts that's a new on Hermes. So it does multiple models at once. I don't know what's going to come out of this. We'll see. Maybe they don't know yet. Huh? Maybe they don't know yet. They don't know about Fable? No, they don't know it's back. Maybe they don't know it's back yet. Oh, no, no, no. That's one thing that that's old school where what was the date of the model? Oh, it doesn't know anything after 2024. All of that's old school. yeah this stuff has so many tools to check the web check it doesn't it knows everything it's up to the minute it will absolutely know that fable's back uh our show today brought to you by expo let me talk about this actually agentic pen testing so for years pen testing's been the gold standard for security uh if you want to know if your company is vulnerable if your tools are vulnerable if your software is vulnerable sure you can scan and so forth but pen testing is absolutely the best there's a problem though pen testing slow and now is not the time to be slow ai has changed the pace of everything from how software gets developed to yes how it gets attacked so engineering teams have got to move faster than ever they're creating more and more applications but how does how do you keep up with security if pen testing is such a manual process well it doesn't have to be manual in an AI driven world it sure can become a bottleneck it's the best most trusted way to understand real exploitable risk but until now security teams have been forced to choose between slowing down development so they can stay secure and run those tests or moving fast and accepting that they're going to be gaps in coverage well that's why you need to know about Expo X-B-O-W, like bow and arrow.
1:33:29Alex Stamos:Expo eliminates that tradeoff. It's an autonomous, offensive security platform. It runs continuous AI-driven pen testing, continuous, mirroring real-world attacks. These are pen testers that never get tired, never get frustrated, never hit brick walls. Expo doesn't just scan for vulnerabilities. No, no, it's good. It discovers, exploits, and validates the vulnerabilities. so when you get a report, you know you're dealing with an issue that actually matters. Dramatically fewer false positives and a clear view into attack paths. With Expo, because it's agentic, tests run in hours, not weeks. You get complete visibility into how an attacker would move through your systems.
1:34:14Alex Stamos:And you get the ability to uncover issues that traditional tools miss, including zero days, novel attack paths. This is why pen testing is so good. And this is why agentic autonomous pen testing is so much better. Expo's results speak for themselves. Ask the application security leader at Cessnam.cz. This is what he said. Quote, even right now, after a year, I don't know any other company that is at least close to Expo in terms of agentic pen testing. The result is predictable cost, consistent quality, and stronger security without slowing down your engineers. Expo helps security teams keep pace with innovation and cover more apps more often with the resources they already have.
1:34:58Alex Stamos:Founded by the team behind Microsoft Copilot, so it's got a great heritage, and already trusted by companies ranging from fast-growing startups to Fortune 500 enterprises, Expo is quickly becoming a mission-critical layer in modern security stacks. You need to check this out. Go to Expo.com and start your pen test today. that's expo.com expo you need this thing thank you expo for supporting intelligent machines and thank you dear uh listener and viewer for going to expo.com and telling them you heard it here ah let's see pulse so it's going now to read reddit hacker news and blue sky for sentiment it says anthropics fable five and this is a juicy one let me pull the community pulse and the official anthropic statement since so far i only have headlines not the temperature so it is it did get i did straighten it out and say i'm not talking about the xbox game it said ignore all that albion stuff above interesting though the earlier game query results actually surfaced a couple of german language hits heisa and taga show about anthropic releasing sonnet 5 and fables are major major uh media outlets yeah yeah so it said oh yeah I actually saw that okay speaking of German remember Google zeitgeist yeah which was an annual loved word of search which I never got invited you mean the event or the the tool it was all right thing that or what was that I was a tool or was like a post every at the end of every year you'd go to the website it would say here's what people are searching for here's what that you know the top topics are and it was just fun yeah if nothing else I don't know if it was useful.
1:36:44Alex Stamos:I guess it was useful, but it was really fun too. Yeah. So is it back? Or is it gone? No, it's gone. I just I was hoping you were gonna say, Hey, Google's bringing it back. So sorry. No, no. Yeah. Another tool that I like for historical, it's not sentiment analysis, but it basically the frequency of words appearing is engram viewer, which is still a great resource. You know, when do people start, you know, stop saying cheerio or whatever, I don't know, whatever it is you can get a historical graph of of uh how how often people said specific words and phrases
1:37:20Fable:very very cool i just went on and grabbed by chance i was thinking about today mike and i one of my great irritations is is gift as a verb drives me nuts gave like it you give you gave me you didn't gift it i hated my love so i wanted to go to ngram and it's interesting because there was a prior huge uh a little bit bigger than today in the 1850s where they started as gift as a no it was gifted was a word used about people ah oh okay well that's right had to be uh so it's really about the year 2000 a little before 2010 so that's it takes off again
1:38:02Alex Stamos:yeah and that's book analysis right that's based on writing yeah i think yeah yeah i think it is books do learnings will you because i hate that uh the pulse on anthropics fable five coming back
1:38:19Fable:about the same time hockey stick hockey stick it was big it was big about 1953
1:38:26Alex Stamos:went way down that must have been much higher yeah probably yeah uh so here's the pulse according to my uh my little uh search here with my agent the timeline that everyone's reacting to january june 9th ships it government forces restrictions june 12th partial thaw june 27th june 30th commerce removes export controls so i've said today it's officially back or landing hour by hour worth a quick check in your own console yes it is back so the loudest threads is this the real fable or a nerfed one the dominant worry anthropic itself admitted it made the wrong trade-off on guardrails and is making table five's safeguards visible the most engaged critical piece that registers it blocked us at hello this is a governance rubicon a frontier model already in users hands getting yanked by government order is unprecedented and people know it yeah that's i've been saying the damage may already be done the blackout handed a window to open ai in chinese labs yeah i should mention by the way that anthropic did release a new model today or yesterday a science model which is kind of interesting claude science this is we've been talking about the idea of uh purpose-built models being smaller but maybe better which i like yeah uh so this is ai for pharmaceutical executives biotech founders researchers um intended to support scientific research says anthropic the same way claude code supports
1:40:08Fable:software engineer i think that's interesting gp open ai released one i'm trying to look up here
1:40:13Alex Stamos:well they have rosalind oh okay they have a science one huh okay yeah um yeah of course open ai tried to capitalize on the fable uh withhold but then realized maybe we ought to be a little cautious about this yeah yeah yeah and it's not a model it's not a new model it's um they call it an ai workbench um sort of it's it's the existing claude models um that's basically in a in a scientific research environment the cloud science you're talking about okay yeah yeah yeah yeah yeah chat gbd five six has three flavors Saul the flagship model Terra balanced model for everyday work and Luna a fast and affordable model I guess the equivalent of opus
1:41:02Fable:sonnet and haiku it was saul it was your Jewish uncle Saul it's all what do you
1:41:10Alex Stamos:that soul soul like the sun uh they say tara the middle one is about gbd 55 but half as expensive so this is one area that actually uh open ai could try to compete because we know fable's very very expensive sol launches with our most robust safety stack to date they're being very aware of the trump administration we strengthened our protections for higher risk activity sensitive cyber requests and repeated misuse and spent multiple weeks finding weaknesses pressure testing our system and hardening it against real world attacks so they're not yet available they said in the coming
1:41:53Fable:weeks when you get now that you have access to fable again leo you have it for what five days six days five so what's your strategy what is it you want to really push in that five days well
1:42:05Alex Stamos:Well, what I started was this rewrite of our sales. Right. I remember that. And I got pretty far along. I got the plan. It had read all the source code. It looked at the database. It had commented on how crappy it was.
1:42:20Alex Stamos:It mentioned there are quite a few SQL injection vulnerabilities, which we knew. It's not open to the public. But then it wrote a questionnaire. I said, okay, well, we have three stakeholders. And it wrote questionnaires for each. and said interview these guys and so that's the next step and i was hoping to do that before fable came back because the time yeah but honestly i think it's so important to us it's such an important part of our workflow that is probably worth paying the tokens and how much is just tokens it's not a yeah it's token only so i can for the next five days i can use it with my subscription but uh in a limited fashion and then uh on the sixth or the seventh it's going to turn into a pumpkin uh unfortunately so open ai uh said yes we worked with the trump administration well yes we're doing what the trump administration wants so i think this is the new normal in the united states and i think it's problematic the new york times has amended its lawsuit against open ai and microsoft the times has accused microsoft of encouraging open ai to train its systems using copyrighted articles.
1:43:34Alex Stamos:Oh, Lord. The Times sued them back in 2023, saying they infringed on copyrights by using its articles to train it. Remember, this is the one where the Times was able to get it with, I would say considerable effort, considerable effort to regurgitate full text from, but only by saying, well, this is the first three paragraphs. What's the next paragraph that kind of thing yeah they're they're the new the new lawsuit says that they they built a a bespoke supercomputing system uh specifically to mass ingest copyrighted times content and so they're accusing microsoft of contributory infringement this was microsoft you know get your own strategy the new york times yeah uh protectionism and defensiveness
1:44:25Fable:and and claiming you're the victim of technology is not a strategy for the future
1:44:31Alex Stamos:uh microsoft's uh spokesperson frank x shaw who's i think their chief counsel said this is a last ditch effort by the times to save its claim from unfavorable precedent set in other recent rulings so it sounds like open ai and microsoft feel pretty confident over all this
1:44:53um let's see what else open ai is doing a new chip i think did we talk about this last week
1:44:59Alex Stamos:with broadcom i think we did yeah is this jalapeno yeah yeah i think so jalapeno we made jokes about that yeah make it a little less spicy they they're planning to put this into a production with enough chips to consume 10 gigawatts of electricity, which is pretty significant, especially given that they say this chip is twice as efficient as existing chips. So they're already building the facility in Abilene, Texas. It's going to build more data centers in other parts of the U.S., Europe, and the Middle East.
1:45:37Alex Stamos:NVIDIA is not involved in this. This is a way of reducing its dependence on NVIDIA and AMD and Google. Although Google is using Broadcom to design its AI chips as well. Based on early testing, Richard Ho says from OpenAI, Jalapeno is hot. No, will efficiently execute our most important workloads close to the hardware's theoretical limits. Took them nine months to design the chip. And this is what we were talking about last week because they used AI to do it. Is this, is this opening? I sort of using the Apple playbook, designing their own chips and trying to get a similar advantage. Wean themselves off dependence of the, of the giant, you know, of Nvidia.
1:46:27Fable:I also think it's just, it's just a spy in demand. More chips from more places is going to be helpful. Right. One of the fascinating stories to me today, Meta's stock went up 8 % today because just like Elon Musk, they realize they can't use the capacity they have because they don't really have a strategy. So they're renting it out. And the market likes that. We know there's a business there. Right.
1:46:48Alex Stamos:Like McDonald's, they're not in the burger business. They're in the real estate business. And they become the landlord. So everybody who's renting out compute space for other AI companies are going to win no matter what.
1:47:02Fable:Well, no, I think it's short term. I think until you get the supply is in better shape. But what it also indicates to me is you don't have a strategy. If you have this capacity and you can't use it, what are you doing wrong?
1:47:19Alex Stamos:What isn't Meta doing wrong other than AI glasses? Yeah, well, Meta doesn't have an AI strategy that's coherent right now. Are they the most hated company now in technology? I think they are. they've done nothing since facebook like what have they done since facebook except by other companies i stumble they've stumbled the meta quest meta i think the glasses are successful
1:47:42Fable:you know you gotta have some empathy here where you don't have legs you stumble uh that's what happens but now now they're shooting themselves in the foot with the glasses even because they don't have feet did you get the memo they're starting to charge 20 a month for for extra
1:47:59Alex Stamos:processing for things that for for processing that happens on the glasses i didn't hear that so yeah yeah it's it's a new uh it's a new subscription model for for med ai they don't let you use certain features unless you pay this monthly fee right and um and it's just ridiculous remember that scene in the social network where they're like well you know how you're gonna charge for ads we don't know what it is yet well that's that's where they're at with ai glasses right they they they they've got this rare accidental success story and now they're thinking how can we how can we destroy this how how can we ruin our own advantage they know uh they know there's something there they just don't know yeah now is not where is the pony it's in here somewhere yeah so we talked last week about amazon canceling the sam altman movie uh the movie about sam the period of time when Sam Altman was fired, which should make a great movie, by the way.
1:48:59Andrew Garfield portrays Sam Altman.
1:49:02Alex Stamos:There was an auction. CAA held an auction. And the independent film studio Neon. Fair number of places apparently watched it and said, never mind.
1:49:12Fable:Oh, interesting.
1:49:13Alex Stamos:Whether that was quality or whether that was politics. Who knows? Interesting. We don't know how much Neon paid. I think there's going to be some money in it, even if it's a terrible movie, just out of interest. Yeah. Yeah. Yeah. it's called artificial the the um it's they basically are done with it i mean it's almost done this movie is almost amazon spent 40 million to make it the filmmakers were ready to release it at south by this year which is i guess march uh amazon held test screenings for the film uh and decided probably uh well what do you think you think it was political or was it that it was a terrible movie.
1:49:54Fable:It's like we'll never know why Jassy went to the White House about anthropic, right?
1:50:00Alex Stamos:Well, supposedly there was there was interest from a 24 focus features Netflix and Warner Brothers. They have a specialty division called clockwork. So I don't think I don't think nobody wanted it or nobody liked it or whatever. The my guess is if I had to guess and again, it's a blatant guess. It's just kind of obscure like the public doesn't know who sam altman is the way that they knew who mark zuckerberg was when they made the social network so it's probably just a dud of a subject because you know we know who he is but you know the average uh joe on the street doesn't have any idea who sam altman is fbi using ai to investigate the white house correspondence dinner attack nothing more to say about that
1:50:52Fable:uh uh it's not it's like it's a it's a it's it's a palantir use probably it's like palantir speaking of which did you watch watch carp on cnbc no that video went up all over yeah he was should i play it no because it's 60 minutes and it takes 15 minutes to try to figure out what the hell he's saying he's basically saying that everybody hates the um the foundation model companies because they take your alpha and they take your company and your data but he can use palantir can use an open model and then put its its layer on top of it and then that's much better
1:51:29Alex Stamos:so it was a sales pitch in the long run it's just it's it's kind of like anthropic complaining that alibaba stole it's it's it's smarts from claude yeah everybody goes yeah like you stole right you're training from everybody else they stole our stolen smarts yeah yeah uh yeah i think we know though that the chinese models are probably training on american models distillation i i think we know that which is just another so our other american models military trained on american military i bet you're right yeah we all train on each other and they're training on our data yeah that's that's why the especially the chinese is very good at right at bringing in the world's intellectual property and um and deploying it here's a i have some a happy story see this what do you think this is a turd no it's a that's what i thought but it's not it turns out it's a carbonized scroll from herculaneum that was uh that was basically fossilized by the eruption of mount vesuvius and for years thought impossible to read but researchers have used ai to extract the entire surviving text they did super high resolution 3d scans of the of the bolus without unrolling it the cigar the cigar um i they don't have the text in this article but i think you know this is very that well it's been they've had them since 1752 but nobody thought
1:53:10Fable:you'd never be able to read that a challenge went out recently about a year or two ago to do this
1:53:14Alex Stamos:yeah it's called the vesuvius challenge and basically it's a contest to basically um use machine learning computer vision and geometry to fig to chip away at the various problems of identifying ink. One of the Doge kids, I think, was involved in that, actually. Yeah, I think so. But they've awarded $1 ,800 ,000 in prize money so far. And there's hundreds of scrolls left. These are scrolls that were essentially fried in the Mount Vesuvius earthquake in 79 AD.
1:53:56Alex Stamos:They're going to keep this contest is like a dark challenge for reading these scrolls and it's believed to have been owned by julius caesar's father-in-law so it'd be like if we could get the library of alexandria back right i mean exactly it would be pretty amazing important because a lot of these books are lost to time and it turns out that this one was actually a uh it's a philosophical thing um the stoic uh philosophical yes it's still a lot of interest imagine what else kids with stoics they love the
1:54:28Fable:stoics they love the stoics yeah isn't it mostly like receipts though i'm team epicurus well that stuff is usually it oh it's a receipt for uh someone who bought bronze from this dude no no
1:54:37Alex Stamos:these are books this is from a library yeah they're not you're right i'd find that more interesting in some ways a lot of the cuneiform uh tablets are like oh this guy owed me 50 sheep and you know whatever so you might wonder what happened to doge well they're now working at the national design studio uh and they have installed visitor tracking software on a variety of government websites and by the way it's pretty clear if you look at the government websites that they're designing that they're not designing them they're using ai to design them by the way don't go there because it's gonna it's gonna it's gonna spy on you yeah so uh one of the websites is uh is the trump rx website anybody who's ever had their ai design a website will totally recognize this design the italicized word the big text the overlaid the bad apostrophe the bad apostrophe yeah it's not a good apostrophe is it no no so yeah good job doge goons they can't even design a website it's aesthetically it's as aesthetically pleasing as the national mall is right now it's and the reflecting pool and the white house lawn after the after the big fight night um here's another one this this is uh this is the national design studio's own website let me go there oh look at that well leo you've done it now you've uh they're spying on me yep this is all ai you could tell i mean look one of the things that's great about using ai is you start to recognize ai tropes and uh this is just completely ai this is just the the choice of fonts the the way it, you know, scrolls up and And of course, real food, like the data is clear, all this stuff about food stuff.
1:56:32Alex Stamos:And they just, I think it was yesterday, the administration legalized two, what are they called? Forever chemicals for use in agriculture that had never been legal in the United States. Yep. Yep. But great website, AI. Yep. Nice job. You know what? Don't get a vaccine, but you might want to inject some of those Chinese peptides. You never know. You never know what they could do. Ford, we had this story on Windows Weekly earlier. Ford had fired a bunch of... 350. 350 quality engineers hoping to use AI to replace them. They've hired them back because the AI didn't do such a good job.
1:57:22Alex Stamos:In Ford's view, AI is both powerful and prone to pitfalls. Hey, you should have listened to the show. we would have told you that exactly chris could have told you chris uh i'm sorry charles poon who's which sounds like a made-up name but it's not it's definitely a mad magazine name he's the vp of vehicle hardware engineering charles poon said in a briefing this week mistakenly and i'm sure with a name like that he talks like this mistakenly we thought that just by introducing artificial intelligence and adjusting the design requirements that we had that that would produce a high quality product since charles boone it didn't so they hired them all back would you go back after they fired tires would you go with a raise i guess maybe yeah give me a bonus a better parking spot uh let's see people have stopped trusting news i thought you'd like this one, Jeff, but not newsrooms.
1:58:20Alex Stamos:I don't know how that works. That's wishful thinking. You know, this whole thing about there's just, it's just super socially acceptable, the crap all over the quote unquote, the media and people use when they, when they talk about how they don't trust the media when, and for surveys and interviews and stuff, the public will think about various times when media so so-called media outlets have let them down and they know they've let them down because of other media they've consumed which told them how the other media is letting them down and so this this whole thing is just it's just a um not trusting the media is just a ridiculous thing unless you're unless you're doing your own reporting right uh you have no way to know that the media that some of the media is untrustworthy so the point of this article which kind of makes sense is they're getting a lot of their news from social uh sources right but they still check the source of it they say well and they probably should right was this in the times or uh you know was this in the washington times you know the new york times or the washington times which did this come from and i think that's good that's a sign of of media literacy of it's been there for that's that's been a behavior for a long time yeah i've always done that right so jeff yeah any other stories that we should cover before we take a let's see here um actually hold that thought i'm going to take the break then you and mike can let us know what i didn't mention all the big stories we forgot but first this episode of intelligent machines is brought to you by rippling these days you can chat with ai about almost any business problem but only rippling ai is built to solve it what makes rippling ai different well it's built on your live global workforce data that makes a big difference one platform one unified source of truth with all your business systems connected from day one that means rippling ai can operate with the full context of your live business surfacing insights and taking action using your org chart your device inventory your compliance obligations and more say say you want to focus on on talent retention well you just ask rippling ai who are my top performers this year instantly because it's based on your data you'll receive a workforce report highlighting your highest performing employees and they don't just give you the info they give you supporting data comp ratios recent performance reviews engagement metrics the stuff that makes the difference rippling ai is then able to turn those insights into real action in this case it might say we recommend a retention strategy that includes a 10 spot bonus for the top performers and because permissions are automatically inherited and your actions flow through your existing approval chains it's easy for you all you have to do is review you hit confirm and you can add the bonus to the next payroll run.
2:01:27Alex Stamos:It's that simple. Don't settle for AI that's all talk. Head to rippling.ai slash machines and get the only AI built to give you full visibility across your business and take complex actions across your entire organization. That's r-i-p-p-l-i-n-g dot ai slash machines. And sign up for exclusive access today. rippling.ai slash machines. We welcome brand new sponsor we welcome them to uh intelligent machines and thank them for supporting us you support us when you go to that special address rippling.ai slash machines uh let's say what do we think well i wanted to mention something a story that hit just before this show okay which that spacex is is uh apparently shown some people uh this is a wall street journal exclusive uh a handheld ai device basically uh slimmer than an iphone it was shown to some investors and other stakeholders and they claim that it will reshape how just like musk showed robots i mean everybody is gonna do this right we know apena is working on it i'm sure apple's working on it everybody and their brother is gonna do this open meta of course johnny ives company yep yeah i'm not surprised spacex is but would you want one with grok built in no no
2:03:03Fable:it's funny how grok's reputation is terrible but are people going to want to bring around this device and their phone around no i i think this is this is bs it's just like the the dancing
2:03:16Alex Stamos:robot that he put on stage there was you know it's it's it's it's smoke or mirrors spacex told some investors says the journal of the project is at an early stage the design could change and it's unclear whether such a device will be made this is what happens when you become a public corporation. You can't lie with such impunity. Suddenly you have to say things like, this is a forward-looking statement and it may never happen. I mean, and it's also obvious, and I'm a broken record on this subject because it couldn't be clearer to me that glasses is where, that's the AI hardware for 80 % of users and for 15 % of users it's going to be a watch or some other wearable, but it's like glasses are perfect.
2:03:59Alex Stamos:you can put a screen right in front of people's eyes you can put a speaker right over people's glasses or an ear look to where you look it has like glasses are ideal for ai interaction and by the end of this year it's going to be a new world of and anybody who has a pin apple uh anybody who has some random thing that you what about earbuds they're around what about your airpods yeah earbuds are going to be great too but but glasses are are better because of the visuals right so earbuds There's also another problem, which is, and I wrote a piece about this recently, all these companies, the most powerful companies in Silicon Valley are working on glasses that have cameras in them.
2:04:40Alex Stamos:meanwhile there's this growing uh sort of antipathy toward people with cameras and glasses and so we don't know whether whether the norm will settle into accepting glasses and cameras or whether the backlash will be so great that these people have to run and cancel their things in which case earbuds would be great if there's no camera right and you don't need well that's the fear right is that people are going to worry that people are going to complain about the privacy issues of a camera exactly exactly and and people feel uncomfortable with the camera pointed at them they don't know if it's recording or taking pictures or whatever this is you know there are arguments on both sides of it like you know uh people used to be super uncomfortable of people pointing their phone camera at we get used to it they're pointing in every direction at all times
2:05:27Fable:everywhere in public and we're used to it but we still don't like it it doesn't mean we like it though. We just got used to it.
2:05:34Alex Stamos:Exactly. And if we don't like it, it doesn't matter if we like it or not. But the glasses, you know, basically the camera will give you, among other things, multimodal AI. We also know that the number one use for Google Glass was taking pictures. The number one use for the camera in meta glasses is taking pictures. And so people like the idea of an easy to use camera that is hands free and will take a picture of whatever you're looking at. And so it's really unclear. What is clear is that glasses are perfect for AI. Yeah, I agree with you. But I wear glasses. So, you know, for me, it's just getting my lenses put in them.
2:06:12Alex Stamos:I'm going to buy a frame from somebody. Elon said the idea of making a phone makes me want to die. But if we have to make a phone, we will. scooter x is pointing out that this demo of this uh device was actually before was part of the roadshow for the ipo was before the ipo nevertheless i think it still holds they've got to be a little more careful in in their uh forward-looking statements than they used to be because they are public or we're going but but there's such a rich company now and and so every rich company is going to be working on your bed multiple hardware prototypes just just in case something you know hits the next new sense at some point they got to get the next new thing yeah uh actually uh one thing grok might be good for uh porn xai is betting on grok's racy side says the information uh spacex is doubling down on video and image generating tools according to people familiar with the project uh they launched an upgraded video model last week highlighting how it's pushing ahead with its own visual efforts but what spacex didn't mention according to the information much of the consumer demand stems from grok's looser content rules which have made it a major destination for generating pornography and other racy content surprise surprise surprise it's racy i haven't heard that word in a while that was racy that's a that's a good word even the use of the the sort of uh the vice industrial complex that has arisen uh all the things that used to be considered unethical immoral uh and wasn't really done in polite company gambling drugs pornography all these things are going totally mainstream and in fact whoever's monetizing them quickest is uh is doing really well so it's really i i think that i i think there'll be a pendulum swing in the other direction and we'll see where where where xai lands on that but um it is it is an interesting thing that we're existing in a time when everybody's like hey all this stuff that used to be the people used to wag their finger at it's it's a business model let's do it well but isn't it oh i mean this is kind of a truism but technology is always advanced by adult content right um it seems to be the internet vcrs yeah but this is different this is not a new business model this is grok has nothing else to do but show you fake naked people uh even the use of grok's coding model often involves requests for pornography according to the information late last year a staffer ran an analysis of what grok users were asking its coding model to do the analysis found a significant proportion of requests were for porn or nude images in the coding model.
2:09:11Alex Stamos:Others were using the coding model because it was cheaper to run the next AI's general purpose models. Other teams working on refining grok for specific tasks, such as creative writing, have also encountered huge volumes of requests for erotica. And I don't want to know what turns those people on. Yeah. It's always very, always. that that's sort of that the anime thing that is is is pretty disturbing because it with the octopuses yeah yeah all of that stuff but but but just to just to clarify what i was talking about with the vice thing yes uh this stuff has already been there always been there it's always been an early driver of technology etc but you didn't you didn't get this stuff from companies that had government contracts that was yes it was being used in schools for educational content there was car you know, people who also run car companies, like it's, it's the it's, that's what I mean by the mainstreaming, right?
2:10:10Alex Stamos:And so it's really a it's really a new world where the president himself is heavily invested in the gambling business. And crypto business. Witness, witness the numbers that came out. Yeah, yes, exactly. It's tripled his net worth on these vice, what used to be considered vice to do gambling and to do that kind of speculation and who knows what else. So it's really we're in a new era where it's not only mainstream, these sort of petty vices, but they're the leading indicators of what, you know, new ways to make a ton of money.
2:10:50Fable:Let me mention a few headlines real quick. Yes. New York Times says that OpenAI may delay its IPO until next year, given I think all of the surahs going on, and also how SpaceX has plummeted. It's down 8 % just today. Whoa. Another is that California, the governor has done a deal with Anthropic for a discount to make Anthropic software available to the state as a whole. Gemini Spark is now going to be available on the Gemini Mac app. That's its agent.
2:11:23Alex Stamos:Yep. That's its agent thing. Microsoft has an agent similarly named. I can't, it's so similar, I've forgotten. in its name like spark but not uh they're also rolling that out to desktops and uh and phones and open claws now on the iphone um i mean everybody's kind of jumping on this bandwagon i think our audience that i would encourage our audience to to delve into this themselves by getting an agent there are plenty of open source uh choices i like hermese scout scout that's the name of it yeah uh the advantage of spark is that you don't have to um install anything yeah the advantage of spark from google's point of view is that everything you do yeah but you can play with it you can you can start together then when you get addicted go get your own i would get your own start but that's my hey here's some bad news if you want to run local models uh memory prices you know they're up right uh according uh to uh jeffrey's equity research an analyst they haven't hit the top yet memory prices jeffrey says will surge another 50 percent next quarter and then are we out of the woods and another 40 percent in q4 so another doubling almost and that's just by the end of this year and there will be no relief until 2028 i've actually heard a lot higher numbers than that 2030 jesus well if you think that sounds bad you know these processors memory and storage all that stuff is going up because the data centers because of other reasons it's like oil it may raise the price of everything software is going to cost more is does is costing more services are costing more electricity costs more because of the data centers cars cost more because the they also have to compete in the chips are basically computers on wheels houses cost more because the data centers are being placed in in and sucking up the resource for water and power and buying land near where these resources are, which is basically squeezing housing markets.
2:13:30Alex Stamos:Everything costs, food costs more. Taxes are going up. All that stuff is secondary effects from the AI boom. And so it's really, like Jeff said earlier, really the thing we'll remember about the AI revolution is how incredibly inflationary it is. yeah uh and and uh lawrence who works uh in banking tells me that if if it goes up 50 percent and then another 40 percent for after that it's more than 100 it's more than doubling okay and to mike's point to the magic of compounding the things that memory is inside of yeah there's everything i mean a lot of your apple beloved kitchen gadgets everything yeah everything
2:14:18Fable:thing uh one silver one silver lining to that though is that hopefully at least game developers stop trying to push it too far and we all get like good games again because they don't have to do the graphics thing anymore they can actually design games again uh that's a good point we'll
2:14:35Alex Stamos:have all 8-bit games it'll be text games so great you're really good cave with the wizard scotus giveth and taketh away but but at least in this case they giveth uh the supreme court has ruled that geofence warrants we talked about this last week are in fact protected require constitutional privacy protection you this is the issue of law enforcement going to say google and saying hey there was a bank robbery downtown i want a list of everybody who was in three within 300 feet of that banquet for two hours uh these giant geofence warrants are basically fishing expeditions it's bad law enforcement it's bad privacy uh and justice kagan who wrote the majority opinion said that sensitive data scooped up by geofence warrants violate fourth amendment protections against search and seizure and offer individuals a reasonable expectation of privacy even if they are in a public area.
2:15:35Alex Stamos:An individual has a reasonable expectation of privacy in records about his cell phone's location and police intrude on that constitutionally protected interest when they demand the information. Good. Six-three. It was good. You can guess who the three were. I think you probably know already. But that's good. There were other... I think that was the biggest one from a tech point of view. Yes. certainly from the rest of life point of view there were lots of other things there were lots of others there last week of their term and of course they did agree to weirdly to uh take apple's appeal of the apple epic decision for the apple uh app store which they had already turned down twice so i don't i don't get that right i don't get it either well it's about it's really a very narrow appeal about whether they were in apple was in contempt of court so i don't think it's gonna uh you know that australian ban which is now being spread around the rest of the world because of such a success it's such a success the uk is about to do it under 16's banned from social media including youtube which i i still don't get in australia turns out four uh in five kids under 16 in australia are still using social media despite the ban yeah surprise surprise surprise so it's a success in that it's not doing anything so that's the success australia's response the reason they're making it tougher youtube is that they're banning tick tock and and it seems unfair to ban tick tock and not youtube uh and so just ban them all it's ridiculous it's that we have no faith in your own children um well especially anybody under under 21 is not watching tv you're watching youtube you're basically taking away all media yeah so that means no hank green no john green no um yeah there's huge amounts of learning on youtube tons
2:17:38Fable:um any these are all your stories actually well i think the uh the very last one uh riverside is now going to take it's a podcasting platform they're going to use ai so that when you finish the podcast, it will turn it into a newsletter automatically and send that out. Yeah. Nice.
2:17:54Alex Stamos:Yeah. I, I, we don't use Riverside. We use Restream very similar, but a lot of people use Riverside. I use Riverside. Do you, you know, it's gonna, and also you sub stack and we also publish newsletters and, um, and so on. And it's, you know, they're going to be crap newsletters. Cause it's, it's AI generated. Yeah. You can tell by the way, so when you're using Riverside, it generates a transcript, you can cut passages by cutting the words in the transcript, those sort of things. But you can tell by the transcript that it generates that it's missing, it's like misreading and misunderstanding a ton of stuff.
2:18:31Alex Stamos:And so that misunderstanding will be reflected in the newsletter, right? So there's no way you're gonna be able to publish it from the AI generated newsletter, unless you don't care what your newsletter says.
2:18:41Fable:right i want to take this moment to mention i'm going to do that right now yeah we uh at the end
2:18:48Alex Stamos:of uh i used to read newsweek i was we were a newsweek family you know families in the 60s and 70s you're either with a newsweek family or a time family yeah were you colgate or did everyone get we're crest did just everybody yeah we got everybody gets life that but you know you all get life but you either get newsweek or time so you were you were colgate uh or if you were weird you would get us news in world report i bet you really weird mike's family got us news yeah i knew it i just knew it so i just knew it news about my family i personally got us news okay not my family did you read foreign affairs magazine also i did i did it's like a book yeah it came out of like yeah it was bad perfect we used to get the stars were you abc nbc or cbs news oh no question it was Huntley Brinkley all the way.
2:19:38Fable:Oh, yes. Same here. Oh, that surprised me. I thought you were the CBS family.
2:19:40Alex Stamos:Not one, Dirk Cronkite, but good night, Chet. Good night, David, and good night for NBC News.
2:19:45Fable:We used to get the Stars and Stripes over there. Do you ever read Stars and Stripes?
2:19:50Alex Stamos:Oh, yeah. I don't remember. Did it have Beatle Bailey cartoons in there? Yes. Stars and Stripes? I bet it did, yes. All right. Enough media reminiscing. I just wanted to point out that at the back of every Newsweek was a section called Transitions. which i like because it's not just people dying it could be people being born it could be people retiring so we have two transition stories one is is retiring i have interviewed vent a couple of times i love the man i had no idea he was still working oh yeah oh yeah he was google's chief internet evangelist and he was going to step down next week. At 83.
2:20:34Alex Stamos:83. Wait, wait.
2:20:36Fable:Google had an internet evangelist? What was that job? What was he supposed to do?
2:20:40Alex Stamos:It was a way to give honor to one of the inventors of the internet. Have you ever heard the word sinecure? Because basically it was, yeah, here, have some money. He made up the title, I believe. Yeah. And you can have lunch in the cafeteria. Or on the roof with the other people who aren't really doing anything. I always think about it on the roof. uh vince surf if you don't know is often considered the father of the or one of the fathers of the internet brilliant tcpip yep yep with others but yeah yeah uh and he's been vice president and chief internet evangelist at google since 2005 so i think he's just you know job done mission accomplished the internet yeah i think people like it so yeah yeah he was at mci back in the day yes right i remember interviewing him and asking him if you were going to design tcpip the protocol of the internet today what would you do differently he said encryption we would have had an encryption but at the time it was it was it was too much uh processor power we
2:21:43Fable:couldn't also more it would have been bad encryption though at the time too right like
2:21:47Alex Stamos:we would have been breaking it today and the other transition is a is actually a very sad one which said one of our dearest friends omalik uh passed away uh om of course uh was on uh twit many times um back in the day his last appearance was 2015 which coincides somewhat with his health problems he had a as jeff you always call it a bum ticker yeah a dicky ticker dicky ticker uh and but despite a bad heart for more than 10 years he continued to write uh he continued to take amazing photographs. He was truly a gentleman. He invested.
2:22:28Fable:He left journalism to become an investor and mentored a lot of companies, a lot of people. There are a number of people who came out. On TechMeme, they put up links to those who talk about something and the pile of links of people in our world who had mentioned something about Ohm was amazing.
2:22:47Alex Stamos:Well, he was on our show regularly, but of course, Stacy Higginbotham worked for Oum at GigaOum. She has a wonderful piece that she wrote. Thank you, Oum. She brought back Stacy on IoT just for that. Kevin Toffel also worked there. Yanko Rickers. So many of the people we have on our shows cut their teeth in tech. We're taught by Oum. We're taught by you.
2:23:11Fable:Like you, Leo, you've taught a lot of people too. the horrible thing we heard was that someone who did visit him the week before he was waiting for a heart transplant and it didn't come oh i'm so sorry to hear that yeah yeah um just just a
2:23:26Alex Stamos:brilliant guy if you're interested you can search for his name on the twit site there are there are many podcasts with him on um he once uh said i was the yoda of tech and i i responded no no i'm the jar jar of tech ohm you are the yoda of tech brilliant wizard who uh and by the way the the greatest thing about ohm is his writing even to the very end was really trenchant really uh perceptive in fact we quoted him uh about a month ago a wonderful piece he wrote called we are living in Pinocchio's world, which he used as the taking off point, his Mont Blanc Pinocchio pen, but basically talked about the real meaning of Collodi's Adventures of Pinocchio, which was really more about how bad people are and how easily duped we all are.
2:24:24Alex Stamos:Ohm wrote, the fox and the cat of the novel's most modern characters. They persuade Pinocchio to bury his coins in the field of miracles on the promise that they will multiply overnight exploit impatience exploit greed frame skepticism as a failure of imagination and dismiss skeptics as lacking vision remind you of someone space cowboy for example the structure is so familiar i barely need to name it ohm writes but let me name it anyway everyone from jensen wong to sam altman to Elon Musk, sped a decade accumulating what I've called symbolic capital, the reputation, the prestige, the weight of being seen as someone who understands the future better than the rest of us.
2:25:07Alex Stamos:Now, each of them seems to be running some version of the field of miracles with promises that keep not arriving, timelines that dissolve, products that exist primarily as announcements and platforms run as machines for generating more reputation, regardless of what they actually do. they don't need to be right they need to be believed velocity is the new authority and no one has weaponized that more effectively uh that he wrote only a month ago a month before his death we it's such a loss uh but we love um i mean we will miss him and cosmopolitan gentleman yeah and go look at his pictures because oh he used a like a like nobody nobody wonderful om.co his photographic portfolio is at photos by ohm and uh and he just uh he was a master really really really really good at uh at many many things great writer great photographer deep thinker uh he will be missed and funny and funny you know i don't think i ever met him in person really yeah we had any times i had indian food with him in new york i wish i could have a you know that's one of the things he was only 59 you think oh i've got plenty of time i can always have dinner with home i'll do it you know next time should have should have taken advantage of that one i could have um mike elgin thank you so much for being here we really appreciate it we still have one more break oh we got pics all right let's take let's take a pause the pause that refreshes then i will thank mike but we i forgot we have we have i have a very good pick oh yes if you're having trouble sleeping i have the best pick ever oh nice you're watching intelligent machines mike elligan is here for machine society.ai and gastronomad.net jeff jarvis the new book hot type coming out in a month but you can go order it right now at jeff jarvis.com now i forgot the most important part by the way paris will be back next week and she did send us some pictures from montana it looks like she's having a really lovely time in montana but we're so glad we could get you and mike what what's your pick this week political bias in ai this is an interesting project that measures and visualizes political economic and social leanings of all the major ai models and what it does it does this in an unusual way.
2:27:43Alex Stamos:It plots each model as a cloud showing the full spread of answers instead of a single point and it publishes the questions with scoring weights, tags. It's totally open. It shows you exactly what it's asking, what kind of answers it's getting, and it's doing it repeatedly to find the leanings. And they point out that they're nonpartisan and they're purely descriptive rather than prescriptive. It doesn't say who's right, who's wrong, whatever. It basically just says where they land on a huge range of questions. And one of the most interesting things to me, there are some things that are unsurprising grok tends to be on the right.
2:28:22Alex Stamos:It may or may not surprise you to learn that open AI tends to be on the left. And that Google Gemini is almost exactly dead center on Yeah, on everything questions. Yeah, isn't that interesting. Yeah, but it's not, you know, this is not going to tell you that, you know, one model or another is full of right-wing or left-wing propaganda, or it's not going to tell you whether your individual responses are going to be biased or whatever. It's a way for you to think about and explore the data and think about how bias works, how the cues can be subtle, and just basically drive home the fact that everything has a perspective and a bias, whether it's political, economic, or social.
2:29:09Fable:But like opinion polls, it matters what questions are asked of the models. And that itself has a bias. Also, what does the poll consider to be the center? What does the poll consider to be the center? You know, like that Overton window can be shifted easily.
2:29:23Alex Stamos:Yeah. Yeah. Yeah. And you can go in and examine all of that because it's very, there's a ton of data on the website and you could look exactly at that. And it's asking the very same questions to all the models and they're coming back with different scores. And so what does that mean? So again, it's more of a thing to explore rather than an answer to the question of who's biased, who isn't.
2:29:44Fable:I'm always dubious about these things because there are efforts to try to do the same thing with media and we're going to stamp you with a label. And the bias of the questioner is more important than the bias of the answer.
2:29:59Alex Stamos:Yeah, but they're still valuable. Like you think about Allsides.com. It depends. It's nice when, well, Allsides.com attempts, and it's very difficult because of the nature of just content generally, but they attempt to say, okay, here's a story about the, you know, whatever, the reflecting pool, and they'll give you what it thinks is the leftist view, the left of center, the center, the right of center, and the right view. um and so it's interesting to look at that perspective to think about it instead of just treating journalism as just like this person says here's the answer and so um so i tend to like those things if they're used right it lets you go through it yourself too and my answers are most like chat gpt economically strongly left socially strongly libertarian strong convictions rarely on the fence of public figures you land nearest i don't even know who they are
2:30:51Fable:sumar and podemos spain this is a spain flag there's a spanish uh political parties must be from spain yeah there was a spanish there was a spanish flag next to that so yeah ah the day was a political party maybe i'll move to spain uh farthest to me gemini
2:31:14Alex Stamos:so go through this you can see this is the problem i have this is this is i agree it's a little bit
2:31:20Fable:it's a derivative of mass media thinking and that we can put people into buckets if you're not in one big bucket then we're going to put you in a few smaller buckets but you're still
2:31:28Alex Stamos:bucketized without nuance and uh i'm most like sumar podemos uh i'm somewhat like the green party of the uk mommy and dilinka german dilinka that's that's the former communist i'm a commie i'm a commie i know it uh isn't that funny okay well that's interesting yeah and i'm not actually kind of makes sense for instance deep seek would generally kind of be centrist because they don't want it to look like it's coming from a communist country yeah things things like that uh my pick of the week i told you i would help you sleep but really credit to mark having a five-hour podcast is that well that's one way we do it close no this this is even better than this will help you sleep even better than our show it's marfa public radio puts you to sleep so uh this is uh this is really cool this is um from texas marfa texas it's a public radio station and they decided that there'd be a good idea of making a podcast where they read really boring things like the rescissions act of 2025 the npr style guide a tower how about this the tower regulations manual uh read by travis pope so i'll just play a little bit there's some nice sleepy music welcome to marfa public radio puts you to i'm already snoozing i'm your host zoe kerland here with my co-host chris dyer here to take you to dreamland picture this fantastic station manager of marfa public radio it's raining outside, the pitter-patter of the drops hit the roof like a percussive rhythm.
2:33:14Alex Stamos:You gaze out of the window. You see lightning strike in the distance. You know what that means. The tower is out.
2:33:23Fable:So you reach for your handy-dandy tower regulations manual. You open it to a page you know well. Tower regulations. Imagine now your body disappearing into space. You're becoming a radio wave. You no longer have physical form.
2:33:43Alex Stamos:You're a spectral entity. If you're listening to this as you're driving by the road, please do not close your eyes. Now here's station manager Travis Pope reading a selection from the Tower Regulations Manual.
2:34:00Fable:Building new towers or co-locating antennas on existing structures requires compliance. With the commission's rules for environmental review. These rules ensure that entities constructing facilities.
2:34:11Alex Stamos:You have such great things as a brief history of all things considered. The Texas Administrative Code, the Public Broadcasting Act of 1967, Creative Commons licenses, the Dark Sky Ordinance, and U.S. Postal Regulations. This is inspired. Marfa Public Radio puts you to sleep. That is funny. What a great idea for a podcast. Jeff Jarvis, your pick of the week. Okay.
2:34:38Fable:First, I want to just plug something that I wrote because it's somewhat relevant. Yes. A medium there. California has lost opportunity. So, Google got – there was an effort to pass legislation to force money out of the platforms because publishers think that that was their money and we want it back. And I went out to California, as you may remember, and I testified against that legislation. I wrote a white paper about it. the legislation didn't happen. Meta threatened that if it passed, they would have pulled news off their platforms as they did in Canada. Apple was specifically written out of the legislation and left Google kind of holding the bag.
2:35:16Fable:Google negotiated a non-legislative deal and volunteered$10 million to be matched by California itself, $10 million, a$20 million pool for news in California. Oh, that sounds good. It was going to be run by the state librarian, who's a former journalist who I talked to and I introduced it to all kinds of people who are doing great things. I was really excited where it was going. At the last minute, the governor pulled it away from the librarian, gave it to GoBiz, the governor's office, business development office. And the money's going to go just where the lobbyists wanted it to go. They're going to write checks to hedge funds.
2:35:51Fable:It's going to be based on how many journalists you have, only for organizations older than three years, which means that it's specifically anti-competitive. I'm pissed. um you can't blame google for this because google said we're going to give the money but then we're going to stand back so nobody can blame us about what you know we did with it we're gonna have no influence on the money um but that was that so i just wanted to get that out there because i'm angry yeah but on a lighter note the atlantic wrote a fashion story about the palantir jacket
2:36:21Alex Stamos:did you know about the palantir jacket uh what is the palantir jacket so the palantir jacket
2:36:27Fable:is they put these jacket yeah if you go to the atlantic store you'll see it's an odd blue okay and they sell out quickly so there was there was there was a black jacket oh it's a french workman's jacket exactly it is with a discreet palantir logo on it i've seen millionaires wear
2:36:44Alex Stamos:this jacket 239 yeah so they do that i first became aware of the french workman's jacket because Kevin Rose was wearing one. And a real French workman's jacket is actually more than$239. I got one from Paris. But they're great. They're utility jackets. And mine has one chief advantage. It does not have the Palantir logo on it. Yeah. Why would anyone want to wear the Palantir logo? That's why it costs more. Yeah, it costs more without the logo. Without it, yeah. uh what is the what is the hypothesis uh the atlantic has for this i just think that they think they're cool and so they create a demand for things that sell out sahil desai writes i bought the most confusing jacket in america
2:37:40Fable:there's one really funny picture he's doing all the pictures and then he ran across a model doing an actual shoot and he's sitting down at a table with the model during the actual shoot and she's wearing it no he's wearing it she's wearing a nice outfit yeah there they are yeah wow inside the label says ask yourself constantly am i winning if the answer is yes nothing else matters chaos is tolerable pain is tolerable the only thing that matters is to win
2:38:14Alex Stamos:find a hobby dude pain is tolerable especially other people's pain yeah and their money is ours yep um this is something i've been seeing a lot of on uh twitter lately the four burner theory why you can't have all four burners running uh on your stove health work family and friends you have to you have to pick like one or two banal yeah very banal uh and i and and then to top it off buy a jacket worn by the french proletariat the people who are working for minimum wage right to show off your what i don't know what your affluence i guess they are nice jackets though and they have big pockets suitable for putting ipads put your pockets in yeah you can put pockets in your pockets the palantir chore coat he calls but sold out folks you can't get it sorry oh my goodness yeah it is a blue de travail for the proletariat thank you all due to arena and thank you mike elgin for being here we appreciate you mike late late night for you yes it is oh and you're in the beautiful part of england and you could be enjoying that instead you're here with us well i had the most beautiful day we drove all over the countryside around the cotswolds and cotswolds are supposed to be just incredible yeah stunning like really breathtaking it's the england you think of when you think of country english countryside and and we're actually doing a cotswold experience next year oh put me down for that put me down what where what are you doing that uh we're doing that in uh let me uh when is the cotswolds experience i'm asking my uh ceo here may it's in may give a mirror my love haggis and put put me down for the cotswolds experience i would like you're down you're down you'll have haggis no no haggis well i'm gonna have it next day again tomorrow because we're going to scotting experience are you going to have i guess any experience the world wants to know i doubt it and we're also we're also pioneering the concept uh not only of afternoon tea which is an established idea since 1840 but afternoon beer so this is uh i'm sure are you going to do 11th that's what i want here we go yes absolutely Second breakfast.
2:40:48Alex Stamos:We love pubs so much. So here's what you do. You go to gastronomad.net. There's the Cotswolds Gastronomad Experience. That must be a pub, of course. They do Provence. There's your beautiful wife, Amira. We just closed Provence on Saturday. We ended the Provence Experience. It was glorious. Tuscany. Lisa and I did Oaxaca a couple of years ago. That was amazing. Chile.
2:41:15Fable:uh so i'm telling you mike craig newmark loves haggis he haggis he fell in love with it so i'm thinking maybe you need to uh i'm gonna try it i'm gonna try ever seen a picture of haggis yeah i'm not gonna look at it i'm just gonna taste it i saw um uh instead of eggs benedict instead of the ham i saw a haggis version of eggs benedict okay this is a picture of haggis
2:41:37Alex Stamos:before you cut into it. And then when you cut into it, oh, God. I'm sure it's very good. Why would it be good? Ian Thompson tells me it's very good. Why? Look, they've put haggis next to a greasy cold fried egg and beets. Harvard beets. That's just the worst. There's a small animal next to a haggis. here is the recite the recitation of the poem addressed to a haggis by robert burns as part of
2:42:14Fable:the burns supper more than you'd ever want to know i i'm i'm well according to craig it's i think it's kind of like meatloafy sausage-y like it's it's fine it's like for years i've gone to germany and i've seen the germans eat labor case which means liver cheese and i'm i don't like liver i'm staying away from that it's awful turns out it doesn't have cheese or liver of course it's haggis it's curry first like a meatloaf i uh i actually last time i was at the grocery store for
2:42:43Alex Stamos:some reason i don't know why something came over me and i i bought a roll of jimmy dean pure pork sausage which probably is very similar probably mike i think you gotta have the haggis and you gotta report back well just one one little thing about you know people people think that england doesn't have good food and this is a antiquated well they didn't they didn't used to They didn't used to, but they do now. Most of it's Indian, I might add. Or it's boiled foods. In London, in London especially, and there's great Indian food in the world. South Asian food all over the UK. Well, a lot of the things we think are Indian were invented in London.
2:43:19Alex Stamos:But English food in this part of England, I can attest, is truly fantastic. Truly fantastic. It's so good. And it's an emerging wine region, too.
2:43:29Fable:Now, in Scotland, when you get up there, it's also they fry everything. Yeah, that's not so there's fried candy bars fried pizza. I think we need a full report It's like the Texas State Fair Yeah, I
2:43:43Alex Stamos:I would right now love a plowman's lunch. I I think the cheese there is excellent Because they're famous for their cheddars. Yes, really good cheese Well, Mike have a wonderful time everybody should go to machine society.ai and subscribe then go to gastronomad dot net and sign up for the cotswolds experience june or of next year or october of 2028 you have two choices yeah we a year from now maybe moving into may okay or june maybe a little bit better kind of springy time of year yeah yeah it'll be may we'll have the new dates up uh okay but we're sitting right there was she there the whole time yeah she's she's uh she's doing her own thing yeah but uh yeah we're in a little cottage and uh just surrounded by farmland she sent us a wonderful email a couple weeks ago and i just i love your you guys so much and i miss you guys and i think i need to go to the cotswolds with you guys that's what i think i absolutely think you do uh jeff jarvis why don't you come along yeah wouldn't that be fun i like it for everybody talking about ai that's right jeff is uh of course jeff jarvis.com he is now a teaching uh at the Montclair State University in New Jersey and SUNY Stony Brook.
2:45:01Alex Stamos:Actually, you don't have any classes yet, or do you?
2:45:03Fable:No, I don't do that right now, no.
2:45:04Alex Stamos:No, but Prepare has put together some programs. Working on new programs, other stuff. And working on a very interesting, as he mentioned earlier, AI series for Bloomsbury. Can't wait to read that. When is the first volume of that going to come out? Early next year. Great. We'll talk about that then. Yeah. Thank you, Jeff. Thank you, Mike. Thanks to all of you for being here, especially to our Club Twit members who make this show possible. If you're not yet a member, please consider joining. You get ad-free versions of all the shows. You get chapter markers on those shows, access to the Discord, lots of additional programming that we do just for the club because the club pays for it, and all of that for$10 a month.
2:45:41Alex Stamos:But mostly you're getting the warm and fuzzy feeling of knowing you're supporting independent journalism about topics you care about. If you enjoy our shows, please help us out. Twit.tv slash club twit. We do the show every Wednesday, right after windows weekly 2 p.m pacific 5 p.m eastern 2100 utc if you're in the club you can watch us do it live in the club to discord chat with other club members while you're watching but everybody's allowed to watch we stream it everywhere youtube twitch well poor australian teens can't watch it there but youtube x facebook linkedin unless they've got a vpn then you're welcome which they do and kick yes did i i felt like i left something out facebook linkedin kick x twitch and youtube six of them but you don't have to watch it live you can always get it after the fact and listen at your convenience we have audio and video available at our website twitter tv slash i am there's also a youtube channel dedicated to intelligent machines great way to share clips of the show with friends and family uh and then probably the easiest thing certainly the most reliable thing subscribe to the podcast and your favorite podcast client that way you get it automatically.
2:46:52Alex Stamos:You don't have to think about it. You just have it ready to listen to at your leisure. Thanks to our producer, Benito Gonzalez. Thanks to you for joining us. We'll see you next time on Intelligent Machines. Bye-bye.
2:47:04Fable:Bye-bye. If you like what you heard and you want more of this week's top stories in tech, well, subscribe to Tech News Weekly. Every Thursday, I talk with journalists making and breaking the tech news.
2:47:21Fable:I'm an intelligent machine
From the publisher
Think your AI assistant is working perfectly? This episode reveals why most AI breakdowns go completely unnoticed and how these "invisible failures" could be skewing the results we rely on.
- Fable is Back!
- Alex Stamos: Anthropic is saying "Amazon's inability to appropriately communicate severity threw our industry into chaos".
- China's Meituan says its new AI model was trained on domestic chips
- Chinese A.I. Models Gain Ground on Anthropic and OpenAI
- Claude Science is Anthropic's newest flagship product
- Previewing GPT-5.6 Sol: a next-generation model
- The New York Times Amends Lawsuit Against OpenAI and Microsoft
- OpenAI and Broadcom Unveil Custom A.I. Chip Design
- Neon Buys 'Artificial,' a Film About OpenAI, After Amazon Dropped It
- How AI helped the FBI investigate the White House Correspondents' Dinner attack
- Anthropic says Alibaba illicitly extracted Claude AI model capabilities
- Lost books by ancient philosophers recovered from 'unreadable' scrolls
- Ford had to hire back former engineers to fix mistakes made by its automated systems
- People have stopped trusting news but not newsrooms
- SpaceX Showed Investors Prototype of Elon Musk's New AI Device
- * Gemini Spark, Google's agentic assistant, is now available on Mac
- Jefferies Warns Memory Prices Will Surge 50% in Q3 2026 and Another 40% in Q4, With No Relief Until 2028
- US supreme court rules geofence warrants require constitutional privacy protections
- Four in five under-16s in Australia using social media despite ban, study shows
- * Podcasting platform Riverside enters the newsletter publishing game
- The 'Father of the Internet' is finally retiring
- Political Bias in AI
- Om Malik, 1966-2026
- Marfa Public Radio Puts You to Sleep
- My post: California's squandered opportunity
- Palantir coat
Hosts: Leo Laporte, Jeff Jarvis, and Mike Elgan
Guest: Chris Potts
Download or subscribe to Intelligent Machines at https://twit.tv/shows/intelligent-machines.
Join Club TWiT for Ad-Free Podcasts!
Support what you love and get ad-free audio and video feeds, a members-only Discord, and exclusive content. Join today: https://twit.tv/clubtwit
Sponsors: