1016: In Case You Missed It in July 2026

0:00This is episode number 1016, our In Case You Missed It in July episode.

0:09Jon Krohn:Welcome back to the Super Data Science Podcast. I'm your host, Jon Krohn. This is an In Case You Missed It episode that highlights the best parts of conversations we had on the show in the past month. My first clip is from episode 1013, where I sit down with Dr. Kathy O 'Neill, Harvard math PhD, former hedge fund quant, and author of the mega bestseller Weapons of Math Destruction, a decade on from that iconic book. Kathy makes the case that what makes an algorithm terrifying is not its complexity, but other features entirely. Listen in. Yeah, we were a lot less vulnerable, I suppose, back then when these automated systems were so much simpler, where random forest or support vector machine is kind of as complex as it gets.

0:57Jon Krohn:I mean, I suppose there's still a lot of room for exploitation there. But now, I mean, even 2016 is when weapons of math destruction came out and a ton has happened since then in terms of AI capability. with large language models, deep fakes, short form video being created by Gen AI, surveillance pricing that has largely become dominant since 2016. Do you think we need like a weapons of math destruction sequel that is going to talk about all these kinds of high powered AI capabilities that make us even more vulnerable? I think the things that make algorithms terrifying aren't their complexity or their technological advancement.

1:43It's a combination of the secrecy, the unaccountability that nobody in particular is in charge of checking if it's right or in charge of mistakes. And the fact that people absolutely have to use them. They don't have choices. They could be flow charts. They could be linear regressions and often are, or logistic regressions. It don't have to be complicated for them to be terrifying and exploitative. So, I mean, a lot of the algorithms that I wrote about in weapons that, you know, are still there. And there's probably the most terrifying algorithms of all. Like I would say, I would argue there are new ones.

2:27Like I would say the AI chatbots that tell kids to kill themselves would rank up there as one of the most terrifying of all. So there's, I'm not saying there's nothing new under the sun, but I'm saying like the ones that I wrote about are still awful and they haven't been addressed. Um, to answer your question though, could there be a sequel? I actually considered writing a sequel. One of the things that I, um, that changed my mind is that everything is changing so quickly that it would like, as I said, it takes four years to write a book. It would be completely outdated by the time it went to China to be published and then came over on a ship that, you know, it's 12 months later to be sold.

3:12Um, and that's why I started my podcast. I like literally started the AI skeptics podcast. Cause I was like, yeah, it takes too long to write a book nowadays. Um, you can't address these issues four years from now, but yeah, I, I guess, I guess, um, I I've always considered the complexity to be independent of the terrifying of an algorithm of a system. Like the, absolutely the most terrifying one of all for me was the one of called like recidivism risk algorithms that like sometimes judges were using to sentence people to longer in prison based on risk score of them someday getting rearrested, still being used, still sending people to prison for crimes they have not yet committed and might never commit.

3:57And still, and by the way, simple as pie. Sometimes they're just questionnaires and And then points are added up like they're it's stupid how simple they are. But I still think they're unconstitutional. I don't understand how they're used. I don't you know, just doesn't make sense to me. But the power is in the algorithm and the power is what is terrifying.

4:17Jon Krohn:That makes a lot of sense. Since 2016, it seems to me like there might be some places where we can now be using algorithms that previously would have been ineffective. So in chapter seven of your book, for example, you mentioned how productivity management models are optimized for efficiency and profitability, not for justice or the good of the team. And for a long time, things like the movement of hourly workers in warehouses, fast food chains were monitored to enable automation and other efficiency gains. But now with large language models, we can be doing things that we couldn't do before. So, for example, Meta announced its model capability initiative, MCI, I think that was this year, just a few months ago, which is a plan to track every keystroke and mouse movement of employees to train its AI models, suggesting that the kind of demeaning degrading of labor that has long affected blue collar workers on, say, factory floors or fast food chains is now affecting even the highest paid echelons of white collar work at places like Meta.

5:27Jon Krohn:yeah, there wasn't really a question there, but I feel like you might have a response. I especially like the way you ended that, which is like, I interviewed truckers and teachers and people who had gone to prison or being denied parole based on recidivism risk algorithms. And they were, you know, let me just give you a little non answer to your non question. Like they, those are the people I was talking to when I was writing the book. And I was like, this is demeaning. This is humiliating. This is degrading. It's dehumanizing. Truckers are working for algorithms. They're not, they're working for surveillance systems.

6:08They're not working for themselves anymore. I used to, by the way, load trucks in high school. And I met a bunch of cowboy, like cowboy truckers back in the day in the eighties, they were just absolutely cowboys. I mean, they were also hopped up on all sorts of different kinds of drugs, but really interesting, very, very interesting, kind men, mostly men and independent, really independent, like independent businessmen, you know, in a very small business, usually just themselves. but the the uh that no longer exists that model of the cowboy trucker is just not possible anymore and that way of life just being removed and being replaced by something that is much much less interesting and sexy and human it's it's it was sad but you know true and this the teachers that were being fired based on almost a random number generator, similar teachers have been underappreciated, underpaid because back in the day we had free labor basically from women like Louisa Mary Alcott, you know, like, or, um, Laura Ingalls Wilder, like they were just like super smart women.

7:27They were only allowed to teach. So they were underpaid. Anyway, The point being that like, I was working like with people who are workers and they, they had a very strong notion, even back in 2014, that when I was interviewing them that like, yeah, these algorithms are working against me. They are, this is something that's happening to me. This is happening to me. And then I remember I was asked to give a TED talk, like the year after my book came out, because it was a, you know, it was pretty popular books, as you say. So I went to the TED main stage and I gave the TED talk and I talked to some audience members and they were so excited about big data.

8:08And they were just like, oh, this is happening for me. This technology is happening for me. It's going to make me more productive. I'm going to have, they were really, Elon Musk came the same time that I came. And I, even back then, by the way, I just was like, this guy is a jerk. and I'm holding myself back because I know you bleep people on this podcast, but people are just super into this idea that Neuralink, they're going to have an automatic connection to the internet and they'll be smarter. They're going to be smarter and faster and more productive and things are happening for them. And I just was like, this is the new divide.

8:48And that was 20, whatever, 2017. I was like, this is, this is a cleavage in our society, which is only going to get bigger. That's what I meant when I said how big data increases inequality and threatens democracy. Like the cleavage is the people who think this systems are working for them versus the thing, people that think these systems are working against them, or it's happening to them, which is even more like sinister way of thinking. But to go back to your point, John, and sorry to ramble, but for me, what's happened with AI and like the engineers who are, whose keystrokes are being measured, blah, blah, blah, which by the way, that happened to me in the hedge fund, like all the keystrokes were being measured then too, just for spying issues.

9:38But yeah, so white collar workers are getting what blue collar workers have had for many, many decades, which is a humiliation and degradation and dehumanization. And like, in some sense, I think that's a good thing. Not, not that it's a good thing. It's not, but it's, it's good to have more people on the side of the workers. You know, it's good to have solidarity to understand, yeah, this is, this is real. This is happening. AI is stealing your ideas, stealing your work, you are no longer necessary because we've already stolen the stuff that we need. Not really true, though. That's the good news.

10:17Going back to my skepticism about whether digital work is really going to be replaced by AI. No.

10:24Jon Krohn:Some stuff will be, but not all of it. Kathy closes on white-collar workers now getting the surveillance and degradation that blue-collar workers have lived with for decades, though she remains skeptical that digital work will really be replaced. In episode 1007, the founder of 80 ,000 Hours, Ben Todd, back for his second appearance on the show, takes that premise seriously and asks, what happens if we do get a fully automated digital worker? Well, we cover where solid ground is left for human careers, why the bottlenecks then move into the physical world, and how fast a robotics buildout could really go.

11:02Jon Krohn:If a frontier lab comes up with a fully automated worker that basically can just plop in, can do anything that a remote worker could do over any time horizon, you know, it can do years of work kind of independently. It checks in at sensible intervals. It does everything. It's imagining that somebody that you've only ever had Slack email and Zoom conversations with, there's no technical reason why you couldn't have that be something completely automated in the future in that scenario like you were kind of when i asked you about where you know there's going to be solid ground for years to come and you were like well just always focus on like that little bit you're going to be able to you know drive a lot more value but in that scenario where there's this fully digital worker i mean it makes the solid ground feel pretty narrow doesn't it Yeah, I mean, just quickly, there's still there could still be legal issues, right?

12:01Like an AI wouldn't be able to own a company, for example, and like there could be liability issues like that. So that would still, but yeah, but I mean, if you actually get the full digital remote worker, then yeah, the bottlenecks move to these things that either have to be done by humans for some reason, such as like legal ownership, or maybe just like the consumers have a really strong preference. for it to be done by human. But then there would still be the physical bottlenecks, right? So there's still jobs that require physical presence. Somebody's got to put those GPE racks together for now.

12:39Well, yeah, I mean, this is kind of what's happening is like a lot of like construction and energy and data center building. These are big growth areas and they're also complementary with AI.

12:51Jon Krohn:For sure. But I've also got to believe that if we have AI systems clever enough, to be complete digital workers, then it's not going to take those digital workers very long to be figuring out how to be automating the hardware stuff too, the physical installs. Again, I suppose this... Yeah, exactly. That's what I mean. Like having physical embodiments, you know, there's still some testing that needs to happen with physical embodiments in a way that, you know, software scales more easily because you can just copy the model weights and you can do that almost for free and almost instantly. Whereas with robotics, it scales a little bit slower because you have to manufacture something and you need to test it.

13:38Jon Krohn:But there's all kinds of ways that you can use simulations in order to be able to train robot arm model weights much more rapidly than from physical real world data alone. You probably still want to do some final testing in the actual real world before rolling out some product, but some hardware product. But, yeah, there's a lot of, yeah, even the construction of these data centers and stuff, you could imagine being done by robots in a world, you know, not maybe just a few years after we have these fully automated digital workers. You could imagine that it being, you know, them starting to make a really big impact in the physical world as well, right?

14:18Totally. And I, yeah, I think people sometimes are a bit too, like they think this might, they kind of assume this will be a very long process, which it might be. But I do have a, I have a Substack post about how quickly could you scale up robotics? Because imagine in, you also need to remember in this world, right? If you actually have a digital remote worker, then like the physical bottlenecks become just like the whole bottleneck on, on the whole economy. So you then have this potentially massive mobilization to try to solve that bottleneck by the world's biggest companies. And so what they might be able to achieve in that type of scenario could be pretty dramatic.

14:59And one analogy you could look at is something like how quickly was airplane production ramped up during World War II? And that was partly done by converting car factories into plane factories. And so like a one very rough estimate is like, if you converted all the car factories into robot factories, how many robots could you make? and just on a kind of like mass per mass basis it would be something like a billion a year and in world war ii they were converted in a matter of years um obviously robots are significantly more complex like the hands are much more complex than cars so maybe that's like a false equivalence but i still we do have a lot of industrial capacity that could be could be converted and it could it could go pretty fast i think i mean also this is a world where you have ai aiding you in all of the steps like we have now like a full well it would effectively be i think it would effectively be superhuman when this is another thing people don't understand is like once you get like a human level digital remote worker you're basically getting like superhuman abilities immediately after because you can speed them up like 50 times so they could like like a day for for us is like a month or two for them.

16:12And there's all the other AI advantages, like they can share their state space across the whole, all their copies. So like any learning can be just like immediately propagated to. Yeah.

16:21Jon Krohn:Yeah. And the same way that autonomous vehicles, you know, already have lower crash incidences than humans, but humans aren't constantly learning how to drive more safely. Whereas based on more data being collected, based on improvements in algorithms and sensors, autonomous vehicles are always improving. And, uh, yeah. And if that 50 X thing, you know, you can, I think, I think the point you were making there is that a task that could take a human a couple of months might, you know, could take hours or days for machines because yeah, you could have 50 agents working in parallel on different parts of a problem.

16:58Yeah. Another big aspect is the kind of coordination aspects where companies are pretty inefficient because it's like there's a lot of communication overheads between all the different people but if you have ai workers like they can all kind of communicate instantly across the across the firm and you know you can also have you can have the ceo effectively like personally supervise every single worker which is obviously like a huge bottleneck now So these AI firms could actually, even if they just had like human levels of intelligence, just through being able to coordinate much better, they might be able to move way faster than human firms.

17:43Jon Krohn:Agents are getting smarter every day, but even the smartest agents get stuck without the right context and the right tools. That's where Notion comes in. With the recent launch of custom agents, Notion became the collaborative AI workspace where teams and agents work side by side. and now their new developer platform is turning that workspace into infrastructure developers can build on. The piece I keep coming back to is how easy it is to ship something real. The CLI authenticates in one line, workers deploy without provisioning any infrastructure, you write your code, deploy, and you're done.

18:14Jon Krohn:For me, that unlocks building purpose-built tools for my custom agents with the predictability and custom logic I need. Think a guest prep agent that pulls of researchers' papers, recent talks, and citation graph on demand. Tools my agents can actually call with parallelism and predictable behavior, not just hope for. Learn more about Notion's developer platform today at notion.com slash superdata. That's all lowercase letters, notion.com slash superdata to try Notion's developer platform today. And when you use our link, you're supporting our show, notion.com slash superdata. Those are forecasts at the scale of the whole economy.

18:52Jon Krohn:My next clip brings things right back down to individual people. In episode 1009, venture capitalist and five-time entrepreneur Steve Mock walks me through the patterns emerging from the stories he collects at his website, aisavedme.org. Chief among them, that the people getting the most out of AI in healthcare are not asking it for advice, but using it to become far better informed advocates for themselves. Stick around for the story of the Australian dog whose life was saved by AI. And you've mentioned to me that one of the big patterns that you've seen emerge from the submissions on AISaveMe.org is AI being used for healthcare.

19:29Jon Krohn:Do you want to dig into that? Sure. So what I'm starting to see now, because we're getting lots of stories on the site, is we're starting to see now is patterns within categories. And going back to my, you know, let's go back to my 84-year-old dad. I'm trying to answer his question, how do you use AI? So there's spot use cases. Oh, you can use it like this. You can use it like that. You can use it like this. And then I'm starting to see what I'm going to call a best practices layer appearing in different categories. So in healthcare, for example, I've got a number of stories. And nobody is saying, I went to AI and asked it for healthcare advice and did it.

20:07Right? By the way, there's people that do that. We read about it in the press. And I think this is where the education piece comes out, which is – and this is a very simple example. But what all the people in my stories are doing is they're going to AI and they're using it to educate themselves on the subject matter of whatever their predicament is. And they're learning how to then ask thoughtful questions to their medical provider. And they're getting – they're not getting better at healthcare from AI. They're getting better at asking educated questions to their healthcare provider. So again, it's helping us become our own advocates, which we have to do.

20:49And that was otherwise a very expensive proposition before the LLMs existed. So don't ask AI for healthcare advice. We get it because we're in the thick of it, but a lot of people don't get it. We're reading stories in the paper where bad things have happened because of that. But the best practice, which I'm hoping rises out of the stories in the site, which I'm hoping we get the word out more, it's these types of patterns, which is use it to become a better advocate. Don't use it to be for health care advice.

21:17Jon Krohn:Makes a huge amount of sense. I already relayed a story to you socially a month ago when you and I met up around a relative of mine who had been diagnosed years ago with Parkinson's disease. and his primary caregiver, whom I'm hoping will actually submit to AICME.org. But until she does, I'm also, I don't feel like I should mention people by name. But she, you know, the point that you're making ties into how nobody is more invested in your health than you and your caregivers. That's great. The healthcare providers, you know, the vast majority of them are great, but they have so many conflicting things on their attention, and they're trying to get as quickly as they can through your case and on to the next one and get through their day.

22:09Jon Krohn:And so, you know, they're not going to spend as much time as you can, you know, as the patient yourself or as the caregiver for a patient, because, you know, if something is really big, a life-changing thing like a Parkinson's diagnosis, you know, it's kind of every day. There's things you need to be dealing with. And you can start to notice things that are different from maybe what your healthcare provider has said or the guidance that they've given. And so, yeah, this relative of mine, it turns out that after years of Parkinson's diagnosis and being given Parkinson's medication, through increases in medication actually making symptoms worse.

22:48Jon Krohn:Wow. So that caused a conversation to happen with AI and say, you know, I'm seeing these strange, like, should this be happening? Like, it seems like things are getting worse, not better. And that led to a situation where, you know, they were able to get a completely different diagnosis. They had to go to a different physician. And, you know, now they're still working through exactly what is going on. But it seems like it isn't Parkinson's. And so the treatment is very different. And so it seems like a path that looked quite scary and seemed like it was getting worse quickly, actually it isn't the path you need to be going down.

23:24Jon Krohn:That wasn't the medical problem. It was a treatment problem. Yeah, you're reminding me. And that's what we're also seeing. One of the stories I have on my site is a woman whose dog was chronically ill, and she had gone to a number of doctors. And the doctors didn't have an answer to what the problem with the dog was. She took all the symptoms, put that into an LLM, and then the LLM came back and said it could be this, this disease that she had never heard of. And then she took that back to her doctors and they said, oh, we can test for that. And then that was what the problem was. So, again, it's helping be your own advocate.

24:01It's a resource to help understand better and work with your practitioners for that.

24:08Jon Krohn:But don't you also have a wild story on your site of somebody maybe in Australia using AI to come up with a treatment for their dog? So actually like inventing a new treatment with the help of AI. So this is Paul Cunningham, and he's an entrepreneur data scientist in Australia. And I had the pleasure to connect with him over this story. But this is well written about. I highly recommend people go read this story, and there's been a lot written about it. But his dog had cancer, had tumors. And the dog was resisting the treatment that they gave. And he then got biopsies of him. This is a pretty crazy story.

24:53This is not what your average person is going to do. He's an exceptional, you know, he's a dog lover. And he said, I'm going to do everything possible. He got biopsies of the tumor. He then went and had their DNA sequenced. He then used those sequences to go and find what possible treatments out there could match. He found an experimental treatment that they could actually match that particular cancer. Then he went and tried to buy it. And because it was experimental, they refused to sell it to him. So he then found out that you can actually make your own treatments through RM &A vaccine labs.

25:31And he went to one of those, had his own custom treatment made. Then they applied it to the dog. And within a month, the tumors had shrunk away. And it's an incredible story. And what's even more incredible is all but one of the tumors shrunk away. And it turned out that it was a different type of cancer. So it wasn't reacting to the other type of cancer. So he did the whole process again. And then last I spoke to him, he had just given that second treatment to his dog. But he talks about now, he said his dog is, you know, last I spoke to him, his dog is thriving, and it's a great, it's an incredible story.