968: Is AI Automating Away All Coding Jobs?

0:00Jon Krohn:This is episode number 968 on AI automating you out of your job. Welcome back to the Super Data Science Podcast. I'm your host, Jon Krohn. Today, I want to talk about something that has been keeping a lot of people in our community up at night, myself included. And by our community, I particularly mean the primary target audience of this show, data scientists, AI engineers, machine learning researchers, software developers. But really, this anxiety could crop up in anyone whose livelihood depends on cognitive work performed on a computer. And the question is, does AI make our careers obsolete?

0:37Jon Krohn:If you listened to my episode last Friday on Open Claw and the Mold Book phenomenon, that's episode number 966, you heard me describe a world in which autonomous AI agents are already writing code, debugging code, iterating on applications, and doing it all without a human in the loop. And if you've now read the viral blog post by AI entrepreneur Matt Schumer, the one titled Something Big is Happening. I've got that for you in the show notes, of course. You may have felt your stomach drop a bit as you read that. Schumer's core claim is that the latest AI models from OpenAI and Anthropic have reached a level where we can describe an entire app in plain English, walk away for four hours, and come back to a finished, tested product.

1:16Jon Krohn:He writes that he is, quote, no longer needed for the actual technical work of his job. That's a jarring thing to read when you're a person who does technical work for a living, and the anxiety is compounded by prominent voices like Anthropic CEO Dario Amadei predicting that AI could eliminate half of all entry-level white-collar jobs within one to five years, or by the IMF's Kristalina Georgieva warning that AI is, quote, hitting the labor market like a tsunami. Or JPMorgan Chase's Jamie Dimon forecasting that America's biggest bank, JP Morgan Chase, would soon need fewer employees. So I'm not here to dismiss the anxiety.

1:55Jon Krohn:The anxiety is rational, but I'm here to provide some much needed counterweight because I think the doomsday narrative is leaving out an enormous amount of evidence. And if you only hear the scary version, you might make bad decisions. You might panic out of a career that's actually going to be more valuable, not less in the years ahead. Let's start with the data because data are what we do, right? A recent article titled Why AI Won't Wipe Out White Collar Jobs in my favorite newspaper, The Economist, presents a compelling evidence-based case, and I want to walk you through some of its key findings.

2:27Jon Krohn:Again, we've, of course, got a link to that article in the show notes as well. But yeah, here are the key findings. First, since ChatGPT launched in late 2022, the United States has added roughly 3 million white collar jobs. Added those jobs. So that includes management, professional sales and office roles. During that same period, blue collar employment has remained essentially flat. So in the very era when Gen AI has been most visible and most hyped, white collar employment has actually grown by millions of jobs in the US alone. Now, you might think, okay, but surely the specific jobs that AI is supposed to be coming for coders, analysts, paralegals, those must be shrinking.

3:06Jon Krohn:No, it doesn't look like it. America currently has 7 % more software developers than three years ago, 10 % more radiologists, and 21 % more paralegals. These are occupations that are regularly cast as AI's earliest victims, and yet their ranks have grown. What about pay? White-collar professionals' wages have held up as well. Since late 2022, real, that is inflation-adjusted wages, in professional and business services have increased by about 5%. Office and admin workers' real wages are up 9%. And if you control for education, age, gender, race, and other characteristics, white-collar workers now earn roughly a third more than blue-collar workers.

3:43Jon Krohn:That premium is nearly triple what it was in the early 1980s, and critically, it has kept rising over the past three years. AI has not, at least so far, eroded the earning advantage of office workers. These findings might seem counterintuitive in the current climate, but they would not surprise historians of technological change. As The Economist points out, we've been here before. In 1982, the Nobel Prize winning economist Vassily Leontief, hopefully I got his name roughly correct, cautioned that computers were beginning to take on first simple and then increasingly complex mental tasks. And the implication was that mass displacement was imminent.

4:19Jon Krohn:What actually happened? Since the early 1980s, employment in management, professional and sales and office roles has more than doubled and pay has risen by about a third after adjusting for inflation. And why is that? The answer is that computers, and I'd argue AI today, rarely replace entire jobs in one go. They automate specific tasks within a job. When a job consisted entirely of routine, repetitive tasks, yes, it could disappear. Think typists and the job that used to be called calculator. It was a human job. but most professional roles are bundles of tasks, only some of which can be automated.

4:54Jon Krohn:The result historically has been not replacement, but upgrading. Technology raises productivity and frees human effort to be directed toward higher value activities like analysis, judgment, and coordination. The Economist's own analysis of more than a hundred large white-collar occupations in the U.S. since late 2022 bears this out beautifully. They classified white-collar roles into four groups based on their task bundles, technical specialists, managers, and coordinators, care workers, and back office employees. Roles that combine technical expertise with oversight and coordination have enjoyed the biggest gains.

5:25Jon Krohn:Employment among project managers and information security experts, for example, has risen by around 30%. Jobs requiring interpersonal care work, judgment, and coordination are also thriving. The only category that has genuinely shrunk is routine back office work. The ranks of American insurance claims clerks, that's tough to say, insurance claims clerks have fallen by 13 % and those of secretaries and admin assistants by 20%. So the pattern is routine work gets automated, complex bundles of tasks get augmented, and entirely new roles get created. That creation part is crucial. AI is already generating brand new job categories.

6:05Jon Krohn:Companies are hiring data annotators to label digital information so that AI can parse it, forward deployed engineers to guide clients through AI implementation and chief AI officers at the C-suite level. The fastest growing white collar occupations in recent years have been those without settled names. Grab bag categories like other mathematical science occupations, which have seen their ranks swell by around 40 % since late 2022 when ChatGPT came out, with real wages rising by roughly a fifth. On this podcast, I'm always going on about how Claude Code is mind-blowing, but now Claude Co-Work is making my jaw drop as well.

6:43Jon Krohn:For example, I recently wanted to quantify how healthy my sales pipeline is for my AI consulting business. I simply asked Claude to estimate my sales for the coming quarter, and it brought info from relevant Google Sheets and my Gmail to create a professional spreadsheet of clients with estimated revenue for each one. Whoa, this might have taken me a day. Instead, it was done flawlessly with Claude Co-Work in minutes. Claude is the AI for minds that don't stop at good enough. It's the collaborator that actually understands your entire workflow and thinks with you. Whether you're debugging code at midnight or strategizing your next business move, Claude extends your thinking to tackle the problems that matter.

7:18Jon Krohn:Ah, and you'll appreciate that I can ask Cowork to show me data, such as my sales spreadsheet, and it provides an interactive chart right in the conversation. For problems worth solving, get started with Claude at Claude.ai slash superdata. That's Claude.ai slash superdata. And check out Claude Pro, which includes access to all of the features mentioned in today's episode. Claude.ai slash superdata. This all echoes research by MIT's Darren Acemoglu and Boston University's Pascual Restrepo, who found that roughly half of America's employment growth between 1980 and 2007 came from occupations in which new job titles and new tasks were being created, roles that were evolving and expanding because of technological change.

7:59Think about it. The job title Data Scientist barely existed 15 years ago, machine learning engineer was not a common role a decade ago, and prompt engineer wasn't a thing even three years ago. New technology doesn't just destroy jobs, it creates jobs that nobody could have anticipated. Now, am I saying everything is fine and nobody needs to worry? Absolutely not. There are real vulnerabilities. Entry-level roles look particularly exposed because they tend to involve narrower task bundles, those bundles we were talking about earlier, with fewer edge cases requiring human discretion. Back office and clerical work is clearly shrinking.

8:31The share of Americans in clerical and admin work has already dropped from 18 % in the 1980s to 10%. And that trajectory is accelerating. New research that I've got for you in the show notes by Sam Manning and colleagues at the Center for the Governance of AI suggests that clerical workers have the weakest capacity to adapt with fewer transferable skills and less scope to move into higher value work. And yeah, worrying is the pace of improvement data because these are genuinely staggering. the pace of improvement on AI models. The research group Meter, M-E-T-R, I talk about them on the show all the time, and I've got a link to them in the show notes.

9:06They track the length of real-world tasks that AI models can complete autonomously. Their data show that the duration of tasks Frontier AI agents can handle with 50 % reliability has been doubling approximately every seven months, with some evidence suggesting that that pace may be accelerating. The latest Frontier models can autonomously complete software tasks that take a human expert over six hours today, extrapolating that trend, and it has held remarkably consistently for six years now. This implies that AI agents capable of multi-day autonomous work are just like a year away. But here's the thing.

9:41Being able to complete a task autonomously in a benchmark setting is very different from replacing a human professional in the real world. Anthropics' own data, drawing on millions of anonymized interactions with their models, show that only about 4 % of occupations use AI across three quarters or more of their tasks. Hardly any roles can be automated in full. As the article in The Economist observes, the one that I've been talking about a bunch in today's episode, today's AI systems have what scholars call jagged intelligence. It's impressive on many tasks, but displays uneven and inconsistent performance.

10:10And being good at 95 % of a task is not enough when the remaining 5 % involves important edge cases and discretion. So what should you actually do with all this? I've got four things. Here's my take. Here's the first one. Don't panic out of your technical career. The data are clear that roles combining technical depth with judgment, coordination, and real-world problem solving are growing, not shrinking. If you're a data scientist or an ML engineer, the demand for your core skills hasn't diminished. It's being redirected into new, more specialized roles. So lean into that. The data science skill set is more valuable now than it was five or 10 years ago, but the job titles and day-to-day responsibilities are evolving fast.

10:52Second, become the person who knows how to work with AI, not despite it. The economist's framing of the cyborg workplace in their article is, I think, exactly right. The future office looks less like the Terminator movies and more like the six million dollar man TV series, human capabilities augmented by machine intelligence. The professionals who will thrive are the ones who can leverage AI to produce work that neither they nor the AI could produce alone. Third, invest in the hard-to-automate parts of your skill set. That means judgment, stakeholder communication, cross-functional coordination, and domain expertise that requires understanding messy, real-world context.

11:29These are the task bundles where AI struggles most and where the human premium is highest. And fourth, stay curious and keep experimenting. The models that exist today will be eclipsed in six months. This is exactly what was told to us by Professor Ethan Mollick, the Worden professor and famed professor a few weeks ago in episode number 962. I would go back to that episode for more tips on how you can be taking advantage of this AI wave if you haven't listened to it. But yeah, stay curious and keep experimenting. The models that exist today will be eclipsed in six months, like the meter data that I was just talking about.

12:02The workflows you build now will need rebuilding for sure. The durable advantage isn't in mastering any single tool. It's getting comfortable with the pace of change itself. Make a habit of pushing AI into harder and harder parts of your work. Find out where it fails. That's where your value is. Now, we shouldn't sugarcoat it. Disruption is real, it's happening, and some people will be painfully affected, particularly in routine back office work and some entry-level roles. That's serious. And we as a society need to be thoughtful about it. But the narrative that AI is about to render all cognitive work obsolete, that there's no future in any human endeavor, as I've heard some people put it, is not supported by the evidence.

12:39Not by the employment data, not by the wage data, not by the historical patterns of technological change, and not by the actual data on how AI is being used in workplaces today. Every major wave of technology from the power loom to the personal computer to the internet has triggered predictions of mass unemployment. Every single time, what actually happened was more nuanced. Some tasks were automated, many jobs were transformed, and entirely new categories of work were created that nobody saw coming. There's every reason to believe AI will follow the same pattern, albeit faster and at greater scale.

13:09These are wild times, no doubt, but I believe they are more exciting than they are frightening, particularly for people in our community who have the technical foundations to ride the wave rather than be swamped by it. The key is to start adapting now, not out of fear, but out of curiosity. You've got the skills, sharpen them, redirect them and build something great with the extraordinary tools at your fingertips. And finally, yeah, that's the meat of the content in today's episode, but we do have some reviews to get through that listeners have been leaving out there. So this one's from Hadassah Galapo.

13:41She's an AI ML research scientist at Valka AI in the US. She said some really nice things. She said that this is one of the most high quality and practical AI podcasts out there. The show creates space for both deep technical, thoughtful discussion as well as genuine human moments. She gives some examples of those moments. She says she's been listening for two years. Thank you, Hadassah. And she consistently walks away learning something new and feeling inspired. Super cool. I hope folks have also been inspired by today's episode and are feeling more optimistic about this crazy AI technology that is changing things so quickly.

14:14Thanks to everyone for all the recent ratings and feedback on Apple Podcasts, Spotify, wherever you listen to your podcasts, as well as for all the likes and comments on our YouTube videos. Bonus points if you leave written feedback. If you do, I'll be sure to read that on air like I did today. I mostly see the Apple Podcasts comments that are in the US, but all in the future. I might switch to other regions to see all the great reviews that you've written around the world. So that's it. I hope you enjoyed today's episode and I can't wait till next time. Until next time, keep on rocking it out there and I'm looking forward to enjoying another round of the Super Data Science Podcast with you very soon.