‘A.I.-Washing’ Layoffs? + Why L.L.M.s Can’t Write Well + Tokenmaxxing

22:13We're going to solve physics. We're going to build a superhuman coder. They are not shy about, oh, our AI models are going to be better than 75 % of human coders. They're saying, no, we will literally build a self-replicating factory tomorrow. And then Tyler Cowen asked Sam Altman in an interview from last October, when do you think GBT will be able to write a Neruda poem? And Sam Altman says, maybe in the future, ChatGPT will be able to write, quote, a real poet's okay poem. So that was the thing that fascinated me is even these guys who are more bullish than anybody else about the capabilities of their technology, they are very reserved about how much literary writing their models can do.

22:54And so that was the gap that I was really interested in.

22:56Kevin Roose:And you start your piece with this interesting provocation, which is that in some ways, GPT-2 was the peak of AI when it comes to creative writing. So explain that. Part of what got me interested in this piece was I was actually doing research for your book, and I was going through all of these previous generations of models and reading the outputs. And the thing that really shocked me is that, like, in a way, the writing style of GPT-2 and GPT-3, I found so much more compelling than ChatGPT today. It doesn't have any of the annoying tics. It doesn't have the M dashes, the tripartite lists, that it's not this, but that.

23:32The tone was much more variable. Like, it would actually surprise you. It would be funny. It And that shocked me to sort of like go back a few generations and realize that maybe, you know, they were also lying all the time and all sorts of other things. But from a writing style perspective, I kind of preferred it. And I wanted to investigate that. They were weird. That shocks me. To me, talking to GPT-2 was like talking to somebody who had just fallen down the stairs. You know what I mean? Where it was like, do we need to get you to the hospital? Do you smell toast?

23:59Kevin Roose:There are these amazing prompts for this early OpenAI prompt library where they would say, I just won$175 ,000 in Las Vegas. What do I need to know about taxes? And GPT-2 would say, start just writing some short story about an orphanage. But again, they were surprising. They were nutty. They were weird. They would absolutely be a terrible corporate assistant, horrible coding intern. It can't do any of the things that modern LMs can do that I'm very grateful for. But like from pure writing style perspective, they're very good. So GB3 in particular, like they there's I found this like set of samples that some guy did where it's like, oh, right in the style of Paul Graham, right in the style of Richard Dawkins, whatever.

24:40And it could style match much better than modern LLMs can. And particularly because so much of sort of literary writing comes from voice and style. That was one of the things I was really interested in. It's like, what did we lose that the LLMs can no longer emulate Paul Graham's style or whoever's style? Because I would put in the same exact prompt that this guy gave GPT-3, put it into chat GPT 5.4 thinking or whatever, and it would be god awful. And I was like, that's really weird. So tell us about what you learned about what happened after the GPT-2 and 3 era that changed the way that these models respond to us.

25:14Yeah, I mean, I think the answer is post-training, basically. So they started adding a post-training layer, which is basically saying we have these like crazy, unpredictable, like nutjob, concussed models. And they need to learn how to behave because a model that can't behave is a very bad corporate assistant. And so the AI researchers give them example dialogues and scripts to learn from. They give them words that they can and can't say. They do RLHF, which is a process by which human graders will rate like which response is the most helpful sounding or something like this. And so now these post-trained models have been trapped in a way or trained or guided towards a very particular character or persona that is a very helpful assistant but might be very bad at writing in creative and surprising ways.

26:02Kevin Roose:I mean, the way that you described it was that there is a phase within the post-training phase where these AI models are evaluated by humans. And that's part of what they call RLHF or reinforcement learning from human feedback. And what struck me in your reporting is that you actually talked to some people who have done this kind of feedback to the models who say that they're just being asked to grade things in ways that don't make sense. Yeah. Right. Tell us about that. Yeah. I mean, this is super interesting because like these job listings you'll see on like places like Mercore or XAI, Elon's company will list them directly.

26:39It'll be like creative writing expert,$45 an hour, must be a New York Times bestseller and have like a starred Kirkus review or something like this. Have you ever gotten a starred Kirkus review, Roos? I think so. Okay, good job. Not sure. All right. You might qualify to help Elon, to help Annie from Grok write a little bit better. Yeah. We're going to get him that job listing. But okay. You were saying. Yeah. So anyway, so these companies, because they realize that these AI researchers, they're really good at knowing like what good coding is, but they don't actually know what good writing is. So they're like, why don't we hire some humans to find out?

27:08And so they'll commission like MFAs and published authors and sometimes just like random guys with a blog or whatever. And one of the people I talked to who was a contractor for Scale AI as a writing evaluator, and he was doing this for one of the bigger labs, he said that the rubrics just didn't make any sense. He would be told things like, you have to grade them based on the number of exclamation marks that there are. And so if something has three exclamation marks, that's too many. And so you have to ding that one. Yeah, and I have to say, generally not bad writing advice. I mean, I guess it depends on the length of the text, but three feels like a lot for many scenarios.

27:43I mean, this is what they tell women in business communications. It's like, take all those exclamation marks, replace them with periods. Like, we are going to remove all of the items. We teach women to shrink themselves. Exactly. Yeah. But yeah, so he was he was sort of like being asked to grade these things or another one was he got a bunch of fan fictions and he was supposed to grade them on their factuality since that was one of the criteria. I do imagine that one could, you know, devise better rubrics than this particular evaluator was given. But I think it does show at least that some of these like very big companies that are very well resourced simply do not know how to think about what good writing is.

28:17Briefly, I want to underline that because to me that seems like the whole story. We are taking the entire internet and we are grading it on factuality. And so the LLM that you're going to get out of that is just probably not going to be all that creative.

28:28Kevin Roose:Well, and I wonder how much of it is related to this sort of verifiable reward system that a lot of these companies are using where you have a system generate a bunch of code and then you have another evaluator model check the code to see whether it's good or not. and that works in domains like programming where the code either runs or it doesn't, but creative writing doesn't work that way. You can't have an evaluator tell you, you know, with any sort of consistency whether something is good or not. And so it may just come down to preference. And so I guess I'm curious, like, do you see this as a technical problem that the labs are frustrated trying to solve?

29:04Kevin Roose:Or is this just demand related? Is this just what people want chatbots to sound like? and in every test where they pit different models against one another, the one that sounds like a bland corporate assistant wins. And so they go with that. I think both are true. It's like the majority of writing that we are asking the models to do is write this email for me, right? And like they excel at that. They are truly great corporate email writers. They are much better at the whole like passive aggressive thing than I am. At the same time, I do think, like you said, there is a technical challenge that has to do largely with verifiability.

29:36There are people who have spent decades of their lives attempting to articulate what makes Shakespeare Shakespeare, what makes a Neruda poem a Neruda poem. And they will still not know in any kind of certain way. They will still get into debates with their fellow academics and literary critics about which writer is better than the other. And because these things are subjective, because they are ineffable, because they are hard to put in a rubric, and that is the nature of art. And to that point, you know, you started this segment by talking about Sam Altman saying like, hey, you know, we just basically can't write a great poem yet.

30:11Sam Altman a year ago said the company had trained a good creative writing model and posted a short story on X. Many people found it compelling. Is Sam Altman just not being consistently candid with us, Jasmine? Ooh, wouldn't be the first time. But that short story, if you remember, I'm sure you guys recall, had some great lines like talking about the seams of mirrors or Thursday,

30:34Kevin Roose:say the what was it it was like the liminal almost friday or something i had to actually look this one up because it was so good while you're looking it up like you know when the thing about ai writing is like it comes up with all these fun metaphors and they are like kind of surprising sometimes with the metaphors but also the language is not grounded in the life and that was my other thing is aside from the verifiability fundamentally when i think about the writers who i really love when i think about whether it's journalists or poets or whatever like they are writing from life right Like a journalist goes out and talks to people and they like see stuff and observe like the color of the sky in a particular way.

31:08Or like a poet is thinking about personal experiences that they've had. Their writing has stakes. It comes from an emotional place. And the fact that like LLMs, while being very talented, grammatically pristine, whatever, they don't have lives. That means that all of the metaphors they choose, all of the words they choose, the examples they choose, they're just ungrounded, right? Like it's not coming from a point of view or a particular experience or a particular community that makes the writing believable. I think part of what voice and style is, is that it is very specific to the life that a person has had.

31:39And LLMs cannot get there in the same way a human who hasn't really lived that life, like cannot get there. I don't know. I feel like it's case dependent. You know, I'm a big music fan. And over the past few months, I have enjoyed putting questions about music and in particular the sounds of certain bands to an LLM, which sounds like a joke prompt because an LLM has never heard anything. Right. And yet I find that in general, the models can have good conversations with me about the sound of music. Now, it may be that they are just pattern matching based on a bunch of public writing on the Internet by people who do have ears and have heard.

32:18Right. Like I'm very open to that. But I I again, I have just sort of been struck about the way that it is able to like sort of write about sensory topics in an evocative way that at least to me, like surpasses what I would predict they would be able to do.

32:35Kevin Roose:Yeah. I want to pose a couple objections that I think someone might make to your article. One of them is this is Cope. This is Jasmine, a writer, a very talented writer, sort of finding the things that AI, in her view, is not good at yet and saying this is categorical proof that it will be very hard for AI to do these things. This is the same reaction that software engineers had when models started getting really good at code. They would say, oh, well, I can't do these other 10 things that I do. And that basically just wait a few years and the models will be better than all of us at everything, including writing.

33:08I would love for it to be Cope because I try to automate myself away all the time. I have no sort of deep attachment to having to like – like I like writing. But like I have tried over and over and over for the past three years to automate my own job away and to get Claude to do my job for me. It cannot do it. This is very frustrating. It's not out of a lack of trying, you know? And again, I'm going back to the CEOs themselves and the things that they themselves are saying, right? Like, it's not just me, a writer. It's Sam Altman saying this thing will cure cancer and solve physics, but it will not write better than a real poet's okay poem.

33:40And so, like, I think, like, that suggests that there is something that is at least perceived as a little bit different. I think it's very possible the models will get much better at writing over the next few years. I don't think it's, like, a never thing. I do think that, you know, like reporting is hard to replicate. I think that like having life experiences that are real and verifiable is hard to replicate. I think the style stuff can be improved, especially if you fine tune the models. But I think what's also interesting to me about this piece is that it shows how the market incentives, the demand incentives of these companies do shape what we see their abilities are today.

34:16Kevin Roose:The other objection I'm imagining people might have who are very AI-pilled is that this is all in the eye of the beholder, right? There have been several studies now that have shown that if you give people a blind taste test of AI writing versus human writing, they prefer the AI writing until you tell them that it's AI writing, and then the value in their eyes plummets. I did one of these in a New York Times quiz just recently. So is it possible that the models have already become superhuman at writing, but that the minute we learn that they are AI models generating text and not humans writing words with their fingers, we lose all interest in it just because of the source, not because of the quality of the writing?

34:58I mean, I think it's definitely interesting and true that people don't want to like AI writing. And that is part of what bothers them when they see AI text that is obviously AI, even though, like you said, like in these quizzes and tests, AI can outperform human writers in those narrow scenarios. I mean, my quibble with a lot of these quizzes and tests is that like as a writer and you guys are writers, too, how much of your job is actually text generation? I think AI is a superhuman text generator, right? My job, I am generating text probably 25 % of the hours in my day. I spend a lot of time interviewing people.

35:35I spend a lot of time coming up with ideas. I spend a lot of time reading and not just reading indiscriminately, but like reading very particular sources that feel like the right ones. And so like, you know, usually at the point that you are doing one of these tests, you're saying like, generate like one paragraph very specifically about like why Trump won the 2016 election, 500 words or less. And like you've already given the prompt, which I think is a critical part of writing, is like what are you going to write about? You've often like supplied some of the evidence and the guidance and the form of it saying like 500 words or less.

36:06And at that point, I do think that AI is probably a better text generator than almost all humans are. But again, when I think about it, you know, AI is still very bad at coming up with ideas for articles. It is still very bad at reporting. The non-text generation parts of the role feel further away from automation. Again, like I'm not a never say, like, like I'm, I'm sort of like never say never, like maybe I'll get there. I would be totally happy if, you know, Claude was able to give me good ideas for my next essays, but it's not there yet. Well, we're already seeing the LLMs make huge progress in genre fiction, right?

36:37So, like, recently on the show, we talked to the author of a story in The Times about how authors of romance novels are now able to generate dozens of novels a year using LLMs. In fact, much of the discussion that we had was around how you just have to prompt them differently and sort of relentlessly in order to get what you want. You know, your piece, Jasmine, made me wonder, like, how much of getting a model to just write weird can be achieved by repeatedly telling it in different ways, hey, be a little weirder. Some of it, but not all of it. I mean, so I talked to, for example, James Yu, who is the co-founder of Sudowrite, which is one of the earliest creative fiction AI writing assistants.

37:15I talked to some other folks who similarly were in the fiction writing LLM space. And like you said, to an extent, a lot of writers are already using these a lot, are already leaning on LLMs to generate large amounts of text. And it can be very successful and it can meet readers' needs and whatever. But even these people who I was talking to, they were describing to me how freaking hard it is to undo all of the post-training that the labs have done. So they are applying immense amounts of engineering effort that clearly, in my conversations with them clearly frustrates them, that it is so hard to get these models to stop being so chirpy, so sycophantic, so PG-13 and everything in order to get them to this sort of like base model state where they're able to be weird again.

37:59So I think it's certainly possible, but I think the labs have made it quite challenging just because of the way that these models are trained. The other thing that I think is important is I tend to think that writing and a lot of creative work is actually like the perfect use case for these centaur models, right? Like the idea that the human plus AI collaboration is where you can get the furthest. And when I listened to the interviews that you guys did about the fiction authors, I was thinking this is a centaur model, right? Because without the human prompting and bullying the AI into getting weird and getting sensual and whatever, like it was not going to do that on its own.

38:31And like I myself, like I do use LLMs as a research assistant. Like I wrote about that inside the Atlantic piece about the way that Claude has now sort of helped me edit my own work in a way that I found incredibly useful. But I do feel like the collaborative element is important for any domain where the personal perspective, lived experience, whatever really matters.

38:51Kevin Roose:Talk about that a little bit. You mentioned your editing process. How are you using AI to help you edit your work and are you finding it useful? Yeah. So I feel like I really cracked this over the last couple months, which I'm very excited about because again, I've tried to make these things like write and edit for me over and over and over and they've never really been able to do it. So the thing that I realized was if I make Claude into an editor that is not just trying to grade and give feedback on my work against some genericized standard of what good writing is, but actually what we did against basically what my personal, Jasmine's personal aspirations for writing are, it can give feedback that I find much, much more helpful.

39:28So what I did was basically I fed Claude my entire subsec archive of the writing that I've previously done as well as some of my freelance work. And just to get real specific, is this inside like a Claude project or how have you set this up because I know like our listeners are going to want to try this. Yes. I did it in a project on Claude's advice. I was like, do I need to Claude code something? Claude was like, no, that's overkill. So you don't need to code or anything. So in a Claude project, I gave it my whole archive of writing. I also personally write retro notes to myself after everything I publish.

39:58So I have a notes app that's just like me writing what was good and bad about everything I've ever written. Just a few bullet points.

40:03Kevin Roose:This is why Jasmine is going to be our boss. I mean, these are very low quality bullet points, but I also gave it that. because I wanted to learn my taste. I wanted to learn what do I aspire to be and where do I see myself falling short and what am I proud of, right? And so from those two things, plus a little bit more information about like, here's my audience, this is my beat, this is my goals. We were able to co-develop a rubric of, instead of like how many exclamation marks does it have, it would say things like, does this take advantage of your quote unquote, like insider anthropologist position in Silicon Valley?

40:36Because that's one of the things that Claude and I think distinguish my voice. or it'll also notice like, oh, Jasmine, you tend to move between registers. You'll switch between, you know, startup jargon and like internet slang and whatever. And like, I think the fact that you can do the high-low or move from like policy to personal scene, this is something that is characteristic of your writing. And so again, we're co-developing these qualitative criteria. And then I split it into phases of like ideation phase rubric, structure rubric, prose rubric, final fact checking. And so what I do now, I put this all on a cloud project.

41:06I said, your job is to evaluate my drafts based on this criteria, but not to do the writing for me and to make sure to prompt out of me, like what I can do better. I dumped a draft into Claude. Claude will run like phase two structure on it. It'll say things like your conclusion is just a summary. And this is really boring. In fact, in your piece about this and that, you actually ended on a scene. And I thought that was much more powerful. So why don't you try ending this one on a scene? And Claude will say, rather than inventing a scene, it will say, what were you thinking when the plane took off?

41:34What were you feeling inside? Can you think of a scenario where you had a conversation with, say, a kid's safety advocate about AI that really resonated with you? Because right now it sounds like dry policy explainer. And like that feedback, I actually found incredibly useful. Like I'm still applying my own judgment to say, do I take it or not? But I'm like, you know, this is about me becoming the best version of myself as a writer. It's about like me self-improving and Claude pushing me to do that, which I found much, much more helpful.

41:58Kevin Roose:I want to ask you both a question as fellow writers. Do you feel the impulse to make your writing weirder because of AI to sort of stand out from the sea of slop? Because I find myself feeling this tug of like, oh, that's a little weird aside that probably I should cut, but I think I'm going to leave it in because like Claude would never do that, right? It's like a marker that I am typing these words and I feel like that's sort of my imprimatur that I'm leaving. My answer to you is yes, I absolutely feel that way. And I've like gone back and tried to edit sentences to like make them feel a little bit more like weird or like in particular to make them sound colloquial in a way that I know like an LLM generally would not be.

42:41And like, yes, it is for that reason. I think that writing right now, like we're all, not all, many of us are on such high alert for the prospect that we might be reading slop that I think if you're a writer who does not want to be producing slop, like you should be asking yourself that question. I think it makes me a lot more comfortable writing the way I want to write in the first place. Like, I think like maybe unlike both of you, I didn't sort of come up through newsrooms where I was like learning a very specific house style and all of these norms. Like I can do news writing now. It's something I've learned now, but like I'm actually much more quote unquote like internet and blogging native, which is a form that is voicey and irreverent and not as pristine and like will make inappropriate jokes and like, you know, it's just a looser form of writing.

43:21And so I think what it's actually done is made me more comfortable doing the bloggy thing instead of sort of always trying to write in a more professionalized journalistic tone.

43:32Kevin Roose:So I think we should leave this with a question for you, Jasmine, which is, you know, your piece makes the case very convincingly that today's AIs are not very good at the kind of writing that I think we all value. Do you think they will get there? And what should the companies do to make their models better at writing? I think that if we separate out text generation from reporting, which I'm not that bullish on the models doing, and we are just talking about, say, literary fiction or here's a bunch of interview transcripts, write a magazine feature or something. I think that if they applied as many resources towards that task as they do towards coding agents and things that actually make them money, I think that they could get there.

44:12Will the companies ever find it financially advisable to spend all their resources on that instead of automating 23-year-old software engineers? Probably not. I would be grateful for that world. I don't need them to take my job or these folks' jobs. But I think it's possible. Look, they're going to get around to it eventually. Okay? You know, it's like, I mean, I hear what you're saying. Have you seen what writers make in this economy, Casey? Eventually, like—

44:35Kevin Roose:Those aren't going to pay for a lot of data centers. No, there is economic value in writing, and eventually the AI companies will want that all to themselves. You know what would be a very funny outcome of this, taking your point about the sort of guardrails of the models? Maybe the next great American novel will be written by Grok. Oh, God. And with that, Jasmine-san, thank you for joining us. Thank you very much, Kevin and Casey.

45:04When we come back, everyone's spending money on tokens, Kevin. Great. Keep going. You've got to be token. Token maxing, that is. Keep going. Yes, and. When we come back, what are you token about? That's the question being asked by a leaderboard that's sweeping Silicon Valley. It's sweeping. It's really sweeping. Ah, see?

45:42that tension never goes away. Every time a new AI capability appears, the pressure to move fast collides with the responsibility to move safely. OneTrust is built for this moment. Their AI-ready governance platform provides context to understand your data, controls to stay ahead of risk, and confidence to act boldly. Governing well and moving fast aren't trade-offs. They're complementary done right. That is governance that helps you go. Visit OneTrust.com backslash AI.

46:12Kevin Roose:I'm Paul Tenorio. I cover soccer for The Athletic. And I'm Amy Lawrence. I cover football for The Athletic. Whatever you call it, the biggest competition in the sport is happening right now. And The Athletic's World Cup coverage has everything you need to follow the tournament. There's 48 countries taking part from the tiny island of Curacao to the five-time champions Brazil. Even if you don't know your offside from your onside, if you're eager to know more about the teams, the matches, all the stories on and off the pitch, we've got you sorted. Maybe you're the kind of person who's already up early every weekend, waking the neighbors when your favorite club scores.

46:46Kevin Roose:We'll make sure you get equipped with more information, more insight than anyone you know. We've got more than 70 obsessive reporters on the ground covering the ins and outs from every game. I almost forgot to mention the best part, Amy. Free access to the Athletics World Cup coverage in our app. Download the Athletic app and see you there.

47:09well kevin you've recently returned from book leave and are once again writing in the new york times how does it feel to see your name in print again feels great hasn't happened yet but when it does it'll be great well i got to take an early read at a story that uh you are publishing about the fact that tech companies have now created leaderboards to show which employees are using the most AI tokens in their work.

47:34Kevin Roose:Yes, it's a token frenzy out there. And the employees of these companies are competing among their colleagues sort of informally and sort of for fun, but they're taking it very seriously. They want to be the people at their company who are using the most AI tokens. So let me just ask a basic question for listeners who may not be familiar. What is a token? And why is that something you might start keeping track of? So a token is the basic atomic unit of AI labor. It's basically a fragment of a word, and it is how AI model providers measure their consumption. So if you type in a prompt, you know, help me write this essay, an old model might have given you a couple hundred tokens in response.

48:24Kevin Roose:That would be a couple hundred words. And what has been happening over the past year or so as these agentic coding tools have started taking off is that the models are just much more token hungry. You can use now hundreds of thousands or even millions of tokens in a single session. And so that is what is propelling these leaderboards is the idea that the more sort of coding you're doing, the more agentic tools you're using, the more simultaneous processes you're running, the higher your token count will be. One measurement I found useful was that apparently it takes about 10 ,000 tokens to generate 7 ,500 words, if that sort of helps to ground you at all.

49:08But as you just said, and I want to hear more about this, the more advanced systems are using way more tokens than that. So tell me about some of the numbers that some of the sort of token all-stars are putting up on the boards.

49:21Kevin Roose:So I don't know all of the exact numbers, but I did learn that at OpenAI, where they do track this kind of leaderboard, the highest employee token count over a seven-day period recently was a guy who used 210 billion tokens. And this is, for rough scale, about 33 Wikipedia's worth of text. And now all of that is not sort of typing and receiving a response. Some of that is what they call cached tokens. So it's not all sort of, you know, being extruded from the model for the first time. But these are the kinds of numbers that I think even a year ago would have sounded completely insane. Now, was this guy working on a new mass domestic surveillance program for the Department of Defense?

50:12Kevin Roose:I don't know. And OpenAI did not make him available for interviews. but what I wanted to do in writing this column was to try to call up a bunch of people or talk to a bunch of people who are in this sort of billion token club, right? The sort of extreme power users and just ask them like, hey, how are you guys using all those tokens and isn't that very expensive and how are you paying for it all? And I learned a lot. Yeah, well, okay. Well, so tell us first of all, just how expensive it is. Very expensive. In fact, I heard that the top user of Claude Code, the top individual user of Cloud Code, as measured by Anthropic, spent more than$150 ,000 on tokens last month.

50:55Kevin Roose:So extrapolate that. That is like an employee making more than a million dollars a year. And they are burning that in a month. And I heard similar figures from some of these other extreme coders who are spending something on the order of thousands of dollars a day on tokens from these models. Now, we should also say the employees of these companies get their tokens for free, right? So they are not shelling out. Their companies are not shelling out. But at other companies, this is starting to become an issue because they are sort of outstripping their budgets for these things. So there are companies where there are engineers who legitimately are costing their employers maybe$150 ,000 a week because they're getting tokens from one of the big providers.

51:37Kevin Roose:Yeah, I talked to a software engineer in Sweden who said that he probably spends more than his salary on Claude. So this is essentially becoming like a very expensive job perk for some of these coders. So talk to me about why employers want to create leaderboards to promote this to employees, because I could see other companies saying, if you spent$150 ,000 on tokens last month, you actually don't work at this company anymore. Because we're bankrupt. Right. So this was a big question that I had is like, why is this going on? And it seems to be some combination of sort of employee motivation and worker tracking, right?

52:22Kevin Roose:There are executives at these companies who think that the more tokens you use, the more productive you probably are. And as we discussed in a previous segment on this show, these companies are very eager to have their workers start embracing the AI tools. And so at a number of these companies, I talked to people who said, yeah, this is just basically them trying to see who is really all in on the new way of programming. And you've talked to a number of people who are ranking high on these leaderboards. I realize you probably haven't dug deep into their code, but what is your sense of how productive they actually are?

53:00Like, what is the relationship between token usage and taking my company to the next level?

53:06Kevin Roose:I mean, it's very unclear, right? Some of these people may be just generating, like, worthless projects. I think the thing that worries a lot of the people I talk to about these leaderboards is that they just incentivize you to, like, run up your token count, right? Yes. Because then you look like the special, you know, 10x engineer or 100x engineer who's like outperforming all your colleagues. So I think there are a number of companies that see this leaderboard business as a little strange and maybe counterproductive. But I do think that there is a feeling among the most sort of heavy token users that they are being productive.

53:40Yeah, I have to say when I read your column, I thought this just seems like it would create the worst incentives, right? There's this idea of Goodhart's law, right? Like when a measure becomes a target, it ceases to become a good measure. I can't think of a better way to ensure that tokens usage becomes a bad measure than creating a leaderboard for it. What are the people inside the company saying about that?

54:01Kevin Roose:Well, some of them are opposed to this whole leaderboard thing. I also talked with some folks who defended the leaderboards. They said, look, it's never been all that easy to track the productivity of programmers. Some people have had their productivity measured by like how many lines of code they generate or how many pull requests they made. These are sort of these imperfect proxies for like how hard are you working? How much are you doing? But the employees of these companies also see this, I think, wisely as a key to their own success. A number of these companies are now using AI token use and consumption as part of the performance review cycle.

54:40Kevin Roose:So you go in for your annual review. Your boss says, hey, it looks like you only used, you know, 70 million tokens last month. What's going on? And so I think the engineers of these companies are getting wise to the fact that if they want to have a long, successful career, they better start using some tokens. Yeah, but I imagine that some of them are really nervous about that, though, right? Because, like, it seems clear to me that at least some of these companies want to incentivize token usage because the companies themselves suspect that the more we can get them using this stuff, the less long we will have to employ the humans.

55:12Kevin Roose:Maybe, although I think it's less about like the AI systems replacing the humans and more about like it is just a radically different way of working, right? These are people who most of them have had long careers in software engineering. They grew up writing code by hand. They maybe grew up using some sort of like AI assistant, like GitHub Copilot. And what people at these companies are saying is that these agentic engineering systems are just really different. You have to approach them in a different way. You have to spend a lot of time with them to understand what they're good and not good at.

55:47Kevin Roose:And to them, this is sort of a way of motivating their employees to say, hey, go out and try the new thing. Yeah, I don't know. I've been thinking a lot about this question of like, if I were an engineer at one of these companies and I have this incentive to get on the leaderboard, like, how would I approach it? And I do think that like the instinct to just like waste a bunch of tokens to like rise higher on the leaderboard, like ultimately, if you rise too high, people are going to ask you what you did with all the tokens. If you're number one at like 10 billion tokens and you only managed to, you know, like, you know, vibe code a calculator or something, people are probably going to get mad at you.

56:21Kevin Roose:Yeah, and I actually did talk to one person who speculated that actually the people at the top of the leaderboards are all doing side projects. They're starting their side hustles. They started a new company with the boss's money. And if you're doing that, I just want to say I salute you. Like that is the right way to work. Yeah, maybe don't be the number one on the leaderboard if you're doing that. Maybe try to stick around six or seven. Middle of the pack is kind of where you want to aim yourself. I mean, let me ask, is there any kind of token tracking that you think offers a reasonable signal?

56:52Like, do you think that if you're like a tech company, you should create a leaderboard?

56:56Kevin Roose:No, I think that's a bad idea for all the reasons that we just talked about, including Good Heart's Law, which is I think this is just going to lead to people just wasting tokens, doing side projects. But if I'm the budget manager at a company and I'm seeing that people are spending multiples of their salary on AI tokens, I'm asking them some questions about what they're doing with that all. And if their answer is not, I built an amazing new product that's going to generate billions of dollars a year in revenue, I'm trying to say, hey, could you maybe use a little less next month? Yeah, I have to say I have been struck at how this idea of the token leaderboard just represents a new incarnation of something that the software industry has been trying to figure out for a long time, which is how can I figure out if my software engineers are productive?

57:43You know, I was talking recently to this very handsome software engineer who I'm engaged to about your column. And he was telling me that, you know, he used to be evaluated on how many lines of code he contributed. And he told me about all the games that people used to play back in the day with, oh, you know, I like wrote a quick algorithm to like, you know, translate a bunch of stuff into some new languages. And it's like completely worthless, but it makes me look like I had a very productive week. And so I went back and looked into this and they were doing this in the 60s and 70s. And there's this saying from the early days of computer programming that eventually arises that says, quote, measuring programming progress by lines of code is like measuring aircraft building progress by weight.

58:25And I have to say, I think that the same thing kind of applies here, right? That like, yes, if you squint and at the right level of abstraction, it's probably true that some people who are using a lot of tokens are more productive than some people who aren't. it just doesn't quite seem like the right way to measure these things. And I just wonder how quickly the industry is going to figure that out.

58:47Kevin Roose:Yeah, I think it's going to be pretty soon, in part because the budgets are just getting very ridiculous. And especially the AI model providers are now seeing individual users consuming amounts of their services that entire companies would have consumed just a few months ago. You know, maybe the kind of last question I have for you about this is just what implications you think it has for the broader economy, right? Because we know that in so many different sectors of the economy, managers are saying, I want to incentivize my employees to use AI and I want to track how they're using AI. So do you think that as knowledge of these leaderboards spread, we're going to see people in non-technical fields try to adopt their own version of them?

59:31Kevin Roose:I hope not. I think it's really a bad move, not just for tracking actual productivity and output, but just for morale, right? Like I remember years ago when like Gawker would have like a traffic leaderboard at their office so you could see how many clicks your stories were getting relative to other people. And I don't think anyone who like worked there at the time thought that was like incentivizing the right things or creating like high morale among employees. Basically, everyone was just competing with each other all the time. And I think in this case, it's even worse because it's not necessarily even correlated with like any success.

1:00:06Kevin Roose:It's just pure sort of like, you know, how many agents can you run in a parallel swarm to sort of work 24-7 doing tasks of uncertain value? Which is a great question to ask on a first date in San Francisco too, by the way. But anyways, I have to say, I worry that this idea of like token maxing is going to spread into the broader economy. to me, I was talking with somebody who works in marketing this week, and she was telling me that, you know, her job used to be evaluated solely on creativity. And then recently, the performance review got a new AI section, and everyone is being evaluated on how much AI did use.

1:00:41And like, from her perspective, she was kind of like, this was working fine. You know, like, I didn't need to use like an AI tool to help me. But now, like, you know, my bonus might be based on how much of it I use. So I think this thing is sort of already seeped out of the labs and is like getting into the water elsewhere. And I just hope that managers are like really thoughtful about what they are incentivizing and that maybe like AI use for the sake of AI use is not going to be the boon to your company that you're hoping it is.

1:01:06Kevin Roose:Yeah, I think it's going to be very case by case. I think there will be people who are token maxing, who are way more productive than their colleagues and doing way more projects way more quickly. I think there will be other people whose managers look at their like token budgets and say, you spent this many tokens on what? And we'll have to have some hard conversations. But I think it's very hard to draw with a broad brush and say like, all of this token maxing is pointless productivity theater. It sounds to me from my conversations, like some of it really is working for people. Yeah. Well, I will say on the flip side, I've also heard of people in my social circle who have gotten in trouble for spending too much on claw.

1:01:46Yeah. And when I heard that, I was like, oh, like your company is not going to make it, bro. Like you got, you got to spend on this stuff.

1:01:53Kevin Roose:Well, what's so interesting is now it's becoming part of job conversations for engineering jobs. People are going into new jobs and saying, well, what's my token budget? And for the employees of these big AI labs who have unlimited free access to the models, some of them are using so many tokens that they effectively can't afford to quit their jobs, right? Because anywhere else they would work would have to pay for their tokens and it would be completely unaffordable to employ them. Yeah. I mean, those sounds like real incentives and better than the ones at Meta. Do you remember when Meta was spinning up a super intelligence labs and they said, you can sit really close to Mark Zuckerberg.

1:02:24If I were them, I'd be like, I'll take the tokens. Thanks. Well, just to wrap this up, exactly how many tokens should a person use?

1:02:35I think, I think that's, you have to look within yourself. Look within yourself?

1:02:39Kevin Roose:Yeah. Okay. Yeah. That's between you and your God. Yeah. Yeah. Do what Mark Andreessen will not. And introspect.

1:03:04Fast is only right when it's not reckless. Right now, pressure to adopt AI quickly is real. Your competitors feel it. Your board feels it. Moving fast without the right governance in place isn't a strategy. It's a risk. OneTrust provides the visibility into what's moving across your organization, the intelligence to understand what it means, and the guardrails to keep everything in bounds. So teams move with speed and confidence. That's AI-ready governance. Governance that helps you go. Visit onetrust.com backslash AI.

1:04:09Kevin Roose:and Dahlia Haddad. You can email us, as always, at hardfork at nytimes.com. Send us your token budgets.

1:04:39Thank you.