Nobody Is Getting New Manager Training for Their AI Team - with Dan Klein, UC Berkeley

24:40Just because you can get there in the limit of infinite everything doesn't mean there's not a much, much better way. So for practicality of people sitting out there building stuff and where most people I would imagine use Claude, Gemini, whatever, you think that there already now should be a, I guess, more of a strategy of saying, let's just look at what the world of models look like and then figure out what is the exact use case we have for our AI use and then decide if there's a better model. And I'm not sure that that is even happening right now. Is that a fair assumption? I think for some things that is like, that is absolutely the right strategy is to say, what are, what performance characteristics do we need?

25:20What do we not need? What kind of model exhibits those performance characteristics with the optimal efficiency, be that data efficiency or compute efficiency at deployment or availability of compute nodes or whatever it is that is your constraint that you care about. And there are going to be some problems that are best solved by just bigger and bigger models trained on bigger and bigger things. Those are problems that involve breadth, problems that involve sort of very complicated contextual understanding. But where you have problems of determinism, reliability, truth, like that is not the best attack we have today, which doesn't mean people shouldn't be doing it that's why people are doing that in terms of pursuits of unreliable general intelligence and reliable specialized intelligence are going to require different methods i was like still curious about the the small team making and it seemed to be such for an entrepreneurial point of view it seemed to be such a big opportunity that you can find a specific area where you can create a specific model you're so not that tiny right because you're basically saying I'd like to make a model for people.

26:26Small is relative. Yeah. So like I understand that. But it's just curious of the opportunity from an entrepreneur to kind of be now looking at, you can actually build a foundational model. You can't compete with the open design of the world because that race is probably done. But you probably could go out and find a bunch of use cases where a specific model will be very useful and then go and make land. Yeah. I think because of the successes on the axes that benefit from scale, people are very much now thinking about scale, scale, scale. And, you know, obviously that requires a ton of capital, a ton of compute.

27:05It requires big teams because anything that's scaled up requires a whole bunch of support structure. But again, our models work in different ways. And the focus of our model is not extracting sort of the full breadth of human knowledge from the information as found. It's, as you mentioned before, we're working on a specific class of kind of interactions, which I would characterize as interactions where you have a person who has all the kind of contextual situation that is relevant to human language in terms of having a conversation, referring back to things that have happened before. On the other side, you have a set of functionalities, you can call them tools or APIs that have logic behind them and ways they can be chained together and semantics that govern what flow of information through those tools means.

27:59And it's the person talking to this orchestration of backend functionalities. And that is a huge class of interactions that share a bunch of properties. They need to be reliable. Like if you say you want three tickets, it says here's three tickets, but it's secretly only booked one. Like that's bad. You're going to find out there's going to be a high cost to that. So on top of that, from the standpoint of the APIs, like there are going to be policies and rules and ways in which these things can and should be used. And those rules may change and they need to be, you know, sort of specified in ways that are changeable and declarative.

28:36And so this is just like for this kind of interaction, we just noticed that the existing models are not only expensive, they're not very good at this. and that being able to build a model that is better, there's an upside to it also being smaller. But you also just can build a model that's better when you architect that model fundamentally to be structured around these sorts of operations. And that's the approach we took. I think it's really important, like, right now there's really two kinds of companies that are dominating the market just in terms of like number of companies operating in these ways.

29:11One is the companies that are very, very big. They're doing everything at like the most massive scale imaginable. And there's companies building these thin wrappers. We're like, what is the technology there? It's like, it's probably some prompt. And I think, you know, I guess you have a very appropriate title to this podcast. I think we are trying really as a community to figure out what is beyond the prompt here. And for us, that is models that have additional control surfaces that have additional characteristics, performance characteristics, reliability, the ability to guarantee certain kinds of behaviors.

29:43I mean, in building things as a society, one of the most powerful tools, if you teach CS101, right, the most powerful tool we've had historically for building large, reliable systems has been modularity, the ability to take pieces, work on them independently and say, this piece, we're going to work on it, but we're going to guarantee you that this kind of input produces this kind of output. And there's a contract and there's an abstraction. And this has been one of the biggest challenges in this AI age is LLNs come with absolutely no contracts beyond you will get tokens out if you put tokens in.

30:15And this is one of the key things that it was clear to me needed to be different for a specialist model that was going to be deterministic. Like there needed to be guarantees that you could make about what goes into the control surface. You need to be able to say how that relates to what comes out on the other side. If you want to be able to build reliable things out of it. So we started there and we started thinking about what kinds of systems you could build that led us to how are they structured? What kind of data do you need? And now we're into the synthetic data training. And it turns out that the amount of data you need to get a certain behavior, that sort of sample complexity question can be different by orders of magnitude.

30:54It's like learning French from a French book versus incidental French uses in English novels. It's just a very different scale characteristic and it's a better operating curve to be on. It doesn't mean the other approach doesn't bring you gains in different cases.

31:10Dan Klein:Can you tell us maybe as just to make it very easy to imagine. What's a quintessential deployment? If you look at this is a textbook deployment of scaled cognition, who's the user? What are they trying to do? What's the impact to their workflows or life or business? It's a great question. So I would say the textbook deployment is, again, the class of conversations, if I put abstractly is conversations between a person on one side and a bunch of APIs on the other. This might show up, for example, in like an enterprise to a customer where the customer is doing, maybe it's a customer support kind of thing, or, you know, changing a flight, changing a hotel, you know, making a purchase, getting a refund, whatever, things like that, where the person comes with all of the context and everything they want to express is in language.

32:03And on the other hand, there's like a whole bunch of APIs that can handle this. Like, Who are you? How are you going to get authenticated? What's your purchase history? Okay, what exactly are we talking about in your account balance or whatever it is, and navigate all of that stuff in accordance with policies. For us, our typical partner is going to be an enterprise. They want to build an agent. This is fundamentally an agent to a model. They want to build an agent. And it's very important to them that the system be reliable, policy compliant, and also secure in a variety of ways. So we didn't talk about this in our system, but the way we've built it lets us make a lot of kinds of information compartmentalization guarantees about where that information will go and what can and can't be leaked through training data, things like that.

32:49And our system has a bunch of advantages there. So our best customer is some enterprise that cares a lot about not making mistakes, not having things like hallucinations or policy violations. and where essentially the conversations they have, they want to automate them, but it's high stakes to get it right. Now, so far, pretty much all the enterprises we talk to feel like it's important to get things right and that their conversations, their customers are high stakes. But if you think about finance or health care or cases where not only do you not want to mess up for your customers, but where the consequences might be health consequences or financial consequences or regulatory consequences.

33:31then there's even this even greater sensitivity to wanting to make sure that the systems are doing what you've instructed them to do that you have audit traces for that that you have control surfaces that you can change the behaviors if you need to change the behaviors and so on why do you think that a open ai when they launched like their health care gpt recently you know what what would be their way of thinking about this because clearly they think that their system will work just fine doing it? So first of all, you also have this issue that any company with a big, you know, hammer is going to go treating everything like a nail, right?

34:07And so you can absolutely take a generalized probabilistic intelligence and try to do something specialized with it in the same way that I can train a person to compute square roots, but my calculator will do it faster and more accurately and with a whole lot less energy use. And so depending on the problem you're trying to solve, there are going to be multiple approaches to it. And constellations of non-deterministic models, clearly people are trying that. Now, you actually do see a lot of news articles about these sorts of things face planting, either because they're not reliable, or they, instead of following your refund policy, they follow some refund policy from Reddit in, you know, 2005, or, you know, you get them off topic and suddenly your customer service agent is talking about something you absolutely do not want screenshotted and shared.

34:56And so there are failure modes to these systems, but, you know, it's certainly you can chain these things together and make a go at it. I just don't think that's, it's not gonna be the most reliable way. It's certainly not the obvious way to me. It's just, if that's the only tool you've got, that's what you do. And for a company that either is building these big models or is wrapping them, that's the approach they're going to take. And for what it's worth, if you're a big model company that sort of cells buy the token you're probably okay with mechanisms that require spending tokens to check the other tokens to check the other tokens like it's just token you same thing with by the way with you know people talk about reasoning models reasoning is a big and maybe ill-defined class of models but a canonical kind of reasoning model would be just to give a caricature as like run the thing 10 times and then look at what you've got and pick the best one that would be a kind of like reasoning for maybe a problem that's like a lock and key verification kind of problem If you're the person who is paying for 10 times the compute, you don't love that as a solution.

35:59If you're the person who's being paid per compute, this sounds amazing. And so I think some of these factors are at play too.

36:07Dan Klein:That's hilarious. The unreliability is actually a feature for the model provider if it's being paid on number of at-bats. Totally. Absolutely. When you teach students about AI, what is the most important thing you think that they should know as they leave kind of like the course? If I had a compact answer, then my course would be a lot shorter. I think that answer has changed. so when i started teaching ai you know something like 20 years ago one of the things that we did early on is we showed this checklist of things that humans can do including like you know playing chess or going to the supermarket and we'd sort of like ask the class like okay raise your hand you think you know you think a computer can do this do you think a computer can do this and you know can a computer play go can a computer drive a car can and in the past 20 years it went from mostly no to mostly yes and so at the beginning we would really focus on these core ideas of like what is ai what are the kinds of problems you know um deterministic versus non-deterministic adversarial versus cooperative single agent versus multi-agent and when we talk about these different specialized kinds of problems which required specialized solutions and specialized representations.

37:34Back then, the reason why you would have some people worked on computer vision and other people worked on natural language processing, other people worked on robotics, was because to get any of that to work required incredibly specialized representations, incredibly specialized algorithms, different kinds of data, different kinds of learning, You know, everything was different. And we would focus on understanding that kind of breadth and understanding what unified it, which at the time was this, you know, this notion that, you know, artificial intelligence, what is an agent? An agent is a system that makes optimal decisions, given its information towards its objective function.

38:11And we talk about that. And we still do all of that because that's all still relevant. But now a couple of things have happened that are interesting. One, what you could talk about is there's a lot more uniformity to AI. As a natural language person, I think it's great that we've decided that language is kind of a good operating system for AI. But very much now, if not an LLM, then at least the underlying kind of transformer technologies are being used kind of very broadly. And so, of course, one of the things we talk about now is that sort of thing. but one of the things that I think is very important now that we didn't talk about before is to start getting into these large-scale societal trends we started talking about digital literacy AI is gonna have a huge impact on how people learn how people work what jobs are available for for people and when the key problem in AI was that nothing worked except maybe like some game playing here or there.

39:00We spent a lot less time on that than now when the technology's downsides are like we could have a whole class and do on those sorts of things.

39:12Dan Klein:On literacy itself. Yeah, I think I saw a stat actually from a online education platform, so perhaps slightly polluted as a source of truth, but something like 1 % of enterprises are investing in skills. It's just, you could say digital literacy or AI literacy, but why do you think that is? I mean, 90 %-ish are investing in the tech, but a small fraction seem to be seeing that there's a literacy problem here. Why is that? Is it again, going back to the kind of objectively it seems like it works kind of a thing? I think there's a couple of things going on. I think there's more like Thomo attached to missing the wave on the technology, like enterprises are being transformed.

40:01You don't want to be the only one that's not. And that feels like it's about the technology. I think people have underestimated the skills and human training aspect of this, that whatever technology there is, you can get more or less out of it depending on how humans engage with it. I think also people just like society as a whole has underestimated how poorly the instincts we have to digital literacy translate, right? Knowing how to find information with search is actually pretty different than knowing how to validate information that comes out of a, you know, a chat system like chat GBT. one way to think about that is people who were doing a lot of writing or researching now are doing something that looks more like review or editing.

40:53And the difference between being a writer and being an editor is really big. But if you're used to being a writer and now some large language model is writing your email for you, you don't think, ah, now I'm an editor. I don't have those skills. You think, oh, it just did my job for me. I can just press send. and I think it's going to take people time because this technology has appeared so quickly I think it's going to take people time to realize that for all the skills that maybe are less necessary there's an equal number of skills that not only are they very necessary but we're not good at teaching them we don't even have good names for what they are right what is the skill of taking

41:29Dan Klein:a fluent looking output and distrusting it well you know one one skill that's a little bit higher order, but I think is a similar kind of a challenge is the skill of delegation and, or you could even say management. You know, I think it's funny for all that we talk about AIs as assistants, how few people have actually ever had an assistant, right? So how do I work with an assistant? Well, I'm learning with this chat bot, like I'm getting my on the job managerial training with a chat bot that's sycophantic and right. It's not, that's not a recipe for success. you know, but people aren't going to AI literacy training in the same way that there's new manager training, right?

42:11Dan Klein:How do you delegate? How do you verify? How do you check people's work? How do you mentor? Right. Those are all, those are things that typically professionals learn over the course of a career. And now we're all given intelligence on tap. And the problem is actually people don't know what to do with an assistant, let alone a capable, you know, junior your employee, right? Yeah. And the optimistic take on this would be, well, people will like learn these things over the course of a career. It just hasn't been a course of a career length of time that we've been working with these systems. And there is more going on because I think a good, if you're working as a human with another human and you're trying to delegate to them, you do trust them to come back and say, well, I actually couldn't find this information for you or I got blocked as opposed to I couldn't find it, but here's my best wild guess.

43:03And I'm not going to tell you it's a wild guess. That would not be good behavior from a human, but we see it all the time for machines. So I think there is the problem that you mentioned, which is that people may not have the skills to delegate and manage humans. And then there's the additional layer that these systems do not act the way a human does in a delegation context, right? especially as far as the sort of metacognition say more about the system doesn't act as a human

43:30Dan Klein:does in the delegation context because it was a little garbled at least for me i just want to make sure because i think that's a really important point to me i think a lot of this boils down to something called metacognition which is i think if you had to put a finger on what systems don't have today you know we've been talking a lot about reliability which is ultimately the feature that they lack today determinism reliability whatever you want to call it if you think about them as cognitive systems. The thing they're lacking is metacognition. In humans, we don't just think, we think about those thought processes.

44:00When you ask me a question, I stop and I think, do I know the answer? And maybe I do, or maybe I don't. If I don't know the answer, I may make a decision to bluff. I may make a decision to just keep quiet or to change the topic or like I get to decide what to do about that lack of information. But the fact that I have an explicit representation of whether or not I have the knowledge, knowledge about knowledge, cognition about cognition. This is metacognition. Systems don't have that. Back to the example of, you know, of what's the population of Berkeley, right? It's just cranking out tokens if that's coming from the parameters.

44:36Whereas a database would be different. A database, you would do the query and you get the answer and you display it, or you don't get the answer and you say, entry not found. So the database does not have the breadth and the contextuality and all of those other kinds of specifications the AI system has, but they are in some sense more metacognitive. They know whether or not the information is present. And ultimately, you know, full intelligence requires both. An LLM today lacks the metacognition.

45:02Dan Klein:So this is, and to your point about the person in the delegation relationship, they would, what you're saying is they have the metacognition or the self-awareness to say information not found, effectively. So a manager comes to me, hey, can you do this? You go, I studied biology, not physics or whatever. I don't know how to do that. Now, I just want to push back a little bit or at least explore the resistance. I had an experience just yesterday with Claude as an example, which can serve as that information not found. Because I read Ethan Mollick's post about giving Claude code an assignment to generate a new$1 ,000 a month business or something.

45:41Dan Klein:I just, I thought that was kind of fun. I just grabbed his prompt, dropped it in Claude code. And it was kind of interesting because Claude, you know, immediately came back to me, Jeremy, I got a level with you. There's no such thing as a business that generates a thousand bucks a month with no effort. Now, what I can do is this. And, and to me, it's, if it were purely a function of sycophancy and one and next token prediction i think it would just probably optimistically say oh you could do this the fact that it kind of to me that was an example of pushing back how do you square that example with uh the definition we've kind of been working with around what language models yeah so i would say there are three levels where what you're talking about happens and what i've been talking about is sort of the caricature the simplest version of a completion engine systems are evolving right and systems aren't just purely um trained to produce mimicries of web text there are additional steps of training there are things like um alignment training instruction training if you do something like rlhf and you are showing the system okay in this situation I don't like that you did this I like this one better and that that training does happen what does that training teach the system to do well it teaches whatever you told it it was supposed to be doing and that certainly governs style so if you talk to one of these models and you get the the stylistic like that's an excellent question and it gets to the heart of the matter like you're like how many times have I read that from from from a model well that's coming from how they were like told in their post training to answer.

47:27Right. And so if they're not just memorizing web text, they're also memorizing the post-training.

47:34Dan Klein:Yeah. Yeah. And so you can be taught to give those excellent question kind of things. That's certainly not coming off of, off of Reddit or whatever. And so there is more training. And as a result, when you, you say, okay, the system said there is no business that will give you a thousand dollars a month. well, where did that come from? It could be there's a web page out there that's like why businesses won't give you$1 ,000 a month. Like it could actually just be that it is regurgitating something that's out there. Like if you ask it how to do time travel and it's like you can't do time travel, there's a ton of web pages that say that.

48:03Or it could be coming from some explicit post training, which is like when people ask this stuff, tell them they can't do it. When they ask this stuff, tell them they can't do it. And this has been told to answer that way. So it is still regurgitating its data. And as these systems get stronger, they will increasingly have a system checking. Like, you know, these systems will increasingly become, if not fully metacognitive, they will start to have those sorts of behaviors. It's very difficult from the outside to tell if the system was aware that it didn't know the answer or whether it was aware that it's supposed to say it doesn't know in that specific context.

48:39And by the way, like, it's not that you can't build a system that looks up some answers. Like, RAG systems look up answers and then decide what to say. so they do have two layers. It's kind of what was designed, right? We were talking to one of the gentlemen who wrote like the attention is all you need papers. And I think his point was kind of exactly that, that the whole thing was made to just answer in a way that you expected it to answer. And that was almost kind of like the objective itself. I know we're running a little bit short time. I have one short question if you have time for it. Absolutely.

49:12We often asking people like how they use it personally. when you use models yourself are you a clock code gbt person or do you yourself now use different models in your personal work yeah i mean i'm going to give you a boring answer to this because it's true it's just kind of boring which is i do use a lot of things because i want to know what these systems do i want to know what their strengths are when their weaknesses are i would say the biggest if there's something about my experiences with large models that maybe differs from a lot of people I talk about, it's I'm often asking questions to which I already know the answer and like using that as a way to check, like, all right, well, what came back?

49:54How much of this was right? How much of this was wrong? And then when I ask a question to which I do not know the answer, I sort of assume that the accuracy rate is comparable. You're kind of, you're calibrating.

50:05Dan Klein:Yeah. I'm trying to, I'm always trying to calibrate and I am continually impressed by two things. One, the breadth and plasticity and flexibility of these models. It's still just like amazing. I mean, like I said, when I started this, it was like, will we ever be able to find the verb reliably in a sentence? And now I've got the system that I can ask it about quantum mechanics and get an answer. Now, I'm not an expert about quantum mechanics, but when I ask it about things where I am an expert, the answers come back like sort of right. But almost always there's like a critical flaw in the answer.

50:37and I know I can't find the equivalent flaws in other areas. And that has really impressed on me how important it is to recognize like all of us, no matter how much we know this intuition, all of us are susceptible to looking at systems outputs. They're fluent, they're confident. The parts we do understand look correct. We assume that everything else is correct. And that's not always true. I think that's a good lesson for everybody to come to take with them and maybe try to test their model or choice. on something they already know.

51:08Dan Klein:One thing I do like the already know. The other thing is like the task that you would, that where there's not a right answer necessarily, or I don't know how to classify this. So I'll give you the exact thing and then you can extrapolate. But one thing that I thought was pretty cool, I had an interview with Time Magazine recently where I was being interviewed, not being the interviewer. And I just, on a whim, I took the transcript and I just said, hey, Claude, I've got a chief of staff that's kind of trained on a lot of my, you know, blog posts, things like that. I just said, Hey, how'd I do? And it was so deeply insightful.

51:45Dan Klein:And then I said, how should I, and part of its critique was you treated this interview like a keynote, not like, you know, when you're being interviewed, your job is to be the sous chef, to help the master chef put the ingredients out. You made this journalist's job more difficult, not more easy. You rambled. You buried the lead. You followed what you're curious about, not what they're curious about. All this stuff. And I said, great. Now can you help me prepare for that? Because I had another media interview. And Klaus said, sure, give me the email that the person sent you. I gave it to you. Boom.

52:20Dan Klein:And I mean, we're talking topics to avoid. Things that you will want to talk about that this interviewer is not interested in. It was, I actually kept it on the screen during my next interview. And so to me, that's like a, it's this whole other class of capability, right? Which is, I don't have the, you know, the means to have like a, a comms expert on my team. Right. But now I have a comms expert or at least a reasonable approximation. And by the way, even if it's not great, it does give me more confidence than the alternative, which is zero. Right. Which is kind of fascinating. I don't know.

52:57Dan Klein:I'm just kind of riffing out loud, but I don't know. This is this is totally right. And I think this gets at what you asked earlier about where are these systems strong? And this is exactly where they are incredibly like mind blowingly strong is the ability to take all that information, cross reference it with like all this context that's out there in terms of how you should run media that's a scatter across the web and to be able to distill things down. That sort of contextuality and breadth is just stunning. And in this case, the fact that it could read it all, pay perfect attention to it all, then give you something that if it were wrong, you would know, right?

53:38And that you were asking it for its generative capabilities. You were asking it to take all of the stuff, mix it together and give you a synthesis. well like that is a place where these technologies are incredible it's just that you know that's not every use case but when you have that use case that's a that's that really lines up well you were in the loop you were being the editor you're like oh i love this i love this and maybe there's something in there you weren't gonna say but that's okay because this gave you a lot i could draw an analogy to early machine translation where like you could read some of it and then there was some stuff that wasn't really even in the language you were expecting and like you didn't get everything, but it was, as you said, better than nothing.

54:16And in situations where something is better than nothing, mistakes are going to get caught by the consumer and where the primary product you wanted was this novel synthesis. Well, that's great. I mean, we can call it a hallucination. It was just a really useful one for you. Yeah, that's cool. Okay, gentlemen, you're out of tokens.

54:35Dan Klein:You're out of tokens. Jeremy Oddly. I'd put this one more in the, hot takes category, probably, actually, Hendrick. What do you think? You know, one thing, the biggest takeaway I think I had from it is, I remember once I was reading a finance newspaper that we have here in the Scandi market, and they were writing about some startup stuff. And as I was reading, I was like, oh, they really don't know anything about startups. They really don't. Like everything, the way that everything was phrased was just very, it was very clear that they completely misunderstood everything about startup and financing and stuff like that.

55:11And then I was like, hey, wait a minute. If there's this wrong about this thing that I know a lot about, I wonder how wrong they are about other stuff too. Right. And I think what he was just making was the exact same point with models. And I hadn't really completely thought about it because I love using those models. I use them increasingly kind of like I trust them more and more just to give me the right answer. But he made this point of saying the population of Berkeley is and, you know, the model will basically have a guess, right? And then I'll put something out. It might be right, but it's probably wrong.

55:46And then, so he was suggesting that you should write something, you should ask something you know a lot about and then kind of understand its limitations. Just to calibrate.

55:54Dan Klein:Yeah. I mean, to me, I think the one caveat, can I put one caveat there? Because I really agree with that tactic and actually love it as a starting point. Say, have a conversation about something that you know deeply to explore the kind of jagged frontier, so to speak. The caveat is I would never recommend someone have a single shot interaction like quiz AI and then see, oh, is its response good? That is kind of, that is demonstrably poor AI collaboration behavior. I think what you want to do instead is like, is engage and respond and give mentorship and feedback and guidance and recognize, wow, the AI is fully capable of evolving its understanding based on what I input.

56:41But I think that was kind of, that's leading to a second point, which is something you always talk a lot about that you shouldn't kind of be working on AI or you with AI. You have like this phrase about just making sure that this is a undirectional kind of thing that you go back and forth with. I think what he was talking to is that people have to change how they work, how they think about what their kind of function is in a workflow. And it used to be that, you know, going back to journalism, you can be a writer, you can be an editor, and it's completely two different jobs. Now, a writer most likely don't know how it is to be an editor, and an editor might have forgotten how it is to be a writer.

57:19and increasingly that when ai is basically giving out this world of abundance you now have to teach yourself how to be an editor and i think you were kind of talking to an editor's function which is it told just told you how many people live in berkeley what you would normally go like hey i wonder if the writer has actually checked that right and so you go back and say hey could you please double check that number is legit at which point the model obviously will do a web search and then probably come back with the right number of corrects. But that is, I think, something that most people are not trained to do.

57:51And so I do think that the conversation had this interesting kind of like insight for me is that we're just kind of used to work in a world of scarcity. And now we're going to be living in a world of at least generative abundance. And we then have to train ourselves to be good at being stubborn about really want to have excellent outcomes because it's going to be easy just to take the first one. We have to be stubborn to be an editor of things that comes at us because we have to be the one who would basically the fact checker because the model will obviously just say whatever and all those different things.

58:27And so I thought that was pretty interesting too.

58:29Dan Klein:I think it's a worthwhile paradigm is his discussion around digital literacy and seeing AI fluency as an extension of digital literacy. So that's a powerful thing. And then furthermore, to realize that some of the ways in which we have learned digital literacy are actually a disservice to us. And comparing Googling versus working with AI, things like that. I thought it was fascinating to think about the fact that so few organizations are investing in skills. And I wrote something down here. Let me see if I can find it. Yeah. This idea of new management training is something that organizations provide folks who've been given teams.

59:18Dan Klein:Nobody's getting new manager training now that they've been given the AI team. I think that that's a really interesting paradigm. And knowing, for example, how to delegate is itself a skill. And if you are a poor collaborator to AI, or if you don't know how to delegate, then you're not going to be able to be successful in a collaboration with this intelligence, not because the intelligence isn't capable, but because you lack the managerial skills to derive the best possible work from this new teammate. I thought that was pretty interesting. A small thing that he brought up, which also I think is just interesting, is that we are trained to spot things that are untrue on the normal web, partly because I think we're visually trained.

1:00:07Like if you go into a website where there's endless amount of banner ads and pop-ups, I think your brain now just goes, this is probably not a very legit website. But we are not, we're used to that things that comes over in a chat window is kind of probably from a friend or it's condensed. And so things that are written out that is lengthy is something that is real thought through because that is what we've seen before. And so I think there's also like the visual kind of the interface design. There's something seductive in the chat that makes us think that is real.

1:00:44Dan Klein:It triggers our impression of thoughtfulness. or he said, I think the word he used was fluency, but the appearance of truth is something that's difficult for us to discern when we are in edit mode or if, especially if we don't know that we're in edit mode. I think it's pretty cool conversation. I think it's, it's a unique conversation in our canon in terms of really focusing on understanding hallucination. What is it? When is it a feature? As he said, sometimes hallucination is the product, right? It's actually the point. And when, when you're not looking for a re, you know, a regurgitation of something that already exists, but you're actually looking for something new de novo to be generated, uh, hallucination is a feature, not above, but then there are other times when your goal is retrieval effectively, that hallucination is a problem.

1:01:32Dan Klein:And thinking about when is what system best? You know, I, I love, I love his point that, you know, if a reasoning system effectively just solves a problem 10 times and selects the best one. The person paying for the 10 at-bats probably doesn't like that. The person being paid for 10x the token usage probably loves it. Now, I think it's slightly a cynical view because I think as far as I understand it, model companies are largely losing money on reasoning queries and things like that. So I don't think they're getting rich off that. But I think it's an interesting paradigm to realize. His example of learning French from a French book is very much better and more effective than learning French from incidental French phrases in an English book.

1:02:20Dan Klein:Yeah, no, that was good too. Awesome. I think that's conclude the conversation today. So Jeremy, you do the backing at the end of this episode. What we need folks, what we need are enthusiastic shares and reviews. You know, I was reminded of, Henrik, I'll just make a plug for a former episode, our conversation with Muhammad Ali, the head of consulting at IBM. I was thinking about some of the things they've done in terms of treating themselves as client zero, proving the economic impact, deploying teams to internal workflow optimization, redeploying freed up labor to revenue producing lines of business.

1:03:07Dan Klein:it's a really cool example and if folks haven't heard that interview they should go back and listen to that as well but with that as always only one thing to say and that is bye-bye bye-bye