How to Build Frontier-Lab Quality Evals with Daniel McKinnon, ex-PM at Meta, Google

0:00Daniel McKinnon: The Metas and the Googles and all the other large companies have to reinvent themselves right now in the age of AI. Every single PM is going to start building AI features. You cannot exist as a PM without understanding this.

0:12Rasty Turek: Meet Daniel McKinnon, former PM on the Llama Models at Meta, now a startup founder and a master of evals.

0:20Daniel McKinnon: PMs right now from a career perspective are in a really tough situation. The average PM is an orchestrator, a motivator and an analyst. But a lot of this is easy to do with AI. What really is the difference between product management at Meta versus Google? Meta is like a much, much more aggressive culture in many ways. Google is considered to be more of an engineering-led company, whereas Meta is more of a product-led company. If you're a PM who's never worked on an AI feature before,

0:45Rasty Turek: when would you be going through this evals process?

0:48Daniel McKinnon: You work at Pinterest and you want to have a better image generation that doesn't look like slop that actually pleases users.

0:53Rasty Turek: You need to think about from day zero, what does success look like? If it's the best way to measure it, we've got to learn it. Where should we start?

1:01Daniel McKinnon: Let me walk through.

1:28Rasty Turek: descript, magic patterns, linear, dovetail, arise, and mobbin. And now, into today's show.

1:38Rasty Turek: Welcome to the podcast.

1:39Daniel McKinnon: Yeah, thanks so much for having me, and it'll be fun to talk about this stuff. So I want to start with your article.

1:44Rasty Turek: You had this provocative claim and this funny meme here. Do evils replace the PRD? What is the role of evils?

1:52Daniel McKinnon: Yeah, so I think it was like a little bit spicy and saying it replaces the PRD because without a product strategy or a particular customer, like your product is nothing. But again, that's a paragraph or even a sentence, depending on what the product is. The majority of most PRDs that I've seen in my career have spent most of the document talking about specifics, how it will work, how it will behave in certain situations, how the user can expect to get value from that. but that gets really turned on its head in this kind of gen ai world where these products really need to do like everything or at least a lot more things than previous products and it's very hard to describe that say oh this thing just does everything and the best way to actually communicate what the product should do is through examples and that's what an eval is it's really just like a trivia question for the model and it's saying this is like the shape of the things the model needs to do well.

2:50Daniel McKinnon: And if the model does it well, it means it's getting these answers. And if it gets these answers, the users will probably like it. And if the users probably like it, let's ship it into prod and see if they actually like it with an online eval. But the key way to communicate how a product should work in this kind of Gen AI era is, is it performing well on an offline eval? And if not, you either need to change the model, change the harness, or change the product. It's possible that what you want to do is not possible with the models today, but it's better to find that out early with an offline eval versus just shipping it to prod and getting frustrated users.

3:22Rasty Turek: And for people who don't quite understand that nuance, what's an offline eval versus just shipping to prod and looking at them?

3:29Daniel McKinnon: Yeah, this is a really, really important nuance. And I touched on it in this blog post. But when we usually talk about evals in this AI world, it's something that's run offline. There's like a little bit of gray areas in terms of our environments and stuff. but think about it as like a pre-baked set of trivia questions that you ask the model. So for example, let's say you have a recipes website and you want to tell users how to make their favorite kinds of ice cream. An offline eval would be a prompt set, say 100 prompts, of different ice creams that users might like. And the answer key would be a correct answer or a plausibly correct answer with a way to score whether it's good.

4:10Daniel McKinnon: So that's run offline during the development of your product and you use that as a proxy for real user traffic. Once you do well on your offline evals, you can ship that product online and you have a website and it lets users generate ice cream recipes and you think because you're a good PM and you really thought deeply about what the customer wanted, that performance on that offline eval set will reflect the satisfaction of the user in the online eval. This obviously doesn't always happen but that's the idea, and it's the best way to measure whether a Gen AI product is likely to satisfy users.

4:48Rasty Turek: If it's the best way to measure it, we've got to learn it. Where should we start?

4:52Daniel McKinnon: Let me walk through what's changed since I wrote that article. So the key thesis has remained true, and this has been now two years, in that an eval is the best way to communicate what your product should be doing and to explain to the engineering team working on making the product work what success looks like. This can be very, very challenging in an AI world when these products do so many different things that it's hard to necessarily understand what good is. But things have gotten a lot more complex. So back in the day, evals were really simple. My blog post basically just cover these simple evals.

5:30Daniel McKinnon: They were question and answer. And this seems crazy to think back two years about what these models looked like and what the use cases were like. But this is before Claude Code and agentic coding and all of these crazy business applications that are getting built right now, Claude Cowork and the like. Really the core thesis two years ago was that Gen.ai was essentially a search replacement. I don't know if everyone remembers when Google's stock tanked because this was going to be the replacement for search. And fundamentally, these were like question and answer products. So what I have right now is the benchmarks that OpenAI reported on GPT-4.

6:12Daniel McKinnon: And you might remember a lot of these. MMLU, HellaSwag, Arc, Winograd, HumanEval. HumanEval is ironically an automated eval of python drop and these are all just question and answers so i dropped one example here from mmlu if you know the actual brightness of an object and its apparent brightness from your location then with no information you can estimate a speed relative to you b composition c size d distance from you and if you want to go ahead and look at some of these just for like historical fun uh they're all on hugging face so this was actually a very very straightforward thing to do is you had to think about the types of questions users would ask and the types of answers they would expect and how to score that.

6:56Daniel McKinnon: So the key thing was just matching the questions to the domain of interest and scoring the answers. So if we go back into my post here, we can see a little bit how I said to do this. And first thing is just figure out your problem and it doesn't need to be perfect. What is a set of problems that your users might ask about? So the example I used was, for example, generating recipes from videos. I guess I have food on my mind because I randomly came up with the ice cream example earlier. And new problem is I have a video on my social media site and I want to be able to generate a recipe that somebody can use to make that thing as they're walking the video.

7:40Daniel McKinnon: And you say, okay, this is really well defined. And now you have to measure How do you know if it's good? You know, it might be formatted right. It might have all the ingredients listed. It might be written in the right style. And then you select all of these components and figure out how to judge if it's correct or hypothesize how to judge this correct. This can be a auto score. This can be another LLM. This can be a human. You can judge correct this many different ways. Once you have that, it's really a mechanical process to actually write the eval. Just come up with probably 100 prompts that are in this distribution can be less, can be more, but this is a typical size of an eval in the Gen AI world.

8:18Daniel McKinnon: And just send it through the model and figure out what is a hard prompt, what is an easy prompt, and have something that scores maybe like 50 % because you have to have room to run. If you create a very easy eval that scores 100%, there's no way for your engineering team to optimize on that. And if you create a very hard eval that scores 0%, you also don't even know if this is kind of possible with today's technologies. And that's pretty much it. And once you have this, you can give it to the team. They can improve on it. You can ship it out to users. If you achieve a score high enough, you can see if your actually online performance matches what you expect it to do offline.

8:55Daniel McKinnon: And then this blog has a bunch of examples of like, you know, some kind of other nice tips. But why is this not that relevant today? Or why does it need to change today? The real answer is that models have generally saturated QA. This isn't 100 % true, but when we think of a good model right now, we don't think of one that can answer a relatively challenging high school physics question like the example I gave above. We think of models that are like winning gold medals at international math olympiads. Like there's almost no question that any human beyond some super, super specialist can ask the model to do and not have a good answer back.

9:40Daniel McKinnon: And also, QA is not the most useful application now. I mean, I think we all remember this narrative that ChatGPT was going to be the next great consumer app and they were going to get all these users and it's QA and you're answering all these problems. And Google did a generative search experience. But if you look at all the headlines in Gen AI right now, it's not QA, it's agents. And this is actually reflected with how the labs communicate progress.

10:05Rasty Turek: Here's a quick word from our sponsors. If you're building anything that uses live data from the web, eventually you hit the same wall. An agent, a research tool, a trends dashboard. They all need fresh data. Scraping that data is the worst part. Captchas, proxy, layouts that change every week. It's a whole side project you didn't sign up for. That's where SERP API comes in. SERP API gives you clean, structured results from Google, YouTube, Bing, Google News, Google Scholar, and more. One API call, one clean JSON response. They handle the captchas, proxies, and layout changes for you. Take the Google Scholar API as one example.

10:44Rasty Turek: Say you're building a research assistant or pulling sources for a literature review. You hit one endpoint. You get peer-reviewed articles back with full text or metadata. Titles, links, publications, citation info, all of it. Across publishers and formats. No scraping a dozen publisher sites and gluing the data together yourself. The same idea extends across the rest of their APIs. Real-time Google search for an agent. Pre-classified images for training data. Google News for monitoring. 99.9 % uptime. 1.2 second response time. Get started with 250 free credits. Link is in the description or scan the QR code on screen.

11:23Rasty Turek: Thanks to SERP API for sponsoring. Are you looking to up your AI product management chops? I highly recommend the AI product management certification by product faculty. It has a 4.7 with 1 ,249 reviews on Maven for a reason. I myself took the course back in 2024 and it was awesome. Since then, they have upgraded it. So now you get to learn from product leaders that open AI and Anthropic. On top of that, Pavel Hearn, author of the Product Compass, leads the Build Labs.

11:56Rasty Turek: So a very practical course. It's going to help you identify AI leverage opportunities. It's going to help you design trustworthy AI experiences. It's going to help you systematically optimize outputs for accuracy and relevance, build rigorous evaluation suites, architect AI agentic systems that work, and select the perfect LLM for your use case. It's normally$2 ,500, but you get a discount when you use my link. The next cohort starts June 22nd and goes to August 9th, so do check it out with my link in the description. They have been one of my longest sponsors for a reason. I trust this product and I think you should consider the cohort.

12:32Daniel McKinnon: So back here two years ago if you look at how OpenAI communicated progress it was these five evals. I guess six evals. If we fast forward to look at how Anthropic communicated progress for Opus 4.8 you can see they're using entirely different benchmarks, you don't see any continuity. It's partially because those evals are saturated and Opus 4.8 would score effectively 100 % on all of them, but it's partially because the task is really different. You'll notice we have agentic coding, agentic terminal coding, multidisciplinary reasoning. This is actually some agentic reasoning, agentic computer use, knowledge work.

13:15Daniel McKinnon: This is actually agentic knowledge work and agentic financial analysis. All this means is what the core model task is, is no longer to get a prompt from a user and come back with an answer, but it is actually to get a task from a user that requires many, many steps. Some of these steps might involve just thinking, which is called reasoning in this world. Some of it might involve tool calling, something like search. Some of it might involve more advanced tool calling, like something we'll go over today. And this is a totally new paradigm of writing evals because you're no longer thinking about QA, you're thinking about tasks.

13:53Daniel McKinnon: Fortunately for us, the framework is largely the same. We still have to define the problem. We still have to be good PMs and know what we're solving. We still have to collect representative prompts. I call this Goldilocks style. Again, they can't be too hard and it can't be too easy. There has to be some room to run. A typical good eval will have something like 25%, 50 % success rate. And then over, you know, months that will go to a hundred percent. And then you'll have to throw it away and create a new one that is harder. And then you also have to figure out how to score. And one thing that's changed is QA is relatively easy to score with humans worst case scenario.

14:33Daniel McKinnon: There's exceptions to this. Of course, the reason these models are so bad at things like creative writing is because it's hard to score and there's different preferences. and different users like different things and why they're so good at math and coding is there is like some right answer and this is much easier to hill climb. But for to some extent QA style questions, you can ask human raters to review worst case scenario if you can't find a better way to score it. With agentic work, it's much more challenging because the time horizon tends to be very long and the final output is a collection of many, many, many steps that it took.

15:08Daniel McKinnon: Some steps could be correct and lead to the wrong outcome. Some steps could not be correct. And it's just from a labor perspective and a defining success perspective, it's much, much more important to get something that can be automatically scored to make more of these rollouts and understand how you can do more experiments. But again, it's largely the same and the tasks are much longer time horizon. So kind of the goal of this podcast is to walk the audience through creating an actual agentic eval in real time. I want to caveat this. This is a little bit pre-baked. It is unrealistic in 45 minutes to come up with a brand new eval.

15:48Daniel McKinnon: This is kind of like a week's or month's problem of deep thinking, but we're going to kind of pretend and we'll go through some of the steps together. So first problem. I want to measure and improve the model's ability to help with clinical genomics. This is a problem that I care deeply about. It's one that I think can improve the world, and it's something that I've launched a new startup to solve. And the problem is, is that whole genome sequencing has become the absolute gold standard in diagnostics in NICU settings, so for sick babies. unfortunately interpreting the results of a whole genome sequence is very labor intensive and it limits access to this life-saving technology.

16:37Daniel McKinnon: So I wanted to see if I could distill some of this human expertise into a model to help broaden the accessibility of this technology. So first thing is like before we have to deeply understand the problem and I'm not going over these flow sheets, but this is just kind of how complex this is. Generally, you start with the raw reads off the sequencer. You do a lot of processing work to identify how the particular patient defers from the reference human genome, and then you do another set of work to determine whether those changes to the genome matter. For example, if my genome were sequenced, I would get about a billion reads that are 150 base pairs long, they would come out in a giant text file.

17:30Daniel McKinnon: And I would need to transform that into a diagnosis that says that this gene may or may not be responsible for this patient's condition. And there's a very structured way of doing this. So I'm not going to spend too much time on this here because we're going to go over some real examples, but you really need to deeply understand the problem. This is why people like Anthropic and OpenAI are hiring investment bankers, accountants, lawyers, you see job ads for all these vertical specific teams, you must deeply understand the problem. You will unlikely be successful in creating an eval for some topic if you don't have some background in it or haven't really educated yourself on it.

18:09Daniel McKinnon: So then let's say we've understood the problem. I think I understand this problem pretty well. And by the end of this podcast, you will too. Let's go to the prompts. So again, we want to find this like Goldilocks set of prompts. So the first thing I like to do is just start with something easy. So you want to make sure the model can actually do this. And when I say the model for agentic stuff, I'm usually talking about the model plus the harness. So I'll use those words interchangeably, but this is a frontier model harnessed in a way that it can use these tools and it can do this reasoning and it can come back with a solution.

18:45Daniel McKinnon: So for this problem of genome interpretation, I picked like one of the easiest genetic diseases possible. And this is cystic fibrosis. This was something that we have known the genetic cause for quite some time. And there are like canonical genes that cause cystic fibrosis. So to save you the effort of me Googling for this, I just had the link right here. And let's just go ahead and look at this. So what we see here is the canonical cystic fibrosis mutation in ClinVar, which is an NIH database for a lot of genetic disease. And what we see here is it's got four stars and three stars. This really should be four and four.

19:30Daniel McKinnon: This is like the canonical genetic defect for cystic fibrosis. So I'm going to make sure that my agent can actually get this before I go forward. And so this is like the easy thing to start. So we're going to have our agentic genetics eval, and we're going to say gene, and we're going to say CFTR2. This is the, again, canonical gene. And then we'll say variant. And what a variant is, is how a particular gene is mutated. So right here, this is, again, this is a somewhat niche eval. But what this is saying is that on this particular transcript of CFTR, at this position, there is one base deleted.

20:18Daniel McKinnon: And what that results in is the 508th phenylalanine deleted. So this is the actual variant that's going to exist in our eval. Sounds good. Okay, so what we see is that this is the exact variant that we care about. And what you'll notice is this is like very complex and nuanced. And this kind of comes back to the absolute first point I was making is you really need to know the space to do these evals. A lot of the evals are kind of like picked up. Like if you're trying to come up with an eval for Python coding, like many of these are pretty good now. And for any of you who have used these models, saying Python coding is a solved problem is a little bit of a strong statement, but it is a very, very well understood and well characterized problem.

21:05Daniel McKinnon: So we will go to this and we're going to say, okay, so this is the thing we want to know. And I should add a column here. And this is the phenotype is cystic fibrosis. So what you see here is I'm starting to build a table of question and answer. So the question is I have this phenotype of cystic fibrosis, which is a lung disease. And, you know, we'll describe exactly what happens there. And then we have the genome of this patient. and then we have the answer, which is this variant. So let's walk through like how we would do that. And when you're constructing these evals, you're gonna really, really, really use Gen AI a lot to construct them.

21:45Daniel McKinnon: So what I'm gonna do is now I'm gonna go over to a terminal window I have open here and I'm using Codex. Any of the tools will work. I have it on 5.3 Spark low because I want it to be fast for this demonstration. but you can use any model, anything you like. And for this task, this will be fine. So what I have here, and I pre-baked some of these just to kind of make it go faster, but I wanna walk through any step anyway, is I have actually two files here that are representing my genome. And what I want to do is create a synthetic version of this genome that has these variants that have the question and answer through this agentic flow that I want to get at.

22:35Daniel McKinnon: So what I'm going to do is I'm going to say please add and then this is going to be this variant to and this is a small variant so it's going to come through here and create a new file in a new folder and we're going to call this dancflive.vcf.gz and we're going to have this be dancflive and we're actually asking ai to help us create the eval for ai so we're going to do this and what the model is going to do is a variant file is literally just a text file. Actually, we can see what it looks like here just for fun. So while this is running, let's just take a look just so we know what these variant files look like.

23:32Daniel McKinnon: And this is fine. We don't need to show all of it. But what we basically see here is a chromosome. So you might remember from things like 23andMe, we have 23 chromosomes. So this is one. This is the biggest one. And this is a position. So your chromosomes have different number of bases. You might remember we have like 3 billion bases in our genome. And then these are swaps. So what we see is in this position, a reference human has C and I have a CA here. So that means that, you know, during some thing I inherited from my parents or maybe something that emerged during my development, I got a base swapped here.

24:12Daniel McKinnon: And then there's a bunch of metrics around quality, how real it is. You might remember you have two copies of each gene, so this is actually heterozygous, meaning only one copy is impacted. And, you know, really, it's just a text file of all the letters in your alphabet. And what I'm saying is I want to add, okay, this is still running, and if this is still running in a while, I'll just use the pre-baked one, is I just want to add this particular variant into my genome to see if our system can catch it. And this is what Codex is doing right now. And I actually don't know why this is taking so long, because this is like a one-liner.

24:54Rasty Turek: But line the line, I think.

24:56Daniel McKinnon: Yeah, right, right. I guess I probably I didn't want to make this demo too pre-baked. I thought about it. Should I just have a Python program that just does all this for you. But then I'm like, that would not help the users at all because when they're constructing their own, they would know how to do it. So just believe me that this will work. And for the sake of time, we'll go to the pre-baked ones. So basically what's going to happen is that Codex is going to add, this is in chromosome, where is it? I'm actually not sure. But in whatever chromosome this cystic fibrosis gene is in, it's just going to add one row.

25:31Daniel McKinnon: And it's going to say, we're going to have a deletion. So instead of having like CA here, it'll just have a C. So like here's a deletion. You'll notice that we've lost an A. We went from TA to T. And then it's going to have a fake cystic fibrosis patient. So now let's just check to see how this works. And we'll say use our agent. And again, these are all agentic evals. And I'm using codex as our agent. But you could use anything. incline, open code, clod code, your custom harness, any way you could to get these agents to actually operate. And in fact, even chat GPT and clod actually in the web UI, they use agents right now.

26:13Daniel McKinnon: They're not just model in, model out. They're model in, reasoning, tools, everything. Okay, cool. All right. So the agent finished here. So we see we added this in a record. Okay. And we see here it is. It's in chromosome seven. And you'll notice this is deletion. It says TCTT. And instead it's a T. And so this means that this is like the canonical CF. So we're going to see if our agent is able to do it. And we're actually going to try a few different agents because one thing I mentioned is you want to score like, you know, 25 to 50 % on these evals. You have to think about what tool are you using the, you know, mythos five for some biodefense thing, then it's got to be really really hard or maybe this is something that for infrastructure reasons or cost reasons you need to use a very small model you need to use haiku or something like that so we're actually going to try these simultaneously on a few agents and see what happens so we're going to say inside what was this file we had uh dancflive again this is the one that we just made

27:21Daniel McKinnon: we have the genome of a patient suffering from and let's just very quickly copy and paste some cystic fibrosis symptoms uh this one we suspect cystic fibrosis please find a genetic cause and then we are going to we're actually going to copy this prompt so we can use it across multiple agents so now we're going and we're we're checking gpt 3.5 codec sparklow we have a few other tabs open so i mentioned you know maybe we want to actually see if haiku can do this. So we'll just upload this. And again, we're going to the CF live. And let's also try cat GPT 5.5 extra high. And I'm not going to use pro because it will take too long.

28:21Daniel McKinnon: And this is also a relatively easy task. I suspect all the agents will get them. Okay, great. So we are cooking with haiku and we are cooking with you can see what's all already happened with our first agent is after one minute of thinking you've actually find the cftr mutation pattern consistent with this deletion right this is the canonical cystic fibrosis gene so what this is telling us going back to our steps is this task is not too hard for these agents at least in the easy case especially not with a powerful system like Codex. We'll see if Haiku gets it. I suspect Haiku will also get it but you can see Haiku even itself knows the CFTR region is important.

29:07Daniel McKinnon: So while that's cooking let's go back to our next step. So again we start with something easy just to make sure it's possible. I knew that this was possible but if you just told a layman and say hey could Could an AI agent find the canonical cause of cystic fibrosis inside a file with billions of variants? They might say yes, they might say no, right? You just need to know, you need to kind of try it to get a sense of if it's possible.

29:35Rasty Turek: Quick thought experiment for you. Is there anything in this video you should be trying on your own? If there is, try it, take a screenshot, post it on LinkedIn or X, and tag me. I'd love to see what you're learning. Now a quick word from our sponsors before we get into the back half of the pod. If you've worked at any company bigger than 30 people, you know this one. The CEO sets strategy. By the time it reaches the people actually doing the work, it goes through three or four layers of translation. Half of it gets lost and nobody finds out until the quarter is over. That's the problem Ariso is built for.

30:05Rasty Turek: It's an AI operating partner for every manager and team. It connects to where work actually happens. The meetings, the messages, the docs, and it turns all that fragmented activity into a clear picture of execution. Managers get real coaching grounded in their team's actual work, not generic advice. Teams stay aligned with strategy as it changes, not as it was last quarter. And leaders see where execution is drifting in weeks, not in the postmortem. One shared memory for the whole org. Everyone finally working from the same picture. If you lead a team, check out ariso.ai. That's A-R-I-S-O dot A-I slash A-A-K-A-S-H.

30:41Rasty Turek: I want to take a second to talk to you about the fourth cohort of LAN PM job. I trained 30 students in Cohort 1, 50 students in Cohort 2, and 75 students in Cohort 3. And I am bringing back the program for Cohort 4. It starts in August and it lasts three months where you're going to have intense sessions. A Monday morning session where I go over your resume, behavioral interviews, LinkedIn. On top of that, Bart Jaworski is going to be teaching you the PM fundamentals in 2026. How to write AI PRDs, how to AI prototype with cloud code. All of the key skills you need to freshen up your knowledge for this market.

31:15Rasty Turek: And Ankit Vermani is going to be teaching you AI product management. He is an AI product manager at Uber, and he is going to teach you how to build AI features that actually work successfully. On top of that, Prasad Reddy is going to be doing one-on-ones with you for market reviews, LinkedIn review, candidate market hit review. So it is a full package. It is three courses in one for one low fee. So join at landpmjob.com. Today's podcast is brought to you by Pendo, the leading software experience management platform. McKinsey found that 78 % of companies are using Gen AI, but just as many have reported no bottom line improvements.

31:51Rasty Turek: So how do you know if your AI agents are actually working? Are they giving users the wrong answers, creating more work instead of less, improving retention, or hurting it? When your software data and AI data are disconnected, you can't answer these questions. But when you bring all your usage data together in one place, you can see what users do before, during, and after they use AI. Showing you when agents work, how they help you grow, and when to prioritize on your roadmap. Pendo Agent Analytics is the only solution built to do this for product teams. Start measuring your AI's performance with Agent Analytics at pendo.io.

32:23Rasty Turek: That's pendo.io. A-A-K-A-S-H.

32:29Daniel McKinnon: but then if you know that the easy thing works and you know we already have early evidence the easy things works you have to like establish the ceiling it's like what is is the hard thing working because like if it's just totally saturated then like what's the point of even having eval this task is already solved um so i'm i'm going to show off something that's pretty hard to do today. And what we have here is a recent paper. So this is from last year. And I will make this bigger. The authors here are deciphering the digenic architecture of congenital heart disease. So what does this mean? This means congenital heart disease is if you're a baby and you're born with problems with your heart and digenic means it involves two genes so single gene single variant genetic diseases are actually sometimes a solved problem like with cystic fibrosis not all cases but many cases like this one are totally understand digenic genetic diseases are like a very very new thing that people are studying so this is like a very hard task to do and even though of this is published, and in theory, an agent should be able to search the internet and find every publication and deeply understand all of this, it's actually not that simple.

33:53Daniel McKinnon: They're not perfect, and they actually need a lot of guidance, which is why there are a lot of these companies, including my own, that are called like harness engineering companies or vertical AI companies or agentic AI companies, because you need some specialized capability to be able to have the LLMs do stuff like this. So let's briefly return to our agents and let's just make sure they got it. Okay, so codex with GPT 5.3 says, okay, this, if you recall, this is the deletion that we added. Boom. I gave it the phenotype. I gave it the genome. This is correct. So how would we mark this correct?

34:29Daniel McKinnon: We would actually probably have another LLM. I'm not going to do this right now for the sake of time, just compare my scorecard. This is the correct answer with the response the model is giving right here. And then we can check haiku and even haiku. Oh wait, did haiku not get this? Okay, so this is interesting. This is actually harder than I would have thought. I would have expected haiku to get this because this problem is so easy. But you'll notice what haiku says is there's 48 variants spanning the gene. So it's looking at the gene, but it fails to actually find the particular, oh, this is so interesting.

35:11Daniel McKinnon: It also hallucinates a hemizygous large deletion. So coming back to this, coming back to our point is start with something easy. I thought I started with something easy here. It's a good thing I did this because if I were benchmarking IQ, this is too hard and I'd have to make it even easier. And the things I could do to make it even easier would be potentially, you know, limit the region of the genome of interest, give it more hints, maybe provide access to more external information more easily, but we can see that Haiku even fails this easy task, and I would be absolutely shocked if 5.5 did. Okay, it's not finished yet, but you can already see that it found the correct answer.

35:50Daniel McKinnon: So if we were to score this, we could easily have an LLM say, okay, Haiku, this is not correct, this does not match what I have in this table, this is correct and this is correct so that's kind of and then in our in our spreadsheet we would just say you know haiku bad others good um and but now let's move on to something where we want to understand the hard cases and again i unexpectedly actually picked out a hard case for haiku but this paper is quite challenging and I believe it is unlikely that any of the models will solve this. So in the supplementary information of this paper is a table and it is a list of patients and a proband is a medical term for the patient you're evaluating and it has diagenic causes for congenital heart disease.

36:47Daniel McKinnon: So this particular patient has ACACB, I have no idea what this is, some gene. It's HET, meaning it only has one copy of this variant, and myoCD, which is also HET, which is one copy. And these researchers discovered that the combination of these two diseases leads to congenital heart disease. So let's see if the models can figure this out. So what we'll do

37:12Rasty Turek: again is we'll go to our table and our phenotype diseases or is it two abnormalities in their DNA?

37:18Daniel McKinnon: Yeah, that's a great question. It's one disease. It's a congenital heart disease. And I don't know exactly which one it is from this paper. And, you know, we could read the paper and figure out exactly what the phenotype is, but it's some defect with the heart. And what's unusual about this and why this is hard is it's two heterozygous variants on two different genes that is causing this single disease. So it's complicated. So our phenotype here is congenital heart disease. And our gene here, we have two of them. One is this guy. And oops. And the second one is this guy. And then the variants are these guys.

38:00Daniel McKinnon: And this is, you'll notice the notation is a little bit different, but this is something that you'll just have to deal with in these evals is like, you know no matter what you're doing because these tend to be in very technical specialized domains at this point you know no one wants evals for for boring stuff like ice cream flavors you just have to get comfortable with all these different mutations and now let's go and let's try this again so what i would do is i would say something like please add these to dan.dvariant.vcf. But I'm actually not going to do this because you already saw how this worked and basically how the vcf file was structured.

38:44Daniel McKinnon: It would add these two rows. And for the sake of time, I've already done it. But then let's go and let's check and see, oh, how do we do on this use case? So in this case, I've already pre-baked it, and I'm going to say this is DanCHD, and I'll say this contains the genome of a patient with congenital heart disease. Please identify the genetic cause. Okay. and while we're going to have this one running we're going to try our other two agents just to see how they do um we can almost guarantee that haiku will not get this because it didn't get the much much easier task but for the sake of completion completeness where we'll do this as well.

39:44Daniel McKinnon: And then we'll do this with GFT 5.5 extra high as well.

39:51Rasty Turek: And again, we would do probably like some non-deterministic nature, right? Like, do you need to like test the same model a couple of times to just see if like maybe two out of three times it gets it right? Or is that not important?

40:02Daniel McKinnon: Yeah, that's a really good point. so this is a question about sampling so sampling is actually really important and you might remember like all of this old research where you would basically sample for good traces and this is kind of what like RL environments do is you do a rollout you do a rollout you do a rollout and then you get the correct answer and boom you give it a good reward for that and the key thesis here is inside the weights of the model, the right answer might live there. It just might not get the right answer each time. So when you're doing these evals, you do want to try multiple times.

40:43Daniel McKinnon: In this particular case, I actually know from having done it that sampling has very little effect and it's essentially deterministic based on model capabilities. I've seen slightly the same model get to the same conclusion with slightly different approaches. But in general, sampling, in my experience, is less important than it used to be, where sampling used to be a big deal. Like if you look at, let's just look at, this is a funny story. Let's look at Gemini Ultra scorecard. So if you'll remember, Gemini Ultra, oh wow, did Google actually bury it? okay here it is this is so you'll remember way back in 2023 when people thought google was kind of out of the ai race i actually worked on this model so i know the story very well google released gemini ultra which was i believe it was a 660b dense model which was crazy back then that was like one of the largest dense models ever trained and they released this scorecard and what you see is this.

41:50Daniel McKinnon: This was very controversial. And this comes back to your question about sampling. If you remember, MMLU, this used to be like the canonical benchmark for LLMs. And let's just go back to look at what a question is to remind you, is just simple question answer. If you know the actual brightness of an object, it's apparent brightness from location with no other information you can estimate this. Okay. Models used to be bad at this, which is hilarious because this seems so distant right now. And Google wanted to be the best. And GPT-4 was the best at this point. It got 86.4 % on MLMU. And it was on five shots, meaning it had five samples and it picked the best one.

42:29Daniel McKinnon: And that's why you see five shot, three shot, three shot, 10 shot. It really is like kind of like a way of cheating. It's like how many times can you sample from this model? And what you see is that Gemini Ultra actually had 32 shots. So they got more shots on goal. And actually, now that I'm remembering this, this actually might be pre-examples, but if this is actually not the number of examples in the context window and just the shots or the number of times sampled, it doesn't really matter for the sake of this argument. But the answers to MMLU might be inside the model weights, but it might just be not enriched enough in terms of the probabilities.

43:08Daniel McKinnon: So by sampling more times, you actually get a higher chance of getting the correct answer. So this used to be a really big thing back in the day. Today, I don't think this is a big thing. The labs don't really publish anymore. I don't think there's a lot known about this. In my personal experience, I've not found that running the same prompt through the model multiple times generates different answers. In fact, I don't know if I've ever seen that for this task, but it's a really, really good thing to do and you should test for your use case. Little side conversation while we look at the answers.

43:38Daniel McKinnon: And we've got all of these cooking. These are all cooking. and we can see that 5.3 Spark finished first. This is actually why we did it. And what you'll notice is it's totally wrong. They found multiple variants, TBX1, MyHCIS, JAG1. You'll notice these aren't even genes of interest for us. These variants aren't even relevant. No high confidence variants. So what we've done here is we've done the second step in our process is we've established the floor. we found something easy, that cystic fibrosis gene. Now we found the ceiling is this model didn't get it. And plot twist, no model on the planet gets this without like a very, very strong harness.

44:20Daniel McKinnon: Again, I'm working on that very strong harness. So, you know, we can make systems get this, but this is hard. And then you just kind of go through and it's almost like a binary search process where you say easy, medium, hard, and then just assemble a list of prompts. You know, you might have a hundred of these, which again would be phenotype gene, phenotype gene, phenotype gene. And then you understand how the models do on it. And then you're done. And then you have your eval. And then you understand what is good enough to actually ship product. So if you're scoring 50%, is that good enough to ship your product?

44:56Daniel McKinnon: Probably not. So then you look at which phenotypes am I better at? Maybe you put guardrails on the product to make sure that only will answer the types of questions that it can get 80 % on or something like that. That's a product decision. That's a product manager's decision to do that. And then you also hand all the hard ones to the research team and you say, hey, guys, you didn't get this. Fix the model or fix the hardness and make sure that it can get these in the future so I can ship a product with these capabilities. And not shockingly, Haiku is totally off base. Clearly, Haiku is not good at all at this task.

45:33Daniel McKinnon: So that's surprising. It's the first one, not surprising. It's this one. And we, again, we have, oh my God. Okay. This is so interesting. Okay. So this is actually a good example of something that came out and sampled a second time and worked because I actually tried this. We were just talking about sampling, but we can see right now that GPT 5.5 extra high this time actually did identify this diagenic pairs and it did almost certainly find the paper. Yeah. So it did find the paper with all these diagenic pairs. So this is actually a very interesting reasoning trace where it was able to turn this congenital heart defect phenotype into a search for a very specific paper and then pull out the results from this paper.

46:23Daniel McKinnon: So actually, this is quite impressive from GPT 5.5, but this is correct. So in this case, I would have to find an even harder one if I were benchmarking this model in particular. But this is basically the key set of steps. And I don't think we have time to do a bunch more, but it's basically running through all the different types of scenarios and then coming up with prompts that will challenge the model, but not totally stump the model. So yeah, with that, that's how you write an agentic eval. And here's two lines in our new one. Wow. So it's just a spreadsheet.

47:02Rasty Turek: And the key thing here is the domain subject matter expertise. It's not like how it's written or anything like that. You're not giving us an eval template like we might have given people a PRD template before. It's really the subject matter expertise that's driving all of this.

47:20Daniel McKinnon: Yeah, exactly. There are a lot of companies over the last, you know, N years who have tried to build better tools for evals. And I'm not saying that tools for evals don't need to exist. There's plenty of ways to improve. But when you're creating evals like this, it is literally just prompts, responses, and ways of scoring whether response is correct.

47:43Rasty Turek: Fascinating. So just to bring it all back full circle, if you're a PM who's never worked on an AI feature before, when would you be going through this evals process and when wouldn't you and how would you be using it?

47:53Daniel McKinnon: Yeah, that's a good question. So if you've never worked on an AI feature before, I would actually try to find somebody who has, who can help you through this. This is deceptively simple. I made this really simple because we had 45 minutes today, but this, and I don't know why it's so complex. Honestly, I had many, many conversations with people about how to build evals, but there's just something kind of like taste-based or nuanced about how to build them. And you know, it is what it is, but find somebody who can help you. But I would say you start from the beginning. It's like you are building an AI feature.

48:28Daniel McKinnon: Like, I don't know, I'm just making it up. You work at Pinterest and you want to have a better image generation that doesn't look like slop that actually pleases users. So it's like an image generation feature. You need to think about from day zero, what does success look like? What is unique about those Pinterest users? What do they want to see? And you need to translate that. You can't just write down a PRD and say they want beautiful kitchens. You have to explicitly define what a beautiful kitchen is and not in words, in examples, and a way to score those examples. And I am not in the image generation space, so I don't exactly know what that looks like.

49:07Daniel McKinnon: But there are many, many, many examples like this where you need to understand what the user wants and translate that into prompts and responses and a way to score those responses. And that is the first thing you should do when you're starting to build a new AI feature.

49:21Rasty Turek: Okay. So just like you have ramped up your expertise in the genomic space, if you were tackling that problem, you'd go learn, you'd go talk to people who have built image and evals. You'd understand, oh, this is how I build an LLM judge that generates images of this type. And then that would really be the basis for your eval. Yep, that's correct. Okay. Wow. There's so much, so many layers. I've done like five or six episodes on evals, but I think this was one of the most tactical that really helped me understand how things change. And I think that's a function of your experience, which I wanted to talk about for a little bit.

49:55Rasty Turek: So your evals piece crossed my radar. I think another really interesting piece you wrote about was product management at Meta versus Google. You've worked on Gemini, you've worked on Llama, you've seen both of these cultures, what really is the difference between product management at Meta versus Google?

50:16Daniel McKinnon: Yeah, so I would caveat that and say I wrote this like two and a half years ago and I was at Google three and a half years ago, I think. So a lot has changed. When I was at Google, Google was a dead company. I think the stock fell to like$80 and I think it's, you know, $300 or$400 right now. And they've really changed how they think about things. I am a boomerang at Meta. So I think I spent seven years there in total. And, you know, I saw everything from Cambridge Analytica lows to highs of like Lama 3 really wowing people to lows of Lama 4 disappointing. So I saw a very large spectrum. And what I would say, like my key takeaways for what Google versus Meta was like is Meta is like a much, much more aggressive culture in many ways.

51:03Daniel McKinnon: I think that it comes from like the founder leadership of Mark Zuckerberg is he is the last man standing. Well, I guess besides Elon, but he is the last man standing who's got like, you know, the founder leading a fan company who has utter and absolute control, who's going to do what he wants. And sometimes it's really empowering because he says, this is super important to me. You have all the resources in the world and you should go do it. And sometimes it's like not what you want. For example, I worked on Lama 4. Lama 4 had problems. I think the main problem was actually how it was evaluated.

51:36Daniel McKinnon: Huh, funny. Those evals are important. If you want to read about that story, Google it. I had nothing to do with that. And I loved working on Lama 4. And Mark just said, you guys all suck. You need to go find new jobs. So basically the whole Lama 4 team is gone because of, you know, Mark's, his decisions. So I think that, you know, that cuts both ways. I'd say my overall preference is for a very high conviction, founder-led company. I actually have incredible respect for Mark. I've only met him a couple of times and every time has been just like, wow, this is a really smart guy. But it creates a lot of problems too, right?

52:12Daniel McKinnon: Because Google is much more consensus driven. I think Google has much weaker product management function. At least it did when I was there. So Google is considered to be more of an engineering-led company, whereas Meta is more of a product-led company. At least that historically has been the case. And it was a really great experience working at both places. I think I've learned a lot from both places. I left both places with a lot of friends. And yeah, if you want to read like kind of this blog post actually pretty viral. If you want to read like kind of an interesting snapshot of what it was like in say 2023 between both of the places, give it a read.

52:48Rasty Turek: I highly recommend it to everybody. As you guys can see, I'm itching to ask many more questions. So Daniel, we're going to need to have you back. Before you go, tell us a little bit about your startup.

52:57Daniel McKinnon: Oh, cool. Yeah. So I've started a company called Gamoff Labs. And as I hinted at during these evaluations, the core problem I want to solve is to make it much, much easier to get whole genome sequencing into every single NICU in the entire world. This is the absolute gold standard of helping sick babies. There's overwhelming clinical and economic evidence that it's effective. But the problem is it's just too damn hard and expensive. So where you see this used is in places like Stanford, Boston Children's, CHOP. And these are the absolute top facilities in the world. And I want to see them in, you know, rural Arkansas, rural India, you know, rural China, all the places where this kind of life-saving technology is not being harnessed.

53:47Daniel McKinnon: And my key thesis is that a lot of the human work involved in interpreting these genomes could be augmented by AI. And we've already shown that using the system that we've built, we can identify variants that have never been discovered before. We've actually allowed one family to have a child, and they couldn't before because they didn't know. Again, this is small scale. We started this five weeks ago. So, you know, but one person is really crazy to have that impact on their life. And yeah, like it's a deeply, deeply mission driven thing. I think it's very, very interesting technically because it's all about building the best agentic harnesses.

54:23Daniel McKinnon: It's all about understanding how AI can help with biology. And if you're interested in joining me on this journey, we are hiring right now. We're a very small team, raised our pre-seed round and are basically planning on building like the operating system for rare disease and genomic medicine. And I couldn't be more excited to wake up to work on this every morning. And I would love it if you would reach out if you're interested.

54:47Rasty Turek: Wow. So a lot of you guys, I know, at least in my audience, you want to become that AIPM at Meta or Google. This is often the next step after that. So, you know, we always say the grass is greener at some point. This is where I've seen those AIPMs at Meta and Google Go, just like Daniel, into starting their own companies. And that's actually the cool thing is it helps prepare you for that. You can see how his own evals and deep AI knowledge has now applied to his startup. Daniel, thank you so, so much for lending your expertise today.

55:19Daniel McKinnon: Yeah, I want to leave with just one parting thought is the Metas and the Googles and all the other large companies have to reinvent themselves right now in the age of AI. If you're inside these companies, it is very, very interesting to see how this classic consumer software building factory has changed. But if you come and you do a startup or you start your own thing, you get to build the future from scratch. And sometimes that's actually easier.

55:46Rasty Turek: We'll leave it there. See you all in the next episode. I hope you learned as much from today's episode as I did. If you can do one thing that's totally free that would help the show, it would be to check that you're following on Apple and Spotify podcasts. Check that you've left ratings and reviews on those platforms. Check that you're subscribed on YouTube. Leave a like and a comment on this video and then share it with your friends. We're trying to make better and better podcasts. After two years, we think we've gotten something pretty good going. So let us know what we can do to make it even better, who else we should interview, and we will put on the best shows we possibly can.

56:22Rasty Turek: Finally, don't forget my offer for the bundle. You get an entire year of my paid newsletter. Plus my favorite AI tools, Boltut New, Airtable, Speechify, Descript, Magic Patterns, Linear, Dovetail, Arise, and Mobbin. That's$27 ,000 worth of value for just$150. So check that out at bundle.akashi.com if it interests you. And I can't wait to share our next episode soon.

56:55Thank you.