From Job Displacement to AI Trainers, Brendan Foody on Work in the AI Age

0:05Hi, listeners, and welcome to KnowPriors. Today, we're chatting with Brendan Fudi, co-founder and CEO of Merkur, the company that recruits people to train AI models. Merkur was founded in 2023 by three college dropouts and teal fellows. Since then, they've raised$100 million, surpassed$100 million in revenue run rate, and are working with the top AI labs. Today, we're talking about where the data for foundation model training will come from next, evaluations for state-of-the-art models, and the future of labor markets. Brendan, welcome to KnowPriors. Brendan, thanks so much for doing this. Yeah, thanks for having me.

0:37Excited to be here. So you guys have had a wild last six months or so. There's huge traction in the company. Can you just talk a little bit about what Mercore does? Yeah. So at a high level, we train models that predict how well someone will perform on a job better than a human can. So similar to how a human would review a resume, conduct an interview, and decide who to hire, we automate all of those processes with LLMs. And it's so effective. It's used by all of the top AI labs to hire thousands of people that train the next generation of models. What are the skills and job descriptions that the labs are looking for right now?

1:13It's really everything that's economically valuable because reinforcement learning is becoming so effective that once you create evals, the models can learn them and how to improve capabilities. And so for everything that we want alums to be good at, we need evals for those things. And it ranges from consulting to software engineers all the way to hobbyists and video games and everything that you can imagine under the sun. And it's really whatever capabilities you're seeing the foundation model companies invest in or even application layer companies invest in, the evals are upstream of all of that.

1:50And are you also helping companies outside of the core foundation models with a similar type of hiring or is it mainly just focused on AI models right now? Yeah. So actually, when we started the business, it was totally unrelated to human data. It was just that we saw that there were phenomenally talented people all around the world that weren't getting opportunities and we could apply LMs to make that process of finding them jobs more efficient. And then we realized after meeting a couple of customers in the market that there was just this huge vacuum because of the transition in the human data market and that the human data market used to be this crowdsourcing problem of how do you get a bunch of low and medium skilled people that are writing barely grammatically correct sentences for the early versions of ChatGPT.

2:35And it was transitioning towards this vetting problem of how do you find some of the most capable people in the world that can work directly with researchers to push the frontier of model capabilities. But we've still kept that core DNA of hiring people for roles, human data and otherwise. And a lot of our customers hire for both. Do you think all of hiring eventually moves to these AI systems assessing people or at least all sort of knowledge work? I think certainly because we're already seeing on most of our evals that models are better than human hiring managers at assessing talent. And it's still like the very early innings.

3:08And so I think we'll get to a point where I'll almost be irrational to not listen to the model, right, where people trust the model's recommendation. And like maybe for legal reasons, we'll still have the human pressing the button and making the final sign off. But we just trust the model's recommendations on who should be doing a given task or job more than we trust the humans. I guess in any field, people say that there's 10x people. There's 10x coders who are way more productive than the average coder. There's 10x physicians or investors or you name it. Do you see that in terms of the output of your models?

3:42In other words, are you able to identify people who are outliers? This is one of the most fascinating things is that the power law nature of knowledge work frames the importance of performance prediction. And that imagine if you can understand like the kinds of engineers on an engineering team that are going to perform in the 90th percentile, right? Or even if you could say, I know that this person that costs half as much is going to perform in the top quartile, right? It frames like how you think about the value that we create for customers and how you think about like the long term economics of the business.

4:15And it all ties back to like, how do you measure the customer outcomes and really go on them? And is it a power law or what sort of distribution is it? Because people always talk about human performance as a bell curve. Do you think that's actually true or do you think that's the wrong way to interpret human performance relative to knowledge work? It's very industry by industry, right? For you in investing, it's the most power law thing imaginable, where it's just the top handful of companies each decade are the ones that matter such a disproportionate amount. And it's the investors that went in those versus if you're hiring factory workers, it's a much more commoditized skill set.

4:50There is a lot less of a difference. And I think software engineering is somewhere in between. it's definitely very power law but I don't think it's as power law as say like the handful of best investors in the world. Do you have a prediction for either because of the distribution of like skill level or the measurability like where you should expect that models are better at evaluation or identification of talent beyond a you know human data first? Yeah so it's really everything that you can measure with text, the models are really good at. Like if you can ask questions in an interview and read through the transcript, the models are superhuman at that across many more domains than one would think.

5:37Like it's not, it's more domain agnostic than I would have initially anticipated. I think the things where models are going to be slower is on the multimodal signals and understanding like how passionate is this person about what they're working on, right? Like how persuasive are they or good at sales? And those capabilities will come, but they'll just take a little bit more time. So that's my mental model for thinking about it right now. Right. So like if I'm interviewing a candidate for one of our companies and they are saying the right words about, you know, motivation level, but I don't believe it, like that might be a next level signal if I have any predictive power here.

6:14Totally. Totally. Exactly. The other thing is that the models are way better at high volume processes. And an example is like, say you're assessing 20 people for the same job and you hire those people, you see how they perform. It's very easy to attribute features of each person's background to how they perform, right? It's sort of the stack ranking where you can understand like this person had this nuance in their interview or this person had this nuance in their resume and that was the thing that explained how well they performed on the job. Versus if those 20 people are performing 20 different jobs, Then it's just this like mess of figuring out like what is causing what things to happen.

6:49It's way more difficult to understand like what features are actually driving signal. And so I think it'll be those higher volume processes that also get automated first. Is there anything that surprises you about like basically the discovered features in terms of, I don't know, any domain that you are working on today that identifies amazing talent? That's a very good question. Or maybe in engineering because it's relevant for many of our listeners. Yeah, I think that one of the really interesting things for engineering is that there's so much signal about a lot of the best engineers online that I don't think people properly tap into.

7:29Right. It's everything ranging from their GitHubs to the personal projects on their website to the blog post that they wrote during college. It's just because there's like it's bottleneck by manual processes. The hiring managers don't have time to read through all this stuff, right? They don't have time to, or with designers, they don't have time to consider every proposal or images from someone's Dribbble profile before doing their top of funnel interviews. And so I think one of the things where people are under-indexing on Signal the most is the things that can be found online. But then a lot of the things that can be indexed on during an interview, like how passionate is this person?

8:08Does this person have the skills that it would require for the job? I think humans are relatively good at. At least they're a little bit more adopted right now. Are there hidden signals for other types of domains where there's less online work? An example of that would be physicians, lawyers. There's a lot of other professions where - Yeah, there's all sorts of these hidden signals. One interesting one we've seen in the past is that people who are based internationally but study abroad in a Western country, tend to work much more collaboratively or communicate better with people. And it's like, they're the kinds of signals that make sense when you look backwards and evaluate them, but are hard for a human without having full context of everything happening in the market to really understand and appreciate.

8:54And there's often, one of the most important things, as you can imagine, is just how intrinsically motivated and passionate are people about a domain. And so looking for signals of not just like on their resume and in their interviews, as well as online, of like, what indicates this thing, right? Like, how do we, and it pertains not just to who you hire, but also what those people should be working on, right? Imagine the nuance between hiring a biology PhD to work on like biology problems versus hiring the person who wrote their thesis on drug discovery to write like problems and like come up with innovative solutions contextual to their thesis.

9:31And there's just so much inefficiency with the way that we do matching and the way we use all of those signals right now. So you're evaluating people. Are you also doing evaluations of the models relative to the people? Yeah, yeah, of course. And then what is your view in terms of the proportion of people who eventually get displaced by these models? In other words, if you can tell the relative performance and you can look at relative output, how do you start thinking about either displacement or augmentation or other aspects like that? I think displacement in a lot of roles is going to happen very quickly, and it's going to be very painful and a large political problem.

10:11Like, I think we're going to have a big populist movement around this and all the displacement that's going to happen. But one of the most important problems in the economy is figuring out how to respond to that, right? Like, how do we figure out what everyone who's working in customer support or recruiting should be doing in a few years? How do we reallocate wealth once we have, once we approach super intelligence, especially if the value and gains of that are more of a power law distribution? And so I spend a lot of time thinking about like how that's going to play out. And I think it's really at the heart of it.

10:49What do you think happens eventually? X percent of people get displaced from white color work. What do you think they do? I think there's going to be a lot more in the physical world. I think that there's also going to be a lot of niche skills. What does the physical world mean? Well, it could be everything ranging from people that are creating robotics data to people that are waiters at restaurants or are just like therapists because people want human interaction, like whatever that looks like. I think that automation in the physical world is going to happen a lot slower than what's happening in the digital world just because of so many of the self-reinforcing gains and a lot of self-improvement that can happen in the virtual world but not a physical one.

11:43Do you have a point of view on like what types of skills, knowledge, reasoning are worth investing in now as a human expecting to stay economically valuable? So Sam Ulman said this thing when someone asked him this about how people should optimize for just being very versatile and like able to learn quickly and change what they do. And I think that resonates a lot because there's so many things that one would think the models aren't good at, that they get very good at very fast, that I almost think you just need to be able to like navigate that quickly. What are the characteristics of those things that you think models will learn the fastest?

12:19Like if you were to say, here is a heuristic, what do you think of the components of that? If it's verifiable. For things like math or student code that are verifiable, they will get solved very quickly. So you want a feedback loop or utility function that you're optimizing against as a model. For things that aren't verifiable, like maybe it's your taste at a founder, right? That's much harder to automate. And it's also a very sparse signal because, yeah, there's just not that much data on it. This is a pretty fundamental research question right now. But like, what do you think are the most interesting ideas about verifiability beyond code and math?

12:50Well, I think that there's ways that you can have certain autograders or criteria that humans can apply. And I'm very interested in that models can apply those criteria. And I'm very interested in how that will play out over time. And there's obviously a lot of other domains where models will take unstructured data, they'll structure it, they'll figure out how to verify it. And it's very like industry by industry. I think it's going to be hard for one lab to do everything there. And there's going to be more specialization as we progress further and further and marginal gains in each industry become more challenging.

13:30How much do you believe in generalization from the code and math type reasoning and intelligence? Like if I'm this much better at proof math, does it make me funny eventually? Me being the intelligence. Yeah, I generally believe in it. But to a certain extent, like you still need a reasonable amount of data for the new domain and to kickstart it. But there's going to be a lot of transfer learning. I think it's very funny when Sarah does proofs. So I think it all fits. She's brutal. Yeah, it all kind of fits. I actually think being bad at proofs is funnier. Okay, let's talk about evals because you're working on the bleeding edge of model capability.

14:10There has been this whole sense of what people call evaluation crisis around like the models are so good and they're somewhat indistinguishable at the fringe of capability today that we don't know how to test them. Ignoring all the issues with people gaming the benchmarks, right? Right. What do you think how like what right ideas are there about evaluating models, especially as they become superhuman? Well, I think one of the most important things is that a lot of the evals historically have been for like zero shot of a model or like a test question. Right. That might be academic when the thing that we actually need to eval is like what's economically valuable work.

14:51Right. When a software engineer goes to their job, it's so much more than writing a PR. It's like coordinating with all of the relevant parties to understand what does the product manager want and how does that fit into the priorities of each team and how does that all translate to the end output of work. And so I think we're going to see an immense amount of eval creation for agents. And that is the largest barrier to automating most knowledge work in the economy. Where should people start? That feels not terribly generalizable. So Sierra has something called Tao Bench that I think people are trying and there are other efforts here, but it is perhaps like more specific to a certain function.

15:34Yeah, I think that people will need to have these by industry and they should probably start with tasks that are more homogenous, right? Like it's going to be for customer support tickets. I think that's a great example because there's like one interface that the customer support agent interacts with. Maybe they call a couple of tools like accessing the database or reading through the documentation, but it's a relatively like homogenous uniform task. I think the things that are going to be more challenging, but also in many cases more valuable, are creating evals for these very, very diverse tasks.

16:13All the things that go into making a good software engineer, that's going to be really hard to do. I think it's going to be a years-long build-out for even some of the verifiable domains because there's so much that goes into a good software engineer of how do they have taste for what is the right way to approach a problem or what are the products that people really enjoy using? And I'm really excited for that. So if you were to counsel people with young kids, say your child is, I don't know, five to 10. Yeah. Should their kids learn computer science? I would probably not push them towards teaching their kids computer science, but I'm not totally against it.

16:49I think that the key thing is, I would encourage them to just like find something that's intellectually stimulating they're really passionate about where they can learn general reasoning capabilities. and those like reasoning capabilities will probably be very like valuable and cross applicable. I like always love building companies growing up and like hustling and doing small things like that and I think that is something that could be helpful but I am skeptical that like the really valuable thing is just people who can code in five years. I think it's much more likely like the people that have these contrarian ideas around what's missing in markets and have the taste of what features and nuances need to go into solving that problem.

17:31You said taste a few times. Are there signals of taste that you feel like you can discover in any domain? Yeah, absolutely. I mean, I think that oftentimes you just want to see the softer signals of how people think about certain problems. And certain people have intuitions, whether it be like the way they approach a problem or if they're looking at different like products, how they notice nuances. Yeah, it's very industry. It's very contextual to the industry, but it's important to measure. How can you score it? What's the positive feedback loop here? We've done a variety of things, but oftentimes we will give people like a problem that as closely as possible mirrors what they would solve on the job.

18:15and then we would see how they compare to other people. And so that helps with scoring it. Yes, and for the thought process is part of that. I know, for example, it's almost like looking at like code reviews or other sort of intermediate work along the way relative to something. We definitely do. One thing I've realized about talent assessment is that a lot of people focus too much on the proxy for what they care about rather than the thing they actually care about. And so ideally, you want to measure the thing that you actually care about. So if it's that person building an MVP of the product, Ideally, you have an interview that's like a scoped down version of doing that.

18:48The place where you need to use proxies is when it's like a longer horizon task where you just want to structure the proxy to get as much signal as possible. And so that's sort of how I think about talent assessment. Yeah. Can I ask a scale of impact question? So if I think about the very largest employers today, like let's call it like low single digit millions of employees. Yeah. Right. Or I don't know anything about contractors and Amazon workers and such. But how many people do you think like will end up doing data collection? I think it's a huge volume. I think the reason is that it all comes down to like creating evals for everything in the economy.

19:25I think part of that will be current employees of businesses that are creating evals for that business so that those agents can learn what good looks like. Part of that will be hiring out contractors through a marketplace to help build out those evals. But it would not surprise me if that becomes the most common knowledge work job in the world. How long does that last? So effectively, people are being brought on to displace themselves. This is true. Is that a six-month cycle? Is it a two-year cycle? What is the length of time in which people have relevancy relative to some of these tasks? There's always a frontier.

20:02Unless it becomes superhuman, right? Yeah, unless it becomes superhuman. Yeah, it's almost like time to superhuman. But I had an interesting conversation, which is that you don't even know that you have superintelligence without having evals for everything. Because it's like you sort of need to understand what is the human baseline and what is good. It's grounded in this understanding of human behavior. Yeah, a friend of mine basically believes that, you know, Nyquist theorem, which is basically if you're sampling a signal, you need to be able to sample it twice the frequency in order to be able to actually extrapolate what it is.

20:30Otherwise, you're not sampling richly enough to know. And so he views that there's some version of that for intelligence. Like you can tell if somebody's smarter than you, but you don't know how much smarter because you aren't capable of sampling rapidly enough to understand it. And so I always wonder about that in the context of superintelligence or superhuman capabilities in terms of how smart can you actually be since it's hard to bootstrap into the eval. Well, so I think like when you take it to the limit and you have superintelligence, what you're saying makes a lot of sense. But another way I think about it is that if we classify knowledge work in two categories, One is like solving an end task where it's sort of a variable cost of like you need to do that repeatedly.

21:07And the other is creating an eval to teach a model how to solve that task, which is like a fixed cost that you do one time. It does seem structurally more efficient for work to trend away from the variable cost of like doing it repeatedly towards this fixed cost of how do we build out the evals and the processes for models to do this themselves. That said, it all comes down to like how fast are we approaching superintelligence, right? If the models are just getting that good that fast, then sure, I don't think we would need humans creating evals very much. But I also then don't think we would need humans in many other parts of the economy.

21:42And so you sort of need to be thoughtful about the ratio of that. Does that create an asymptote in terms of how good these things get? Or do they start creating their own evals over time? I think that they'll play a role in creating their own evals. Some kind of bootstraps. Yeah, where they might come up with certain criteria for what a good response looks like. And humans validate that criteria. However, I think you often need to ground this in the experts in that particular domain. Sure. But I'm just thinking like MedPalm or something, right? Yeah. MedPalm 2, where the output of the model was better than the average physician.

22:13It was basically like a health model that Google built. Yeah. And they would use physician panels to rate outputs of the model versus individual physicians, and the model did better by far than individual physicians. At some point, it should do better than the physician panels, where feedback from the physician panel should make the model worse. right. In other words, if you just are all dead off of individual physicians, the model already was going to get worse. And so there's a little bit of this question of how much, when does human scoring create worse outcomes because the humans aren't as good at a task?

22:43Well, I think the models will be able to delineate between the valuable human knowledge and the human knowledge that's not valuable. And that maybe you have doctors that create like a bunch of evals for this particular task and the model realizes like, wow, like I see the mistake that the doctor made on these particular tasks, but I'm going to ignore them. And like, here are the things that seem insightful or the things that I can learn. And the models will, yeah, use that data and value that data immensely. The other thing I'll say is that I think it's easy to look at these evals and the rate of improvement on the evals and just like think we're a lot closer to super intelligence than we are.

23:21But the truth of the matter is like, there is a lot between being really good at suite bench and like replacing software engineering, right? There's like all the coordination problems we talked about. There's like so much else that goes into that. And I think that we're just going to need a lot of evals for tool use. We're going to need a lot of evals for agents. And that build out is going to be a lot longer than a couple year time horizon. How do you think about incentives for all of these like expert knowledge workers? Because the opportunity cost for a great software engineer with like taste and architectural understanding is a great job at Merkur or another, you know, interesting tech company versus some of the geo arbitrage on just basic knowledge work does not exist in, you know, as the skill level increases over time.

24:09That's true in coding, it's true for physicians, it's true for finance people, lots of areas where you might want evals and labels. Totally. I think that it'll definitely become more power law over time, which means that like the best people are going to, of course, make an incredible amount of money. Do you think it's more just turn up the dial on like what any piece of information is worth from the higher skilled workers? Yeah, yeah. But you also want like the evals at the frontier of what the models can't do. So it might be that for like a very well-scoped problem, like answering a medical question that someone has, you might need to get the like, you know, world class doctor that is like, oh, one of the handful of people that's able to be better than the model of that very well scope problem.

24:54But for like, the broader agentic problem of like, how do we, you know, talk about this case in a way that the like patient is receptive to how do we then like, you know, coordinate with these set of tools to, you know, help to complete the diagnosis and, you know, send whatever emails at X time. Like, I think for those kinds of things, I still expect that the like bulk of the bell curve, people that are closer to the like mean of the distribution will be able to contribute for a longer period of time. What do you think is the biggest shift that nobody's really anticipating that's coming? It could be domain specific, it could be broader.