#325 Phelim Brady: Why AI's Future Depends on Human Judgement

22:43Phelim Brady:Yeah, particularly as sovereign AI develops, I mean, these different regions or countries are developing models that are trained on local data so they're more culturally attuned. But I'm interested in the persuasiveness project. How did that work? I mean, can you describe that? I mean, you have people in your database and they talk to a model or they read outputs from the model or what's happening there?

23:22Craig Smith:Yes, there were many manufacturers there and it was an experimental workflow. So essentially, people were split into randomized conditions. And as a result, there was kind of an AD style test of whether participants, if they were exposed to a back and forth conversation with one of, I think, about 20 models, and these models were instructed to persuade the participants to agree with one of political stances using different rhetorical strategies. So the experimental methodology was fairly sophisticated. And then the participants were measured on whether they agreed with the issue before they were exposed to the model and the conversation and after, and then as a result, you're able to measure the difference in how different models and different different strategies informed how much these participants changed their minds on the topics that were at hand.

24:25Phelim Brady:Yeah, but the interaction between the participant and the model, is it like, you know, have a conversation with this model for 30 seconds and, you know, about some prompt, some topic or is it that there is a prompt and the model responds and the participant reads the response or is the participant actively engaging in a conversation with the model

24:58Craig Smith:it was i believe it was the latter so an active engagement multi-turn multi-turn conversation i'm not sure if the i don't have the exact constraints to mind i think it was perhaps a minimum number of interactions that were required in order to accept the data point. But yeah, it was certainly more sophisticated than a just read the output of a model. It was a kind of a live interactive style of experiment. Yeah.

25:22Phelim Brady:But that's a really interesting use case. So you need a certain level of political awareness. I mean, how did you pull together that cohort?

25:37Craig Smith:yeah in this case this i think goes back to the representation and the generalizability of the of the audience so trying to get a sample of of the real world so that the the outcomes of that experiment are more likely to generalize to real world scenarios and to a real world context was a crucial criteria of of that that kind of project and we have the tooling built into the platform which allows you to do census-matched sampling, which means that the resulting participants you get broadly match the population on key criteria, such as age, ethnicity, political affiliation, things of that nature.

26:19Yeah.

26:20Phelim Brady:I mean, you said you don't do much sort of labeling anymore. Is that right?

26:26Craig Smith:That's right. We never did a huge amount of, let's say, the simple kind of commoditized style of labeling. One could call the post-training evaluation data sets a form of data labeling.

26:37Phelim Brady:Yeah, yeah. Actually, that's what I was getting at. I mean, you know, in the early days of supervised learning, the focus was on labeling, particularly for computer vision. And then just in the last year, as these models have matured and become more widely used, and as there are more models competing, evaluation has sort of come to the fore. How long have you been doing model evaluation?

27:09Craig Smith:Yeah, we've been working in that area, I would say, for two to three years. So I think post the chat GPT boom, the shift of user-driven model evaluation has shifted towards, as I said, kind of more complex audience requirements, more expert participants and a requirement for more kind of scientific rigor. And that's been a natural fit for our platform capabilities, which was the original wedge within this space. We've obviously invested pretty heavily in additional capabilities and frameworks and the ability to kind of convert human judgment into useful signal for these model developers.

27:55Phelim Brady:Yeah. Yeah. And that's, yeah, as the models, I mean, evaluation on benchmarks has gotten sort of a, people have soured on that to a certain extent because you can, there are all sorts of problems. You can train your model to the benchmark or, you know, the benchmark somehow ends up in the training data and all sorts of things. So people don't trust, you know, a model winning the math Olympiad anymore. That at one time was a big deal. Now they're more focused on this human evaluation. does that is the market like blowing up for you guys that end of the work yeah i think that is a

28:42Craig Smith:key driver of our expansion in the market this shift away from like the academic benchmarks or exam-based benchmarks which the frontier labs use to kind of advertise their relative performance there's such a strong incentive as you said to to either to unintentionally gain these benchmarks and leaderboards so their usefulness is decreasing as they become saturated and essentially the models are able to all pass with with flying colors but then it's becoming harder to judge the the real world evaluation and i think that's where platforms like ours come to their own and in trying to simulate the real world environments in which these models and agents are interacting with real people trying to achieve real problems and then evaluating that performance is ultimately the real the real goal that's ultimately going to drive kind of the the economic impact of

29:40Phelim Brady:these models have in in practice yeah you guys you know are part of an industry you say you have I've forgotten the number. I heard million, was it a million or multiple millions or something on the platform? On the evaluation side, if you were to look globally, do you have a sense of how many people are involved in evaluation through platforms like yours?

30:11Craig Smith:I don't have a strong estimate of the global number of people involved in this type of work though I would say it is it's increasing and also the the types of people who are likely to be involved in it over the next couple of years are likely to shift considerably as the labs kind of shift their focus towards knowledge work and understanding the capabilities that these models have in kind of all of the areas of the economically productive labor, folks within all of those industries are going to have some involvement in evaluating and providing feedback to the tools that they use day in, day out in order to start to improve their performance, not only on these academic benchmarks, but actually how they're performing in real world conditions on real world problems.

30:59Phelim Brady:Yeah. I mean, there's evaluation of the frontier models. That's one thing. But then there are all these applications built on top. Is it companies building applications that come to Prolific to evaluate their applications? Or are there enterprises that are adopting solutions who want to evaluate? Increasingly, both.

31:26Craig Smith:I would say that the labs are leading the way as they have the talent in-house to understand the methodology in the frameworks that are required to build out benchmarks. And the majority of labs will have in-house teams and in-house evaluations that are accustomed to their requirements and are not necessarily always one-to-one mapping to the public benchmarks. But as enterprises start to build on top of this infrastructure and build agents for their specific use case or their specific context, they increasingly also need this evaluation, not only of their own tools, but also because there is such a wide choice now of which open source or frontier model to use, even evaluating which base model to use in the first instance is a useful piece of data and is not necessarily obvious up front.

32:14Craig Smith:But then as they build their application, in particular understanding, I think safety and reliability and trustworthiness in real-world conditions is often they have a higher bar for that kind of robustness in specific applications than perhaps the infrastructure layer is going to have.

32:29Phelim Brady:Yeah, yeah. Yeah, so enterprises, I mean, if I'm an enterprise and I'm building an application and I have a choice of, you know, eight different models to hit for my inference, you know, through an API. Yeah, you know, I've seen sort of casual rubrics where, you know, Claude is good for this, ChatGPT is good for that, and, you know, Gemini is good for something else. But, I mean, who knows on your specific use case. So are you having enterprises come to you and say, we need to figure out which model works best in our application and do A-B tests, as you say, or A-B-C-B tests or something.

33:21Craig Smith:Yeah, I mean, I think that's you precisely laid out one of the challenges that we want to tackle, which is many of this evaluation is based on vibes and based on intuition that domain experts have. We want to bring a layer of objectivity and rigor to this evaluation, which I think is going to be particularly important in enterprise applications and particularly in regulated or sensitive domains like healthcare, finance, law, things of that nature. And then on the second part of your question, that choice of base model is just the first step, right? I think there's many more steps than in terms of testing for safety and having a continuous flywheel of feedback, both from automated evaluations.

34:03Craig Smith:We've talked mostly about human judgment evaluations, but there's also a whole strand of technical and automated evaluations, which is a complement to the human judgment. And then also understanding the delta between the performance and outcome that they're driving towards for their application and then the gap between that and their perhaps fine-tuned model or customized model. And then critically, the tooling and the products to help them bridge that gap between the capability that they're after and the kind of baseline capability that they get with the existing model.

34:38Phelim Brady:You mentioned agents at the very beginning in the context of ensuring that you have a human on the other end doing the evaluation and not an agent. But the agents are being employed in evaluation and they're getting better. Is there a reason why your industry wouldn't eventually fade away as agents play that role?

35:06Craig Smith:Just to be clear, we're active users of agents and LLM as a judge and similar tools in order to augment human performance and make sure that we're using the relatively expensive component of human judgment in an intentional kind of cost effective way. I think there's a few reasons why I think at minimum it's going to be a long time before

35:31Phelim Brady:the human component dies out.

35:33Craig Smith:Firstly, I think the human judgment is where the alpha is. So if you're looking to push the capability beyond what is already available, almost by definition, you are not able to build an automated evaluator that is able to assess that gap in capability. So you need human judgment, human judgment there. Of course, in cases like chess, the models have gotten to a stage where the human judgment is no longer kind of a net benefit to improving the models. But I think chess and similar verifiable domains are a fairly small set of the useful tasks. And wherever there is kind of ambiguity or subjective opinion required, human judgment is going to be in the development lifecycle for a long, long time, as well as I think crucially the trust and safety component where I think at minimum having an escalation path where the model confidence is low or where you want to cherry pick and just make sure that the models and evaluators are performing as expected.

36:37Craig Smith:So certainly I think the human judgment and the human's role in this lifecycle will change. I think the relative value that it provides is going to increase, if that makes sense.

36:49Phelim Brady:Yeah, absolutely. Yeah, and I didn't ask that question as a challenge to your viability. I'm just imagining the future, but that is one of the really interesting things about AI. I mean, these large models is they encode human knowledge off of text, and there's a lot of ambiguity and a lot of misinformation included in the text. And then we'll ultimately need humans to guide the models. And it sounds, you know, you didn't come back with a number, but it sounds like there must be millions of people around the world engaged in this. Do you think that's an overstatement? Millions?

37:37Craig Smith:I don't think that's an overstatement, for sure and also i think the many more people are going to be involved at minimum in a passive way as more and more of these models are rolled out in as as products and i think just maybe touch on your other other point we've worked in with academics in the field of human computer interaction for for a while i think that field is shifting maybe towards like human ai interaction or human agent interaction so i think there's a lot of work to be done in actually understanding like how to get the best of both humans and AI systems, and how do you make the collaboration and the engagement between them better than the sum of its parts, as it were.

38:16Craig Smith:This is not necessarily kind of obvious how it's going to work at Apri, and I think will require quite a bit of intentional effort in order to understand how to develop systems that work well for humans and make sure these agents and systems are human-centered and are supportive of folks' work.

38:36Phelim Brady:How much AI do you use in a prolific? I mean, either in, you know, managing the platform or sourcing participants or, yeah, I mean, is there AI in your platform?

38:51Craig Smith:Yes, and an increasing amount. I think there's an interesting fact that AI is quite a convergent force in technology in the sense that increasingly, I think products are evolving to the magic AI boxes where you type in your request and you expect the AI to do its magic and return its result. I think the very early version of the we've rolled out is on the data collector side, the audience requirement tool. So you just express who you're looking for in natural language and we go away and we use the data that we have on our participants in order to find the right people. rather than you needing to really think about the and or requirements of how you select for those people.

39:33Craig Smith:And similarly on the contributor side, instead of asking them a battery of questions to understand who they are, we're able to offer them a path where they're able to interact in natural language through audio and video. And then we're able to use that richer qualitative data in order to extract what's important to the other side of the platform and improve that matching. and I think that abstraction will increase where the magic box will you'll ask it instead of find me these people it will be okay and design me the data collection tool and please go away and actually execute on the the work and then ultimately maybe automate the full workflow so the researchers or data collectors kind of stay in the outer loop of the project and designing the hypotheses and the outcomes that you want but increasingly the the agents and the AI tools that we build allow them to automate a lot of the details.

40:27Phelim Brady:Yeah. And you're doing that now or that's on the roadmap?

40:31Craig Smith:This is the roadmap. So the two examples I gave at the start of the define the right audience through natural language and the understanding of the contributor's background through AI interaction. Both of those are live products and I think early examples of more to come. Yeah.

40:48Phelim Brady:How does the customers, whether it's an individual PhD student or a foundation frontier model company, how do they pay? Is this a subscription or a pay as you go?

41:04Craig Smith:Yeah, it's a fully usage-based billing cycle. So we will recommend the recommended pay to the contributors, and then we take a portion of that payment. But it's fully usage-based. So you can go from tens or hundreds of dollars through to tens of millions. And it's purely just the amount of data that you collect and the amount of time that you ask from the contributors.

41:28Phelim Brady:and how much of your business is kind of retail in that sense that you know people are coming on the platform for a short project and then they're off and how much is it big enterprises that have longitudinal studies going on that they need you guys for yeah we span the spectrum

41:56Craig Smith:of budgets and wallet sizes and use cases. I'd say the vast majority of users have recurring use cases. If you're in the business of collecting data or running research, typically you have many projects, or even if you move companies or universities, bring your preferred tools with you. So yeah, we see the majority of customers come back on a repeated basis.

42:19Phelim Brady:Yeah, you mentioned polling also. Do people use you to run surveys?

42:28Craig Smith:Yes, again, this is a use case that is supported. And again, I think something that we're increasingly interested in as the complexities of running high quality polls is becoming more and more challenging for a variety of reasons. Harder to reach people, harder to incentivize people. So I think that's an interesting role that we can play there. Yeah.

42:53Phelim Brady:And on the compensation side, each project is the contributors are paid according to the project. So there isn't a standard hourly fee or task fee that they're earning. And is it enough that people, they can supplement their income certainly, but can people earn a living income as a contributor?

43:21Craig Smith:We don't optimize for folks who want to earn a living on the platform. So primarily it's a side hustle or supplementary income. And yes, all of the projects or the pay is based on some combination of the length, complexity of the project and complexity of the requirements on the audience side. But all of that is made transparent to both sides of the platform. People have that openness ability to kind of, it's not a black box in terms of us going away and not showing how the sausage is made. The data collectors and the participants are able to interact with each other and see those sides of the platform as well.

44:00Yeah.

44:01Phelim Brady:And where do you see Prolific going? I mean, as I said, I would guess that there's growing demand as more and more companies release these models. But then there's also, it seems to me there would be opportunities, for example, on polling, where not only do you source the respondents, but you could run analytics on top of those responses. I mean, if the data is flowing through your pipeline. So where do you see Prolific going?

44:40Craig Smith:Yeah, totally. I think it's exciting use cases across a range of applications. And ultimately, our ambition is to become a full stack human data platform. So provide a global, high quality participant pool where you're able to access both the breadth and depth of humanity. the tooling, whether that's through partnerships or through our own tool to tap into those humans for a wide range of different applications, and then supporting the methodology and the tooling really to advance the frontiers of research and AI. And yeah, much more work to be done on all of those strands. And yeah, many exciting opportunities, both in the developing transformative or human-centered AI, and then also really understanding how this is changing and influencing human behavior and really being the platform that provides this authentic, high-integrity human data in the age of lots of AI-generated data as well.

45:42Phelim Brady:Yeah, and I know I'm up past the hour. Do you have time for another question or two?

45:50Craig Smith:Unfortunately, I probably need to drop one more question. Absolutely.

45:57Phelim Brady:Yeah, there's last year or in the past year, there's been this boom in humanoid robots. But the critical block bottleneck for humanoid robots is data collection, you know, because with LLMs, you've got the internet full of text, but humanoid robots, you need, you know, data from humans manipulating objects or, you know, identifying things in scenes. Are you doing any work on data collection for humanides?

46:30Craig Smith:Yes, it's not the core of the work that's done on the platform to date, but is, I would say, an area of open discovery. In particular, you mentioned you had Faye Lynn on the podcast recently, and she probably talked quite a lot about world models being a required environment in order to accelerate the training of embodied AI. So we've actually done some very interesting work on integrating prolific into virtual environments so participants can take part in data collections in VR. So I think we've gone from kind of text, images, video. I think world models and virtual reality are an interesting modality that we'll definitely explore in the coming years.

47:14Phelim Brady:Okay.