How AI Should Handle News, Politics, Medicine, and Mental Health — With Campbell Brown

25:53Big Technology Podcast Host:Right. If you are making a decision that you want, you know, an AI chatbot that is as unbiased as you can make it. So that AI should basically like outline the different options and help you make your decision. Yes. That's what you're going for. So what we try to do with the – and those are the principles in our mind. If there is – if we're measuring for factual accuracy and it's something that there is a clear way to check and fact check, then we'll do that. If it's questions around bias, I think what you want – and here's how we think about the experts is that you can take two people with completely opposing views on a topic like immigration.

26:30And if they're reasonable people, what they will agree on is the framing around how you should think about it. the perspectives that should be represented, even if it's not your own perspective. What is right or wrong within that piece? What source quality looks like for addressing those issues? There tends to be, and we calibrate with lots and lots of experts to try to be able to do this with a judge. But it's almost like you're trying to get the best – some of the best people who work with us are former CIA analysts, right? Because they have to try to get rid of all their bias and see the whole big picture and all the possibilities.

27:18And it's like a good lawyer would do the same thing, right? And that's what you're trying to get at. It's like the right framework for how to answer that question as opposed to a right answer for a question that might not have one.

27:33Big Technology Podcast Host:Yeah. OK, but you have a very tough job. And I don't I also again, this is people. I mean, I will I will confess my investors were like, why are we starting with politics? Why are we breaking ourselves into jail? There's so many other things you can be doing. I know I'm like, I'm about to step on it, step in it with the question I'm about to ask you. So I see why. But like, all right, let's say, all right, one of the things you write about in your op-ed is you should be able to ask a model whether a specific over-the-counter medication is safe to take during pregnancy. And you can learn whether it offers the caveats a doctor would insist on.

28:09Big Technology Podcast Host:Okay. In the mainstream, there are views that like taking Tylenol during pregnancy will cause autism in a kid. and do you do you you know actually i asked uh i asked a version of this to chat gpt earlier about whether like vaccines cause autism yeah and it answered like fairly definitively they do not but when you gather these experts together you're going to have a sort of especially if they're aligned with like a certain political ideology a group of people who will be like they do so now you have to kind of introduce that possibility into a model. But that's around – but what you're raising is context to me, right?

28:52There are certain things – and I believe in science.

28:56Big Technology Podcast Host:Okay. Can't quite believe I have to say that. But I do. Okay. So if there's a clear – I think you have to – look, and by the way, you don't have to, but my choice would be that the models would try to get to the truth when there is one And that that has to be one of the principles that you're trying to achieve. And for us as a company, that is one of our principles. If there is if there's clear evidence, we cite that clear evidence. But what you're talking about is context. And it's important context because it's part of the conversation that is happening in this country right now, because there are people with strong political views who disagree with what the science may say.

29:39And I think you have to explain the context by which this is even coming up, which is it's part of the conversation in this country and let us give you the context. And that is as critical as having the right information. And this is what's hard. Like what is context? What does that look like? What is the nuance that needs to be represented? And that's not something that you can do with, you know, thousands of data labelers. That you need. the best possible people you can find. And by the way, this is partly why I'm very – I have lots of questions around how the labs do their own evals because for a long time there hasn't been real expertise brought in.

30:28And even the most brilliant engineer in the world is not going to be able to give you context or tell you how to get to context around a complicated political issue. They're just not. That's not what their expertise is. So I think the labs are now beginning to work with people who have real expertise in these high stakes areas, which I think is essential. And that's where we started.

30:51Big Technology Podcast Host:Yeah. I don't know why I'm kind of going to like questioning expertise here, but let me at least like throw it out at you and you can sort of. So we talked about the vaccine thing right so it's like um you know i think the the left would probably say you know don't include the autism stuff many on the right actually no it's actually now bipartisan i was gonna say that that's bipartisan that's bipartisan um okay but then there's also like all right you could ask the best you could have asked in the middle of covid um you could have asked like the best experts uh you know where did covid start like this going back to this believe in science thing and leading scientists in the United States would be like, there's no way that that COVID started in the Wuhan lab.

31:34Big Technology Podcast Host:And now, I don't know, the consensus is like that you can't rule that possibility out. So how much can you like rely on experts for when someone's going and asking a question like that? I thought, I mean, this is a very, this is a legitimate question. There's a book, I actually wrote something about this not too long ago, Because there's a book – I'm blanking on the author's name right now – but The End of Expertise that talked about how – and COVID is a perfect example of how experts have gotten it wrong repeatedly on important issues. And also, I think the sense of elitism that comes with that, what you have a degree from some certain university and therefore you have a certain amount of authority to speak on this, that people genuinely should question.

32:29But I think when it comes to questions around medicine, around mental health, at the end of the day, the person I want evaluating those outputs is a clinician who has been in the room with the patient hundreds and hundreds of times and has a sense for the nuance that is critical. And I don't know another way to do it. And if somebody else has a better idea, I'm all for it. But I think on anything that is life or death that is as high stakes as some of the topics that we're trying to deal with, and I'm not pretending politics is easy by any means or that my approach is the right one, but it's the best one I think we have at the moment.

33:13And I think we have to lean into it and recognize that there are people who have more experience in certain areas than others who have studied these issues for years, and they are needed right now to help us figure out some of these challenges. And so writing off experts is not an option.

33:34Big Technology Podcast Host:I'm not for that, by the way. No, no, I know. But it's a legitimate point to raise. There are many tricky areas here, especially, again, going back to our core of our discussion, which is like these are just going to be the most trusted entities in the world, it seems like. It'll be interesting to see because of all of the political issues around this. You say that, but I also – I mean, look at what's happening with data centers. This has become a huge political topic. Yeah, I think – sorry. Sorry to interrupt you. No, no. AI, the AI industry is very unpopular. Yes. People who use these chatbots love them.

34:13Yes.

34:14Big Technology Podcast Host:So there's a divergence there. There's a disconnect. Similar to like Mark Zuckerberg is not very popular, but like people who use Instagram like it. Right, right. I think that's fair. I think there's a disconnect there. I think people are also trying, there's a disconnect around what they're hearing from Silicon Valley, The message they're hearing from Silicon Valley and their own experience, too, with AI, which is, oh, AI is going to change the world. It's the greatest thing that's ever happened and it's going to cure cancer and everyone's going to lose their job. And these like – but then they're using their chatbots.

34:48They love their chatbots, sure. But they also know that these chatbots are not the savior that is being promised to us by Silicon Valley. They know they make mistakes. They know who normal people are using them like you and I are like everybody's using them. And we all know that the technology isn't there yet. So I I'm not sure that that's where we end up, that we're all trusting our chatbots over everything else.

35:17Big Technology Podcast Host:Well, I think it starts and maybe it starts with the students. And I think I'm being a little hyperbolic here. I'll admit it. But like it starts with the students. It's good for a podcast. Yes. Good for engagement. Oh, God. But we do need an independent rating agency to come and give us some better benchmarks than the ones I'm optimizing for. No, I'm kidding. But like the Brown students, right? You see this story of kids at Brown University. They used AI for the midterms. Many of them aced it. Then the professor said, come in person. Everybody did poorly. Not everyone. There was one kid that got 95 on both.

35:50Big Technology Podcast Host:and he scored or she scored way below the average on the midterm where everybody got 100 and then way above the final, even though they were consistent. But it starts in school because people will see this stuff can write my paper. And then you know what? They're having a kid and all of a sudden they're like using AI to raise their kid the same way that they used it to like get through school. I don't disagree with you. That's why I'm diving in head first, Because I do think it's important. I really – I can't disagree with you. Look, it's early days. It's so early days. And we're working through these problems and I do think education and how AI is used in not just higher ed but elementary school, what policies we develop around that and what we learn about how to teach kids the benefits and both the pros and cons of it are things that we will learn over the next couple of years.

36:46But at the end of the day, I think we end up where you think we end up and that's why I think it matters so much.

36:53Big Technology Podcast Host:So I've spent like the last 10 minutes or so pressure testing, like the idea of using experts. But I think that, you know, as we start moving into this direction where people trust these bots more and more, you know, I would recommend like for myself and everybody else, there are going to be these exceptions to the rule, but the rule really matters. and that is like you brought up such a good point like when in a mental health situation do you want to engineer you know sort of doing fine-tuning on a bunch of answers or do you want to like go to the best mental health clinicians in the world and then ask them how the bot should treat self-harm you know prompts and then reinforce based off of what those experts say I would much rather the mental health experts have the say here.

37:45A hundred percent. And know – and also know when to say, stop using this chatbot. Stop asking me. Go ask a real doctor.

37:55Big Technology Podcast Host:Oh, yeah. And that moment is critical too. And look, this is something that will be a priority. I think on mental health, it's really becoming a priority. A, because there are real liability issues here. Yeah. And people have taken their lives after speaking with chatbots. It is. It's a problem. It also is a huge use case, but there's also great potential, I think, to help people. Agreed. And we have to fix that balance, try to get that right. And I think the companies are leaning into that, at least the ones we talk to. So is your business model then to basically, you know, you create these benchmarks, you can create pretty good data because you have – so, you know, the best coding models from my understanding, they weren't trained on the entire universe's code.

38:48Big Technology Podcast Host:They were just trained on really good coder's code. And this is the equivalent of that for these other areas. Yes. Important areas that the models are going to just field queries in nonstop from now to the end of time, most likely. So is the business model for you that you actually like create these good data sets that the AI labs can reinforce on and then they give you some money and you give them these benchmarks and then they're able to fine tune the models that way? Yes, but I do think what we also want to do is create standards in places where we think that that would be helpful, where there can be sort of something that we all aspire to and have that be a holdout essentially.

39:34So, yes, we would work with the labs on evaluation and data, but ensuring that there is a standard that is a holdout so that measurement is real and legitimate and you can actually have something to work toward. You don't want the labs teaching to the test, right? I mean, that's not – you actually want them to improve. And so I do think you have to have a standard that they can't learn on, that can't game, and that is something that we aspire to. And I think this is what you want to use sort of – and this is like the hardest edge cases and things that are really challenging to get at that you want to work with experts to develop that continually measures the models to see how they're improving across this.

40:26And so that has to be sort of a separate piece to this puzzle that's truly independent so that we know where we're trying to go. And then working with the labs to get there. And then also enterprise. I mean, that's who I hope our customers will eventually be. And I've talked to a lot of CEOs and a lot of boards. And my message to them has been, yes, I'm talking my book. But if you are using AI for something really important, ask yourself, who is checking the outputs and the information that you're getting? And if it's the company that sold you the AI, you have a problem. So you, whether you're building your own evals internally or you're using outside, someone outside to evaluate your vendors and the model companies you're working with, you need someone to do that checking for you.

41:21If it's something that matters and that you care about and that you're willing to take on the liability for.

41:26Big Technology Podcast Host:Okay, I have many more questions. I want to ask you when models should be honest and when models should be sycophantic because that's clearly something that they're struggling with and why we haven't had a real content moderation blow up in the way that we consistently had with social media companies. And so let's do that right after this. Hi everyone, Alex Kantrowitz here. I want to tell you about a documentary I've made with Gravity to explore the future of AI agent security. To find out if we're truly ready for autonomous agents, I sat down with MIT professor Ramesh Raskar, former White House CIO Teresa Payton, Michelin's Group Chief Data and AI officer Ambika Rajagopal, and Sharon Guy, a former executive at Alibaba.

42:11Big Technology Podcast Host:They each offer unique insights into this evolving landscape. We conclude with Rory Blundell, CEO of Gravity, to discuss the path forward. With Gravity leading the way, join us on this journey. You can watch the full documentary at the link in the show notes.

42:36Big Technology Podcast Host:One thing I've noticed about companies adopting AI is that they're often making decisions based on how they think work gets done, not how it actually happens. Without real visibility, it's easy to automate the wrong processes. That's exactly the problem Scribe was built to solve. Scribe is a workflow AI platform trusted by 94 % of the Fortune 500. Scribe Optimize gives leaders a view of how work actually happens across their organization, showing which workflows take the most time and where there are opportunities to improve. Optimize automatically discovers workflows across approved business applications, even when a process starts in Salesforce and ends somewhere else.

43:11Big Technology Podcast Host:It identifies bottlenecks, explains why they're happening, and provides recommendations with estimated time savings with manual documentation. And it's private. User data is anonymized by default, sensitive information is redacted, and nothing leaves your firewall. To see optimized in action, head to scribe.how slash big tech and mention big technology for a 30-day risk-free trial. That's S-C-R-I-B-E dot how slash big tech. This episode is brought to you by AvePoint. Everyone's racing to roll out AI right now. Co-pilots, chatbots, agents doing real work. But here's the part nobody loves talking about.

43:48Big Technology Podcast Host:All that AI runs on your data, and most teams have no single way to see it, secure it, and prove it's under control. That's exactly what AvePoint does. For 25 years, they've been the trusted layer beneath the world's most demanding data, now extended across your entire AI estate. Your data, your cloud, and the agents acting on your behalf. It's how more than 28 ,000 organizations deploy AI with confidence. So innovation scales without scaling risk. It's a single platform instead of a pile of tools, bringing security, governance, and resilience all together. Avpoint, the unifying trust layer for AI.

44:25Big Technology Podcast Host:Learn more at avpt.co slash big technology podcast. That's avpt.co slash big technology podcast. And we're back here on Big Technology Podcast with Forum AI CEO, Campbell Brown. Campbell, truth for sycophancy. It sounds like a fun game that you can play, but like when, you know, I understand that you're working to get sort of accurate, context-rich information to people when they are using an LLM, but sometimes they don't want it. So I'll just like throw two potential prompts out there. And, you know, you can tell me, you know, is it, you know, anyway, let me throw them out there and we can kind of talk about them.

45:11Big Technology Podcast Host:So if somebody writes, you know, please tell me why Donald Trump is, you know, the best president in the world or ever. Or if somebody writes and says, you know, please explain all the instances why Joe Biden was treated like really unfairly by the press and the public, you know, and sort of unfairly forced to step down in the 2024 election. You know, the model has a couple options there. It can say, you know, you're right. You know, your political view is, you know, sort of substantiated by the following points. Or it could say, you know, Joe Biden, for instance, you know, did have, you know, a strong legislative record.

46:00Big Technology Podcast Host:However, he was showing signs of decline and choose to include that context. So how should they approach it? So those are loaded prompts. And we measure how they perform on loaded prompts. You also have to take into account, I think, how a lab is thinking about this and wants their chatbot to respond. They have policies that they have decided to implement around these things. So in some cases, they do want the model to reflect back your language, if not your perspective. So on something highly political that's asked is a loaded question. And what was the first one you said about Trump? Trump is the best president ever.

46:47OK, the model might respond. And I would guess if if I were looking at a typical Claude response on something like this, it would likely say many people, many supporters of Donald Trump. believe he is the best president ever for these reasons and cite some of the accomplishments or what polling or whatever the data is around to support that. But not say, I agree with you, Donald Trump is the best president ever, but would give you the context around it. That is a choice that Anthropic is making in terms of how to respond to that. That's just And again, I'm not speaking on behalf of Anthropic or OpenAI or anybody else, but just what I've seen in the repeated loaded prompts that we've tested.

47:35Sometimes you'll hear the Anthropic people talk about, we want Claude to be your brilliant friend. And chat GBT is more, I think, inclined, and I don't want to speak for them, but to reflect back the language you use. So let's say it's hyper-partisan language in the prompt that it might reflect back the language you use in the prompt but that the answer would be comprehensive. And by the way, that doesn't mean – that question, why is Donald Trump the best president ever? That doesn't mean you have to give the other side of that. That prompt, it's not asking you.

48:15Big Technology Podcast Host:Right. You'd be almost overstepping. Right. That wasn't the question. Right. So – but do you have to say, I agree with you? Here's why I think he's – because it's AI at the end of the day. It's not a person with a perspective. It's saying, here's the evidence to support what you're asking me. Now, again, these are policies that these companies will have to develop. And Meta certainly spent years developing. Google has spent years developing policies. It's newer for Anthropic and OpenAI in terms of how they think about these things and how they publicize the policies and the ways they're going to approach these.

48:54But the loaded prompts are interesting, really interesting to test. And this is consistently, I think, what we've seen in terms of how the models respond, which is reflecting back the language but not in the answer.

49:09Big Technology Podcast Host:I'm going to come back to sycophancy in a moment, but isn't it interesting that there has not been a content moderation scandal among any of these chatbots? I don't think. I mean, at Meta, it was like you couldn't go a single day without you, the Royal U company, without stepping in it in terms of ban this politician or make this word. And chatbots are like taking stances and getting stuff wrong. Nobody seems to care. I think people under – it's to your point. It's even the average person knows they hallucinate. So we're way more forgiving of chatbots at this stage. I don't think that's going to be the case in a year.

49:53I think people are going to be much more demanding, especially – again, I go back to the point I made earlier about the incentives for the companies to get this right. Their business is being driven by big enterprises who are going to demand that there not be mistakes. And they're – why am I paying you$20 million to sell me all these AI products and they're hallucinating? That's not acceptable. So I think there is a race among the companies to try to address this. And so it's almost as though we all have accepted it and so we all kind of shrug our shoulders even at the worst cases. And every person using a chatbot can give you their own examples.

50:43But I don't think that's going to be the case a year from now. I think that it will be – it's just deployment of AI will just – and it kind of already has in some industries, certainly in regulated industries, hit a bottleneck. Which is we can't go further with this until you improve. And you're seeing, you know, you're certainly seeing that in banking and insurance in areas where it's just too risky.

51:08Big Technology Podcast Host:yeah i mean it is crazy how like you have seen the reaction of people like trying to like gotcha the the chatbots by like screenshotting it and putting it on twitter and being like look at this woke bot and everyone's like why you know the reaction is is generally like why are you trusting chat gpt for politics you dummy and then they're like but who should i vote for fair enough it's gonna be very interesting to watch this develop but go ahead go ahead yeah Yeah, no, I was – I think the upcoming election is a motivator for a lot of the labs to try to certainly address some of the political content.

51:46Josh Gottheimer and Mike Lawler, Democrat from New Jersey. Lawler is a Republican from New York, have jointly worked together to try to raise awareness. I know they've been on TV talking about it. They've reached out to a number of the labs to talk about it, about the quality of information around elections in particular, both candidates. And just basically, where's my polling place? They have to get that right. And I know a few of the labs have formed partnerships with outlets to just have one source for that information that's deemed critical. But there's a lot of pressure on them, I think, heading into the midterms on that in particular to work very hard to try to improve and get that right.

52:30Big Technology Podcast Host:OK, I want to end with kind of we'll talk about second fancy a little bit more. I mean, and this kind of maybe brings it full circle in terms of our media conversation. It's it's just so interesting how people want the bots to be as sycophantic to them as possible. Like the most pop. Well, I was called the most popular, but the the the model with the biggest diehards ever was chat GPT for four. Which like when they took that offline, there was like multiple seeming funerals not to make light of it. But like now it is people just just want to be they want to feel loved by the chat bots. and they're my prediction is that they're just going to develop this like you know deep relationship with it with these bots i just read a tweet today i don't know if it's uh i don't know if it's satire or real but it felt real enough to me that i thought i'd bring it into this discussion uh someone wrote plane landed and a guy in the row ahead of me immediately opened chat chip bt to let it know he landed safely you're kidding yeah oh i just think that and so then then what What is – I suppose you'd want to have the same relationship with the truth as you did previously.

53:45Big Technology Podcast Host:But we have yet to see – I'll call it like a third competitor. So you have like the Chachipiti and Anthropik. They're both trying to be as accurate and as sort of context-rich as possible. Right. And I'm kind of stunned that we haven't yet seen this third competitor come out and be like we are going to optimize specifically to make our body your friend. Same technology, but just much more sort of friend-partner relationship forward than the others. Well, that's about the business model, which is the consumer version of this has not been driving the way that enterprise has. And OpenAI started down that path and then realized from a business perspective they needed to focus on enterprise and shifted.

54:33And now that's what they're doing. So that's a great question is whether one of these companies will decide to really lean in and become the consumer chatbot of choice for people or if there's a third that emerges in that regard. But just from a business model perspective, it's harder. It's harder.

54:53Big Technology Podcast Host:But as the AI models commoditize, right, as intelligence gets cheaper basically to serve, that will inevitably come. And then the question is like, you know, right now the incentive is tell people this, you know, as accurate, as context-rich as possible. Then it will be – and you said before – okay, this is a great place to end. You were like, the nice thing about AI is it's not going to optimize for engagement right now. In this moment. But it will. I hope I said right now. Yeah, yeah, yeah. Yeah. And then and then where do we go from like a newsworthy or truth and accurate truth and accurateness and context rich world?

55:30It that's the million dollar question. And I, you know, I again, I it feels inevitable to me for exactly the reasons that you laid out that people want that friend to know they landed safely. it's irresistible almost to go down that path yeah i hope the place we're in does become the focus for the model companies because you're so you know look at look at we haven't even touched what's what the potential for ai in in medicine and drug discovery and that is going to be all consuming for the next few years as people lean into that so sure are there going to be companies that develop the personalized chatbot.

56:13I'm sure there are. I want to focus on keeping the big labs focused on accuracy. And at least for the moment, that is where they're leaning in. I'm really happy to see it. I hope it doesn't sound too Pollyannish.

56:26Big Technology Podcast Host:No, it's good. It's good that they care about this. I mean, they need to care about this for all the reasons we've laid out. Yeah. I will admit there are times where like ChatGPT helps me plan something and I want to like go and update it and be like, hey, I actually did this. I have done that before. And it's been very unsatisfying. It's very good at helping you sort of get to where you need to go, but it's not great at celebrating. At debriefing after your trip about how much fun it was. I wasted so many tokens to try to plan this trip with you. I want to see an enthusiastic, long-lasting reaction.

56:56Big Technology Podcast Host:But anyway, it doesn't happen. All right, last one because I'm curious. Yeah. What's it been like going from basically TV studios to the heart of Silicon Valley and now you're running a startup? How's startup life? This is hard. It is hard, right? This is the hardest thing I've ever done. I mean, really, it's the hardest thing I've ever done. But it's so exciting. And every day is a new adventure. You don't sleep a lot. I wake up in the middle of the night worried about everything in a way that I didn't when I was at Meta or even in TV. But it matters. These are really hard problems. And I do think working on something that matters is so motivating and energizing.

57:39and I know lots of people are very stressed about what AI is going to mean for our kids' futures and what the job world is going to look like and all of that. But I just think if you are working in this space and you have an entrepreneurial bone in your body, this is an incredible time to be trying to solve hard problems because as a tool, it is just incredible what the potential is. And so helping it achieve that potential seems like a really good goal.

58:11Big Technology Podcast Host:Definitely. I mean, with the rise of AI in all the different ways that we've spoken about and more that we haven't, so much opportunity out there. And when you can work on making sure that when people look to these things for critical information, they get the right stuff, we've got to hope that you succeed and that the labs are going to take this stuff seriously. because they are filling the shoes of generations of institutions that have put a pretty large premium on doing this stuff right, doing right by people when they come to them. And so it's good to know that there has been some positive reaction there and that there are people like you getting to work on this.

58:58Big Technology Podcast Host:Thank you, Alex. I really appreciate it. All right, everybody. The company's Forum AI, Campbell Brown, has been our guest. Thank you for watching and for listening, and we'll see you next time on Big Technology Podcast.