The AI trust problem: Can you trust your AI?

0:00It's great to get things fast and it's great to have the powerful capability of AI behind you. But if you can't trust the information you're getting, you can't rely on it. It's not only of no use, it's actually dangerous and harmful.

0:14Jason Mudd: Welcome to On Top of PR with Jason Mudd.

0:20Jason Mudd: Hello, I'm Jason Mudd with Axia Public Relations. Welcome to a new episode of On Top of PR. Today we're talking about the trust problem of AI and how we can solve it. My guest today is Mike Ryan, CEO and co-founder of Bullet Point Network, an AI-driven investment platform built on an AI plus one philosophy that combines AI with human expertise to speed investment decisions. A former Goldman Sachs partner and global co-head of equities and former member of the Harvard Endowment Investment Committee, Mike brings more than 20 years of experience in investing, leadership, and advisory work. Mike Ryan, welcome to On Top of PR.

1:02Great to be with you, Jason. Thank you so much for having me and for that generous introduction.

1:07Jason Mudd: Yeah, I'm glad to be here. Glad to have you here. I'm very interested in talking more about this trust problem that's at the heart of AI and how we might solve it. As we talked about earlier, you know, our audience is going to be public relations, communication and marketing leaders working for a public or private enterprise typically. But we also have a lot of public service and public sector and government employees listening as well as nonprofit. And, you know, I think a lot of those executives are excited about how PR is helping to influence AI citations and how, you know, AI models are depending on earned media coverage and news stories and third party credibility about a brand or an organization to cite, reference and refer or recommend them.

1:51Jason Mudd: But internally, AI adoption is facing some challenges about trust. And so I'm glad we're talking about that today and excited to dive into that a little bit more with you. Great. Yeah, I think it's one of the most important issues that we have. I think trust and safety are issues that are critical in any activity that we do. And AI has really changed the equations on both of those. You know, today we'll probably talk about trust the most. But ultimately, it's great to get things fast and it's great to have the powerful capability of AI behind you. But if you can't trust the information you're getting, you can't rely on it, it's not only of no use, it's actually dangerous and harmful.

2:31And so what people are trying hard to do is figure out the best ways to improve the reliability and trustworthiness of the AI agents and the AI information that they're now.

2:41Jason Mudd: Yeah, I agree with you, Mike. I think that's a great observation. Tell me, how did you get into this space? What was your prompt to transition into AI? No pun intended, but yeah, I have always been an investor and an analyst. Really, I love analyzing businesses in particular and thinking hard about what the factors are that really should drive a decision. A lot of that in my business is some combination of stories and numbers. You have to hear the narrative and understand the vision and understand what the anecdotes and key factors are about what's going to make this special and unique and successful, but you also have to then quantify it and understand things like revenues and growth rates and margins and valuations.

3:28And so that's a task that I've been doing. It's a set of jobs I've been doing for three decades, doing that kind of work. And I love doing it. It's fascinating. Technology has always been a big part of it because gathering information, technology has been an amazing advantage. We're now at a stage and have been for the last five years or so where you can literally use AI to help you analyze information. And as your audience, I'm sure, knows, you can also use it to help produce content, to actually generate information. And so we have sort of a massive transformational change that's gone on here.

4:06You have more information than you can handle. You have a tremendous capability to filter and analyze and now a growing capability to produce both written content and video content and images. And so how to do that best is something that I think everybody is spending time on, whether, as you said, nonprofits, I'm on the board of several AIs on the agenda, corporations, you know, big public companies, of course, and then private companies from, you know, small to large are spending a tremendous amount of time and investors are funding it, are fueling it really with capital. And I think it's a really major trend, maybe the most important trend in our society.

4:44Jason Mudd: So one thing I know we want to talk about is this idea that the biggest barrier to enterprise AI adoption isn't capability anymore. It's simply trust. And so when we're thinking about trust, are we talking about the trust of the internal leadership and management of the company? Is it employee trust? How would you define the challenge? Absolutely. So it's a great, great call out. I think the biggest impediment is not being able to trust the answers that you get from AI, not being able to trust the information that it gives you or produces for you. Think about it this way. We all meet people in our lives, right?

5:22And we have people that work for us and work with us. And if those people one time in 10 say something that you know to be false or is absolutely wrong, you're going to stop trusting them. You're going to think that they're not a reasonable person to get information from to base decisions on. And so none of us would willingly hire or collaborate with a person who's just going to one time in 10 give us sort of bad information, false answers, or what's now in the world of AI sometimes called hallucinations. We don't need that in our midst. But yet we're all using AI more and more. And frankly, we're going to continue to do so because it is getting better, more capable every day, incredibly fast and incredibly powerful at doing tasks.

6:04And so I think the trust I'm mostly talking about is can the users of information trust the answers they're getting from AI?

6:14Jason Mudd: Yeah, I like that a lot. And that's definitely one of the challenges. Anecdotally, I play trivia about twice a week with a group of people. And, you know, there's some people that we play with off and on, right, that'll come in and they're just so confident in their answer. And, you know, then when they get it wrong, you're kind of like, oh, I don't know. You know, you start second guessing them whether you mean to or not. Right. And so I think it's the same thing. We've all kind of been burnt by, you know, AI, you know, with information that we either clearly knew wasn't right or later found out wasn't correct.

6:44Jason Mudd: And even just yesterday, I was working with AI to draft a message. And, you know, I'd gotten farther enough along. I said, OK, we'll put it in my inbox as a draft and I'll review it later. And then before I even could review it later, I got a reply from the recipient. And I was like, wait, what happened here? So, you know, I went back and checked and, you know, it was like, oh, whoops, I accidentally pressed send instead of, you know, followed the command to just save it. And even though I told you that I saved it, I actually ended up sending it. And that was a trust issue for me that, you know, set our relationship back a little bit.

7:17Jason Mudd: I bet it did. Not with the recipient, but with my, you know, preferred AI platform, I'm now starting to kind of say maybe I don't want to give it that offer, that ability. No, I hear you loud and clear. And, you know, those kinds of things are happening all the time. And so it's almost like a tale of two cities or a real paradox. I mean, some of the things that AI does for us and around us just amaze us, right? The speed and the power of it is breathtaking. Other times, it's simply getting things just wrong or doing something that you didn't want or authorize. And you're wondering, how did that happen?

7:55And one of the things that's been built in and trained into almost all the large language models is this tremendous solicitous tone and this very positive reinforcing communication style back to you. And so you'll say, hey, what happened here? And it'll say, you're absolutely right. But yeah, I don't want to be absolutely right. I want you not to do it the next time or I didn't want you to do it the last time. And so we have to kind of work on that. And so we can talk about how we do that. I mean, the way we handle it and what we're primarily dealing with is two things. We're dealing with investors who are making decisions about whether and how much to invest in a business.

8:33So they are typically reviewing many, many opportunities, hundreds, thousands of opportunities that come their way. They're filtering them down to the few that deserve serious attention. They do lots of diligence and analysis on those few. And then they make final decisions with conviction on the one or two to actually advance capital to. That's one of our main use cases. The other one is we work with companies. We work with the portfolio companies of those funds and some other companies. And they're usually making strategic scenario models to plan their business, to analyze and understand where their opportunities are, where their risks are, how their results can be amplified.

9:15And then they're communicating that. They're making internal communications to their C-suite and their executive team, making board presentations. And then they're making external presentations often to capital partners that are going to fund them in the next financing round, to M &A partners that they might want to do an M &A transaction with. And so we help with those two use cases, and both of them involve a lot of information, a lot of synthesis of stories and numbers, and frankly, some amount of prediction about the future. You're forecasting things, you're telling stories of opportunity and talking about where you may take your business going forward.

9:53And so that's actually pretty hard work, that last part in particular, synthesizing and forecasting the future and then communicating it. And so we've really worked on optimizing the trust factor and the trust layer on top of what the models can do.

10:06Jason Mudd: And how are you doing that? It's a combination of technology and workflows. The main phrase that I think most analysts know all too well is garbage in, garbage out. If you put bad inputs in, you're going to get bad conclusions out. It doesn't matter how smart you are or how hard you work. So the first thing we try to do is create a lot of filters on the source pile, the source information that the AI uses. A lot of that is now done by some pretty sophisticated database architecture that we've developed at BulletPoint Network. But you can also do it as simply as just making sure that you only include or upload sources that are relevant and accurate.

10:46We then do some pairing and prioritization because a lot of our clients are using literally millions of pages of documents and many, many dense spreadsheets of numbers. And we're combining all that. And the way AI works is it has what's called a context limit. So you type in a prompt, you give it source information or it searches the Internet, and it can absorb or read only so much. And that's the context window. So what we do is we pair every prompt with the most relevant sources in that walled garden of trustworthy information so that you're not overloading the AI with too much. I mean, you might feel good.

11:29Hey, I've loaded all the information in. AI is giving me an answer. But it's not actually reading all the information. It's sampling it. So we want to give it the most relevant things to sample from on each and every prompt. That's step one. Then step two is we believe very strongly in transparency, right? Sunlight is the best disinfectant for problems. And so we give the user traceable, clickable footnotes that take them not just to the document where it came from or the spreadsheet, but to the exact area, the exact page and paragraph or cell and number that it came from. That's really helpful.

12:04So people can do some spot checking. They can also look at those areas that prompted and delivered the answer to them. And they may want to human read more information around that to give their own brain a little bit of infusion. But transparent footnotes and source citations are critical, traceable, clickable, usable footnotes. The last thing that we are big believers in, and this is maybe more unique to our business, but similar, I think, to marketing experts who are very quantitative in some cases, numbers are really important. And so having a trusted spreadsheet be the calculation engine for any calculations that your AI is relying on, having the ability to sort of control and understand what the inputs are, what the formulas are, what the outputs are in that spreadsheet, and then have the AI look at both files of words and folders of documents and also trusted numbers from a spreadsheet on which we can build with our models scenarios like upside, downside, base case, and make lots of visualizations, charts, and tables.

13:04It's great to have all of that visual benefit, but only if the numbers underlying it are trustworthy. And so we're big believers in the AI plus one concept. We're the AI bullet point network, but you're the one. And you should have control over the thesis, over the sources, over the prompts, and over the flow that's used. So that's basically how we do it. We have a combination of people and technology and a purpose-built workflow to enable you to drop in at just the right time in just an easy way to iterate and understand and collaborate either among a team of humans or you with a team of agents or a combination of people in AI.

13:44Jason Mudd: And are you describing an AI-enabled advisory support? Is that what you're talking about here? Yeah. In large part, it's AI-enabled advisory support for investment advice, investment analysis, strategic finance advice for companies and strategic decision-making. We're really talking about decision-making support. Okay. Yeah, that makes sense to me. So you're also talking about, we were going to talk about how off-the-shelf AI is powerful, but it isn't built for high-state communication. Can you kind of explain that to us? Sure. Well, we're big fans of off-the-shelf AI. So we have connections and APIs.

14:20They're called into all of the open AI models, all of the anthropic cloud models, all of the perplexity models and Gemini models. We use them all. We also have our own proprietary models that we use and train. And so we believe that there's a lot of value in the off-the-shelf capabilities. It is important to recognize though, they've largely been trained on the internet. They've largely been trained on a huge volume of information that's available on the internet. They have their own limitations because they're trying to be all things to all people. So they're trying to give restaurant recommendations in Monaco, healthcare advice for your mother-in-law, information on your own financial plan, and the ability to sort of, you know, fix an HVAC system in your house and many, many other things.

15:09That's an enormous remit to do. And so what you have to often do is sort of curate with your own domain expertise, information and training and workflows that are exclusively focused on what you're doing. And that's something that our AI in the world of investment analysis and strategic finance decision-making can do particularly well. There are other domains that do that very well for marketing and other activities, healthcare. I know of several other platforms that do it in those other domains. But you want to have domain expert information available and feeding and training any model that you're using.

15:49And then frankly, you want to have human domain experts. I mean, one of the good things is while people are anxious about AI for lots of reasons, and we may or may not get into all of them here today on the call, but one of the reasons that people are anxious about it is they worry if it's going to eliminate their job, you know, just to speak freely and bluntly. And I often say AI is not going to take the job of most smart people, but a person using AI better than you might take your job because there's almost no better time to be a domain expert or to be someone with great judgment, real experience, real creativity, and the ability to persuade, which I bet many of your audience has in droves, particularly the last point, communication and persuasion.

16:36And fundamentally, a human that is well-versed in how to use AI, how to use quality control and features to make it more trustworthy, and himself or herself has domain expertise, judgment, and experience, they're more powerful than they've ever been in human history. They are the person who can be 10x more productive, 10x more impactful, and more.

16:58Jason Mudd: Yeah, I typically agree with you that, you know, the people who are hesitant to adopt AI, at least in the last several years, are the ones who I think are really far behind. And then you see it when the light bulb comes on and they're like, oh, my goodness, this is so powerful and I can do so many more things. You know, it's fun to watch that transformation happen as well. So I know one of the points you want to talk about was how AI can reduce the constant rebuild of the senior communication or of senior communication. What does that mean exactly? Yeah, well, you know, I grew up doing this work, as I said, you know, building analysis, presenting it to my team, my boss, the various investment committees, management committees and commitments committees and the like.

17:40And frankly, when new information came in, it seemed inevitably to come in in the evening and not in the start of the day in the morning. And then you'd have to really update all your materials, update your spreadsheet model, update your slides. You'd have all kinds of work to do. And, you know, technology in general and AI in particular can really cut through that in a dramatic fashion. We can actually set up, you know, capabilities and agents and workflows so that the updating process is trivial. It's done with ease and with speed. And what that does is it enables the producer of the work, you know, me, 20 years ago, to think and be, you know, using their brain for real judgments and real discussion and debate with others, not to sort of rote mechanical updating of words and numbers and names and paragraphs and charts and tables.

18:35And so that's a really important thing. Now, at the end of the day, you're always going to read it and quality control it before you turn it over to your CEO or to your capital providing LPs or to your investment committee or your boss, whomever your boss is. And so I think having an AI plus one mentality in general is a good one. But when you're handing it over to important people, it's something that we all do. But you want to be able to go into that where you are bright eyed and clear minded, not exhausted from having just done a lot of mundane rote work.

19:05Jason Mudd: Exactly. I feel that same way many times in how we offer our services to clients as a PR firm, right? We're trying to make it so that the head of communications within the organization isn't down in the weeds of the day-to-day tactical and blocking and tacking of their role where they can be more readily available on as a trusted advisor to the leadership team and also available to think strategically as opposed to worrying about getting back to their office and knocking out a deliverable or some other tactical work that needs to be. be completed. And so I think if you use AI the same way, you can elevate your thinking and your mindfulness about the organization and thinking more as a strategist than worrying about kind of the day-to-day activities.

19:50So I couldn't agree with you more. I mean, as I said, I think it's going to create a more extreme barbell than we've seen before between sort of the exceptional high quality thinkers and everyone else because the gap is just widening as people that are really able to elevate and give critical insight, judgment, and persuasion are going to separate themselves because everybody can basically do things in an automated way, often with AI, that cover the basics.

20:22Jason Mudd: Yeah. So, Mike, if somebody's interested in learning more about you and your company or connecting with you, what's the best way for them to do that? Sure. Lots of people connect with me on LinkedIn, Mike Ryan, Bullet Point Network, and our website is bpnai.com. And we have lots of information on our website and you can connect with me over LinkedIn or check us out on the website. All of our companies and many of our funds actually use PR in their own lives because they know that they have to get their message out and they're all trying to optimize what the search engines will say about them. And now increasingly what ChatGPT and Claude and the AI engines will say about them.

21:02And so they have a lot of focus on how they get their message out. And the one thing that I'd say I've found from the best firms that we work with doing that, which is in your neck of the woods more so than mine, is it's pretty much now become about content, about producing interesting, thoughtful content. People are much more likely to find that. And the AI engines are much more likely to read and absorb that. And then that helps build your authority and your brand in effect in a way that, you know, direct advertising, just talking about your features and your product and you doesn't really do.

21:40The AIs are really figuring out. And that's why some people are getting boosted to the top of AEO and getting AI to sort of recognize them. It's because the AI is reading your content, reading your thought leadership and your authority in whatever area you operate.

21:54Jason Mudd: Yeah, that's exactly right. That's what we're finding as well. You know, our clients that have hired us to create earned media coverage for them are seeing that they're getting more leads from, you know, AI referrals and references than they have in the past and certainly more than they're getting nowadays from, it looks like, search engines. So if people can stay on top of that, I think that can be very valuable for their organization. Absolutely. And there's a tricky concept and a new term that's finding its way into our society, which is called AI slop. You got to be careful that you're not producing a lot of generic AI output that really isn't that insightful or deep.

22:32Just because you can produce it fast doesn't mean you want it or want to distribute it. You want to make sure that you have something meaningful to say. And one little tick and trick that we recommend people use is some version of a 20-60-20 strategy where you, a smart human, do the first 20%. You come up with the ideas. You start the process. You get the ball in play. Then AI does 60 % and deepens it, advances it, gives you critical material that supports your direction. And then you are the last 20 % and you put the conclusion together with your own nuance and insight, wrapping it in. And so people that let AI start usually end up with AI slop.

23:17People where a human provides that first cut, then let AI take the big middle and then you put the final finishing touch on it. That seems to be the right recipe for most of our users.

23:27Jason Mudd: Yeah, I completely agree with you, Mike. In fact, we've talked about that a little bit on this show, the exact same concept and borrowed originally from kind of, at least for me, the source was the idea of that's how Steve Jobs used to lead his teams where, you know, give them the first 10 % of the input, work closely with them to identify, let them do the next 80 % of the work and then check in at the last 10 % to make sure it was in line with what you were hoping and expecting. But, you know, whether it's 80 % or 20%, you know, I think it all kind of works out just fine. Yeah, I think if you're Steve Jobs, it's probably a 10, 80, 10 thing.

24:02And maybe, you know, that's generous. But for most of us with our own individual projects, it's probably worth our time to give it the first 20, which is a little bit of 10 on the direction and a little bit of 10 on the, what are the important things that are in my thesis or on my mind? And when you get them, they get the AIs going. But got to be real careful because, you know, when I was a kid, I wouldn't take a ton of family vacations, but they mostly followed a pattern of my dad sitting in the front seat with my mom and the kids in the back. And somewhere along the way, occasionally he would say, good news is we're making great time.

24:34The bad news is we're going in the wrong direction. It wasn't a great maps or Google maps and so forth. So you don't want AI to take you fast in the wrong direction.

24:43Jason Mudd: Yeah. I like that a lot. I like that a lot. Mike, thanks for being here and sharing your insights with our audience today. I really appreciate that. You've made it clear they can connect with you on LinkedIn. We'll put a link to LinkedIn, your LinkedIn profile, as well as your company website in the episode notes to make

25:03Jason Mudd: Thank you.

25:28Jason Mudd: dot com slash on top of PR. That's rate this podcast dot com slash on top of PR. Once you do that, screenshot your review and email it to us at podcast at Axia PR. The reason we need this is because we don't know who Jaybird in Toronto is and we want to be sure to send them a thank you for the review. Should they be the one that we draw? Once you enter, you stay in the drawing until you win. So one review goes a long way. It takes about two minutes to get entered and your name could be the very next name that we call. So thank you for taking a moment to help us. We appreciate you and your loyalty.

26:02Jason Mudd: And I'm here to help you stay on top of PR. And I hope that that was accomplished today. With that, this is Jason Mudd signing off from Axia Public Relations, hoping something great happens to you today. Be well. This has been On Top of PR with Jason Mudd, presented by ReviewMaxer. Be sure to subscribe so you don't miss an episode. And check out past shows at ontopofpr.com.

26:54Bye.