AI Series 2.0 #3: Jonathan Lister-Parsons, CTO and Co-founder @ PensionBee: Navigating AI Hype with Human-Centered Innovation

27:29It's rubbish in, rubbish out at the end of the day, isn't it? This episode is part of the Tech Leaders Podcast AI Series 2.0, brought to you by BDigital. At BDigital, we empower leadership teams to maximize value from their technology investments. Our data and AI services help businesses fully embrace the potential of artificial intelligence. We prepare your data, we streamline your processes and ensure robust governance so that AI works effortlessly to drive real results. Curious to To learn more, visit bedigitaluk.com or follow us on social media to discover how AI can transform your business.

28:21You've been talking about pension B, and it seems like your raison d 'etre is about customer experience and customers, how they interact with pensions. How can generative AI or how has generative AI improved your ability to deliver what you want to deliver to your customers, your vision? So just by way of context, when we've just internally discussed using AI or even before AI, when we were talking about things like chatbots and that bringing that automation directly to the customer's interactions with us. What we've always been really mindful of is two things. One, a pension pot is a really significant financial asset where our main service is combining pensions.

29:13So people are bringing over often 10 ,000, 20 ,000 plus pounds to pension be. And their pension pot is, I think when you sort of average it out across the country, it's the second most important financial asset after residential property. So we have to be very mindful of people's sensitivity around the level of service and the level of trust that is required, especially from a new entrant to the market when you're asking for trust with your lifetime savings. Maybe now at 10 years, some of the trust is implied, but I still think it's a question of respect for what you're engaged in. you're engaged in a really serious matter you're trying to make it as easy as possible and a really pleasant experience but you know it's important to take it seriously so that's one thing to be mindful of and then the other thing as well is the spread of ages and you know sort of not just age but but the wide demographic profile of our customer base because it's a mass market product and so you know a lot of people in that in in that sort of multi-dimensional bell curve of are more comfortable speaking to humans about certain things or not having, you know, the magic of bleeding edge technology sort of waved in their faces when it comes to their financial products.

30:37You know, so there's kind of an interplay with the first point I was making. I think, you know, there are many businesses where putting something in that feels really exciting and cutting edges is into the customer interfaces is totally fine. And if it's a bit messy, it sort of doesn't matter because the stakes are not very high. So I think perhaps what I'm obviously getting to is that we've never been massive fans of sort of putting the robots right in front of consumers. So let's take it back a stage, put it backstage and look at where the robots can be helpful in the business. And I think there are many opportunities.

31:12I think just to stick with the kind of customer service aspects of things, somewhere where I think generative AI can be particularly helpful, but particularly large language models. And it's not so much the fact that it's generative more so much that it understands unstructured information. We get a lot of input and feedback from our customer base. You know, we've got lots of different feedback channels. We capture communications with customers over live chat, email and phone, you know. And so you can take transcripts of phone conversations so we can generate, oh, and then that doesn't include all the public feedback channels and the emails, the specifically feedback emails that we get.

31:50there's a lot of text-based information that we can gather together. And it's obviously beneficial for us to be able to analyze that in close to real time and in high volume and be able to do things like look at the trends, look at where people are having problems, look at where people are actually being very positive so that we can sort of see that stuff has worked. If you try and do all of that analysis work manually, it's a really difficult job. So it's really, you know, we're in the middle of a trial at the moment with our data team to look at what happens if you sort of take all that text and put it into a big, you know, analyze it through a big large language model and, you know, do things like sentiment analysis and topic clustering and that kind of thing and use that to generate a dashboard that replaces, you know, that kind of almost impossible to do level of human work or, you know, complements the existing work that was already being done with a much deeper level of analysis.

32:47So I think that's a really exciting, promising area and obviously something that's in common across a lot of businesses. And then I think other areas that are really exciting. We're still at a relatively early stage with looking at the role of AI in software engineering and product development. But I can see why that would have a lot of promise, because a lot of the areas where that can immediately be useful is reducing boilerplate work or stuff that's in common and helping with design work. And that seems very promising. and then looking into our operations, we're running a lot of very complicated pension transfers for customers and being able to, again, work with diverse and unstructured information and then turn it into something that is a little bit more structured that we can then derive insights from and take actions from at scale rather than it being a manual job.

33:39That also seems very promising. So there's a lot of opportunities, I think, to use AI smartly although I'm actually a bit of a cynic when it comes to AI generally because I think it's just it's going through a hype cycle and I think as I've said probably on your podcast but also in other places I think it's very easy for people to assume that you take technology you drop it into a business and it works like a silver bullet and it makes everything better and AI is almost like kind of like almost the ultimate version of that because it has so much promise you know when you when you see the uh you know the youtube videos and some amazing experiment you don't see the 99 failed experiments but you see the one that works and so people think oh i'm gonna put it in my business and boom you know i don't need to employ anybody anymore it's going to run the show and then of course it doesn't and there are so many human reasons why that doesn't work and actually you know certain fundamental problems that we're seeing around controlling hallucinations and the questions around trademark you know problems and privacy problem more broadly.

34:42There are just lots of complexities that were papered over, you know, for a long time that have allowed people's expectations around AI to go a bit through the roof. So I think part of, you know, part of my role actually is trying to help steer a narrative, you know, for the business, like what should our expectations around AI be and what's actually realistic. I think that's such a good answer. And you and your position as CTO saying that I think is actually very powerful because it also gives other CTOs who may be tuning in and other CEOs who might be tuning in permission to be able to have that conversation with their own teams.

35:21Because I think what can often happen, particularly with this emerging tech and particularly with the amazing kind of case stories and the amazing examples that some people are sharing, is the hype does feel very real and people end up feeling very vulnerable if they're not taking advantage of this new technology in order to drive ROI, drive efficiency, drive productivity gains. And they're worried that everybody else is doing something that they're not. And so there's this kind of hunt for the magic piece of software that they can implement to unlock all of those gains. And so for you to, in your position, to sort of have that confidence to say, you know, it may or may not always be the answer.

36:05Let's do this very thoughtfully. let's do the analysis and let's not necessarily jump in the hype cycle, I think is actually a very powerful thing for people to hear. And I'm certainly a big proponent of always making time to really analyze and get clarity on what is the problem you're trying to solve and therefore what might be the appropriate way of solving that problem, rather than, as you say, just jumping in and assuming that there's some sort of silver bullet software that's out there that you're just somehow not finding. I think you see this a lot with emerging technology. I think one of the differences between generative AI and, say, previous waves, like passion about the metaverse or the crypto waves, blockchain, yeah.

36:53Where's your blockchain strategy? What do you mean you don't have one? You know, that sort of thing. I think the difference is that people can go on to chat GPT and just use it themselves. It's a consumer trip technology, which I do think is one of its massive advantages because it's driving adoption and experimentation in a way that you would really be challenged to do with blockchain and with the metaverse. They were dependent on businesses figuring it out and kind of launching solutions. Whereas it's not really like that with Chen AI. People are just playing about and even via the APIs and things, it's quite accessible to, if you do have some software development skills or just use CheckGPT to help you or Copilot, you can kind of get some stuff off the ground.

37:41So there's a huge amount of very grassroots innovation and that is very exciting. And that kind of goes back to what I was saying right at the beginning of the podcast about what excites me around it. But certainly from a business point of view, sort of the global collection of boardrooms where people sort of go, right, there's this new technology, so what's the strategy? they go well the strategy is to watch it for about 12 months see if it goes away or if it actually works and then maybe do something about it especially if there are some market solutions because you know guess what not every company in the world is an r &d lab or indeed a tech company selling tech solutions to other businesses who then deploy them in their in their business you know we're an end consumer or b2c business so we've always got to be you know very much thinking about well is what we're doing actually helping to solve any customer problems here or is this technology for technology's sake you know which which is something that that i'm not a big fan of which could be fun uh you know in hack day yeah we were talking recently karenza weren't we about ai washing and how things getting badged as ai which is just automation or just even like just normal tech not you know just code basically and it just seems to be and i think that's part of the hype cycle and i think we'll see less of that but but can i just go back because you said you've got a few um not reservation but you're skeptical about a couple of things around the hype cycle of ai what else are you scared what are you fearful of anything jonathan anything you're particularly pessimistic about in relation to the proliferation of it of ai generally in it doesn't have to be in specifically to financial services just more broadly i'm not i'm not fearful of the upcoming AI apocalypse of jobs.

39:28Yeah. You know, I think that's, I think that's definitely not something that I worry about because I think about, I think the, the real opportunities, at least in the, in the short term, short to medium term, probably are AI combined with human beings. It's sort of AI supplemented humans and, you know, via better tools. I think that's a, that's a huge opportunity to help people to, you know, upskill and to increase the impact in their work. And of course, you know, it generates jobs, like we've seen in all sorts of different ways over the decades, you know, technology generates new jobs as the economy moves on.

40:07So I'm not particularly fearful of that. But I think what I am a bit more fearful of in that regard is people who are themselves fearful that their job is going to be automated or that the technology should be stopped at all costs because it's going to have these negative impacts that people are often from not particularly strong position of knowledge talking about. And, well, I mean, how can you? The technology is not really that old and we haven't seen so much of the societal impacts. So I think to have a really, really strong opinion that we're absolutely going to destroy 40 % of the jobs in the economy.

40:49It feels, I just don't understand how somebody who takes themselves seriously can have such a strong opinion about that. But when they're trusted because they've got a position, or they're a published author, or they have a role in companies or in society, or they even just become a meme, then I think that then generates all these consequences from people who don't spend all day thinking about technology and AI. They just hear about how their, you know, if not their job is going to be disrupted, maybe their son-in-law's job is going to be disrupted or their cousin's job. And they suddenly, you know, start to panic.

41:22And I think where that then turns into, you know, sort of societal pressure for not going down a sort of efficient route to getting the most out of technology and becomes, you know, in some cases actually political. That I am fearful of. fearful of and it's part is it's not even partly it's probably mainly because i can't really forecast that myself you know i feel like it's a risk but and it feels like it's chaotic and that you know could go in all sorts of directions but i don't really know where it could go and none of them seem particularly attractive and trying to steer the whole of society down like a sort of you know moderate point of view in regards to a very rapidly developing area of technology how on earth is anybody ever going to do that you know that's almost impossible to do that because everybody's going to go you know their own separate ways and i just think there are so many ways to misinterpret the potential of of ai both from a hype cycle point of view also from like a you know predictions of the apocalypse kind of point of view and that yeah it's it feels like it's a fairly chaotic area to be at the moment so that's i think that's probably my main fear is like that lack of control around the narrative in some respects.

42:38Yeah, I mean, technology replacing jobs has been happening since, you know, for forever, you know, like the Industrial Revolution, the Luddites used to destroy machinery, didn't they? Can I think it back to what you were saying at the start about social mobility and thinking, do you think that there are going to be applications where AI will help people who have had less advantages in life in the way that they've started and less exposure to excellent education and supportive mentors and supportive environments. Do you think AI will help fast track some of those people who have had more disadvantages in life to end up kind of catching up and being able to play in a level playing field?

43:23Yeah, it's a great question. And I think take that back to education in the education system and kind of continuous education throughout life. I think that's a good place to start looking. Before generative AI took off, we were definitely seeing the rise of a type of education where you essentially give people an iPad or a tablet and they follow exercises at their own pace. And hopefully the software is able to provide something of a personalized experience and to help them to learn and then the human teachers they are there to act like a coach and to help unblock people and I'm quite attracted to that as a model I think you know you need to be a little bit cautious with it because there's still a lot of value in in having people interact with other particularly children you know interact with others and but you know just putting up putting all the negative side effects to one side for a moment.

44:24I think the ability to take a teacher and go from, I'm a teacher in charge of a room of 30 people, therefore some of them are going to fail, you know, being that being the reality in many classrooms to this room of 30 people is able to follow personalized educational journeys, and I'm here to support them. And I can do that to a higher level with more people because of this software. So I think that was already something that was in motion. And because it's usually deliverable over cheap hardware, over the internet, and it's software, so it doesn't have to be particularly expensive. That's something that you can see being applicable globally as well.

45:09So I feel that that's a very promising area for technologically driven kind of automation in an educational capacity. And I think AI, if you say, well, what can AI do in that world? I think AI is going to be a better tutor. And I always remember this book, I think it's called The Diamond Age by, I want to say, Neil Patterson or something like that. And it's this classic, maybe early 2000s, late 1990s science fiction. And it kind of invented this, the idea of a tablet that had a completely personalized life coach and educational assistant on it. And it was built for this one little girl who was the daughter of this amazing inventor.

45:57And the culmination of the book is there's a scene at the end where there's a shipping tanker and it is full of Chinese babies. And they've all got these tablets. and it's like well this is what happens if you know this technology gets in the hands of people who can can duplicate it is it becomes this thing that can train up you know a generation of people to a much higher degree of of kind of you know knowledge and education that's a classic book and it's probably inspired a lot of people throughout throughout the years alan kay at xerox Park was really famous for coming up with a really, really similar sort of thing that in theory inspired the iPad, the Dynabook, it was called.

46:42I think this idea of technology and personalized education has been around for a long time. And I would love to see people who are good at producing AI tools going into that area and making that more effective. That's really interesting. Neil Stevenson, I think you were referring to, Jonathan. And I haven't read the book. I Googled it, so just to be clear. But yeah, we'll put that on the show notes, by the way, if anyone's interested in reading that. I might check that out. It's a great book. It's a great book, yeah. I just wanted to ask you about regulation in the financial services sector. Have you seen any regulation or anything that's reacting to the explosion of AI technology in your industry, which I know is very heavily regulated traditionally?

47:26That's a really good question. I think the FCA, they brought in the consumer duty last year. And it's a really interesting framework for protecting consumers and obliging businesses to have a duty of care with their customers and with consumers. And it's a bit different from the FCA handbook and the sort of thicket of regulations. it's more kind of principles-based. And, you know, you read through it and you're like, oh yeah, there's lots of sensible stuff in here. And because it's principles-based, when you think about the use of AI, all of the same principles apply. You know, so is there going to be any consumer detriment if we do this?

48:12You know, that's something you're obliged to think about. And I think that's really, you know, that's really positive because it gives us the ability to do new things, but to be accountable, held accountable for how we use technology in the service of consumers. And I think with GDPR, which is something we've had since 2018 around data protection, that has existing legislation in there or existing parts of that legislation that you can easily apply to AI. So rights around automated decision-making and the concept of a data protection impact assessment so that if you're using new technology, You have to document how you thought through the risks of that technology on your end users.

48:59And I think, again, you know, very, very sensible and very positive. So it'd be interesting to see how, you know, like the EU's AI Act is going to have consequences and where it will come into conflict if it does at all with existing legislation like GDPR or existing regulation. I don't think that there is a gap at the moment. I think AI is technology. It's new, but it's for the businesses to continue to follow the rules that have already been laid out. And I don't see that there's a huge gap that's suddenly exposed by the arrival of capable AI tools. Yeah, for sure. I suppose more companies will be utilizing AI for more things, won't they?

49:43So I'm sure it's something that regulators will be keeping an eye on. It could become a fascinating, fascinating space. But what about security, Jonathan, in terms of like, obviously, you guys look after a lot of very sensitive data. Are there any security considerations that you've had to react to in relation to the use of generative AI tools? Yeah, I think there's proactivity around this. It quite quickly became clear that with OpenAI, for example, ChatGPT, that if you were putting some information in there, they were not guaranteeing that they weren't going to essentially read it and take it and use it for their own training, or indeed look at it with a human pair of eyes.

50:25and so we quite early on this year issued guidance around you know the safe kind of safe use of of ai tools and that involves not putting customer or corporate data into into you know public public tools where they'll take your data unless it's a tool that we vetted and where there's an agreement that the data is not going to be taken by the third party. But I think I find that sort of almost surprisingly opaque area of these tools. They're not really front and center with that privacy sort of stance. Apple are, you know, and Apple often are when it comes to privacy. But even then, you know, you've got this kind of blurring of boundaries where if you've got something that the local models in Apple intelligence won't cope with, they're going to send your data to open AI.

51:21And then you go, well, hang on, is that still going to be covered by the same privacy standard? Or has that been lowered to a lower back, still acceptable privacy standard for me, you know, as a consumer or as a business? Or is it just kind of out of scope entirely and, you know, it's best efforts? And so it's just, it's not a simple, it's not a simple area. And I think the safest thing to do as a business, unless your information security team or your data protection team have cleared that the tool is not going to take your customer or your company data and just don't put it in there. Yes, maybe that limits what you can do, but you can anonymize data with not too much effort before you put it into the tools and still get a lot of the value.

51:59That's been a major kind of area for us, just making sure that everybody's clear on those guidelines. If you were designing a kind of a smart data policy for technologists and software engineers in the UK, is there a particular guiding principle that you think everybody should be working towards, bearing in mind the fast pace of change in tech? Well, in a way, it's frustrating that we don't have more of that because obviously GDPR made a good stab at making data portability something that was a bit more normal. But given we are where we are, what would I say would be a good principle? I'll tell you what I think about data portability.

52:41I think where we're really held back is when companies start products, the assumption is that data needs to sit centrally in order to be useful. And that immediately makes it very hard to implement data portability effectively for data to be smart data that you can move around in different places. And about 15 years ago, there was this sort of brief period where people got very excited about this thing called vendor relationship management that was this idea that people would have their own data on their own computers and they and or maybe online in lockers or vaults and that they would share their data with with companies and um you know the companies would bid for their business and it it's great but it it never worked and i think you know today we're seeing an interesting resurgence, people being interested in local first software applications, offline first software applications, because, you know, you get really excellent performance.

53:50And there's been a couple of useful inventions that allow for the conflicts between data that's manipulated sort of offline or hasn't been synced yet to be reconciled with other data. So, for example, in a Google Doc, you know, if you go offline and you make a bunch of changes, then you can reconcile those into the document when you reconnect, even if your colleagues have already made some other changes in the doc. That stuff used to be a lot harder than it is now. So there's a kind of community around these offline first applications. And some of them are motivated by data privacy and wanting to keep data with the end user.

54:30So I'm not sure what the policy is, but I think promoting that initiative and trying to make it easier for a business to create something where the data is held with the consumers by default, and where they don't have this kind of overriding need to put everything in their own locker, you know, and sort of talk about the value that they can extract from data. I think that'll be quite beneficial. And, you know, maybe one way that you could do that would be to sort of strengthen the rules on consent around what you can use data for. If there's no value in building up a data set because you can't go and sell it to marketers or marketing companies, or you can't, you know, you can't use it for corporate purposes that haven't been already opted into, then, you know, there is a little bit kind of less point in just aggregating it together.

55:20And you've got all of these risks around data breaches and things, which you kind of don't have so much if there is no one central repository where the data is stored. So maybe we're talking, maybe there's, you know, there's a government workshop that needs to happen around this idea, to come up with the policy. But I think that's the combination of kind of technological underpinning and the sort of corporate attitude and attitudinal adjustment that needs to take place for smart data to be a success. Otherwise, it's always going to be on the periphery and people will say, that sounds like a good idea.

55:53But then they'll go off and they'll do centralized data again and again and again because it's where the money is. Good stuff. Well, Jonathan, it's been absolutely fascinating. We could have gone down many rabbit holes along the way, but I wanted to cover a lot of topics. And I think we've covered a lot of ground in this conversation. I just wanted to ask Jonathan about what does the next 12 months look like for PensionBeam? Well, I think it would be remiss of me not to start with, you know, the US. It's obviously, it's a big area of growth for us. And we've only been in a sort of live in the US since August.

56:27So we're very, very early in the journey there. And so over the next 12 months, What I'm hopeful to see is us getting off to a really strong start in the US and starting to build up that brand over there that we have here. Are you spending much time over there yourself? I haven't gone yet. We made a decision quite early on to keep our product development over here in the UK to begin with, which has been great from the point of view of making it really easy for people to build on the knowledge that they've already built through the UK business and collaborate with the team rather than sort of starting from scratch over there or putting whatever it is, a five-hour time zone difference between engineers that need to work together.

57:12So no, I haven't gone over yet. But we see the team on Google Meets and on Slack on a regular basis, so I feel like I'm getting to know them. I hear the chippies are lousy. The chippies are lousy over there, Jonathan. Oh, I know. Don't tell me. But I do love a bagel. I do love a bagel. Look, guys, I'm sorry we've gone way over. I was obviously enjoying myself. Me too. Yeah, it's been great. Thank you so much for having me on. Yeah, likewise, Jonathan. It's been brilliant to see you. Thank you so much for coming on.

57:49So, Carenza, what did you think? Jonathan's a great guy, isn't he? Really, really interesting, yeah. Very thoughtful. very very very high sort of clarity in the way that he communicated as well beautifully articulate which i absolutely absolutely enjoyed listening to i love the fact that i mean it just seems like not not only is pension be such a successful company but the fact that the management team is exactly the same as the founding team i think is a huge testament to the characters of the individuals who are at the heart of the organization. 100%. That is a great data point for a good culture, isn't it?

58:30To have low attrition at the top end like that is a testament to the culture that they've got there. Yeah, Jonathan's great. I really like when he doesn't give you answers on the fly. He's very considered, isn't he? He thinks about the question and you know you're getting the best possible answer. And wow, did we get some good answers. They were fantastic. It was some brilliant content there. I mean, everything from his take on the software engineering stuff right through to ethics, regulation, and so much more. It was a brilliant, brilliant conversation. And I think it's just really great to see British fintech companies doing as well as what those guys are, disrupting such a legacy industry like the pension industry.

59:17It doesn't get disrupted very often. But these guys are actually doing it, Carenza, aren't they? They are. And I loved his founder story when he was describing that magic moment when they realized that this needed to be done. It was wonderful to hear that and understanding their own lived experience as to why it was needed. I thought that was brilliant. And I also liked the bit where he talked, he gave some really, really powerful advice about when to use AI and when not to use AI. When we had Jonathan on first time back in 2021, again, as Jonathan pointed out, they had just gone public, but they've come on so much since then.

59:54And, you know, I was even impressed with them then, but I mean, they've come on leaps and bounds since then. But yeah, I really wanted to get Jonathan onto this series because I know he's got quite a more of a cautious view on generative AI, the proliferation of generative AI. Wasn't quite as skeptical as I thought he would be, but nonetheless very cautious. And I think that's very healthy. And I think an industry like the pension industry and financial services at large, generative AI is going to become a major conversation over the next couple of years because people are already using AI to manage their funds, their pension funds and so on and so forth.

1:00:34So I think it's just going to be really interesting to see how the regulators keep up with the innovation and how all this plays out. But yeah, it was brilliant to get Jonathan's take on a number of elements of the proliferation of generative AI. Yes, and I think because he's an early adopter in his own right, I mean, he's someone who loves technology, who even has on his own wall, as you saw, the kind of the Time magazine front cover. I have to say, you've got unbelievable eyesight seeing that because I would never have seen that. Or maybe you've got a big screen, perhaps. But yeah, well done.

1:01:14You know, he's someone who's an amazing early adopter and yet he is measured. And so I think when you've got that combination of someone who really loves tech, is so passionate about it, it's their life's work. And yet they have like a healthy, measured approach about what to use and what not to use. And his advice around kind of the strategy is, you know give it a few months if not up to 12 months just to see what it does and whether it sticks or whether it goes away I thought that was excellent advice and I think it will give a lot of confidence to a lot of leaders out there who are fretting particularly if they're in a technology role where they are having to kind of hold the fort and almost like hold back the horses when the rest of the company want to kind of maybe rush to adopt and as as he puts it Gareth you know there is no silver bullet.

1:02:12And I think the more of us that are able to help kind of give that sense of reassurance, I think it will really help people make better, wiser decisions. Absolutely.

1:02:28This episode was brought to you by Be Digital. Be Digital support leadership teams to optimize cost and get more out of technology investments. Be Digital and the team have unrivaled experts with technology license management and data remediation and are therefore perfectly positioned to help prepare organizations for AI technology capability. And on the last point, BDigital have developed a cutting edge AI readiness assessment, which provides tech leaders with a platform they need to make well-informed decisions about AI adoption strategy in 2024 and beyond. Go to BDigital UK to find out more and get in touch.

1:03:11Thank you.