Using generative agent simulations to drive better experience outcomes in healthcare - Interview with Sri Narasimhan of CVS Health

30 Jul 2026 · 43 min · 16 chapters

Ask about this episode

Ask anything about it. ChatGPT or Claude reads this page and answers with the times it was said.

Connect VO and ask about every podcast you hear, including the moments you saved. Add to ChatGPT · Add to Claude

In short

CVS Health uses generative “agentic” simulations (agentic twins/digital-twin style) to mirror how consumers/patients make decisions, then run “dress rehearsals” to predict experience outcomes before launching healthcare products, messaging, and workflows.

Guest background

Sri Narasimhan is VP of Enterprise Customer Experience and Insights at CVS Health. He leads consumer experience and insights across CVS retail pharmacies, Aetna insurance, Caremark PBM, and healthcare delivery (Oak Street Health, Signify Health).

Key claims

Traditional research struggles with declining survey response rates and limited ability to test new scenarios. Agentic twins amplify the customer voice by simulating individual decision trade-offs, enabling speed, fidelity, scale, and reach. CVS validates twins against human studies and monitors drift; a “human at the steering wheel” controls confidence and directs follow-up testing.

Notable examples

Message testing and ad testing drop from ~6 weeks to minutes (2–3). Pharmacy choice drivers include location and insurance; adherence insights surfaced “refill anxiety.” PetRx research found emotional devotion to pets matters, but connectivity to the veterinarian and efficient prescription fulfillment are key. Twins enable ongoing access to hard-to-reach specialty patient groups.

Written by AI. May contain mistakes. Listen to the episode to check what was said.

Chapters

Tap a time to open that second in VO

Overview of CVS Health and Its Operations

0:46 to 2:37

Sri explains the structure and services offered by CVS Health.

“CVS Health is one of the largest healthcare companies in the United States.”

Understanding Consumer Experience at CVS

2:38 to 4:24

Discussion on how CVS gathers and analyzes consumer insights.

“Now, you alluded to it about something that you've been up to recently.”

Introduction to Agentic Simulations

4:25 to 6:13

Sri discusses the concept of agentic simulations and their relevance.

Challenges in Consumer Centricity

6:14 to 8:13

Exploration of the difficulties in achieving consumer centricity.

“We used to put a chair in every meeting and we'd be like, you have to think about the consumer.”

Amplifying Customer Insights with AI

8:14 to 10:29

Sri explains how AI enhances customer interactions and feedback.

“So one is it's really important and rich source of training data.”

Building Agentic Twins for Consumer Insights

10:30 to 12:39

Details on how agentic twins are created and their purpose.

“So we go and get panels of customers like you would in traditional research.”

Understanding Individual Simulations

12:40 to 14:00

Discussion on simulating individual behaviors for better insights.

“Like if it was just what they did in the past, we have tons of models and things like that that do that.”

Leveraging Generative Agent Simulations in Healthcare

14:00 to 20:42

Explore how generative agent simulations can enhance customer insights in healthcare.

“So now we can actually twin those groups where traditional research panels, super costly, and you can only get them one time.”

Impact on Business Processes and Consumer Experience

20:42 to 25:27

Learn about the significant business impacts of using generative simulations, including speed and fidelity.

“So I would say speed, fidelity, scale, reach are the four right off the bat where it's had a massive impact for us.”

Governance and Ethical Use of AI in Simulations

25:27 to 28:00

Understand the importance of governance and ethical considerations when implementing AI-driven simulations.

“And do you think that, and one final kind of thing on this specifically, because you talked about being transparent about your approach and how that's kind of been responded.”
Show all 16 chapters

Understanding AI Twins in Healthcare

28:00 to 29:24

Learn how AI twins are used to improve decision-making in healthcare.

“who are actually using the capability and that human at the steering wheel is able to look and say, you know, okay, this one is a good, we have a low confidence on this response.”

Advice for Implementing AI in Complex Organizations

29:24 to 31:10

Explore best practices for introducing AI capabilities in large organizations.

“And that's a journey for us, but we're working on it.”

The Importance of Team Engagement and Governance

31:10 to 34:02

Discover the significance of team buy-in and governance structures for effective AI use.

“I think one of the biggest ones for me that it's unlocked is really product and service development.”

Transformative Potential of AI in Customer Experience

34:02 to 36:28

Understand how AI simulation can enhance customer experience and insights.

“So that's about it for my big questions around kind of what your New York can approach.”

Key Advice for Improving Customer Experience

36:28 to 37:24

Learn essential strategies to enhance customer experiences and add value.

“but it amplifies the consumer voice in a way I've never seen.”

The Unifying Power of Sports

37:24 to 40:37

Reflect on how sports can bring people together and foster community spirit.

“Shift your mindset from being like, what is the value I can extract from my customers to what is the value you can add to your customer?”
Hear the part that matters, and keep it.Open this episode in VO. Double tap your headphones to save a moment as you listen.
Get VO free

Transcript

Automatic transcript. May contain errors.

0:00So welcome to the next edition of the Punk CX podcast. With me today I have Sri Narasimhan. Now I checked that was the correct pronunciation of his name and I think I got it nailed according to Sri, which is great. I'm really happy to say. But Sri is the VP of Enterprise Customer Experience and Insights at CVS Health. Sri, welcome to the podcast. How are you doing? Thanks Adrian. Yes, 10 out of 10 on the name. You nailed it on that one. So LTR on name. Excellent. So, Tri, if I could ask you, maybe just give me a bit of a thumbnail sketch on you and the work you do across at CVS Health. And also explain a little bit more about CVS Health for people that may not be stateside and may go on it.

0:43CVS Health, don't I know that name? Yeah, so CVS, well, I'll start with CVS. CVS Health is one of the largest healthcare companies in the United States. So as part of our portfolio of companies, you have retail pharmacies, which a lot of people experience as our CVS pharmacies, if you've ever been to many locations throughout the country. We also have a health insurance arm, which is Aetna Health Insurance, which is one of the largest health insurers in the country. And then we have Caremark, which is our PBM, which is a pharmacy benefit manager, just prescription benefits for consumers. And then finally, we have healthcare delivery, which is a primary care business where Oak Street Health and Signify Health, where we actually treat patients and serve patients with healthcare.

1:27In terms of me, the function, I think what's most relevant here is I lead the consumer experience and insights function where we are focused on understanding the consumers better than anybody at the company. of the campaign. So consumer sauce, as I mentioned, would be people that go into our stores, members of the insurance business, and Caremark business, and then patients, and then also clients. So we have a large B2B kind of segment in those insurance businesses. So understanding what those people experience using a variety of signals. We've run a large survey program, about 18 million surveys.

2:03We look at Google reviews. We look at social media data. But now increasingly, increasingly using AI, we mine things like call transcripts, digital sessions. We've now run a, we have an always on MPS for every single member in our insurance businesses, but, you know, using 200 operational variables. And now this kind of new concept, which I think we're going to assess today around agentic simulation of consumer behavior. So leveraging all that and working with the business to drive action and change to, to improve the experiences as much as we can. and deliver the best in class experience. We want to be the most trusted company in healthcare.

2:38Nice. Now, you alluded to it about something that you've been up to recently. And I know I got wind of it because you recently authored a white paper with a gentleman by the name of June Park, who is a similarly CEO. And it outlines your kind of approach, key findings, and some broader implications of using generative agent simulations in healthcare now is that sort of like like a digital twin type of thing and if so can you tell me a little bit more about what you're doing and kind of how it works because it all sounds very fancy it is it is a digital twin uh the thing is i think you can't say uh you have to use the word agentic today it's like required if you want if you want someone to listen to you you have to throw agentic in front of everything i you know if you go to i think you can buy an agentic cold brew now oh yeah i didn't know if you know that it's a cold brew and i'm just yeah but i think the concept is similar to a digital twin so the idea is now with the way ai is developed you can actually mirror human thinking decision making and you know how people balance risk and so if you can do that and say i can create a twin an agentic twin of you adrian i can then put you you know, generalize your thought process, your thinking to new scenarios.

3:55So I can take you and put you in another scenario and say, okay, given these parameters and this situation, how will Adrian respond, both in terms of your preference, but also in terms of your choice, of what choice will you make? Now, imagine if I do that for hundreds of thousands of people. Now, all of a sudden, and this is the concept of what we're doing to the Agentec Twins with Simile, we will have you know an army of customers with us throughout our entire process and always on and ready to help us build better experiences test learn and run a dress rehearsal on anything we're launching and that's really the concept here is you know can we we used ai to actually amplify the consumer voice is the way i think about it because we can bring them now the thought process of the consumer into any new situation and i can get a bit kind of nerdy on the technology kind of just like any anybody else but i'm also quite pragmatic in my kind of like outlook and sometimes think well this is all very nice technology but is it just technology for technology's sake or is it that you're actually aiming to solve an actual kind of problem that you were facing organizationally i mean so what problem is it that you're trying to solve with this new approach i mean where's the gap if you like with it because you talked about all the data all the data sources that you can have where's the where was the gap that you were that you're trying to solve with this new approach yeah and it's kind of funny to say that i i solved consumer centricity by building an army of robots but that's kind of where where we went um because consumer centricity is hard we talk a lot about you go look at some 10ks you'll see it in every 10k someone's like i want to be the most consumer-centric company but consumers are difficult to talk to you know surveys are difficult as you as you know and i listen to your podcast and you hear a lot of people come on here talking about how difficult it is to get feedback via survey those response rates are declining getting populations of customers to come talk to you is difficult even the data mining generally speaking most of the data you're looking at is exposed so something happened to someone and you're observing what happened and then you know you're banking on history repeating itself, but you're not taking that consumer and putting them in that situation and seeing how they'll respond.

6:10So for us, this was kind of like the old gimmick we used to do. We used to put a chair in every meeting and we'd be like, you have to think about the consumer. That chair is the consumer. I know everyone in CS is probably a version of that. Now I actually have hundreds of thousands of customers in the room. So if you think about it, it's that's the problem we're solving it's we are bringing the customer voice into every conversation from start to finish soup to nuts we're bringing them from product development through the develop that development process now in terms of even testing and knowing what's going to happen before we go live so the consumers experience something that we know from the ai simulations of them that they got value out of and that they found the experience you know beneficial so to us it's solving that core you know centricity problem and i think simulation is it's right you know right now it's still new i think we've been at it for a little over a year maybe just under a year and it's only going to get better but it's it's basically allowing us to see what will happen and see how consumers are going to experience these things before it actually happens it's a little bit sci-fi man honestly it's like uh you know what you're in westworld or something but or the matrix but it's it's really got that that ability and we're using it actively to make decisions and and build experiences and you know so far the early returns have been wonderful and so before we get on to the kind of like you know the impact of stuff i mean because i also know that i speak to lots of different people and we they talk about simulations and digital twins and and all these different sort of things and particularly around i've i've talked i've talked about this on the podcast before i remember being at the uh the the qualtrics event kind of earlier in the year and they because they've got a big market research arm of their sort of like business they've started talking about kind of building these kind of like simulations and stuff as well and one of the biggest questions that came out of that because there's there's you know when you talk to that market there's a lot of there's a lot of market researchers that are in that market and they're a bit like oh or people go does that mean you're not going to talk to customers real people anymore with kind of like pulses and things it's like so explain how this kind of like how it fits into that that whole sort of thing yeah a hundred percent and i will say like one of the funniest things about ai i've observed is everyone has an existential threat from it like it's like you know it's like immediately like oh man it's going to happen to me but i think like one of the things that we always think about at the company is it's not we don't think about it as replacement we think about as amplification like how is it going to make us you know more productive and and reach groups that we weren't able to reach before and i think that that's that's the way we we think about it here um just just in kind of broad philosophical terms it's it's additive it's going to grow it's going to increase our impact i think that we're never going to stop talking to customers i mean that's the business we're in even as not just me and cx but the company right we have to learn real people the way i think that this is kind of shaping up you know is one we use the customer data to help improve the twins like we know a lot of what you hit on it a lot of this is a change management exercise like when you're when you're asking people to trust an agentic simulation you need to show the proof that hey you can mirror what people are actually thinking and doing.

9:36So one is it's really important and rich source of training data. I think that continues to happen even with surveys. Like, you know, I know surveys get a bad rap, but surveys are really important in terms of understanding motive and like what customers, why they're doing things. So we can continue to use that as training data. But beyond that, the conversations we have with customers are more meaningful now. Because with the twins, I don't need to, I can ask them the questions that are more straightforward, more, you know, decision based and things like that, and really focus on the real customers on smarter things that we can talk about more around motive, more around understanding what makes them tick, and all those things that are probably more interesting as we try to, you know, influence behavior.

10:21So for us, it's made us more effective at talking to customers. But of course, we're never going to stop talking to customers. It's continue to be a core part of what we do it probably just changes the types of conversations and maybe the frequency of conversations i don't think it's ever going to go away we're always going to talk to real people so don't want to get into the process of how we build real people because that's probably give us an kind of like an 18 an 18 kind of like rating but let's get into the kind of the process of thinking about how do you build the agentic kind of twins you talked about all this kind of like kind of data and the use of ai and and stuff i mean how did that come about i mean is that the where assimilate kind of comes in as they have this kind of like process that you that helps build that how does that how does that all work yeah and you know and june park and his team are great partners and wonderful thinkers in the space and june kind of wrote this original paper on all of this way back when they have the twin engine but just in broad terms how it works we take behavioral data you know that we can that we can glean from you know this would usually be a panel.

11:25So we go and get panels of customers like you would in traditional research. People consent to all this, just to be clear. It's all consent to data. But we'd pull like their behavioral data. So experiential data, observational data, things that they would do on the internet or in more public spaces, social media. We then conduct an AI interview with them. In that AI interview, it's actually, I've taken it. It's a really interesting interview. It's not just like, hey, how do you feel about X? How do you feel about Y? You're presented with actual choices. Like you are presented with like, you have choice A and choose B.

11:57Why, what would you choose and why? And it gets that kind of how you think and how you trade off on a decision. And then it goes into more of like external signals, like media diet, people consent to share, spend data. So you build this rich twin off of all these different sources. So it's combination of survey data, observational data, you know, external signals, and then these interviews to build a profile of a person. And the way I think about it is it builds like their decision profile. Like how do they make a decision? How do they actually think? And then you can take that profile and generalize it to new scenarios.

12:35So it's not just like what the question I always get is how is this different than machine learning? It's not just what they did in the past. Like if it was just what they did in the past, we have tons of models and things like that that do that. it's more about how they think and make trade-offs which is where this is really unique and it allows you to generalize to new scenarios okay new new kind of situations and did you have to inform the models or almost like frame and what was going into the models through using your own sort of personas and sort of like descriptions of different kind of customer in inverted commas types and when you can then ingested all that kind of panel data and maybe supplementary kind of data did the actual kind of modeling throw up other personas that are modified personas that you hadn't thought about because do you know what i mean because the power of some of these kind of models is just like astonishing in many ways to come up with sort of like going oh you've got all these people but you might not have thought about these folks over here's a subset did that happen well so it's what's unique about the approach we're taking here is we're actually simulating individuals so we are creating an individual like an actual person like adrian shrie you know and what what you do is when you take these individuals you then scale it up you do so many of them that you have scale okay so what that's what i think is because there's a lot of all like similar twin capabilities that are like hey we're simulating like a 75 year old or whatever it might be here you're you're actually simulating individuals so it's really mirroring what you would do with the traditional market research study because now I have these huge populations so we you know we've created right around 100 ,000 at this point over 100 ,000 twins that we are able to actually take that large population and to your point we can get and filter it and get to more granular groups but you would basically think of it as having 100 ,000 customers across all of our different groups twin now to your point though there is important context like you know what we would do is we would create a bank of customers for say pharmacy experiences where we ask questions in that interview process that i described that were related more to the pharmacy same thing with insurance and stuff like that because you could just create a general population one and maybe it would work but i think it's important to probably with our estimation we want to get the kind of context that is important to that experience because we're going to ask and we do ask some very specific questions right so we want to make sure that we give them the we get the right context from that individual uh so that's that's kind of the difference in approach so you it's to do your point you do end up with these nice pop niche populations but it's sort of because we've just cast a broad net of our consumer base okay now you do want like what's really awesome about the tool and capability we have some real we have like specialty patients we have like bl groups that are really hard to get to.

15:34So now we can actually twin those groups where traditional research panels, super costly, and you can only get them one time. And then, you know, it's really, that's it. It's hard to find them again. Here we've now twinned them. So we have them for good. So that's one of the biggest advantages here is that we can get to these sort of populations that are more difficult to get to. Oh, okay. And tell me about the impact of this approach. Because you say it's been developing this now for what's it a year or so kind of like kind of now kind of what's been the now that you've got to the point where you're you've kind of built these twins and you're starting to as you say you're starting to kind of use it in your in your decision making what's the impact you've um you've seen on the on the on the on the business yeah so i'd say there there's probably four big areas that it's delivered a ton of value one is speed of course and that's what everyone will kind of latch on to because we're just driven by speed and efficiency but you know take message testing so like add like i run our insights group so i add testing message testing things we want to get in the market traditionally with six weeks we got to get a panel you got to go test it this is now down to minutes and i say in like two to three minutes like we could probably run this thing that fast with the same level of fidelity.

16:52Now, the way we measure fidelity, we compare it to what the actual human studies would say. So there, we're down to a couple of minutes. So that's been unbelievable. And that goes across a lot of different studies, but message testing, just one that pops is something that we do all the time and really can do quickly. Same thing with ad testing, advertising. On the fidelity side, the second would be fidelity. If you think about traditional methods, you end up with a lot of fatigue. you have to actually extrapolate out. So think about like a choice model you would run. You have to, humans can't respond to more than like five choices.

17:27So you give them five choices and, you know, then you have to do that over and over again. And that's why you, people pay, you know, tens of thousands of dollars for conjoint analyses and things. Here, you don't have that problem that you can show these twins as many scenarios as you want, and they will give you that choice. And so we use it a lot for understanding choice models. So for us, it's like, you know, what are the actual drivers of pharmacy choice and understanding kind of what's more latent risk and what's actually a primary choice driver for instance like location and whether or not the insurance is taken there's one of the number one these are one and two drivers of whether someone picks a pharmacy but you can get to those next level choices as well and you know one example that we had in the white paper is i think around adherence so understanding medication adherence we now get to the what are the different parameters and there's no fatigue and there's no what i would say consumers sometimes are a little bit afraid to share what what you know drives adherence you don't have that problem here uh you know you can ask more detailed questions and more questions that kind of get at that to understand the drivers of adherence one of the interesting things i think we found was a lot of it was refill anxiety so we have new patients they were like you know how am i going to actually get this script over and over again and it's one of these things we kind of uncovered that would have been probably a little harder to uncover if we had used just you know normal patients because they may not want to admit that oh yeah i'm a little bit afraid of how do i refill these prescriptions um the third value i think is scale so you know we now don't have to run these small pilots we can run these scenarios and tests on huge groups of people the example there i always talk about is our pet rx product we were going outside out with you know serving pets and one of the you know and one of the things that we investigated were one of the things that are going to resonate with people to say hey cvx can play in the space on pet medication we learned really interesting things about how people feel about pets from the twins one of the things they don't actually they don't actually view giving the medication as an act of a chore They view it as like an act of devotion.

19:37And there's all this messaging we had around appealing to people's love of their pet. But what's really fascinating and came out of the twins was, yeah, that's all important. But what they really want to know is, are you connected to their veterinarian? And much like with a normal pharmacy product, will you be able to get this script to them in an efficient way without them having to do a bunch of work? So it was like, hey, it wasn't enough to just be like, hey, we emotionally understand how you feel about your pet and we get that. You had to get to the point where you're like, but also we are good at this.

20:07We know how to do this. And fulfilling the script isn't that different than filling your human script. The last one, reach. Like I mentioned, we can get to these really hard to reach populations. So people with specialty conditions, various hard to reach conditions, populations and groups, we can now get them and twin them and then we'll have them always on. And that helps us actually understand how do we maintain adherence and how do we make sure that people are getting the communications in those really hard to reach groups that are going to be most effective for them to stay on medication. So I would say speed, fidelity, scale, reach are the four right off the bat where it's had a massive impact for us.

20:47Awesome. I mean, one thing I wanted to ask is that when you were talking then, you talked about like ad testing and messaging and stuff. And so are you going to be able to only test sort of text-based stuff or can you do multimodal sort of things? Could you show the twins kind of like images and that they'll be able to respond to the look and feel of something? Or how's the level of sophistication in terms of, you know, what kind of input can you put in to get a response? Yeah, we show them text, we show them videos, we can show them figmas. So we can show them quite a bit. And we're working towards two things we're working towards with Simile are being able to put them in a more virtual environment so they can actually experience a virtual store.

21:31And then also we're working on agent to agent. So, you know, can we actually, you know, get twins of colleagues and twins of, you know, our patients and have them interact and then see how that interaction could be shaped? You know, and I would say just broadly, it's an emerging technology. So even the four I gave you, we're learning all the time and new and different things. And like, you know, using it in ways that I think are more akin to like a pilot. So the four examples I gave you are really focused on like what more traditional research, but using it more as like, you know, okay, we're going to release this experience or we're going to do this.

22:12How are people not only going to react, but how are they going to behave? Like one of the biggest, I think, unlost of the capability is the say-do gap. So like, you know, even in the example you're given around different media, we can put different things in there to show them like, hey, this is the new experience. This is the new workflow. How would you behave? Is it going to increase your adherence? Is it going to increase the rate you call us? Is it going to, you know, what are the different actions that are going to happen from that? And that to me is where there's just a massive opportunity because it's not just a research tool.

22:47like how do you feel about x y and z which is also really powerful but it's more around what will you actually do and that's where it's kind of becomes a crystal ball if you will the joke we have here the oracle of wound socket is i think what someone we're based out of wound socket but that's that's kind of where i think it ends up nice so you're you're using ai to do this as these simulations i mean and i'm assuming that you're sort of like you're using it more broadly in the business. I mean, how are your customers or patients responding to this issue? Because we see conflicting reports that people go like, you mentioned it, they kind of say, do sort of gap.

23:25People go like, ah, I'm using it every day, but I'm not really sure I trust people that kind of like use it and that are responsible with my data. So it's a bit that sort of thing. So how are you seeing people respond to your customers and patients respond to the use of AI in your sort of daily operations? Yeah, it's kind of interesting because I think what you observe, and we've seen this with the twins, we also do some human research is like, I think people will also always approach things a little apprehensively, but once they get value from it, they tend to lean in a lot. And you see that a lot with like, even, you know, public internet of AI sources, I think you, people are always a little bit skeptical, but then once they, they get really valuable information quickly, then they tend to adopt fast.

24:08And I think for us, the response to things like the white paper and us being super public about doing this has been positive. We've seen it in the media and elsewhere. And I think ultimately, from a consumer's perspective, we're building better experiences. We're able, we don't have to, it's increasing our speed to market because we're able to test with these twins. We're able to understand how they're going to be affected, how they're going to behave. And then when we go live with something, we're more certain it'll work and deliver a better experience. So I think overall, I think that, you know, people have been pretty excited about the use cases here with AI, particularly this one, because it's really designed in the truest sense to make us more consumer centric.

24:52I mean, I know you did that joke earlier, but we're bringing all these consumer voices into everything we do now. You know, a common set of language that you hear internally here at CBS is, did you run that by the twins? You hear it all the time, which I'm pretty proud of. But what that means is, if you want to boil that down, is what do the customers think? And that's like language that's really hard to get internally at a major corporation. And now we have it all on. So I think the level of consumer centricity here will ultimately really positively impact our consumers in a way that we're just going to build better experiences for them.

25:27And do you think that, and one final kind of thing on this specifically, because you talked about being transparent about your approach and how that's kind of been responded. And you talked about this internal cultural sort of dynamic, people are going to have yes to twins yet. That feels like almost like added layers or added sophistication to a governance approach. like transparency and then a cultural kind of change it's almost like it's it means that because people talk about governance right particularly around ai how you can protect and manage outcomes and outputs and things but it feels like by being transparent about things and then almost driving this cultural change you're almost adding layers to that because it's not just about guardrails and policies and stuff it's almost a bit like here's an approach and here's a cultural change Do you feel that's going to drive better governance in terms of how you manage this and the impact that it has?

26:29It's a really good question. I think it's driven demand, and it's probably more of an onus on governance, if that makes sense. I think being so open, public about the capability, what it brings to the table, even internally, we've done that. We've really showcased it internally all the way to the top of the company. and so there's been a lot of demand to use it and i think one of these things with these cape one of the thing with all these capabilities as you know from the cx world is like this concept of democratization because where i think there's a potential real value at here is like you know you democratize the capability and then you kind of can bring you know put all these customers in people's pocket but man is there risk with that too because you know these mirror humans so if You know, just like a human, you can lead the witness and you can kind of if you don't pose things the right way, you can get yourself in trouble.

27:22So we've built a pretty strong governance around it. And I think you get the nail on the head. I think anyone going down this path, you better have governance structures set up. So what we've really done is we've built a validation engine. So we're constantly validating against human responses. we also built a validation engine around drift which is sort of yeah how to compare to the population so these these twins don't drift away and end up giving kind of outlandish results or or something that's a hallucination and then i think the biggest thing that we do is we have a human at the steering wheel so my team really detailed and wonderful researchers are the ones who are actually using the capability and that human at the steering wheel is able to look and say, you know, okay, this one is a good, we have a low confidence on this response.

28:15We should treat it as directional or, Hey, something's going wrong here. We need to go look at the underlying twins. And one of the things we've challenged is basically don't look at it and just immediately say, this is wrong. It's not what I expected because the twins could be giving you something really valuable that you didn't know, but go, if it's something that's really challenging to what you really new and seems very out of character, go check it and go test it and go look at it. So I think that human at the steering wheel component has given us a lot more confidence. And especially when you're rolling something out for the first time, Adrian, as you know, when you roll out any of these capabilities, so much of it's predicated on the change management and trust of the organization.

28:54So what you can have is us go out there in these early days and say the twins said something. And then the consumer behavior when we actually launch it is completely different because that would immediately erode trust and the capability. And, you know, even if that was something we could explain, it would be difficult to kind of get that trust back. So building that governance structure, the validation of the humans at the steering wheel is so essential for this being effective that we're taking that approach. And then we're figuring out how do we give access to the broader population within CVS Health, but in a way that's controlled, that's making sure that we have the right guardrails in terms of confidence of the response.

29:36And that's a journey for us, but we're working on it. Absolutely. I mean, so you said that that would be the big piece of advice is getting the governments right. I mean, what other sort of advice would you have for someone? So say that they were thinking about doing something similar to CVS Health, they sound like my company or some of the complexities that my company has and they're the groups of different customers that we serve. And that sounds like it would be really useful for us. I mean, if they wanted to, say, replicate your approach, what sort of advice would you give them? And where do you think this sort of approach would be applicable, do you think?

30:16I mean, I think it's probably in a complex organization that serves a number of different kind of customer groups, but you might have different thoughts. Yeah, I mean, I'll start with the second question and I'll get back to advice. But I think the applicability here is broad. I mean, it's anywhere where you want to understand how your customers are going to respond, behave or react to a product or a service or something you're launching, a communication. so you can really build these ai twins of of pretty much anyone as long as you can get that core data that i talked about so get them for an interview get them for all those different components so i think that's where probably more i would get i would say more on the consumer side we've had a lot more success there but anything like a large consumer business where you're where you're looking to test and change things, I think it has value.

31:10And it's been super valuable for us. I think one of the biggest ones for me that it's unlocked is really product and service development. So bringing customers way earlier in the process, because where you traditionally have this is you kind of get a little bit down the path and then someone's like, okay, can you go run this test with consumers? But at that point, the product's already kind of been baked. And then you have things like some cost bias and it's difficult to kind of backtrack. Now you can bring them way more, way further up in the process. So anything where you're developing a product or service, I think has tremendous value.

31:42To your point on advice, I would say we already hit governance. The other thing I think is that there's a change management with your own team that I think is really important. And you kind of hit on it earlier where there's, everyone kind of goes through this existential crisis with AI. And this capability in particular is really a challenge to the way traditionally things are done. You face a little bit of that apprehension from your own team. And I think one of the ways we've overcome that, though, is just showing the amount of value and impact you can have to the organization and how bringing this capability forward gives you just a bigger seat at the table.

32:22Because if you think about it, most of these traditional research approaches, those four things I hit on are really challenging speed fidelity scale and reach and those things are difficult to to solve and so that honestly limits the impact that many traditional researchers can have now that you've solved those things you're gonna have a bigger seat at the table your work is gonna be more impactful and more powerful and we've seen that at CVS and you're gonna be able to serve and be more productive in a way that you weren't before like we can now you know if you think about it these twins are also available and they're not kind of costless to us so in terms of you know we pay for the the service but now we can use them over and over again that's there you know we we can now reach groups that traditionally didn't have the budget for this stuff right they weren't they wanted to run a consumer study they couldn't afford it so i think getting it into those getting your team to buy in is really important uh because again you know They're the ones that are going to be at the steering wheel and driving it.

33:26When you can get them involved, not only does it increase the impact of the organization, you also get a better product. My team is constantly going back and working with the Simile team and working with others to make sure that this is the best capability we can have. You can't forget that employee engagement line, I guess, that we always talk about. You have to have your own employees engaged, the CX professionals, the Insights professionals. Because if they're hesitant and detractors, you won't get off the ground. So governance and getting your team engaged are two of the biggest lessons. Awesome.

34:02So that's about it for my big questions around kind of what your New York can approach. I mean, unless there's anything that you've missed out that you want to highlight before I ask you some quick fire stuff. Yeah, sure. I would just end on the twins just be saying like, this is the start. I think the way the line we always use internally is like, we're, we got to be where the ball goes or, you know, that old Wayne Gretzky and go where the puck is. Hey, don't go where the puck is, go where it'll be. So I think this is just the start of this capability and where it could be. And I'm excited that we're at the frontier of it, but we're still learning and getting better at it.

34:40And I do think the implications of this simulation and using the simulation is going to be incredible. And I think you're going to see it. and I know you're a CX leader, a thought leader. I think it's going to kind of take that approach that you saw in the early 2010s with operational CX. And I think you're going to see kind of this evolve into operational simulation. And I'm really excited for that journey and seeing where this all develops. Because I think history is going to repeat itself when you saw all those, you know, the medallias and the Qualtrics and all those folks emerge. you're going to see something similar here with the simulation capability in my in my opinion all right you know i can imagine when you know i've seen all these kind of like developing capabilities with with kind of people talking about it well you know everybody's trying to own the or the experience orchestration kind of layer everybody's talking about it but then let's say they've got somebody who's going to you know let's see you're you're like a marketer or something and you're spinning up a an outreach campaign whether that's for acquisition or for loyalty or kind of whatever it might be and you're using an agentic platform to you create a brief and you kind of spun up some ideas and you've interrogated your audiences and all these different things and you come up you've drawn on your your banks of you know almost approved kind of content and all these different sort of things and it's got to come up with these different kind of ideas and try to optimize it for channels and stuff and it's somewhere along that kind of process you can imagine that you can have the twins kind of plugged in and go like does does this make sense?

36:13Yeah, no, exactly. Right. I think that that's where it's going to be, it's going to fit in to so many other things that we're looking at and doing. And like, I just think it's creating this, it's amplifying the consumer. I go back to this line, but it amplifies the consumer voice in a way I've never seen. And I mean, I've been at customer experience and insights, you know, for over a decade. And I would say this to me is the most transformative technology. have observed. Nice. He says dropping the mic, moving on. Let me finish off with some quick fire questions just to wrap things up. First of all, best advice question.

36:55If I were to ask you to boil it all down, Sri, to one piece of advice, ask people to complete a sentence in order to make this easier. The sentence is this. If you want to improve your customer experience, Sri says do this. Complete that sentence. As you could probably tell, I get too excited and all my answers are really long-winded. But I would say the biggest thing to me is to shift your mindset, just broadly, not even with the twins. Shift your mindset from being like, what is the value I can extract from my customers to what is the value you can add to your customer? I think we too often as companies think about, okay, how do we change consumer behavior?

37:38How do we shift the value equation so we get more value? I think it's way simpler than that. If you add value to a consumer and you add value to someone's life, they're going to be loyal. They're going to spend more. They're going to come back more. And I think we need to flip that equation. So it's really, you know, in this order, it's basically value for the customer and then value for the business. And I think that's the biggest advice I can give someone. I would just add one more. I think, I know you asked for one, but two. I think don't get single threaded through one signal. I think in this day and age, even in this conversation we've had, there's so much data and capability out in the market that just drives your knowledge of consumers.

Read the full transcript

38:23And I think it's never been like this before. And I think each of these data sources tells you something interesting, right? Like, you know, the digital session mining, call center mining, that tells you what happened. The always on MPS is really like, what is the behavior of those 200 operational variables? The surveys give you the kind of motive or what are people, why they're behaving in a certain way. You have to, and the agents, the general agents, what will happen is what the simulation tells you. So you can stitch all this together and it creates this profile that you just never had before.

38:55So don't, you know, for those that are in the more of the measurement side, don't get single threaded. You have to think about how you tie all these and stitch all these things together. Ruffins, great advice. So one final thing before we wrap up. should you tell me a good news story tell me the something that is the most interesting positive exciting thing that you've seen in the last week and i think you're going to tell me at the time of recording we're in the midst of the world cup but that's fine so i think you're going to tell me something which might not be well it's time related but it's a good thing anyway yeah well i will say the game last night we recorded this after the u.s lost to belgium that part was not good that was that was for me to watch but i think it's been cool to see just how and we were talking about this right before we joined but how the world has kind of come together you got to forget you know every four years that this happens and then the four years you kind of forget how soccer and sport bring the world together and just observing all these different countries and different u.s cities has been really great because it's been awesome to see all the stuff on twitter or on Instagram and all this stuff where people are talking about how they're experiencing the U.S.

40:07and how they love it and how they love the people and how the people love them. And I know you were mentioning you're from Scotland and the Scots kind of taking over Boston and Boston embracing them. That was such a cool moment to see. And you just see that over and over again in all these different parts of the country. So it's really, you know, been warming my heart because, you know, there's a lot of noise and a lot of, you know, conflict in the world. And this kind of reminds you that we're all just people. We're all just people who are trying to live our best lives and watching everyone together do it together and kind of have this harmony has really given me hope for the world again.

40:50So it's been awesome to see. And the beautiful game really makes it a beautiful world. Perfect. I mean, I love that. I love that idea. I love that sentiment. I love that reflection. And because it's a message of hope, right? When people left to their own devices and that are passionate about something and they get together and they can celebrate things. And it's the broadest sense is that actually broadly people are all right left to their own devices. That's a great mic drop moment. people are all right that's what i really do i think that's what we're learning people are people are really good together excellent listen tree um congratulations on the on the work that you're doing with the uh with the with simile and the the simulations i think that's super super interesting and like you say i think we're we're only just starting down that road to see where the impact kind of goes and how it all kind of like stitches together and so that's that that's brilliant so thank you for sharing your time and your insights and your expertise kind of with me today that's been super cool yeah thank you i appreciate the opportunity it's been great talking with you and i love the podcast so i'm i'm honored to be on so thank you thank you again for the time you're most welcome thank you wow what a great interview i hope you enjoyed it i know i did find out more about me and the work that i do at adrian swinsco.com do leave a review on your favorite podcast platform.

42:18And if you have any comments, feedback or questions about the podcast, then feel free to send me a message to podcast at adrianswinsco.com and do tune in again. Thanks very much.

From the publisher

Today’s episode of the Punk CX podcast features a chat I had with Sri Narasimhan, the VP of Enterprise Customer Experience and Insights at CVS Health, where he leads efforts to drive meaningful organisational changes based on customer, client, and patient feedback.

We talk about a whitepaper that he recently co-authored with Simile CEO Joon Park on using generative agent simulations in health care, the problem they are solving with this new approach, whether that means they will no longer be doing research with ‘real’ customers, how they have built their AI “agentic twins”, the impact of this approach and how their customers (patients) are responding to their use of AI.

This interview follows on from my recent interview – Start with Strategy, Iterate Like a Band – Interview with Craig Crisler of SupportNinja – and is number 597 in the series of interviews with authors and business leaders who are doing great things, providing valuable insights, helping businesses innovate and delivering great service and experience to both their customers and their employees.

More from Punk CX: Customer Experience Insights with Adrian Swinscoe

All 58 episodes
Using generative agent simulations to drive better experience outcomes in healthcare - Interview with Sri Narasimhan of CVS HealthPunk CX: Customer Experience Insights with Adrian Swinscoe · 43 min
Listen in VO