Generative AI: Augmenting or Replacing Research?

0:00Season three of the Built Right Podcast is right around the corner, but we've got one big change coming your way. The Built Right Podcast is now the Talking AI Podcast, and we've got a lot to talk about in AI. In the Talking AI Podcast, we'll be having in-depth conversations with both AI experts and early adopters of AI. That way you can understand how the technology works and how early adopters are beginning to implement and, more importantly, get value from AI. Our guests range from AI research scientists to founders of AI products to industry leaders putting AI to work in their business. While you're waiting for season three, go ahead and subscribe on your favorite podcast platform so you don't miss an episode.

0:41And make sure to leave us a comment about the AI topics that you want to hear about. So get ready to talk some AI in the new Talking AI Podcast, coming your way August 6th.

0:58Welcome to Built Right, a podcast by Hatchworks where we help you learn to build the right digital product the right way. In each episode, we'll deconstruct the layers of successful product development, break down popular trends, and offer real advice to help make sure your product is Built Right. We may not have all the answers, but we've built a lot of digital products across a lot of industries, and we've seen a thing or two. Let's get into it.

1:33Welcome, Built Right listeners. Today, we're chatting with Nisha Iyer, CTO of Conote. Conote makes qualitative research fast and easy with its AI-empowered platform, helping transcribe, analyze, and organize user interviews. It's a tool built for user researchers, which at Hatchworks, that's a big part of what we do. So we can definitely sympathize with that type of tool. But welcome to the show, Nisha. Thanks. Great to be here. Yeah, excited to have you on. And today we're going to get into how generative AI, and more broadly, just the democratization of AI, will fundamentally change user research and more broadly, user experience.

2:15But let's start off there. Nisha, why user research? Why this problem? What part of user research is broken or needs help? And how's CoNote looking to solve it? What gap do you see in the market right now? Yeah, great question. So just real quick intro. My background is data science. I've been in the industry for about a little over 10 years. And my last company, I was working at a startup. I've been there for five years and had built a tech team and had come to a point where we're doing product development. So with product development comes user research, right? Like to build a good product, you need to understand your users.

2:58That's how you get to product market fit. That is how you really build what people are asking for versus what you're building in your own head. So we did a lot of user research there. And I worked directly with a small group that did the product development. One person was a UX designer and then an engineer and a data scientist and myself. And we did a bunch of user interviews and went through the process of distilling them and really pulling out insights. And it was tedious. It took a long time. It took a lot more time than I had expected, you know, just from my technical background. And I was pretty overwhelmed with like the amount of information that we had to consume.

3:45Like, you know, you do the interviews first, record the interviews, transcribe them. And by the time you sit down to really distill what's what has been said, like the important themes, the important takeaways, You have to pretty much go through the interviews again and go through every transcription, you know, like the basic affinity mapping technique where you're taking post-its and grouping themes. And it just takes a long time. Like it took, you know, a week to two weeks because you don't have like that set aside time to just dedicate to the distilling of research. And so what I found myself doing with my little team was just taking shortcuts, like being like, okay, like I remember this, I remember that and being like, and then like internally thinking like this isn't the right way to do this.

4:31Like I'm 100 % biasing the, you know, I'm biasing my findings by thinking like hearing the things that I really wanted to hear, obviously, like that's just human nature. So what actually happened is that I had a project come up where there was like some kind of commitment to do 20 interviews in a period of two weeks and then distill the research. And I was like, this is insane. Like from my experience with research, I was like, this is a crazy requirement. And I and I thought like there must be some tools, like there must be some AI platform that does this. Like we, you know, we're at the age where this should be available.

5:12So I started Googling for it and I couldn't find anything. I was like, this is insane. So I called my friend at the time, my coworker, and now my co-founder, one of my two co-founders. And I was like, hey, dude, like we should build this. Like we can do it together. Called my third co-founder and we all talked about it and all agreed that it was a huge pain point of, you know, not being able to synthesize research in a speedy amount of time. And then also like just that, that unbiased synthesis, right? So that's how this came about. Honestly, it's just from personal pain points, which I think is a great way to build a product because you've actually experienced it.

5:54And you're like, I want to use this to solve my problems. Yeah. Quick break in the pod. If you're listening to this podcast, chances are you've been thinking about how to actually use AI inside your business. And that's exactly why we built the AI Opportunity Finder. It's a free tool that helps you uncover high impact, tailored AI use cases based on your business, your goals, your pain points, and your industry. No fluff, no generic use cases, just real ideas that fit your business and the ranked by ROI potential. It takes about three minutes to run, and it's like having your own personal AI strategist for free.

6:27If you want to try it for free. Check out the link in the show notes or go to hatchworks.com backslash AI dash opportunity dash finder. That's a great explanation. And you're bringing me back to my product days where we would do user research interviews and I would always schedule like an hour after the user interview to like debrief, go through it again. And it's like, you know, that's a two hour block there. And then to your point, you got to synthesize the whole thing. You forget stuff. You mentioned bias, but there's also recency bias where I'm going to remember the most recent interview more so than the other one.

7:03And then you have like for us, we would have these Miro boards where it's just huge with all these insights. And it's like you're trying to connect the dots. It's it can get messy. So like I can I can feel that pain. It's it's bringing back some some memories from those days. Yeah, exactly. A hundred percent. It's just like, how do we and then and so like just to continue on, it was just like, like this journey has been quite serendipitous. Honestly, I ran into my upstairs neighbor and she now also works for Kono and are with us. And she was a user researcher and I told her the idea. And she was like, oh my God, like this is going to make my job so much easier.

7:45Right. And I, and like, I like I'll stop there, but I just want to like touch on that as well, because it's not like, oh my God, it's going to take over my job. It was more like, this is going to make my job so much easier. Yeah. And I love the point too, like you're hitting on the pain points of the speed element, but there's also the quality piece for the bias. So there's some core like value points you're starting to hit on. But I was, I was digging through your LinkedIn and your, your CEO, James, and I'll mispronounce his last name. Frasier. Frasier. Okay. But he had this like interesting quote that was out there, a survey by user interviews and said, UX researchers were the most tentative of all roles to use AI with 44 % saying they've tried to use AI, but avoid it in their research.

8:34But like by comparison, CX professionals at 5%, data scientists, 9%, product managers, 21%. What do you think is the reason behind that? Why are you user researchers in particular, like less likely to adopt this technology that could potentially make things easier for them? I mean, honestly, I think it all boils down to like, fear of the unknown. If you look at like, if it 9%, right, data scientists are 9%. Like we, most data scientists understand exactly what's going on at the bottom level, right? Like it's we're, it's mathematical, there's no like magic, there's a lot of inference on based on similar words and, and words transformed into number numeric representations.

9:24And that's where like, it all stems from. So I think like, the number one thing is fear of the unknown. And, and then it just goes into like, I don't want this to take away my job, like, it's not do it. And then like, so then I feel like I would get on the defensive of saying, like, AI cannot do my job the way I'm like, better than me, it's not going to replace me. So I don't trust it. I think instead, like, where we could go with this is AI is augmenting my job. Like I can actually focus on the important pieces versus like the tedious nature of things that I could actually like bring to the forefront using a tool that does what I would be doing over a week or two weeks in a matter of minutes, right?

10:07And then I can spend the time taking those insights and making more inferences and pulling more information out of it, I can also speed my research cycles up. So I think that like that, that fear, like we've heard it, you know, we do our own user research with Conote. And I think it's just like, what's going on under there? Like it's a black box. And, and I think that like the way that I would talk to people who had that fear is that it's not a black box. It's just, it's like something that I can, I can help explain and walk through. I think that would just get boring though, because it's super technical, but it is all, you know, it is all related to similarities and semantic understanding.

10:52And AI is also not here to take your job. Like I will end with that. Yeah. And that's an interesting theme we've had across several episodes we've done lately is there is that fear of the unknown, that fear that it's going to take my job, that It's going to replace me. But this idea of a co-pilot, it's enhancing my skills. It's making me better is a theme we've continued to hear. I was chatting the other day with, and I'm trying to find the episode. It was Brennan at, I think maybe that's episode 15. We'll see where it is, what it launches, but it's hyper context. They're solving this tool for one-on-ones and performance reviews.

11:34And they had the same kind of idea with HR was like, you know, AI cannot solve this for me. There's no way. But what was interesting is like with the latest, you know, just craze over the past year, you know, chat GPT and all that kind of stuff. they were able to play around with it and get a sense of how things can work. And it kind of opened up their minds a bit. I don't know. Has some of that happened with user researchers as of late, where we've had this crazy hype cycle with generative AI, where people see some of the power with it? Because I think with user research, it's so qualitative.

12:06I think that's one of the big hiccups there as well. It's like, you know, this is qualitative stuff. It's not ones and zeros. But with generative AI, it adds that kind of semantic piece to it, to your point. Yeah. Yeah, no, I think that there is a growing acceptance and people want to use this. When they start seeing the way that generative AI can augment their research versus take over, I think people are more accepting. Like, I think we just actually spoke to someone recently that's getting on the platform at a large corporation, and they were a little skeptical at first. And then we introduced Conode as it gets you 85 % of the way, right?

12:51It's not doing all the research. It's just getting you to a point, jumping off point, where then you can take those findings and build your own insights. And that helped her feel better. She's like, oh, okay. So it's not like just giving me this output. It's more so like giving me stepping stones to get to that output. And I think when put like that, researchers seem to be more open to using the tool or using products like this, like any kind of generative AI products. You know, there are a couple out there in the market that seem to be getting some kind of traction. I can talk to those later.

13:28But like, I do think that like, it's like, and still in like the early adopter phase, right? People are still weary and we have to show people at Conote the reasons why they should be using it. And I think what we're doing for that is building a lot of user education, showing people how they can use the tool to augment their research and giving examples within Conote of how you can do that. Yeah, it's an interesting kind of product. And then getting into the marketing problem where they may be problem aware, but not really solution aware and trying to migrate them down that path. Let's get into like kind of short term evolution about AI can impact user research versus like longer term.

14:12And in the short term, I'd be curious, and this may be even be like functionality within Codenote or stuff on y 'all's roadmap. like what's the short-term view of how generative AI or AI in general is helping the user research process I mean is it simply just churning through this long interview and it's spitting out the insights like where do you see it today and then like what's like the crazy like you know utopian future of what it could be in the future so right now the power of conote lies within you know like we are actually moving pretty fast. We released our initial beta live July 18th, and we've already had a couple of releases since.

14:58The big, powerful generative AI piece right now, or so like, I just wanted to take a step back. Like we, I don't think Kono is 100 % generative AI. We have layered models. We do use traditional machine learning layered with a couple of like large language models. And I think to that extent, like there's already like power there. And that's why we call it an AI engine versus like just like Gen AI, right? And what we're doing, like the big powerful piece right now is that you can upload hours of research. So multiple interviews, and then you can synthesize. So not only can you synthesize, you can transcribe with the interviews and see the transcriptions, see the diarization by speaker, and then highlight key quotes.

15:42You can then synthesize your interviews. And then under 10 minutes, you will get the key themes and then the keywords within each theme. And those keywords directly relate to sentences within the transcripts. So let's say like I get four themes. I can click into those. I can then see where each speaker, like if I had five interviews, I can see where each of those speakers said that mentioned that theme. I can then click into detailed view where I can actually hear the speaker saying it so I can get sentiment. And I can also, you know, bookmark this and build a presentation that I can send out to a stakeholder that may be interested in like some of the key quotes that were said over, you know, like eight hours of interviews, which is usually would take, as you know, usually would take so much more time.

16:33So, yeah, I'll stop there. That is our current like big bang of our AI engine. And we definitely have some other plans ahead, but just wanted to stop for any questions. Yeah, it's an interesting point, too. You mentioned like generative AI is the hyped up word right now. But like machine learning and you as a data scientist knows like there's some of this stuff's been around for a very long time. This is not necessarily a new thing, right? And there's so much power just in like machine learning and a lot of the things there as well. And I'm curious too, you know, it seems like every day there's another Gen AI, you know, product coming out there.

17:20Like, how do you see differentiating when, you know, I feel like a lot of this, the tools with AI have been a bit democratized to where people have access to these large language models. You kind of mentioned that's not the only core point to your tool, but how do you build a moat when it's so much easier now to integrate some of this technology into a tool? How do you all think about that? I think we have to really think about the user, right? Like, sure, everyone can access these APIs and build and integrate them into their product. Are they actually thinking about the UI and the UX? Like, that is a key piece of co-note.

18:00Like you want to have, and as you know, like also being in product, like you want to have a really intuitive like journey when you get to an app, right? So you could integrate an API and build like all the tech and be amazing and stacked and everything. And if you're building a SaaS product and don't have like an intuitive front end, people are just going to stop there. Like they're not going to know how to use, get from point A to point B. And what Cono, so I have James Friscia as the CEO and co-founder, and then my third co-founder is Cameron Ridenor, and he's chief design officer. And so his background is UX, right?

18:39And so not only, like we live and breathe these problems, we get in touch with people that live and breathe these problems. We have people that also work with Cono that do. And I think that our moat is, A, that we're not just simply consuming APIs, right? We have other pieces of infrastructure around them on the AI side that actually enhance and empower us to be a little ahead of, not a little, a lot ahead of some of these gen AI companies that are just simply consuming and using prompts for some of these APIs. And then secondly, like just the fact that we have such a deep understanding of the user and are focusing on that when building our product, right?

19:25Like the experience, the interaction with the app. And if you're listening to this and are curious, like please go check out Kono.ai. It is live and free. And Matt, I'm not sure if you've checked it out, but when you're on the application, compared to many other competitors that we've checked out and tried out, there is a very simplistic flow to get to the pain point that we're solving for, which is really being able to speed up your research process. Yeah, I think part of the benefit there is you're very focused in on a particular type of user, which is user researchers, right? I think so many folks, and we see this with a lot of clients too, they're trying to serve too many different people.

20:11And then you get back to user experience. How can you build, you know, not simple, but just intuitive user experience when you're trying to serve different groups? Do you have even within user researchers a persona within that that you're targeting? Or is it user research is kind of the core? Like, is there a type of user research you're even more granularly focused in on? Maybe not a type of user research per se, but definitely a type of user researcher that is, you know, interested in synthesizing multiple interviews and has a research cycle they can't keep up with. or potentially, you know, like where I'm trying, where we're trying to drive people is the fact that research is more important than people give it.

20:57Like it takes so much time. The research cycles are longer than development cycles, right? Like I, like if I'm thinking about dev, I think of CICD and DevOps in CICD and just in general agile principles, a sprint is two weeks. There is in no way that like researchers think they can finish a research cycle in two weeks. However, with Kono, you could do a week of interviews, and then synthesize and be done and ready with new, with new findings for the next sprint. And I think that is a missing piece in the entire like end to end process. Like I have tirelessly worked with development teams and like lead engineering and data science teams.

21:37And the missing gap is that they don't get the user, they don't understand thoroughly like the user research part right like they it's like a game of telephone like 10 people have spoken to each other before the engineering team hears what they need to build and they can get so in like uh you know like just in deep in the rabbit hole of like hey this is how we're going to do it technically and not be thinking of like the actual user problem and that's where i really want to like you that's where kono comes in right kono gives you the ability to add continuous research to CICD. So like in my, in my mind, it should be CRCICD.

22:18Like that should be instilled in the development process. No, I love that. And you're speaking our language. When we talk about built right, we talk about building the right digital solution the right way and building the right one. It's a key element. It's user research. And I love the concept you're talking about where it has to be continuous. And this is what we preach as well. Like so, So much of the time, it's like, all right, let's go do our research. All right, discovery is done. Let's go build the thing. But it has to be built into the process. So I love that idea of it's, you know, you think of CICD, same type of thing.

22:52You need that feedback loop built in as you evolve the product. It seems like y 'all are kind of, you know, dogfooding it a bit by using the product yourself. I'm curious, how much of the roadmap are y 'all defining as you use the product versus feedback from customers? We try to definitely take more of the customer's feedback just because they're using it as their customers. But I do have to say, I listened to a podcast recently and I started to listen to it. And I was like, let me just put this through Kono and see what happens. And it just was able to distill the key points. so fast. And going back to like the roadmap ahead, you were asking about, in our next release, actually, October 13, there's a really cool feature coming out that's called action items.

23:44So now not only do you get themes like that have been synthesized during the process, but you actually get the items that to action on, right? So like, this is what your users have talked about. These are the actions to take. That came from us using it and from feedback, Like, I think like, I wouldn't say 50, 50, I'd want to say like more like 70 users, 30 us if we had to put a ratio to it. But I think we end up all seeing like, the great thing is we, I think we all end up like coming up with very similar pain points. And one of the main pain points we heard is like, this is great, but it doesn't give me items to like where I need to go next.

24:21So, so I ran the initial, like the first round of Kono interviews we did before user interviews we did before we had started building our product, right? We had just like a prototype. I ran those interviews through on dev through the action items feature, like with that to see what the action items were. And it actually gave me action items that were the features that we ended up building, which is crazy, right? Like it told us like users want audio recordings, Users want ability to like integrate with Zoom and Google Meets. Like, so I think that's, that's like, I kind of got off on a tangent, but that's what happens when I get excited.

25:08I think that's something that we've heard from users and from, we've also experienced that we're really excited about. And then, yeah, like, I think it's cool that we get to use it as we do our process as well, because it definitely makes us realize like what is like, you know, like sometimes you can just be drinking from the fire hose. Like you think of really cool ideas, but we use it and we're like, this is annoying. We need to change it. Like we spot the little things too. So yeah, it's a good mix. Yeah, that's interesting. And, you know, getting into the, let's get into like the future state, like, you know, way in the future, you know, where do you see the practice of user research going if things continue to evolve where AI is continuing to evolve?

25:52Is there a future where it's not even a real person doing the interview? And do I, at some point in the future, have an agent or a bot that's collabing with somebody else and performing this research? Do you ever see a future where it looks like that? Where does your mind go when it starts to wander of where things could be way in the future? I mean, I don't know about like, yeah, like great point. And I think people like wonder about that. But like, for me, I think there's like a degree to personal interaction. Like if you're a bot interviewing me right now, I feel like, sure, maybe like in some years there will be AI that's able to replicate each of us very well.

26:35But I do think like that human to human interaction is important in being able to, you know, like what 94 % of cues are like communication is nonverbal right like I think there's a lot to uh process that's outside of just like a conversation that I'm sure AI will be able to replicate but I don't know if I'm like yes like we want to make everything computer like like you know that like in the age of AI and just take away the human element I think more so like the way I I see Cono evolving in the future is being able to scale across, right? Like not like becoming so focused on like automating the entire user research process, but being able to scale to all types of research.

27:19So like to be able to reach product market fit and to really understand our target audience, we want to focus on user researchers right now. But to be able to like scale, I think where we go is just redefining all types of research, right? Like, how do we help in the political space? How do we help in academia? How do we go into like specific types of research? And I think that's where I see Kona moving. Like, that's where we're going in the future and not like, I don't think I don't see us adding a component where like, we're gonna build in AI bots that like, interview people or, and so like, once again, like, that's, that's why I feel like we're not taking away anything.