The ultimate guide to product operations | Melissa Perri and Denise Tilles

23:38And they're like, I see them just like not working on strategy. I'm like, why are you not working on strategy? And they're like, I don't have time to do that. Right? I mean, here like, like stopping all these fires. I'm figuring out like what template I should put my road map in. Like I said, it's not hard. They know what template it should be in. But think about if you now can delegate that to somebody else and say, I want it in this template. Go roll it out, right? Like go roll it out, go train everybody on it, find the right software for it, do it, right? It frees those people up. The leaders to go work on strategy.

24:10And like, that's why you're spending so much money for a product leader anyway. That's why you spend so much money for a product manager anyway. So to me, it's these things are not impossible to do. It's just like, do we have the right people doing them? For pms that are trying to figure out what remains on their plate. What are the kind of the pieces that a pms should keep and not offload on a product ops person? They don't want to offload decision rights. You should never be like outsourcing your decision making to a product ops person. That's not the point. Like they're, they're the product manager for the product managers.

24:43That's how I think about it. Right? So their decisions should be around how do I operationalize great product management here? But as a product manager, you shouldn't be delegating to them to make any decisions about their product. And that's not their job. And if they are coming to you and saying like, you should build this, like, that's not their job either. Right? That's not that's not the right thing. So making the actual decision and then putting that decision into play with your teams. Right? That's the type of work that you want to hold on to for a product manager. So while a product ops person can help you point you in the right direction.

25:17Let's say to find customers, maybe even help you get in touch with the customers, like send out the email to invite them to a meeting or operationalize that. Hopefully, like, that's automated. I would love a product ops person to automate that type of stuff. Right? You are going to be doing the user research. The product ops person you can go to and maybe say, Hey, this is a type of problems that we're understanding. And maybe they help you find other pieces of information around the organization on those topics and bring it to you. But they're not going to be the ones like reading through all this information and being like, you should build this.

25:49Right? They're also not going to be the ones who are like, you say, Hey, I made this decision about what to build told the team about it. The team like got together. Can you go monitor the developers and make sure they're building it on time? Like, no, they're not project managers. They're not the people who are going to be on top of your developers, like watching them build things, making sure that everything's out on time. That's not their role either. They're not going to handle hard stakeholder conversations about trade ops for you. Like all of those things that you are going to want to keep ownership of.

26:19Because at the end of the day, as a product manager, your job is to produce outcomes. And you do need to make sure that you're monitoring those outcomes and moving towards them. You don't want to offload their responsibility. So I took a couple notes of just things that the PM continues to own. What are in quotes, you know, no one owns anything. Strategy, vision, advertising, resource allocation, trade -off decisions, hard conversations around trade -offs and stakeholder input and things like that. Is there anything else in that kind of bucket of just like stuff of PM? A product manager continues to be responsible for.

26:55Well, there's a big piece you're missing. A good market. And that's where products can really make a difference in terms of, you know, connecting those teams, but also being, you know, sort of the first point of contact for sales, first point of contact for product and enabling just some efficiencies. You know, we want to call it process or method, but just trying to help break down the silos. A company I'm working with right now, there's big challenge with that. And it's a large enterprise level company. So how do we break down all of the different silos between departments and your team leads and whatnot.

27:31So it's really talking about like, here's how we're going to do it. Here's some sort of simple templates. We will use it training sales, training product and then getting that into play. And doing the book and doing our research, that was a sort of consistent story point from a lot of people that we interview that go to market with a really big pain point. So I think to what Denise was saying, like is product apps will take on a lot of the coordination of those types of things and make them standardize templates. As a product manager, like your role in informing what the go to market is doesn't change, right?

28:03Like you're still putting the inputs in there. So maybe product ops will provide, for example, like templates and help say like, Hey, here's some different pieces that we need to make a go to market plan. For example, here's a template. Here's what I gather from people, but like you as a product manager, still have to fill out your parts. You still have to go talk to the sales people. You still have to like work with the marketing people to make sure it's positioned correctly. But the product ops person can help with the kind of like program management around it. Of getting those people together, having consistent templates, creating the cadence for when you review those things, making sure that they all align and they're all in one place.

28:41So you're not going to like find them everywhere. And then making sure that like go to market process is consistent across the organization. So that it's not like, you know, as Denise said, it's not like everybody's reinventing the wheel and somebody ought to go to market team has to figure out how to go to market differently with a different product team. Awesome. That's a really handy and important addition. You mentioned project management. I imagine many people think product ops also takes on project management. What's the good way to think about project management versus product ops? There's also program management.

29:11Do you have a kind of a way of thinking about those roles? The way I like to think about it is however, whatever they're doing in terms of the three tellers product ops is really thinking about. As I mentioned earlier, increasing the speed and quality decision making and all of the pillars sort of play into that program manager. I think is really thinking about larger company initiatives and, you know, the duration of their work is ongoing versus project manager. They're responsible for a certain project that's, you know, had a time box and an end date, but it does get a little withy for sure.

29:43I want to go back to the three pillars real quick and go one level deeper to help people understand what they actually entail. What are the jobs of a product ops person? So just we could even keep this sort of brief, but just let's say with business data insights. What are the functions and jobs and things that a product ops person would be doing for your company? You know, a lot of companies will have a data science team or business, you know, intelligence team and you don't have to reinvent the wheel. You don't have to have your sort of internal product ops type of intelligence team. But it's about connecting those and making sure you're putting that sort of with a product lens and especially say with finance too.

30:19You don't want all the product manager sort of hitting up the CFO for that month's revenue or having questions. So it's about accessing all of these inputs and putting it, you know, putting it through a product lens and then making sure that product managers actually know what to do with it can action it. So it's essentially running queries for you generating charts and graphs and recommendations based on what the data is telling you. Basically, it's doing all the data stuff that a PM what would you be doing? Is there anything else there? Also helping the leaders too, right? So we're talking about it from a product management perspective, but I think this becomes this one especially becomes really critical for leaders and executives, right?

31:01A lot of times in like I sit through board meetings where we we talk about AR. We talk about retention. We talk about net retention. We talk about all of these business metrics, right? And they're great for monitoring the health of our business, but where product ops comes into play and business data and insights is how do we monitor the health of our product? So like as a chief product officer, AR is interesting to me, but it's not as interesting to me as AR by customer segment, right? It's not as interesting as AR by product line. It's not as interesting if I take that and then look at it by retention or adoption by product or feature set or adoption by customer segment by products, right?

31:42And when we start to put those lenses on it, they now become a really powerful strategy insight. So what I think these people do and do really well is even at the executive level, they help you, you as a product leader or like, I need these types of insights, right? This helps me. This is how our business runs and this is how I want to set the strategy. These people make those dashboards, make those repeatable insights for you. And then ideally, they know the data so well, they know your business so well that they can surface up other opportunities for you as well to look at data that you didn't know was there or can find these interesting trends.

32:18So they are very much like the people digging into the data. They're just doing it with more of a product lens instead of like Denise said an overall company metric lens. And it's not just in service of the product managers and I just wanted to like point that out because I think this becomes so powerful for leaders because if leaders don't understand those types of metrics, they can actually monitor their strategies. They can't go back and say like, oh, we decided to go up market into the enterprise. Look, we've got enterprise revenue. But are you actually looking at where the enterprises are like adopting different product lines?

32:53Right. Are you looking at how the enterprise uses different features? And if what if certain enterprises are using, you know, these types of features are they less likely to turn? Right. It's those types of insights that I think are really important on this lens. As a PM hearing that somebody else will be doing this feels really weird. Like this feels like such a core job of a PM is to spend a lot of time with data, try to find opportunities, try to find things going well, not going well. But I think the message again is not you shouldn't be doing that as a PM and it's not bad. If you are really good at it, it's probably just don't have a lot of time to do it well.

33:29If you can find someone that's really good at this and has time that doesn't have all the other thousands of things in their plate, you'll actually end up finding more interesting. And you're probably missing a bunch of stuff because you don't actually all the time. Is that generally the way you think about it? Yeah, and I think too, like the business and data insights people, they're not going to be experts on product, right? Like they're almost always not like we had a couple analysts at product slabs and they were, you know, X McKinsey X Delight. They didn't know anything about product. But so we taught them the product piece and we were the ones who were like, here's the interesting data I know I'm going to need to see, right?

34:08And they were able to pull those things together and views where we could actually dig into it. But that was a jumping off point on the quantitative research where we're like, okay, now we know that you need to go do the quality data. And I think for product managers, you still need to be super comfortable with data. Like if you can't read these charts, if you can't see understand trends, if you know, and a lot of times too, if you're putting this stuff in looker or you know, a bi tool, like I tell you, you can pull ad hoc reports yourself. It's just that you don't have to craft a sequel query to do it, right?

34:37You still need to understand the relationship between data and product. And you still need to understand like what good data looks like. You need to understand two things like causation and correlation. You need to understand that if I put out this crazy like marketing launch on Thursday, like that's why all of a sudden we got like a million signups on Thursday and we didn't get a million signups on Wednesday, like they still need to understand all those things. And I don't think that becomes any less important. And I don't think understanding interpreting trends becomes any less important as a product manager is job.

35:07I just think it's about putting data into the hands of product faster so that it's not about like you having to fight your way through bureaucratic processes at your company to get that data. Right. So like product ops can help streamline things that we know we're going to look at repeatedly, right? Like there's a bunch of stuff that we know we should look at. So put them put them in a dashboard. Like why should I have to go pull that report ad hoc every single time, right? Put them in a dashboard, put them in a report. Same for like board slides. We had like one of my friends who's the chief product officer of Forsta Brian, Brian Buddha.

35:43And he said like I love product ops because when we prepare for board meetings, I know there's a certain set of information that is going to be a lot of work. So I think that I'm going to have to put together for this board meeting. And then when we go do it manually, it becomes obsolete by the time the board meetings over. And then I got to start again and prepare for the next three months and do the board meeting again. Right. So it's like you don't want your data to be obsolete. You want this to be in a repeatable fashion for things that you know are repeatable. But it doesn't get you off the hook of still looking for trends and looking for insights.

36:17You fell in love with building products for a reason. But sometimes the day to day reality is a little different than you imagined. Instead of dreaming up big ideas, talking to customers and crafting a strategy. You're drowning in spreadsheets and roadmap updates and you're spending your days basically putting out fires. A better way is possible. Introducing Giro product discovery, the new prioritization and road mapping tool built for product teams by Atlassian. With Giro product discovery, you can gather all your product ideas and insights in one place and prioritize confidently. Finally replacing those endless spreadsheets.

36:53Create and share custom product roadmaps with any stakeholder in seconds. And it's all built on Giro where your engineering team is already working. So true collaboration is finally possible. Great products are built by great teams, not just engineers. Sales, support, leadership, even Greg from Finance. Anyone that you want can contribute ideas, feedback and insights in Giro product discovery for free. No catch. And it's only $10 a month for you. Say goodbye to your spreadsheets and the never ending alignment efforts. The old way of doing product management is over. Rediscover what's possible with Giro product discovery.

37:30Try it for free at Atlassian .com slash Lenny. That's Atlassian .com slash Lenny. We had a guest on Casey Winters and he has this perspective that when you have ops operational people, that's generally a sign that something is not efficient and it could be made more efficient with software and product. Not everything can be productized, but do you have a perspective on that that ops is often assigned were like software. Hopefully someday could do itself. You still need people to oversee those programs. So I don't think they would fully be obsolete. It wouldn't be like this. This whole role goes away completely, right?

38:09But they're the people who should be looking at what can we do to optimize and streamline this and not do it with human components. Right. I think that's why a product management mindset went so well to a product ops function too because you're like, how do I use software or tools or processes or frameworks to help. Fill in some of these gaps and standardize it and then let it run. And that's what I was meaning by like the shared services model, right? If you really think in that format, it's not like, hey, I need a team of 10 ,000 product ops people. I think you're doing it wrong if you're doing that, right?

38:40Like it should be a well run lean team who thinks about how to leverage tools for this or building your own or whatever and your products for the product team. Right. And that's how it should be see. So what I mean, like it shouldn't be obsolete one day. If you are building, let's say products for the product team, like you still need somebody to look over that product and make sure it's relevant. And at the pace of everything changing, like it does today, you know, so many things are so different than they were five years ago. You want somebody to be looking over that and making sure they're still up to date.

39:13It's still relevant. It still works for our company and stuff like that. But you're going to have a smaller subset of people doing that. It's not going to be like, oh, I need a product ops team for like every product manager that's on here. Like that, that's not how I should work at all. I think that is going to make a lot of people feel better hearing that. Do you have any rule of thumbs or waves thinking about just like how many product ops people you want per product manager for team? What's like a simple heuristic for just like how many people you may need? I don't have a hard and fast rule on that one, but I would say like if you're out of one to one ratio, like you're doing it wrong.

39:52You're the one 100 you're doing it wrong. Yeah, too. I also think it's like so so when we talked about it to with this hybrid versus shared services model. If you have a hybrid model, let's say and typically this is a symptom that your data is not well instrumented. Let me put it that way as well. Sometimes you need a stop gap holdover until you can well instrument your data or something, right? And that could be as long as it takes for certain company. In this case, like we said, you might have like a business data and insights person aligned to every director of product to overseas multiple scrum teams, right?

40:29You might have one aligned to every VP of product, depending on how much help you need to get the product out of like the product data out of things. Now, if you have a very well instrumented data set like we were talking about at doodle or something like that, you're probably going to have a way less people, right? Because the product managers are armed and capable of going in there and being able to pull the queries themselves. Do their ad hoc research themselves because they made it accessible and you made it possible for them to go do that. So I do think there's a balance there between how good your company is instrumented both in like for everything like for all three of these pieces that we're talking about relative to the size of your product ops team.

41:10And it might take some more manpower at the beginning to get it going, but ideally you streamline this team and it becomes pretty lean and pretty small and they're overseeing multiple programs or software systems that run themselves. Well, it gives me a lot of hope that this isn't going to become just like another massive org within a company is looking at companies that are incredibly efficient like ramp and deal that are very few employees very few product managers. And they have product ops people and so that tells me there's a lot of leverage that you can find from just like there's maybe I don't know one or two is probably not a large team of product ops people there.

41:48We say to like like get started with one person like we describe these through pillars and you know repeatedly through the book where like hone in on what's the most important part for you. That's going to help you right now that we talk through and then just like take one person and throw that out there and usually you can get so much leverage from that that if frees you up to do a lot of the stuff that you need to get done and then when you're ready and you see the next hurdle and it's not something that first person could take on then you might add one more person right there are different expertise.

42:19I think between the three different pillars and the people who would oversee it and that's something to take into account as well. So like we said in business data and insights like that person typically is not an expert on product hopefully they get ramped up on it working with you but they're typically not an expert on product and a lot of times these people are not coming from a product management background. That's going to be actually a different person that I would look at to help with the governance and product pieces like that person needs to be a product background person because they're usually helping.

42:48You know roll out the roadmap stuff coach people on how to do it helping to define the different systems and processes that you need on your product operating model and if they have no experience with product management at all that's going to be really hard for them to do and it's probably not going to be great and we do see this trend of people throwing agile coaches at this. And agile coaches who have never been a product manager before in a well run company are going to struggle there because they're going to revert back to agile processes and optimizing for things like scrum but they're not going to do what that role is designed for which is to help product management processes like.

43:28We don't usually need another agile coach telling us how to run a stand up we need people to come in and help us figure out like who's invited to these cross functional roadmap reviews what kind of inputs do we need on there and what kind of decisions do we need to make coming out of it and how do we communicate it at the correct level to executives as well so we're not digging into gira at these meetings in front of a CEO right like. CEOs don't care about what's in gira well they shouldn't let's put it that way should and and they but they want to know like what are the big pushes that we're doing to help us reach a strategic objectives right so it's helping.

44:05They know a little more about the right size of communication the different cross functional teams what product managers do on a daily basis and what's important to them and that's really critical in that role so. So that's where I think your teams might just be slightly bigger from a perspective of more than one person not i'm not saying hundreds of people but like you might need a couple different people in this organization because the roles are slightly different like exactly for the market research in the customer research thing to like you might need somebody with a user research background for that somebody who didn't research jobs is a great person for that.

44:40And it's pretty unusual I think to see companies start out with an entire team I worked with sam's club and they were planning to do it that way that mostly we see it sort of starting organically I think the way Christina to our mentioned. In her story of being a PM and feeling the pain and having the empathy of wanting to make it better and that's typically sort of the generation of of these roles that it might be so and. You know being allocated part time from their PM role is finding out that they really do enjoy this sort of enable that aspect of it and that's where it grows. Awesome so let's just lean into this topic of just how to start rolling it out at your company and you already talked about a bunch of tips there and by the way if you buy your book at the end there there's this like really beautiful little guy there you go in the camera of just like a little.

45:26Yellow brick road of all the steps that it takes to roll it. Yeah, it's candy land. So on this topic of just who to start with sounds like you recommend start with one person and what I'm hearing is pick one of these three pillars that you think is most. Highest leverage potentially to take off your product manager and play does that is that right. Okay. Okay and then so what else should people be thinking about in starting to roll this roll out starting to build this team. Yeah that's a great question so one of our case studies is shantaro met sui at amplitude and he created the role of amplitude and you think about it's kind of meta right so there I think they're creating a tool that helps in terms of product operations but they're enabling it there as well so he's done a really great job of getting that setup and and really being a thought later there so.

46:14The case study that we chat with him about was introducing it and how do you get it going and how do you build the momentum and show the quick wins and that was his top tip was making sure that you set. You know understand first of all you may have a perception of what the biggest needs are but doing your listening to or doing your user research some research sprints. Understanding there may be a huge opportunity but it also sounds massive in your team of one where can you make the most difference quickly where can you have the most impact so identifying those celebrating those wins making sure everybody understands that and then sort of showing what's above the line with a person of one that you know the capabilities you'll have and then below the line.

46:57Things that you may not get that you could if we did think about building the team and sort of setting the expectations as well. Do you suggest they try to hire someone that has already been a product ops person is it okay to hire someone that hasn't then they just kind of become a product ops person how important is that experience I think if you find someone that's done the role of product management awesome if someone set it up somewhere me look at Blake Samic right started from Uber to strike to AI clearly. You know he's got a model going that's really successful so somebody could get it like Samic I would say definitely do that.

47:34Yeah I think I think it's going to be hard to pull away from open AI yeah yeah he's leaving there right now just started got a lot of work out of them. But I think it's a lot of LinkedIn requests right now yeah yeah it's like. I'm sorry sorry for your inbox I think too it's it's about if you're going like all it right if you're if you're just trying to get buy in for this and let's say it's originating from a CPO or like a product leader who's like hey let's start this out but the company is not ready to invest all the way you might pull somebody from a from a different function let's say have them be the product ops person and try to rapidly demonstrate value if you are like a CPO or.

48:15Leader and you're like I got budget I know this is important i'm totally bought in let's go right you're probably better off hiring let's say somebody who's done this before somebody's experience but also somebody who could be more of a VP or director of product ops and then they can go do the hiring and figuring out do we pull people from other functions and like stream line it. And that helps free up the leader as well from not having to go find 8000 that it's never going to be 8000 people but like eight people to do that function right so that would be a way to look at it as well so if people are not quite sold on it you might want to start with one function.

48:51Figure out where the burning you know issue is demonstrate that value show that it's like something that you want to actually invest in and then you might want to hire in a leader or help build up the team from there. It feels like that first hire so important because if they don't do great the whole role of product ops starts to get a bad tinge within the company and so there's a lot of pressure on making sure that first person succeeds I guess more reason to buy your book and make sure that they do it right. If you feel like you as a leader can coach the product ops person and you have time to coach the product ops person let's say through being a good product ops person and getting started with it and directing them you're probably good at taking somebody who doesn't not have experience but has the rate skill set and then you can teach them.

49:38If you have absolutely no time to coach this person and they're not super self directed let's say so they're not going out taking classes reading the books doing that type of thing probably want to hire somebody who's done it before at least that portion of it and then that person can help you know coach other people grow the function it's the same thing with product management right like we look at these teams. Of a lot of you know especially in a lot of transformation companies we have a lot of product managers who've never been product managers before and a lot of leaders who've never been product managers before.

50:06And they asked me do I hire in experience leaders or what and I'm like well if those product leaders need to go coach other product managers then you need somebody who's experienced in there right you need somebody who knows how to get that work done if they don't have time to learn if you're not on a timeline to actually like teach these people these things and get them up to speed then you need somebody hit the ground running so I look at it for like how much time do we have to demonstrate value how much coaching is available to get this person into the right the right mindset. And the right skill sets do this in execute and I think that that's needed in almost every role not just this one.

50:42Right and one thing I wanted to mention was whether to hire products or product manager person for the role with the background of product product officer product management you know there's not a ton of folks out there that have a products title but as you're looking you know dig in because this person may have been doing a lot of the work but as a product manager or within that title so there's a lot of people out there that probably would fit that profile but not with a proper products title. What are the key skills that you find are really important for finding this person especially if they don't have this role my guess is dependent on the pillar if they're data focused research focused or process focused what do you suggest to make sure you're looking for when you're hiring this person with the business data and insights like you're looking for somebody who's really good at interpreting data telling stories with data somebody is good at communicating to many different types of stakeholders about the data and putting it into useful ways what I wouldn't hire for that role as a first person is like a data it's not a database engineer right like that's not what we're doing we're not like like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like we're not like WD and Ray will say that these things we will say that these things we żeby or something like this Devons that have a link toyd Kevin shake up your back to興 bend Well, also believe it's not a debt doctor's projects or mashing It's way, it's a debt traffic It's way too obvious They shouldn't be the ones coming up with all the questions you need to answer.

52:54It'd be great if they surface them in -site. But you ask them the questions, they can go get the answers. What about for the other two pillars? Well, in terms of the process and practices, I think this person really needs a super high EQ, right? Kind of understanding what the needs are, but also has a good, sort of, spidey sense of how much methods do we need to think about bringing to the team? And thinking about how those things get introduced, and it's not a mandate, right? But it's like a suggestion of how we can work. And typically, product managers will be pretty open to that, right? Because if they're like, I don't know what the roadmap template is, I don't know what the roadmap cadence is.

53:35Here's some guidance. Awesome. Now, I have to think about how to do it. I'm just going to do it. So I think someone who has has a lot of experience, sort of understanding, sort of the underlying tensions and opportunities. And then feels good and understands how to implement the systems thinking, and also understands it's not as set it and forget it, that they're constantly reevaluating the processes, the tools are these working for us. So, and then understands, I think, and more broadly, as my CPO is getting ready for board reading, what are the inputs they're going to need? It's getting ready for the QBR.

54:13Are the PMs ready? Do we have a cohesive story? Is anybody taking the time to look at all of the different presentations to make sure that we're sort of giving the same perspective or sort of building towards a certain strategy that everybody's focused on? So that would be my advice. I think for customer and market research part, you're looking for somebody with more of a user research background here, but process oriented. So I'd look for somebody who knows user research gold, knows really good tactics for that, because it can help create the tool kits, get the right prototyping, the usability software in there, they know what good interviewing looks like, that type of stuff.

54:54But they're also got this kind of need to make things better. So they're like, I need a creative system to do this. They're good at operationalizing stuff. That's, I think that's a skill for everybody. They're like, I can build a system to fix this. That's actually a really good interview skill that I would ask about product ops. And I never came up with the, like I never thought about that before. But it's like, tell me about some kind of process or some kind of thing that you had to do in your job that you really hated, and that you ended up just trying to automate away. Or like build a system around it to make it better.

55:24That would be a great interview question for anybody in those roles, I think. With a user and market insights person too, there's not a ton of people out there doing this, but there is a little research ops movement out there that I think could be really valuable here. So Jen Cardello, who is our case study on fidelity, she runs their user insights team there. And she's our VP of, I think, is user insights. And she does oversee all their user researchers as well, but she also oversees a research ops team. And the research ops team is responsible for building their purchase of end database. They also help build toolkits for people to do user research.

56:01They oversee any of the user research tools. They also go out and train other people in doing good user research. So with them, that is, that looks like not everybody's allowed to go talk to customers in financial firms like this because it compilines reasons. They make sure they certify people to be able to go do good research up to certain points and they get levels for how far they can go so that they can democratize a research and help put it into their hands. And then if there's compliant issues around different research studies, they come back to Jen's team and they use the researchers will help them complete it.

56:33So it's about how we, you know, there's a lot of legal things about what you can say and what you can't say to customers and stuff like that. What you can ask them and that they're navigating those complexities around there. So Jen, Jen comes from a research ops background. Like she comes from a whole UX background, but research ops is like her thing and she's fantastic at like setting up that stuff. And she and I worked together at Athena Health and she did that there. So I watched her put that into place and it was amazing and now she does it at fidelity. So if you find somebody with that background, golden, but if you find somebody who, if you can't find somebody like Jen because there's only one Jen, you should look for somebody who has at least a user research background, probably some kind of UX background.

57:14They're really good at doing that and they want to operationalize it. Jen's about to get a bunch of LinkedIn requests too. Sorry for your inbox Jen. I feel like the research team is going to hate people now for pulling her into product ops. Would you suggest product ops report to generally, at least to start on a new site? I had a product. C -P -O. Hmm. Such a clear quick answer. I love it. And how do they find time to train and work with this person? Are they kind of like the right hand person that helps them just make everything more efficient? What's that relationship like? I think definitely like the right hand person.

57:52And we say to a lot of C -P -Os, especially like high growth companies, like make your first tire to product ops person and help you get this data out and start looking at it. Because that helps them with forward meanings. It helps them set strategy. And usually when you walk into a girls' age company, that's the first thing that you need to do is make sure that it's working, that you need to set it. Typically when you're getting hired, there's usually a strategy problem. And that person is like your right hand man, right, trying to operationalize that. So I definitely think that you're going to be guiding them there.

58:23I think this comes back to your question, though, about do you hire somebody with experience or do you hire somebody who's new to it? If you as a C -P -O don't have a lot of time to train up somebody on product ops, because you've got 8 ,000 fires to fight, then hire somebody with experience, right, who knows how to do it and operationalize it. If you're like my biggest issue is business data and insights, for example, right? And I just need to get my data so that I can do the strategy pieces and then I need to think through and like work through what I want product ops to look like. Maybe then you just hire the data analyst.

58:57And it's pretty if they're competent in the data analysis piece, it's pretty easy. I haven't done this myself to teach them about what types of information you need to see. As a product person, they're going to need a lot more handholding at the beginning, because they're not going to know all the different cuts of data. But it's not an investment of an inordinate amount of time, right, to be able to get something valuable back, right? It's not like training for 40 hours a week and then waiting six months to see results. It's more about I need you to go pull these types of cuts. Here's why. Let me explain to you this so that you learn it and then you can think about it next time, but that's going to help you there too.

59:33So I think it really depends how fast you need product ops, fully rolled out, and then how much time you have to train people and then where you're starting from there and how big and how much buy and you have to grow this thing from the get go. Awesome. Maybe the last question. I'd love to go through a quick case study of a company you worked with and just kind of share how maybe how you rolled it out, what you ran into challenges you or had to overcome and maybe the benefits and impact that adding this role had. When I was at Athena Health, we were doing a, Athena Health has always been a software company, so let me point it that way.

1:00:11But they didn't have a formal product management role and they had just implemented it when I came in. So the chief product officer brought me in. He did not have an extensive product background. So he said, I need to train all these product people and figure out what to do with this organization. So I came in to help him do that. We had over 360 product managers. We had 5 ,000 software developers there. And it was a massive platform, like $8 billion in market cap, I think. Electronic Health Record System. So this is where I started to realize we needed product ops. Like this was me discovering this.

1:00:47So I'll tell you how we rolled it out and probably what we would do differently next time. But we had trained all the product managers. People were starting to use a lot of the things that we were teaching. And we saw that the maturity was getting a lot better in the organization, which was fantastic. But then we started to run into these problems. And these problems that we found could not be solved by just training product managers. And that's where the concept for product ops came up. We also realized we had way too many product managers. Like just way too many product managers, there was like one person reporting to one person all the way down.

1:01:19And we were like, this is not helpful. So we ended up training everybody, teaching people about what the role was. And then thinking through as we encounter these other problems, like what else do we need besides product managers? And product ops became one of the things. We also had people actually move out of product management into other roles. We had people become data analysts. We had people become user researchers. We had people go into other parts of the organization. But a lot of people have to retrain to actually just self -select it out of product management. And some of them did come to us and say what else is there.

1:01:52And when we looked at the product ops role, we said, okay, what are the big fires that we have to fight? That's just not from a lack of skills perspective. And that's a big part about product ops. It's not a replacement for product managers or product leaders not having product management skills. It's to help skilled product managers and product leaders do their job better. So this will never replace the fact that people don't have the skills to do their job. So where we ran into issues was one getting insights back to the executives on what the teams were actually doing. So the CEO and I were sitting there trying to set strategy and set the vision for the company.

1:02:32And I was helping him formulate it into written forum and help him deploy it and think through where we wanted to go. And he was like, in Jira digging around in Jira, trying to find information on what people were working on. And I was like, you're not going to find that in Jira, especially when we've got hundreds of thousands of tickets for 5 ,000 people. You're not going to find this in there. And that started to show me like, hey, he's looking for this. What do you want to see? He's looking for a portfolio roadmap of what everybody's doing. And he wanted to see what are the big pushes we're making from a feature perspective and how do they tie back to our overall strategy and our goals?

1:03:13What's going to help our retention? What's going to help us get new customers? What's going to help us move into the enterprise which was a big thing we were doing, going up market into hospitals. And we had no transparency into the allocation of R &D on that and also the word maps on that. So one of the things that we were trying to do in product ops was build that view. Right? Like try to figure out how we get people to put the right information into Jira at the right level. So we actually had to train people on how to write at the times we only had Jira. So like, epics in Jira that were not just like build a button.

1:03:46Right? Like there were more, there were more substantial than that. They actually had some meat behind it so we can look at it. And then we had to go out and find the right software to roll that up into a portfolio of views so the executives could get the insights they were looking for. We also had to build a way to track the OKRs that were deployed. Right? And actually see where it was. So we had to build the dashboards for that. So we started there and that became really important because that was a big issue was just the executive visibility. How do we make consistent roadmaps across the organizations?

1:04:16How do we get visibility into what's going on? And as we started to identify more and more things, we said we're a huge team. We should actually have somebody overseeing this. Data and insights was a really big issue in the company in general. And we knew we had to instrument things better and at the time they brought an amplitude and they were starting to put an amplitude everywhere in the organization but it wasn't fully rolled out yet. So we had these people who were going around trying to help the individual product teams get the information out of amplitude. And we said, we need this to be more of a consistent thing like a consistent program.

1:04:52So that kind of sparked the need for having our first product operations leader. So we ended up creating a VP of product operations and somebody moved from the product management role into that. She was much more of a process type person. She wanted to really help arm the teams into being able to get good data out there but she understood product management well enough where she knew how all this stuff worked and she wanted to create the systems internally. So reporting into her, we had a data, a business data and insights team that was overseeing amplitude rollout and they were also putting people around the director level overseeing usually like five to let's say eight scrum teams.

1:05:32On the director level, sometimes smaller, depending on the product, we had a business data and insights person embedded at that level to help get the ad hoc reports out now because we weren't well -instrumented. We're still making the programs in the shared services at the top level, but these people need to make decisions today. So how do we get them to do that? So she oversaw that team. She had somebody directed there. And then we also had the people looking at the portfolio, views and the governance and the rollout and the rollout of that getting put into that as well. So that helped us kind of get going with that and on the other side.

1:06:08So this didn't fall under product operations at the time, but like I said, Jen Cardello was doing research ops there and leading this team around the user insights and getting all, she got the participant database out. That was fantastic. She got out a bunch of different user research tools. They made a design systems database too. That helped us be able to do prototyping a lot faster and have consistent design processes, which was amazing. The head of UX reported it into the chief product officer. So it's still fun to follow under the CPO, but it reported into the head of UX on that side. And that was totally fine because we just collaborated with them like pretty much all the time.

1:06:43So that's how we started to roll it out and get going and that's where that need was. And it became so much better to get the insights that we needed out there. And then what happened was actually Athena Health at this time was really wild. You can private. So this is where I left, and a lot of leaders left at the same time, but they ended up restructuring it. And they actually kept the product ops team. So now to Davenport overseas product ops team, he was a chief of staff for the chief product officer at the time. And he's been building it again, taking it, taking the stuff that worked, and then building onto it.

1:07:18And they're actually one of our key studies in the book as well about what Tim's doing now and how he's orchestrated as well to help with OPEX and CAPEX and accounting type issues that they were having to. So Athena Health has been through many different restructuring since I've been there, but they have always kept product ops and their current chief product officer is said that he will never go anywhere else that doesn't have product ops. Like that's how much he believes in it. Wow. What a testament to the value of product ops, 100 % retention on the role through all these transitions. One of the interesting things you said is within this product, this VP of product ops managed a bunch of different people in teams, which is really interesting, because I always imagine product ops VP would manage product ops people.

1:08:01Is that common where they lead? Say there's a data team you mentioned, if you other teams? Yeah. In this case, we did have her managing the business data and insights people. And they were kind of data analysts. I want to say they were like data engineers or anything like that, right? But they were people who were really good at pulling SQL and analyzing data from a product perspective. We actually moved, I should say this as well. We moved a lot of people who didn't want to be product managers, but were good at that out of the role and into that role. So they had some product background. They've been trained in product.

1:08:32They were really good at the data pieces, but they were more suited for that than they were suited for product management. And like I said, a lot of people opted out. They're like, give me out of this role. I don't want to do this. They wanted more of like a transactional type role or or diving into data. And a lot of it came down to, I think people under, under anticipate how much they're going to have to deal with stakeholders. And once they have to, they're like, oh, God, I don't want to do this. And I see that like over and over again when it happens with product management. But so she oversaw them.

1:09:01But we did have, they did work closely with the data people on the CTO side. There was a whole data team on the CTO side who were doing more of the database administration and the instrumentation of things. And they were also helping to roll out like amplitude, instrumentate correctly and building like the right views and things like amplitudes or the other product analytics or other tools that they were using. We did not have Tableau at the time. That was something that was like added later. But they were in charge of like kind of utilizing the data and trying to build those insights. Amazing.

1:09:34With that, we reached our very exciting lighting round. I've never done this with two people. We'll see how it goes. So you can pick the question you want to take or both answer. Here we go. What are two or three books you've recommended most to other people? I'll take that. Of course, escaping the build trap is true. The other one that I recommend is called Traffic. It came out this year by Ben Smith. Kind of around the growth and invention and growth of Huff Poe and Gawker and whatnot. I was in media at Conte Nass then. So kind of peripherally part of it. It's an exciting ride that has us, you know, an ending that we all know.

1:10:12But it's a good story. Good tale. The Art of Action, I think, is a fantastic book on strategy. And I always recommend this to people. And it's kind of like out of the realm. So I had a great product management book out there too. But I like this one because it's like a sleeper hit, I think, in the product management community. It is a fantastic description of deploying strategy and how you can tell if strategy is well deployed in organizations or if there's gaps that you need to fill. So I find that when people read it, they go, oh my god, we have all of these problems. And I'm like, yep, that's a strategy deployment and a strategy creation problem that's that's pretty apparent.

1:10:46So that's my favorite book to recommend to people. I love Teresa Torres's continuous discovery habits, fantastic book as well, just to give a shout out in the product world too. Next question, favorite recent movie or TV show that you really enjoyed? Deutschland, 828689. It's on Hulu, highly recommended. About a huge journey. I'm like pedal. Yeah, really great. I am going to go, I just watched the house of Usher on Netflix. I love the, it's Halloween right now. So well, it was just Halloween. So I was watching all the scary movies, but I love the Netflix genre of everything from the haunting of Hill House.

1:11:30Those were great. Amazing. I started watching that and then I gave up quickly, but I should give it another chance. My wife and I have been stuck on love is blind, classic. Same, same good one. What a terrible, wonderful show. It is. It's the high low. Next question, do you have a favorite interview question that you'd like to ask when you're hiring people? When was the last time you changed your mind about something really important and why? So do they have sort of a learning mindset? Do they have self -awareness? Can they acknowledge sort of where they were and how they evolved? So that's usually a pretty insightful question.

1:12:06Yeah, mine's probably tell me about a time that you failed. And what happened? That one's like an interesting one too, because if nobody has an example of a time that they failed, that right there is an interesting sign. To I feel like everybody tries to turn it into a positive story. They really try to spin it. And I'm like, I don't even want you to spin it. I just want to tell me what you learned. It's fine to say what you learned at the end of that. I think that's okay, but they're like, I'll put this happened and all these things were great afterwards. And I'm like, I'm not looking for a fantastic outcome.

1:12:38I'm just looking for, did you fail and did you learn something from it? Right. Do you have a favorite product you've recently discovered that you really like? Well, there's like one I'm going to have to plug. It's not that I recently discovered it, but I talked about all the issues that we had at Athena Health with the portfolio management system. And after that, I discovered Dragon Boat. And I'm now on their advisory board. But it helped me solve that problem so much of like rolling up everything that was in Jürrett sits on top of it and being able to look at it in a great view. So like that was so, so good.

1:13:13Another one that I have not been involved with, but that I just really love because I think they're doing great things with again in the product ops world is dovetail. And they are a research repository out there. And they're also putting AI on top of it to help generate insights into all the research that's been done. So I have no affiliation with dovetail, but I really love what they're doing because I've had that problem of us rolling our own research repository like over and over and over again. And now we've got a tool out there to do it. I guess the one I would mention is this to Lee. And some product leaders that I'm working with right now are sort of playing around with it.

1:13:51And it's kind of an interesting tool that's kind of a single workspace so you can connect your strategy and discovery and delivery visually. And kind of guides people along those paths as well. So interested to play with it more, but it picked my interest. Do you have a favorite life motto that you often repeat yourself, often share with someone come back too often to help either with work or with life? Mine's both, especially with teams that I'm working with or cross -functional stakeholders that if you try to serve everybody, you serve no one. So it's about honing in on who you're actually building your product for, who your persona is, and then with my children in terms of, you know, try to be everyone's friend, but it may not happen.

1:14:35So kind of goes both ways. What's the worst that can happen? With most of the things that I've done. And to me, when I get scared about something or trying to do something, or even like interpersonal reactions, right? Where I'm like, God, I have to have this hard conversation with this person. Like my brain, I try to tell myself like, what's the worst that can happen? And usually when you sit down and start to think about it, it's not as bad as you anticipate. So for me, that's always helped me manage like my anxiety around difficult situations. We're taking a leap and trying to get out there.

1:15:11And I think in my younger days, I was a little crippled by overthinking and by into anxious about things and not taking big leaps. So I've tried to stick by that and keep asking myself what's the worst that can happen. And it's usually not as bad as you think it is. I love that. Final question, since there's two of you, we'll end in a really sweet way. What's one thing you really admire about each other? Melissa is kind of a legend, right? But behind the legend, she's just a really great, collaborative person. So of course, we got to know each other really well, writing this book together. We worked together before that.

1:15:49But it was a really great way of staying connected to a former colleague, someone in the industry. And especially during COVID, it was a really great way of sort of building towards something that was bigger than the both of us. So what I love about her is that she's really revered by so many folks, but really approachable and really truly wants to help people. Oh, thanks, Denise. Well, I love about Denise is she's really this calming patient presence. And she approaches everything with so much professionalism, but also can really get to the meat of the problem, help calm down certain situations, right?

1:16:34We're escalated, turn them around. And she's got this like patience and calm about her and everything that she does that helps you focus, helps you concentrate on the big issues. And she can really steer people in the right direction. So to me, she's always, I've always admired watching her like coach and lead people. And she let her analysts at products labs as well. And when things would go, you know, slightly awry or a situation would go out of hand. Denise was always like, nope, we'll just fix it. And dove in and was able to, you know, to make it better. And I always really appreciated that.

1:17:10Oh, Shax, we might have had a few of the what's what's the worst it could happen moment. Yes, right? Always. Amazing. You two are awesome for folks listening. Make sure to grab their book product operations. You can find it on Amazon. I imagine share where else people can find it, but two final questions. Working folks, find it online if they want to reach out. And how can listeners be useful to you? Product operations .com. You can, we've got some downloadables that are available for folks and questions. If you want to get in touch with us, how can they help us? I'm trying to think here. Tell us your stories about product operations.

1:17:46Yeah, we want to hear like how people are implementing it, what's working, what's not working. Like we're always learning about this too. And I think we, we said in this book, it's still, it's taking off now, right? But it's by no means standardized. So we want to hear about what's happening. How is your company been successful or not successful with it? What have you seen work? What have you seen not work? So definitely reach out and let us know about it. Also, you can help us by buying the book and leaving us an Amazon review if you do read it because that does help authors. And we self -publish this one too.

1:18:14So that always helps us on there. You can find both of us on LinkedIn as well. I'm Melissa Jean Perry, I think on there. But if you look for Melissa Perry, I usually pop up. And Denise Tillis. And then if you want to contact us, yeah, productoperations .com. We got a form on there to reach out. Productoperations .com. And is that the best way to find where to buy the book or their other sites? You recommend people go check it out. Yeah, and we do. We just got it. Self -publishing has been fun. But we just got it where it is distributing now to like Barnes and Noble and other bookstores. So you can usually find out anything.

1:18:48But if you go to the website, it will have the most up -to -date information on where you can find it. Then if you go to your bookstore and if you, some people are against Amazon. So if you go to your bookstore and ask for it, they will be able to order it for you. Amazing. Thank you to For Being Here. Thanks for having us. Thanks for having us. My pleasure. Bye everyone. Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast.

1:19:23You can find all past episodes or learn more about the show at Lenny's Podcast .com. See you in the next episode.