Using AI to evaluate employee performance with Rippling’s COO Matt MacInnis

0:05Hi, listeners. Welcome back to KnowPriors. Today, Alad and I have a spicy one. We're here with Matt McInnes, the COO of Rippling, the juggernaut workforce management platform that unifies HR, IT, finance, and more. They're launching a new AI product that looks at the work output of employees and generates performance management signals. Sound terrifying? Let's discuss. It's so good to have you. Thank you for having me. So I think a lot of our audience will know Rippling or use Rippling, but for anybody who's missing it, what does the company do? Yeah, It's an all-in-one platform for HR, IT, and finance.

0:40We do all the boring stuff, but the important stuff to help you run your company. So we want to eliminate the administrative burden of running a company. That's all the official language, but most people come to us and say they need payroll, and we have payroll. They need a device management solution. We have one of those two. So we do all that stuff. The rumor is many hundreds of millions of dollars in revenue growing fast. Anything else you can say about scale? It's going well. We've got about 3 ,500 employees. We've got tens of thousands of customers using the platform, so I'd say we're doing something right.

1:11I guess also one of the things that you all have really pioneered is this notion of reintroducing compound startups or bundled products across a suite of different things. How many different products do you offer now, and what's the velocity in terms of adding new ones? We have on the order of like 25 unique SKUs that a customer can buy from us. Products come in different shapes and sizes. And so like we ship small new things every quarter. And then we definitely do like big things every couple of quarters or so. We're about to ship like scheduling, which, you know, again, sounds unsexy, but is actually really cool.

1:44We shipped an applicant tracking system for recruiting, you know, tacking these sorts of things onto our HCM suite. We do a lot of this partially because we have so many founders in the business. We have over 150 people who have started companies that now work at Rippling. It's like an explicit strategy to go out and try to either give talented entrepreneurs whose business ideas didn't quite work out like, hey, hand raised, I've been there, a safe place to land and continue either pursuing what they were interested in or do something new at Rippling. And so that's worked out really well for us on the velocity front for shipping new products.

2:11The compound startup thing obviously has been in the zeitgeist a little bit in the valley. It's just obviously a huge tailwind for us that businesses generally want to consolidate as much of their software onto a single platform as they can. So we're going to keep pursuing this and keep recruiting awesome, talented entrepreneurs and ship new stuff all the time. Makes sense. And I guess one way to think about your business is almost like instead of one company growing at a certain rate, you're like 25 startups all compounding from a smaller base, which I think is very exciting in terms of the potential upside.

2:38It makes like if you're one of the things that I wished I had learned way earlier in my career as an entrepreneur was just like basic corporate finance, like understanding an income statement and a balance sheet and how those things play together and what investors look at in that context. And I get it now, for the record. I think I've mostly figured that stuff out. But when you look at the income statement for Rippling, and you think about the 25 businesses or just like the major product suites like IT and finance and spend, they're sort of subscale businesses at some level on their own. And then in aggregate, you have this beautiful top line picture.

3:13But from an efficiency standpoint, it's okay today, but there's clearly just going to be this like blossoming of efficiency over time for us as these different suites all play to one another. In SaaS software, most people don't totally understand that like for scaled businesses, the unit economics of your business converge at the cross-sell motion. Like your new logo sales motion is super important, but as you sell your products into your ever-growing customer base, like your economics start to look like that more than the new logo sales motion because you always get a lot more in the cross-sell bucket.

3:43And so for us, the compound startup thing also has these beautiful financial dynamics that we have a lot of things we can sell to our existing customer base over time. And that's helped us a lot economically. And one of the things that you're here to talk about actually is a new product that kind of ties together a lot of the other ones in some ways. Do you want to talk a bit more about what that is and how you all are starting to move into AI? Yeah, I mean, the pendulum swinging toward consolidation has these obvious surface level benefits, right, of better sales efficiency and customers being able to save money by not paying multiple sales teams to acquire them.

4:16But that's like super basic, like it's really surface level. Where the magic really comes is where there's something common underneath all of these different applications that you're building that provides you with either a scale advantage or what I like to call kind of like your vibranium advantage. So you have some sort of superpower at the core of your platform that lets you do things that other companies just sort of look at and think like, how the hell did they do that? Like, why are they able to do that? And we can't. And for us, it's like our deep understanding of the employee graph and about employee data.

4:44So everything that we build runs on these common rails of like a deep understanding of data about the employees in your business. And so the question is like, what happens when you start to marry all of this data in a single platform? Like what's some cool stuff you could do with it? And then you toss in the question of AI and like what could a large language model accomplish with this data and this like real understanding of its structure and its history? And that was one of the big questions we started asking ourselves a few years ago and started investing in this new thing. So there's a new product that we're just releasing called Talent Signal.

5:12It's the ability of this system to read the work product of employees and like marry the data that we have on whom you've hired into your company at what job level with what, you know, all the basic data about their job history married together with the actual work product that they produce to yield like an insight into how those employees are doing. And, you know, this is obviously going to be super powerful and useful. And it's sort of this thing that I think everyone knows is coming at some level, that like AI is going to contribute in some way to evaluating human performance. And so we knew there was an opportunity here, and that's what Talent Signal is going to deliver.

5:50It actually feels like a pretty big break to be looking at what you describe as work product, because traditional HR and IT systems, they don't necessarily have that work product data in them. Whether you've had a job as an individual contributor, you know, for some period of time reporting to a middle manager. And like, I did that for a while early in my career at Apple. And the real sort of crunch point in your relationship with the manager comes around performance review time, where you have some opinion on how you've done. And your peers and others around you have an opinion on how you've done.

6:24And your manager has an opinion on how you've done. And everybody gets in a room and after they've written the feedback, you know, they do this thing called calibration where managers try to hold themselves to a common standard and they all try to hold one another accountable to a standard way of evaluating against the rubric. But the truth is like the manager has never really hasn't really sat there and like looked at everything you've done, particularly if this is over like a six month time horizon or a 12 month time horizon. And they just don't have enough time to do that. And so there's a bunch of like really interesting articles written about this in many different sources.

6:57I recently read one in HBR where they talk about the manager vibe. So like if the manager has a good vibe about an employee and there's ambiguity about their performance in the review process, then that opens up this massive gaping hole for the vibe to be the basis of the performance review. And likewise, if you have a negative vibe on an employee and there's some ambiguity about their performance, then they're gonna drive that negativity through that crack like a Mack truck. And the question is, how do you get around this tendency in these fundamentally human processes? And the answer is, we go to the source.

7:29You bring the facts to the discussion. And so Talent Signal, by reasoning from the work product only, it doesn't have access to demographic data. It doesn't know your race, ethnicity, your age, your work location. It just knows this is the source code you wrote or these are the customer interactions that you had as a support agent. And then it generates this thing called a signal that is a stamp, basically, that says that this person is high potential, this person is typical, or this person is in need of attention. We call it pay attention, but they're effectively at risk. And directs the manager to go and spend time with them, but it surfaces all of these concrete work product examples that the manager can use to go and have a good coaching conversation with the employee.

8:13Do ICs get to see it? The ICs can see it when the managers let them, and this is actually a thing that we've debated quite a bit. Talent signal is not making employment decisions. It's just giving this independent signal to the manager about how the employee is doing. A calibrated signal. Yeah, well, it is calibrated, and that's actually really important, because one of the pieces of data that we do feed the model is someone's job level, and then we try to calibrate that actually across all the companies that the data is trained on. Does it end up showing calibration relative to both the individual company and overall pool?

8:45We don't separate it out. We give you only the localized version. And so you tend to see like a pseudo normalized distribution. So you see like in a population of like 50 engineers, you'll always see some people who are flagged as being high potential. And you'll always see some that need attention, even if in the global model, you know, they were all skewed. It's a really good company. It's a really bad company. Yeah, exactly. You know, because it's not particularly useful otherwise. And this is all stuff that is part of this early access program that we're doing. A couple of things that I should share about this, because I think your listeners are going to clue in now, like, wow, the stakes on this are pretty high.

9:20Getting this right is awesome, but getting it wrong sounds kind of dangerous. We are doing this as part of an early access program. And the way that the product works is that it generates one signal one time for one employee at their 90-day mark. Even people who have been at your company for like three years, we can generate a signal, but we're only going to base it on the first 90 days of their work product. And the reason that we're doing it this way is because companies that look at this can see, okay, this thing actually made like a pretty darn good assessment at day 90. And it took us like 12 months to figure out that this person was not a fit for our company or that this person was going to be an exceptional member of the team.

9:53That builds trust in the model over time. And we think like, I don't know if you guys have ever heard the Overton window concept, right? Like this idea that people are only ready for a certain amount of change and how they think about a certain problem. And for us, it was actually really important to contemplate in the design of the product that we not stretch the Overton window too far. And also, by limiting it to the first 90 days, we get to build trust with the employees, with the managers, and have them kind of understand the implications of this thing and whether it's accurate for their particular circumstances.

10:18And over time, we can expand how it's applied. These are all the different issues that we've contemplated as we've gone along. But if somebody's been around them for three years and we have a 90-day signal, is that still relevant to that person who's been around for three years? Nope. Highly not likely to be like incrementally useful information at that point. The idea there is like, hey, here's what we would have said at day 90 for this person. It's backtesting. Yeah, it's backtesting and establishing some level of credibility for the model because we obviously done a bunch of testing with this and thought it was quite accurate.

10:46And certainly instilled confidence in us that like it's a useful signal. So a lot of the use of it is actually for new employees versus people who've been around at a company for a long time. This version of the product, V1, like as we step into this baby steps, is to do the 90-day signal for new hires. And so the more people you hire, the more useful it is. So high-growth companies, obviously, are going to get more value out of this initially. But the sky is the limit, obviously, as this thing evolves and we all gain more trust in the model. I want to talk about risk, too. But what is the aspiration for how this changes performance management?

11:17For me, the sort of motivating factor here, honestly, it's the bad manager. If you're an employee and you are working in the bowels of the organization on hard problems, your manager, a little lazy, doesn't sort of recognize the quality of your contributions, shows up at that calibration meeting with a better vibe on somebody else and they get the promotion. Talent Signal walks into that environment and slams your work product down on the table. and says like, what about this? I can give you a concrete example of an underrepresented profile at Rippling when we were building this product. She was an engineer in India who was working on one of our toughest problems and she was singled out as a high potential employee.

12:03And she was in fact pretty early in her tenure at the company. And we paid attention to that and we talked to the manager about it. And it was sort of an eyebrow raising moment where she was kind of lifted from obscurity by the model that was like, I don't know what your vibe is on this person, but like, man, they seem to be contributing at a high level. And here are concrete examples of how they've done so. So the lazy manager who doesn't like represent things the way they ought to is held accountable by their manager when they look at the total organization through this tool. And it does a better job of representing the employee.

12:30And obviously I can talk all day about like lifting people from obscurity, but it also has the team performance impact of signaling that someone needs support. If they're not performing well, if they haven't ramped well, giving that signal to the manager and the manager's manager knowing about it is hugely valuable to overall team performance too. So the vision here is like to have an independent, when I say independent, it's independent of the biases of the manager. It's independent of all the noise that sits in the company and just cuts at the heart of one vector on this employee, which is their work product and gives them a chance to shine.

13:06You just imagine this is the first time in kind of the recent history of the concept of performance management in companies where there is an orthogonal input that can really upset with facts how people are doing this. What did you learn from dogfooding at Rippling? We started talking about this internally quite some time ago. And as the product has gotten more mature and as we've talked about it more and more with employees, the feedback from employees has been super useful to informing the policies that we set up. I'll give you a couple of examples. No one's allowed to make any significant decision using the model alone.

13:42So anytime you talk about employment decisions, promotions, that kind of thing, you're not allowed to just point at TalentSignal and say, well, it said X. You've got to have your own independent assessment of the inputs. So it's like manager reasoning about work product, TalentSignal point to them too. TalentSignal is like a cheat sheet, but the manager has to do what is fundamentally a human process, which is to evaluate the whole person. The policies that we've set up internally prohibit blind following of talent signal and require the manager to express judgment around what they saw in the study.

14:17And like, look, we dog food the heck out of everything at Rippling. Parker, our CEO, he runs payroll for the company. Like every pay run goes through him. He also approves every expense above 10 bucks. It's a lot. We can talk. We want to talk all day about that. Sometimes I just eat the$10. You know, like, what's the point? Of fighting Parker on the expense policy? Amazing. It was Uber comfort and not Uber X. But anyway, the AI stuff, he's obviously very close to the development of this. And the employees have been, I would say, really thoughtfully engaged in balancing being good sports as dog fooders, but also sort of making sure that their own rights are represented in the development of this technology.

14:55One of the biggest objections I can imagine, especially as you get to evaluating people whose job might be like, you know, classic middle manager, I make other people successful. It's about like collaboration or focusing people on the right tasks that is not captured in a concrete work product. What's your response to that? I mean, so first of all, TalentSignal focuses on individual contributors in terms of developing signals. So for salespeople and support agents and individual contributor engineers, it only does a signal for them. We haven't gotten into the game of managers yet. That's going to be interesting for us or for someone to dig into.

15:32But there is this question of what's it looking at and is it sort of like this overlord looking at everything that I'm doing? What we needed to do in the development of the product was find the highest correlate, like find the best R-squared. Like, what is the input you can give the model that is most predictive of the output, which is, were they promoted? Were they terminated for performance? You know, did they stay at the same level for a long period of time? Just in general, what was their career outcome in the period studied? When we did these sort of preliminary studies, the screaming signal was work product.

16:07You know, like, if you want to know if someone's a good engineer, look at their contributions. Like, look at their source code. And don't just look at like, you know, definitely don't just look at how much they do. But like really reason about the quality of the code contributions. Think about security issues. Look at pull requests. Look at comments on pull requests. These foundational models do an excellent job of thinking about source code and writing source code. And so they're actually really excellent engines for assessing the quality. That was one of the coolest things I thought about seeing the demo.

16:36Like when it was looking at assessment of, for example, like maintainability, extensibility. Yeah. Because that requires like code reasoning. Yeah, it has an opinion on this, and it's able to express it really eloquently. And then the manager has to go in and use their own judgment. I'll give you another example. This is an example from a customer who's been using the product. So the CTO of one of the alpha test companies went in and saw that somebody he didn't think was a very strong engineer was flagged as high potential. And he was like, okay, that does not jive with my priors. It's my vibe.

17:06He had priors. It doesn't jive with his vibe, right? It's not my vibe about this employee. So he goes in and looks at the source code and he goes, oh, I see what's happening. He's like, I wrote all this source code. And we're like, huh? Like, tell us more. And he's like, well, this employee has been struggling. And so I've been spending time with them shoulder to shoulder, like writing code and coaching them through this stuff. And like what the model has picked up on is this really high quality contribution that only happens when I'm sitting next to this person. And I was like, aha, okay, cool.

17:35So it's sort of like an unknowable misattribution. How do you think more generally about managers? You mentioned that you don't currently assess them. Andy Grove used to always talk about how the output of a manager is the output of their team. And that's how you're supposed to assess them. So to some extent, you could argue you have some signal you can aggregate up. So when you look in the product, it does aggregate at the manager level to show you the sort of distribution of high potential, typical, and needs attention employees. That part, we're sort of saying to customers, like, use it informationally to sort of spot where there might be hotspots.

18:05but don't totally judge the manager on a basis. So the question is, is that a reflection of hiring or is that a reflection of execution? So I guess it's hard to sometimes tease those things out. But now you kind of get why we call it talent signal because it's a signal. It's like, uh-huh, okay, the little yellow light bulb going off over here. So I was just curious, how did you converge on this as a thing that you're going to do for AI? Was it a big exploration? Was it more like, hey, we actually have something here that we've aggregated data. This AI seems to be good at interpreting certain types of data.

18:32I'm just kind of curious how you landed here of all the things that you could do with foundation models? We thought about a lot of the obvious AI use cases. I'm going to zoom out for a sec and maybe toot the company's horn a bit. You have a dollar and there's a bunch of things you can do with it. Back to this corporate finance topic. Like there's a bunch of things you can do with that dollar. If you invest that dollar back in the front and out the back of the machine comes two, like don't take the two, put the two in the front, get four out the back, put four on the front, get eight out the back.

19:01This is why SaaS software businesses run at such a deep cash deficit over the course of their early years. Now, if you can't do that because you don't know what technology you're going to build next, if you don't know how to invest it in sales and marketing to go and acquire the next customer, if you don't know how to invest it in R &D to go and build the next product that's going to generate incremental revenue, then you might do something like stock buyback. And that means that the most creative idea that you could come up with with this cash that your business is generating is to just juice the share price.

19:30And even worse is a dividend. because like now I can't even do that. I'm just gonna like literally just gonna give it. I don't know what to do with this money. I'm just gonna give it back to you. Like, what would I do with this money? There's like such a bad signal on a company if the best thing they can think of is a dividend. Now, by contrast, companies like Rippling and many companies in Silicon Valley not only know what to do or think they know what to do with the next incremental dollar, but they want even more dollars than they have access to. And so they use equity capital to go out and get a bunch more cash that they can use to pump in the front of that machine and get even more dollars out the back.

19:59You look at some of the highest performing companies in Silicon Valley and they reach profitability or that, you know, some of them do at least. It's still centered around one idea or one product that they have done a really good job of scaling up. And one of the like super unique things about Rippling, and it's like so easy for us as a team to take this for granted, is that we have this massive list of projects that if we were to go build them, they would turn into revenue. Like we know the next product we want to build and the one after that and the one after that. And the only challenge is like, can we hire enough engineers and not run out of money?

20:33You know, because we know that in the long run, this is all going to work. Oh, wait. I think a common objection in like classic, not rippling Silicon Valley, do one thing well type companies is, it's really hard to focus on that many things. It's really hard to do that many things well. It's hard to keep it cohesive. How do you teach the sales team that? How do you think about cohesion? You just work harder. You know, like you just get the right people into the right jobs and get enough leaders into the business who can deal with sort of the fractal of complexity. And this is also the traditional enterprise sales playbook from the 90s, right?

21:03And I think it's almost like we had an era of 10 years in the 2000s where we forgot about this. And everybody became single point products. And then there's you guys, there's HubSpot, there's Datadog. Like a variety of people have built out these sort of bundled products. And the cross-salm mission around a single sort of core either system of record or type of identity or something else. So, I mean, there's the old saying from Netscape, from the Netscape days where all of innovation is either bundling or unbundling or some variation of that. So now we're in an era of bundling again. Yeah, history repeats itself.