#273 Benjamin Encz: How Ashby is Building the Future of Hiring with AI

0:00With the features in Ashby, I can actually describe these criteria. So I can put in these things I'm looking for, and then we'll have LLMs read the resume, and it will indicate whether or not these skills are present. And once you start interviewing someone, we believe that is something that requires human judgment kind of almost all the way through. A huge part of it, touched on early on, is like this data piece where, you know, we help companies understand their process better and optimize it. It's a pretty recent thing that companies have really invested in automating the more complicated parts of the process.

0:26And we're definitely seeing more of it, but we're definitely not fully there yet. Our sponsor for this episode is Xtreme Networks, the company radically improving customer experiences with AI-powered automation for networking. Xtreme is driving the convergence of AI networking and security to transform the way businesses connect and protect their networks to deliver faster performance, stronger security, and a seamless user experience. Visit extremenetworks.com to learn more. That's extremenetworks, all run together, dot com to learn more. I'm excited to get started. Thanks for having me here.

1:13My name is Benji, co-founder and CEO of Ashby. My background before starting this company was all in software engineering. And then in my most recent company, I was a director of engineering and pretty much spent all my time hiring engineers onto my team. That was really hard and competitive. It was back in San Francisco in 2015 to 2018. It was very hard to hire engineers. And so I got really deeply involved in recruiting, recruiting tools, our overall recruiting process. And I had a lot of issues with the tools we were using. There was like a lack of automation, a lack of data and insights. And that was kind of slowing us down on this really top priority.

1:53And so, yeah, that was almost six and a half years ago. And eventually decided that I think we should build something from scratch with more modern technology that's really focused on companies that are hiring top talent in really competitive spaces. And that brings us here. We're basically offering an end-to-end recruiting platform for in-house teams. So it covers everything from posting jobs, getting people to apply, sourcing passive candidates, moving around for the interview process, and all the way for extending offers. So that's kind of the scope of the product today. And we serve about 2 ,700 customers, a lot of them in a technology space.

2:28Yeah. Yeah, that's right. I mean, isn't Zuckerberg offering$100 million salaries? I don't know how real that is. Yeah, I know. The rumors are, but probably not totally off for all the data points we're seeing. Yeah. I mean, certainly total comp, I guess, if you include options and stuff. But I guess my first question is, what were the problems that you saw? You don't have to name other platforms, but what were the problems that you saw that the other platforms weren't offering that you wanted to correct with Ashby? Yeah. One of the key ones was data and analytics. So I think for some background there, you know, a lot of the other platforms were started around 2012 or before.

3:23And back then, recruiting teams weren't used to working with data in like a day-to-day basis. But as hiring has become more competitive, a lot of recruiting teams are not operating more like a go-to-market team, like sales and marketing. And they're increasingly have an operations team supporting them. and with that a lot more data is being used to make decisions to optimize the process to figure out you know what is our comp strategy where are we losing candidates all these kind of questions and we weren't able to answer these with these other products um it was a lot of manual work and so reporting was one key piece and then the second part was kind of automation across a lot of different areas of the process this was like pre-LLMs but still a lot of companies were trying to become more efficient at this competitive hiring process.

4:09And so they were using an applicant tracking system, which is the name for this category of software we're in. But then they were adding separate products, one for automating interview scheduling, which can be really time consuming and manual, one for automating outreach to candidates that you're trying to source for your company. And so the second opportunity we saw was bundling all these products together. they bring like this core operating system, this applicant tracking system, but then also add all the automation in a single product. And so these are probably the two biggest insights we had back then.

4:41Yeah. And the data, what kind of, you're talking about data about the candidate pool or data about churn within the organization or what kind of data is important in this case? Today, we're really focused on the recruiting process. And so if you think of that, it's like a funnel with a lot of different stages, and there can be drop-offs at different stages. I remember when we were hiring for my engineering team, we had certain channels where candidates were coming from, where we only had a 10 % offer acceptance rate, which is pretty abysmal. If you extend nine offers or 10 offers, you only get one candidate to actually join.

5:22You're spinning your wheels. And so it's really about all these metrics throughout the process. And that also includes like how long does it take to schedule candidates? Where are candidates dropping off? Why are we losing out on offers? Where are good candidates coming from? How many referrals are we getting? All these operational questions in the recruiting process is what we are really focused on today. And it's kind of surprising if you haven't worked in a larger recruiting organization, there's a surprising amount of complexity in actually scaling a team. yeah and your customer base are they small and medium are they big corporations or a lot of them are they tech companies or companies that are not in the tech space but are hiring tech teams yeah so we started more in tech because that's where there was the most competition for talent and so that's where our product got the most uptick because if you have a competitive process then you need more data to make decisions.

6:18And then today, really sort of, you know, one to above 10 ,000. So our bigger customers include Shopify or Snowflake, like large public technology companies. We also have OpenAI as a customer who's not huge yet, but growing extremely fast. And then we also have a lot of earlier stage companies to like ramp kind of probably in the middle of the pack. But then also, we have a lot of like five person startups joining us too, because they know that hiring and talent is really important. And they want to start with right product that can kind of grow with them over time uh and then we are expanding into non-tech as well but tech is definitely kind of still the bigger portion of the customer base yeah and and this platform is used by the hr departments at companies it's not a platform sort of a platform that candidates, you know, post their resumes to, and, you know, like Indeed or something like that.

7:16That's right, yes. So we use primarily by the recruiting teams within HR and then the hiring managers. But a lot of candidates know about us now. If they've been applying to a lot of tech jobs or AI companies, then they will see the Ashby brand out there quite frequently. But there's no candidate, the candidate only interacts with the company through ashby but they don't have their own profile or anything like that that's right yeah uh because it yeah i mean the the question is really i mean managing the process is one thing but finding the candidates is increasingly difficult isn't it what do you do is there anything in ashby that that helps uh hiring managers or hr managers identify candidates or is it once they have a candidate, then Ashby takes over?

8:09There are a few different parts of the product, but in general, we support what is called inbound channel. So all the people that are applying to companies, so Ashby helps post these jobs and then collects all the people that are expressing interest in your company. Then there's also other modules. We have a sourcing module. If I'm interested in your background, you're not applying to my company, we can find your contact information. We can send you an email outreach and try to compel you to interview with us. So we also have an outreach piece. In terms of finding candidates, most of our customers in technology are kind of using LinkedIn as a primary place to identify people that they want to reach out to if they're not already coming inbound.

8:47But the interesting thing in the job market over the last two or three years in tech is actually a lot more people are coming inbound than it used to. It used to be a difficult channel, but a lot of our customers are now making more than half of their hires from applicants that are coming into Ashby. and for applicants that are coming into the platform we do help them use free ai i'm actually identify kind of the potentially best fit talent in what is increasingly a bigger pool of applicants so we do help with that part yeah and the applicants i'm and i'm interested mostly in in the ai sector yeah applicants coming in through whatever channel are they primarily people looking to change jobs or are they people coming out of master's programs or even PhD programs who are looking for jobs?

9:41Yeah, we have a mix of all sorts of talent, depending on the strategy of each of our customers, but it's usually a blend of like tenured people and internships new grads um so it's a pretty broad range yeah and is there something uh on ashby that helps a hiring manager evaluate the candidates or is it is it simply sort of tracking them through the process i don't mean simply but yeah yeah i totally get your point um so uh where we have deployed ai move and then we're going to go a bit deeper in that but one of the key areas we launched a little bit over a year ago now is at the kind of resume review stage, we can help the team identify the potentially ideal candidates.

10:26And so one example is I'm hiring product managers for my team today. I actually want people to have a product background. Ideally, they were a founder before and have a software engineering background. That's like pretty hard to find combination. And the last time we posted a role, we got about 2000 applicants. But with the features in Ashby, I can actually describe these criteria. So I can put in these things I'm looking for, and then we'll have LLMs read the resume, and it will indicate whether or not these skills are present. So instead of reading every resume line by line, I can have AI help me scan the resume, find the relevant skills, and then let me filter through the applicant pool that way.

11:02And that's something that really wasn't possible pre-LLMs in a reliable way, but has really changed how a lot of companies use this inbound channel, because Now you can sift through like thousands of candidates more efficiently. And so we help with that part. Once you start interviewing someone, we believe that is something that requires human judgment kind of almost all the way through. We have a lot of like features that help manage the process, but we don't really take the judgment out of the process. We think that's human's job for sure. Yeah, that's interesting. I just saw an article, and my wife and I have talked about this a lot because, as I said, you know, we have a generation through our kids and nieces and nephews who ask for advice.

11:51And, you know, I'm sort of old school. I tell everybody, the resume should all be on one page. And I saw an article recently saying that that's bad advice. because all this stuff is getting read by uh by llms now and what what is the case i mean can you send in a five-page resume is the llm going to scan it is it to the point now where the more information you have the better for that's a great question yeah um i definitely think you want to have your key accomplishments on that resume because um otherwise it's hard for for anyone to identify them. That said, in our case, our customers really do a combination of this AI-assisted first pass of trying to figure out which resumes to look at first, but then you also still have a human looking at it.

12:45A five-page resume is probably not optimal either. So probably one to two pages is still good if you can bring in your key relevant skills for that job. But if you have closed$100K deals or you have shipped code in a certain language, you to have these details on the resume because that will help um in case people are looking for these skills so but it's definitely changing yeah you have to kind of optimize right now for the machine and the human kind of at the same time ideally yeah yeah um and and and uh i guess the llms these days are smart enough uh that it's not simply picking out keywords it's uh it's understanding what's being expressed on the resume.

13:31That's been an exciting change, I think, where recruiters were historically pretty good at keyword search, but keyword search is very literal. And so if someone used a slightly different phrasing or an adjacent skill, they would be missed in this kind of pile of resumes. And now the LM actually has this kind of semantic understanding, which we've seen really helpful in terms of accuracy. And it's one of the areas that LM's are actually really, really good at. So that's why we decided to ship that. Yeah. And, you know, a lot of jobs I hear, you know, there'll be an entry level job or even more senior job.

14:10And companies will get 10 ,000 applications, partly because now, you know, you don't have to lick an envelope and find a stamp. I mean, you can do it all online. so people sort of flood the zone. What percentage generally, is there some metric, what percentage of resumes make that first LLM pass? Yeah, that's a great question. I wish I had prepared a step on my end because I actually have just data on our trends report. And the way to think about it is our customers don't actually have the LLM do the first pass and then human is actually side by side. So let's say we get 2 ,000 applicants. we look at every resume but we start with the ones that seem like the strongest fit we go for them first with the mindset of like these are probably people who want to move forward then we see the bottom people that don't seem to meet most of our criteria to me just mass applied they're not actually a good fit for all and we can go through the mindset of these are probably not a fit so just speeds up the human review but it's still always human going through and making a decision on each of the resumes um the interview rate um that's a good question i think it probably have somewhere between like one and 10 um on average it really depends on the volume of interviews but it's kind of roughly what you can think of and you did touch on a point there where we are seeing more tools on the candid side too that make it easier to mass apply to jobs and that increases the numbers but doesn't necessarily always mean there's a great fit so yeah that's right yeah yeah it's probably a good benchmark yeah uh and and when you say that it's done side by side it just seems i've never hired at that level but it seems that uh you know if you get a thousand resumes in you would just shoot them all through the llm first and at least have them rank them is that what ashby does or we can rank them but we don't do like what companies do in our space which we shy away from is we don't say like you're an 87 out of 100 and you're a 52 out of 100 instead we take these objective criteria that you defined and we just say like could we find on the resume yes or no um and if yes where's the evidence for it and so it's really like an assistant reader that's very objective based on the criteria that you set up which is a little different um yeah we think it's kind of right way to go about it yeah and then uh uh the the the humans then go through the top 100 or something if there are 10 000 resumes come in yeah it depends on the team there's a lot that go through every single one but if you have the system to read resume can take like a second or two just validate like okay this person they applied for a software engineering job but they actually never worked as a software engineer there and you know you make a quick gut check and then you move forward but the different approaches we do generally recommend that humans will look at every everyone um but yeah yeah um so how has this uh industry and ashby in particular how do you think it's changing hiring practices is it making it easier to find uh the right fit uh I mean, is it, you know, it just seems like there's, you know, on the one hand, there are a lot of people that have, again, I'm talking about AI, that have gone to undergraduate or boot camps or something that have a feel for AI that can code.

17:51and those are the kinds of people who are not finding jobs and that people like Meta are laying, or Microsoft are laying off, right? Because a lot of that can be automated. But at the top, there's kind of a shortage of talent. That's why you have talk of$100 million annual salaries. Yeah. um is uh where do you see the market i mean are is it is that top tier talent growing at all and if it's growing where is it coming from is it international students sort of i mean i'm talking about the u.s market primarily we can talk about international but uh is it uh you know people coming into the country is that people coming out of graduate programs and is there enough of a pipeline to meet the need uh and what what do the the the people that went to boot camps expecting to have a high-flying career uh what do they do it's a great uh set of questions and i'll try to take them one at a time i think the interesting thing about this narrative of entry-level jobs and AI and automation, I don't think it's perfectly accurate.

19:15I think a big part of what we saw in the last years is kind of a readjustment of the 2021-22 period where a lot of these tech companies hired like literally tens of thousands of people in a single year. And they pulled so many, they pulled so much talent into companies that are just overstaffed for the size they should be at. And that's even outside of AI or automation. There's like, if you talk to someone at Google or Microsoft, there will be stories of teams just being too big to actually move efficiently. And so I think AI is like a good story of like why these redundancies are happening. But in practice, it's often just like we hire too many people and we need to right size the business.

19:54So I think that's a big part. Then there is definitely still a really strong competition for talent, but it is more at the senior level. There's just a lot of supply of people with zero to two years experience. So I think that's a challenge right now with this dislocation we had in the market. We're adjusted from this big overhiring, and now we're pulling back. Where people are pulling back the most is at the entry level, where people that join at that level, they aren't that productive. They're not that accretive to the business. You invest in them for future growth, but if you can also just hire senior talent instead at a higher price, then that's kind of what companies are opting for a bit more.

20:32In terms of practical advice, I think startups still have a hard time to hire talent, especially early stage ones. So if you have any hands-on experience actually building something, even if that's on the side, and you can demonstrate that, I think that's a great place to launch a career. There are a lot of scrappy companies out there. They're looking for people that can actually build stuff. So especially if you have coding skills, putting projects out there, side projects, posting about them will give you a good amount of visibility. Yeah. Yeah. And that top tier talent, is it coming out of graduate schools or is it coming from overseas?

21:09There's a mix of all sorts of channels, but I think in general, there's still a big shortage of software engineers that are meeting the bar for like a senior software engineer at any technology companies. Every company, every startup, every founder I talk to has a hard time hiring as many engineers as they'd like, but the background can be all over the place. I mean, there are obviously a lot of the traditional you know stanford grads or however grads but when it comes to software engineering um it's much more about the practical skills and there are a lot of people that have an untraditional background um and just have demonstrated that they can code on the side i may never even completed a degree so there's no real there's no real strong pattern there um yeah yeah and and does ashby uh i mean what do you do when someone submits a portfolio for example or is that later in the process tends to be later for design roles it may happen early but it's part of that human review so you know the the lm again will read resume we'll try to find if you have like the key criteria and then even we'll take a look at that as kind of the first step i think asked this other question earlier that I also wanted to touch on of like how do we help companies kind of solve these talent problems or get better at hiring and a huge part of it that I touched on early on is like this data piece where you know we help companies understand their process better and optimize it and that can include things like you've hired you know 10 software engineers in the past here's how many applicants it took you to hire a single person so if you post this job you know roughly how many applicants you need before you will make one hire then you can use that to decide how long you want to leave this role open.

22:50Then you may look at your process and you notice that you have a drop off at a certain stage. And then you can take a closer look at that and see if these particular interviews are too hard or if they're interviewers that are grading too harshly. And so a lot of the optimization that Ashby offers is around this kind of data and data analysis that helps you refine your process. Yeah. Yeah. That's interesting. And how often do people, I mean, again, from your data and analysis, hire the wrong person? Is that a problem? I would guess it is. That is certainly a problem. And it varies pretty highly.

23:30We actually don't have, like, perfect visibility into that because we do end at, like, the hire process. We did roll out something about a year ago. We call it quality of hire surveys. And that is something that will trigger a survey to the hiring manager at a cadence that you can choose internally. We do a 30, 60, 90 days. So we started to collect this post hire data and we can see like where better people coming from, how does it correlate with interviews? But yes, people definitely hire the wrong people, you know, interviews are highly imperfect. But that is a big area that we are helping our customers on is like collecting the data post hire.

24:02Then again, understanding which interviewers or which interviews are more predictive of performance, which channels tend to produce better candidates. you can help with all of that. But yeah, it's a big problem for sure. Yeah, and the channels, what are the channels? Yeah, so inbound, broadly speaking, is one. And then within that, you can have different job boards. It could be LinkedIn, Indeed. Depending on your industry, some of these will give you better fit candidates. And then outbound is another one. So again, if we have a team that's going to reach out to candidates that are not actively applying and interviewing and we source them, that's an important channel.

24:38referral is the other channel that tends to be really important early on where you're asking your employees to submit people that they have worked with in the past and obviously have like some pretty deep knowledge of their skills and then internal mobility is another one and that's actually increasing for our bigger customers especially at a time when they aren't hiring as much externally um you see more people move internal um into different roles uh and then if we look at our data internal mobility is obviously one of the best channels so you already know you already know so much about this person um based on your organization um and then referral can be another generally good one if you have a good referral program in place because if you have strong talent on the team then the chance that they know strong talent can be quite high yeah that internal thing is interesting does i mean organizations are big uh it's does it typically happen that one department will post a job company-wide and people will apply or or do you can you scan an organization and say uh you know here are people with the proper skills here's their current job yeah we could replace them with a cheaper person and move them over to this i mean And are there tools to do that?

25:56Yeah, that's a great question. So today is definitely more deformer where people are applying and expressing interest in roles. But we are rolling out AI functionality to generally find past talent that you've interacted with in Ashby. And so if you think about our customers, they've hired each like a couple hundred or a thousand people, but they have sometimes millions of candidates that they have some touch point with in the past. And what we can do now with AI is resurface these automatically. So if you're opening a new job, you can say like, hey, you're all these people that you've interviewed before or hired before, or they expressed interest in your company that you didn't interview and revisit them.

26:31And that's something that's pretty exciting because it's something companies have been trying to do for many years, but pre-AI, it was really hard. And the research kind of started from scratch, but now we can use all this historical data to service people, including internal candidates. That could be a good fit. Yeah. Wow. And so I would guess you saw the opportunity, you guys jumped into it, but a lot of other people are jumping into it. How do you see the market and how are you growing? Yeah, we are more than doubling year over year for the last couple of years. I think can pretty confidently say that in our space, we're probably not a fastest growing vendor.

27:12A big advantage we had is coming into the space in 2018. When again, all these other solutions were five, six years older, and the talent market really changed in especially in software around like 2015 or so, there was like much more private capital. the competition again for like senior engineering talent really exploded the number of startups exploded and so we really built for this new way of acquiring talent and these more sophisticated recruiting teams that work with data and so we have a pretty big leg up I think because we had we were lucky in the timing and we had it inside a couple years before everyone else and then we have been able to build this all-in-one product so instead of buying like a lot of different solutions we have a single platform for our buyers to kind of run this entire process and that especially useful now with ai where we can use all these different touch points it can be we're about to launch call recording so if you have conversations with candidates all of these are recorded all the emails that you're writing all the feedback forms all the notes all this data is in a single platform and that creates some really valuable data that's hard to replicate so yeah yeah yeah that's interesting and so building the product out I mean you're doubling every year do you have to is there a lot of pressure to keep expanding the product and what's the vision where do you see Ashby going yeah it's a good question I mean our fortunately our customers are really happy with the velocity at which we're improving the product and shipping features that's something we focused on pretty early both my co-founder and I have an engineering background and I spend a lot of time hiring engineers so we were able to build a really strong engineering team um the so people are really happy with what we have today we do definitely have quite ambitious plans to expand into areas that we're not quite talking about probably yet but um uh today we're really focused on on this kind of recruiting challenge and especially with ai there's a ton of opportunity to rethink a bunch of areas of the product that we built a couple years ago pre-llms So that's kind of our immediate focus.

29:16And is it getting to the point that the hiring process, all of the sort of tedious work is being automated, and so hiring managers can really focus on interviewing and the candidates? or has that always been the case? It's just taking care of by back office people. Yeah, it's definitely not always been the case. Part of my reason to start Ashby was when I was in the hiring manager role, about half of my recruiting team was spending all their time coordinating interviews because there can be pretty complex interview loops. We used to do like four-hour onsites, and we need people with skills and need to go in a certain order.

30:07And so we had a recruiting coordination team where all their time was spent just scheduling interviews. And that was one of my insights back then was like, hey, I think computers are really good at organizing calendars. So let's try to automate that. But it's a pretty recent thing that companies have really invested in is like automating the more complicated parts of the process. And we're definitely seeing more of it, but we're definitely not fully there yet. But the goal of Ashley is to get to the kind of core competencies, which are how do you make your company attractive to people such that they want to express interest.

30:40How do you evaluate candidates because we think that requires humans and then how do you close them once they have competing offers? That's really where we're kind of focused on and we're trying to eliminate everything else. But even things like scheduling interviews today, we can automate 80%. You know, with LLMs, we're going to try to get it to like 90, 95. But some of these problems are incredibly hard to solve in like a real world setting um so there's still a good amount of opportunity left yeah that's right i mean things always sound easy in theory but in practice there that's also the unintuitive thing about ai um how lms work is like you see the news of like you know models solving some math olympiad problems but they can't actually be an executive assistant because they don't have all the context necessary they're they're kind of smart on unknown problems they've seen before but not very smart on things we need a lot of context so yeah and actually that's a that's an interesting point is does ashby do anything to build context yeah you know maybe the personality profiles of the people on the team that the candidate would be joining or something like that now today it's really more about all this data we collect and we started making that more accessible.

31:58So we now have this thing we call candidate assistant. So if I go to your profile and actually I can ask any question, the way I would ask Chachapiti a question, and we'll pull from any kind of data source we have that can be past conversations, feedback, anything. And that's been really helpful because I can go in and be like, okay, you know, why is this candidate left their most recent job? If I was an interviewer asking a question, historically it would have been really hard to find it out. Now I can get an extra transcript with the conversation. And so we're collecting all this data and then making it available to AI and to the end user.

32:32And there are more and more workflows where we can start embedding that kind of stuff. And then another thing we launched a couple of months ago, which is really exciting for candidates, a lot of people have experienced interviewing somewhere, maybe going for three to four rounds of interviews and then getting a very generic rejection email, which is like, sorry, we're not moving forward. Because it's actually time consuming and hard to give compliant feedback on a process, but we made it pretty easy by introducing a special email token you can include in your email, and we'll automatically pull all the feedback, we'll sanitize it, and we'll give you a nice version of feedback that a candidate can take home.

33:07And we're just going to do that in two seconds, where before it would have taken 30 minutes per candidate to compile that. And so there's some really interesting stuff we can do with all that data to also humanize the process a bit more yeah and and hiring successfully you were saying that uh one of the things that ashby does is sort of track when people drop out of the process why they drop out of the process what are the reasons that uh the hiring yeah process would would lose somebody yeah um so there are obviously two broad categories one is you decided not to move forward with a candidate and then the other is a candid pulled out of the process.

33:49For candidates pulling out, speed is a really important one. The best candidates are not in the market very long. They'll have multiple conversations. So it's actually an area where actually with the product and all the automation helps. We can bring someone through an interview process in like 20 days if you set it up really well. If your competitor takes 30 or 40 days, then you may have signed an offer before they finish the process of another team. So speed is like a huge area that we're focused on. Other things, compensation, companies are getting better now about posting that upfront, having these discussions early, but that can cause misalignments and waste a lot of time for you or misalign the compensation at the last step of the process.

34:25And when it comes for you rejecting candidates, what we really try companies to focus on is try to eliminate candidates as early as possible because it's going to be more efficient. What you see in miscalibrated interview processes is a lot of people make it through all the way to the last round and then most candidates drop off there. that means your team has wasted hundreds of hours interviewing candidates they didn't actually hire so these are all the knobs that you can kind of tweak in the process yeah um and and you know as the economy becomes increasingly uh digital presumably demand for this kind of thing will grow how do you see the market developing yeah i think yeah the way you can think about it we are benefiting most companies that are either hiring competitive talent or are hiring at a large volume.