How LLMs Are Reshaping Recommendation Systems

0:00Newsfeeds and recommendation systems have long relied on deep learning architectures that score each candidate item independently. As LLMs have matured, they have opened up a fundamentally different approach, where a system can reason about content the way it reasons about language. However, that power comes with a fresh set of engineering challenges around cost, scale, and evaluation. LinkedIn recently rebuilt its newsfeed to treat content recommendation as a sequence modeling problem. The general approach is to predict what a user will want next, much like an LLM predicts the next token in a sentence.

0:38Tim Yerka has worked at LinkedIn for 13 years and is currently a VP of engineering. In this episode, Tim joins Matt Merrill to discuss how LinkedIn re-engineered its feed, how the team combines LLMs with traditional signals, managing inference costs at massive scale, steering content quality using natural language policies, and more. Matt Merrill is a software engineering leader with over 20 years of experience building and scaling software teams across enterprise and product-focused organizations. His background is in backend development, cloud architecture, and distributed systems design. He currently architects and delivers software products and leads a team of engineers at Dept Agency.

1:22You can learn more about his work at code.theothermattm.com.

1:39Hey, everybody. I'm Matt Merrill, and I am here with Tim Jerka, VP of Engineering from LinkedIn. So today we're going to talk a little bit about what LinkedIn's doing with revamping their social feed with AI. But before we start, Tim, can you tell us a little bit about yourself and your current role at LinkedIn? Thanks for having me, Matt. As you mentioned, my name is Tim Jerka. I am VP of Engineering at LinkedIn, and I work on our consumer products. So what that would be is things like the LinkedIn feed, search experience, profile, pretty much anything you will engage with when you open up the LinkedIn app.

2:10And I actually started here 13 years ago as an engineer on the feed. I was an AI engineer, one of the first AI engineers building the LinkedIn feed. So it's been pretty amazing to go from actually building the thing to now see the thing operating at just massive scale with 1.3 billion members. It's been quite a journey. So recently, you guys have rolled out some pretty major updates to how you're making the LinkedIn feed relevant to users. And I know that's welcome news to me. And it's probably welcome news to a lot of people who may not have been otherwise engaged with LinkedIn because of content that was irrelevant or they thought was annoying.

2:44So I'm really excited to hear about what you did and also how it intersects with AI tooling, which I think this audience will really like. So let's start with just what was the problem, right? Like what was the overall problem and why was it challenging? I mean, I can maybe start by rooting it in a problem of scale. Like we've, since just 2020, we've gone from about like 690 million members to 1.3 billion members. We've scaled considerably. And we have about seven new members joining LinkedIn every single second. And so with that scale comes a lot of challenges. Number one, you have tons of new industries coming on the platform.

3:20People in new roles, new opportunities. They're looking for something different from our platform. And there's a lot of people coming with evolving expectations of what they need from their professional social network. Like starting from 2020, I mean, the world of work has changed a lot. We went through a global pandemic. We then went through the entire like AI LM revolution, which we're probably ostensibly still in the middle of. And people are just turning to LinkedIn more than ever to get this information about like, how do I navigate this? How do I reskill? How do I learn about what's happening, keeping pace with things that are happening in my industry?

3:54And so the problems we had to solve were twofold. One is make sure that we could kind of scale to this very kind of granular topical matching and representation of the entire LinkedIn feed ecosystem and make sure we're giving people the right content at the right time with timely insights. The other is with seven new members joining every second, you have to do a pretty good job in the first few seconds of a person's session, narrowing down what they actually care about. And so we almost call this like the zero shot experience. And both of these things together are kind of challenges we were solving.

4:25Number one is like, you need to get that signal very quickly to help the people get the insights that they need within a few seconds of onboarding. And they're not necessarily like endlessly scrolling on LinkedIn. They're trying to get a job done. They're joining to find a job, to learn something, and you have to really quickly get them that information to make sure you're maximizing the utility of LinkedIn. For people who are diehard users, I mean, there's a lot of things that even you alluded to, like we started seeing the emergence of AI slop, engagement bait, like we needed fine-grained control to make sure that we are focusing on the highest value conversations, people who are authentic, verified, and making sure that we have the capabilities to keep the ecosystem steered in that direction.

5:05And so this new foundation not only kind of up-leveled the fundamental infrastructure modeling capabilities on retrieval ranking, it also gave us these fine-grained controls to solve a lot of these problems from like faster time to signal world awareness, being able to do the zero shot session kind of stuff, but also being able to steer the system to make sure that we're giving the highest value content to our members. So was the primary driver new user stickiness and a nice side benefit was existing user engagement or was it all of it? It was across the board because one thing that like you'll see is there's kind of two classes of content that we surface.

5:44You know, LinkedIn has historically been this very network-oriented product where like I connected with a bunch of people. But the people you connected with aren't always going to be the people that are giving you the cutting-edge insights for where you're trying to learn. And so you'll actually see in your feed a lot more what we call unconnected content, suggested content, that is supposed to be really topically oriented to like, hey, I'm trying to learn about what supply chain risk means in the context of like, you know, like anthropic and cloud models. Like, how do I go deeper on that? What does that mean for my company?

6:09Or an electrical engineer trying to learn about modular reactors, like try and go deeper there. So like you need to get deeper into where the subject matter experts are, even if they're beyond your network. And so it's not just solving the short term zero shot signal. It's also be able to map the interesting content based on where you're trying to learn much more quickly. Yeah. And does that also extend to your outreach, right? Like I know I get emails with content that I might be interested in from LinkedIn. Does it, does it also include that or is that outside the scope of what you're doing?

6:39It's in scope, but right now we're starting with rolling this out in the feed and we're iterating a lot there. This stuff kind of, I mean, it cascades across our products. Like this is stuff that we build out and make sure that we have this feature rich capability everywhere. You might encounter content on LinkedIn. Cool. So let's dive right into the interesting stuff. So like one of the things I was reading about, there's a really interesting blog article that you guys posted about this goes into a lot of detail, which is awesome. you're using sequential time-based interactions to drive relevancy.

7:10And this is not something I personally have seen much about in the AI space. So can you talk a little bit more about that? Yeah. I mean, the underlying premise is actually quite similar to how you would think about LLMs, which is LLMs are really just sequence models for language. They're trying to predict, given the token, what are the next tokens that come after? And then stringing together like a comprehensive sentence, paragraph, whatever the context window is. You can do the same thing with content in the feed or any kind of item in a recommender system where you're trying to predict sequences of items versus sequences of words.

7:45And so there's a lot of benefit of this compared to like the old style MLP architectures where like you might have heard of these like feature crossing, DCNV2, deep and wide feature crossing, where you're like, you dump tons of features into the model. You try to find all these feature interactions. That's then your like deep learning model. And every time you give it a new item, it weights all the features in that model and it scores the item. And it's like, here's the score of this one and this one and this one. And then you sort. Sequence-based models are actually saying, contextually, we're going to learn that once somebody has viewed a piece of content about modular reactors, then they're probably going to go deeper in direction X or Y.

8:22It starts like modeling out a sequence of where you would go next, based on all the data points that we have of every single expert on LinkedIn reading about any number of topics. And so it has a lot of benefits then of starting to go extrapolate, you might be starting at like an introduction to PyTorch, but like you go like 15 levels down the sequence, you might actually be trying to build like a re-implementation of like an LLM locally on your machine. And so like, can you reconstruct that sequence of how people would go down a content path and help kind of understand the engagement patterns from that vantage point versus de novo, like rescoring every single item, you know, just using all the features in the model.

9:02That is super interesting. And also just, we'll get into this a little bit, just mind blowing at that scale. If you can't answer this, totally understand, but did you build this on any sort of open source or more widely available model? Or is this like something you guys have completely come up with R &D from your own perspective? As with anything, we pull from, you know, the research that we see around us. There's a lot that has to be uniquely built for LinkedIn. And so it's not like anything off the shelf works for us. It requires us to piece together something unique. To my knowledge, this is like fairly unique at this scale across like this kind of the system of the sequence based models and ranking and then the LLM based retrieval.

9:38Like it pulls from concepts from Lodlager, including some stuff that we've built internally. In terms of open source, like we do leverage like open source components in what we build. And it's honestly through experimentation that we figure out where we have to build things in-house versus open source. You try lots of different frontier models. You try lots of different approaches. And you kind of see what converges to solve the specific problems that we have in our scale, our professional context, the kind of fine-grained controls that we need. It's tough for me to visualize what this architecture might look like.

10:10But I'm assuming you're using multiple models in different places. I mean, we're continuously. like if you think about the kind of life cycle of we have offline environments where we will train models using some interesting like an engineer has an idea of like adding a certain feature to the model or trying a different model architecture offline we can run like thousands of these tests offline and replay them on historical data and say like did we do a better job ranking items in terms of like engagement then we have to actually like bring these online to see like do the members actually like it like not in simulation but like in in the real world and so there we then like we will split up our traffic and we'll run hundreds of experiments while there so at any given time we're running tons of tests to try all sorts of stuff and you may have seen some of this already you know publicly like making sure that we're trying to focus on like authentic voice conversations making sure that we're not you know rewarding ai slop making sure that our suggested content is really well targeted to interests and being able to better model member affinity to different interests.

11:11Like all these things are variations of tests that we're running at any given point in time. And what's the engagement signal? Is it clicks? Is it time? Is it all of it? Yeah, so as a standard, you know, in industry, it's what we call a multi-head model with a multiple objective optimization. And so it's kind of a little bit of everything. You want to make sure that you have predictors for different ways that people will engage with things. That means positive predictors of like, are you going to be commenting on this are you going to be resharing it are you going to be spending time on it also negative predictors are you going to skip it are you going to be like is it going to be in a viewport for a second just pass over it that's like a negative prediction that we try to do as well and you combine all these together into one multi-objective optimization to actually then ultimately optimize for i mean for us it's really legitimately trying to get the most interesting conversations in front of people that we feel like it's gonna be time well spent but it's a combination of all these factors that then ladders up to that.

12:08That makes a lot of sense that what defines engagement is a very complicated thing in and of itself. So yeah, using these models to help track that makes sense. And maybe to like illustrate it even more, the weights of all these different actions are personalized on a per user basis. If you're somebody on LinkedIn, who's like in marketing, like you're trying to get like follower reach and comments, like you're trying to achieve something different than somebody who is prepping for their job interview is like, I just want the content that's going to show me how to like nail the job interview at LinkedIn, right?

12:39You're not going to be looking to get like a post and to get comments and like reach. And so you have to adjust the model and how it works depending on the actual intent of the person using the model. And it's literally by user. This is we're not talking about roles or personas or anything like that. It's very unique to each user. It's fully personalized. Yeah. Wow. That is awesome. So let's double click a little bit into that evaluation. That's something that I have personally been, I am not working at nearly the scale you are, but I have been doing a lot in my job is just how you define what is good in this space.

13:14Right. And we just touched on that a little bit. So I think the thing is like over time, how does this happen and who is doing this, right? Like I can't imagine, well, I don't know, maybe I'm wrong, but I can't imagine they've got armies of you, you know, PhD trained in AI doing this. So are these engineers, are they product people? And like, how often are you retraining at full scale versus just incremental improvement over time? I'd love to hear more about that. So in terms of training, we incrementally retrain constantly. So like we're pulling in the latest data as soon as we get it, retraining, updating the model.

13:51The more interesting part is the evaluation of how you're actually sure that this stuff is good. And I mean, there's a number of ways you do it. There's the more traditional ways you eval, which is like A-B tests, and you look at your kind of like metrics and you're like, okay, what happened to A-B? be. But for something like retrieval, where you're using LLMs to actually retrieve content that is relevant to a person, you actually want to be looking at some sort of, we call them policies, right? You define a policy for what consists of high quality content. This is something that's defined by our product managers.

14:22And these can be evaluated using people like, you know, we've called tastemakers, people who are like really well attuned to the product strategy and where we want to take things and make sure that content is timely and insightful, et cetera. So you'll have like a timeliness policy and like type of policy, but also you can start encoding this into like auto eval. You can have GitHub copilot CLI essentially doing revs of like, Hey, I looked at a bunch of content that was surfacing retrieval. It missed the policy in these places. Like, do you agree with me or do I need like some feedback to like iterate on, make sure the policy is reflecting what you intend.

14:57And so you get into that kind of like active learning loop. And so we do both. And it's been pretty incredible to just be able to have that capability. It gives you just much more sense and fine brain control in terms of, are we actually doing what we aspire to do? Like, is this actually timely content? Is this actually interesting for this particular audience? You know, in the old days of ML, you kind of hoped that that was the emergent property of your system. You know, like you put out a model and it's like, we optimize for this thing. And we hope that like it surfaces interesting stuff now you can actually kind of validate it you can say like you know for people in nuclear engineering are you actually surfacing stuff that is relevant to their industry or is this something that is either like general purpose humor or something that's not as fine-tuned and so you can get into that level of granularity and it's sounding like it's kind of happening in in parallel almost yeah which is wild it's almost like an organism Awesome.

15:50That's very interesting. And about the who, you said it's product managers, tastemakers. How are they inputting that signal in to correct that? What does that look like from a user experience? I mean, there's a number of ways they will directly edit. I mean, at the end of the day, it's a text file that's checked in to GitHub, you know, and like they've written a policy almost you can think of as a prompt. We've built out or agenda tooling around that. So like they can iterate on that prompt by getting some examples from GitHub Copilot CLI and it's like, hey, here's a few examples. Here's what I think I would score them based on what you told me.

16:28And they can be like, no, you missed the point on this one. And the Copilot might then update some of the words of the policy to make sure that then it's reflecting kind of the intent of what we're trying to do. So this is a very engineering centric, like CLI based process. So your product owners are very technical. Everything is version checked and like, you know, yeah. Wow. Oh man, that sounds great. So we can see live over the course of a week, places where we've evolved our opinion on like, you know, what constitutes inauthentic content, like people giving kind of AI slop style replies, and we can fine tune to make sure that we address that.

17:07You're building agents that can write code, summarize documents and automate workflows, but they're missing one thing, awareness of the world around them. X-Weather combines enterprise-grade weather intelligence with agent-ready APIs, natural language capabilities, and an MCP server built for tools like CLAWD, Codex, Copilot, and modern IDEs, so your agents can adapt workflows, automate responses, and make better decisions based on real-world conditions. Backed by Vaisala, whose instruments fly on NASA missions to Mars, X-Weather delivers trusted data and unique insights that go beyond conditions to actual impact, from real-time lightning strikes to road surface forecasts.

17:43Start with 15 ,000 free API calls every month and pay only for what you use as you grow. Your full weather stack for developers by developers. Start building for free today at xweather.com. This episode of Software Engineering Daily is brought to you by Warp Build. AI is writing more code than ever, which means GitHub Actions is running more than ever. Your GitHub Actions bill is now a function of how much AI code you generate. And every engineer knows the feeling. You push a commit, and then you wait. Warp Build makes GitHub Actions twice as fast at half the cost, with a one-line change to your workflow.

18:18Linux, macOS, and Windows Runners, in Warp Builds Cloud or your own, enterprise-ready, SOC 2 Type 2 attested, and trusted by teams like Sky from Comcast, Bitcoin, and Braintrust AI. Get started with$50 in free credits at warpbuild.com. You're shipping faster than ever with AI coding agents, but those agents don't vet the packages they pull in, and they don't have security context built in. Ori by Endor Labs fixes that. It plugs directly into your editor via MCP, catching vulnerabilities, blocking malicious packages, and flagging exposed secrets in real time. No separate tool to switch to, no dashboard to babysit, security that fits how you actually build.

18:56Teams using Ori see 10 times fewer security tickets and six times faster fixes. Free for developers. Get started at www.endorlabs.com slash A-U-R-I. And I'm very curious about like the day-to-day for an engineer in here, right? So I mean, even your product people are working in like a code-based workflow. So what is like an average engineer doing in this environment? because I think that's where a lot of people are wondering, like, okay, how do I fit in with my existing knowledge? I might not be a PhD in AI. How do I fit in and start to prepare myself for what this world might look like? There's no average engineer.

19:38That's fair. Let me start with the people who really are moving 1 ,000 miles a minute on this stuff are people who are generalists who can traverse more parts of the stack. So they not only understand how the AI models work and how to fine tune like an LLM to distillation. They also understand how to integrate that into like the back ends of the infrastructure. Maybe they even know how like prompt engineering works and like all that kind of is a super powerful skill set for people to kind of cross over the boundary of decision-making into like, what does this product actually look like? However, that hasn't negated the role of having some specialized skills.

20:15So to get an ML model to production in front of this many people, you need to make sure that you have nearline pipelines that are taking all the tracking events from the client and like propagating them in nearline to all these feature stores to make sure i know this particular post by you know jaylen bronson was like you know like the 25 ,000 time in the new york metro area and these metro areas which is all in real time being updated to then know where we target this stuff so like all these like nearline signals need to get adjusted to the You then need to run this massive model in inference on a GPU fleet.

20:49And so you need inference engineers that are optimizing the QPS throughput of the stuff because we don't want to overspend and buy every NVIDIA GPU out there. You can do a lot of things to increase the throughput of these systems. Sometimes you can offload some of the stuff to CPU workloads, keep it in the GPU workload. You have infrastructure engineers looking at that. You have AI engineers that are training and deploying. and a lot of engineers that, like I said, are generalists that are then making sure that these things all kind of work seamlessly together as we ramp to our members. And so the average engineer is somewhat of a misnomer because you still need a multifaceted team.

21:22And we almost think of it as kind of like these pods of teams where you might have like a product manager that's helping shape some of the taste and you have a designer that's designing the feed card and infrastructure engineer and a backend engineer. And so one of these pods might be like working on a particular model end to end and bringing it to like the members. That's fascinating. At a system of this scale, how many pods and how many people roughly on a pod? I mean, it was a couple hundred engineers to bring this to life, but like not everything's conducive to like these pod style. Like you have some deep infrastructure optimization work, but like where you're moving fast on a new product idea and like some way to like improve the feed, that's where like these pods can move really fast between like an idea iterating on it, deploying it and having that closed kind of loop ecosystem where the team is making their decisions and running with it and using agentic tools to develop.

22:11That makes sense to me. You need the generalists to be able to react quickly, put those puzzle pieces together fast and get it presentable to the user, so to speak, or even out to the user. And then you need those really deep specialties to be able to tweak the fine details to make sure it's not too slow, that it's not completely irrelevant, that it's not costing too much. Yeah, that's really interesting. It sounds like that's a lot of people to be able to get this done as opposed to, No, we actually did it with less people, which maybe it's just me wanting to see this through rose-colored glasses and save our industry.

22:45But I'm curious what your take is there. I mean, these are like, you know, systemic shifts that we've been working on for like 12 to 18 months and like iterating our way into this. And you're shifting from one model architecture and infrastructure to like this GP fleet. Like that does take build out. Like there is the upfront build out. Our infrastructure team is phenomenal. They work on lots of things beyond this. Like they've been working on feed in like heads down mode for the last couple of years. But that's not the only thing they do. Like once we got the build out going, you know, we are running and operating this with a much smaller team and focused on like using the platform to build interesting new features and getting interesting content in front of members.

23:21Our infrastructure teams are then moving on to like other interesting building out of like agentic platforms and they're kind of reading the next wave. And so it's not like each of these is a permanent large investment. You got to do the build out to then leverage this platform to do interesting things and build compelling experiences. Yeah, the investment's got to come in there. Yeah. So let's go back to scale because the scale is just absolutely incredible. And I think that there's kind of two things that I'm thinking about. One is just how is this architecture inherently set up to scale just from a conceptual perspective?

23:55And then there's also the hardware infrastructure part of it, too, which you've kind of hinted at. And I'm curious if you can describe a little bit of like how the interplay and how you thought about that to be able to create something of this scale with this cutting edge technology. When we started this, it's kind of like try to do everything on like GPUs and like see the best experience we can create for our members at the end of the day. Like you start there and you quickly realize, well, this isn't going to scale. And so that's where you find something that's compelling at small scale, test to 1%, 2%, 5 % of members and say, okay, there's something here.