Mastering Developer Experience at Shopify | Eytan Seidman, Director of Product

0:00It's really critical that that champion that I mentioned has a strong perspective on where this thing should go. and is really the person who should be taking charge on driving what might be very controversial decisions and not just kind of agreeing on, oh yeah, we'll just take some of this and some of that. And so that champion role has a really critical thing, which is like, hey, I have a certain thing I have in mind. Here's the outcome I want to achieve. Here's why that outcome matters. And I'm going to work like heck to make sure it happens. Is seven day cycle time good for your team? What about 150 lines of code in a pull request?

0:35Visibility into these metrics is not enough. You need to see how you stack up to overall industry performance in order to actually understand and define what elite performance looks like for your team. If you want a holistic view of your engineering performance, download Linear B's 2024 Software Engineering Benchmarks Report. Based on an in-depth analysis of more than 2 ,000 dev teams and 3.6 million PRs, Linear BeerBee's benchmark report offers invaluable insights you can use to build your improvement strategy. Head to the show notes to get your copy today. Thanks so much for joining me. So we are back with another in-depth episode of Dev Interrupted.

1:15I'm your host, Conor Bronson, and I'm delighted to be joined today by Eitan Seidman, Director of Product at Shopify. Welcome to the show, Eitan. Thanks, Conor. It's great to be here today, and I've been looking forward to this. Yeah, Eitan, you have an amazing background here. You've been a founder and VP of product at Oyster.com, which was acquired by TripAdvisor. You're ex-head of product at Compass, and now you're doing some amazing stuff at Shopify. And I want to hear all about what Shopify is doing for developers. You're a new additions.dev. But first, we should give our listeners some background about how Shopify is approaching development.

1:50Pretend I'm an engineering manager and, you know, like many of our listeners, maybe I'm looking for a new role and considering taking a role at Shopify. What sets your engineering and product orgs apart? Yeah, it's a great question. I'll give you the slightly long-weirded answer, which I definitely give to candidates when they're interviewing, because I find it's the best way to answer it, and I think it works pretty well. So I think first and foremost, it starts with being a mission-oriented, founder-led company. And that really cuts through a lot of BS on its own. So I think all companies have missions, but I think Shopify is probably better than most in a lot of ways.

2:25Our mission is to make commerce better for everyone. We're here to really build phenomenal software for anybody who's looking to get started selling online or really anywhere in the world. So it's kind of first and foremost as being mission led, founder led is a key part of it. The second thing is we hire people who are well aligned to that mission and who are really awesome at their craft. So at the core, it's a company for craftspeople, right? If you are amazing at your craft, whether it's design or UX or product, data science, every crafter is led by a manager who is also a craft and excellent at that craft.

3:00And all the way up to Toby is really the case. Let's get a second thing I'd say. It's also like a very high agency, high context culture. It is not a culture where there's lots of process, there's lots of bureaucracy. We try and really limit the number of like policies we have in general or anything of that nature. we're really treating people like adults, counting on them to show up every day, do their craft really, really well and to come with ideas. And then lastly, it's a culture that really thrives and expects people to think from first principles. Anytime we approach a problem, we try and understand, okay, well, what is true about the world today?

3:40And what are kind of key principles that are going to guide us as we go out and build this next piece of software or go and build this feature? And so I think when you take all those things together, right, it's like it's a mission company. We care a ton about having phenomenal crafters who are really great at what they do. We give them a lot of agency, give them a lot of context. It creates an environment where people can do phenomenal work, where you can come and say, hey, I want to go and fix this thing. And here's why it's really important. And here's why it matters to Shopify. And we go off and do those things.

4:10And I would concur. I think that's a really exciting thing as someone who's coming into Shopify's engineering org. I think, oh, I can be impactful here. I can have a high degree of agency. But I can imagine when people onboard, there are also challenges that come with this. If there is a lot of context within the culture, that means there are a lot of things you need to understand to do your work well. It means you have to adjust to the approach the team's taking. How do you make this work for new hires? Yeah, it's a great question. It is definitely one of the challenges. And one of the things that people ask me very often, I actually did an interview this morning, like, you know, what are some of the challenges for folks joining?

4:46And one of the things is that there is a lot of context that you have to understand about commerce and about the product. Shopify, at the end of the day, is actually really one product, right? We're not like a multi-product conglomerate. We really have what is a single product for the most part, which is a Shopify admin and the storefront that helps you control that storefront, all the apps that go along with it. And so there is a lot. I think one of the things that we do is we tell people like, hey, don't worry actually about contributing anything of meaning in your first, say, 60, 90 days. What we really want you to do is to learn and to really understand the code base if you're working engineering, to understand the product design, to understand all the product decisions that have been made as much as possible.

5:32So that as you go about doing things, you understand kind of what you're doing. Now, we do get people up to speed. We've got ways to do that, checking in, basically pull requestings of that nature to get people going. But part of it is just giving space and opportunity for people to get that context. Yeah, it's a really interesting one because I think a lot of software developers coming into a new role at an organization like Shopify. Yeah, they do want to get that first PR and whether or not it's a highly impactful one. They do want to get a sense of like how does our process to actually getting code in the database work.

6:06I'm curious how this meshes with this deep like product led ethos, founder led ethos that clearly infuses Shopify. how do folks kind of figure out what to make an impact on post that initial learning period so typically it depends on on who you are but like if you're a typical kind of crafter working in software engineering um what you'll do is you'll either glom onto a project that uh your lead might say hey i think you can be really impactful on this project here's why it's a great one for you to learn about the area uh and then as you kind of you know spend more time in the product, you're going to start to have your own desires and things to work on.

6:47And so the nature of it is like, you know, a good chunk of what we do is kind of, if you will, comes from top down strategy, but a good amount also comes from just bottoms up, hey, here are projects that we think are critical to building out the infrastructure of Shopify. We invest a ton of our engineering cycles in just improving the underlying foundation upon which all of our products are built. And we're actually, we're huge believers in that, that in order to build great features, we have to have great infrastructure with us, our database infrastructure, our API layer, you know, API rate limiting technology, all those sorts of things are kind of critical infrastructure pieces.

7:27And so it's kind of a combination of like, hey, some of it's coming from your lead. Some of it over time, though, we expect to come from the crafters themselves and to say, hey, I want to go work on that thing or I want to go champion that thing. And so one of the things that we have is every project has a champion. And like I said, sometimes it probably starts off, you're working on a project that somebody else is championing. And then over time, you're graduating to become a champion yourself. It's really interesting to hear how much of this work is kind of bubbling up from the crafters within Shopify.

7:57Do you have a breakdown that you track of what that percentage is of, hey, this is tops down, tightly aligned to business priorities versus, hey, we're discovering these are things we want to work on? It's not something that we track on an ongoing basis. The observation I've had, though, is that as the organization has become, so I work on our dev platform, which is helping enable developers to build applications for Shopify. We've been in kind of in this formation, if you will, for about four years now. And one thing I've noticed is that as that organization becomes really high context, really understands the strategy that we're trying to achieve, there is this kind of meshing of top down and bottoms up.

8:40right like you don't see like a lot of massive deviation because people are like okay hey this really makes sense i get where we're going as a company so in our case you know we've been investing a lot in building out a platform alongside of our product and so there's a lot of things you need to do to invest in the underlying primitives the right dx to enable the use of those primitives the ui layers on top of those and so people start to see okay this is the next piece of the puzzle that we need. And so you just don't have like a lot of this like shock of like, oh, wow, I'm so surprised that like we're going in that direction now.

9:14And this, I presume also is a major argument for a high retention culture within Shopify's engineering product orgs, where keeping craftsmen around, having folks who want to stick around for several years, as you have, really increases this ability to tightly align with the approach of leadership and kind of the business priorities, while also giving folks agency to go out and act. And as we've seen in research from Google and elsewhere, when folks in their roles feel that they are able to make an impact, it's a high indicator of happiness and success within the team. And it obviously boils up within the company as far as successful initiatives.

9:55And you mentioned an initiative that is very commonly being talked about today here by elite engineering orgs like yours, developer experience. And because you're building both this external facing dev tools for Shopify developers, but you also have to consider the internal development culture at Shopify. How do you think about devX or developer experience in your role? I'll talk mostly about external developer. So the work that we do for first party, third party app developers, theme developers, people who are building on top of our platform, that could be a merchant themselves. So take like an Albers, for example, they might be building custom applications or I'm a big fan of Adam's shoes.

10:38They might be building custom apps to sell Adams, things of that nature. We need pizza. So these are all merchants on Shopify who need to extend the platform themselves. But there's also third party developers who are building apps to sell in the store. um when when we look at those like those kind of two large group of developers um what we think about are our first and foremost is capabilities right what are the key underlying things that they can touch and manipulate um and yeah and leverage as a part of it and so we've actually been doing a lot of work to increase capabilities and to just to ground that in reality like hey i want to build something that allows me to manipulate checkout based on the contents of the cart.

11:20So we have a product called Shopify Functions that enables developers to do that, which is basically, it's a commerce-oriented function that takes as input the cart and then outputs a set of mutations to that cart. So that's a good example of capability. And we've been running with that and increasing capabilities for developers, again, both these merchant developers and our third-party application developers. And that's really important, right? Because the more capability that you have, the more platform opportunity there is for everybody, and the more merchants can get done and utilize your underlying commerce stack.

11:53And then the second part is the actual just DX of the building of the experience, right? So, hey, when I go to build this application, what is it like? And there we focus on kind of a couple of things. Like first is that getting started experience, right? How fast can I get started? And so we all the time are pushing ourselves and saying, hey, from the time I go and do Shopify app create initially to an app to the time I can like get that kind of like dopamine of like, hey, this thing works and it does something. How fast can it be? And how and how can we remove as much friction as possible for it?

12:29So that's kind of like what, you know, kind of the Yolo dev, I want to get started, I want to move fast. And the second part of DX is when I am working as a member of a team, perhaps a couple of people, three people? And how can I actually make changes on an ongoing basis where I might have my local dev app, I might have a staging server, and then my production server. And so both of those really matter a lot. How fast can you get going? But then do you have the capabilities to actually work inside of like, you know, a team environment? It's interesting, because I think most of the conversations we have about developer experience on this show are much more internal facing, where it's, hey, I have a team of 100 devs.

13:11How do I keep them happy? How do I make sure they're being enabled? Or, hey, I'm a principal engineer. Here's how I want my team to be enabled. Here's how I want to kind of like guide this organization. So it's really cool to hear your perspective of kind of this dual tracks of like, we both have to consider our internal space. And as a company that's building a dev tools platform and a leader within that space, you have this major consideration. And often, Shopify is used both by like major teams and also small business owners. So there's these like whole other set of considerations where it's like, how do I help an individual or like couple people make our tools be more successful for them?

13:47I'll just add one thing to the prior point is that the same tools that we give developers to build these applications or themes who are merchant developers or third-party developers, we actually use those same tools ourselves when we go to build first-party applications. So Shopify has a kind of core product that comes in the box, but then we're also building apps on it, just in the same way that, you know, hey, iOS has an app that you can download from the App Store. So those exact same tools we're utilizing ourselves. So we get a real feel for like, okay, how well is this platform working? Does it have the right capabilities, the right primitives?

14:24How is the DX? So we got a lot of really great insight to that. So how do you define that success? What does success look like for you on this platform you're developing? Again, kind of zooming out a little bit, I'd say that Shopify is very much a product experience company, right? We care a ton about how well is this piece of software crafted? How easy is it to use and get started? Again, whether that's for our merchant experience or for our developer experience. And we put a lot of weight on the qualitative experience, right? The notion of product sense matters to us a lot. And it's for every craft, right?

15:00It's not just product managers. It's every craft kind of has a sense of like, is this thing working well? And so we have like kind of our own internal, I refer to it as like OGF, overall goodness factor, right? How good does this feel? How well is it working? And then we've also got kind of qualitative indicators of the OGF, which is like, is it, you know, when we hear from merchants or developers, what are we hearing from them? And so that kind of creates for all of us like this internal scorecard that we have of like, well, where are the big gotchas? Where is the big friction? That context piece you mentioned earlier.

15:34Yeah, exactly. Right. You've got a ton of context. You use the product daily. You're talking to customers, whether it's developers or merchants on an ongoing basis. And so you just have this knowledge in your floating in your brain of like what good should feel like. And again, but not just feel like, but also like where are the gaps? Where are the things that are really holding people back or holding you back? But then beyond that, you know, things that we look at, if you kind of look at those two, those two segments of folks that I mentioned, when it comes to like, meaning the getting started fast group, and then there's like, I'm working in a team group.

16:05When it comes to the getting started fast, we have a lot of really good indicators, right? We know, you know, what percentage of people who create an app actually succeed and get it in that app installed on a store. We know the time, how long has it taken them to go do those things. So there's a lot of kind of quantitative data that we can accumulate about how well that experience is working. When it comes to the second group of like working, you know, who are working with many devs, let's say, you know, me, you and another person are working together to go build an application. There, it tends to be a little bit more qualitative because there's things like CICD integration, things of that nature.

16:41But we still have a lot of good data points that help us, you know, that give us a good sense of whether things are working well. Do you benchmark against other industry sources or other teams, or is it mostly an internal benchmarking of, hey, we feel like success is here based off of these qualitative signals that we get from our customer base and from our internal surveys? Yeah, I'd say it's mostly the latter, right? It's mostly like our own internal benchmarking of what do we think good looks like and why. Because it goes back to some of that first principle of thinking that we spoke about, right?

17:14Where we have a clear opinion on how the software should be used and why we think it's overall a good thing. And so we can say, hey, this is what we think good looks like. So, for example, one of the things that we look at is within the app developer platform space is, you know, How is our CLI being utilized and how many folks are using declarative configuration, right? Configuring their application, utilizing kind of a source code file rather than doing it like imperatively in a UI. So we track that data. We don't look at external benchmarks there. I don't know if they're available, but mostly we have a good sense of, hey, we think the vast majority of people should be using this because strategically it makes a ton of sense even for them, right?

17:55Hey, all of my config can be checked in as code. I can easily duplicate environments if I go and need to. So that's a good example of a feature where we have our own sense of what good looks like. Awesome. I do love that because you are a company of crafters, of builders, of developers who are kind of building for developers, you have this internal compass that you're able to develop as you are participating in this process and communicating with customers. I will shout out, since you asked if they are available, we do have benchmarks available at linearb.io slash benchmarks. that benchmark a lot of these quantitative signals around software development and what success looks like.

18:32And there's some really interesting ones coming on qualitative too. So if you ever have a chance to look at them, would love your feedback on that. I'd be curious to understand more about how the day-to-day functioning of Shopify engineering and product teams kind of works, the social circuitry, so to speak. How are teams constructed given this high degree of autonomy and this kind of mission driven focus? So it'll start typically, again, people will have a good degree of context from strategy that will come from leadership, or that's Toby himself, or other leaders in the company. Those docs and posts are floating around internally a lot.

19:12So people have a good degree of, okay, this is where we're headed. We have kind of loose roadmaps like any other company would about, hey, here are kind of the key things that we want to be true in, say, six months or 12 months or 18 months. And then what happens is we have our internal mechanism to create projects and for people to kind of basically, like I said, attach themselves to that project. So every project has a champion. That champion is sometimes a product manager, but very often might be a designer. It might be a data scientist. It might be an engineer who's championing a project because they feel passionate about it and because they feel they're really high context to go do it.

19:51So, hey, we actually recently, I'll tell you about one that we released recently. We used to kind of have like three slightly different CLI experiences for apps, themes, and storefronts. And a really talented engineer on the team said, hey, we should kind of unify these to give like a really consistent DX, whether you're building an app, a theme, or a storefront. And so we actually just kind of announced it in June as a part of our edition, the unified CLI. So whether you're building any of those three, you get a very kind of familiar DX about it. And again, like I mentioned, that was championed by this engineer.

20:26A couple of other engineers, you know, attached themselves to that project. And then, you know, there was a product manager and designer that worked on it as well. So it's really kind of taking this loose roadmap and then people kind of form into teams around, but it's around the project. The project is kind of the key atom, if you will, that is the thing. And that project has all the artifacts attached to it, whether designs, whether those are UX designs, tech designs. It might have a short brief attached to it. It explains, hey, these are the key assumptions I have in the project and these are the key outcomes I want to achieve.

21:09And then we have a mechanism for both peer review and also leadership review of that project. It's a really interesting process because I know there are other companies that leverage something similar, but what it almost reminds me of is government in a collaborative fashion of like, hey, we have people who are creating a bill or in this case, a project and proposing it. And then you get people to sign on and say, oh, we think this is a good idea. You get buy-in broadly and then go execute it. It's a really interesting system because of how, I would say, democratic it is in a lot of ways. And I'm sure, as we kind of alluded to earlier, that brings a lot of team satisfaction benefits and also works really well given the high degree of context that your developers and your builders or crafters have, given that they're using these same tools every day themselves.

22:00So it's a fascinating kind of alignment here where I think a lot of people listening would go, oh, well, my product, my devs don't understand it well enough yet. I need to kind of give them more guidance. And I would almost challenge folks in the audience and say, like, are you giving your teams enough context, enough opportunity to get context? Are they being able to be close enough to the bare metal? I know it's not gonna work for everyone, but I love that you're kind of talking through this process because I think it's really interesting to think about how other product leaders approach this.

22:30I mean, I think that there is a lot of that to it. I'd say there's one key difference is that like versus, say, government is that we, this is kind of like a quotable thing, is like we actually are very anti-compromise. Kaz, our CEO, has a saying. He's like, everybody loves rainbows, right? You know, but if you like take the colors, imagine like a painting of a rainbow that is like still wet. If you take the colors and like, you know, mush them together, you get brown. And like, you know, it's a really gross brown. And nobody likes that. And it's far uglier than a rainbow. And so we believe that like it's really critical that that champion that I mentioned has a strong perspective on where this thing should go and is really the person who is should be taking charge on driving what might be very controversial decisions and not just kind of, you know, agreeing on.

23:27Oh, yeah, we'll just take some of this and some of that. And so that champion role has a really critical thing, which is like, hey, I have a certain thing I have in mind. Here's the outcome I want to achieve. Here's why that outcome matters. And I'm going to work like heck to make sure it happens. And I appreciate you clarifying that because, as you mentioned earlier, like this is a high autonomy culture, a high ownership culture. Yeah, I hope my metaphor isn't isn't making it feel too like, oh, we're getting bogged down in legislation. Like, I think that's very clear. That's not Shopify's approach, given how much y 'all do ship.

23:56And given the incredible stuff that Shopify recently rolled out at your new additions.dev, one of your two times yearly events and releases that you roll out, I'd love to understand more about what Shopify is shipping for software development teams and how you see that impacting software development for the millions of developers who are leveraging Shopify. That's a great question. So additions is this thing that we do twice a year, which basically gives a roundup, like almost a change log of everything that we have shipped over the prior six months. And we try and make it as complete as possible.

24:37So it's not actually just like a few things. It really is showing everything. And that's why, like, you'll see there's, you know, 100 or 150 pieces of either new or true functionality that gets released as a part of it. Our most recent one that we just did in June, just to kind of set the stage and give you some context, zooming out a bit, it was called Unified. And, like, when a piece of software first ships, right, like I've been at the startup stage, it's, like, clean. There's no, like, duplicative system. So there's like no migrations, like everything like works totally flawlessly because like this thing is like brand freaking new.