Sidekick is an AI Shopify expert

25:35And if a buyer has a question like, hey, what's your return policy? Or like, where's my order at? They can interact with the merchants through Inbox. And so one of the things that we're offering today is that we look at all the merchants policies and all the other things that they've given to us and then can help formulate answers for those common questions, right? It's like, well, what's your return policy? It's like, well, we're pretty sure that this is the answer. And then we can suggest that to a merchant who then says, yep, that's right. Or if that's not right, they can adjust it to be correct and then send it.

26:02And so merchants love that because it saves them time for answering a lot of those repetitive questions. And then those ones that are a little bit harder, they can write them themselves just as they would before. And then the last one that I'm thinking about is like, again, going back to that product and merchandising kind of task is that oftentimes merchants are uploading a lot of these at the same time, right? And so like, they don't always capture all the metadata in the first go. And so this is, again, where we've created models that actually can help with that. So like, if you upload an image of a white flower dress, we have a model that can actually understand what that picture is and suggest that, hey, maybe this is a white flower dress, and this should be categorized under dresses and like, under cotton, right?

26:40And like, maybe it can suggest, oh, it turns out if you upload multiple different colors, it's like, well, maybe you want to create product variants. And so that's some of the other technology that we're kind of working on today is like, using the data that we have from merchants, enabling them to more expressively describe their products through our sites. With Inbox, is Inbox part of Sidekick or is it adjacent or parallel to Sidekick in the way that you see it? No, it's a completely separate offering. So merchants have to choose to install, again, going back to kind of the platform thing we talked about earlier, they choose to install the Inbox app, which Shopify builds.

27:12And then within the Inbox app, you can choose to use this behavior or not. You mentioned also the ecosystem a little while ago, And I'm curious how, as you have created these new AI-enabled products, how has the ecosystem been plugging into that? Is that something that's possible? What kind of interactions do they have together? So today we have a very extensive API through GraphQL that we expose. Like the data we just talked about, right? The categorization. So whatever the merchant decides, like we make a recommendation, they say, yes, that's correct. This dress is actually a dress. Once they save that change, that information is then available through that product description API.

27:50I see. Matt covered a nice breadth of generative text, generative images, product understanding, all of which are kind of adjacent to or image generation is sort of a different algorithm under the hood. There's also a direction where we can think about something that I've, it's been a gift that keeps on giving for all of my career is that the sorts of machine learning techniques that work with text often also work with commerce. And so this goes back to old-fashioned matrix factorization for recommender engines was also useful for understanding text. RNNs, useful for looking at sequences or sentences before Transformers took over.

28:34Good for text also. Boy, you're taking me way back. We did many whole shows on RNNs, and that seems like the Stone Age now. Yeah, the Stone Age of the 20-teens, right? That's right. That also, a lot of the techniques, you could always just peek at what you're doing in language and come up with a cool idea for e-commerce and vice versa. And so this has not stopped. I mean, transformers have kind of taken over everything, but there are not quite transformer architectures, but heavy attention method, transformer-like architectures that can look at sequences of behaviors of merchants, of buyers, the people that are shopping with our merchants.

29:14Those are sequences too and can be processed in an analogous way in order to understand what is the next step on the journey for a merchant and how can we help them get to that journey? What are the likely ways we can simulate that? And that's been sort of one of our frontier cutting edge areas that we've been applying ML. actually mike's answer reminded me i want to add one more product that always slips to mind here and it also blends with the ecosystem thing that we talked about um so as i'm sure you've talked with other folks on the show about um one of the exciting parts about llms is the ability to write code and we talked about the graphql api and so one of the other exciting applications that we've done is for our developer ecosystem um is enhancing our developer docs in the way that like we now have an integrated tool that assists developers in writing code.

30:04You described, you're like, hey, I'm looking to find the product category. Can you write me the query to do it? And it will literally write you the GraphQL query. You can copy and paste that, put you right in your application. So I think we're still in the early days of figuring out how, like, it's such a dramatic shift for engineering to, like, figure out how we apply these elements. And it's exciting to see these new applications to, like, existing documentation sites and, like, just unlocking power and making it more straightforward to develop apps.

30:48What's up, friends? I love my eight sleep. Check them out. Eight sleep dot com. I've never slept better. And, you know, I love biohacking. I love sleep science. And this is all about sleep science mixed with AI to keep you at your best while you sleep. This technology is pushing the boundaries of what's possible in our bedrooms. Let me tell you about 8Sleep and their cutting edge Pod 4 Ultra. So what exactly is the Pod? Imagine a high-tech mattress cover that you can easily add to any bed. But this isn't just any cover. It's packed with sensors, heating and cooling elements, and it's all controlled by sophisticated AI algorithms.

31:30It's like having a sleep lab, a smart thermostat, and a personal sleep coach all rolled into one single device. And the pod uses a network of sensors to track a wide array of biometrics while you sleep. It tracks sleep stages, heart rate variability, respiratory rate, temperature, and more. and the really cool part is this. It does all this without you having to wear any devices. The accuracy of this thing rivals what you would get in a professional sleep lab. Now, let me tell you about my personal favorite thing. Autopilot recap. Every day, my 8 sleep tells me what my autopilot did for me to help me sleep better at night.

32:08Here's what it said last night. Last night, autopilot made adjustments to boost your REM sleep by 62%. Wow, 62%. That means that it updated and changed my temperature to cool to warm and helped me fine tune exactly where I wanted to be with precision temperature control to get to that maximum REM sleep. And sleep is the most important function we do every single day. As you can probably tell, I'm a massive fan of my eight sleep, but I think you should get one. So go to eight sleep dot com slash change log. And right now they have an awesome deal for Black Friday going from November 11th through December 14th.

32:47The discount code CHANGELOG will get you up to$600 off the Pod 4 Ultra when you bundle it. Again, the code to use is CHANGELOG. And that's from November 11th through December 14th. Once again, that's 8sleep.com slash changelog. I know you'll love it. I sleep on this thing every night and I absolutely love it. It's a game changer and it's going to change your game. Once again, 8sleep.com slash changelog.

33:28Going back to something that you had mentioned earlier in the conversation, you had talked about magic and magic enabled and stuff. Could you tell me a little bit about that? I may have misunderstood. Is that a product or a supporting technology that you guys are using? I mean, it's kind of the way we refer to things. We should have done a better job explaining it. So all the things we just talked about, we consider part of the magic brand at Shopify. So it's like the product taxonomy stuff, the text generation, the sidekick, the image generation in the background. So those are all magic features that Shopify offers.

34:03Gotcha. So it's kind of the, the AI enabling brand, uh, that's around all these things. So as you, you know, we've kind of talked a little bit about, uh, about Sidekick and we've gone through, there was another thing I was wanting to ask you about, and that was, uh, how your current array of kind of AI enabled capabilities that we've been talking about, how are you thinking about that going forward? Where are you looking at? How are you, you know, are you going to add any more in there, uh, that, that are announceable yet, or maybe at least alludeable to, how are you thinking about kind of where you're at today versus kind of some of the things you might be doing in, you know, in the fairly near future and stuff.

34:43And we'll get into farther future a little bit later, but. I can't answer today, unfortunately, other announcements that are coming, but I think we can talk about generalities of like what's interesting, right? Fair enough. That Mike talked about of like, you know, applying old techniques and new ways around commerce specific things. I think predictions and customization around that are interesting. I'd say me personally, I think the thing that I get excited about is the other modalities that are out there. So going back to that reference from earlier, ChatGPT was cool. I feel like I had a second bout of ChatGPT when the voice mode came out.

35:18I don't know if you've played with it, Chris, but it's absolutely incredible. During the typical day, I have an ongoing... I probably, I, this is really terrible that I would say this, but I, I probably talked to chat GPT more than I talked to my wife. Thank goodness. She doesn't want to hear me any more than she already does. So she won't hear this on the show. So, um, but yes, I, I have an ongoing conversation about a plethora of topics. So, which beckons back to the fact that this is moving so fast. Um, and you guys are, as you guys are having to kind of match your customer needs with products that support with the technologies that are driving that forward.

35:56What are some of the things that you're thinking about now for maybe as you go forward into the future? And more specifically, how are you thinking about handling the risks associated with changing technology right now? We've talked a little bit about constant experimentation and everything, but there's also a point where you kind of have to make investments in different directions and trade offs and stuff like that. Other than the experimentation of that to support as this increasing line of capabilities that you guys are offering, how are you thinking about that risk directions and stuff? Do you think that commercial product offerings, for instance, one topic that comes up all the time on the show, do you think open source is going to overcome that and kind of take over since things are slowing a little bit on the frontier models in terms of the gains they're making and open source seems to be catching up faster?

36:47How are you guys thinking about problems like that as you're dealing with these business issues in your company? I think it will likely be hybrid. So like, I know we talked to kind of like our strategy is like everything all the time. So I can't imagine a world where the commercial offerings completely take over. And I really, I don't know if I can imagine a world where open source entirely takes over either. And I think that's probably a good thing for the world. Like, I think that's what drives the innovation, right? It's like a competition between the two. And there are some things that one is good at and the other is not.

37:14So like, I can't imagine a world where we aren't using both at Shopify. Can you talk a little bit about kind of how you see the strengths and weaknesses, recognizing it may change tomorrow, given how fast things are moving? But like when you look at that kind of what we might go with a commercial offering like ChatGPT or one of the other several biggest competitors versus the open source and probably the foundation infrastructure that you guys will have. How do you guys know where to go? Like, how do you know to go to ChatGPT as an API versus using a foundation model you're storing in your infrastructure?

37:45Well, I think it goes back to Mike's favorite point from earlier of evals. We got to have our compass because without the compass, we're lost. So that's how we answer which one. But I think the other part of like you're asking, like which one is good at what at this point? And so I think in general, the way to frame it is like open source, the power is in the control. It's like you're guaranteed to run this exact model with this exact set of training data with this exact outcome. So it's very predictable. You have way more control over the training process and like the post training process and like you just get a lot more knobs, right?

38:17But also with great power comes great responsibility, right? Like there's a cost to operating all those knobs and knowing what the correct values are for that. So for problems that you have pretty fully defined and you know exactly how you want to do it, it's like open source is great, right? The commercial models, fewer knobs, but the thing that's great about that is like, you know, the defaults out of the box usually work pretty well, right? And so like, I think if you're looking at things that are early on in prototyping, like the commercial models work great, right? They can get you from zero to one real quick.

38:46And then when you get to that one, you realize they're like, oh, well, I want to get to 2.0. And sometimes it necessitates a need to that shift to an open source model to get that extra control out of it. There's kind of this question of what size of a problem are you trying to solve? If you're trying to solve a central do everything, the co-founder that you wish you had model, we're going to need to pull out all the guns, right? And really put everything we can into making this, leveraging as much power as we can, right? And the question is just, what is the most powerful for this task at this time?

39:21There's another side of things where maybe we're trying to solve problems that aren't supersized problems, but they're more manageable problems, right? Or maybe we need to do it at scale. And of course, the commercial models are getting faster and faster and cheaper and cheaper. But when you need to do something at scale, it might be worthwhile to distill a model from some patterns and then run it at scale. We have billions of products, if you look over our entire history, doing that at scale for the product that Matt described, where we understand all the different attributes and the taxonomy and we normalize the description of those products.

40:00that's a true engineering feat that we need to work on. And that may not be a great idea to send that to GPT-01, right? Absolutely. One question, I'm guessing, Mike, this is coming to you. The last two, three years, we've been so focused on LLMs and generative AI capabilities. And I know in general, the industry is starting to kind of also kind of pull back and look at some of the other things, things we used to talk about, other technologies in the AI space we talked about a lot. It seems that some industries, things like reinforcement learning and CNNs and things like that, depending on the industry, I think in my own experience, as I've talked to different people, some find utility in these other architectures with other purposes and some don't.

40:46How are you guys? Are you really primarily focused on LLMs and generative or do you have use cases where some of the other technologies that we haven't talked about as much lately but are still very much out there in industry, are they coming into play for you guys? Yeah. So this might be where we appeal the technology. I need one more layer, right? At a base level, any neural net is a universal approximator, right? And so if we have enough data, there is a big enough neural net that we'll solve. Just an MLP, right? A fully connected neural network. And so the way I tend to think about it, whether it's a CNN or a heavy attention model or an RNN, whatever it is, all that's really in the business of doing is making it so that, even though there is a number of neurons, it might be way too many neurons, right?

41:34And we might need way too much data in order to do that. And all of these are really just tactics for reducing the amount of data that we need in order to approximate the patterns that we want. Right now, for sure, heavy attention models, whether it's traditional transformers or evolutions, like you might see in of the original 2017, tensions is all you need, transformer architecture to what you see in Lama. These are kind of like tweaks and they're still very multi-head attention focused. There are other techniques like the one I described for e-commerce that are making substantial changes, like removing the soft max out of multi-head attention, which is a, you know, sort of like having the sigmoid as our activation function 10 years ago.

42:16It was just a mistake and a sociological mistake at that, right? So seeing small changes like that that maybe move us out of transformer architectures, I think that we're definitely in an era where that makes sense. There's also kind of combinations of things, right? So you mentioned reinforcement learning. There's GNN architectures. And these are actually compatible with vision transformers for planning and reinforcement learning, using transformers for your aggregation functions for a graph neural network. And so it's not an either or. It's that now we have another tool for either, in the former case, modeling the world so that we can do a good job at our Q learning and in our policy learning, or do a good job in capturing the right information when we have a graph structure of how we've organized the different kinds of nodes, the different kinds of, in our case, merchants and products and buyers.

43:12That being said, there's another way of taking your question, which is, look it, when are Transformers going to be done and are they gone already? That was on my mind as well, actually, because everyone's talking about, okay, what does the post-Transformer world look like? So, yes. I think I can say this, and I say this to my students pretty religiously. I'm going to make this claim with full understanding that you should never make a prediction that will be falsified in your lifetime. I'm pretty sure Transformers is not the last architecture out there. It seemed for a decade that CNN was almost synonymous with vision since 2012, right?

43:49And now it's not. And if you would have asked me in the late 20-teens if I thought it was, I'd probably say the same thing. It's a heavy thing. It's a heavy bet. CNN seemed to be the top of the hill, and they seem to do such a good job with image classification. It's hard to imagine what will replace it, but probably it's not the last chapter of the story. And I think that you could probably say the same thing for Transformers. So with the, a little bit ironically, and you've sort of kind of covered this territory a little bit with that last answer, Mike. From each of you, we usually finish the show really wanting to get perspectives from our guests on kind of what the future looks like.

44:32And with each of you addressing kind of different areas, you probably have somewhat different answers based on your focus and stuff. And Mike, recognizing that you've already kind of touched a little bit on the future, but I'm actually, despite your comment about not making predictions that might prove falsified in your lifetime, I'm going to ask you both to kind of do that. If you're looking out and I'll let you kind of decide on what timeframe works for you, but maybe, you know, beyond the short term, waxing poetic a little possibly and trying to say, you know, what would you, what do you think you're going to see?

45:09What, what do you want to see and how might your various jobs and how your company serves customers? You know, how do you see this fast moving their twists and turns all on the way that catch us all by surprise? How do you see that playing out from each of you? Matt, if you could lead off and then Mike, I'll come back to you for that. I think what's most exciting about this, I mean, when I grew up, I remember when we first got the internet and like it was like the first ISP out there and there was like a dial up modem and there was like a BBS. Like it was just that was like the first wave of technology to me.

45:41And like that's how I got into this field. And then I feel like the mobile revolution caught me by surprise. Like I think at the moment, like I knew when the first iPhone came out, I was like, I need to have one of those. But what I didn't expect was like how much the world would change after that. And it feels like this time around, I wasn't a big believer in Web3. I was like, what is this Web3 business? But like, I feel like this is, again, that same kind of shift. So I'm just going to ignore Web3. I think this is the real Web3. It's AI. And so like, how does this play out? Right. Like, I don't think what's different this time is that like for the last, I don't know, 70 years that we've had computers, like we as humans have had to conform to like how computers work.

46:18Like at first we wrote assembly code. At first we wrote like literally bits. Then we wrote assembly code. Then we're like, well, maybe we should have languages. And it's like, okay, so we're slowly crawling there. And then the next revolution is like, oh, we should have point and click. And so these boxes. And so now we have a world where everybody spends, you know, eight hours a day clicking on little colored boxes and then typing into other colored boxes, like characters on a keyboard. And I think what's fascinating to me is that we've shaped who we are to conform to how computers work today.

46:48but i think this point in time and i'm sorry i'm giving me 10 years out from now no it's fine all that's gonna change like i think that whole like all these browsers that we click buttons on to like set settings like all that is gonna go away i think it's gonna be that we interact with like an agent or some you know amorphous entity that it's like and instead of you know listing all the steps of like first search for this then click on this link then do this thing it's like be like i would like to buy a box of toothpaste and it's like the agent's like great do you want to buy one for that ships tomorrow or do you want one that's like cheaper but ships next week and you're like the cheaper one and then it's like done right you didn't fill out a credit card form you didn't click through 10 sites you didn't do any of that so like i'm gonna put my bet on that it's like i just think the web will change again like i think we've gotten so used to sass and all these models and like i'm just i don't know how long it's gonna take that's why i'm like 10 years out i don't know it might be three years out but it might also be 20 but i don't think we're gonna be typing in boxes in 20 years from now.

47:44Good answer. Mike, back to you. Yeah, I think I'm going to flank this from both directions. So I spent a little bit of time in the self-driving space years ago. And this was during the, I think, the maximum hype period for self-driving, where everyone I talked to said, oh, well, people won't even need to drive in five years. And this was more than five years ago. And on the one hand, just across the bay, Waymo is giving rides to people, right? Not at scale yet, but it is, right? For sure. A lot of people said my son would never need to get a license and he's about to get his driver's permit, right?

48:27So just to draw an analogy, I think it's reasonable to expect what Matt is expecting of having kind of like a self-driving assistant that can do that. I have all the respect for Matt in the world because he was careful about how he's going to do the time, not five years or whatever. There's probably going to be a little bit of difficulty smoothing down the edges for that. And luckily, a crash with a self-driving assistant is far less dangerous than a self-driving car. So it could be that we get imperfect models there. I'll take an extra five boxes of toothpaste. That seems like an okay for the better scenario.

49:03So that's one side of flanking it is, we very well will. There will be a self-driving moment for these assistants. And how long out is completely on the same page with Matt. We're going to hit some bumps before we actually get there. One thing that I feel very confident of is that we are going to change the way we organize and access and utilize information. This is going to be a forcing function that we really haven't seen since early search days for the internet, which is also a way of just completely transforming how we organize and access information. And a lot of people you will talk to will already say that they go to their favorite LLM first before they go to a search experience.

49:53And there are also a whole host of product and interface and questions like that about what's going to be the best way of doing this. But it is also, once again, piggybacking on something that Matt said, it is incredibly significant that we are now speaking in the same language, quite literally, when we want to access and refine the information we're looking for. And that's something that's really never happened before. Well said. Well, gentlemen, thank you very much for coming on the show. It was really interesting. I learned a lot and thanks for sharing your perspectives going forward. I hope you guys will, will come back as things evolve and you have more things that you want to share with the audience.

50:39Thanks for coming on. I'd be happy to. Thanks Chris. Thanks Chris.

50:49All right. That is our show for this week. If you haven't checked out our change log newsletter, head to changelog.com slash news. There you'll find 29 reasons. Yes, 29 reasons why you should subscribe. I'll tell you reason number 17. You might actually start looking forward to Mondays. Sounds like somebody's got a case of the Mondays. 28 more reasons are waiting for you at changelog.com slash news. Thanks again to our partners at fly.io to Breakmaster Cylinder for the beats and to you for listening. That is all for now. But we'll talk to you again next time.