Clones, commerce & campaigns

0:03Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, productive, and accessible to all. If you like this show, you will love The Change Log. It's news on Mondays, deep technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for The Change Log wherever you get your podcasts. Thanks to our partners at Fly.io. Launch your AI apps in five minutes or less. Learn how at Fly.io.

0:37Well, friends, I'm here with a friend of mine, Michael Greenwich, co-founder and CEO of WorkOS. We're big fans of WorkOS here. Michael, tell me about AuthKit. What is this? How's it work? Why'd you make it? WorkOS has been building stuff in authentication for a long time, since the very beginning. But we really focused initially on just enterprise auth, single sign-on, SAML authentication. But a year or two into that, we heard from more people that they wanted all the auth stuff covered. Two-factor auth, password auth, you know, with blocking passwords that have been reused. They wanted off with, you know, other third party systems.

1:12And they wanted really WorkOS to handle all the business logic around tying together identities, provisioning users, and even more advanced things like role-based access control and permissions. So we started thinking about that more, how we could offer it as an API. And then we realized we had this amazing experience with Radix, with this API, really the component system for building front-end experiences for developers. Radix is downloaded tens of millions of times every month for doing exactly this. So we glued those two things together and we built AuthKit. So AuthKit is the easiest way to add Auth to any app, not just Next.js if you're building a Rails app or a Django app or just straight up Express app or something.

1:52It comes with a hosted login box. So you can customize that, you can style it. You can build your own login experience too. It's extremely modular. You can just use the backend APIs in a headless fashion. But out of the box, it gives you everything you need to be able to serve customers. And it's tied into the WorkOS platform. So you can really, really quickly add any enterprise features you need. So we have a lot of companies that start using it because they anticipate they're going to grow up market and want to serve enterprise. And they don't want to have to re-architect their auth stack when they do that.

2:20So it's kind of a way to like future proof your auth system for your future growth. And we had people that have done that. People that started off and they're like, oh, I'm just kicking the tires. I'm just doing this and then poof, their app gets a bunch of traction, starts growing. It's awesome. And they go close Coinbase or Disney or United Airlines or, you know, it's like a major customer. And instead of saying, oh, no, sorry, we don't have any of these enterprise things and we're going to have to rebuild everything. Just go into the WorkOS dashboard and check a box and you're done. Aside from the fact that AuthKit is just awesome.

2:50The real awesome thing is that it is free for up to 1 million users. Yes, 1 million monthly active users are included in this out of the gate. So use it from day one. And when you need to scale to enterprise, you're already ready. Too easy. You can learn more at offkit.com or, of course, workos.com. Big fans. Check it out. 1 million users for free. Wow. Workos.com or offkit.com.

3:37Welcome to another fully connected episode of the practical AI podcast. In these episodes, Chris and I try to keep you fully connected with everything that's happening in the AI space and and hopefully share some things that will help you level up your machine learning game. I'm Daniel Whitenack. I'm CEO at PredictionGuard, where we're deploying a platform for private and secure AI, and joined, as always, by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? Doing very well, Daniel. I'm podcasting from outside today. That's exciting.

4:17It's a cool November night, but since I just moved house and I don't have a place to sit, nothing but boxes here, we're talking about AI outside today. This is an outside AI day. Yeah, yeah. You live in a place where it's possible to be outside reasonably comfortable in November right before Thanksgiving. Yeah, it's a bit colder up here in the Midwest. We're definitely getting to that Midwestern time when the Carhartt jackets come out and the beanies. And yeah, it's a good time of year. It means Thanksgiving is upon us. That's right. We got Tofu Turkey coming up here. Tofurkey is imminent. Yeah, forthcoming.

5:03So exciting. There's better ones than other ones. So this isn't a podcast about Tofu Turkey or Tofurkey, but there's some that are better than others. And we'll maybe let people hop on their own minds if they're exploring that territory. We need some AI generated Tofurkeys coming at us. There's got to be some intersection. That's right. Maybe Tofurky is using AI to generate ad copy this year, which reminds me, I don't know if you've been seeing all the things in the news, Chris, about Coca-Cola's ads, AI-generated ads. Have you been seeing any of that? Have you seen the actual ads? I have not seen the actual ads, but I have seen some of the news talking about it.

5:48Yeah, yeah. So for those that aren't aware, Coca-Cola, you know, every year Coca-Cola kind of creates these iconic Christmas time ads with the Coca-Cola truck and, you know, the polar bear and things like that. and this year at least I don't know if it's all the ads but there's at least one ad I haven't been following the exact details but there's at least one ad that is fully AI generated or at least driven by AI generated video clips or images that sort of thing and I've seen it on the streaming services so on on you know I forget which ones whether it's prime or they sort of all have ads now because it's basically like cable at this point but all of them have ads so i've seen the the coca-cola ad on the on the streaming services and yeah i think uh maybe those that haven't seen it out there should go watch it i think it's interesting that there's certain elements of it that give you that that ai generated vibe right where you could kind of tell But it definitely evokes the character of the sort of Coca-Cola ads.

7:01And lots of people don't like it. Lots of people think it's interesting. Some people on LinkedIn I've seen said, well, if AI-generated video is good enough for Coca-Cola's Christmas ads, then who is it not good enough for at this point? Which is maybe a hot take. I don't know. Any thoughts, Chris? I'm just kind of amazed that people are surprised by that these days. You know, it's like, uh, you're going to see this stuff everywhere. And so, yeah. Okay. Iconic thing. I got it. But yeah, I mean, I would have been almost surprised if they hadn't. Yeah. And, um, yeah, if you just search for Coca-Cola ad, I think the, it's the real, real magic holiday ad, which is also a bit ironic that they titled it real magic when it's definitely not real but yeah you can watch it it's pretty interesting i i think it's uh whether or not it's it's really really good ad material it's i think a sign that for sure ai generated video is is here with us for the future so you had a few some months back the actors you know going on strike but i i just think that it's one of those things we have a long way to go uh you know not just in entertainment but in most industries where it's gonna you're gonna see corporate videos that are ai generated i've already seen that uh i may not have seen the coke one but i i you know i've seen corporations that are doing it it's the way it is now yeah certainly companies like synthesia and heygen and these video generation companies for training videos for you know multiple languages all these sorts of things.

8:47There's a lot of use of those. I've definitely seen it. Disruption. Yep. Speaking of disruption. Oh, boy. We haven't talked about this yet on the show, Chris, and I don't think either of us have a desire nor maybe, at least on my part, any sort of profound opinion on this topic other than the fact of what it means for AI, but I saw an article in Time about what Donald Trump's win means for AI. So if you're listening to this podcast at a time sometime in the future when it's not election season, maybe you're looking back on this and you know what Donald Trump's second term meant for AI. But at this point, we don't necessarily know, although that we could make some guesses, which we can talk about.

9:39But yeah, we're about to go into the second Trump administration. So if you're listening to this at some other time, that's the time that we're talking about this. And yeah, so interesting. We've seen, maybe just as a reminder, we've seen the Biden administration do some things as related to AI, including the executive order on AI, which we did talk about on the show. That was episode 244. So if you're wanting to know if we refer to that and you want to know which, you know, the details and the interesting pieces of that executive order, that's episode 244, which we'll link in the show notes. But yeah, interesting.

10:27Any initial takes on as a practitioner, what this means for us? I can tell you what I hope it means. And I hope, you know, during the first Trump administration, he didn't know very much about it. He brought in some corporate folks to, you know, put together some committees and they, you know, there was a little bit that came out of that, a website and stuff like that, but it didn't impact us too much at the time. And so I think part of me hopes that maybe it will be gentle. Let him talk about rolling other things back, but maybe he's not aware enough of AI to do it. But of course, it's been another four years and who knows where that's going.

11:12So a little bit nervous to see where his policies take us, but I hope he's more or less hands off. Yeah, yeah. So in the Time article, this is a quote from that article, which we can put in the show notes. It says, Trump's own pronouncements on AI have fluctuated between awe and apprehension. This sort of, you know, describing it as a superpower or very alarming, right? Often in the same sentence. Yeah, maybe so. But one of the things I think that has been kind of promised, maybe as a part of just undoing some of the things of the Biden administration, which I think we can expect more generally, is a promise to repeal the executive order on AI, among probably other things.

12:04And I think citing the hindrance of innovation, you know, this kind of anti-regulatory take on a lot of things. So there's a promise to to repeal that. I'm not enough of a lawyer slash politician slash political analyst to know what exactly that undoes. because the executive order, I think, kind of has its tentacles in a variety of things that it touches that are maybe not immediately related to the executive order, like the NIST, AI risk frameworks, and those sorts of things. So I don't know exactly how that works out. Maybe that's a point of confusion on my part. Yeah, my concern is there are some things that I think if you didn't just have a knee-jerk reaction to anti-anything that Biden did, that there are actually some things that the current administration and the incoming administration should be able to agree on.

13:05And one of those that's not AI, just as an example, is the CHIPS Act, which is kind of trying to bring semiconductor capabilities back online in the US. And if you're an administration that's anti-China or, you know, in the China-Taiwan concern, then you would think that that's at which Trump has said he is. You would think that that's actually something that both sides of the aisle could agree to. But he's also said he's going to repeal the Chips Act as well. And I fear that this that the executive order, since it is something he can repeal with just the stroke of a pen, might suffer that. And yet I think that he would be making a mistake regarding his own administration.

13:50I think that would create problems. Yeah. The article that we're referring to even talks about this, that there's some statements about, you know, we're going to need more, more chips and more chip production. Right. But at the same time, as you mentioned, the Trump campaign has attacked the Chips Act. I saw certain things, of course, are still in progress that would, I think, fit the America first chip production piece of that, including I just saw in the news that Intel was awarded up to$7.9 billion under the CHIPS Act to help build or expand chip plants in Arizona, New Mexico, Ohio. and Oregon, including 1 billion plus later in 2024.

14:45So some of this I know, especially the Ohio plant and all of that is, I think, in progress. I don't know the exact details of that. But yeah, some of this is in motion. So it is a bit confusing to me. I'm sure that CEOs of large companies are on the edge of their seat and trying to get audiences with the right people and understand what's going on. I'm assuming, again, I don't know how all of these things work under the hood, but I'm assuming there's a lot of that shuffling going on to get a read on the situation. Yeah, I would hope that if there's anyone out there listening that might be a part of the incoming Trump administration, making America great again is exactly what the CHIPS Act was intended.

15:34And frankly, I think the AI executive order does the same. So I think I'm hoping there's no knee jerk on those two things, despite the comments. Maybe he'll let them continue. Yeah. What is your take on the potential perspectives on open source or closed in the Trump administration? Any thoughts on that in terms of how that may be influenced one way or the other? I don't really know at that point. I think it comes down to whoever, it depends on who's in the cabinet potentially and probably more specifically who's working on staff at the White House and what their takes on it are. And I couldn't speak to that.

16:18Yeah, I've seen a mix of takes on that. I think there's one perspective that, while China has benefited greatly from open source AI, right? Not only have they been model builders and actually producing a lot of technology in the AI space, but they've also benefited a lot from, you know, meta and US AI technology. So there's kind of one side of it that would be, well, let's, you know, lock that down in the same way that they might try to restrict exports of other things or that sort of thing. But I've also seen the other take on the fact that, you know, you're basically anti-regulation and it would be kind of not within the character to be restrictive in terms of the open source AI world.

17:16So I think it's a little unclear. I'm certainly one that kind of views the future even more importantly in security, privacy conscious industries really driven by open, self-hosted models. I think that's really the way that you ensure security, privacy, transparency. Yeah. I think there's a lot of ambiguity at the moment because if you look at traditional conservatism, if you look at Ronald Reagan, you know, because a lot of Republicans really look back to that. Open trade is huge, but we're also having Trump talking about tariffs. That's been the news of the week. And, you know, that's kind of the antithesis of that.

17:59And so it's kind of hard to figure out where the ball is going to land on those.

18:17Okay, friends, one thing I'm obsessed with now is using Notion AI. Yes, I'm a Notion user, a power user, I would say. I use it for myself. I use it for changelog. And it's just so effective to organize everything. And with the addition of Notion AI, it is a single tool that does it all. It searches across Notion, other apps, generates docs of my own style. It analyzes PDFs and images. It can chat with me about anything. Now, you may know Notion as the perfect place to organize your tasks, to track your habits, to write beautiful docs, and all the things you could do with Notion. But adding Notion AI with the context of your work is simply revolutionary.

18:56And unlike specialized tools or legacy suites that have you bouncing between six different apps, Notion is seamlessly integrated, infinitely flexible, and yes, it is beautiful on the eyes. And yes, I am obsessed with Notion AI because I could pretty much ask it anything. And it's just there for me. It's super fast and is relevant to all of the stuff I have inside Notion. You can try Notion today for free by going to notion.com slash practical AI. That's all over case letters, notion.com slash practical AI. Try the powerful, easy to use Notion AI today. And when you use our link, you're supporting our show.

19:38which is awesome. Again, notion.com slash practical AI.

19:54One thing that was kind of brought up in the midst of this talk of the Trump administration and AI is this sort of AI and China discussion where there's a thought, you know, AI is kind of thriving in China and maybe China is pulling ahead in AI. I know we've talked about this on the show before. There's kind of this discussion of China and AI every time policy decisions are discussed on this show and kind of factors in. And one of those things that I think is relevant is just the dominance of Quinn-based models in recent times. So if people aren't aware, one of the things that I think is interesting to follow recently is Alibaba's Quinn family of models.

20:46That's spelled Q-W-E-N, Quinn. The latest of these is the Quinn 2.5 model family. And generally, these Quinn 2.5 models are quite impressive. They generally top the open LLM leaderboards in various categories. You'll see them in the top spots. So obviously, these are Chinese models. That is, they're models being built by a Chinese company, Alibaba. The CEO of Hugging Face, Clem, is quoted in one article I was reading of, you know, Quinn 72B is the king and Chinese models are dominating. That's a pretty clear statement. That was earlier in the summer, but I think we've seen continued domination of these models.

21:47Any interesting takes on that, Chris, in terms of how you've seen the model landscape shift from closed model providers to open to maybe more geographically diverse and certainly China being within that? I'm in an industry. I'm in defense and intelligence where obviously we're not going to be using Chinese models. And so we have not been focusing on that. We, of course, keep track of everything out there, but that's not one we're likely to use. but I'm really curious in kind of outside of the sector that I'm in. I'd love to get some feedback from people on what they're uptaking. I think there are a lot of industries where they're not going to care either way on that and they're going to go for the best models on the leaderboard, but I haven't actually talked to anyone who's done uptake.

22:39How about yourself? Yeah, and maybe this is an interesting little diversion here because I think some people don't understand the potential security risks as associated with this sort of model. So we say it's a model produced in China. Some people would be uncomfortable because of China's use of data or ways that they would use this technology. But if we look at the model itself, so you can go to Hugging Face and just search for Quinn models. So the Quinn models are open in the sense that you can go to HuggingFace. It's a repository of models. You can literally go to the Quinn model. You can download the weights of the model and load that model into infrastructure that you control.

23:34So this model, when you think of the model, is composed of parameters and model code that runs that model. And so if you go to the model on Hugging Face, you can download that. Now, similar to like if you were to go to GitHub and you look at all of the repositories on GitHub, some of those repositories on GitHub will have security considerations or licenses that won't allow you to use them or, you know, sources that you don't trust. right? It's a little bit interesting here because these models are kind of loaded into code that is maintained by HuggingFace, the Transformers library or other serving frameworks, right?

24:18So if you're self-hosting the model, meaning you're pulling the model down from HuggingFace, the files, and you're loading it into code that can serve that model, that model serving is under your control and you are downloading those files, meaning you can inspect them. It doesn't mean there's no security vulnerabilities associated with them. But ultimately, all of that is under your control. That is a different scenario than if you were to connect to an API that is serving the Quinn model, which there are ones from Alibaba and others, where this model is actually hosted as a product of a Chinese company.

24:59You're sending your data to that API product, which is then processing your data and giving you a response back from the model. So I just wanted to emphasize there's kind of these two scenarios here. So one, in one scenario, the security vulnerability is really related to the model files that you're downloaded. Is there any security vulnerability in those model files, which there could be? Is there any third-party code that's used when you load those model files, which there could be? and what serving framework are you using to serve them, which could have security vulnerabilities. In the other case, you're relying on someone else's infrastructure, which isn't under your control, which might be under Alibaba's control.

25:43So these are just different concerns that you want to weigh. And I thought that may be good to highlight because some people may even want to experiment with the Quinn model, like in a thing like LM Studio or something like that. I'm not vouching for all the safety considerations that might be in in your mind but it's not like I don't think when you use Quinn in LM studio there's some sort of phone home to Alibaba going on necessarily in the in the underlying code that's running that I think in like U.S. government circles just to clarify something I think it's more policy than necessarily so I think you're going to have some agencies that are downloading all the models and reviewing and inspecting and stuff like that.

26:27But I think for typical usage, you're looking at more of, I think you're much more likely to see a U.S. agency or corporate that is serving the U.S. government going to be focusing on meta versus Alibaba. I think that's just a policy issue. Yeah, yeah, yeah, for sure. I think you're right. I think I've just seen a lot of confusion around this. It's like... No, it's good clarification. Anytime you use a Quinn model, it's stealing your data. But there may be ways to use this in a way that is appropriate for your scenario. Sure. Likely, like you say, if you're working in defense or something, that's going to be a different consideration than if you're hacking together a cool AI agent on your side project, you know, for personal purposes.

27:12Those are very, very far apart on the spectrum. So, yeah, very, very interesting, though. Also, there's some recent development. So we're late in November already, but this is, I think, about a week ago, something like that. Quinturbo 1 million was released, a sort of new version of this, which extends the context length of the Quint 2.5 language models from 128K to 1 million tokens. So that's to kind of give a context, some of what's cited is like 150 hours of transcripts or 30 ,000 lines of code or these sorts of things. So lots of context can be put into these models, you know, which is a trend that has continued.

28:01And I have my own opinions about, but it does seem to be a trend that continues. Go ahead and share them. You can't hang that out there and not go there now. Yeah, well, I just think if you think about the typical, the most common enterprise cases that I run across in working with customers, most often these fit these scenarios of what I like to think of as something that could be done by a college level intern. right so you have some very clear instructions to do this sort of workflow and it might be multi-step it might be a complicated workflow but it's all like you can break it down in a sequence it's um there's instructions there so anecdotally if you go to a college level intern and you say go into the warehouse out back there's you know rows and rows of documents now do this task for me, right?

29:01That's a much harder thing with a higher degree of potential failure than if you go to the warehouse and you find generally the section that's relevant to a task and you say, hey, you know, look at these couple folders of documents and do the task. You're much more likely to get a better result. And I think these models, you know, anecdotally behave similarly. And there's some evidence for this in terms of the forgetting of what's in the middle of the context, which has been observed, you know, in academic research. And I'm sure people on this podcast will be like, no, Daniel, that's solved. You know, whatever.

29:42It's just my own sort of experience and anecdotes in terms of what has been found to be useful. It's just, yeah, a million tokens is a lot. So possibly more than most people are going to need. I know people that have been on this podcast and are peers of mine that totally disagree with what I just said. So that's okay. We're all kind of figuring it out as we go along, I guess. So Quinn 2.5, that intersects with some of our discussion around the China-America debate. but there's a variety of models that people might be interested in in just taking a quick look at that have popped up over the last weeks and I don't think we've, it's been a while Chris since we've done a here's a buffet of new models type of brief disclosure and there's a few interesting ones so there's one that is from DeepSeek which previously released a series of really good coding models.

30:49But they've released DeepSeq R1 Light Preview, which is kind of fitting in this chat GPT-01 or OpenAI-01 kind of world, which is this going to pause and think about things sort of world where it's trying to solve very complicated math benchmarks or other things. And so you see actually this DeepSeq model in many cases for certain benchmarks, maybe even doing better than O1 preview in a number of benchmarks. So I think this is further evidence that this gap between the closed model providers at the frontier and open model providers is just closing so rapidly. It's, in my opinion, basically not distinguishable anymore in a lot of things that people want to do, whether you want to use an open model or closed model.