The state of open source AI

0:06Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related technologies are changing the world, this is the show for you. Thank you to our partners at Fastly for shipping all of our pods super fast to wherever you listen. Check them out at Fastly.com. And to our friends at Fly, deploy your app servers and database close to your users. No ops required. Learn more at fly.io.

0:43Welcome to another episode of Practical AI. This is Daniel Whitenack. I am the founder and CEO of Prediction Guard, and I'm joined as always by my co-host, Chris Benson. How are you doing, Chris? Doing great. How's it going today? It is going awesome. I don't know if you heard, but it is the advent of Gen AI. I'm participating in this advent of Gen AI hackathon with Intel. So people are getting hands-on with a bunch of open source models and different hardware. So I've been in Slack all day, answering questions and seeing cool prompts and seeing cool output. So it's just been a ton of fun. What's the most interesting thing that's been in terms of what you've seen so far?

1:26I'm just curious before we go in. The first challenge, so we're only a day in. The first challenge was to generate a series of images that kind of go in a sequence, kind of like a comic strip that tell a narrative. But there were some really amazing ones. One's kind of a child growing up and then him having a son. And the images were really compelling and the narrative was really interesting. Yeah, just very, very creative output is something that I've noticed. And today is all about chat. So we're going to see some chat bots popping up in the hackathon and really looking forward to that. Sounds like fun.

2:03Yeah, and you know, the hackathon is all centered around these openly accessible or open source or permissively licensed generative AI models. I think it's really fitting because we have with us Casper, who is a longtime open source enthusiast, but also one of the contributors to the recently published State of Open Source AI book from Prem. So welcome, Casper. It's great to have you with us. Hello. Yes. Yeah. Great to be here. Yeah. Well, I mentioned you're a longtime open source enthusiast. How did you kind of get enthused about open source AI specifically? So what was your own kind of journey into open source AI, maybe kind of leading up to this book and what it's become?

2:51That's a good question. I've been around for long enough that AI didn't really exist as a thing back when I got into open source. And it was honestly just purely a hobby. I never even considered it as a career. This was, I must have been, what, 15 years ago or something. And I, in fact, I felt ashamed and embarrassed every time I was working in open source because it felt like I should have been spending that time working on an actual career, right? It felt like it was just a toy. I had a very long commute together between my home and the workplace on a train. And I was just coding away on my phone.

3:24I actually installed Debian site loaded on my Android. And yeah, that got me hooked on open source purely as a hobby. And I mean, if you contribute enough and you're happy making mistakes in public, you know, eventually it builds something that loads of people start using. It spirals out of control. Before you know it, it suddenly turns into a career. So I probably entered into this whole space in an unconventional way. I didn't intend to, you know, make things that would become famous, but they just wound up becoming famous, which is quite pleasant. I mean, there's pros and cons because also things that become successful aren't necessarily things that you expect to become successful.

4:00You can put a lot of effort into something and the world determines it's not really of much value. And so they don't use it. And something you barely put much effort into could explode. So that was my sort of background. I'm kind of an academic slant as well. So I did a lot of machine vision type things in university. Didn't really want to shoo on myself into any particular one area though. And also I didn't want to do pure academia, right? I much prefer industry and having stakeholders and actual products that you build at the end of the day. And I mean, there's pros and cons definitely to both.

4:34But yeah, so that's obviously how I wound up like the rest of the industrial world, seemingly moving towards AI, because that's a buzzword. And that's what everyone wants you to work on effectively. So yeah, what started off as initially being machine vision pre-machine learning became machine learning type machine vision type stuff. And now, of course, LLMs are all the rage. So that's why we thought of doing a bit of extra research and try and consolidate all of the noise out there and various different blog posts, people effectively shouting into the ether. And we thought we might as well write a book and release some of our research in the wild, get some feedback on that before we actually start building more things.

5:14Yeah, that's awesome. And you even allude to this in the sort of intro to the book, this sort of fast paced nature of the field and a lot of people feeling sort of FOMO, like how do I even categorize all of the things that are happening in open source AI? So maybe one kind of general question about the structure of this. Chris and I have worked through some of these categories in various episodes on the podcast, but sometimes it is hard to sort of think about like, how do you categorize all the things that are happening in open source AI? Because they do go beyond just models, but they include models and a lot of things are sort of interconnected.

5:59So how did you kind of, was it organic in how the structure of this book came together? Or how did you come up with the major categories in your mind for what's going on in open source AI. And that's what I was really wondering as well. You literally said, Daniel, exactly what was in my head just now. So I just... Yeah, we're in tune. Yeah, no, I mean, it is a big ask because, I mean, my philosophy in general is that the universe exists as a cohesive whole. And, you know, we split it up into different subjects like physics and chemistry and maths just as a way for humans to actually parse everything that exists in a small little bite-sized chunks.

6:36But they're not really independent subjects, right? And the same goes with AI. I mean, there's so many different categories of AI. So, I mean, the nice thing about working in the open source space is that there's lots of different people you can have conversations with, get some feedback. Everyone kind of chipped in their own ideas about how to, let's say, break down a book into different chapters. Ultimately, I think what made the most sense is that it doesn't matter too much what those chapter titles are. It's more about the content within them being, let's say, not too repetitive and actually distilling the ideas that people are talking about.

7:09And if you can do that really well, it maybe almost doesn't matter quite how you subcategorize things. But I would say Filippo Pedrezini is probably the one who came up with the actual final, let's say 10 chapters. But then past that in terms of actually writing those chapters, probably about a dozen people have actually worked on them, which is again, really nice that you can do this in the open source space. No single person is really the author of this book. It seemed fairly obvious to me, based on my own particular passion and research, that licensing should definitely be a chapter. And that's something that developers often neglect because it's just sort of outside their field of interest and expertise.

7:47And it's just a bit of red tape that maybe they have to be aware of in the back of their mind. But yeah, so I mean, I basically wrote a chapter on licenses, which I think everyone else was happy about. Nobody else wanted to do it. But sure, I mean, it was just effectively topics that we felt are big major things that there's a lot of confusion over. Maybe we ourselves were confused about it as well. So like evaluation and data sets, what's the best way to evaluate a model anyway, right? So that seemed like a big topic. Let's make that a chapter. So it seemed fairly organic coming up with these titles.

8:17And of course, as we were writing this, again, it was all fully open source in the whole writing process. We thought maybe we should split up a chapter. So we split up models into two chapters, let's say one for specifically unaligned models versus aligned models. So it was an iterative process. Yeah. On that front, I definitely hear the passion coming through for that sort of licensing element of that. And I see that up front in the book. And maybe so I'm also very, very much like we've mentioned on the podcast multiple times that people need to be reviewing these things, especially as they see, you know, whatever 400 ,000 models on on hugging face and kind of parse through these things.

8:58But could you kind of give us maybe the pitch for engineering teams or tech teams that are considering open models, but might not be aware of the kind of various flavors of openness that are occurring within kind of, quote, open source AI? Could you just give us a little bit of a sense of maybe why people should care about that? And maybe just at a high level, what are some of these kind of major flavors that you see going on in terms of openness and access? Right. Yeah. I mean, I suppose first I should have a disclaimer, which is the quiet part that nobody usually says, which is almost a counter argument.

9:40It might not matter because in practice, nobody is going to sue you if you do something illegal, unless you're fairly big and famous, right? that's just a harsh truth and it's very frustrating that you know laws and enforcement are tend to be two separate things and there is a precedent in law that you're not meant to create a law unless you know definitely you can enforce it so to a large extent a lot of these licenses out there are questionable in that regard the other thing is a lot of these licenses are not actually let's say tested in court they're not actually formally approved by you know any government or legal process.

10:18So it's not necessarily legal just to write something in a license. You should probably be aware of recent developments in the EU, for example, they've proposed the two new laws, the CRA and PLA, two new acts, I should say, that are effectively saying the no warranty clause in all of these open source licenses might be illegal if you are in any way benefiting, let's say monetarily, even if it's indirectly. So you're a company releasing open source things purely for advertising purposes, but you're not directly gaining any money from it. We're still going to ignore the no warranty clause. So yeah, there's interesting stuff in that space.

10:49But I would say as a developer, the things that you should be aware of when it comes to model openness is that there's a difference between weights, training data, and output. Those are the three main categories, really. So licenses usually make a distinction with... Well, it's not licenses, it's more about the source. So are the model weights available? That's often the only thing that developers care about in the first instance, because that means they can download things and just play with that, right? But if you actually care about explainability or in any way alignment in order to figure out how you might be able to make a model aligned or unaligned or whatever you want to do with it, you probably do need to know a bit about the training data.

11:29So is the training data at least described, if not available? And when I say described, as in more than just a couple of sentences saying how the data was obtained, but actual full references and things. So a lot of models are not actually open when it comes to the training data. And then of course, the final thing is the licensing around the outputs of the model. Do you really own it? Are you allowed to use it for commercial purposes? And even if you are, it's highly dependent on the training data itself, right? Because if the training data is not permissively licensed, then technically you shouldn't really have much permission to use the output either, right?

12:02So I think even developers are kind of confused about the ethics around the permissions. So certainly legally, we're super confused as well. I have two questions for you as follow-up, but they're unrelated, but I'm going to go ahead and throw both of them out. Number one, the quick one I think is, could you define what an aligned model versus an unaligned model is just to compare those two for those who haven't heard those phrases? And then I'll go ahead just as you finish that and say, and what's the reason that I noticed, you know, licenses is addressed at the very top of the book. And is that framing the way you would look at the rest of the book or is that more just happen chance that it came there?

12:41I was just wondering how that fits into the larger story you're telling. Yeah. So for those who don't know, unaligned models, it's effectively, if you train a model in a bunch of data, it is by default considered unaligned. But in the interest of safety, what most of the famous models that you've heard of do, like chat GPT, for example, is add safeguards to ensure that the model doesn't really output sensitive topics, issues, anything illegal. It's still probably capable of outputting something quite bad, but there are safeguards. And the process of adding safeguards to a model is called aligning a model, as in aligning with good ethics, I suppose that's the implicit.

13:24Gotcha. Thank you very much. And then I was just wondering, like I said, the positioning of licensing at the front, is that relevant or is that just happen chance? We did sort of think of an order of chapters, let's say, and licensing just seemed like a good introduction, let's say, because it's before you get into the meat and the details of actual implementations and where you can download things and where the research is going, let's say. Well, Casper, as you were just describing the kind of framing of the book and also some of these concerns around licensing, I'm wondering if we could kind of take a little bit of a step back as well and think about what are some of the main kind of components of the open source AI ecosystem?

14:07The book kind of details all of these, but what are some of the big major components of the AI ecosystem, maybe beyond models? Because people obviously have maybe thought about or heard of generative AI models or LLMs or text-to-image models. But there's a lot sort of around the periphery of those models that make AI applications work or be able to run in a company or in your application or whatever you're building. So could you describe maybe a few of these things that are either orbiting around the models, if you view it that way, or part of this ecosystem of open source AI? Sure. I mean, there's huge issues, I would say, regarding, let's say, performance per watt, effectively electrical watt.

14:56There's a lot of development in the hardware space. And, you know, we have new Mac M1 and M2s, which might actually mean you can fairly easily do some fine tuning and or at least inference on a humble laptop without ever needing CUDA. it seems like there's a lot of shifts and paradigm changes when it comes to the actual engineering implementations. WebGPU is a big upcoming thing, which, I mean, it has technically been going on for a decade or more, but it might actually have reached a point where possibly we can just write code once and it just works in all operating systems on your phone. You can get an LLM just working wherever.

15:33But yes, I mean, there's effectively a lot of MLL style problems. It's one thing to have a theory of how to actually create an LLM, but quite another thing to actually train a thing, fine tune it, or deploy it in a real world application. So there are a lot of competing, let's say, software development toolkits, desktop applications. And I don't think anyone's really settled on one that's, you know, conclusively better than anything else. And really based on your individual use cases, you have to do an awful lot of market research just to find something that's suited to your use case. I ask this because we've had a number of discussions on the show about sort of training, fine tuning, and then this sort of prompt or retrieval based methodologies.

16:17So from your perspective, as someone that's kind of taken survey of the open source AI ecosystem and is operating within it and building things, what is your kind of vision for where things are kind of headed in terms of more sort of fine tunes getting easier and fine tunes being everywhere or kind of pre-trained models getting better and people just sort of implementing fancy prompting or retrieval based methods on top of those. Do you have any opinion on that sort of development? I know it's something that's on people's mind because they're maybe thinking about, oh, this is harder to fine tune, but is it worth it because I'm getting maybe not ideal results with my prompting?

17:01Yeah, no, it makes sense. I would say basically, if you're not doing some form of fine tuning, you're not producing anything of commercial value. Effectively, it's very much like hiring an intelligent human being to work for you without them having any particular expertise and not even knowing what your company does. That's what a pre-trained model is effectively. So you do need to fine tune these things or add some amount of equivalent, anything else that's equivalent to fine-chewing, let's say. In terms of things that actually predate LLMs, I think there's a lot of stuff that is very useful and even maybe far more explainable that people seem to be discounting just because it's easy to get some result out of an LLM just by prompting it.

17:42So people view it as good enough and they start using it even though it's maybe not safe, right? So one thing I would really recommend people look at is embeddings. Just by doing a simple vector comparison in your embeddings, you can find, you know, related documents. You don't really need an LLM to drive that because LLM is effectively, instead of explicitly making an embedding of your query, you know, conversing your query into a vector and then comparing it to other vectors in your database that correspond to, let's say, documents or paragraphs that you're trying to search through, your LLM is automatically doing that entire process.

18:17And it might make mistakes while it does that, right? It's going to paraphrase things, which it might get wrong because it can't have been due simple basic mathematics. It doesn't understand logic, right? So yeah, whenever it comes to things like, let's say, medical imaging, where there's a lot of interest in how can we use AI to improve this, people tend to get frustrated with how slow the uptake of AI is. But there's a reason for that, which is explainability is important, right? So the way I see things going is, yes, far more fine-tuning, more retrieval augmented generation type stuff, so rag stuff.

18:48And then also probably push into explainability. I don't really think there's much explainability in LLMs right now in general. Everyone's been so focused on LLMs with large vision models are kind of one of the newer things on the rise. What is your take on large vision models in the future and how they start integrating in? I was just, Andrew and Guy is talking about some of them now, and I would love your take on it. Sure. I mean, we didn't quite get to covering this in the book. I mean, that's how fast-paced things are. So multimodal things are super interesting. To me, my feeling is that it's effectively gluing together existing models into pipelines.

19:28And it hasn't been historically something that I was that interested in because that's more an application and it's not so much something you need to research per se. It's very similar to how the OpenAI people were very surprised that ChatGPT exploded in popularity, even though technically the technology is quite old. It's just, you know, you lower the entry barrier a little bit and then everyone actually starts using it because they can, right? So to me, the multimodal type stuff is similar. It could result in really innovative new companies popping up and new solutions that are actually usable by the general public.

20:00But in terms of the underlying technology, it doesn't seem that particularly novel to me. As you kind of looked at the landscape of models itself and the licensing of those models, the support for those models and underlying ML ops sort of infrastructure, the support for an underlying kind of like model optimization, you know, toolkits and that sort of thing. some people out there might hear all of these words like oh there's these llama two models and there's now mistral and then there's you know now ye and uh like all of these as you were going through and researching the book and also kind of doing that as an open source community can you orient people at all in terms of the kind of major model families so you already distinguished between sort of models and unaligned models.

20:52Is there any kind of categories within the models that you looked at that you think it would be good for people to have in their mind in terms of, hey, I have this application or I have this idea for working on this. I maybe want, I'm listening to Casper. I wanna maybe fine tune a model. I've got some cool data that I can work with. Where might be a sort of well-supported or reasonable place for people to start in terms of open LLMs or open text image models, if you also want to mention those? Sure. Yeah. I mean, because there's just a new model basically being proposed every day, I mean, often it's a small incremental improvement over a previous model.

21:32So in terms of actually trying to compare them from a theoretical level without looking at their results, there isn't really much to talk about in terms of large model families. They might be in an extra type of layer that has been added to a model in order to give it a new name, let's say. Nothing particularly stands out there. I mean, we do have a chapter on models where we try and address some of the more popular models over time, the proprietary ones and then the open source ones. But I would say nothing particularly stood out to me over there. I suppose the more interesting thing in terms of actually implementing something for your own particular use case is starting with a base model that has pretty good performance on presumably other people's data that looks as close as possible to the data that you actually personally care about.

22:19So you don't have to wait too long when then fine tuning it on your own data. So for that, I think the most important thing is to take a look at the most up-to-date leaderboards, right? And there are quite a few different leaderboards out there. We do also have a chapter on that. And that was interestingly also a nightmare to keep up to date because the leaderboards themselves are also changing regularly. new leaderboards are being proposed for different things. And take a look at a leaderboard, pick the best model performing there, and then start doing some fine tuning. That would be my MO. This kind of gets to one of the natural questions that might come up with a book on this topic, which is things are evolving so quickly.

23:00And you mentioned kind of the strategy with this book being to have the book be open source, have multiple contributors. and I'm assuming part of that is also with a goal for it to be updated over time and kind of be an active resource. How have you seen that start to work out in practice and what is your hope for that sort of community around the book or contributors around the book to look like going into the future? Sure yeah I mean like for the evaluation data sets thing we already have you more than a dozen leaderboards, just the names of the leaderboards and links to them, and then what benchmarks they actually implicitly include.

23:41And yeah, we have comments at the bottom of each chapter, which are driven by GitHub effectively, powered by utterances, which is this integration tool helper. So you don't need to maintain a separate comments platform, let's say. It also encourages people to open issues, open pull requests. If we've made any mistake or something is out of date in the book, I mean, we definitely encourage people to fix things or complain about things, which I suppose it's also good from the perspective that nobody can sue you for writing something wrong because in the first instance, what they really should do is just correct it, right?

24:15You can't really open a cold case. And for that reason, I think it's also lowering the entry barrier for people to contribute in the first place. They don't have to worry about what they write and whether or not people will disagree because if they disagree, they can fix it right. They can start a discussion. Nobody's going to immediately file a lawsuit. And yeah, so we've had quite a lot of interesting discussions already on the individual chapters. The other thing that we highlight is that as soon as you make a contribution to anything, your name is automatically displayed at the bottom of the individual chapter, as well as the list of contributors in the front.

24:46So yeah, it's a good way to get your name as a co-author in a way of a book. I mean, it's a 21st century book as well. So it lives fully online. Everything that is committed to the repository is automatically built and published immediately. And before we get too much further, some people in the audience might be wondering, like, I mentioned the name of the book, and of course, you can find it by Googling it, I'm sure. But what is the best place to find the book? And then also, as a contributor, you mentioned the links at the bottom of the pages, but I'm assuming there's a GitHub associated with the book.

25:19Do you just want to mention a couple ways for people to find it? Sure. I mean, the easiest is probably to go to book.premai.io. Yeah, apologies that there's an AI and an IO. It seems to be a thing. But yeah, so book.premai.io or I mean, you can also just probably Google PremAI and you can find our Github, which is also github slash premai-io. That's a thing. All the AIs and all the IOs. Exactly. We have quite a few repositories that, I mean, some of them were just archived right now because we're constantly running different experiments, changing the entire architecture of the things that we're building.