#218 Jeff Boudier: The Future of Open Source AI Development (Hugging Face)

0:00Well, with Hugs, this new solution, we're trying to make it as easy for an enterprise to build their AI features with open models as it is with closed models. with the key difference that they can host everything within their own environment. And so HUGS, this new solution, is the way for the enterprise to take all these great proof of concepts, prototypes, new features that they've built using closed models, and bring it in-house and hosting the model themselves so they can really control their own AI building. Even if you think it's a bit overhyped, AI is suddenly everywhere from self-driving cars to molecular medicine to business efficiency.

0:46If it's not in your industry yet, it's coming fast. But AI needs a lot of speed and computing power. So how do you compete without costs spiraling out of control? Time to upgrade to the next generation of the cloud, Oracle Cloud Infrastructure, or OCI. OCI is a blazing fast and secure platform for your infrastructure, database, application development, plus all your AI machine learning workloads. OCI costs 50 % less for compute and 80 % less for networking. So you're saving a pile of money. Thousands of businesses have already upgraded to OCI, including MGM Resorts, Specialized Bikes, and Fireworks AI.

1:35Right now, Oracle is offering to cut your current cloud bill in half if you move to OCI. This is for new U.S. customers with minimum financial commitment, and the offer ends at the end of the year, December 31, 2024. See if your company qualifies for this special offer at oracle.com slash IonAI. That's IonAI all run together, E-Y-E-O-N-A-I. So go to oracle.com slash IonAI to see if your company qualifies for this special offer. Hugging Face is the leading open platform for AI builders, people who build their own AI using open source libraries from Hugging Face and models, data sets and applications that are contributed by our community.

2:34and at Hugging Face I lead product and growth so in a nutshell that means all of the commercial activities of Hugging Face and my role is to help companies build with Hugging Face technologies and to build the business of Hugging Face. Yeah I want to get a little bit of background before uh i talk about what you guys do i mean most people are familiar with the leaderboard the model leaderboard uh but i had uh a friend who may or may not work at hugging face now a french friend come to me uh three or four years ago uh he was looking at hugging face and he said what do you think of Hugging Face?

3:27Sort of getting the temperature on what the community thought. And I didn't know anything about Hugging Face. And so I have a couple of questions around the origin of Hugging Face. When did it start? Where did it start? And who came up with the wonderful name. Well, it started eight years ago now. And it was started by Clément Delongue, Julien Chaumont and Thomas Wolfe, who started with a very different sort of mission. Initially, they wanted to make AI more approachable, and maybe even fun to chat with. and if you roll back the tape to eight years ago it was kind of a crazy idea because the ai that we know now really didn't work at the time that's right we had we didn't even have at the time transformers and so it was really building upon the earliest tools available in deep learning and then Transformers came along and that changed everything and that changed the company as well.

4:48Right. And part of the origin story of the name, of course, with Hugging Face being a mobile chatbot for teens at inception, the Hugging Face name maybe makes a little bit more sense, but also behind it was an ambition, an ambition and sort of a bet to be the first company to IPO with an emoji instead of a three or four letter ticker name on the stock market. So, yeah, we're keeping up with that bet. So it doesn't refer to the alien monster. It has not. I am not familiar with the alien monster. Are you referring to the alien movie? Like the thing that comes and grabs? Yeah, that's not the kind of hug.

5:42That's not the kind of hug that we're going for here. Yeah, that's what I assume. But is there a French idiom that it refers to? No, no, no, no. No, no, it's the emoji. That one. Okay. The one with the two hands. Oh, I see. Yes. The hugging space. now you get it yes it's that cute little emoji with two hands extended that wants to give you a hug to give you a hug okay uh and and it started out as a suite of tools of when you say to help developers build and and is it still a suite of tools primarily uh and how does you know you you host hundreds of models if not more at this point and i've always wondered what does it mean to host models do you operate a private cloud where people can access these models and you have them you know in in virtual machines on some data center somewhere what does it mean to host models And then do you have a suite of tools for people to build models?

7:06Yeah. Yes, we do. And I feel like, Craig, you're up for an update on Hugging Face. Today, we host and offer over 1 million models publicly on the platform. It's any kind of model you can use to build AI features. It can be models for generating text like you would with an assistant like ChatGPT. It's models to process text. If you want to figure out whether an email is important or not or summarize something translated. It's models for audio. So everything that we're saying right now is going to be transcribed by a transformers model into text. with diarization to know when Craig is talking and when Jeff is talking.

8:01It's models for images. So you can create images from text. You can understand images, caption them. It's even models for video. Actually, the number one model on the Hugging Face Hub today is a video generation model. So you can create video clips from text. And this goes on and on to every facet of machine learning or AI. It goes with models specifically for biochemistry, for time series, to understand any kind of content. And so Hugging Face today is more than a suite of tools. It's really the leading platform for anybody who wants to build their own AI using open source technologies and open models.

8:54It started as something very different, as I said, but what really changed everything for Hugging Face and for industry is the advent of the transformers type of model and transfer learning. Transfer learning, which is the ability to take an existing off the shelf machine learning model and then improve it, adapt it with little data. And that's why today we have a million freely accessible models on the Hanging Face Hub. And we have hundreds of thousands of data sets. So all the data that is used to improve models or create new models altogether. And there are hundreds of thousands of hosted applications on the platform.

9:45Yeah. Yeah. Yeah, well, let me ask about what does hosting mean? As I said, do you operate a private cloud? Is it a list of links then that link to models that are on other clouds? Or is it a link to GitHub repositories? I mean, what does it mean to host? So you can think as its simplest expression, if you go to the Hugging Face hub, which is the name of our website and our cloud hosted platforms where all the models, the data sets, the applications are available. So that's HuggingFace.co. You will have access to all the models, the data sets, the applications. You can try them right there from the page.

10:44So they're available for inference. You can see all the details that have been provided by the model providers, like Meta provided LAMA 3.2, the latest iterations. There is a variety of different models there. You can see all of their details. And then you can use them in whatever Hugging Face open source library or external compute platform. And so that's where the unique ID, so the model ID for that particular model or data set can be used directly by companies, data scientists, machine learning engineers, software engineers, simply by using the hugging face ecosystem of open source libraries.

11:31The most famous thereof will be the Transformers library, library, which is today how all AI builders are accessing all of these new models. And so you have sort of the visible part of the iceberg, right? Which is this Hugging Face Hub, this website where you can find all these resources. And then you have the hidden part of the iceberg, which is the vast ecosystem of open source libraries so that people can actually use all these resources and then build their own AI with them. yeah um so uh i'm i'm not quite understanding yet the um so uh i build a model uh i'm a big company not meta not open ai not amazon i'm i'm a big company you've never heard of and I build a trillion parameter multimodal model, and it's open source, that model exists in my data center, wherever my data center is.

12:51If I'm going to host that model on Hugging Face, what does that mean? Does that mean I port the model over to HuggingFace's data center? Or does it mean that I make my model available to HuggingFace through an API call? or does it mean that I put my model weights in a hugging force repository so that people can use the weights on their own models? I mean, how does that work? It's kind of a combination of all those things. And maybe as you built your own model, you have been using HikingFace open source libraries, right? Maybe you've been using our Transformers library to do fine tuning of an existing model.

13:56Maybe you've used our Nanotron library to build a pre-trained model from scratch. But in any case, if you want to host the model on HikingFace, what that means in practice is uploading the model weights within a repository that is hosted within the Hugging Face cloud. And you can choose to make this model private or to make this model public. We actually have as many models that are hosted privately on Hugging Face by companies who want to collaborate on their models. And they want other teams to be able to access them, access them using the Hugging Face hub or the Hugging Face open source libraries.

14:40And so that's what hosting the model means. Of course, if you choose to make the model public, then you can contribute it, assign a license to it so that other people can use the model. And to use the model, they will use the open source library, which has implemented the code of the model. And so most of the models today on the Hacking Face Hub, they're either transformers models or they're diffusers models or they're sentence transformers models, etc. which means that you can use those model weights directly within the open source library to do things like serve predictions, that's inference, to do things like adapting the model, that's fine-tuning, to do things like evaluating the models, see if the predictions conform to what you want the model to do.

15:40Yeah. And tell me how it's different from GitHub, which I'm much more familiar with. Yes. Yes. So GitHub is sort of a collaboration platform around code and is built from the ground up for software developers. And for machine learning, for artificial intelligence, you're using kind of different processes. you're using kind of different artifacts, right? Where you're dealing with large data sets. You're dealing with humongous files, which are the model weights we talked about. And you're interacting with these things with completely different tools. And so you can think of the Hugging Face Hub as sort of a GitHub that has been built from the ground up for machine learning to work with model weights, to work with data sets, to work with all the open source ecosystem of libraries that are used by AI builders to do AI.

16:45Yeah. And do you compete at all with GitHub? I think it's very different sort of a feature set. And I think AI builders will use Hugging Face to host their models, their data sets, to work with models with data sets. And software developers will work with GitHub to host their code and review code. What's interesting is that we are adding a lot of collaboration features on top of the Hugging Face hub. And maybe that's something that is not fully yet appreciated by the tech community, which is that today, the hub is really the central place where all the collaboration happens within companies, between companies, between model providers and model users, etc.

17:46So today on Hugging Face, you can look at a model and start a discussion and propose some changes via pull requests. You can discuss things and create new projects together with your team or with the community. What's interesting, though, to your question is that the audience, the AI builders, is sort of a community that has evolved a lot over the past four years. And when I joined Hugging Face four years ago, that community was very much researchers, AI researchers and data scientists. And these researchers and data scientists were using Hugging Face day to day to share models, work with models, etc.

18:39And today, that audience really has expanded first to engineers, machine learning engineers, but now to software developers. And so now we have more and more software developers who are sort of AI native or AI first, and they build AI features using the new set of tools that is being designed for AI. And so by and large, that's Hugging Face. So that's why today there are over 5 million AI builders using Hugging Face. yeah and when you say there are a million models uh hosted on hugging face as you were saying a lot of those are uh models that are uh you know people companies have an account and their developers are working on the models on the hugging place face platform they're not uh necessarily open to the public.

19:39But there's a large segment. I'd like to know what percentage of the million plus that is that are open and they're open source. How does someone navigate that? Yeah. That is a great question. and it is at the heart of the product experience of the Hugging Face Hub. And when I mentioned a million models, I was really referring to the ones that are open source, meaning they're freely accessible. You can go out there and you can check them out and download them. We host overall, there have been over 3 million models, datasets, applications that have been built on Hugging Face. But again, there's the visible part of the iceberg and the hidden part of the iceberg.

20:30And the visible part is all those public models. And the hidden part is all the private models. There are over 100 ,000 organizations that have been created on Hugging Face in order to be able to collaborate privately building AI. So it's building as a team, like private models, sharing private data sets, building demos for the internal team on AI features, and doing all of this as a team. And to your question, like how do you navigate all this? So one answer is leaderboards. And you had mentioned earlier, the leaderboard. And I think by the leaderboard, you mean the open large language model leaderboard, which is the reference today.

21:21if you want to compare the most popular large language models, so the ones that generate text from text. And there are over, I think, 10 ,000 of those models that are evaluated on this particular application, the Open LLM leaderboard. But as I said earlier, today on the HikingFace Hub and with Transformers, you can do any kind of machine learning application, and that might be text, but that also might be audio and that also might be images or videos or 3D, etc. And even if you look at text, you might be interested in how models perform against academic benchmarks, mostly in English, but maybe you're interested in how they work in a different language.

22:14And so there are hundreds of leaderboards today on the Hugging Face Hub so that if you're interested in building an application for arabic speakers then you can look at the arabic llm leaderboard if you're interested in speech to text then we have a special leaderboard for that so leaderboards is definitely a very popular tool to navigate through the diversity of models available on the hub and then we have other tools and if you visit today the HikingFace hub so it's HikingFace.co and go to the models you will see on the left side of your screen a sidebar of filters and you can filter every all models of the hub for maybe their task is this classification translation transcription you can filter for the library that you're going to be using the model with or the license that you want the model to be in or the language that you want the model to be supporting so that makes it easy to go from the million public models all the way down to the the compatible models for your use case and then everything is sorted by popularity so you see very quickly what is the signal from the community in terms of usage the number of downloads the number of likes of the models and that's how data scientists and software engineers today use the hub to quickly find models that should probably be considered for their use case.

23:52Yeah, and these are all open source. So there are developers continually contributing code to the models. Is that right? Yes. So open source AI is sort of an umbrella term right and then we can go into specifics of what that means for a model a machine learning model typically we talk about open weights model meaning like the weights of the model are accessible and and yes so the model can be published by the contributor with whatever license they want to apply to the model and we make it super easy on the hub whenever you are creating a new repository a new model or data sets to choose the license that you want to apply so that people can build ai in confidence and the you also mentioned applications being hosted yes so people are using these models to build applications uh is there are there leaderboards for different kinds of applications how does someone find the applications and are the applications uh free to use are there what sorts of licenses are attached to them uh if they're built on open source models yeah can you talk about that yeah of course so as i said there are hundreds of thousands of applications today that have been built by the community and by and large these are sort of demos demos of what people can do with a model demos of ai systems and different models working together and so whatever you see on x or linkedin or whatever social network demo of a machine learning model if it's an open model then it's probably a demo that is hosted on hiking face So that you can just go on that particular website URL and click and add an image or type some prompt and see how the model behaves.

26:09So these are the HikingFace machine learning applications. We have a special name for them. We call them spaces. So if you go today on the HikingFace Hub, there is a tab for models, a tab for data sets, and then a tab for spaces. And it's really incredible to see what the community builds every day. We actually feature the spaces of the week. So if you go to hikingface.co.spaces, you will see the spaces that we feature. And of course, if you have an account, then when you log in, you see all the trending models, that assets and spaces. So it's a great way to discover what the community has been up to.

26:52Yeah, when you say trending, so there's a leaderboard for applications as well? So, if you will, you can see everything on HikingFace through the lens of what is trending, right? So what is trending means what is used, what is being liked by the community. So yes, that's one of the main things that you see when you open Hugging Face. If you're one of the five million AI builders that use Hugging Face every day, then you open the website and you see what is trending today, what has been trending over the past week. You see all the activity around your own repositories, what your team has been up to, the people you follow, etc.

27:39Yeah, that's fascinating. And it seems to me that must be a tremendous resource for entrepreneurs that want to pick up an application and commercialize it. How easy is that and how often does that happen? Well, it happens every day and happens all the time. In fact, we had a fun stat that popped up recently, which is that on Hugging Face, every 10 seconds, there is a new model or data set or application that is being created. And I think it speaks to how AI has changed how we build technology. Where it used to be that in order to build a technology, a feature in an app, the first thing you would do is to write a million lines of code with all the rules that define how it should work.

28:48And today we have a very different approach, which is if you want to build a new feature in your application, you're going to start by looking like, is there a good model that can do this? And you can start from the model. And so in that way, AI or machine learning is the new way to build technology. And that's why we see so much activity happening on Hugging Face as businesses build their own AI using Hugging Face. Yeah, you have a wonderful overview of what's happening in AI through Hugging Face. And you can see what's hot and what's decreasing in interest. I would imagine, and again, on the leaderboard, I would imagine you're hosting agents now.

29:56Do you have a leaderboard for agents? I mean, that's kind of in between a model and an application. Yes. There's lots of conversation right now around what do people really mean by agents? But yes, we definitely have a framework for people to build their own agentic systems using Hugging Face models. We actually have a great demo implementation of that within our own sort of open chat GPT, if I may call it that. It's Hugging Chat. It's free to access. You can go to hf.co slash chat and you'll be presented with an interface that looks familiar to you if you ever used ChatGPT. But the catch is that the models underneath HuggingChat are all open models and the application itself is open source.

30:54And within the open source application, we have implemented an agentic sort of system so that you can define the different tools, which can be AI models or they can be AI applications like spaces that you want your large language model to be able to use as an agent and then create your own assistant to do that. And really creates, it only takes a couple of minutes to do. So yes, we have all the tools and the resources and the open source libraries for companies to build their own agentic systems. And it's not so much like one leaderboard or one isolated feature. I think it's something that's enabled by everything that we built.

31:44Yeah. And where do you see in the trends? What is falling off the radar, is getting less attention? Where does the activity seem to be going? And do you have any sense of what's next, given what you're seeing on Hugging Face? Yes, of course. And this is, by the way, fully transparent with the community. As I said, if you log into HikingFace today, you will see exactly what people are doing, what's trending, etc. And what we are seeing is that the pace of innovation has really accelerated across all different modalities. and when i looked today at the top 10 models on hiking face and by top 10 i mean the most trending through downloads and likes and activity on the platform i think it's the first time that i saw that eight uh the top eight models were actually not natural language processing not text uh text generating models like all the top eight models are models to generate videos models that can take images as well as text as input that's aria models that can generate images from text like flux models that can transcribe speech into text or create speech from text.

33:29So it's really, really exciting to see that. And I think it's the first time in the short or long, depends when you started the history of AI, that all these different disciplines, which used to be completely separate fields within the research community, are now using a common set of architectures and a common set of tools whether you're creating or processing text images videos audio etc you're using a transformers model or maybe a diffusers model but that's also partly transformers model you're using the same set of tools you're using the hugging face hub to access resources and to host your own models you're using the the Transformers library to do fine tuning and evaluation, et cetera.

34:23So I think it's really, really exciting. And we see a lot of cross, cross contamination is not the right word, but you see a lot of techniques, enriching fields that have been proven in others. And that's been definitely the case where we see all of the great work that's been done by the diffusion community, mostly focused on creating images like low-rank adaptation that is now benefiting the natural language processing, the text AI field, and we see the same thing happening the other way. So it really goes to show the power of building in the open, of collaborating so that everybody can progress much, much faster.

35:20Yeah. I just had somebody from Mozilla Foundation on talking about open source. Okay. And I think they're doing something with you to try and complement your work. but there everyone seems convinced that open source is going to win but the most powerful model out there remains a proprietary model and at what point do you think it costs a lot of money to to build these huge models and to meta's credit it's It's continuing to invest. But at what point do you think open AI will finally pull ahead of the others to a degree that people aren't going to be able to catch up in the open source side? Or at what point do you think OpenAI will suddenly sink to just another model in the sea of open source models and at that point have to succumb to the open source impulse?

36:51Yeah, I think it's something where we've learned a lot over the past few months. And to come back to your original question, yes, we've done a lot of great work with the Mozilla Foundation and Firefox in particular. So I mentioned Hugging Chat. It's now available directly within Firefox. You can go and activate it so you can use Hugging Chat within your browsing experience, which is really, really cool. But to go back to your main question, I think what we've seen over the past year is that the top performing open models have been catching up to the top performing closed models and the pace at which they are catching up has been accelerating.

37:35And we're not anymore in a situation where if you want the best performing model, you have to use a closed model. Today, we have open models that are playing right there with the best closed models from open ai or anthropic and so the situation has really really changed and and i think that's gonna really shape how companies will build their own ai i think we kind of are living in an anomaly right now and this anomaly is one in which all the companies that have been sort of woken up to the promise of AI for their business when they witnessed what ChatGPT enabled. And they've woken up to this reality of AI and they're sort of struggled to quickly figure out what it meant for their business, for their product, for their customers.

38:48And in this sort of race to find solutions, they've quickly adopted closed models as the easiest way in, as the easiest way and the fastest way to be able to put something out in the market. it. And I think that's a very unusual way for companies to think about creating new technology and creating new core technology that's important to them, that processes their customers' information, etc. Generally, when companies create their own core technology, they want to do this in-house where their data is safe, where they understand what they're doing, where they're building for the long term. And you can only really do that in AI if you're building with open source.

39:47Because if you're building upon a closed model that's hidden behind an API, then maybe next Monday the new version changes and you are completely oblivious to it. And if your customer experience changes, there's nothing you can do about it. And so I think we're living in this sort of anomaly where companies have not really figured that out. and what's what I think will happen next is that they will internalize all these new sort of AI proof of concepts and advanced beta features etc so that they can control their own destiny now that we understand that to build new technology you need to think about how to build it with AI The only responsible thing to do for an enterprise is to control and build it themselves so they know what's going on and they can be accountable for it.

40:56And so, yeah, I'm really excited about this new era. There is no more sort of excuse because we're not anymore in a world where only the closed models can do what they can do. And what's missing is sort of the making it as easy part. And that's one piece that we're really working hard on at Hugging Face. This is the reason that we launched what we called Hugs. And you were asking me earlier about the origin of the name and what is this weird Hugging Face thing. Well, with Hugs, this new solution, we're trying to make it as easy for an enterprise to build their AI features with open models as it is with closed models with the key difference that they can host everything within their own environment right either building their own technology and hosting their customers data on say AWS that they can build within their own AWS account and same thing with with other clouds and so HUGS this new solution is the way for the enterprise to take all these great proof of concepts, prototypes, new features that they've built using closed models and bring it in-house and hosting the model themselves so they can really control their own AI building.