OpenAI introduces custom model building with ChatGPT

0:00OpenAI is expanding their custom model training program. So today on the podcast, I want to dive into what this means, why this is important, and talk about some of the interesting use cases that we're seeing from this fine-tuned model. There's a bunch of companies that are actually using these fine-tuned models that you probably didn't know about. I personally didn't know about. There's some very interesting use cases. I'll talk about the features, what's going on. Let's get into the podcast. So the first thing that I want to talk about here is just the fact that obviously this is a use case that a lot of people have been using.

0:32This is essentially the custom model program, and it's set up essentially to help companies create generative AI models that are tailored for specific use cases. So you can imagine if a law firm or a healthcare provider wanted to use ChatGPT, but they're like, hey, it doesn't know all this like really specific stuff about the healthcare industry. They would be able to provide extra data and they would fine tune a model. it's essentially chat GPT or GPT-4, but with all this extra industry-specific knowledge. And this is what, you know, they're kind of helping people to do. So custom models and all of this was actually introduced at OpenAI's first developer conference.

1:11If you remember, they had their big dev day. And this essentially allowed businesses to work with OpenAI researchers. So this wasn't like you could do it yourself. You pretty much had to go collaborate. And I actually saw some, like, I saw some reports saying that, like, you had to spend, like, a million dollars or something crazy in order to do this. So it definitely wasn't for everyone, but there was a bunch of people doing it. So since they launched this, they did a big blog and they said, quote, dozens. So I'm assuming, I don't know what, like dozens, like a handful of dozens. I don't know.

1:42There's probably like somewhere between 12 and 100 customers or companies that have specifically done this with them and worked with them on this custom model program. But OpenAI said that they needed to expand the program to quote unquote maximize performance. I'm not 100 % sure what maximize performance means in this context, but what I can assume is that this is probably a good way for them to make more money. So if performance is, you know, financial returns, then this is a great thing to do. And of course, for companies, like I'm not saying they're, you know, they're selling something bad here.

2:14I'm just saying that's probably their main motivation. So one thing that is interesting, there's a new feature here and it's called assisted fine tuning. So this is added to the custom model program. But essentially what assisted fine tuning is, is of course you can go and they kind of have two options. The old option, which was like OpenAI was going to help you do this. And they have a new option where you can actually do this yourself. So it's kind of like self-led. You can go bring in your own data. You can go train this yourself without having to use OpenAI. And that is going to be cheaper.

2:46And I think it's going to be a really interesting option for a lot of companies. So there's a bunch of different custom trained models that are being that are being developed right now on top of OpenAI's kind of base model of GPT-4. And I think this is a great example. So they had a couple examples in their blog post that they talked about when OpenAI kind of made this big announcement of these new features. I mean, the biggest feature here really is that like this program already existed, but now you can go and do it yourself without needing to spend a million dollars to get access to one of the researchers.

3:15So in my opinion, that's huge news. They talked about a couple different companies that have been using them. So one is SK Telecom. This is a big telecom company in Korea that I like. Like I've been hearing about them do a lot of investments and a lot of moves in AI. So this isn't a shocker to me, but they fine-tuned GPT-4 for help with a bunch of like specific telecom related Korean conversations and some other things that they were doing. Another famous one that I didn't realize had a partnership with OpenAI on this is Harvey AI. This is a company that raised millions of dollars earlier last year.

3:52and I remember when there's all the hype around Harvey AI because it helps lawyers with legal cases and stuff and I remember a lot of the hype was like I'm sure there's way more now but there was like 15 ,000 law firms that were on the wait list to use Harvey AI so I think it's very interesting I watched a demo of this actually in action and essentially using Harvey AI they were able to feed a bunch of specific data in relation to like law for law, the law field. And I think they actually gave it like a bunch of cases. So if you know lawyers, essentially what they do is when there's a lawsuit, they're going to go look at a bunch of old cases, how they were resolved and try to find what the precedent was on an issue.

4:35And this is something that Chachupiti notoriously struggled with. There was like, you know, a lawyer that asked it for, you know, some sort of like legal, some sort of legal precedent on an issue. And it completely invented a case that never existed on like some airline lawsuit he submitted the case he got in big trouble because he completely submitted a fabricated case that never happened it was a hallucination and this was kind of like a big example of now some of the issues that could arise with these ai tools so that being said lawyers had a lot of skepticism and that's i think where harvey ai was born and why it's important is because to avoid problems like that um to i i'm sorry i was just thinking i'm like that would have been like a genius marketing move for harvey ai to essentially hire the lawyer to like just you know uh do that and then whatever his consequences are pay for them so that it can just prove how why you need them and you can't just use chat chippie out of the box but anyways probably not what happened this is good conspiracy theory anyways um it came up with a really great uh if you watch it side by side with chat chippie t and with what actually they were able to fine tune harvey on it comes up with really great results so i saw a demo where they were like asking it for precedent on a specific issue.

5:46ChaiGPT gave kind of a three paragraph blurb about what you'd expect, right? Like it just kind of pulling from whatever it had. And then when Harvey was asked, it was able to use the natural language processing, like the LLM kind of of GPT-4, but with all the data in context, it was able to actually outline four real cases. It can add links to those cases. And then it's able to, yeah, essentially outline like actual useful data and a bunch of different questions and things that were being asked in relation to kind of like case law. So very, very useful. And of course, not possible just out of the box with GPT-4.

6:23I think what's interesting here is OpenAI said that they think most organizations are going to create their own customized models for their specific industries. This is actually the same thesis I have as I'm building out AI Box, which is a no-code AI app builder marketplace. We believe that there's not going to be just, you know, the handful of big players we see today, but that there will eventually be thousands of AI models in every industry that are specifically good at doing different things. And so that's why we're building out AI Box, so that we will have a platform that people can access all of these different models on one account, right?

6:56You don't need to make 20 different accounts to go use all of the different best-in-case, best-in-class AI models. You can go get them all on one account, and you can also mix and match them together, link them together, chain your prompts together. So that's what I'm building. And I'm excited to see that OpenAI sees the same future that essentially I'm building for. What is interesting here to me is that OpenAI right now, they're getting close, apparently, to reaching around$2 billion in annualized revenue. And they're planning a$100 billion data center with Microsoft. So right now, I think that these kind of custom model training that they're working on right now is seen as kind of a way to sustain some of their revenue growth.

7:35while they're also trying to reduce, like there's a huge strain on their model serving infrastructure. Like this is, it takes a big toll. So these fine-tuned and custom models, I think are a lot more efficient. And I think because of that, that could help alleviate OpenAI's, like obviously this is historical. They're, you know, this compute capacity challenge that they're facing right now. And I think this could actually kind of help to alleviate that. So what's interesting is OpenAI actually introduced some new model fine-tuning features for GPT 3.5, right? Their free version that anyone can access.

8:10And this includes a dashboard for model quality comparison, third-party integration support, and they're kind of starting with weights and biases and some tooling enhancements. What I think is interesting, though, is that details about fine-tuning GPT 4, which was actually made available in early access during dev day, they didn't actually say anything about that, So I'm not sure when that's going to come out, but I'll definitely keep you up to date. I think this is a fascinating new development we're seeing out of OpenAI. It solves some problems for them. I think it helps generate extra revenue.

8:41But overall, I think this is amazing for the community as companies are going to be able to make some incredible fine-tuned models on top of what OpenAI has. So I'm excited for that. And I'll keep you up to date on whatever amazing innovations we see there. What I would love for you, if you learned anything in this episode, if this was interesting in any way to you, it would really, really make my day if you could leave us a review on the podcast. I would be super thrilled to hear your feedback. You can, if you're on Spotify, drop us some stars for an Apple podcast. Go ahead and leave a comment.

9:10Really, really appreciate it. But I hope that you have an amazing rest of your day.