Next-Gen ChatGPT: Exploring the Power of GPT-4's Latest API Release

0:00What can 160 years of experience teach you about the future? When it comes to protecting what matters, Pacific Life provides life insurance, retirement income, and employee benefits for people and businesses building a more confident tomorrow. Strategies rooted in strength and backed by experience. Ask a financial professional how Pacific Life can help you today. Pacific Life Insurance Company, Omaha, Nebraska, and in New York. Pacific Life and Annuity, Phoenix, Arizona.

0:57Yeah. almost made OpenAI somewhat of a kingmaker because certain companies that they have given this early GPT-4 API access to have went on to raise millions and millions of dollars because, you know, they were beating on all their competitors with a superior product. With this announcement of this new AI model being available, they are also announcing that GPT-3.5 Turbo, Dolly, and Whisper APIs are all also generally available, which, you know, a lot of people have had. I've had those integrated into my different platforms for a number of months now because it was actually back in March that they originally launched the API for ChatGPT in general, which obviously made a big impact.

1:34A lot of people integrating ChatGPT into their products. As previous to that, there was one called DaVinci, which was not quite as powerful. Now, back in March, they originally launched that, you know, ChatGPT API, but it's taken them all the way until now to actually get this GPT4 version publicly available, which I believe is going to have a massive impact as I personally do virtually nothing on regular chat gpt i always use gpt4 because i find the outputs are so much better um you know like 20 30 40 better i would say on the quality of the outputs and so now that they have this api available virtually every product that already has chat gpt integrated into it to do a lot of really impressive um different moves or plays in ai is now going to get a 20 to 40 upgrade for the entire industry which i think is really impressive so earlier this month they released their first update to the chat-based models and they said that they're they're envisioning a future where essentially these chat-based models can support any use case so today they're announcing the depreciation plans for some of their older models which are using the completions api and they recommend that any users adopt the chat completions API, which essentially is, they're essentially moving from text completion to chat completion.

2:57And I'll talk about that in a second. But as far as chat should be, or as far as GPT4 goes, this definitely is their most capable model. And they've said that millions of developers have requested access to GPT4 and their API since March. And the range of innovative products leveraging GPT-4 obviously is growing every day. We see new tools popping up all the time. And I specifically have talked to different people at investment firms and at startups that are asking me like, hey, because of your podcast, do you get any like special ability to get onto chat GPT or GPT-4's waitlist and get access to the API sooner?

3:36A lot of people really want this for their product. I've heard from people specifically that say, hey, we built this awesome product, but we feel like we've kind of maxed out as far as it can go without getting the GPT for API. So I think this is going to be a massive step for a lot of those people. And this is how they're going to roll this out, though. This isn't just for every single person in the world. Today, all existing API developers with a history of successful payments to OpenAI can access the new API. So essentially, if you have been using, you know, chat GPT's API for your current business, you can get it.

4:13Otherwise, you will have to wait. But they did say they plan on opening up access to new developers by the end of this month. So you're gonna have to wait about a month if you haven't been integrating it already into anything you're doing. So anyone that's seriously building will already have done this and be ready to go. But new businesses will have to wait 30 days. And then they're going to start raising rate limits, which brings up a whole nother, you know, point which is the fact that they're going to have rate limits they're going to have limits to how much you can actually use this if you know for gpt4 right now um it gives you a certain amount of you know uses per hour and conversations per hour you can have with this thing which is going to be kind of interesting and perhaps a little sketchy for some developers depending on what this looks like i need to look into this but if the rate limits are bad right it's not letting you have a lot of usage, then perhaps this isn't something people want to implement immediately if it's going to curb the usage of their users.

5:10But that'll be left to be said as we got to look into that. And they said that that's going to depend on their compute availability. So based on the stability and sort of the overall readiness of these models for production scale use, they're also going to be making the other models I mentioned earlier publicly available gpt3 turbo dolly and whisper and they are also working on safely enabling fine tuning for gpt4 and 3.5 turbo which i think is going to be really really interesting and they expect that feature is going to be available later this year but being able to fine tune gpt4 to whatever your custom needs are i think is going to be quite amazing apparently they introduced chat completions API back in March.

5:55And they said that now that accounts for about 97 % of their API GPT usage, meaning people are using it for tools where they're essentially in sequential order. It's like a conversation and it's not just, you know, completing a paragraph or a piece of text. So essentially people are taking the chat GPT model and the concept and they're implementing that into everything they're doing versus older models that were more focused on just one query or filling out one response. Now they've integrated this where everything is having a conversation. They said that the initial completions API was introduced back in 2020 to provide more of a freeform text prompt for interacting with their language models.

6:42And they've since learned that they can often provide a lot better results with a more structured prompt interface. So the chat-based paradigm has proven to be really powerful in handling the vast majority of previous use cases and new conversational needs. So while it's currently providing a little bit higher flexibility and specificity. So in particular, chat completions API structured interface, which is like system messages and function calling. Okay, so pretty much what they're talking about though when they're referring to all this is the fact that when they do some of these API calls. So when you are like, when a company, not necessarily on chat, but when a company is setting this up, they can set up a system call, a user call.

7:29And then of course, there's the AI model call. And what that means is that the system can, you can have a prompt written into the system that says, you are a customer service chat bot. You always respond in a friendly and happy manner. And that is a system prompt that is always integrated into every conversation that is happening. And the reason why they separate that out from putting that into the prompt itself, which is, you know, coming to the user essentially, is because it's a lot less prone to prompt injection attacks, where they're essentially figuring out what the prompt is. And since user provided content can be structured separately, and it can be separated from the actual instructions themselves.

8:10So I think that's really interesting. As far as the depreciation because that is a big part of what is about to happen here they're about to depreciate a lot of older models they said because they're continuing to kind of invest in the chat completion api versus the convert versus the um the older versions they said our efforts to optimize our compute capacity or i guess they labeled that as one of the other reasons is their efforts to optimize their compute capacity they said in six months we'll be retiring some of our older models using the completions API instead of the chat completions API.

8:44So while this API will remain accessible, we will label it as legacy on our developer documentation starting today. So if they're not going to completely kill it off, people that have it integrated with their tools will still be able to use it. And the reason why I think this is important is because the way they treat these older models today with the new release of GPT-4's API is the way they'll eventually treat GPT-4's API when they have to depreciate that as well. And so I think developers are keeping a really close eye on this depreciation schedule, essentially, and how they handle this. Because this is going to play, like, if they do this for everything, every time a new API comes out, of course it's better.

9:20So you assume you want to upgrade. But some people, right, maybe they've made a software or their developers made a software. They don't know how to edit it. You need to make sure that you have the ability to continue to upgrade these as you go. otherwise you know some software will be stuck in the in the dust and eventually it could get depreciated and break if you're not staying on top of this and upgrading these tools they said they're going to keep it labeled as legacy but it's going to be still accessible they plan for future models and product improvements to focus on the chat completions api and they don't have any plans to publicly release new models using the completions api so essentially they're saying they're making a paradigm shift away from the completions API and to a chat completions API, meaning they're going to start having this where it's structured, where you can give special requests to the model in addition to the prompt.

10:12So that are separated out, you know, you can give like an overarching set of instructions, which could be as simple as you or a customer service bot respond friendly, or it could be, you know, a five page PDF documentation that is, you know, know the instructions and then everything that someone talks with this model about is you know going to be based off of that so starting on january 4th of 2024 next year older completion models will no longer be available and will be replaced with the following models so they have you know some older models a lot of people are familiar with because they've been open source but they got ada babbage curie da vinci and all of those are going to be replaced with the new ada 2 babbage 2 curie 2 davinci 2 models which are this kind of new architecture of chat completion api so they're going to completely switch that out and any applications that are using stable model names from for base gpt3 models so ada babbage curie davinci will automatically be upgraded to the new models listed above and the new models will also be accessible in the coming weeks for early testing by specifying the following names in the API call.

11:22So if you're a developer, this is kind of important to you, but if you change the model name in your API calls from ADA to ADA-002 or from Babbage to Babbage-002, you're actually going to be able to test in the coming weeks using those models by changing that. So I think this is going to be really interesting. I think overall, I think OpenAI is handling this very well, the way that they're depreciating older models the way they're auto upgrading those older models as well i think is really good um it's good to see but overall i think this is a really exciting time to be in ai this new gpt4 api getting general availability is absolutely massive for the industry i think we're going to start seeing some huge improvements from a lot of the different ai tools we see out there so this is a very exciting thing to follow into the future