The Birth of Code Llama: Meta's New Open Source AI for Code Generation

0:00Really the headline here is that in a move aimed at intensifying its position in a very crowded generative AI sector, Meta has open-sourced CodeLama, which is an advanced machine learning system capable of generating and explaining code in natural English, right? So it can generate the code on the one hand, and it can also explain what is, you know, happening in that code. So this release comes on the heels of other AI models by Meta for tasks like text generation, language translation, and audio creation. Meta has really been pushing, like, hard. They've been at the forefront. And I will give them credit in a sense that everything they've been pushing is not a productized product, essentially.

0:41They're releasing a lot of these as open source systems. A lot of these are things for researchers and developers to use. And you have to give them a lot of credit for that specifically. A big part of this is they want to make sure that they have the most cutting edge AI researchers on their team at Meta. And so this is a good way for them to, you know, say, look, we're creating cutting edge stuff and we're releasing it and it's getting users. And that's a really big part of attracting top talent. And so I think meta is doing a great job. I still think like, it's really interesting and left to be seen, whether they're going to try to productize this, whether they're going to try to turn these into, you know, like revenue generating streams, I think that'll be very interesting to watch, perhaps, at the moment, they're saying, you know, these are still relatively small compared to our other revenues from advertising, essentially.

1:26So I think this is very, very interesting to see where this goes. But But CodeLama stands alongside a lot of other AI-powered code generation tools, right? We have GitHub Copilot, Amazon CodeWhisperer, there's StarCoder, StableCode, PolyCoder. But this really has the capability to complete and debug code across a variety of programming languages. So these aren't the only ones, but here's a couple big languages that CodeLama works on. They have Python, C++, Java, PHP, TypeScript, C Sharp, and Bash. So in a statement shared with TechCrunch earlier, Meta kind of elaborated on its philosophy towards the open approach.

2:04It said, quote, At Meta, we believe that AI models with large language models for coding in particular benefit most from an open approach, both in terms of innovation and safety. Publicly available code-specific models can facilitate the development of new technologies that improve people's lives. is by releasing code models like CodeLama, the entire community can evaluate their capabilities, identify issues, and fix vulnerabilities. I think really the machine learning system is an offshoot of Lama 2, which is a text-generating model that Meta had previously kind of open-sourced. But unlike its predecessor, which can generate code but kind of lacked quality, right?

2:43Lama 2 wasn't known for incredibly high-quality code. But unlike that, CodeLama is a lot more focused and sophisticated. So the training data for CodeLama includes a mix of publicly available sources, and it kind of emphasized the relationship between code and natural language. So CodeLama models come in a bunch of different flavors, you could call them, with each parameter sizes ranging from 7 to 34 billion. And it was trained on a massive amount of around 500 billion tokens of code-related data. So some specialized versions are kind of optimized for Python and understanding human instructions, with others being fine-tuned to generate helpful and quote-unquote safe answers.

3:25And it's up to you which one you'd like to use. So for those unfamiliar with machine learning, terminology parameters are essentially the learned parts of a model that essentially define its capabilities to solve problems like text or code generation. So tokens, on the other hand, represent just raw text data, right? Like how much data went in to actually train this. So the expansive parameter count of CodeLama really kind of reflects its ambitious capabilities. Meta says that their 32 billion parameter model is the most effective code generating model that has been open source so far. And I think the potential impact of a tool like this is it's really big.

4:02And I think this is also especially given that GitHub's Copilot is already in use by over 400 organizations. And, you know, like I, for one, have many of my developers that are using GitHub Copilot to build things and are seeing some really solid gains from it. But apparently, it's increasing developers coding speed by around 55%. This is according to GitHub's claims, of course, so take it with a grain of salt. But additionally, a recent survey by Stack Overflow found that 70 % of developers are either already using or plan to use AI coding tools this year. And I also have a lot of developers that aren't necessarily using GitHub Copilot, but are literally just using GPT-4, asking it questions about code and getting really good insights and responses.

4:43Now, of course, it's not perfect. They say a lot of times it will gaslight them about a question driving crazy. But at the end of the day, it does have a lot of really good tips and it helps a lot with coding. So, I mean, for all the gaslighting, they continue to use it so they couldn't live without it and use it every single day. So it's got to be doing, you know, something pretty, pretty solid. However, I do think that the rise of these kind of AI assisted code, these aren't without their pitfalls, right? Research conducted by a team affiliated with Stanford revealed that these tools can inadvertently introduce security vulnerabilities into applications, which is definitely a concern for virtually every developer.

5:18I think that there's also the kind of contentious matter of intellectual property. So some code generating models are trained on copyrighted material, which is potentially, you know, having some big legal risk to businesses. Code lama is not entirely foolproof either, right? This new one that's just coming out. So I think while it won't actually generate ransomware code, if asked directly, it will comply when request, when like you, if you ask it and you frame it in a less, and it kind of like a roundabout way. So meta acknowledged this in a blog post advising developers to conduct safety testing and tuning specific to their application of the model before deployment.

5:56But despite all of these challenges and right, there are a lot of risks. I think meta really imposes minimal restrictions on code lama's usage So unlike other versions, I believe lama like the first version of lama that they came out with had like a ton of Usage restrictions. You weren't allowed to use it for Corporate anything you weren't you're only really allowed to use it if you're a researcher just testing it And so it's really I don't know wasn't super useful This is different this time around around essentially They're just requiring that the model not be used for malicious activities and that a license be requested if deployed on a platform with more than 700 million monthly active users.

6:33So that's right. There's a handful of companies in the world that have that. But if you're going to use it on a platform with more than 700 million monthly active users, you're going to want to license. Otherwise, pretty much anyone can use it. And then the thing with, you know, not having malicious activity, like obviously that's just in everyone's terms of service. That's, you know, it goes without saying, I'm sure malicious people will use it, but at least meta can say, we told you not to, right? So I think meta aims for CodeLlama to find applications across diverse sectors, such as research industry, open source projects, NGOs, businesses.

7:05And I think it's really hoping that it's going to inspire development of new tools and commercial products based on really kind of the Lama 2 model. So very interesting space to watch and it looks like an incredibly useful tool.