AI Operations and AI Engineer: The New Careers AI Is Creating

0:01Today on the AI Breakdown, we're exploring the new careers that generative AI is creating. The AI Breakdown is a daily podcast and video about the most important news and discussions in AI. Go to breakdown.network for more information about our YouTube channel, our Discord, and our newsletter. Hello friends, quick note before we dive into the episode. Today is sort of nominally a holiday in the US, at least it's a bank holiday. And so for this episode, I'm not doing my normal brief followed by the main episode. We are just doing a longer main episode about what I think is a really interesting topic.

0:30We will be back to our normal schedule and format tomorrow. Enjoy. Welcome back to the AI Breakdown. One of the big questions around artificial intelligence is how it's going to impact people's careers. In fact, part of what has made this such a radical growth industry with so much attention focused on it is that unlike many hype-y technologies, the first time, the very first time in fact, that many people use a tool like ChatGPT or MidJourney, they have a sense that the way that they're going to do their jobs is very, very likely to change. And some, in fact, probably expand from there and think to themselves, well, maybe the entire nature of the jobs that we will be doing as a whole is likely to change.

1:12A big goal for me with this content is to help people navigate that transition, whatever it ends up looking like. And so today we're going to explore how AI is already transforming how people think about their careers, and in particular, what new types of roles it's actually creating. Now, part of the inspiration for this piece was a recent article in Semaphore called The Rise of the Novice Coder. Can AI turn every employee into a developer? It sets the scene with the story of Leo Sagara. Gara was a 36-year-old father of two who was previously the building maintenance supervisor at Vanderbilt, but who, thanks to AI, has now built more than 20 software programs that Vanderbilt is using for things like faculty onboarding, tracking transactions, and the way that he did this was with an AI-assisted program.

1:55He used the Power Platform from Microsoft, which is a no-code platform that is designed to help people who aren't coders actually build software apps. Now, apparently it was Microsoft's event for this piece of software last week. And so Semaphore also covered some of the other ways that people are using it. From the piece, Charles LaManna, who runs business applications for Microsoft, said some of the biggest users of Power Apps are in corporate finance or human resources department. He said Toyota has become a major Power Apps customer because it's instructed its employees to use the service to make improvements for logistics or other areas.

2:25LaManna said Power Apps are filling in the gaps not met by companies making business software. He said digital demand is basically unquenchable. The more SaaS services you have, the more you have gaps between them. Now, where the author Reed expands this article out to is a discussion of just how much coding is perhaps the core use case or killer app of the current generation of LLMs. He writes, Coding, that unexpected LLM skill, may end up having a much bigger impact than the natural language component that has captured most of the public attention. It isn't just Microsoft's GitHub pioneering the space.

2:55Companies like Replit and Sourcegraph, among others, are getting traction by focusing on generative AI software development. Other companies like Nomad Data are using code to supercharge LLM prompts. For instance, you might ask a chatbot to gather a bunch of data from a whole slew of corporate documents. Behind the scenes, the LLM has been instructed to create a Python script to gather the data. The combination of automated coding with natural language is a potent mix that hasn't really been exploited yet. And then here is the key line from Reed. This likely won't lead to software engineers being replaced by AI, at least based on what we know about the technology today.

3:26More likely is the incorporation of code into more aspects of daily life and more people participating in that transition. It's taking full advantage of the powerful computers around us both at work and at home. For years now, companies have been yearning for employees to become citizen developers and take it upon themselves to build software and improve productivity. Automated code generation could help make that a reality. We often call today's tech consumers users. We may have to come up with a new term for what comes next in the age of AI. Directors might be a better one. Okay, so in our exploration of how AI is likely to impact people's careers, one question that this piece rises up is to what extent average professional non-coders right now will have development become a part of their skill set, presumably in an AI-assisted way.

4:06Well, rather than fully trying to answer that question, let's actually continue on and talk about how AI is impacting people who are already engineers. As a way to frame that, let's read a piece by Latent Space's Swix back from June called The Rise of the AI Engineer. Swix writes, Emergent capabilities are creating an emerging title. To wield them, we'll have to go beyond prompt engineering and write software. Swix begins the piece, We are observing a once-in-a-generation shift right of applied AI, fueled by the emergent capabilities and open-source-slash-API availability of foundation models.

4:38A wide range of AI tasks that used to take five years in a research team to accomplish in 2013 now just require API docs and a spare afternoon in 2023. However, he continues, there are no end of challenges in successfully evaluating, applying, and productizing AI. He points out evaluating and selecting which models, be it GPT-4, Claude, Hugging Face Llama, or something else, tools, from the most popular chaining, retrieval, and vector search tools like LangChain, LlamaIndex, and PineCone, to the emerging field of autonomous agents like AutoGPT and Baby AGI, and then of course there's the news.

5:09On top of this, he writes, the sheer volume of papers and models and techniques published each day is exponentially increasing with interest and funding, so much so that keeping on top of it is almost a full-time job. And this is where SWIX gets to the culmination. I take this seriously and literally. I think it is a full-time job. I think software engineering will spawn a new sub-discipline, specializing in applications of AI and wielding the emerging stack effectively, just as site reliability engineer, DevOps engineer, data engineer, and analytics engineer emerged. The emerging and least cringe version of this role seems to be AI engineer.

5:43He continues, every startup I know of has some kind of discuss AI Slack channel. The thousands of software engineers working on productizing AI APIs and OSS models, whether on company time or on nights and weekends in corporate slacks or in indie discords, will professionalize and converge on a title, the AI engineer. This will likely be the highest demand engineering job of the decade. AI engineers can be found everywhere from the largest companies like Microsoft and Google, to leading edge startups like Figma and Notion, to independent hackers. They are making three

6:31In the next section, Swix describes how an AI engineer is different from a machine learning specialist or a data engineer. And basically it comes down to this idea of applying or productizing AI as opposed to building out the underlying models. Now one interesting note is that this isn't the much hyped prompt engineering job that so many have discussed. In fact, we'll talk a little bit more about that in our next section. Instead, this is something that is more technical. It really is a subcategory of engineer, not just a non-engineer using AI tools to now be able to code. It's a very different role, in other words, than what we were talking about just a moment ago.

7:05OpenAI's Andre Karpathy actually commented on this post. He wrote,

7:38scale training of transformers on supercomputers. In numbers, there's probably going to be significantly more AI engineers than there are ML engineers and LLM engineers. One can be quite successful in this role without ever training anything. Okay, so let's take a moment to pause here. We talked first about how non-coders can use a new set of AI-powered tools to actually start to write software that fills in specific gaps in their software stack at work or that solves specific problems that they need solving. We talked about how there is this new category of engineer, which is specifically about the implementation and the productization of AI software and putting it into practice.

8:11Well, Rachel Woods is exploring something that certainly feels related but is ultimately different than the AI engineering role, which she has been calling AI operations. A good way to get into this might be to read her pinned post on Twitter right now, where she writes, for the last few months, I've become obsessed with the problem of how do you build an AI powered business, aka how do we actually use this stuff? It honestly started because at the beginning of the year, I did hundreds of AI advising calls with businesses. And while I was able to answer questions and help point them in the right direction, the meaty question they really needed answered was, how do I use AI to power my business?

8:42Pre-ChatGPT, this wasn't really a question on people's radar, much less one that had good answers. Underneath the question are all sorts of sub-questions. What should people learn? Who should businesses hire? Where should businesses start? Why do certain things? Why not do certain things? What ethical guardrails should exist? And more. I think we're still at day one in answering these questions, but I'm also proud of the progress. It's early for AI operations, but definition breeds clearer investment. and focus, and is an important step for getting real-world businesses' value out of AI. Now, in a post on her Discord, she expanded out what this definition of AI operations actually means.

9:13Rachel writes, From what I've seen across what businesses need, will need, where the field is headed, and where the talent gap lies, there's a need for a holistic definition for people who are able to do a few things. Drive AI literacy, implement AI solutions, and manage those efforts. What I get most excited about, as someone who has worked on cutting-edge AI research teams, is that I don't actually think the traditional ML engineer PhD is the right archetype for this. I think those people need people who are grounded in real-world business problems and who can speak enough of the same language to usher solutions into organizations.

9:43Here, Rachel describes that a little bit further, talking about how the idea of AI operations is putting AI into the systems that businesses run on. This is a follow-up piece of advice to the video where I shared the story of a founder I talked to who is literally telling their team they will get managed out of the company if they don't learn chat GPT by a certain date. So the flip side of this is if you want to be extremely sought after in this next AI wave, don't stop at learning how to do prompt engineering. The skill set I'm seeing every company wish they had and be willing to pay pretty high salaries or consulting rates for is AI operations, which is basically being able to take prompts.

10:21Yes, you still need to learn how to do prompt engineering, but put that into systems, processes, and automations that can be used to help a business really truly scale their operations. While prompt engineering is probably not a sustainable career, I really think AI operations will be. Now, really, really importantly, and I want to make this point crisply, this isn't just the prompt engineering that you heard so much buzz about a few months ago. As Rachel again tweets, underrated skills in AI ops, system design, task design, data set design. Prompt design is important, but get good at these and you can start solving very interesting operations problems.

10:53Now, what's really interesting is that one of the big themes we've been talking about for the last couple of days, and frankly, the last couple weeks, is the extent to which the theme of this fall when it comes to AI seems to be integration, application, actually putting these tools into settings and circumstances where people and businesses can use them. The AI ops role that Rachel is talking about is effectively the traditional business's world's bridge between the old world that didn't have these AI-powered tools and the new world in which AI-powered software and systems will underlie so much of what happens in very traditional businesses.

11:25Someone, some type of person more specifically, is going to be needed to help companies figure that out. And so here we are, less than a year into the post-ChatGPT world, and the broad brushstrokes and outlines of these new types of roles, both for people who are engineers and for people who are not engineers, are starting to come into view. One of the things that makes this space so exciting right now is that despite the fact that hundreds of millions of people have tried these tools, a tiny fraction of those folks have actually figured out how to integrate them into their workflows, and even fewer have figured out how to take those skills that they're developing in real time by trying and bring them into the companies that they work with or for.

12:03There is going to be so much opportunity for individuals who can help translate this new AI world to clients, to customers, to the businesses that they work with. And I think that starting to understand these categories of roles might help people frame that work just a little bit more. Anyways, guys, hope this was an interesting and useful piece. I'll include links to everything I talked about here. Big up to for his work on popularizing this idea of the AI engineer. And actually, congrats to him. He has a summit going on in San Francisco right now with the other folks from Latent Space called, of course, the AI Engineer Summit.

12:34And Rachel, as well, is working on helping people actually put this into practice. You can check it out at theaiexchange.com. For now, though, that is where we are going to leave things for today's AI breakdown. I appreciate you guys listening or watching as always. And until next time, peace.

12:53Thank you.