The AI engineer skills gap

0:03Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind-the-scenes content, and AI insights. You can learn more at practicalai.fm. Now, on to the show.

0:47Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How are you doing, Chris? Hey, doing very well today, Daniel. How's it going? It's going really well because I have a close friend joining us on the podcast today and a previous guest. We went through the Intel Ignite Accelerator program together in different companies. And yeah, just really excited to have with us Ramin Mohammadi with us, who is an adjunct professor at Northeastern University and also lead principal AI engineer at iBase-T.

1:30Welcome, Ramin. It's good to see you again. Yeah, thanks then, Chris. It's always great to be back. Yeah, yeah. I've been excited to talk through these things. And even before the show, obviously, you're kind of living in two worlds. You're living in the industry world and you're living in the academic world. And you've kind of been living in those two worlds for quite some time, which is interesting because you have a perspective on like how, for example, data scientists or AI people or machine learning people are being trained and what those people are actually doing in industry, which I find really intriguing, especially because so much has changed.

2:18I guess maybe that's a good initial question is, is my perception right that like the role of an AI person or a data scientist or a machine learning person in industry like that, the day-to-day life of that person has really changed dramatically over the past even few years. And I'm curious if the academic side has kept up with that. Yeah. So I think that that's an interesting question. I think we need to break it down into multiple sections. Because, I mean, let's just start first, do a quick review of what has happened, you know, because we're talking about the complete transformation of the AI and data science job market.

3:07I mean, if you remember, and it was about a decade ago, back in 2012, our business review, they called data science the sexiest job of the 21st century. Yeah, that's why I got into it. Obviously, that describes what I want it to be. If you think about it, that one phrase, it kicked off a massive gold rush. Everyone wanted it. Universities were spinning up the new master programs overnight, and the promise was pretty simple. Get a degree and learn a little bit of machine learning, and you're insensibly employable. That promise feels almost like a myth now. I mean, if you talk with any new graduate today, especially someone looking for their first role, the feeling is totally different.

3:54It's brutal. The market is absolutely brutal. We see job posting for entry level. That job requires about three years of experience. The demand has changed. It's shifted fundamentally. It's not about what do you know from the textbook anymore. It's about what can you build, can you deploy, and maintain a real scalable AI system. It's kind of like that's the new currency of hiring. I think one time, Chris, I don't know if this was us that came up with this discussion, But I remember quite a while ago, we talked about kind of like full stack data scientists or something like that. The idea being like you could figure out what kind of modeling you needed to do.

4:42You could do the prototyping and POC, but you could also like deploy something to actual cloud environments or something like that. I mean, that seems like quite a tall order, Ramin, because you're basically saying like, be a software, like a proficient software engineer, but also be an infrastructure person. And also, and there's this, I don't know, I've heard a lot of people say there's not really like a full stack engineer doesn't really exist. So, yeah, is it, I guess, from that perspective, how much of what a data scientist or machine learning or AI person fits into those different buckets at this point, whether it's software engineering or infrastructure work or actual like knowledge of differential equations or statistics or something?

5:34I think that's also a great point. So if you think about back to DataSense job, the idea of DataSense job was that your job is kind of done once you got a good score in the notebook. You know, that the classic my model has 95 % accuracy on the test data and you're good, you pass it to someone else. And then if you remember, I think it's around 2020s with some resources like Google Cloud rules of MLOps, you know, it laid out this new reality. that successful ML needs a homesuit of real engineering escapes. The things like containerization with Docker, CICD pipeline automation, monitoring, and you have to know if your model actually works in the real life, and then you need to monitor it.

6:21And after you deploy it, you need to basically look for the drifts. So industry made it really clear that job wasn't just build a model anymore. It's kind of like you need to own the pipeline. and then if you think about it all of a sudden the analysts or data scientists went from just being a simple analyst to be engineers who build and maintain the intelligent systems and so just as that engineering power was being raised by MLOps along comes the second maybe even bigger tidal wave, you know, the generative AI and that becomes like around 2023 explosion that you can see that Stanford Ford AI Index basically, they mentioned that this was not just a cool new tool.

7:06This was an automation event. You know, I immediately attacked the entry point in the field that they could do those jobs, basically. You know, it's kind of sort of this shift was drastic from the data scientists to ML ops engineers and all of a sudden AI, basically. In addition to that, there's so much more diversity in, you know, as we were talking a moment ago about the notion of the full stack engineer, especially at the entry level, trying to fit into this. And the notion of like, what is full stack is changing fairly rapidly. There are a lot of different options out there. And not only do you have to try, would that entry level student have to try to fit in to the notion of what, you know, an organization is looking for, but there's all these variations on that.

7:50And if they're not in the right variation of what that organization is looking for, in terms of this abundance of skills that are required for that given position, they're still out of luck. I mean, it's really a crapshoot for students today in terms of trying to find the right fit and represent their own ability to fit to the organization that's looking to hire. I'm really glad that I'm not out there in the job market in that way right now. It would be brutal. Yeah. So I think that's true. It's like, if you think about it, as this AI wave comes in and this series of automation tasks, basically, A, I made certain things simpler.

8:32Those are like the types of tasks, like a bullet per task that you always used to give to the new hire, basically. You know, it's kind of like the groundwork. And for someone who's an early hire, a recent graduate, those type of job were kind of like the first step on the ladder. For example, you write a complex SQL query to get the data, make simple pythons, and get your hand dirty with the company's data. you learn about it, and also you show your skills. But now it's no longer like that. So you need to basically find the correct fit, what they exactly want, what they want to build. So I showed that I can build that.

9:10And there was this study from OpenAI and University of Pennsylvania that they look at this task exposure to large language model, and the takeaway that they had was pretty simple. Any repeatable task that used to be given to juniors I highly vulnerable to AI basically and innovations. So if a junior analyst used to take all this afternoon, write the SQL queries and make the dashboard, now AI can just write it with a great prompt, right? So basically the economic case for hiring a big group of trainees and have them to do the work and has evaporated. There's kind of like a change. For example, I used to hire lots of interns to basically help with the development and speed up the process.

9:59And since AI shift, to be honest, I just use AI for all of those tasks. So this has been this big change. And of course, we are seeing this shift in hiring strategy kind of everywhere in big tech or even in startups. They're just stop hiring for potential and they're starting hiring for proven capabilities. It's kind of like the paradigm has changed. New companies these days basically are forced to bring in 50 juniors or spend a couple of years to train them. They'd rather to hire five or maybe 10 people that already have built or developed some complete system from day one. So it's kind of like, if you think about it, that new entry-level jobs is technically what we would call mid-level engineers a couple of years back.

10:47You know, shift is really bad. And with this kind of with this new bar, it's not like that, you know, you don't need knowledge. So all this, you know, deep statistical knowledge, heightened skills, they're all essential. But they are just, at this point, they are kind of prerequisites. They are the ticket to the game. They are not how to win it. You know, it's kind of, it's history. You need to prove that you can build. The company wants what you build. And then you go for a hiring. I'm wondering, because that bar has been raised, like you say, the kind of mid-level positions that we used to call mid-level or maybe the entry-level ones now, how does that change?

11:27Because, I mean, maybe this is a negative view that I'm about to give, but I'm very pro higher education. But I also think like even whether you look at computer science or data science sort of education, a lot of that does not even before before the recent shift that you talk about, it didn't always connect to what you were actually going to do in your day to day work. Right. So now not only does it not connect to that entry level kind of day to day work, but does it now even increase that divide where like the like how how could we possibly train people to come in as mid level kind of data science folks?

12:12Because I think if I'm interpreting what you're saying correctly, it's not that AI is making data scientists no longer relevant or AI or machine learning people no longer relevant. It's still very relevant. It's just the stuff that entry-level data scientists or machine learning people used to do and kind of level up on, that's no longer available. So where are they going to do that? And is it even reasonable for us to think that universities could help get them up to that level, I guess? Yeah, I think so. I will answer to that question in two sections. I think one part is about where is academia stance right now.

12:55And then the second part would be talking about the industry versus academia right now. So let's just start with where does academia stance? You know, if you think about it, and I kind of call this, I don't want to be negative, I call it educational bottleneck, you know. And to be clear, the first thing is that, you know, the faculties that we have in CS, ML, data science department, they are all brilliant. You know, they are like world class at teaching the fundamentals, the math, theory, history, the research, that foundation is non-negotiable. You need it. But the curriculums often just stop there.

13:29And it used to be also kind of like that. and it's about the theory and leaves basically this huge gap between what the student learns and what employees actually need for them to do on the first day. As an example, a student might spend the whole semester learning about the math and all sorts of like optimization, back-replication techniques and stuff like that, which is necessary. But as soon as they graduate, they basically see this job market that wants them to deploy on the Kubernetes or they know how to work with all different cloud resources, you know. So they know exactly how the engine works, but they actually never tried to drive a car into traffic.

14:07And that, you know, there was this new post by Andrew Engie recently that he argued this urgent shift in education. I'm going to paraphrase what he said. He said, knowledge is great, but skills are greater. Meaning that in the field that's moving this fast, you have to teach the practical skills to get the work done. You know, you need to basically give the capacity to get meaningful work done by having a proper knowledge and proper training. So this is exactly what the job market is selecting for now. So that's the view that I have on education at the moment. And the second part that we can basically talk about is like a comparison between where is like a way of industry versus academia.

14:53And there is a really good study by MIT, a recent study basically that the stats are staggering. They say that right now about 70 % of the AI PhDs are just skipping academia and go to job market, go to basically industry directly. And that's a huge brain drain for the universities. And the second is that, which is the real clear, and I'm sure you know about this, like 96 % of the major state-of-art systems comes from industry labs, not from universities anymore. So a university is already falling behind. And then companies like Google, Meta, OpenAI, they are the one that's defining the frontier now.

15:37They are building the tools. They are setting their standards. And that's the absolute core of the bottleneck. Academy curriculums moves on a cycle of years. getting a new course approved, like updating a textbook, it's slow. By the time a university approves one new course to be, like let's say, for example, LLM application course to be added to curriculums, the tools have already changed three times. So the entire framework is really different because it took a while. And that has happened to me also. I developed a course and take years to get approval to teach that course. And then you need to go back and update everything that you were planning to teach because the industry has changed over.

16:26Well, friends, when you're building and shipping AI products at scale, there's one constant complexity. Yes, you're wrangling models, data pipelines, deployment infrastructure. And then someone says, let's turn this into a business. Cue the chaos. That's where Shopify steps in, whether you're spinning up a storefront for your AI-powered app or launching a brand around the tools you've built. Shopify is the commerce platform trusted by millions of businesses and 10 % of all U.S. e-commerce, from names like Mattel, Gymshark, to founders just like you. With literally hundreds of ready-to-use templates, powerful built-in marketing tools, and AI that writes product descriptions for you, headlines, even polishes your product photography.

17:11Shopify doesn't just get you selling. It makes you look good doing it. And we love it. We use it here at ChangeLog. Check us out, merch.changelog.com. That's our storefront. And it handles the heavy lifting too. Payments, inventory, returns, shipping, even global logistics. It's like having an ops team built into your stack to help you sell. So if you're ready to sell, you are ready for Shopify. Sign up now for your$1 per month trial and start selling today at Shopify. dot com slash practical AI. Again, that is Shopify dot com slash practical AI.

17:53So, Ramin, I love I love how you highlighted this kind of divide between academia, industry, like what that is in reality. Anecdotally, I remember actually last I think it was last year or maybe a year and a half ago, I lived by Purdue University. I was like walking through campus and they were just finishing their, they had this new building, right? And so this was 20, whatever, 2024, right? And it said like Hall of Data Science, right? And I thought that my immediate thought in my mind is like, in 2017, you could have created a Hall of Data Science. Now you need a hall of AI. Like you're building the wrong hall.

18:42To their credit, I think they actually, so I just looked this up while we were talking. They did rename it Hall of Data Science and AI. So to their credit, they at least caught up with the name. But yeah, I guess like obviously you're an educator. And so you see that there is value in trying to have these formal education serves a purpose and is different from maybe on the job training. What do you think or have you seen examples where this sort of practical skills are built up in an academic environment rather than just kind of the theory or the knowledge as you were kind of drawing the distinction there?

19:34Yeah. So actually, that's something that we have been doing for almost the last three years. So I basically developed this course, this MLOPS course at Northeastern University almost three years ago that we have been ongoing. So the idea was this. This is like about three, four years ago. I was this hiring manager and I used to do lots of interviews for our team. And I always basically interviewed smart, motivated, good school basically candidates. But most of them struggled with the same thing. They understood the theory, but they couldn't build anything. They couldn't ship anything. And that's when it clicked for me that, okay, if the industry, I personally as someone who was in the industry and academy, expect these students or these basically candidates to build a real system from day one.

20:25And then I know in the industry we don't teach them that. Could we do something about it? So I started working on this course. I built this envelopes course that every semester right now we have about 150 to 170 students within one class, like a huge classroom. And instead of just learning the concept, they start by choosing a domain that they actually care about, healthcare, finance, sport, robotic, and whatnot. Then as a team, they spend the entire semester on building one real product. And this real product, it's not just homework assignment. It's not a toy example. It's a real working system with deadlines, milestones, deliverable, just like a real, like an actual ML and software team.

21:10And the best part of that is we wrap up the semester. You know, the way that we wrap up the semester is that the students basically present a product at our ML Ops Expo, which is a full industry partner event we have been holding over the last, I think, two years now. This year, for example, we partnered with Google. So we are hosting on, in two weeks, December 12th, at Google Main Campus in Boston, and where our students are pretty hyped to come there, but they would basically, what they do, they show, they demo the actual product that they have built. And so the whole course is simple. You don't just learn ML anymore.

21:51We teach you how to build with it, you know? And the idea for me was to give the students this hands-on experience that companies are looking for right now. And honestly, watching the students go from, I have never deployed anything before. Me and my team, we built a real product this semester. That's kind of like the best part for me. One, at least hypothesis that I have here, which I would love your opinion on, Ramin, is on one side, you have highlighted how this kind of gap is widening even between the theory and where you need to come into a job at a mid-level. At the same time, this revolution of Gen.AI has been happening, which in some ways, to your point, some of those things are the things that are being automated by AI, but it's also enabling maybe this younger generation of software engineers, AI people, to actually perform at a higher level out of the gate, but in a different way.

22:59So not like there's kind of a burden on maybe us as prior generation data scientists and machine learning people to understand that students and new hires need to, from the start, be doing their data science work differently. So just by way of anecdote, we were talking about this a little bit before the show that, you know, my wife owns an e-commerce business. Black Friday, Cyber Monday just happened. I, you know, day to day in my company, you know, I'm not doing as much kind of hands on work on the product as I was given my role as CEO. But it was like, it was nice to go back. So for like four days, I helped them during the sale.

23:47And I just sat in a room doing like customer lifetime modeling and like updated forecasts for 2026 and looking at churn and analyzing like customer journey and all this stuff. And number one, it was a ton of fun. But I was kind of coming at it from that perspective and kind of reentering some of those things that maybe I hadn't done as much for a little while or even, you know, maybe since the previous year when I helped them with forecasting. Like I was able to get tons of that done so quickly because I was having AI honestly write most of the code for me. The thing, though, was I still had to play the data scientist to get from like point A to point B.

24:36There was no way that I could have just said to any AI system, like, hey, I want to write a three-sentence prompt and get out all of the lifetime modeling and forecasting and all of this stuff. I still had to play that kind of data science orchestrator and know what the things were, know what modeling techniques were relevant, know maybe what trade-offs were and other things. So do you think on the one hand, it's maybe depressing that the academic kind of industry gap is widening? But on the other hand, maybe there's, am I right that there's an opportunity to actually like lean in for these students in terms of different ways of working to like get to a higher level faster?

25:27I'm not sure about getting to the higher level faster part, but I saw a new talk recently by Neil Ahoy over at Google, and he made a great point about this data science job. And he basically was saying that the data science job is not gone, but AI is just forcing them to change dramatically. It's no longer about analyzing the data or building certain sort of dashboards to stuff like that. As we say, you can just wither the knowledge, just prompt it properly, and just having the data and just build that quickly. So there are certain types of tasks that you used to do for trying to climb the ladder to learn more and more, but that they are not the same anymore.

26:14And the expectation is not for you also to do the same task, because if a company is hiring you probably these days they want more but i think it is a really great point that for hiring managers or for someone that's when you hire someone on your team or have someone new juniors on your team you need to also account for helping them to like mentoring them properly to be sure that they can evolve and learn otherwise we basically take this cognitive ability from them because they everyone if you just ask everyone to just build build and they just use AI, they don't, they never are going to learn basically how to build.

26:51So we take that cognitive ability away from them to just build new, faster products. Yeah, I think you're really onto something there in terms of, you know, like, one of the things that I have done for the last few years is I'm a Capstone sponsor for Capstone projects at Georgia Tech in the College of Computing. And so as, and I'm doing that from my nonprofit role, as opposed to my day job. And so when I work with different teams there, I think one of the challenges is they're kind of bringing what they know, you know, certainly Gen AI capabilities have helped them, you know, step up a little bit along the way in terms of figuring it out.

27:32I think the areas that I've noticed that they're still struggling, the students are, there's, you know, going back to Dan being a data scientist over the weekend, instead of a CEO in that moment is he's bringing all that business knowledge, you know, years and years and years of business knowledge and understanding about what's really needed in that. And I think that's, you know, that's one of those things that is part of the struggle with junior level is there's the kind of concept of I've learned tools in university, and I'm trying to bring them to bear. And they're not always the right tools for the organization they've joined.

28:09And they don't necessarily know how to combine that with all the other tie-ins that that organization may need that were not necessarily something accommodated in their academic development. And so, you know, that's kind of exacerbated by the fact that now with Gen AI kind of replacing a lot of those junior roles coming in and, you know, how do you ramp up? It does seem, to your point, like things are actually getting like, even though we have new amazing tools in, in the form of JNI capabilities, it seems like things are getting harder to bridge that gap. And, and I'm not sure how you do that.

28:48Cause it's a combination of both kind of the, the experience of being in the real world, along with fast moving, you know, a fast moving technical landscape to navigate. Are you seeing that from your side with students and, and how are you tackling some of those, subtleties that are there? Yeah, actually, definitely. So two weeks ago, I sent out a survey to my students, and I asked them basically to take a couple of questions. And I specifically did it for our talk. And so as part of the survey, basically, there were some questions. And one question was, which is 60 % basically of the students, they say that they are taking online courses on top of what they are taking in the school.

29:31And another question, 82 % of the students say that they're participating in hackathons in order to learn how to quickly to build. And about 46 % of the time, they are attending workshops. So they are building their own parallel curriculums through side project, open source contributions, or certification through AWS Google. And that's exactly it. The portfolio kind of has become a new credential. It's no longer about your grades. dislike about what you have as a portfolio. And this is also important for us to kind of like a dose of reality that this self-learning path isn't easy and isn't equitable.