#269 Jason Hardy: How Hitachi Vantara Is Powering the Future of Enterprise AI

0:00Pragmatic AI is how we have a pragmatic approach to providing AI capabilities. i.e. I'm not promising the world to our customers. I am painting a vision that we can get there through this very methodical approach where we think through it. We are very agnostic. What we wanted was to create something that's easy so that it can be as much about the solutions that we bring to the customers as much as a factory for customers to build their own AI outcomes within. Our strategy, obviously, is to sell product, is to solve these problems for our customers. But it's not to just sell them a bunch of GPUs and walk away.

0:39It is to make sure that they're getting the desired results to actually see that outcome. That 90 % failure rate is well worth that 10 % success rate because it more than pays for itself. So that I think we'll continue to see. Now, obviously, again, it's just like going to the gym. The more you exercise, the more you become, the more better you get at these things, the more success you see. So from our standpoint, as we see more engagement, as customers start to work on this more, they'll start to see fewer failures and more successes. My name is Jason Hardy. I'm CTO for Artificial Intelligence at Hitachi Ventara.

1:17I've been with the company for, oh man, almost 13 years now. My kids have seen nothing but Hitachi. So it's been fantastic. I was actually a customer beforehand and I came over through a transition. And yeah, I've been really enjoying it ever since. My background has been in data and in data analytics, not from a data science perspective, but how we actually apply data into solving for some critical problems. And that brought me into our data portfolio, supporting our object storage technology and things like that. And just through the natural progression of technology and how the market evolved um i spearheaded the effort to create an ai brand um behind uh hitachi vantara or within hitachi vantara um and that is hitachi iq so that is yeah this laborious process that i went through and i love every minute of it and now i'm i'm leading the charge uh as far as what ai means to us as an organization and and how we can uh create a different paradigm within the market and really help our customers create something that's relevant and start to see a lot of value from this technology.

2:33Yeah. And can you sort of put Ventara and Hitachi IQ in the context of Hitachi, which most people know from at least my age as an electronics, consumer electronics company, but obviously it's much more than that. Yeah, it's a massive. So yeah, so we are a division. We are several layers deep inside Hitachi Limited, the larger multi conglomerate Japanese organization. I think there's now about 900 business units inside of Hitachi Limited, and we're one of them. So we range from everything from trains and nuclear power and transformers and consumer electronics, like you said, like rice cookers and things like microwaves and things like that, all the way into the IT capabilities from what we do inside of even consulting services into product engineering for customers, as well as obviously engineering our own products.

3:38Hitachi Vantara, formerly known many years ago as Hitachi Data Systems, is kind of like the IT arm, effectively, of Hitachi Limited, as in we sell data storage technology to many, almost all of the Fortune 100 and beyond organizations, everything from commercial to large enterprise to government and beyond. So from our standpoint, it's a bit different for a company who makes trains and transformers and things like that to also have this very predominant and prestigious data platform business. And really the core behind that is the fact that we understand across all of the big things that we do, that data is still the most valuable thing that our customers have and that even ourselves create.

4:33So having a capability to deliver on data outcomes, data technologies, whether it's storing it or activating it or consuming from it or whatever it is, it's very critical. And it's critical to how our customers succeed. So this data business living inside of this massive Hitachi organization is actually a natural fit because of that need for manufacturing and energy and transportation and logistic. All these different types of businesses rely on it so heavily. So for us, it's really capitalizing on what we're good at. And IQ, Hitachi IQ, which is the AI portion of Hitachi Vantara, takes that one step further.

5:16As we know and have been said in the market frequently that data is the fuel for AI. And I wholeheartedly believe that. So again, it's this natural fit for who we are as a company being very data centric. and how we can use that and capitalize on that capability we have to then drive AI outcomes into the market. Now, taking that one step further, when we look at Hitachi IQ, yes, it is the infrastructure, yes, it is the compute and the GPUs and the data and how we harness it and the IT pieces of it, but it's also having an awareness and bringing in other parts of the Hitachi Limited machine to help customers realize the relevancy and to align with them on their thinking in the industries that they work inside of.

6:00So we actually make trains. So we can actually do AI for trains. We make, obviously, all the power grid stuff. We can do AI inside those systems and into those business units and customer types and financial services and et cetera, et cetera. So really, our strategy and everything that I created around Hitachi IQ and how we're executing is as much on the infrastructure side and where we've been really good at for 50 years now from a Ventara perspective to as much as the outcomes and solutions we create with that point of view and that expertise in the industries that our customers execute within.

6:37So it's blending now across those. And just like we're having a data business inside of a very large manufacturing company like Hitachi, now having an AI piece of it inside of a very large manufacturing company like Hitachi is kind of that natural evolution to how we can create a lot of really pragmatic and capabilities out there. Yeah. So Hitachi IQ isn't a product or one system. It's a collection of... It's a brand, yes. A brand, right. It is infrastructure. We do have SKUs and GPUs, and we partner with NVIDIA on collaboration and bringing product in. But you could consider it a kind of an umbrella around what we do for infrastructure.

7:23So we have NVIDIA GPU systems. We have storage technology for parallel file system functionality. But it's also the solutions that we bring into it as well. And we call those the Hitachi IQ 4 solutions for an outcome. 4 of, I'm sorry, 4, the number 4 or? No, 4 like F-O-R. So Hitachi IQ 4 manufacturing. Hitachi IQ 4 energy. Right. And are those SaaS products or is that a consulting business? So it's not SaaS. It is consulting, absolutely, because it is so heavily bespoke, but it is also turnkey. And I know that's like, wait a minute here. So what we've been able to do, and this is through decades of having a capability in the AI space.

8:21We've been delivering outcomes to customers to do tremendous things with AI for decades. What we've been able to do is through that learning, we've built product. Now we built product in a bespoke way for a customer or for an outcome. What we've been able to do is take that product that we've built and poured it into Hitachi IQ's portfolio so that we actually have skewed outcomes effectively. that say, hey, we have a fault detection system or a guided repair platform that uses generative AI where you feed it a service manual or many service manuals in some cases. And as a service engineer, you can describe the problem and it will give you a diagnostic tree.

9:07And it has an LLM that helps feed that and power that process. That is an outcome that lives inside of Hitachi IQ that we sell as a capability. Now, it's not saying, hey, you buy Hitachi IQ in this frame and then you run just this thing on it. That's just one of the many things that can be used inside this shared resource from an infrastructure perspective. So that's why I say it's as much infrastructure and what we deliver into data centers as it is the outcomes that we put on top of that to then help our customers realize the potential for AI to improve on their business and give them a bigger capability.

9:46Yeah, I see. Can you tell me what pragmatic AI is? Yeah, of course. Yeah, so AI has been kind of framed as this, and I agree from the big vision of it, it is going to transform how we do business. Agentic AI is going to drive our productivity, increase it exponentially. Obviously, there's a lot of technology hurdles and things like that that we have to get to, but we're starting to see some of the fruits from that labor. Now, what pragmatic AI is, is how we have a pragmatic approach to providing AI capabilities, i.e. I'm not promising the world to our customers. I am painting a vision that we can get there through this very methodical approach where we think through it.

10:41But day one, don't expect it to completely revolutionize how you do anything. Day one, you start to cut your teeth on it. AI itself, generative AI and agentic AI especially, they're still going through teething pains. I call it kind of it's going through puberty right now. It has to grow up very quickly. And especially at the rate at which it's being adopted and the rate that's being evolving, it is growing up and maturing very quickly. It's being forced into it. But we still have to be practical about how we think about this. And we have to be grounded in reality that, yes, there is some fantastic capabilities that can come from this.

11:20But let's be pragmatic here. Let's go after things that maybe that low-hanging fruit, that where we can see some success or we can learn from it. But let's also not try to transform the core of your business step one, where you end up on front page news when something goes wrong. So I call it going after the peripheral of the business, kind of that edge, and then get maturity in that space in those use cases and then start to work your way into the core as you gain and exercise that muscle. Yeah. How is generative AI changing the game when it comes to AI infrastructure demands? It's increasing it exponentially.

12:02When we talk about traditional AI, that machine learning capability, even things like robotic processing is a form of AI. It could be done in CPU. It could be done. I have Raspberry Pi at home that I like to tinker with and stuff. You can run AI on that. Now, at the same time though, that is not generative AI. Generative AI has created this mathematical paradox of how the transformer operates. And so in that, in the core of how generative AI works, it has caused this exponential increase in the need for processing power. Now, there are certain ways to cheat the system and you can do things you can do with lower end systems.

12:43I run LLMs on my Mac, on my MacBook, which is great. It's great to be able to experiment with on that. But when you look at it from an enterprise perspective, especially adopting generative AI, it has such a tremendous amount of one data necessary to build relevance within the model through things like retrieval, augmentation, or even how you fine tune your model. But it's also how your end users interact with it. So it's very Wild West almost where it's not a predictable workload. So now the more you use it, the more you're like, wow, this is really powerful. and I can see how it can improve on my business.

13:21But at the back end, to get to that point takes a lot of effort, but also a lot of infrastructure to be able to do that. And as your users get more accustomed to it, it only increases more in consumption because they know, again, it's exercising that muscle. What we also see too is that users, if you don't give them access to it, they will go figure out a way to use it anyway. And in most cases, it's using things like ChatGPT or Gemini or Grok or any of these open and available free systems. Nothing's ever free because that data is being utilized for training. So we call that shadow AI. So there is this, again, it's a huge amount of inner infrastructure.

14:05But at the same time, your users are going to figure out a way to do it if you don't make that investment into the technology. Yeah. Yeah. What are some of the mistakes you see? Because everyone's experimenting with generative AI and now agents. Yep. There isn't that much in production yet. But what are the common mistakes you see that lead to failures? Yeah, a lot of it would be the data. And it's kind of multiple folds here. One of them is data, where your data estate, not 100 % of it, but take that use case that you're going after. Let's say it's, again, the guided repair thing or the maintenance thing.

14:47If your data is not in a state that you can accurately consume from it, it's a garbage in, garbage out problem. So obviously, the output coming from the system is going to be flawed because the input going into the system to give it context is flawed. So this unprepared or unclean data, for lack of a better term, causes a lot of this consternation around, well, I'm not getting what I expected out of it, so therefore it's a failure. Well, maybe it's because we didn't do all the pre-work ahead of it. Also, another area that I see is where customers have an expectation that AI is a product. It's not a product.

15:28It's an outcome. It's a solution. It's a tool. It's not like SQL or it's not like a database even at the most natural form. It's it is an outcome. It is a solution set. So in that is having that expectation that I'm buying a product. So I just should work flip of a switch. And that's definitely not the case. So it's having an unhealthy expectation going into it, going back to being pragmatic about it. The other side of it, though, too, is where I see that there's this rush into AI executive level saying, we need to do AI. So they're trying to figure out AI and the right parties aren't involved.

16:06Having a successful AI outcome, generative or otherwise, really focus on generative and agentic. It's a multi-discipline thing, as in you need IT as much as you need legal and HR and the solutions that you're trying to solve for to be involved in it. So don't just expect IT to give you AI and what they go turn on co-pilot or something like that and call it done. That's not a success. So having all the right parties involved, having them evaluate, okay, major, what are my KPIs, me from the point of view that I have in the role that I'm in, and how does that play into the AI outcome? So not having all the right people involved really causes problems there.

16:49So that's why what we built is this service to even help bootstrap some of this that we call discovery. And its intention is to pull all the right parties together. Its intention is to pull all the, to analyze and understand the data, to understand the business problems that we're solving for and make sure that everything aligns so that when we do go build a proof of concept or go into a pilot, that everything is set up beforehand to be successful. And occasionally it still fails. Maybe it's just not the right time or the, or you don't have enough of the information. Your information is clean, but it's just there's not enough detail or the expectations weren't right, whatever it is, or it's successful.

17:31But the point is, is that you're getting all your ducks in a row beforehand before you start to click go on that. And then obviously that leads into a lot more successes. And this ideation process also helps flesh out what are the use cases we should even tackle? Yes, we have many problems inside of a business. Again, let's start at the edge, what are those low-hanging fruit things that we could use to show value from an AI perspective? Yeah. What is over-provisioning? Over-provisioning could be a lot of things, yeah. Why is it a problem? Yeah. Over-provisioning from a compute perspective basically turns into where you're throwing far too much workload at an environment where I'm over provisioning it for, it can be kind of twofold here.

18:26I'm over provisioning from a, from I'm throwing too much at it and it's not, it's being oversubscribed. It's being taxed. So you start to see these, these choke points along the way. The other side, so now as I'm interacting with it from an end user, if I have to wait a minute or 30 seconds imagine if you went to chat gpt or whatever generative ai platform you use and you type your question in or whatever it is you're willing and then you hit wait and you hit enter and you don't get a response back for 30 seconds 45 seconds a minute you're gonna think something's broken here and i'm going to refresh the page i'm going to try again well that's because behind the scenes there's not enough resources to be able to support that ask that you're asking and that's an over provision problem oversubscribed problem.

19:11The other side of it, though, is where you over provision your resources, and you don't have enough workload to keep them all busy. So now you've spent too much money to actually to support the need that you have. Now there is obviously where you want to have some margin in there. But But yeah, from an over provisioning perspective, it can be either not enough or too much technology to support what you're doing. And then obviously there's cost implications from that. Yeah. And so Hitachi IQ, do you guys have your own suite of models or are you the solution provider and a model agnostic? We are very agnostic.

19:54What we wanted was to create something that's easy. Now, what we have done though is in our solutions, we've embedded models. So at GTC 2025, it was Jensen announced the AI data platform program we are a part of that and we are a part of the nvidia 8i data platform program and that really is the initiative to bring data and inferencing closer together um and and it's really important for storage companies to to provide that because obviously we're the keepers of the data so in the solutions that we build for this we utilize things like llama nemotron um which is an NVIDIA model that was created.

20:34But at the same time, too, we use other models, maybe LAMA 4 or LAMA 3 even, or different whatever you can get from Hugging Face that we picked at the time. We also are building our own models from a Hitachi perspective for the industries that we work inside of. Those aren't released to market yet. Those are still under experimentation and under design and being built. But we really designed it to be agnostic so that It can be as much about the solutions that we bring to the customers as much as a factory for customers to build their own AI outcomes within. And that's why NVIDIA's AI factory concept makes so much sense is because it is this agnostic engine that they can use our tools.

21:18They can use their own tools. And then our tools include things like NVIDIA AI Enterprise. Or it could just be a bunch of compute resources that they use for whatever they need to. And they never call us again, except for when they need more resources. So it's really goes both ways. And it's up to the customer on how they want to consume it. Now, we prefer, obviously, that they stay in kind of the ecosystem that we've created for them. And we build these solutions with them and all that great stuff. But it is designed to be flexible so that the customer can do what they want. Build the future of multi-agent software with agency.

21:56That's A-G-N-T-C-Y. The agency is an open source collective building the internet of agents. It's a collaborative layer where agents can discover, connect, and work across frameworks. For developers, this means standardized agent discovery tools, seamless protocols for interagent communication, and modular components to compose and scale multi-agent workflows. Join Crew AI, Langchain, Llama Index, Browserbase, Cisco, and dozens more. The agency is dropping code, specs, and services. No strings attached. Build with other engineers who care about high-quality multi-agent software. Visit agency.org and add your support.

22:53That's A-G-N-T-C-Y dot O-R-G. Yeah, and are most of your customers, is the attraction that Hitachi has these other business units, so you have expertise in, as you say, railways? uh so if if uh an analogous uh company whether it's railways or something else in transportation when they're looking for a partner uh they they say well hitachi does this already so they would have a better handle on what we need that's a part of it i i think it kind of goes it's it's a couple things one obviously it is yes we have we have an expertise that's unique that where we build the thing that we can also support from an AI perspective.

23:48So we know both ends of it. We're also a manufacturing company, a heavy manufacturing company. So we understand manufacturing and how AI can improve on supply chain and the process and all that stuff. So it's as much that though, as it's also that we're very trusted. We've spent 50 years building a brand, Hitachi Antara, Hitachi Data Systems, on a 115 plus year old brand being Hitachi. In that, Hitachi is synonymous with quality. So from our perspective, it's as much that our customers trust us because we've been working with them for, could be 30 years in some cases, so that they know that when we say we're doing something or we stand behind something, that we mean it.

24:36we stand behind it so it's gone through its paces at the same time too it's also that yes we are innovative that we're creating new things and that we're creating these things under the the point of view of how a customer would use it practically does it actually solve a real problem how would we address that problem we do it through this way because we see the same thing and then obviously having that relevance to the customer from their point of view. Yeah. Everyone's talking about agents and agentic workflows and agent building platforms. What's Hitachi IQ doing in that regard? Yeah. So we're obviously it's very new and very fresh.

25:22We are currently exploring what is the best agentic framework for us to really integrate in and bring our capabilities out. Obviously, we are doing a lot with NVIDIA. So we are using NVIDIA's IQ blueprints in some cases for a lot of the agentic capability. And so that brings in reasoning models and things like that. Like I said, the Lama Nemo Tron is an example where reasoning is super important for agents so that they have agency and can kind of work through this. But as far as how agents, the agents work together and how they communicate and all the things that have to happen from a workflow perspective through the entire agentic pipeline, we are still working to see what makes most sense.

26:13we'll have some decisions made here soon obviously it's we're spending a lot of time on this um so to say that we haven't figured out yet would be an overstatement but to say that anyone has figured it out yet would also be an overstatement so it is there's a lot of experimentation going on right now to figure out what makes sense but at the same time too it's mcp a to a for the agentic stuff it's there's a lot of new things that have come out that it's like okay what makes sense what doesn't, where do we invest? And also, of this, it's not just IQ from what we're doing on the AI front directly, but of these new capabilities, what also should be integrated into our other storage products?

26:55What should be integrated in to, again, like being a part of that AI data platform program, where do we draw threads in or have threads that go into other parts of our portfolio so that we can pull these things closer together and improve on the overall experience? Yeah. Are you also one of your clients, other Hitachi business units? I would guess that you're. Absolutely. Yeah. And do you end up building solutions that then you productize and make available to others? Yeah, we call it customer one. So we are as much customer one for what we build internally and take to market. and we do have other customers who are like, hey, we want to do this thing.

27:40And okay, great. We'll build this together. So very much we eat our own dog food, drink our own champagne, whatever it is that you want to use. But we definitely work together internally to solve our own problems and then take those to market as much as we come together with a customer. We have current active engagements doing this, solving some absolutely complicated global problems where it's us, it's our business unit partner, it's NVIDIA and the customer working together to solve some really fundamental issues to where it benefits us as much as the customer and every other customer after that, because we are going after these really big, I don't want to call it a moonshot because we are seeing improvements, but these are very complicated things that we're solving.

28:34And like I said, it's as much for us as it is for our customers in some case. Yeah. And how do outside customers work with you? I mean, do they come to you, you know, we want to integrate AI into our operations. We don't know how. Or do they come to you saying, you know, we run a fleet of ships around the world and we want to optimize our scheduling with AI? I mean, do they come to you with a specific problem or do they come to you sort of Greenfield saying, you know, come help us become an AI company? I'd say it's that. And it's also where we kind of, we spark curiosity through conversations that we have where it's not even about AI.

29:34Obviously, we have a very deep Rolodex of customers. So it's, I have a dedicated set of solution overlays, sellers, that are, all they talk about is AI capabilities. and they go ideate with customers and inspire them like, hey, this is the potential of it. So it's as much as them coming to us saying, hey, we want to be AI enabled or, hey, we have this problem we're trying to do. We've done this in manufacturing. It's like, hey, we're trying to solve for this problem. Hey, we actually have an idea for this and we work through it and like we solve the problem and they're very happy and then we go on to the next and the next with them.

30:13So it's as much us inspiring them as the customer coming to us saying, hey, we have a problem, or hey, we're thinking about this, and then having that conversation and fleshing that out to the point where it's like, yep, we can start to solve that. And this is the resources and what it will take to actually get there. Yeah. You know, AI is becoming so important in all industries. When did Hitachi IQ start as a brand? and how important it would, I would guess that it's going to become a really core offering within Hitachi. Yep, absolutely. So we launched IQ officially at GTC 24. So a year and some change ago, GTC is obviously Nvidia's big event that they have in San Jose.