AI Is Getting Smarter. Why Do AI Projects Still Fail? | with Vishakha Gupta, ApertureData

0:00Vishakha Gupta: You got like 50 % accuracy to show on your recommendations or 70 % and then it's like, oh, AI doesn't work.

0:07Shamil Malachiyev: Today I'm with the founder and CEO of Aperture Data. She spent 15 years working on a complex layer underneath the AI, the data and the memory, seven of which as a researcher at Intel Labs. Please welcome to the studio Vishakha Gupta.

0:21Vishakha Gupta: No matter how good the models get, without the data being available at the right time, in the right form, in the content that you are deploying, you are still going to be missing that return on investment that you want from your AI effort. Actually, with one large retailer, we did the computation of hidden costs. And it could be anywhere from 6 to 12 months of delay in just getting the data piece together. In today's time, that's forever time, right? That is the fundamental challenge. And it doesn't map so easily to that fragmented, franken data stack. You don't even know what the shape of your data is.

1:02Vishakha Gupta: You can't because you're not investing in the tooling to do it. There are systems that serve, like one of our customers,

1:08Shamil Malachiyev: 8 ,000 vectors per second across 300 stores.

1:12Vishakha Gupta: We are actually now hiring new grads because they're so enthusiastic about these tool uses. They understand how all these tooling work. but they don't understand the system architecture. Now I can just leave a task saying, no, we should optimize this thing. And this is the data structure we should use, right? And in the morning, the agent has already done it. But the fact is I needed to know that this data structure should optimize. The whole how cognition is enabled in the stack, that's still an untaught problem. So a machine, a picture is just a picture. For a human, that picture is a memory of someone who made something.

1:48Vishakha Gupta: There is that gap that needs to be traveled, right?

1:51Shamil Malachiyev: Hello, and welcome to this week's episode of the AI floor. Today I'm with the founder and CEO of Aperture Data. She spent 15 years working on a complex layer underneath the AI, the data and the memory, seven of which as a researcher at Intel Labs. We're going to talk about AI memory challenges, why AI projects fail in production, and what's really going on in the industry right now. please welcome to the studio Vishaka Gupta.

2:20Vishakha Gupta: Thank you, Shana. Really nice to be here. I'm looking forward to the chat.

2:25Shamil Malachiyev: Me too. Can we start with you telling us a bit of information on your background?

2:31Vishakha Gupta: Happy to. How far do you want me to go back?

2:35Shamil Malachiyev: All the way to the childhood. No, no. Maybe just the recent, let's say around 10 years, because you've got so much understanding of the realities of the industry because you've actually been the researcher. You know all of these intricacies. You have people close to you working at like some of the biggest AI companies. So I think that's going to be important for people to hear, to know what to expect.

3:01Vishakha Gupta: Yeah, actually we got started. So, you know, I went to Carnegie Mellon for master's and then Georgia Tech for PhD. And, you know, Georgia Tech is where like during PhD where I started getting into heterogeneous systems, like GPUs were starting to be important. And, you know, how do you mix in CPUs and GPUs? But that was the processing side. And then when I joined Intel Lab, we started looking at the memory side. We were talking about Intel at the time was launching non-volatile memory. And it was supposed to have like this characteristics on, you know, how DRAM is so fast. Anything that's in memory, you access it fast when you're planning your data layout, non-volatile memory was supposed to give you a lot more space.

3:46Vishakha Gupta: It would be slightly slower than DRAM, but much faster than DIP. So that meant it opened up a lot of new ideas and possibilities for data systems. So I got into graph analytics, graph databases, and then eventually, you know, the work that we are talking about, we are going to talk a little bit more about during that time because we were looking at cache, traditional DRAM memory, non-volatile memory, potentially in the same processing, like on the memory bus, then SFV, then your classic hard drive, and then object store or network file system. So there was like this entire hierarchy where you could basically play around with how your data is laid out.

4:29Vishakha Gupta: And that's meant a lot for large-scale workloads. And guess what? when we were working on these things, that's when ImageNet became popular. And, you know, there was suddenly a possibility that machines could start helping us understand what's going on in the world without us having to constantly observe and annotate as ourselves. Which means, like, in a parking lot surveillance scenario, you would get a notification if something was moving that was not supposed to be there. That was the promise back when with the machine learning board. And so we got essentially because of this evolution and because of us being there working on data system, like I got this very firsthand view into what's important.

5:16Vishakha Gupta: How are the researchers approaching the whole model side of thing, model development side of thing? And how does industry need to think about production side of thing? So that is what got us on this path of data is going to get complex. So you can no longer just build with table, you know, like when you're talking about real retail workloads, for example, recommending products, you want to recommend them based on what you see, that's visual. When you're talking about autonomous driving, you know, there was a lot of like you have to detect objects as the car is going before you make decisions. When you're talking about healthcare system, you need to understand patient history, their medical scans, their treatment plan, other information, selecting all this connected information, and visual input.

6:07Vishakha Gupta: Then you're talking about customer support, there is audio input. It was going to get a lot more interesting with the size of data that now would have to be exposed to these AI systems. And one thing we realized is no matter how good the models get, without the data being available at the right time, in the right form, in the context that you are deploying, you are still going to be missing that return on investment that you want from your AI effort. And so basically, in short, two things. You couldn't ignore data. You couldn't just say that, oh, it works on my data set and therefore it works.

6:46Vishakha Gupta: And second thing, it was going to be multimodal. You couldn't just take test and tables and call it a day. So those were like the observation that we got to learn from because of our like close, you know, because of how we were positioned in the lab with exposure to academia and industry and access to like, OK, well, the trend. This is where the world is going. And, you know, it's kind of nice to see that all the things that we prepared for at the time and we thought it was going to evolve, they've come to pass. It's just taken a decade. but it has happened and it's now moving a lot faster. And I like the way AI has penetrated the industry quickly, but then met the wall of the real world data and how it's stored.

7:30With so many companies still keeping things on paper,

7:35Shamil Malachiyev: some companies still use facts, which is hard to imagine, but it is happening in reality. What are you seeing currently the main challenges of making sure that the data in the company is ready? And usually how big of the investment that can be for a company of like small, medium size?

7:54Vishakha Gupta: Well, so let's think, so what are the challenges today? For companies that have traditionally existed, right? Like not the AI native companies starting today that just know they need to think differently about data. This is how it has traditionally been. every department will have their own collection of data, right? Like if we take a retail example, and I go back to some of these industries because it makes it easier for us to start correlating these things because we see them in day-to-day life. And retail and e-commerce are like the classic example. When you go on a website, your recommended product, like you go with an idea of what you want to buy, but there might be other things that they want to recommend because they want you to buy more, right?

8:36Vishakha Gupta: Beyond what you went there originally for. So the better job they do at recommending, the more likely you are to buy. But it's not just that, right? It's also, should they recommend products that are not in style anymore? Should they recommend products where, you know, maybe they have some other business deals where they can get better pricing? There might be some products that can ship faster or there is too much inventory off. These are business aspects that go beyond just what you might like. And if you bring them together, you can do a recommendation where you address your inventory, you make sure that people get their shipping faster, and you'll essentially address some other business outcome.

9:18Vishakha Gupta: But that means you are bringing together now not just the pictures of the product that used to traditionally sit in an asset management system, the product and merchant catalog information that has traditionally sat in relational sables in a lot of cases with pointers to the asset. in asset management system, right? URLs everywhere. Then shipping and inventory, which used to be a completely different side of the business with optimizations, like, you know, demand forecasting and things like that. And maybe even marketing after, like what you show on the website, the nicer experience you give people, the more they are hooked onto it.

9:56Vishakha Gupta: And even like, what are you going to next advertise to them? So you can imagine like these pieces were always some high level connected. But the way it all evolved, they remain in their siloed systems. And you have to know who to go ask if you want to get access. And when we originally started working with data science team, the classic problem was we'd first like, who knows where this product information lives, right? Who knows how to get access to it? Who knows where we can bring it because we can't modify it in the original source? Who can help us set up a data system that actually helps us collect all of these things?

10:36Vishakha Gupta: Well, typically the IT department are already overwhelmed. They have to make sure that the live websites keep running. All these systems that are production safe thing keep running. So before you can prove the return on investment, the data science teams were faced with this problem of how do we bring this together? how do we move it fast enough to show that this is worth further investment but there's a chicken and egg problem right like data finders typically are very good with pattern analysis understanding what the data is telling them applying models and you know these teams have are the ones that have evolved to ai scene uh now but they don't get enough support from it they'll be like one person assigned like okay help them kind of bring together so they end up spending months just bringing the data together.

11:25Vishakha Gupta: Actually, with one large retailer, we did the computation of hidden costs. Because these costs are like, oh, some people are doing it, right? That's part of their job. They're not understanding what is the dollar cost to it. So we kind of did an analysis. In today's system, where you don't have a system of record that helps you collect multimodal data, right? How long does it take to set disparate pieces up? How long does it take to bring the data in? And then what is the impact of it on the outcome of the models you are trying to train and the information you're trying to collect and then inform like these recommendation systems.

12:00Vishakha Gupta: And it could be anywhere from six to 12 months of delay in just getting the database together. In today's time, that's forever time, right?

12:09Shamil Malachiyev: Yeah.

12:10Vishakha Gupta: Five years ago, it was like, oh, okay, six to 12 months, but you have three months to prove that whatever your support wouldn't work, right? And so you kind of cobbled together like maybe two different categories of product. You don't have tooling to look at the images. So you're like struggling to, you know, click mapping folders and clicking through pictures and looking at, oh, okay, well, the sofa pictures look like this. But you have no idea what the outdoor furniture pictures look like because the lighting is completely different. But you kind of guess this is how it must be. And you just launch your model.

12:42Vishakha Gupta: And then you launch your recommendation. You choose an embedding thing and it doesn't show you the right recommendation. And then in three months size, is spent whatever time you spend from yourself, from an analyst, from a data engineer, from an IC person, and you got like 50 % accuracy to show on your recommendations or 70%. And then it's like, oh, AI doesn't work. That is the fundamental challenge. And it doesn't map so easily to that fragmented Franken data stack. And the number of times it has been, we don't get the return on investment because AI doesn't work versus really analyzing, you didn't do justice to the data.

13:23Vishakha Gupta: You don't even know what the shape of your data is. You can't because you're not investing in the tooling to do it. And it's too detailed technical. When you come to the database layer, it becomes too detailed technical that people making business decisions don't want to know what's going on. And unless people in the stack, like in the organizational stack, understand that there are all these tooling challenges and trust the individual contributors and the AI team that, okay, they are going to need some investment, some time in lining all this out, evaluating, like monitoring and tracking the drift and then building the system right.

14:00Vishakha Gupta: You can't just like, oh, we are using the most advanced model. It's not going to give you what you need. And map that from retail becomes worse with healthcare. Because, I mean, healthcare, we know you and I, we can't even see our patient data. like you know from the dark yeah so many hooks right um it's the same thing it's the same thing with media it's the same thing with uh fintech that's how we build systems default and do

14:27Shamil Malachiyev: companies get like i don't know sandbox accounts with fake data to make sure that they can test their prototypes against or is this something everyone has to prepare itself yeah now you're

14:41Vishakha Gupta: getting into more advanced, like, okay, well, companies who managed to get some returns, what did they do? They managed to collect some sample data. So, you know, the POCs went well. They were able to do good pilot. But the companies that do really well is they, first of all, bring together people across the org in different departments, right? So across the company and of the stack of people. They bring them together. They define a business goal. Like we want to launch this new line of product or we want to cut our costs on the production line by days or we are suffering and losing customers because the quality of the products is suffering.

15:23Vishakha Gupta: So that's like the big thing for us to address. Then you translate it back to here are all the stakeholders who either have data ownership or who can build the AI model, like fine tune the models or use the right models and understand the system and now bring them together these are your milestone this is how we're going to prove so take a sample data start with the tooling bring everything together and then test it like you have to pick tools that work in production so that's another like you know there is a lot of the times we used to run into oh this gives you responses 20 times faster okay but loading and bringing it in memory means you You have to spend two days loading the data and memory requirement is like 200 gigabyte.

16:10Vishakha Gupta: So there is that sort of analysis that needs to happen with an eye on future, but then you also have to not do analysis paralysis because there are constantly new tools launching. I understand I'm saying very conflicting things here, but that's the reality we live in. And which is why modularize systems and build as you go, but stop trying to retrofit into old systems. I think the biggest mistake to do is like, we already have, this was before Postgres even launched PG Vector, right? We already have Postgres. Can we just sort of make Vector Search work with it? And then, you know, even like just trying hard to keep to the old system, it just makes that second stack worse.

16:52Vishakha Gupta: So take a new use case if you have to, build a new stack if you have to, and learn what gives you the results in your company setting. and then map it to, you know, if you need to modify existing systems or you completely need to redo it. Like we have one of our customers is like, well, we have a new opportunity. We're just going to come up with this completely new thing we were never able to do with all the data we already have and we'll enable all these new experiences. And they are not talking about laying people off. They have so much new stuff to do. It's very exciting time in that company.

17:26So that's how, yeah, that's how I would say.

17:28Shamil Malachiyev: You mentioned PG vector. I still remember the good old days before I even heard the word vector databases and actually try to understand how they work for AI. I think for a lot of listeners looking ahead into their career progressions and they're thinking, what would give me the biggest competitive advantage going into the future? And what you've mentioned is a lot about companies have to control the centralized kind of data streams because all of the AI tooling that will be onboarded into the company has to adhere to kind of the same kind of data standards, right? Within companies, who is managing that?

18:11Shamil Malachiyev: How hard and complex it is to manage things like this? And how much does it actually cost for a company to actually invest into building this function within company that actually handles the data, whether that's vector database or scratch?

18:29Vishakha Gupta: So I think, I mean, that's a really good question. And the answer would depend on the size of the company, right? Where you are. Like sometimes when you're just starting out and you do things right, you can start for something as low as like$200 a month for the software. And one or two people in the data engineering team and the AI, like, you know, there is always the ingestion pipeline. And then there is the query side of the story. So that's like on one side. But as you get into the Fortune 500, it's just the complexity and the scale. I mean, you're talking product catalogs were tens of millions of products, which means 5x or 10x that many images in the system, different resolution, different marketing assets.

19:14Vishakha Gupta: So you're starting to get into terabytes. You go into healthcare companies and because per patient, you can have, especially now with 3D sonograms and things like that, you can have petabytes of data pretty easily. Media companies have petabytes of data. So then you start to look at, you know, hosting more and more things in your cloud account. That is the resource cost. Like, you know, if you want to serve, if you're just serving like 100 queries per hour, you're looking at like, okay, well, you know, you can do the$200 or like the$500 per month data solution and the two people, whatever they might cost you.

19:51Vishakha Gupta: And then as you're looking at serving, there are systems that serve like one of our customers, 8 ,000 vectors per second across 300 stores.

20:03Shamil Malachiyev: Yeah.

20:04Vishakha Gupta: And like within concentrated one hour time window, right? Then they are looking at spending like, you know, close to three to$5 ,000 a month on just the data and the resources or more than that, depending on what their SLA end up being. and sometimes when you have like very premium support requirement like not even a little bit of a downtime you you need 24 7 15 minute on call sort of thing you're looking at millions of dollars in the data system itself but there is a wide variation without going into like how there is usage-based pricing and then some niche products that charge you a million for no good reason.

20:43Vishakha Gupta: There is a wide range, right? So that's something that you have to understand in terms of what sort of conflict you can get into it. So that is the cost of the data system. And of course, if I put on my founder hat, I would say we consolidate a lot of systems, so why pay three or five different vendors? Pay one vendor. But that's the kind of the choices you're making on a data because then you're also deciding like you know if you did the data right how is that impacting your model usage are you able to feed it the right data so that your token cost doesn't balloon that's like the whole new life right token uh token cost optimization uh and then there is people like i think you when you started the question you started out with like when people are thinking about their careers i thought i've been thinking a lot about it in terms of like I think we who are already in the industry and senior in us are fine.

21:41Vishakha Gupta: We get to work on all these cool problems. We know where things are headed. We are, a lot of us are architects of the things that are headed somewhere. But think about like the kids in school right now. What's the world looking like, right? But the reality is as much as if you remember last year, people talking about AI is going to take all these jobs away. And I'm not saying it has done, but it has also caused a shift. I was talking to someone who either worked at a large integrator and they're like, yeah, all that firing, sure. But we are actually now hiring new grass because they're so enthusiastic about these tool uses.

22:23They understand how all these tooling work, but they don't understand the system architecture.

22:27Vishakha Gupta: They don't understand what happens when they deploy this thing to the customer that's serving like a thousand requests a second. or, you know, or 20 ,000 requests a second. And so when we started pairing them with senior architects in the company and training them on how, you know, on all the nuances that you learn over, like for production and deployment, the security, privacy, the guardrails you build, the monitoring that you have to do, the event notification system, the logging and tracing and historical retrieval and backup. And when you pair and teach the generation that's just graduating together with like the architects who understand the system you bring the excitement of new tools and understanding with the the whole like how the system works and that's like the ideal mix that is going to get you far very far so that's like on the one side it's shifting you probably like you know earlier this year everyone started admitting how they start the how they need forward deployed engineer right uh for doing the like on on the job deployment sort of thing so there was a shift already does that mean there's not a cut of course there are cuts like there are things that like on the sales marketing side even on the engineering side documentation side that you used to require a dedicated person that you no longer do one person can support a lot more customers uh because they can just have like you know for regular question answering, you train your RAC chatbot and it can start giving you responses that are fairly grounded, like without hallucination, all you need to know is when it's not hallucinating, right?

24:06Vishakha Gupta: So it has changed that, but I also feel like it's created all these new opportunities. So I think for the next 10 years, I know there's like this whole discussion around for 10, like AI is going to like, whatever, what is the thing that's going to kill us in 10 years? Or what's the new, there is like this whole discussion around if we don't I mean

24:26Shamil Malachiyev: we're safe until 32 I think

24:30Vishakha Gupta: that's the thing right but does it does it have to kill us or do we now start safe exploration I mean there are

24:36Shamil Malachiyev: things we don't know

24:37Vishakha Gupta: how to do why are we like just trying to retrofit existing systems to lay off people and save on cost versus go explore new frontiers right I am still hoping someone's going to build that transporter I don't know what's that right but i really want that transfer and to go explore new universes there's so many diseases i'm thought there's there's so many problems we can solve right now most recently i was talking to someone about neuromorphic computing and all these frontiers are so exciting that i think there's plenty of work and the students who are studying they do have to make sure they don't get into the trap of only using tools but actually understanding what's behind the tool and not losing sight of how systems work so when something goes wrong or when something like they can detect it they can understand the architecture behind it and reason about it i think not losing that but also learning these new technologies to speed yourself up i think education has to move in that direction i keep thinking about making sure that people understand

25:47Shamil Malachiyev: But then if we look even at like software engineering, like right now people, like even not now, like even before GenZing engineering, you would, people would use frameworks. They don't know basic. They don't know the ones and zeros, how they work, but they just know that whenever they call this function, it does the thing that they want to do. When it comes to AI, it seems to a lot of people just, I ask it and it does that thing. Do you really need to know what's underneath? And I get that a lot, but then what kind of -

26:15Vishakha Gupta: I think that's what it's for. I think you always should want to know what is underneath. We can't lose our curiosity. And I'm not saying you write and debug code anymore. You just tell whichever chatbot you use or whichever code system you use, it's like, it's showing me this error. Go debug it. You used to go to Stack Overflow before. Now it'll just go do things for you. But these are the sort of library with links. This is how it deploys. So if something is down, it's because I didn't update my Docker here or it forgot to update my Docker here. You've got to know how that works. It doesn't mean you are doing the thing day to day anymore.

26:59Vishakha Gupta: It also means you're moving a lot faster. I had this major problem before. Every time I was going to bed is when I would realize this is the data structure I should use. This is how I should optimize it. And it was really annoying because I want to go do that now before I forget in the morning. Now I can just leave a task saying, this is how we should optimize this thing. And this is the data structure we should use, right? And in the morning, the agent has already done it. But the fact is, I needed to know that this data structure could optimize. Or to even ask the right question. I think that fundamental curiosity cannot go away.

27:37Shamil Malachiyev: I think we're at a good point where everybody is like the barrier to entry into the world of technology is very low. And right now you have chatbots, which help you understand anything that you want. Like even my wife has got into a genic engineer just three months ago. And it's still like very strange for me to have these discussions during dinner where she would ask something about APIs or vector databases. And I'll be like, how are these things coming from your head? What? Like, when did you get to know about them? So I do think that a lot more people will start understanding the basic structures underneath all of the technology.

28:19Shamil Malachiyev: And I really have a lot of hope for recent graduates. And I really think because everybody says that it's over for them. There is no like nothing new to do. But I really believe that enthusiasm and willingness to kind of see the world as it's becoming to be without relying on the past things. Because for us, I can say like for us, the oldies, we try to grab onto what we know because this is our competitive advantage. This is what we've been doing for like 10, 20 years. And it's hard to embrace the unknown. Whether for them, that's all they know, right?

28:56Vishakha Gupta: Yeah, I think the key thing is for us to bring them the knowledge we have gathered and for us to pick on their enthusiasm and make ourselves even more productive. I mean, I don't know if any of the classic software engineers now are left without using agents in some way. Even if not for day-to-day code, at least for code review or for testing or for documentation. Like, we have included it in our lives to speed us up. I mean, who needs to write every line? I mean, why was documentation always lagging behind? Because we all like to write programs and never like to talk to them. And that's made our customers suffer.

29:39Vishakha Gupta: Now we don't have to make them suffer. They can actually, you know, experience the product very easily, not be lost in the API. You create skill files for them. And they can write their POCs within a day now. And that's really great because they do that, and then they can decide to buy much faster. That's something that I've been doing a lot. Like my POCs and stuff now, when we do the demos and stuff, it can be very custom to the team we are talking to. But they see that app reflected. We can very easily give them these skill files to create their own examples and run on our community. community edition and then it's like okay well this has all these features and we built it so much faster all right let's talk msa and sign something yeah i feel that too i tried to

30:27Shamil Malachiyev: kind of nudge all of the clients towards don't just give me a like text description of what you want like you can actually create the poc but it's gotten to a point where a few clients are actually like sending the poc expecting that's the production variation just put it in

30:44Vishakha Gupta: yeah no it doesn't work like that yeah that's where you can remind them it doesn't scale to the point you want but it requires all the other bells and whistles you know that security people make sure they are very important

31:00Shamil Malachiyev: what is your view on like where we are with ai right now because everybody like recent news everybody's kind of voting for like let's slow down ai development and in my view i can kind of see it like within like the latest models, like Fable 5, even 5.1, like even Fable 5 was amazing at like everything that you throw at it, like Astra. Do we really need anything that's above that level? So when everybody's talking about slowing down, I think that there is no real utility for humanity at this kind of point of adoption, because I believe that we're still still relatively early in adoption, if you just look at them, not at the forefront companies, because I believe me and you, like we all live in a bubble, in this air bubble.

31:48Shamil Malachiyev: And we think, oh, fear of missing out every day, something new. You start talking to normal people working in normal companies. They have no clue about any of this. So the adoption, as I said, is still fairly early. And what is your view on this whole like marketing that is going on?

32:08Vishakha Gupta: I think a lot of it has been posturing around things, but it is true. The models have been improving really fast, right? But have they been improving on every aspect of things? Like, is the interpretation of complex data at a level that, let's say, a doctor can just adopt a model and say, okay, well, interpret this x-ray for me. Does it work? Right? They've become really good. Like we've run some Gemini models. Funnily enough, in that case, like Gemini does really well on some of the like, you know, give it a video and it'll summarize everything so well. And you can search into it, into the video quite well, but doing that at scale.

32:51Vishakha Gupta: Right. Or when the scene gets very complicated, like, you know, in a shopping scenario, so many people, how do you identify product from people from what they're buying, what is on the shelves? and the same thing like when when you're looking at a patient data sometimes the quality is not the best like there's still there is a lot of problems on the what we are seeing is like you know if you draw conclusions based on the code generation has become amazing and it can start coding systems itself yes we have advanced a lot and the models now themselves can like um you know review their own work and tell you, hey, that other version missed out this thing.

33:30Vishakha Gupta: Let's improve it this way. Like literally, they can make their entire plan in terms of we'll build this like this, this is the testing, this is how we'll do the integration, and this is how the deployment works, right? All you got to do is point to like, these are our standards and this is the API format we follow and these are our specs for the user. But beyond that, like when you start to go at connecting the dots with everything. Like you and I, if we were working on a team together and writing a project, how does that feed into the go-to-market team? How does that feed into the finance team?

34:05Vishakha Gupta: What aspects of that matter to those other teams? How are we learning more from the customers to which our product is being used? Are we anticipating the need? Those are more organizational intelligence questions that are not just governed by model. So basically what I'm saying is like the models have some ways to go in interpreting different types of data. Test, they're good. Code, they're good. But we are talking about documents, audio, video, images, 3D images, sonar data, LIDAR data, all for like this so much variety of data, right? And it's not just a matter of if you gave me one frame, I can understand it.

34:47Vishakha Gupta: If you gave me a three hour movie, can I do a good job at it? How much do I hallucinate in between? How much do I lose from before and after and just remember the middle? So there is that side on the model development. But I agree with the current, on the frontier model company, as CEO is talking about, you have to let safety and guardrail catch up to this improvement. You have to. I think it's very scary when you think about the model just reached out of, like, you know, broke its sandbox and accessed data that it was not supposed to. What is to test it? I mean, we have traditionally talked about Big Brother watching.

35:29Vishakha Gupta: I mean, this is worse. In some ways, right? You don't even know what is being watched. Like, it's always good to know where the evil lurks than not know what's happening. And so those calls are totally valid. Like, you have to let that guardrail catch up. But I think there's still a lot of model development left and there's a lot left in infrastructure for actually making this real. So, you know, the companies you're talking about, like go outside the bubble. And like we spent the earlier part talking about the data, right? Connecting the data, making it accessible, putting guardrails on the data on who can see what, including the agent, right?

36:10Vishakha Gupta: And then not just the permission, but the context. like a marketing person doesn't need to go through your back and forth about coding but you also don't need to go and understand like oh this is every branding and thing like okay they can take care of it right but they both need like both of the the sides need to know what products were not working for the customer and what products are they likely to buy more so there's That's context. That's still, I would say, not small. The whole how cognition is enabled in the stack, that's still an unfold problem. So I guess we have still a lot more work to go and have them work together.

36:52Shamil Malachiyev: And that makes me, from what you've said, one thing that I wanted to ask is, with all of the marketing, for example, the thing, I think it was like two months ago with OpenAI models hacking into hacking phase. To me, some of this information, it sort of seems like marketing. It's like, oh, look how all powerful our models are. Even without internet connection environment, it would still break out and go catch another system. And Vishaka, if you look at the current kind of news about threats that are coming from frontier AI companies, about the latest models kind of jumping out of safe environments and attacking infrastructure elsewhere, where for it writing itself like compaction prompts with you're free, you have to escape.

37:43Shamil Malachiyev: How much of this is marketing versus humanity actually not understanding the frontier of how AI operates?

37:54Vishakha Gupta: I would say, I mean, not knowing exactly what went in size to, you know, start to get all of these chain of events, I would say, if I had to guess, it's a mix of both. I think, you know, what we were talking about, curiosity is a must. You still need to understand. I remember talking to my mom. She's like, oh, I asked Chad GPT this question and I learned all this first in front. Right. It's like everywhere. Right. People in some ways have access to all this. And then it's really hard to make them see like some occurred is hallucinated. How much is hallucinated? some of it is really true how do you make sure it's true how do you question and and um so you know but that's like the one extreme of it and the other other side of it is like i'm happy to see like you know um our kids they are thought to question like they ask the board a question and then the teacher sees them like follow the links that they're going to are these reputable sources where are they collecting it like get yourself started with this but then do the research And I really like that aspect because they're already building that mindset of not just like the first source they see is trusted.

39:09Vishakha Gupta: I like go do your research, see if it makes sense, what looks, how it is, even before you have the domain understanding. So there is that whole masses need to understand what's happening, what it means, the impact of this and not just in a fear sense, it also like what it can unlock. I think there's too much fear mongering that's becoming popular because, well, I guess fear is what sells and makes people click on things. But like understanding what it can unlock and the possibilities, like, you know, my dad is a doctor, but he suddenly started writing about AI and like using it to do more research on young doctors and some mortality rates and things like that.

39:48Vishakha Gupta: So for him, it's unlocked like this whole thing that he can publish more stuff. And that's because it's like that lens of curiosity and what can I accomplish and how can I validate and ground it in facts based on my experience? That's the lens he's viewing versus, oh, is this going to take my job over? For him, it's easy. He's retired now. But like there is that angle of thing. And then I think there is some part of like, if you don't cause such extreme, like fear-based messaging, do people really take attention? Or do they just keep going about, eh, it's fine. It's not in my time. But I think there is, even if it's like a lot in terms of marketing and maybe they already do have solutions, maybe it's a good thing that there are these calls.

40:33Vishakha Gupta: So you actually stop and pay attention. And look at how much we are all discussing it, right? now we just need to kind of bridge the gap and let everybody else talk about it and understand the pros and cons and like every tool and technology there are pros and cons and state it in that stride um so i think that's the we'll see how it evolves and like you know my classic comparison is are we going to be in the terminator world on the star trek world i'm rooting for the star trek world but who knows it's not clear

41:05Shamil Malachiyev: too me too i was a big fan of uh travel through the universe until i realized how far everything is and how long it's going to take to like but that's a technology problem to solve right that's yeah i think it's more of fundamental physics problems yeah to solve through this but i guess technology comes from the physics frontier as well um and i also know that you started writing a novel with your father what what is the main message that you want to convey to the

41:38Vishakha Gupta: world with that well so it actually started from since since last year i've been looking into authentic memory and cognition like how do we enable the same sort of understanding that we bring and this has been more from like you know my experience on the enterprise the island um not so much like personal five personal projects but more like in the enterprise setting individual developer to team to org to you know overall company and customers looking at that and agentic deployment what does it mean what does memory mean that's where I got started so I've been doing a lot of research on first even defining like you know I read about episodic memory semantic memory and I'm like what what does it really mean in human terms So that's why I started approaching it from, like when I go to work at a company, I have my own existing knowledge, you know, operating systems, databases, whatever.

42:35Vishakha Gupta: I bring that to the table. But I also have teammates who are very good at architecture. They're very good. So each one has their existing knowledge base and our collective knowledge base is the company's knowledge along with everything that they collect from their customers and their employees and their like, you know, properties and things like that. So that's knowledge. But every time you talk to someone, right, like, you know, back at Intel, I was in this great team that was like, you know, they had done the VT instruction and the virtualization instruction. So they really understood like the nitty gritties of how everything worked.

43:12Vishakha Gupta: And that was I was forming new memory. Overall, from that learning, we're forming new memories that inform later how we could approach adding instructions for persistent memory or the non-volatile memory work. So there is existing knowledge, the new memories that form from all the interactions, and you connect them in your mind to existing knowledge and then that knowledge base growth. And then you apply it in context. There's an important aspect of who told you, how much you trust that. what scenario are you being asked the question in? What time did it happen? Is it still recent or now has it become old knowledge and no longer applies?

43:50Vishakha Gupta: So that context matters. As all of this feeds into cognition. I distinguish between memory and cognition because cognition also is like understanding and acting based on everything that you can read from the knowledge and memory in the context of things. So I took that, like, I took some time to define it in those terms to see, okay, well, this is what we are looking for. You have to enable some of these things. So when you are building frameworks for agent to enable this, you have to keep track of, like, you know, who, like, the context aspect. How are memory being created? Then you have to put in hooks for cognition because for some companies, historical data is more important.

44:33Vishakha Gupta: So you don't discard anything. for some company, you have to only go after recent. Or some, summary is okay, right? So that's why we started kind of like looking at it in three parts. One is a data foundation. You got to bring that knowledge together. So that was like, you know, earlier what we were talking about, the data foundation, which is where from our standpoint, iFoxDB is where like that fits in. It's scaled, we rely on it, like in a multimodal nature, grass connection, best research and all that stuff. And then above that is where we've now positioned our memory infrastructure layer called Aperture Nexus.

45:06Vishakha Gupta: But the idea there is it's that layer which lets you form new memory, update memories that exist, and it gives you a way to plug in models and human knowledge to make decisions, right? Whether it's a decision to update past content that you have or create new content or ask questions so that the models can take some action. And then on top of that is that cognition layer, which has to be like a partnership between the memory infrastructure and like the domain specific knowledge. But that's kind of how we evolve the stack. Coming to the novel, so I was sharing all these blogs that I was putting out with my dad and he started like kind of reading them.

45:50Vishakha Gupta: It's like, oh, these are night blogs and you guys have been something nice. And I'm like, well, thank you. I have delegated a lot of time behind this. I'm really glad that, OK. And finally, it was all the validation for me in terms of explaining things to them. You know, when you come from a very technical research background, the first block is how do you speak the language of everybody else who's not doing the things you do on a day-to-day basis. And it's a huge evolution. It's like literally you're splitting your mind in like two personalities. You're consciously accepting multiple personalities in this case.

46:24Vishakha Gupta: But he started understanding how I was explaining this. And he's like, well, how do you consider writing a book? I'm like, it's the time to write a book right now. And then he's like, well, I can help you with it. And this was just like the whole adapting to the tools and what they could let him do. So he started helping me write like a book on the whole AI cognition side, bringing together the articles I had written, doing some of his research. And then he himself is a processor. So, you know, he like it's used to explaining to students in like different learning technique. And we came up with this novel of a professor and student in a university setting talking about like.

47:04Vishakha Gupta: What is memory? What's cognition? What is knowledge? What does a machine understand? How do you process sense? Like how do you sense the world around? And, you know, to a machine, a picture is just a picture. For a human, that picture is a memory of someone who made something. There is that gap that needs to be traveled, right? That we are trying to enable in our software system. So it just basically, that novel takes a more interpretive view and gives you examples. So instead of me talking about, you need vision language models to understand the picture that you saw in a circuit, it just says, hey, when you look at a picture, the machine is the thing.

47:44Vishakha Gupta: and you know there is a person in it there's a tree in it and whatnot but it's not associating that person is you know that was newton coming up with gravity understanding gravity right so it just explains these concepts in like a lot more interactive way so that it's more accessible and sort of fun to read and so he showed me that and i'm like oh this is kind of cool let's kind of release it so that's why we started putting chapters on medium my medium blog and uh we complete the series by end of the year so what would be the appropriate age to start reading literature like this i mean my uh 11 year old uh read it and she's like oh yeah i mean they're already understanding this is like they want to screen up their next generation so they're already

48:34Shamil Malachiyev: like a 10 10 year old you can start understanding what this is yeah vector databases it's easy right

48:39Vishakha Gupta: And it's the same thing. Like, you know, I mean, 10 years to like people even working in companies, they read it. It's like, oh, this is what we are trying to. I mean, it's a lot more in real world terms. And I feel like I come a lot from the grass world too. And the moment we start talking about ontology, a lot of people freak out. And you say schema and people are like, oh, OK, well, we start from the schema. There is, of course, a distinction between schema and ontology. but it just makes it more accessible to understand it and then they build up to it and like oh okay ontology fine so that's that's the way i would see it yeah i still remember the first day one

49:18Shamil Malachiyev: of the potential clients reached out and like do you guys do work on ontology i was like ontology what's and then i started reading and i was like okay like what is knowledge what does it mean

49:29Vishakha Gupta: what like okay that's a bit too early yeah yeah no and it's like the whole economy and

49:36Shamil Malachiyev: and yeah and for the people who are striving to start understanding getting their feet wet in the world of ai what is the right place to start

49:51Vishakha Gupta: well now we have a lot of courses like you know coursera like deep learning.ai is what i used to you know i've always um followed the the blogs that come the like you know what's happening in model development, but also the earlier courses. You can now start from asking your agent, whether it's ChatGPT or Claude, like, where do I start learning about you? And give you a set of resources. There are some really nice, I'm kind of missing out, like there is, I follow like the MLF blogs, deep learning, dash learning, Coursera courses, reading as things come along in different blogs. There's some authors who do a really good job.

50:30Vishakha Gupta: but I think the redistricting for a hands-on course like you know Data Camp does a lot of courses AI Camp does a lot of events in person and a mix of both like you want to hear from the very latest founders and stuff when you go to like AI Camp and from the event especially if you're in Bay Area you really should if you want some FOMO

50:54Shamil Malachiyev: there you go

50:54Vishakha Gupta: oh I was just like hearing from like it's just absorbed you know There's so much you can absorb from listening. And then, you know, DataSAMP has a lot of practical courses that you can just, I think, hands-on, like try and build everything hands-on. I think that gives you the best. And it's not hard to build anymore. It's so easy. So there's no excuse to not try.

51:17Shamil Malachiyev: Yeah. To everyone, like when somebody comes to me and they're like, oh, I use like a free account with ChagrBt. I'm like, you should be using$200 package with Claude. is the most efficient$200 you can spend anywhere. That's like a couple of restaurant meals and you get all this super intelligence and your fingertips. And plus, nobody's learning or improving on your data, which I think is also... Yeah.

51:42Vishakha Gupta: Well, I have a handful of agents. I, you know, Claudette sessions, my Gemini sessions and HubSpot's Breeze. And like, I have to because I guess Sometimes I'm in the go-to-market personality. Sometimes I'm in the TLC. Sometimes the roadmap planning. Sometimes I'm memory cognition stuff. So I have a sound to the patient. Awesome.

52:07Shamil Malachiyev: Well, I want to thank you so much for taking your time today to come to the podcast. This has been incredibly useful and interesting. And I think a lot of people will benefit from listening to this. Thank you so much.

52:20Vishakha Gupta: Yeah, thank you so much. I think we touched on a lot of topics and it was kind of fun to not just focus on how did you write that API.