The timelessness of vector databases | Pinecone’s Ram Sriharsha

0:05Welcome to Dev Interrupted. I'm your host, Andrew Ziegler. And I'm your host, Ben Lloyd Pearson.

0:39from Sean Godecki talking about work politics for engineers. Another thing that caught our attention in the news is celebrating one trillion webpages archived on archive.org, an amazing achievement, but also a month of celebration for all things archiving and recollecting digital history. And a Coinbase engineer shared more details about their AI use case for code generation, which taps into a story we covered a few weeks ago. And last but certainly not least, the largest Anthropic enterprise AI rollout to date just happened. A lot of really interesting things happening in the news. What do you want to dive into first, Ben?

1:18Well, first of all, I'm super excited for our guest because I just started using Pinecone this week to build some agents for here or workflows that Devinterrupted. Super fun tool to use. But let's talk about the other tool, the other AI tool that I really love as well, and that's Anthropic. I want to hear what's going on here. Yeah, so the largest anthropoclod rollout to date just happened for Deloitte. And Deloitte's making clod available to 470 ,000 technologists across its global network and organization. Talk about a huge adoption wave of folks picking up this AI tool for what's really the first time inside of this large traditional enterprise.

1:59eyes. This marks a really powerful transition of AI being an everyday tool for knowledge workers. And this was covered in an article we read from Hithy Cyborg, which has actually quite a number of really interesting articles from this week. So we're going to make sure we embed his really great roundup in this post so you can go check it out. Also featured in there is a Google bug hunting bot called Codemender who hunts bugs in your code base overnight and OpenAI being valued more than SpaceX. So be sure to go check out those articles. They're really great. But the Anthropic and Deloitte announcement is an expanded alliance that brings to AI closer to knowledge workers in a way never done before for its type of industry.

2:39And as part of this, Deloitte understands the challenges that it's going to be heading face first into. So they've actually established a Claude Center of Excellence. And this is a center of trained specialists that are going to create implementation frameworks for how the Deloitte technologists are going to adopt and roll out AI. So this represents a really fascinating learning opportunity for the rest of the industry to pay attention to what's going to matter to Deloitte at their Claude Center of Excellence, because this collaboration is representing, you know, the work of both sides, both Anthropic and Deloitte.

3:15So you're talking about folks in the seat using AI to have massive impact for a huge company, global company at scale, and a technology firm making that model possible. So there's so many fascinating things happening in the story. Ben, what do you think? Yeah, you're right. I mean, there's a lot of reasons that I love this story. And part of it is, you know, our producer, Adam, he was showing us the Claude's new ad campaign that they're out with right now. It has slogans like keep thinking, like the AI for problem solvers. And I feel like those two slogans like pretty much describe like the role of consultants, you know, like Claude really is for consultants, in my opinion.

3:54But there's like two kind of things that sort of stand out about this to me. The first is that, you know, Claude is the most verbose GPT platform that I use, like by far. You're absolutely right. And in the hands of a consultant, I feel like that could be very dangerous. Like, I'm sure there's a lot of people at Deloitte that are going to be like introduced to this tool for the first time, probably going to be making all the same mistakes that we all did when we first started with GPTs. One piece of advice, if there's anyone from Deloitte listening, don't ask Claude to write your emails. It just always writes a novel, and no matter how many times you ask it to make it shorter, it will never be short enough.

4:29But more importantly, Claude is this incredibly powerful artifact generator. And when it's connected to useful data sources, as I'm sure consultants at Deloitte have, it can produce really useful components in seconds. You can build UI and UX components. You can build data visualizations. You can summarize things, any type of thing in any way that you would want to. Create automation scripts based on a database schema. Like, you know, there's so many different things that you can do with Claude because of how it's designed to generate artifacts, particularly if you're like connecting it to an MCP.

5:04So, you know, I think smart people with this power in their hands can do incredible things in an extremely short timeline these days. So like, first of all, kudos to Deloitte. Like, you know, I think you made the right decision picking Claude for your organization. Like it is probably the best tool out there for an organization like Deloitte. Yeah, totally agree. Yeah. But I like that you brought up the Claude Center of Excellence. Like in the article, it says it's staffed with experts tasked with developing internal implementation frameworks, sharing best practices and providing ongoing technical support.

5:40So like, in other words, they have AI consultants for their AI consultants now, which is kind of crazy. But Andrew, I'm wondering, like, you know, how long is this like we have our own internal AI consultant expertise? Like, how long is that going to be a marketable trait? Like, is that on a limited lifespan? Do you think? No, I don't think that's limited at all. I think we're actually just, that's why I'm paying such close attention to the Claude Center of Excellence because I think this is a model that you're going to see replicated across a lot of huge organizations and standardized organizations, also highly regulated ones, as they begin these same size rollouts.

6:17Also, I think you're going to see organizations and groups like this inside of governments relatively soon as they start operationalizing around how they and their knowledge workers are using AI. And I think that a lot of this is going to come down to a continued practice of learning and development. And also probably to, you know, to everyone's misfortune, probably a very robust certification ecosystem from all model providers and the folks that are using them at scale. I'm looking at like some of the largest tech companies in our industry use this model. You know, if you have a platform and tooling that hundreds of thousands or millions of engineers are using at scale every day and on top of all of your documentation and resources, you also probably have certifications and training and levels of excellence that you can accredit to the people that are using your tools.

7:09These are the AWSs and sales forces of the world where, you know, you go on LinkedIn and you see folks in these ecosystems with, you know, 50x cert trained like in their LinkedIn title. I think we're actually going to see a lot of this around AI skilling and specialization, especially around applying and using it at scale. So I think that this is just the beginning of a trend, in my own opinion. Yeah, that's actually a really great point. You know, I'm just waiting for like that man behind the curtain moment, though. Like eventually we're going to find out one of these organizations that are setting up their own like AI center of excellence.

7:43It's just going to be a bunch of AI agents behind it and secretly, you know, and it will eventually be. Hey, AI is great at telling you how to use AI. I mean, once again, it's like if you're not using AI to write your AI prompts, then you're already missing out on a flywheel. So it makes sense to me. Yeah, I actually think I just discovered for the first time this week documentation that was written for AI, not for me. So let's move on to our next story then. And this one's kind of a continuation of a discussion we started last week. They kind of got a lot of attention on LinkedIn about politics and software engineering.

8:15And I love this because it's almost the exact opposite of the discussion we had last week. And it's practical advice from someone on how they influence politics at their tech company. So what do we have here, Andrew? Like you just aptly summed up, well, last week we covered Matthias Lima's article about don't avoid workplace politics, giving advice to engineers about why they need to participate in the influence, you know, network at their jobs and both in a soft and hard way by being contributors, by also showing up for conversations where they need to weigh in and give their best advice. And this stirred up a lot of conversation on the Hacker News, got a lot of attention and in fact influenced in a response article from Sean Godecki.

8:55He's a writer whose articles we've covered here on DI a few times, And he was prompted to write this after reading that article. And his opinion, you know, is that software engineers, they're often fatalistic about company politics. They think they can't influence or change anything, or it's pointless. And they're just kind of along for the ride. And that kind of nihilism and cynicism can drive a lot of negative work around work politics. So he really breaks down how stakeholders, organization leaders view and work with software engineers to achieve their goals, to help better educate an engineering reader about how the mind of their manager counterpart might even work.

9:38And he also gives some practical advice on how to actually navigate that power network with your own passion projects. And some of the advice he gives is, I would say, starts to actually veer into a little more of a cynicism itself around how engineers work and typically view their work inside of an org. Something think he called out, for example, is that if you want to make a high profile project successful, then one way you could do this is to with like a pet project, making it available for what he labeled as a political campaign within your company, even calling it as much as the flavor of the week for what engineers want to get done.

10:17So if you want to migrate the billing code or rip out a build pipeline, or rewrite like a Python service into Golang, or, you know, he gives these examples of what he labels pet projects for engineers, then he gives advice on how to dress that up for the flavor of the month that your executives are worried about. That way that you can go after what they're trying to achieve while still achieving your pet project. And, you know, he's not wrong that org politics, they shape those technical outcomes, but his framing does make it sound like the only rational move is to become like a savvy opportunist rather than like a principal planner with your innovation.

10:55And ultimately, you know, those politics are inescapable. But his framing does assume that engineers are kind of tools within the political game and not agents in their own rights. I would really challenge engineers in response to this article, you know, to understand how their work aligns with the impact of their organization. But, you know, Ben, what did you think after reading this article? Yeah, I'll push back a little bit on the notion that he's saying you have to be a savvy opportunist as much as I myself like to view myself as an opportunist. Because he does point out, there's one section in this article that the easiest way to actively work to improve your political position is just to help high profile projects be successful.

11:36And yeah, sure, sometimes it's hard to like, be on be chosen to be on a high profile project. And, you know, it can be difficult to accomplish great things. So it's, you know, the work to do good work at a high profile project is difficult. But from a political perspective, simply just putting your best foot forward and always doing what you can to push projects you are involved with in a positive direction, that is the simplest way to make positive political pressure on an organization. But beyond that, yeah, he does focus a lot on opportunism. I think it is really worth pointing out, particularly, I think this is a challenge a lot of engineers struggle with, is that you might have the best idea, but it may not be the right time within your organization for that idea.

12:22And really what you should be focused on is like saving that type of project or any project that you care about for when there's a top-down mandate that aligns with it that you can kind of just include it with. So yeah, you brought up some of the great examples that were in that. And, you know, one thing to keep in mind is the stuff always kind of comes in waves. Sometimes it's cyclical. You know, the longer you're with an organization or the longer you've been around organizations of different types, you can sort of learn how to sense these things. And then, you know, at the end of the day, when it comes to opportunism, luck is kind of a controversial topic as well.

12:54You know, sometimes it just strikes completely at random. Sometimes you have to work to actively make it. But the way I've always kind of approached that luck, you know, to generate an opportunity is, you know, really luck is only like 10 % of the equation. Like you could be the luckiest person in the world, but if you're not ready to take advantage of that luck when it happens, then none of it matters anyway. So, you know, put in the 80 % and when you can, and just be ready for that moment where you were luck may strike or opportunity may strike that you can, you know, apply additional political pressure beyond just doing the right thing.

13:28I really do like that he's breaking it down for engineers just to help them understand like this doesn't, you know, because of the discussions I got on LinkedIn around this, there was a lot of mixed feelings. Like some people felt very strongly with that previous article on, you know, getting involved with politics and other people just felt like there's no way they could do it. And I don't think that's really true. I think everyone can do something to positively influence their organization politically. Yeah. I think that goes back to the fatalism viewpoint that Sean rightly called out in the beginning that that is, you know, a lot of the response.

13:59So I'm interested to see what the third article in the saga will be, you know, maybe next week that we cover because it does seem to be a heated and divided topic. I also like that you called out luck. I think that's really self-aware, but also ties back to an article we covered recently from Charity Majors where she acknowledged her own luck as an engineering manager and how it got her to the places and the opportunities she had. And she called out the same thing that you did. You know, luck is part of the equation. It's also about showing up and building every day. Yeah. Yeah. So let's move on to this next story, which I've been salivating to cover.

14:30related to Coinbase. And how much of the code is written by AI? So what do we have here, Andrew? Yeah, so there was an article we covered, or rather a tweet we covered from the Coinbase CEO, Brian Armstrong, earlier this year, bragging about how 40 % of the code at Coinbase was AI generated. And so we recently actually scooped this podcast interview from Syntax Podcast. And this is with Kyle Sessmet of Coinbase talking about the types of code at Coinbase that are generated by AI. And what he talked about is how this code is mostly tests, particularly in TypeScript around various parts of the infra there at Coinbase.

15:09And so, you know, Ben, I think that's right nail on the head of your prediction here. So I'm going to hand this over to you because I know you probably have a lot to say about this one. Yeah, for our dedicated listeners, if you remember back, I made some guesses about what Brian Armstrong actually meant when he said 40 % of their code was written by AI. And I love this interview because this is exactly what in that episode, what I said we need more of. Less CEOs with big numbers that they just tweet about and more engineers on the ground telling us exactly what they're doing with AI. So it turns out I'm right about a few key things about how Coinbase is using AI.

15:46First is that most of the code that AI is writing, as you mentioned, is for testing. It's not for product innovation. It's handling toilsome test work that they actually weren't doing before. To that point, Coinbase also has extra requirements that AI written code must be fully tested. So this creates this feedback loop where if you're using AI to write code, you're required to write more code because you're required to do additional testing that your non-AI peers don't have to do. And then the third thing that I was right about is they don't set goals around the number of lines of code that are changed by AI.

16:22They only suggest that as a best practice, just around the concept of, you know, keeping your code simple and your pull requests small. I really, I listened to a bunch of sections in this, in this podcast is really nice to just get like on the ground reporting firsthand of how Coinbase is actually using AI within their organization. So yeah. You absolutely called this one too. When we, when we covered this originally, this was exactly what you said that you thought it would be. And then we are learning that that is what it is. And And not even necessarily in a bad light, you know, we're just calling it what it is.

16:55It's code that you rightly have labeled as it didn't exist, didn't need to exist beforehand. It's great that they're doing that now. They probably should have always been doing that. But the fact of the matter is, is that it creates all of this extra framework around actually having to do it. So that just as a clear reminder to anybody that's adopting AI coding tools within their code base is to understand how it does impact all of that downstream code. and to understand what it really means when you have more code. Is that good or bad? Was it code that needed to exist yesterday? It's important to be aware of that stuff.

17:28Yeah, and it's one of those social media effects. If you see someone on Instagram that seems to be having a better life than you, you're just seeing one version of them. You're not necessarily seeing all the realities that are behind those pictures and videos. The same goes with executives tweeting about how much code is written by AI at their company. Don't judge yourself based on that. But if we got it all wrong, and maybe you know more than us at Coinbase, we'd love to talk with you about it. So please, reach out to us. Absolutely. All right, I think we have one more story, and it's about the Internet Archive.

18:01What's going on here, Andrew? Yeah, so this month, the Internet Archive, you know, that very diligently archives all copies of all pages on the Internet that it can get its hands on, is celebrating one trillion web pages preserved and available for access this month on the Wayback Machine. And, you know, the Internet Archive has been around since 1996. It's a crucial part of the Internet and how we can access documentation, things from before. It's basically the archaeological record of the digital age. And so celebrating One Tree and WebPages Archive is a huge celebration. And they have a month-long lineup of events, both virtual and in person, as well as lots of opportunities to get involved in the preservation efforts.

18:44We just wanted to call that out here. So if that's of any interest to you, please definitely go to the Internet Archive and check out their event lineup, which we'll also include below. You can continue to pay it forward so the next generation of Internet users can continue to benefit from the same kind of preservation effort. Yeah. Are there any other websites out there from the 90s that we still use today? I have to wonder. From the 90s? Yeah. I mean, are you still not logging in to feed your Neopets? I think that says a lot about you, Ben, than anything. Back when Flash was everywhere. Yeah, I mean, it's extraordinary to see this organization last so long.

19:23And it's a cause that I deeply believe in. I think information preservation is an often overlooked thing. It's often why I have difficulties with the current state of Western intellectual property law, because a lot of information just disappears from the world long before there's the monetary incentive to preserve it. So yeah, support the Internet Archive. They do a wonderful service for the internet for all of us, and we should help them back. So, Andrew, who's our guest this week? This week, we're sitting down with Rams Raharsha, the CTO of Pinecone. So stick around for this discussion. Are you tired of slow, inconsistent code reviews?

20:02Meet LinearBee AI, your new AI-powered workflow assistant built to supercharge your team's review process. With LinearBee AI, you'll get automatic PR descriptions, AI-generated review suggestions, and instant insights that help your developers fix issues before a human even looks at the code. No more waiting, no more guesswork. Just faster, smarter, and higher quality reviews powered by AI. Visit LinearBee.io to bring AI into your code review process today. Hello and welcome back to Dev Interrupted. I'm your host, Andrew Ziegler, and today we're sitting down with Ram Srirharsha, the CTO at Pinecone.

20:43Prior to joining Pinecone, he was the vice president of engineering at Splunk, where he began as a senior principal scientist. And before that, he was a product and engineering lead at Databricks for their unified analytics platform for genomics. Ram, welcome to The Dev Interrupted. We're so excited to have you here. Thanks, Andrew. It's great to be here. We're here at ELC, the Engineering Leadership Conference here in San Francisco, and you gave a talk today about vector databases. Why are vector databases still important? That's a great question. In fact, my talk goes into exactly this topic.

21:15The reason I wanted to give the talk in the first place was, first of all, there's a conference where people are talking about AI. Agents are a hot thing. It's an emerging technology that people are starting to get familiar with. Through all this, people may not realize that at the core of AI is search. And at the core of how AI does search is vectors. So I wanted to give a talk to frame the problem from that perspective and to kind of maybe educate the audience a little bit that if you take this to the logical conclusion and think about how do you now scale search, you have to have vector search and vector databases at the core of those.

21:49And it's a new way of motivating vector databases for a new audience. I think people who are familiar with recommendation systems, people who have done information retrieval, they kind of get it. But people who are working on agents, people who are trying to build generative AI applications and so on, may not necessarily realize at first glance that, oh, the thing that underpins AI, emerging AI, is actually search. So I just wanted to give a new framing to that problem. And so would you say that it's like framing a building block that folks should be using and getting them educated about how they're used to build higher level software?

22:22Yes. You know, vector databases are really important in the AI space. And I really want to understand more from your perspective of why their prevailing importance in search will probably never go away. and understanding the role that the vector database plays within the engineering world for people that are building agents or working with AI tools. And most importantly, what are the ways that engineering leaders can think about those kinds of tools, those primitives, those catalysts for change and use them within your own organization? What kind of perspectives do you hear from leaders? Yeah, that's a great question.

23:01So first of all, the reason why vector databases don't go away is that If you're building agents, you need knowledge. And you want to be able to externalize this knowledge. You want to be able to audit it. You want to be able to redact it. You want to make sure it's secure. And the moment you start thinking about that, you need vector databases, and it becomes a foundational component. And now it becomes the question of how do you take this foundational vector database, foundational language models, and put them together to build a stack for your agent. What I see people often do is they start simple.

23:36There are good recipes out there today called RAG, Retrieval Augmented Generation. There's a lot of knowledge by ourselves. We have lots of blogs. We have lots of templates of how to build RAG systems. We even have what we call an assistant, which allows you to just drop unstructured data and ask questions. So we tend to try to lower the barrier to entry to this kind of building agents as much as possible. So that's where I would start. If I'm an engineering leader who's trying to empower a team to build agentic workflows, I would start with something like an assistant, start consuming it, start building knowledgeable agents, and then slowly go deeper into the stack as you kind of gain more expertise and also start putting more guardrains.

24:25So when you work within that iterative space, it's about just starting and then measuring what works and what doesn't. What else matters as an engineering leader? How can they equip their engineers to really be thinking about unlocking more value? It's a great question. I think to me, starting simple matters. Putting evaluation frameworks matter because LLMs hallucinate. No retrieval is perfect. You kind of need to know how to tweak or tune your retrieval steps. You kind of need to know what to do about hallucination with language models. You need to know when your agent is actually returning the right relevant results.