Security at Coinbase with Philip Martin

0:00A cryptocurrency exchange is a digital platform that allows users to buy, sell, and trade cryptocurrencies. These exchanges face unique security challenges that require specialized threat assessments and planning. Coinbase is a U.S.-based cryptocurrency exchange that was founded in 2012 and has evolved alongside cryptocurrency as a technology. Philip Martin is the Chief Security Officer at Coinbase. Prior to Coinbase, Philip built and led the incident response and security engineering teams at Palantir and was a U.S. Army counterintelligence agent and Arabic linguist. In this episode, Philip joins the podcast with Gregor Vand to talk about his career and security at Coinbase.

0:42Gregor Vand is a security-focused technologist and is the founder and CTO of MailPass. Previously, Gregor was a CTO across cybersecurity, cyber insurance, and general software engineering companies. He has been based in Asia-Pacific for almost a decade and can be found via his profile at Vand.hk.

1:14Hi, Philip. Welcome to Software Engineering Daily. Hey, Greer. It's great to be here. Yeah, Philip, thank you so much for joining us today. You are the Chief Security Officer at Coinbase. So we're going to be hearing all about financial security, obviously around cryptocurrencies. currencies. First of all, we're going to be hearing a bit more about what you did before Coinbase. So what is your path to Coinbase and how did you get into this industry at all? Sure. That goes way back, really. I knew I wanted to be a security practitioner when I was still in high school. So I taught myself to code in my parents' proverbial basement.

1:50My parents don't have a basement, didn't have a basement. But I taught myself to code in high school. This was back in the 90s and started doing web design for local companies. I had a good time, taught myself C, Perl, JavaScript, et cetera. And then ended up going to San Jose State for a bit for computer science, dropped out because it was incredibly boring, and joined a startup at the time that was building, it's called Cobalt Networks, that was building Linux-based appliances for small to medium-sized businesses, large work groups within larger business, things like that. Really pretty ahead of its time, but got to do some pretty cool stuff there working on IPSec and other features of that device.

2:36And from there, we got acquired by Sun Microsystems at the time, which was a behemoth, right? It was, I don't even know, call it 50 ,000 people globally at the time. And got to do some really interesting work around the Linux kernel, getting some of our Sun's hardware working with Linux, which at the time was unheard of, and got really bored of that, quite frankly. These huge behemoth organizations, it's stereotypical going to meetings about meetings to then hold the meeting about the issue, as opposed to just moving ahead and fixing something. And so I left Sun and made the obvious next step of going into the military where I focused on – and really, this was more about being a little bit burned out on computers and software engineering than anything else.

3:23But when I went in, I was like, okay, what do you want to do? And I was like, well, what has as little to do with computers as possible that also isn't the infantry? And they're like, this kind of intelligence thing. You should do that. It's all about people. And that's totally true. It is all about people. And it taught me a lot about how to really interact with people, how to work with other human beings who are either like or not like myself. Great experience. Got to see a bunch of cool missions, do a bunch of cool things. And the process really rekindled my love for security, for secular. So I left the military and then went to work for Amazon, which is like fascinating technology challenges at Amazon.

4:01For me, the mission didn't really resonate. I wasn't super excited about what we were doing in the world. So I left Amazon, went to Palantir, where a friend of mine who I'd served with was working at the time and absolutely loved it. The mission was there. The technology challenges were there. It was a small 300 % or so company at the time. So lots of agility and ability to sort of move outside of my defined box. And then my boss left. And I wasn't really excited about... I didn't have like, oh, here's my next step within Palantir that was really exciting to me. And so I had met some of the folks at Coinbase previously because they were working on some really fascinating, I'm sure some of the challenges we'll end up talking about in this session.

4:47At the time, I didn't know much about cryptocurrency. Some of the other folks on my team at Palantir had gotten into mining really early. At the time, I told them something to the effect of that pretend internet money is not really going to go anywhere. And I regret that decision quite a bit, obviously. I was aware of it broadly. I hadn't really ever considered the security challenges inherent in a cryptocurrency or in running an exchange or custodian or what a fundamental shift it was in an asset and how one protects assets. But as I started to learn more and talk to the folks, I started to get really, really intrigued about both how critical security was and is to Coinbase.

5:29It truly is the one existential threat I think the company has faced since the very first day it started, as well as how much work there was to build new things in furtherance of that goal. Because we protect, I don't remember what the last quarterly report number was, but hundreds of billions of dollars in cryptocurrency. But underneath that, there are hundreds and hundreds of millions of private keys to be managed. And there are insane insider threat risks, right? When you can move money digitally in this way, irrevocably, right? The human element of security becomes incredibly interesting and very, very difficult to control for.

6:11The amount of money adversaries are willing to spend attacking us really is just proportional to the assets we have on platform. And so there's a very direct monetizable piece of the company at risk there. So we see attackers who are willing to spend a lot of effort and time and money and focus attacking, much as Coinbase, but really everybody in the cryptocurrency ecosystem across the entire chain, all the way from the end user to the exchange, to the custodian, to the software infrastructure that's supporting those things. So software supply chain attacks are fascinating in crypto. They actually happen outside of nation state sponsored hacking activities.

6:49So that was just like, for me, it was just catnip. And I've been at Coinbase, it'll be nine years in April. And really, honestly, people ask, if you stayed at one company that long, Coinbase has not been at the same company the entire time, right? It's gone through a number of evolutions as any organization like this would. But at the end of the day, For me, it's been that consistent presence of significant security engineering challenges in an environment where failure really matters has been a recipe for something that I just cannot get enough of. Yeah, that's awesome. I mean, I like the kind of theme, I guess, just throughout when you did actually get bored, you kind of made a switch.

7:35And I think a lot of people don't maybe do that enough. I mean, it's not to say jump around, but it is sort of, I like that you completely did a hard left, you know, and you went from technology into intelligence and then brought them back together. And obviously, as you call out, you've been at Coinbase now nine years, and clearly there's still an intellectual challenge there for you, which is great to see. So, I mean, you've kind of touched on it there, but again, just in case any listeners are not aware, just a sort of very, very brief, what is Coinbase? That's a fascinating question. So for the vast majority of people out there, you experience Coinbase as one of two things.

8:10You experience Coinbase as Coinbase.com, which is a retail brokerage. That's what we would call that, right? Similar to your pickup brokerage, your Schwab account or Fidelity or E-Trade or whatever, where we provide the access and tools to buy, sell, trade, store, transmit cryptocurrency. Now, there's more than that. There's also staking. There's a bunch of other stuff. We'll just keep it simple to start with there. So it's either that or their customer Coinbase Wallet, which is our self-custody mobile app that allows consumers to do all those same things with cryptocurrency, but without using Coinbase as an intermediary.

8:52You can go transfer, trade on things like DEXs and similar things. You can do all that, but not having to depend on a third party to store and facilitate those transactions. Got it. I don't have a huge relationship to crypto. And it probably is because I got there very early and used a different exchange that fell over very fast. And that was the end of that. So I think what's going to be a theme throughout today is the fact that Coinbase is such a established and I think that runs through a lot of the, if you go to coinbase.com, etc. It's all about we are the most secure, the most established exchange.

9:28Yeah. Now I should note, we're way more than that, right? We do a bunch of stuff on the institutional side or on brokerage. We have a qualified custodian. We're into derivatives trading. There's a bunch of other pieces to the puzzle. But for the vast majority of users, that's sort of their relationship with us. So now we sort of understand, at least at a very high level, what the Coinbase is. You came from, I mean, ultimately you came from tech and then sort of what we might call traditional intelligence and then now into crypto security. What mental models or frameworks did you have to actually rethink when you came from, I would say, traditional intelligence into Coinbase?

10:07I'm not sure if it's rethinking so much as putting the proper context, right? Perhaps that's a distinction without a difference. But I think that I actually took quite a lot from how the government, for example, thinks about designing systems that protect classified information to how we think about building systems that protect cryptocurrency. From a system design perspective, there's a lot similar in the way that I think about those two things because the problem space is actually almost the same. right? When you think about what's the problem space for crypto, it's, well, I have all this value that at its core is tied to a, not a short string of digits, but not a huge one either, right?

10:54Something that if you could, you know, if you could memorize 20 digits at a time, you could smuggle a key out of a place. If you could see it, memorize it, do it piece by piece, whatever, right? Not dissimilar from the classified data problem, right? Of what you're actually trying to protect is something that can be in someone's brain. When we think about protecting cryptocurrency in that way, we think a lot about, not to jump around analogies here, but I will anyway, we think a lot about that just sort of like radioactive material. How do you safely manipulate radioactive material? Well, you don't.

11:25You build tools to manipulate it. So it can be at arm's reach and you can be protected from it. You're not exposed to it or exposed as little as possible. It's a very, very similar philosophy that we would take, or we do take in thinking about how we design systems that actually those core, core systems that touch private keys is in a very, very similar way. I also think that from, again, from that design perspective, very firmly embedded in our design process is that people shouldn't be trusted individually, right? That we want to see systems, system design, queer, it really does require a conspiracy for things to go intentionally wrong.

12:06Because conspiracies among humans can be fragile things. We want to introduce that additional risk into a bad act we'd have to do in order to cause bad things to happen. So lots of stuff like that, right? Where there, I think, are actually a huge number of parallels in how we think about secure systems design. Yeah, makes a lot of sense. I mean, as when you were talking about your time in the military and you were saying, you know, it was all about humans and at the end of the day, security basically is humans. It just so happens that there's this sort of interface that we all use called a computer and internet, et cetera, but it's ultimately humans and what kind of human figure out versus another one.

12:40I think that's very, very true. And I think it's, it's an element of security that you can't overemphasize. And I'm not talking, you know, when I see that some people think, Oh, security training or whatever, and sure, fine, right. Educating the human about, about the risks is a component of good security, but more about understanding how humans are going to interact with the system, right? More about the understanding that the vast majority of humans that use the system that you're building don't actually care about the security or insecurity of that system. They care about getting their work done for that day.

13:13And so that's their incentive. And instead of sitting here and trying to say, well, no, I'm going to make them care about security. I think people should sit there and say, how can I make the fastest path to that human getting what they want to get done done, the one that takes them through the most secure path? How can I make security the feature that they want in this system design? And I think we think about that a lot as we think about building security systems in Coinbase. And of course, there's a certain amount of fiat that you must use this or you must do that. You can never get away from that entirely.

13:49But I think at the same time, we spend a lot of time and effort thinking about what roads do we need to pave for our engineers, for our whoever it is, to get their work done in a way that is safe and secure, in a way that they are safe and secure, because they know that if they are, they're going to be faster and more efficient, more able to get their actual thing done that they care about. You could even call it some security humility. No one cares as much about system security as the security folks do. Let's just say that and be done with it and ask ourselves, okay, great, what do they care about and how can we position ourselves that we're delivering that to them in addition to security?

14:32Yeah, I completely agree. I work in effectively email security, but I have to keep pointing out that email security is, it doesn't technically matter that a bad email lands in your inbox. It matters that the human interacts with the bad email so to your point it's it's around like how can we always ensure that people are getting what they need to get done but gracefully avoiding the the bad stuff effectively absolutely that kind of brings us on to you know because that's effectively phishing so just talking about you know the kinds of attacks we might see at coinbase i mean what would you say the kind of classic question was like the most common kind of attack and and also just how is how have attack types changed over i don't know the last four to five years yeah i mean Coinbase gets attacked the way every other company on the internet gets attacked.

15:19We see the phishing. We see the web app attacks. We see all the same things that everyone else does. And I say a lot, people sometimes ask me, wow, Coinbase, the security must be so different. It's absolutely not. It's fundamentally, ignore a couple of things on the back end that I'll talk about in a second, but it's fundamentally web app security at a large corporate enterprise. So you get all the same social engineering, phishing, perimeter stuff, web app stuff. We have vendors. So vendor, third-party security, we have all the same attack types everyone else does. Now, where it gets interesting is where we get into the cryptocurrency-specific stuff.

15:56Because that creates a bunch of really interesting and unique security threat surface for us. everything from, hey, we're interacting with smart contracts. Or actually, in some cases, we're writing smart contracts. How do we write smart contracts in a way that is safe and secure, which is a really interesting set of problems? By occasion, it's sort of like, what if you're able to travel back in time and pick up a C programmer from 1970, fast forward to 2025 now, and ask them to write a secure application? They couldn't possibly, right? Because their whole classes of attacks have been invented that they would just have been unaware of, even as a state-of-the-art practitioner.

16:41Back then, a very, very similar problem in smart contract security is that whole classes of attacks are getting invented on a regular basis, right? As people really explore the boundaries of the security of the language, of how the compilers operate, of how the underlying network processes the byte code and the message types that are sent and all of this stuff together, right? It's changing and it's updating at a pace that is shocking. It shouldn't be shocking, not after nine years, right? But it still is sometimes to be shocking how quickly the space moves. And so we spend, we have a whole team, right?

17:19A blockchain security team that does nothing but work on the various pieces and parts of that from smart contract security to protocol security, secure protocol design to how we store process, private keys, the whole sort of ball of wax there that is really the unique bit of how Coinbase sees attacks. The other unique bit, as I talked about before, is just how much is at stake for Coinbase. And so while we certainly see our share of spray and pray phishing that went to us and a billion of our closest friends, we also see some very, very targeted attacks that attackers clearly spent time and effort and money executing, which is really awesome from the perspective of a security professional.

18:02There were a couple of, one I've heard of before, but one I hadn't. If I go to coinbase.com and there's obviously a lot on there about security because educating users is of huge importance. There is a term, a trust trading scam. What is that? So I say a lot, right? There are no new scams in the world. They're just sort of scams that have a different coat of paint on them. And so something like a trading scam is really sort of a confident scam where a bad actor is one way or another, and there are many sort of variations of this convincing a victim that they are going to teach them how to trade crypto or clue them in on some great investment opportunity or whatever.

18:43The pitch varies quite a bit. And getting that victim to, in some way, shape or form, transfer money to either the attacker or attacker-controlled address or wallet or something of that nature. And look, there are as many scams as there are scammers, probably more out there. And so I think the really important thing that we focus on when we talk about scams is sort of two prongs to that. The first prong is, it's about, to your point, educating customers and potential victims out there, getting the information in front of them. Because what we see on our end is that educated folks, meaning that folks who have heard about these scams before, who have some inkling of what the shape of them and what they might sound like are much less likely to be victims to those scams than people who are encountering them for the first time.

19:37That seems intuitive, but the data backs it, that is actually true. And so we want to get in front of as many eyeballs as we can. Now, the hard thing about that is that scan details change. And they change not just over time, but they change in reaction to our education efforts and the different controls we put in place. And so when we talk about this stuff, what I talk about tends to be in fairly generic terms, it tends to be a, I equate it a lot to what I might call real life security or security device people have heard growing up for ages. If it's too good to be true, it probably is. That applies just as much online as it does in real life.

20:17If you're being pressured into making a decision, if someone is pushing you to move faster than you're comfortable with, that's the moment when you take a beat, step back, ask yourself, talk to somebody. And the third thing is financial decisions should not be secrets. If someone is telling you, hey, I need you to do this thing, but don't tell anyone about it. Don't talk to your trusted loved one, your brother, your mother, whoever, don't tell them about it. They're not going to understand. That should be a red flag for folks. And these are very common. In fact, in most scams, we'll see some combination of one or more of these tactics, whether it be pressure, isolation, or what have you.

21:00And those should really be, I encourage folks to talk, not just educate themselves, but talk to their loved ones about this stuff and communicate these core concepts. One of the most fundamental things I talk about a lot is that it's almost independent. So almost anywhere I go to talk to people. People don't need to hear what I'm talking about from a consumer protection standpoint. The audience is probably like, I would bet your audience is in at least the top quartile of educated, educated at meaning aware of online scams folks out there, which is great. I'm very happy for that. But the folks that need to hear what I'm saying are not listening to software engineering daily.

21:44They're not reading the Coinbase blog. They're not attending, you know, Ripple's Swell conference in Miami. They are, you know, reading, they're, they're watching Good Morning America. They're, they're reading the AARP magazine. They are, um, they're in other venues and consuming media differently. And so two things, one, we do a lot of work to try to get out in those, in those media channels too. But, and this is, you know, if I can encourage your audience to do one thing walking out of this, one thing that will improve security for everybody, it is have a conversation with one person you know who you think might be at risk for being scammed online.

22:27You know somebody, I promise you, right? A cousin, an aunt, an uncle, a neighbor, a friend from a social group, whatever it is, there's one person that comes to mind for almost everybody. I'd worry about that person if a scammer called them. Go talk to them. We have a bunch of resources on Coinbase's blog. We've done some animated stuff on Coinbase's YouTube channel for sort of scam awareness. There are other resources that it doesn't have to be Coinbase, right? It's more important to me that the information gets out there than it's a Coinbase source for it. But that is really a rallying cry. I push everywhere I can, right?

23:01Is talk to the people in your life who might be at risk because you want it to be you that has that conversation with them first. Not a bad guy. Yeah, I think that's a great call out. I have sort of aging parents and I think they've managed to avoid most things, but they certainly have been targeted in the past. But luckily they do know to message me. Perfect. Anything that looks a bit suspicious, they just say, what about this? And I'm like, yeah, that's a scam as well. Even my wife got a very sophisticated phishing scam recently. And I was almost saying, no, I think it's probably correct and then we did some extra checking on it's like no that's that's also a scam so yeah i think it's a great call out just as you call out our audience here probably more aware of things than than the average person but we all know someone who's not so that's a great call out just want to kind of just talk about mechanisms for a second i mean one thing that sort of jumped out at me here in singapore is you know we have this thing called sing pass so that's a sort of government digital identity and it's used to log into you know if i log into my tax or just any other government kind of portal and then you know it can also be used to log into coinbase and i'm kind of curious around what that adds to something like coinbase and we've just got a little anecdote here which i'm curious if you've got sort of something that sort of resonates alongside this which is on my apartment door i've got this you know i didn't put it on there it's a very fancy lock it's got fingerprint it's got digits it's got all these things and one day the batteries run out and we didn't actually know how to jump start it there is a way but we we didn't know at the time so we call the locksmith and we expect he's going to come with some sort of special rewiring device or something and he comes with a giant metal bar and i was like no no no this is a it's a digital fingerprint you know and he's like yeah yeah yeah and he pops out the eye hole and he puts his metal bar and he opens the door from the inside which just for me this was just like an eye-opening moment, I thought, geez, this is security in a nutshell, right?

24:57We can put all these amazing mechanisms on the front. So just kind of going back to things like SingPass and obviously multi-factor, et cetera, et cetera, what kind of stands out as mechanisms that do work? And again, why something like what does SingPass, for example, bring to Coinbase? So as you mentioned up front, right, half of Coinbase's mission is be the most secure, be most trusted, as we say. The other half is be the easiest to use. And you could think about that as being two pieces of a mission that are in tension with each other. I don't think they are in tension, or perhaps they are occasionally, but they don't have to be.

25:30Really, it is a challenge to us to say it is unquestionable that we have to be the most trusted exchange, because if we are not, then the business dies. However, if we are so secure that no one can remember their password, no one can log into their account, then we have failed in our mission to actually be easy to use. and thus we have failed in our larger mission of encouraging the economic freedom in the world right so we have to balance it so things like sing pass or you know equivalent in the us would probably be closer to like logging with google or whatever we see that as making things a little bit easier for our customers while at the same time in our overall sort of security architecture design we don't outsource to a thing right so even if when you log in with your with your sing pass you still have to present whatever 2FA you have configured hopefully it's a yubi key if it's not i encourage you to use a yubi key we see that that is by far the safest method of two-factor that is out there but the the oh do you have a listeners know i i like passkeys i'm curious do you have any so leaning on passkeys versus yubi key the interesting thing for me about passkeys is that they're not like a passkey is not a passkey is not a passkey from a security design perspective right what's backing that passkey where it's stored right their implementations where that passkey the backing private key might be on a google drive right it might be on a password manager with a bad password we can't know it from point of perspective there's no way for us to and in fact i think that's actually a gap in the protocol that i know our team has been working with or working to address is what we would love to know is is this passkey hardware backed or not yeah i think that's that's a great call out if we could know that then we could have a lot more faith in that.

27:19And even if maybe there's a world here where we're big advocates of using risk models in situations like this. So this is not how it works today, but you could imagine a future world that says, well, hey, you're logging in using username pass, you're using a software-backed pass key or non-hardware-backed pass key, and your IP address is new. And maybe there's three your form or other factors, we're not going to let you log in like that. Whereas maybe if it was a hardware back passkey or a YubiKey or something, we'd say like, okay, you physically possess a thing that is letting you make this assertion.

27:56So we have a lot more confidence that you were you and not a bad actor who got access to some sort of digital repository where that passkey private key was stored. I think that's a really good call out. I mean, I've always appreciated passkeys from a sort of conceptual standpoint. And I was working with them at least two years ago and the thing is as things have kind of advanced yes okay things have got easier for users in theory but as you call out it's almost become too easy in some in some respects that you know i can log into my password manager with literally just a password and then suddenly that's my passkey and i don't think was really right ever supposed to be how things were i'm noticing that my my password manager is now saying they're going to start backstopping that with with my email address.

28:40And of course, this is a lot of what I work on, which is sort of backstopping accounts. Right. And then you get back down to the locksmith who pokes out the eye hole and uses the stick to unlock the door, right? Is everyone is depending on a different layer to be the ultimate backing authority. And then someone comes along and says like, well, that layer is vulnerable. And then the whole thing sort of collapses, right? I think this is really the hard part about modern secure system design is systems have gotten to the point, they're so complicated today. This is probably true in the past as well, but from my perspective, they're even more complicated today, that it's hard to even know that that eye hole exists in the door.