The Architecture of the Internet with Erik Seidel

0:00The modern internet is a vast web of independent networks bound together by billions of routing decisions made every second. It's an architecture so reliable we mostly take it for granted, but behind the scenes, it represents one of humanity's greatest engineering achievements. Today's internet is also dramatically more complex and capable than in its early years. Eric Seidel is a network engineer at Cloudflare, where he focuses on automating global network infrastructure. He joins the show to discuss his unique journey into tech, the fundamentals of how the internet works, the border gateway protocol, peering versus transit, Cloudflare's architecture, networking in China, and much more.

0:45Gregor Vand is a security-focused technologist, having previously been a CTO across cybersecurity, cyber insurance, and general software engineering companies. He is based in Singapore and can be found via his profile at van.hk or on LinkedIn.

1:15Hello and welcome to Software Engineering Daily. My guest today is Eric Seidel. So welcome, Eric. Hello, nice to be here. Yeah, great to have you here, Eric. We've met very briefly a couple of years ago. I say back in Singapore. I'm still in Singapore. You're now in Austin. You were then and still are working for Cloudflare as a network engineer. And we're going to be hearing all about how Cloudflare works, as well as just kind of general principles of how the internet works today, which is going to be very interesting. But as per usual, we'd love to just hear a little bit about your background before sort of leading up to Cloudflare.

1:52I gather you have spent a lot of time in Asia, but also like China specifically, which is obviously very interesting from an internet perspective. So yeah, can you just give us a brief history? I think like a lot of people in the tech industry, and I found I'm not unique as this, I have a rather non, I'm what you might call a non-traditional, non-standard background, very varied background. I came into the tech industry a little later. It's not my first go at tech or understanding tech or working with tech. I mean, I've been doing tech-related jobs since I was a student worker back in university in the late 90s, early 2000s.

2:31And then I kind of drifted off away from like computing and internet altogether and towards a lot of language learning. I was classic students, Latin, Greek. I mean, I was training to be a classics teacher at one point, actually. And then I ended up going to China for a while, teaching English there and learning the Chinese language. Spent a while there learning Chinese because I just really enjoyed a whole thing of learning languages. And then after a few years in China, I kind of drifted back towards tech. And within a year of coming back to America, I found in March 2020, about less than a year after I came back to America, I found myself at Cloudflare.

3:11And I've been working at Cloudflare ever since. I first kind of started as a customer support engineer at Cloudflare, very much networking focused, helping customers with their networking related issues and how to use Cloudflare networking related services, in particular, our Magic Transit product. And then from there, I kind of migrated into engineering as a network engineer. And since then, I've actually become a systems engineer on the networking team where I focus more full-time on just automating networks or automating our network, I should say. And I mean, so through that time in China, were you sort of like keeping up with tech or doing any kind of network related things?

3:52That's how I developed my interest in networking. Actually, I started doing a lot of networking related things in China. I ended up building up my own ASN in China and running it with like public IP addresses and stuff while I was in China and things like that. That's really where I got into networking to begin with. Yeah, because I think most of the audience are probably aware, but the Great Chinese Firewall, so just networking in China when it comes especially to anything that needs to leave the country or come into the country from a network perspective is just a whole different kind of ballgame, right?

4:23Yeah, I think we might be getting to this later in the podcast, but yeah, networking in China works very different from the way it works in mainland China, I should say. works very differently from the rest of the world. And that like very much influences how we deliver our products in China versus like ROW, rest of the world, as we say. Got it. Okay. Yeah. And as you say, yeah, we will get to that in more detail later on. Awesome. So let's start kind of with the fundamentals. So kind of we might term it sort of just internet fundamentals and sort of where Cloudflare fits into all this. So yeah, where do we start with this?

5:00I mean, at the most basic level, the internet is just a big collection of networks. We've got tons of different networks that make them up. I've made previous reference to ASN, Autonomous System Number. The network is divided into what are called autonomous systems, which are like kind of freestanding independent networks in their own right, identified by a unique number called ASN. And they all connect to each other. They all peer with each other. and all together in aggregate, that makes up the internet. Now, once you get a little lower though, what you have is you have different tiers of networks, things like that.

5:35At the very top, you have kind of what are like tier one providers. These are very big networks like Telia as an example, GTT as an example, Nippon Telephone and Telegraph, NTT in Japan is a big example. And they are what we call transit provider networks. And they have networks that though they might have certain focus on certain regions, like NTT might be more APAC-focused, GTT, I think, might be more EMEA-focused, Europe-focused, that area. Cogent might be more like North America-focused. Initially, they run networks that span the globe, big networks that span the globe. And they engage and they're called tier ones because they are networks that can reach the entire internet without having to pay any money.

6:22They basically engage in settlement-free peering with each other. It's like their mutual benefit. Like, yeah, we all connect to each other, exchange our customers' data with each other. And by connecting to each other and exchanging our customers' data with each other, that's the internet. And in addition, you have all sorts of other networks. You have smaller regional ISPs that connect to one or more tier one transit providers. They might have downstream customers of them too. Like their customers connect to them that then connect to like the transit providers. Then you have tech companies like Cloudflare is an example of one.

6:55Google, AWS are examples of others where big content provider networks. We connect to tier one networks too for transit. And in addition, we do lots of interconnections with all sorts of other networks on the internet to like broaden our connection base and reduce the latency. It just reduced the general cost and increased the efficiency of delivery to customers. And so, you know, we've talked about tier one. And then if we, I guess, look at edge routers, edge routing, like they, I believe, use the BGP routing protocol. Could you maybe just, what even is that? And how does that sort of then fit into things?

7:30So I find the best way to explain BGP, like with a lot of things, is to like start with the problem that it solves. Like what's the problem that it solves? You mentioned edge routers. Edge routers, like we have all these networks. These networks have the concept of an edge and a core. The core of the network is like it's all their network, like right deep in the bowels of their network. If you're an AT &T customer in America like I am, I'm deep in the bowels of their network. Like my local internet connection connects to something deep in their core. The edge or the border, we call it the border as well, the edge of the network or the border, that's where it's actually connecting to other networks, where traffic is leaving and entering their network from other networks.

8:11And the routers that do that, we call them edge routers. Now, when we have that, we have these networks connecting, like their edge routers are connecting to routers on other networks. They have to basically provide a roadmap to the other routers. We all know the internet is made up of IP addresses and IP prefixes, even like the most basic examples, 192.168.1.0 slash 24. That's, of course, a private IP network, but it works. Public IP networks are numbered and work the same way in that respect. With their public networks, they have all these public networks, like your public IP address, like my public AT &T IP address.

8:50It is part of an AT &T prefix. Now, how do other networks know that I'm an AT &T? How do they know? Like, yeah, I have to send my traffic to AT &T, not only AT &T, but maybe it's best to send it to this AT &T router in this region. How do they know that? That's where BGP comes in. BGP, it stands for Border Gateway Protocol. It runs on these edge routers. And what basically do is you establish BGP, these edge routers, like AT &T edge routers, CloudFlare edge routers, whatever they might be, if they have a direct connection to each other, like if the AT &T router has, say, a direct connection to a CloudFlare edge router, like a direct point-to-point fiber optic Ethernet circuit, then these two networks will run a BGP session over that point-to-point connection, and they will exchange their prefixes, like in the CloudFlare case.

9:45These are our prefixes that you can reach here to the AT &T router. And the AT &T will say, these are all of our prefixes that you can reach these prefixes through me through that same session. And that kind of provides the roadmap. And it's made for all of these point-to-points. We generally have one with the exception of IXP sessions. IXP is where we have multiple sessions. Each point-to-point, it's one BGP session. And that's what BGP basically is. It's building the roadmap of the internet. It's telling everyone where the IP addresses are. Got it. And this might be quite a basic question for some of our listeners, but not for me anyway.

10:23How often do these change? Because if this protocol is all about exchanging the roadmap, yeah, how often do they change? Do they change? You have to look at it in terms of individual sessions versus aggregate. Individual sessions, maybe not much. All in all, aggregate, it's changing all the time. The internet is whole. Like the IPv4 internet right now has, I think, over a million unique prefixes. The IPv6 internet has, I think, about 200 ,000 unique prefixes. That's changing all the time in aggregate. And it also can depend how far up the chain you are. If you're like a small regional network, maybe you're a customer of, say, you're like a small office network maybe, and you're a customer of AT &T and you've got a BGP session with AT &T.

11:06Or not even AT &T, just your regional ISP. Your session with your regional ISP might not change that much. It might be very stable. Once you get to your regional ISP, the sessions with all of their customers will be very stable, but some of their customers may be changing, which means their sessions with their upstreams might be a little less stable, a little more dynamic, because every time one of their customers makes a change, they make a change. It gets propagated to their upstream, and their upstream sessions make a change. Once you get to the tier ones, like Telia and AT &T is a tier one, two, Telia, GTT, NTT, all these, they're having a lot of changes.

11:46It's very dynamic because anytime, not just one of their customers, anytime a customer of a customer of a customer of a customer makes a change, that'll get propagated up to one of their sessions. So they've got sessions where like, yeah, there's lots of changes. It can be very dynamic. So it really depends where you are on the internet. The closer you get to the tier one, the heart of the internet, if you will, and what we call the default free zone, the DFC, it gets more dynamic. So many questions I could be asking, but I know we're always going to run out of time if I ask everything. But one sidebar question here is tier one, AT &T sounds like the main US player.

12:22Does that mean that they're just sitting on a massive asset that at some point someone else could come and buy? Could another tier one be created in some way? The basic standard of tier one, from at least the way I understand it, and I think that's the standard way, is that tier one is a network that it can reach anywhere on the internet without having to pay another network. Okay. That's a good definition. Yeah, that's basically what it is. And AT &T can reach, from what I understand, if they're still tier one, I understand they are, they can reach any other part of the internet without paying.

12:53And the reason they can do that is because they have such a large network, such a large customer base. And I'm not really the finances person or the business decisions person, but it's reached the point where like, yeah, other networks, they've all made the business decision that this is a network that it's worth doing settlement free peering with. Okay. Well, that kind of actually leads us pretty nicely then into peering versus transit. And you did touch on it briefly earlier, but yeah, let's talk about that. And obviously we're going to get into what that means from a basic financial standpoint as well, I guess.

13:26So we have our edge routers. They're connected with lots of other networks with fiber optic ethernet and those BGP sessions I mentioned. A lot of them are peers. We do settlement-free peering with them. What they do is we send them our prefixes. They send us their prefixes and maybe the prefixes of their direct customers, which means we can use that connection to reach like their network and other networks that are their customers maybe, right? What we cannot do through such a session, what if there's like some other network out there that's not their customer at all? It's a customer of some other network.

14:02We can't reach it through them. Like they're only going to provide us with the reachability to their network and their customer networks? If we've got, say we've got some other network way out there and we're not directly connected to them and that other network is not a customer of any of our peer networks, how do we reach them? That's where transit comes in. Basically, when you're buying a transit session, buying a transit service, you are buying a connection to the entire internet. You pay them and they give you what we call a full table or a full view of the internet. They won't just send you like maybe a thousand prefixes of just like their network and their customers.

14:39They will send you a million IPv4 prefixes containing like every other network on the internet. They will send you all 200 plus thousand IPv6 prefixes of the whole internet. And they've given you that full view of the network's internet so you can reach any other network on the internet through them if you so choose. And I guess in Cloudflare's context, this is sort of table stakes because Cloudflare to provide, we can maybe do a very brief history of Cloudflare in a second, but the product provided initially and then a massive suite of products now, Cloudflare has to assume it can know where to go across the entire internet or to be able to provide its services effectively.

15:21Yeah, we have to make sure all of our data centers' basic requirements. We have to have a fully functional connection to the internet that has a full view of the internet. Yeah. Tired of babysitting autoscalers and overspending on cloud costs? Meet Thoris, the platform that makes engineers heroes to their finance and business leaders. Thoris intelligently manages Kubernetes clusters, automatically right-sizing and scaling workloads while preventing downtime from traffic spikes. It anticipates usage and capacity needs, so systems stay fast, reliable, and efficient, without constant tuning. Teams using Thoris cut cloud spend by 40-60%.

16:00Thoris predicts compute and GPU demand before it happens, keeping performance smooth and costs in check. Stop wasting compute and guessing your resource needs. Let Thoris handle your autoscaling, so your teams can focus on building. Find out how much you can save with Thoris. visit thoris.ai and try our cloud savings calculator. So let's move on to the global scale, the architecture of Cloudflare. I mean, I definitely didn't prime you to be a history expert on Cloudflare today, but maybe you could just from any basic information, where did Cloudflare start? Where is it kind of now? Maybe sort of from a product perspective or just product suite, I guess.

16:40And then we can talk more about like how that's actually then manifested at a technical level through architecture. Sure. I'll focus on the part of Cloudflare I know the best, based on not only my networking team, but previously in customer support. The aspect I'm most familiar with for Cloudflare, I think a lot, is the security aspect. We provide a lot of security services. And I guess you could say the classic way we do it, which is kind of the OG way of doing it, Cloudflare-wise, is we provide like the CDN edge where basically we sit between our customers' origin servers, like their web servers, our edge servers are between them.

17:17And their customers will not directly connect to their web servers. They'll connect to our edge servers. And then we might provide services like caching, like we'll cache some of that for them. We provide a lot of security services for them to make sure like, yeah, if there's a DDoS attack, not only do we strive to absorb it, we strive to like filter it as well. As much bad traffic as possible, drop it as much good traffic as possible, make sure the customer gets or make sure it's served by cash or whatever means. That's the OG, like I said, from there. I worked a lot in customer support on a product called Magic Transit, which is kind of expanding that philosophy to layer three, which is basically networks like with their own ASNs, like I mentioned.

17:59They can put themselves behind our metals where we basically establish using PNI or using GRE tunnels or some other means to like connect from our edge servers to their edge routers. and we sit in front of them. And before any of their traffic, layer three traffic reaches them, it goes through our metals first and our metals filter it and then send it to them. That's another product. And that's kind of where I came to Cloudflare. Yeah, where we are very much, at least from the perspective I see in Cloudflare, we are very much a security oriented company providing security services to our customers in addition to CDN services and even like edge compute services as well.

18:40And speaking of edge compute, in addition to our workers' product, we're moving more and more into the AI realm as well. Yeah. So let's talk about then the sort of pure infrastructure that's required to drive this. I believe just a couple of numbers off the top of the charts, if you like, as over, I mean, this was what, two years ago, at least, was over 300 global points of presence. And I think it'd be interesting to understand what those are in over 100 countries, 44 million HTTP requests a second on average. These are just sort of numbers to help the audience maybe just get a vague sense of scale here.

19:12And again, this is two years ago, so it's only probably gone up since then. We've gotten bigger. I'm sorry, I can't provide you with the exact numbers. No, no, it's fine. We're getting bigger. It gets bigger all the time. Yeah. So let's talk about point of presence. What does that even kind of mean and what are they handling? So that goes back to that edge networking concept. So we have these networks, they want to connect to each other, right? Where do they connect to each other? The point of presence. It's like a data center someplace where like, yeah, we've got routers there. We've got infrastructure there.

19:41The other network has routers there. They have infrastructure there. We can connect to each other there. And it's not just case of points of presence. Like with us, it's not just edge routers as well. Like we'll have our fleet, we'll have some of our servers as well there. The thing about our service, the way we've got them configured is we like to have them configured generally to run the whole Cloudflare stack. Basically any of our Cloudflare services should be able to handle any of our customers' needs and services. Got it. So how does this then like flow through to data center, I guess, design, if you like?

20:13I believe it's sort of the concept idea of traditional versus multi-colo POP or MCP, which is a different MCP to what maybe a lot of our audience are used to hearing about. So yeah, so let's talk about those. Yeah. So the way I like to conceptualize it, to understand how we approach it, you kind of get an idea of the internet topology itself. is like, just with like human topology, you've got areas where population density is very low, relatively low, like rural areas. And then you have areas more urban where population density is very high. The internet interconnectivity on the edge is kind of like that as well.

20:49You have areas, like Austin is an example, even though it's a big urban area, like where there's not as much interconnectivity happening, right? Not many networks connect to each other, peer with each other in Austin, where a lot of the peering, a lot of the internet connectivity happens, closest to me, is the DFW, the Dallas-Fort Worth area. There you have like lots of networks coming together at lots of data centers in that area, all connecting to each other and peering with each other. Now, the internet almost works the same with the city metaphor. If you've all these networks connecting to each other, guess what?

21:24You've got lots of companies, including us, including all sorts of other companies. They want to park their servers and infrastructure there. They want to park their infrastructure close to where the connections are happening because the closer you are to where you're connecting to other networks, the quicker it is to get your data onto the customer network and to the customer. Like if you're some area where it's like you're far away from where interconnectivity is happening, then you end up at the core of your provider network. You have to go through your provider network before you reach the customer's network.

21:54Whereas at the edge, like, Yeah, it's easier to just immediately start going to the customer network. And that's why data centers and points of presence will follow those contours. So like DFW, lots of data centers, lots of interconnectivity. We have a big point of presence there with lots of servers there. One of our biggest IAD, Ashburn in Northern Virginia, that's another huge area where there's just tons of network interconnectivity, which leads to tons of data centers, tons of infrastructure, everyone parked where all the networks are meeting each other to get to the customers quicker. So, I mean, I seem to remember this from a talk you gave a couple of years ago.

22:34This was a bit of an eye-opening moment for me to sort of understand, I think, why we were in Singapore at that point in time. And I still am. And Singapore is a listed region, for example, when you go to AWS or GCP. and i think i was always wondering why all the regions effectively kind of the same across different products and across different providers they can't always offer all the regions if you want to call it that but but a lot of them do still follow like you know southeast one is singapore and southeast one is is the same in gcp aws etc and this is kind of i guess what you're explaining here which is same building pretty much just where all the providers kind of have there same building or if not same building close enough that you can get like a metropolitan network connectivity in the form of dark fiber or like waves connections relatively inexpensively yeah and it does still require agreements between everyone to kind of say like i'll connect to you you connect to me and that kind of all helps yeah it's still based on that basic system and it comes together like where this happens it's most efficient if there are certain points on a map where it all happens.

23:42Yeah, I guess if we then bring back in the edge router aspect to this, which is that's edge routers are just so I'm getting this clear, like not the opposite of a data center, but they are the bits that do sit outside of these sort of mass transit areas. Is that correct? Well, I mean, they're the ones that are actually doing the connectivity. So we try to get them close to the most convenient places to connect them to as many other networks as possible. Okay. And just in terms of redundancy, we did actually have an episode a couple months ago, just talking about subsea cables and just how, you know, when they get cut and like, what can cutter damaged.

24:18And we were focusing a bit more on how they actually get repaired. We had someone on from one of the big news outlets who'd gone and been on one of these boats and just seen what work is needed to actually repair these fiber optic cables. But either those or any other kind of problems, I guess, because we're still talking about a lot of cabling for most to this, how do you factor in redundancy and how to even think about that? Yeah, I mean, you pointed out a tricky question. Subsea cables can be tricky because they carry a ton of capacity. There's only so many of them. And when they get knocked out, it can take a long time sometimes to fix them because they're under the ocean.

24:57You have to go out there with boats and things like that to fix them. So we do run a global backbone network. And the thing about it It does span the globe. It circumscribes the entire globe. Basically, we try to make sure we have a lot of diversity. We have backbone links going through multiple different sub-C cable networks. And then we have lots of transit providers as well, too. Like if we lose some backbone capacity because of a sub-C cable deal problem, we can move to transit. Where the trickiness comes in is sometimes with some of these bigger sub-C cables, not only does it impact us, not only does it take us out, take out some of our capacity.