Doug conducts analysis of events and trends across the global Internet for Kentik (previously Oracle Internet Intel, Dyn Research and Renesys).Follow Doug on LinkedIn
I joined the Internet Society as a Technical Engagement Manager in November 2016. Currently, I serve as the Senior Manager, Internet Technology - Asia-Pacific and as part of the MANRS initiative. MANRS (Mutually Agreed Norms for Routing Security) is a global, community-driven initiative encouraging network service providers, Internet exchange points, and cloud and CDN operators to implement basic network security measures to secure the global routing system. I'm based in Sydney, Australia. Prior to joining the Internet Society, I worked with a system integrator based in Australia as a chief technologist. I have a vast experience in the service provider sector, as I have spent almost a decade at Cybernet (one of the largest ISP in Pakistan) leading the Network Operations and Projects team. I have been regularly presenting on various technology events like SANOG, APNIC/APRICOT, MENOG and other national NOGs (Network Operator Groups). Currently, I serve as the chair of Routing Security SIG at APNIC.https://www.internetsociety.org/author/siddiqui/
Phil Gervasi: Many people might be surprised, may be unpleasantly surprised that routing on the global internet has for many years been built pretty much on trust relationships. In years past and to an extent today, there was very little to protect the entire internet community from malicious people spoofing IP address spaces and entire autonomous systems, and there was very little governance protecting the security of routing among providers and all manner of other organizations with an internet presence. Relationships that the rest of us actually using the internet every day rely on without even realizing it. MANRS or the mutually agreed norms for routing security was started specifically to set forth security principles and guidelines for routing security on the global internet. With us today is Aftab Siddiqui with the Internet Society and a key person in the MANRS Initiative, and we'll be discussing MANRS and what he and his organization are doing to protect internet routing for all of us around the world. My name is Philip Gervasi and this is Telemetry Now. Aftab, thanks so much for joining today. I've been looking forward to this for a long time, especially after doing so much research. I have a lot of questions for you. Looking forward to picking your brain on some things. But before we get into that, would you mind giving our audience just a little bit of background of yourself, your experience in the networking industry, and of course the MANRS Initiative?
Aftab Siddiqui: Right. Thank you, Phil. Thank you for the invite. Yes. I'm Aftab Siddiqui working for Internal Society. I have spent close to 15 years in this industry, spent most of my time in the service provider sector, ISPs, running routing protocols most of my life, and then looking at how they are behaving on the global routing table. Then I end up with Internet Society and it's been more than five years now, and I have been working mostly on the MANRS Initiative and Initiative, which is supported by Internet Society since the very beginning, and I've been part of this initiative since then.
Phil Gervasi: Great, thank you. And I would like to get into what the Internet Society's all about a little bit more, but first I'd also like to introduce our co- host for the day, Doug Madori, no stranger to Telemetry now and I'm sure to our audience as well. Doug, welcome and if you wouldn't mind introducing yourself to our audience.
Doug Madory: Yeah, sure. Yes, I'm Doug Madory, I'm the director of internet analysis for inaudible and I have been doing BGP analysis in one way or another for the last 12 and a half years or something, when I started at a company called Renaissance. And I started in that job doing data QA, just finding errors and stuff in our products that were based on BGP. I got into the subject, felt like I could find a lot of novel, interesting things and the rest is history. I've been doing that as a main part of my job every day for over a decade. And so, some of the issues in BGP are routing security. That's probably one of the biggest issues that people grapple with in this space. And so, I've had, since MANRS began a number of years ago, I've always had overlapping interests with that group and the folks involved, starting with Andre inaudible. And there'd be a lot of back channel discussions of if we're seeing something, we would have a conversation or something we can help them with. We've always had some common interests and tried to be supportive of each other, ourselves as BGP analysts and subject matter experts and then their effort at trying to advocate for routing hygiene, is what we like to call it. Just best practices in B2B routing.
Phil Gervasi: Great, thanks Doug. And so, to get us started, Aftab, would you give us a explanation of what the Internet Society is all about?
Aftab Siddiqui: Yeah, why not? Well, Internet Society is a not- for- profit organization based in Virginia, in US. It's a 501 C3 organization. But its main purpose is... I mean, if you go into the vision, it's a broader vision, which is internet is for everyone. But what they have been doing for more than 25 years is to make sure that the internet is available to everyone and it remains open, and the statement we use is globally connected, secure and trustworthy. Secure part is where the MANRS comes in. We have multiple other initiatives which is related to supporting the internet exchange point development, making sure the technical community most importantly is very well- connected with each other. Sustainability of these technical communities. Making sure that, as I said, the globally connected to provide community networks where there is no service and internet connectivity, help them with the connectivity as well. And of course, making sure that the policy work is in line with our mission and vision. Provide the policy support in different aspects as well. In general, this is what Internet Society has been doing for more than 25 years.
Phil Gervasi: Is MANRS an initiative of the Internet Society or is it something that's more broad?
Aftab Siddiqui: It is an initiative. I would call it an initiative supported by the Internet Society because if you look at the history of the MANRS, as Doug mentioned a moment ago, that it started early in 2014. It wasn't called MANRS at that time, it was called Routing Resiliency. It was more of a, " Okay, fine, let's sit together and discuss how routing is important, what are the issues in the routing and how can we all resolve it together?" it was more of a round table or what we call it these days, a fireside chat, which started with some of the industry leaders and who were very much entrusted in the routing security at that time. And then it started rolling from there and then it became an initiative which ISOC, Internet society said we will support them wholeheartedly. The support is mostly comes from the staff member as myself, and other staff member who makes sure that there is a consistent support available to the community. But of course it is run and managed by most of the community members. For example, if I can quickly explain the structure of the MANRS, we do have a steering committee. A steering committee has elected members from the community and they have multiple tenures, and we have done two elections so far. And we have a chair, who's Warrick Mitchell based out of Port Australia. We have a co- chair who is at the University of Washington as well, Andrew Gallo. They are the people who make the strategic decisions about the MANRS. Of course, the operational and secretariat work is done by the staff, including myself, but we have made sure that the overall strategy is always made by the community themselves, rather than Internet Society.
Phil Gervasi: Routing security is certainly a poignant name, but MANRS? I mean, that really works. I get it. That's great. But I want to ask why is it routing security, global routing security that is your main concern? What is the specific problem that you've identified that you're trying to solve?
Aftab Siddiqui: That's interesting. Okay. As we were discussing this morning as I'm attending Apricot in Philippines, Manila, and this is what we are discussing, that wire writing security. And the point is, and I raised this point quite often, that whenever we talk about cybersecurity and whenever you read about a cybersecurity paper or anything, most of the time they are trying to address anything above layer three. Nobody talks about what is happening on the underlying layer. And that is the biggest problem at the moment that you need to send packet from one point to another, you need to transfer your data from one point to another. That requires security as well. Why nobody's concerned about that. And we have seen in the past that so many incidents have happened in the past and it is actually the baseline of how the data is transferred from one place to another. Same as the case for the physical security. I mean, people put armed guards in front of the data center, but then they are not putting any security in the routing layer then, well, there is a problem somewhere. Yeah, that's why we believe that it is very important to, for at least someone to pay attention to that. There's a lot of people who do pay attention. What we are doing is making sure that we gather them all around so we make sure that everybody is equally entrusted in that.
Phil Gervasi: And from a technical perspective, there really is very little that we can do to prevent an individual or an organization from advertising routes and prefixes out into the global routing table. Are we trying to prevent, now, security vulnerabilities that will ultimately lead to sections of the global internet being taken down or maybe even rerouting of traffic for malicious purposes? What are we actually trying to prevent from happening here?
Aftab Siddiqui: We have seen all three of them, all three of the categories which you have mentioned, and we see them on, at times, daily basis. I mean, I do not remember any single day when you do not see a single event not happening at all. Yes, we can always debate whether they were malicious or not. In my opinion, they're in many cases, these are just mistakes. Sometimes just honest mistakes people do make on the internet. Well, because the problem is... The best thing about the internet is of course it's a network of networks. It means if one network make a mistake, it'll be propagated quite quickly. That is one problem. But of course when there's so many incidents happening, we cannot just close our eyes and say, " Everything is just a mistake." Of course there are bad actors on the internet running some networks and they do want to exploit these mistakes people make. And that's why we have seen some bad actors trying to hijack some Ether wallet. It was back in, I think 2018 or 19 when this happened. And then there was a recent case in South Korea where when another crypto exchange was attacked in the similar manner. And then rerouting the traffic is also very common. Again, it can happen because of a mistake, but we have seen that some state actors and malicious entities have rerouted traffic to jeopardize the traffic, plus make sure that there is an outage in some part of the world or in some countries. It's everywhere. It's hard to say, it's hard to pinpoint that it was malicious or not, but it is happening, so we have to treat it as a problem to resolve.
Phil Gervasi: Okay, interesting. It's not just about security in the classic sense, like bad guys trying to do bad things, but it's also about routing hygiene. Or in other words, how people do routing on this interrelated global internet, that thing that we have. And ultimately, propagating best practices among engineers and organizations to do routing correctly.
Aftab Siddiqui: Yes, it is. The most important thing we are trying to convince every operator, number one, manages technology agnostic. We are not hell- bent on one form of technology, " Oh, you have to do this only." We are saying we would like to achieve this target. No matter how you want do it, that's fine. There are best practices to achieve these things, let's follow the best practices. Best practices designed by the people who have been operating networks for decades. Best practices which are very well understood by the ITF as part of the BCPS and the RFCs. Follow those to make sure that the reliability and resiliency in the internet remains as it is.
Phil Gervasi: And that makes a lot of sense considering that the internet, the global internet is the mechanism that we use to deliver services and applications all over the world. And a lot of those applications and services are mission- critical. I mean, there's governments relying on it. Healthcare, our businesses, educational systems. Now, what you mentioned though is that you are seeing issues occur almost every single day. I have to imagine that there is some sense of urgency in what you and what your organization does.
Aftab Siddiqui: Yes, absolutely. I mean, it's a famous thing that mostly we are concerned about things which are newsworthy, but it doesn't mean that it is not happening. And again, another point we mentioned most of the time is that just because it did not impact you doesn't mean it did not happen. If it is happening in one part of the world, most likely you will not be impacted on the other side of the world. Or if it, is, for example, if something happened in Australia where I'm residing, I'm pretty sure half of the US will be sleeping at that time. If something happens, yes, the impact will be very small, but of course we need the network reliability is more important at that time now, that we have to make sure that something, if something happens, it is addressed properly. Yes, things are happening, we are not aware of them on daily basis.
Phil Gervasi: We did discuss very briefly that pretty much anyone can advertise whatever network they want into the global routing table. What is MANRS specifically addressing with regard to routing, security routing, hygiene? I have to imagine that the ability to advertise whatever you want is one of those things, right?
Aftab Siddiqui: Yes, absolutely. It is one of them. Let me go through the actions, what we call it in MANRS Initiative. For example, the first action is that you make sure that you only announce what you should be announcing only. Nothing less, nothing more. Of course you can do less, but please do not announce anything more than that. And based on certain rules and regulation, for example, what you can announce, we are talking about IP address resources, so IPV 4 or IPV 6 addresses. And you only announce that because you have been delegated from your respective RIR, which is regional internet registry. If your RIR has allocated certain resources to you, you can only announce that. Make sure that you do that. Plus, if you are a service provider, you have downstream customers. It's your responsibility to make sure that what you are receiving from your downstream customers is also authentic. It's also verifiable before you send it to the internet, so that your whole customer cone is your responsibility. This is the most important action, what we believe is the BGP filtering. Do not announce anything you are not supposed to announce. Yes, as you mentioned, this is one of the biggest issue that, and in the past it was just open to everything, you'll announce everything, which is possible you can. But now in today's world, it is much more important to make sure that the right filters are in place for any announcement. Now, moving into the second action, which is anti spoofing, where networks can change the source address, and when the traffic goes out on the internet, of course they will not come back to the same network because someone has changed the source address. It will go to the specific source address. That creates amplification attacks. And more than 25 years ago, iF I'm not mistaken, the BCP 38 was released. And since then, everybody is trying to convince the operators to implement that, but it's still not there yet. It's far away because of course there are some technical limitations in the operator side, but of course that's not been implemented yet.
Phil Gervasi: If I could just interrupt you, can you quickly explain what BCP 38 is?
Aftab Siddiqui: Yes. BCP 38 is, for example, as I mentioned, it is the source address validation. For example, you have a customer which is using certain IP addresses. It could be anything. Just for the example, I'm giving a private address example. It is for example, 192, 168.00/ 24, right? It's a slash 24 address and they are using it in their network. Well, it is your downstream customer. And when they send traffic, whenever you receive traffic, you are always entrusted in destination because you need to send it towards the destination. Enormous circumstances. You do not look at what from where it is coming in from because it's coming from your customer. You just believe in that, but you do not look at that what is the source address of that? In routing, it's always the destination. If you do not look at the source address, what will happen if I change the source address? Of course you will send the traffic out, but it'll not come back because the source address has changed. It'll go to the address where it is pointing it to now, which is the new source address. And if 100 networks do the same thing, you create an amplification attract towards one destination. Every source address belongs to, for example, network A. And every response coming back will go to network A, and then they will receive a distributed denial of service attack, which is the DDoS. Not implementing source address validation is one of the causes of the DDoS attacks. Not the only one, of course, but one of the causes, yes.
Doug Madory: But in a nutshell, BCP 38 is just saying you shouldn't be sending out traffic with a source address or return address that's not destined for the customer or where you're getting that traffic from, inaudible that. But yeah, you should be checking the source address on outbound traffic to make sure that the person who's sending it actually can receive it. Otherwise, you create this scenario where you can launch amplification of attacks. And this is something that gets talked about quite a bit in the space and it's the source of a lot of DDoS attacks.
Phil Gervasi: And honestly, these two actions that we've been discussing thus far, route filtering and then the anti spoofing mechanisms, they probably prevent a lot of the issues that could potentially happen with global routing. What else is MANRS doing specifically to address this issue?
Aftab Siddiqui: And it goes directly into our action number three, which is the coordination part, which is the human part. And that is really very important. And what we have seen in the past that, again, as we have said so many times, whether malicious or not, things do happen. Incidents do happen. If something is happening, for example, how do I reach out to the network operators? How do I reach out to the person who's responsible for that network which is generating a problem, which is generating an attack, which is generating my resources from their network? Which we call it misorigination or the normal term some people use is hijack is. How do we contact them? And this is where the action three is most important, where we say, " Well make sure that your network is contactable. Make sure that there's enough information available which is publicly verifiable that anyone n can contact you." I mean, you don't have to put your personal numbers on the internet, but of course your network, you should be reachable via email, an email which belongs to a team, which is monitoring it. A phone number in case just in case there is a knock phone number or service or help desk, phone number is there. We request all the operators and it's mandatory in actions that you have to have an email address available through the RIR who is databases and through peering DB, which is, again, peering DB is one of the most important, I would call it a portal for network operators across the globe, which they use to make sure that the relevant information is there. We encourage everybody to, every operator, whoever joins us, if even they don't join us, it's okay. Please do the best thing. Yes, this is the human factor embedded in the action three.
Phil Gervasi: The people part makes a lot of sense to me. Those open lines of communication are going to do a lot, I have to imagine, to securing those relationships between providers and other providers, providers and customers on the global internet. In fact, it reminds me of that example of when you get a phishing email and you see that it's from somebody perhaps an executive at the company, all you might need to do is call up that executive if you can and ask, " Did you send this email?" And then verify again, just an open line of communication to help secure your network infrastructure. Now, that aside, I know that there are some somewhat technical mechanisms that we can use to secure our global routing such as RPKI and ROA. Is MANRS involved with that?
Aftab Siddiqui: Absolutely. Well, let me share you some insight. As I mentioned, the best thing about MANRS is... Of course I'm associated with it, so of course I will call it's the best thing. But MANRS is technology agnostic. When RPKI is still in the works at IETF, the global validation part was still there. How we were encouraging operators to do the global validation. Well, through the historical system, now we can call it is the IR, which is the internet routing registry. We are encouraging everyone to make sure that they create route objects and make sure that information is up- to- date so that everybody else can verify this information. That was going so far, of course. And if anyone who's listening and do understand IRR, then they can... I mean, they must be smiling right now that how bad the IRR ecosystem is. Yes, but that was the only solution at that time. Fast forward, little couple of more years since 2014. RPKI started taking really good shape. The relying party software were getting better, RIRs were better in terms of providing services. The vendors were getting better in terms of providing the support for the RPKI. Things were getting really, really good. Then at that time it was of course being technology- agnostic, it was added that now it is much more important to have RPKI support in the global validation as well. We now encourage every operator who has already joined or who is applying to join as a participant that do create ROA as well, which is route original authorization, which is an equivalent of the route object, but of course it is verifiable through a PKI infrastructure, and it is much more secure. You do not have to update it on daily basis. Of course, it's a renewable certificate, it'll be updated automatically. These are the things which helps us secure it much better. And of course, we do support RPKI in the best possible manner. And just FYI, I did a routing security tutorial just today at Apricot, and the focus of that managed tutorial was only RPKI. We are pushing our RPKI to the fullest. Yes.
Doug Madory: Here are some of the advantages. The RPKI, when we talk about RPI, we're really talking about RPI, IROV, route origin validation is one particular application that sits on the RI infrastructure. The hope is there'll be future applications like ASPA and PGP sec, which we won't get into probably here, but those will also rely on RPKI. It's been a, I think synonymous where we use RPKI to stand in for the one application that we've got built, which is RV. But just in a nutshell, the advantages is that, especially as somebody as an analyst who's trying to understand what's going on in internet routing, I think I have probably agree with this, that at least RPI, ROV, there is one correct answer internet wide, that there's one ground truth. It may not cover every AS or every route, but because not everybody's made rows for all their routes, but it is a reliable way to validate something. In contrast to the previous IRR based solutions, you had a numerous sources of information and some of them were of different levels of quality. And in fact there are a couple that are of low level of security as we found out in recent years, where bad actors have been able to input entries to whitelist their bad routes. And that's not something that's possible in the RPI OV. The other issue is that each network ultimately if they're going to do route filtering, is going to implement their own solution. And those solutions can differ depending on how their design choices, engineering choices. It becomes a little bit unpredictable as to what's expected of any network. You could have two different big networks both doing IRR based filtering and they could behave a little differently from one to the other. And that unpredictability and that's not great for the system. We don't have that with the RPI OV. It is a uniform thing. And yeah, the fact that we've had a lot of adoption in the last couple years is a really great step forward on this topic.
Phil Gervasi: Oh yeah, absolutely. I completely agree with you, Doug. I mean, the fact is we are making incremental steps forward to secure global routing. It's an iterative process like everything else. And it's very interesting to me that these four actions that we've been talking about, to me, they don't seem standalone whatsoever. They really all just seem to be so intertwined that it's like they're four components of one big action. I mean, routing, of course filtering has to do with routing. There's people involved with these relationships and with the configurations and all these processes. They all go together in my mind. Now, Aftab, you mentioned several keyword words here that I want to call out. You said the word join, you said the word member, you said the word encourage. What do you mean by that? What does it mean to join MANRS? What does it mean to be a member? And when you say encourage, I have to assume that you are not spelling out explicit configuration, but encouraging best practices and more outcomes and then encouraging folks to do what they need to do to get to that specific outcome. And just to piggyback on that, what is a member? What does it mean to be a member of MANRS?
Aftab Siddiqui: Well, yes, these are really very important words. For example, one thing which is we call the networks who are part of the MANRS initiative, we call them participants, because they are participant in an initiative. They are not joining an exclusive club, of course. We call don't them members, we call them participant in this initiative. That is slightly unique thing in the MANRS Initiative. Number two is we do have an implementation guide. We do have implementation guide and it has snippets of configuration from different vendors. We have it from Cisco, from Jennifer, from Arista, from Nokia, from Microtech. It's all crowdsourced. It is done by the community and it resides in a publicly available GitHub repository, and anybody can go and update it and send a poll request, and we review it every quarter that if we have received any request or not. Yes, we do have an implementation guide. We do provide the guideline, but our focus is on the outcome, as you mentioned. We would like you to do, we would like you to achieve this. If you want to achieve it with a different set of configurations, that's up to you. We are not forcing you to do this exactly. Implementation guide is just a guide. If you are not familiar with this, you can simply copy paste and it'll work. But if you know your platform, if you know your business requirement, a technical requirement better than anybody else, of course, then you should implement the way you are running your network. But we provide the guideline, but of course you do it yourself. We are more concerned about the outcome.
Phil Gervasi: A participant isn't required to use your configuration examples. I mean, I get it. They're encouraged to do so because they are best practices. But ultimately there isn't like an audit or anything, is there?
Aftab Siddiqui: No, there is no specific configuration audit. The audit we do is for the outcome again. Once someone apply to join, we look at all the global publicly available data sources and see if they have done something wrong in last six to three months time period, thankfully. Now we do have a data source since 2019, since we have started collecting the data from all the public sources, we can go back up to 2019 easily. But our focus usually is that in the last three to six months, what have they done? If they have done something wrong, we go back to them and say, " Well, can you explain why this happened and what measures you have taken since then to rectify this problem?" And things like that. The audit usually is from the outside, not from the inside.
Phil Gervasi: MANRS is a technical community that service providers, well really any organization with a public internet presence can join and then undergo an audit from your folks that look at their peering and routing and advertising activity over the last few years to determine if they are following best practices, and therefore trustworthy with regard to global routing. Insofar as any organization is going to make a mistake here and there because we are people. And then that organization can in turn go to the rest of the world and say, " We are a participant of MANRS and therefore are trustworthy."
Aftab Siddiqui: 101%, yes, we do believe-
Phil Gervasi: Oh, 101%. I'm I'm flattered.
Aftab Siddiqui: Yes, we do believe... I mean, that one person was for the last statement where you said, " Well, technology's going to break." Yes, that is the most important point. Things will go bad no matter how good your network is, and we have seen it several times. It is the most important bit is how quickly you identify the problem and you resolve it. And initially we used to say, " Well, you have to make sure that what you are doing on the internet," but now we also recommend network operators that please make sure that you have some visibility of your network from outside as well. Use some tools which can give you what your network is doing from the outside perspective rather than just from the inside. Please have some eyes and ears looking at the routing table or using some tools which can help you provide this information.
Phil Gervasi: Oh yeah, it's very clear to me that MANRS is certainly concerned with the technical but is also very much concerned with the people part of this entire thing. People, processes, workflows, team culture of a network operations team, and I guess a network security team as well. Now, does MANRS provide any internet resources, any actual infrastructure to help their participants in securing their networks and then securing their global routing footprint?
Aftab Siddiqui: Of course, we do rely on the infrastructure support from the Internet Society, which is provided by them. The infrastructure is provided by Internet Society. We do have our training labs. One is in Sydney, another one is in Zurich, Switzerland. And the labs we provide to our participant just for the training purposes, if they have any gaps, if they want to test something, of course we cannot replicate the network of anybody else, but we do have live internet feed, we do have peering so they can test. I mean, if they're going to break it, they're going to break up our lab infrastructure. They are more than happy to do that. We do provide that support just to give them some level of confidence that if you do, this will be the outcome. They can test try, we provide the training, we provide tutorials, live demos to help them understand better before they go back and implement it into their own networks.
Phil Gervasi: I mean, a lot of this conversation has felt like it lent itself, it was oriented towards service providers. But I know from my own experience as an engineer, I've worked with some enterprise organizations, so not service providers, but that we're so large global and with so many locations and so many end users that it really felt like we were running a service provider network in a lot of ways. Would you say that MANRS is really more oriented toward the service provider space or are you also trying to accommodate folks in the enterprise world as well?
Aftab Siddiqui: Well, initially it was we target the network operators or the ISPs, which you call it. But because our point of view was, and to some extent still is that if you secure the bigger transits, you secure the blast radius, you reduce the blast radius. That was our initial target. Make sure the top 10 or top hundred or whatever numbers we can come up with, make sure that we can target them first, bring them on board and make sure the most of the trans provider across the globe are part of this initiative. And we have, to be very honest, we are very successful in that. We do have some of the largest operators joining, they have already joined us, and some are already in discussion for quite some time to join us as well. That is going really very well. But your point is absolutely correct, some of the enterprises are... Some of them have more routers and more nodes in their network than some of the large network operators. Yes, that is important. To have them on board is more important and that's why we are renewing our efforts toward the enterprises. We are trying to reach out to them. Of course, what happens is most of the enterprises do not come to these network operators group events or other events. Now we are trying to reach out to them through different associations. We are partnering with FS ISAC or HS ISAC for the health industry and the financial industry and working with Isaka to make sure that other cybersecurity folks who are from the enterprise sector do know that what we are doing, what we are trying to achieve, so that they can participate as well. Yes, we are trying to reach out to them, but yes, the network is growing, the internet is growing pretty fast.
Phil Gervasi: Yeah, that's an understatement. Aftab and Doug, where would you gentlemen say that we are in the adoption of these technologies in the greater community? And I'm specifically thinking ROA and RPKI.
Aftab Siddiqui: I mean, you summed it so nicely. And I was just looking at the data while you were talking about it and I just looked at it. For example, Facebook, Google and Netflix. And even Amazon, if you just look at the ROA uptake of these organization, it is close to 100 percent.
Doug Madory: Yeah, I guess I have a glass half full perspective on this, given that this is trying to improve global routing security is a really difficult thing. You have to have thousands of entities around the world all do something. And also a lot of times the motivation for them to implement RPI RV to reject invalids, they're doing things that are going to benefit others. And so, that's even a harder sell sometimes for them to allocate resources to just... That's not going to necessarily benefit them directly or at least initially. But I guess there's been a lot of progress made in the ROV world in the last couple of years. And so, I wrote a couple of blog posts around that I did with Joe Snyder's. Fastly looking at... There's two steps to RPI ROV doing its job. Step one is are there inaudible created? Basically people who have created some sort of attestation that says that this is the correct state of the routes. By the numbers, they're very low. It's maybe a third of the routes in the global routing table have rows that are valid. The analysis that we did at combining with some of the net flow from Kensig was that we could show that even if those were just a third of the routes, there's a lot of important companies that are doing this. And so, it ends up being over half of the traffic because not every route pushes the same amount of traffic. And so, that was like a hopeful message that we maybe are further along than we thought if we're just counting routes, because not every route pushes the same amount of traffic. And then the flip side is if a route becomes invalid, is it getting filtered? And so, then we can take at any given moment there's a bunch of persistent misconfigured routes that are RPI invalid and therefore getting filtered and we can use those to study what is the effect of filtering at the moment? And again, you have a lot of benefit created by the global backbone carriers, inaudible, Acogent, inaudible who are filtering invalid routes, really for the benefit of the rest of the internet. They don't necessarily benefit themselves. But I wish to appreciate the effort and engineering that went into this. But because of those backbone providers are dropping invalids, that means that when there's a routing leak a big origination leak, any of those routes that had rowers will be protected to some extent. And to use the phrase the Aftab used a minute ago about reducing the blast radius, the propagation of those bad routes gets dramatically reduced. That's present day. It's not a solved problem, but we do have to take a moment and appreciate that we've come a long way, but there's plenty still to do. Aftab, do you have anything to add on that?
Aftab Siddiqui: Yeah, it has changed a lot in last couple of years or maybe I would say in a few months it has changed a lot. The only problem is I see while looking at the data is we are missing out on a lot of financial sector. If they just jump on this one, well, we have most of the problems are resolved, because most of the incident we see, which with mostly are of malicious intent are towards the financial institutions, but unfortunately they are not looking at it. That is the only missing point. Otherwise, the big operators network operators are implementing the route region validation and the eyeball networks are implementing, are creating the ROAs. It is a really good match, but it's just the player in the middle, which are the health sector and the financial sector. Unfortunately, they are not looking at it at the moment in a nice manner, I would say.
Doug Madory: These are the companies that push most traffic. That's what creates that. If you look just bits per second, where are they going? They're going to rouse with valid rows. That's a good thing because that means that there's a potential for the RPK system to protect the traffic that's going there. We didn't have that just a couple years ago, so this is a relatively new phenomenon.
Phil Gervasi: Gentlemen, I'm going to stop us here. We're getting close to an hour and this has been a thoroughly enjoyable conversation. Aftab, thank you so much for joining. It's been really interesting to me as a former engineer from the technical side, of course, but also to learn about how the people side of securing global routing is just so important and a focus for MANRS and that the organization exists in the first place to address this problem. Again, Aftab, thank you so much for joining.
Aftab Siddiqui: My pleasure. Thank you for the invite. Thank you. Thank you, Doug.
Phil Gervasi: Great. And so, Aftab, if folks would like to reach out to you online to ask a question or maybe if they have a comment, how can they do that?
Aftab Siddiqui: Well, I am mostly available on Twitter, so they can reach out to me @ aftabsiddiqui, or you can find my details on the Internet Society website, which is siddiqui@ isoc. org. You can send me an email if you want to reach out.
Phil Gervasi: Great, thanks. And Doug, always a pleasure. How can folks reach out to you online, again with a question or a comment?
Doug Madory: I am still on Twitter and LinkedIn. Those are good ways to reach me. I also have an account on Mastodon. Just look up Doug Madory. I don't have any clever Twitter handle, just my name.
Phil Gervasi: Great. Thanks, Doug. And you can find me on Twitter @ Network_Phil. I'm still active there. You can search my name on LinkedIn, Philip Gervasi. I'm all over the internet, and my blog networkphil. com. And if you are interested in being a guest on the show or if you have an idea for an episode, please reach out to us. We'd love to hear from you. You can email us at telemetrynow @ inaudible. com. Until next time, thanks for listening. Bye- bye.
Do you dread forgetting to use the “add” command on a trunk port? Do you grit your teeth when the coffee maker isn't working, and everyone says, “It’s the network’s fault?” Do you like to blame DNS for everything because you know deep down, in the bottom of your heart, it probably is DNS?
Well, you're in the right place! Telemetry Now is the podcast for you!
Tune in and let the packets wash over you as host Phil Gervasi and his expert guests talk networking, network engineering and related careers, emerging technologies, and more.