Internet Routing and Our Shared Responsibility

The internet is a place of global cooperation. Its operation, security and on-going maintenance is the responsibility of all network operators worldwide, no matter how big or small. Businesses and individuals depend on the internet to be always up, always performant and always secure.

Over the past few years, incidents involving routing security have become more prevalent, and the impact of these incidents can affect the entire internet. While these types of issues are often due to human error, there have also been malicious attacks designed to purposely disrupt internet services, or even capture users’ financial information.

Due in part to these well publicized outages, the Border Gateway Protocol (BGP) has come under fire for being inherently insecure. This does have some elements of truth as it was first designed in 1989 and did not have security in mind.

At its core BGP is a TCP application that is used to exchange network layer reachability information (NLRI) between autonomous systems. It was built to be reliable, scalable and flexible.

It is reliable in that it uses TCP to build sessions between BGP neighbors. It also includes its own reliability methods such as keepalive and hold timers, and loop protection features to ensure the information it receives is accurate.

It is scalable because it can handle thousands of individual sessions, and millions of prefixes. Today, the global routing table (also known as the Default-Free Zone or DFZ) is around 840,000 IPv4 prefixes and 100,000 IPv6 prefixes. Most internet facing routers will receive the full DFZ from multiple ISPs. That means that routers running BGP will have many millions of prefixes in its routing information base (RIB). It then needs to decide which of all these prefixes is the best path to a particular destination and there is a long list of “path attributes” BGP can use to make this decision. It is quite impressive that a protocol written so long ago when the internet was a fraction of today's size is able to handle the scale we see today.

It is flexible in that over the years, there have been numerous enhancements to BGP. I mentioned earlier that BGP is used to exchange network layer reachability information. Back in the day, this was basically just IPv4 addresses. Now BGP can exchange all sorts of different NLRI such as multicast prefixes, IPv6, all sorts of MPLS labels, and MAC addresses. The way people are using BGP has also changed, now it is almost exclusively the protocol of choice when designing massively scalable datacenter networks. This is due to the fact that BGP is proven to be reliable, scalable and flexible.

In terms of security, there’s many approaches network operators can take. The responsibility to keep the internet up and running falls into the hands of the many thousands of engineers all over the world who run the networks upon which people depend. In this spirit of cooperation and to do our part in keeping the internet secure, we’re pleased to announce that as of January 28, Equinix Metal has joined the Mutually Agreed Norms for Routing Security (MANRS) for CDN and Cloud Providers, a global initiative open to any network operators who commit to securing their network.

MANRS participants agree to implement a specific set of security and best-practice related guidelines at the network level. These guidelines are aimed at increasing the security of our own network, and the greater internet as well. To join MANRS there are 5 mandatory actions that a network operator must implement. These are:

Action 1: Prevent propagation of incorrect routing information

This action involves ingress filtering of prefixes received from all non-transit peers. This helps to prevent hijacked IP space from propagating around the Internet. At Equinix Metal, we apply inbound filters to our peers. These filters are built from IRR objects and are applied automatically to our peering sessions.

Action 2. Prevent traffic with illegitimate source IP addresses

Most DDOS attacks use spoofed source IP addresses to attack something on the internet. By preventing these spoofed IPs from leaving our network, this attack vector is greatly reduced. At Equinix Metal, we apply egress filters which allow only traffic sourced from ours —or our customers’ — IP space to leave our network.

Action 3. Facilitate global operational communication and coordination

When incidents do occur, it is important that network operators can quickly reach engineers in other organisations. To do this, up-to-date contact information must be accessible in the common tools we all use. Equinix Metal maintains contact info in PeeringDB as well as the relevant RIR databases. We also have a looking glass tool which allows anyone to peak into our network and see the path our routers are taking to a particular destination.

Action 4. Facilitate validation of routing information on a global scale

RPKI aims at security routing information by validating that the ASN originating a prefix is the legitimate owner of the prefix. As more network operators impmement RPKI, it becomes harder to hijack prefixes which reduces the attack vector. At Equinix Metal we have ROAs for all our public IPv4 and IPv6 space and we drop any prefix with an invalid validation state. We also maintain our IRR records in RADB via automated tooling.

Action 5. Encourage MANRS adoption

Like any standards-based approach, programs like MANRS can only succeed if there is a global push by all network operators to abide by the guidelines. As more organizations participate, we collectively reduce the impact and frequency of security related incidents that affect the entire internet. Equinix Metal encourages all its current and future peers to participate in MANRS and adopt the principles of good routing security.

By following these five principles our network and related operations are hardened against the types of issues described earlier in this blog. As more operators join and follow the MARNS guidelines, the impact of routing threats will be reduced, and the overall number of incidents will also decline.

We strongly encourage all network operators to implement the above security features. Many of the enhancements are simple to implement and can be done without causing any downtime for your customers. Even something as simple as creating ROAs for your prefixes will help. It is also very encouraging to see that more and more networks are starting to drop prefixes that are marked as invalid.

We’re proud to be doing our part to keep the internet secure, and are committed to continued effort in maintaining high levels of routing security.

If you’d like to learn more about internet security, there please take a look at the valuable information on MANRS.org. We also recommend reading up on RPKI and learning how to implement it on your network. If you have questions, please reach out so we can together build a better and more secure internet.