Connecting your Metal VLAN to AWS

In this guide, you will learn how to connect your Equinix Metal VLAN to a VPC that you own on AWS. This guide sets up the underlying networking elements directly, which gives you greater control. There is a simple, less-featured method using an Equinix Metal simplified connection in the console, which will be described in a later guide.

Before we get started, let's look at the architecture of the completed connection:

On the AWS side is a VPC, with multiple subnets. The IP address range for the VPC, and for each of the subnets, is as assigned by you. The VPC connects with the rest of the world via a standard AWS Transit Gateway.

On the Equinix Metal side is a VLAN with IP addresses assigned by you as well. The VLAN may or may not connect to standard Equinix Metal Layer 3 networking via an Equinix Metal Gateway.

The VLAN is connected to an Equinix Metal Gateway, which is connected to an instance of Equinix Metal Virtual Routing and Forwarding (VRF). VRF is connected to an Equinix Fabric Virtual Connection (VC). The VC is connected on one end to the VRF, and on the other end to the VPC Gateway.

Prerequisites

The prerequisites for connecting your VLAN to your AWS VPC are:

• An AWS account
• A VPC created in your AWS account, along with a VPC Transit Gateway
• An Equinix Metal account, with a project in it
• A VLAN deployed to your Equinix Metal project
• Configuration information

The configuration information is the following:

Item	Purpose	Example Values
AWS VPC	CIDR range for the VPC	10.20.0.0/16
AWS Subnets	CIDR ranges for the subnets	10.20.10.0/24, 10.20.11.0/24, 10.20.12.0/24
Metal VRF	CIDR range for the VRF	10.50.0.0/16
Metal VLAN	CIDR range for the VLAN, which must be within the VRF range	10.50.10.0/24
AWS ASN	The Autonomous System Number for the AWS side	64550
Metal ASN	The Autonomous System Number for the Metal side	64600
Primary VC subnet	Subnet for the primary virtual connection; this should be very small, as we will use just two addresses	192.168.15.0/30
Primary VC IP	IPs within the primary VC subnet to assign to each side	192.168.15.1 Metal and 192.168.15.2 AWS
Secondary VC subnet	Subnet for the secondary virtual connection; should be very small, as we will use just two addresses	192.168.15.4/30
Secondary VC IP	IPs within the secondary VC subnet to assign to each side	192.168.15.5 Metal and 192.168.15.6 AWS
MD5 password	A password to use for BGP peering, MD5-hashed	Clear text of "What A Great Guide!", hashed to dbc29529874661f92e089593c8adc10b

See the Amazon Web Services documentation for help setting up the AWS components here. When complete, you should have your account, a VPC with a Transit Gateway, and several subnets in that VPC.

For Equinix Metal, we have guides to help with setting up your account, organization and project.

Once you have your project, in the Console, select Networking:

Then select VLANs:

Then click the "Add a VLAN" button:

In the dialog that appears, pick a Metro and any VLAN ID that is convenient for you, or let Metal pick it automatically. For our example, we will use VLAN ID 200 in Washington. We are using Washington, because it is the same metro where Amazon's us-east-1 region is located.

Deploy VRF

With the VLAN in place, you now need a VRF that will be used to connect to Equinix Fabric. The VRF is under "Networking", like "VLANs". Click on "Virtual Routing and Forwarding":

then click "Create Virtual Router":

In order to create the Virtual Router, you need a few pieces of information:

• A name for the VRF. It doesn't have any inherent meaning; it just needs to be useful to you. We will call it "Cloud".
• The Metro. This must be the same metro as the VLAN you are connecting to. In our case, it is Washington.
• The ASN. This is the Metal-side ASN we selected earlier, 64600.
• The allowed IP ranges. These are CIDR ranges that will be "behind" the VRF on the Metal side. Whatever ranges you pick here, these are the ranges that the VRF will tell Fabric, "send those addresses to me, I can handle them." This must include:
  • CIDRs used for Metal Gateways. We used the range we reserved earlier 10.50.0.0/16
  • CIDRs used for subnets for the Fabric VCs. We selected the primary and secondary ranges of 192.168.15.0/30 and 192.168.15.4/30, but VRF only allows up to /29, so use 192.168.15.0/29, which covers both.

Click the Create Virtual Router button when you're finished.

Once the VRF is created, you need to reserve IP ranges to use from within the allowed IP ranges. For this particular VLAN we use the range we described above, 10.50.10.0/24, which is within the larger VRF allowed IP ranges.

In the console, click on the VRF:

This brings up the VRF details. Click "Add IP Reservation":

Then enter the range 10.50.10.0/24 and click "Submit Request":

Deploy Equinix Metal Gateway

We now have a VLAN and a VRF. Let's create a Metal Gateway to link them together. "Metal Gateway" is under "Networking", like "VLANs" and "Virtual Routing and Forwarding". Click on "Metal Gateways":

Then click "Create a Metal Gateway":

In order to create the Metal Gateway, you need a few pieces of information:

• The Metro. This must be the same metro as the VLAN you are connecting to. In our case, it is Washington.
• The VLAN. This is the VLAN that the Metal Gateway will connect to. We will use VLAN 200 which we created earlier.
• The IP block. This is one of a reserved public IPv4 range, a private IPv4 range, or a VRF IP range. Since we are connecting to a VRF, we will pick "VRF IP", and then the IP address reservation from the VRF.

Deploy Fabric Virtual Connection

There are several steps to deploying the Fabric Virtual Connection (VC):

1. Create the connection in Equinix Metal to Fabric
2. Create the Fabric VC
3. Accept the connection in AWS
4. Update BGP peering in Metal

Create the connection in Equinix Metal to Fabric

From the left navigation bar, select "Interconnections":

Then click the "Request a new interconnection" button:

There are two kinds of interconnection you can request, AWS Direct Connect and Request Interconnection. Although we are connecting to AWS, do not select AWS Direct Connect. That is a simplified version, easier to deploy but not as flexible. Instead, click the "Request Interconnection" button:

The next screen has two options for interconnections, Fabric VC and Dedicated Port. We won't cover dedicated ports in this guide, as we are focused on using Equinix Fabric. If you are interested in learning more about dedicated ports, please see our dedicated ports documentation.

There are two kinds of Fabric VC, Metal-billed and Fabric-billed. Either can be useful for this kind of interconnection, but we will use the Metal-billed one. Read the interconnections documentation for more information on the differences.

To create the connection, you need a few pieces of information:

• A name for the connection. It doesn't have any inherent meaning; it just needs to be useful to you. We will call it "AWS."
• The Metro. This must be the same metro as the VRF you are connecting to. In our case, it is Washington.
• The connection speed. Higher speeds have a higher cost. Since we are just creating a basic connection, we will use the lowest speed, 50Mbps. You can select a higher one if you need it.
• Single vs redundant. Each connection is exactly that, a single connection. If there are issues anywhere along the line, whether hardware issues such as a port on the AWS or Equinix end, or software problems, your connection will be out of service, at least until it is fixed. Like all good networking, you should plan for redundancy. We will use a redundant connection for this example.
• Connect to VLAN or VRF. The VC is a layer 2 connection between layer 3 routers with BGP support on each size. On the cloud provider side, the layer 3 routing is provided for you. On the Metal side, you can set up the layer 3 routing and BGP manually with software on Metal devices and additional VLANs connected directly to the VC, or you can let Metal do all of that for you with a VRF. We will use a VRF connection for this example.
• Primary and secondary connections. Since we selected redundant connections, we need to specify where each of the two connections attaches on the Equinix Metal side. In the case of a VLAN, that would require two distinct VLANs. Since we are using VRF, which is Virtual Routing and Forwarding, we can use the same VRF for both connections. This is a key advantage of VRF connections over VLAN connections. We will use our "Cloud" VRF for both.

Click the "Submit Request" button.

Once the request is submitted, you need two things:

1. An email to your address telling you that the service token is ready to use. You should get two such emails, one for each of the redundant connections. You know it is ready by the "Service Token Status" line.
2. The service token itself. This is a long string of characters that you will need to create the Fabric VC to Equinix Metal. It will appear in the emails, as well as in the console after you submit your request.

Create the Fabric VC

In this section, you'll create a Fabric VC between the VRF on Metal and our VPC at Amazon.

Hop over to the Fabric console at https://fabric.equinix.com and log in.

Once you are logged in, you will see the Equinix Fabric portal:

On the top menu bar, go to "Connections" and "Create Connection":

There are three main options for a connection: A Service Provider, An Equinix Fabric Customer, and My Own Assets. Click A Service Provider, then in the search box, type "AWS". This will filter the options below to just a few. Select "AWS" and click Select Service.

This opens up the connection options for AWS. There are two types of connections, normal and high capacity, and single or redundant for each.

We don't need high capacity for this example, so select the normal capacity option.

After you click Create Connection, you need to enter the following:

• The connection type, in our case, Redundant
• Your AWS account number
• The destination, which should be the same region as our Equinix Metro. Since we deployed to the Equinix Washington metro, we select "Ashburn (us-east-1)"

Click Next when you're done

In the next step, configure the connection type. Since the interconnection request gave us a pair of service tokens, under "Origin Asset Type" select "Service Token".

Click the "Enter Primary Service Token" button, which opens up the details window. Enter your first service token, and click Enter Service Token.

Repeat the step for your secondary service token.

When you're done, you should have a page that looks like this:

Enter a name for each of the connections, select bandwidth to match what you selected earlier, and click Next.

Review the details and click the "Create Connection" button.

You should see a page showing the connections created:

Notice how it says that the connection configuration is incomplete, and you must accept it in AWS. That will be the next and final step.

Accept the connection in AWS

Log in to your AWS console. In the search box, type "Direct Connect," and select it when it appears in the Services list.

Once in the Direct Connect section, select Connections from the left navigation bar:

You should see a list of connections, including your new ones, named "AWS-VRF-primary" and "AWS-VRF-secondary". Both are in "ordering" state. Fabric has requested to connect to AWS, but your account must accept the connection.

Click on the primary connection, and then click Accept it:

Repeat for the secondary connection.

When you are done, the Connections page will show both connections as "pending," until they are finished setting up.

Once they are done, they will show as available:

At this point, you have a connection from your VLAN, via a Metal Gateway through VRF over a Fabric VC to your AWS Direct Connect connection.

You now need to link those connections to your VPC Transit Gateway. This is a standard AWS operation.

If you do not have one, create an AWS Direct Connect gateway. Note that you need to provide the Amazon-side ASN. We will use the one we listed earlier, 64500.

When the Direct Connect gateway is set up, it will show as available. Click on it.

There are two sections to complete the connection:

• Virtual interface attachments are the connections between the Direct Connect gateway and Direct Connect links.
• Gateway attachments are the connections between the Direct Connect gateway and AWS-side networks, in this case the Transit Gateway.

Start with the Virtual interface attachments. Click Create virtual interface, and enter the following configuration data:

• Type: Transit
• Name: pick a representative name; we will call it fabric-vrf-primary
• Connection: select the connection you want to use; in our case, AWS-VRF-primary
• Interface owner: my account
• Direct Connect gateway: the Gateway we created, equinix-fabric-connect
• VLAN: can be any number, but leave it blank, and AWS will tell you what is available when you click Create
• BGP ASN: This is the ASN from the far end, what VRF announces, which we configured earlier, 64600

Expand the Additional Settings by clicking on the triangle and enter the following data:

• "Address family": IPv4
• "Your router peer IP": The IP address on the Metal side 192.168.15.1/30
• "Amazon router peer IP": The IP address on the AWS side 192.168.15.2/30
• "BGP Authentication key": The hashed BGP password dbc29529874661f92e089593c8adc10b

Click Create virtual interface.

Repeat the same steps for the secondary connection, using the following information:

• Type: Transit
• Name: pick a representative name; we will call it fabric-vrf-secondary
• Connection: select the connection you want to use; in our case, AWS-VRF-secondary
• Interface owner: my account
• Direct Connect gateway: the Gateway we created, equinix-fabric-connect
• VLAN: can be any number, but leave it blank, and AWS will tell you what is available when you click Create
• BGP ASN: This is the ASN from the far end, what VRF announces, which we configured earlier, 64600

Expand the "Additional Settings" by clicking on the triangle and enter the following data:

• "Address family": IPv4
• "Your router peer IP": The IP address on the Metal side 192.168.15.5/30
• "Amazon router peer IP": The IP address on the AWS side 192.168.15.6/30
• "BGP Authentication key": The hashed BGP password dbc29529874661f92e089593c8adc10b

Click Create virtual interface.

Finally, connect the Direct Connect gateway to the Transit Gateway. Click the Gateway associations tab and then click Associate gateway.

Select your Transit Gateway and enter the IP addresses you want to make available. In our case, we will allow the entire VPC of 10.20.0.0/16, but you can have one or more subsets.

When you are done, the Transit Gateway will show as associated. It may take a few minutes.

Update BGP peering in Metal

In the Metal console, click on Interconnections. You now will see the connection in the list under Fabric VCs, with a status of Pending, which eventually will change to Ready when all of the other steps are done. Click on the connection.

Select the Primary Port tab:

At the bottom of the page, you will see the Virtual Circuits listed. Click the three bars button on the right to edit it:

This opens the "Manage Peering Details" window. Enter the following information from our configuration table above:

• "Peer ASN": AWS-side ASN 64550
• "Subnet": Primary VC subnet 192.168.15.0/30
• "Metal IP": IP address of the Metal side of the connection 192.168.15.1
• "Customer IP": IP address of the AWS side of the connection 192.168.15.2
• "MD5 Password": dbc29529874661f92e089593c8adc10b

Click the "Update Virtual Circuit" button.

Repeat the process with Second Port, using the secondary VC subnet and IP addresses.

• "Peer ASN": AWS-side ASN 64550
• "Subnet": Primary VC subnet 192.168.15.4/30
• "Metal IP": IP address of the Metal side of the connection 192.168.15.5
• "Customer IP": IP address of the AWS side of the connection 192.168.15.6
• "MD5 Password": dbc29529874661f92e089593c8adc10b

Click the "Update Virtual Circuit" button again.

Test the Connection

The setup is complete, and now it's time to test the connection. We will deploy one server on each end, an EC2 instance in AWS, and a Metal server in our VLAN.

Since this is a guide on the connections, and not on deploying AWS instances or Metal servers, we won't give detailed descriptions here. There are a few things to keep in mind.

For the AWS server, ensure:

• It is in a subnet in the VPC that is connected to the Transit Gateway
• It has a public IP address
• It allows ICMP traffic from both of the private ranges: 10.20.0.0/16 for the VPC, and 10.50.0.0/16 for the Metal VLAN
• Save the private IP of the server. In this example, AWS assigned 10.20.11.170

For the Metal server:

1. Deploy the server normally in the Washington metro.
2. Switch the server networking type to hybrid bonded, so you can SSH to it from the Internet while also connecting to the VLAN, then attach it to the VLAN and assign an IP address. Use the Equinix Metal hybrid bonded networking documentation for detailed instructions.
3. Add the route to the AWS VPC via the Metal Gateway, via ip route add <AWS range> via <Metal Gateway>. In this example, that is ip route add 10.20.0.0/16 via 10.50.10.1.

root@c3-small-x86-01:~# apt-get install vlan
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
  grub-pc-bin
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  vlan
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.4 kB of archives.
After this operation, 51.2 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 vlan all 2.0.5ubuntu5 [10.4 kB]
Fetched 10.4 kB in 0s (41.5 kB/s)
Selecting previously unselected package vlan.
(Reading database ... 74606 files and directories currently installed.)
Preparing to unpack .../vlan_2.0.5ubuntu5_all.deb ...
Unpacking vlan (2.0.5ubuntu5) ...
Setting up vlan (2.0.5ubuntu5) ...
Processing triggers for man-db (2.10.2-1) ...
Scanning processes...
Scanning processor microcode...
Scanning linux images...

Running kernel seems to be up-to-date.

The processor microcode seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

root@c3-small-x86-01:~# modprobe 8021q

root@c3-small-x86-01:~# echo "8021q" >> /etc/modules

root@c3-small-x86-01:~# ip link add link bond0 name bond0.200 type vlan id 200

root@c3-small-x86-01:~# ip address add 10.50.10.2/24 dev bond0.200

root@c3-small-x86-01:~# ip route add 10.20.0.0/16 via 10.50.10.1

With everything set up, first try to ping the Metal Gateway:

root@c3-small-x86-01:~# ping 10.50.10.1
PING 10.50.10.1 (10.50.10.1) 56(84) bytes of data.
64 bytes from 10.50.10.1: icmp_seq=1 ttl=64 time=0.169 ms
64 bytes from 10.50.10.1: icmp_seq=2 ttl=64 time=0.236 ms
64 bytes from 10.50.10.1: icmp_seq=3 ttl=64 time=0.215 ms
^C
--- 10.50.10.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2042ms
rtt min/avg/max/mdev = 0.169/0.206/0.236/0.027 ms

That worked. Now, try to ping the router in AWS:

root@c3-small-x86-01:~# ping 10.20.11.1
PING 10.20.11.1 (10.20.11.1) 56(84) bytes of data.
64 bytes from 10.20.11.1: icmp_seq=1 ttl=55 time=2.06 ms
64 bytes from 10.20.11.1: icmp_seq=2 ttl=55 time=1.78 ms
64 bytes from 10.20.11.1: icmp_seq=3 ttl=55 time=1.89 ms
^C
--- 10.20.11.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 1.780/1.909/2.064/0.117 ms

Finally, ping the AWS instance, using the IP from earlier:

root@c3-small-x86-01:~# ping 10.20.11.170
PING 10.20.11.170 (10.20.11.170) 56(84) bytes of data.
64 bytes from 10.20.11.170: icmp_seq=1 ttl=55 time=1.87 ms
64 bytes from 10.20.11.170: icmp_seq=2 ttl=55 time=1.64 ms
64 bytes from 10.20.11.170: icmp_seq=3 ttl=55 time=1.95 ms
^C
--- 10.20.11.170 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2004ms
rtt min/avg/max/mdev = 1.640/1.821/1.952/0.109 ms

Conclusion

You've now connected you private VLAN on Equinix Metal to an external VPC on AWS. You've configured gateways on either side of the connection, and you have a connection between them via Equinix Fabric Virtual Connection. This setup means you can communicate securely between the two networks.