Connecting your Metal VLAN to Azure

In this guide, you will learn how to connect your Equinix Metal VLAN to a Virtual Network (vnet) that you own on Microsoft Azure.

Architecture

Before we get started, take a look at the architecture of the completed connection:

On the Azure side is a vnet with multiple subnets. The IP address range for the vnet, and for each of the subnets, is assigned by you. The vnet connects with the rest of the world via a Virtual Network Gateway.

On the Equinix Metal side is a VLAN with IP addresses assigned by you as well. The VLAN may or may not connect to standard Equinix Metal Layer 3 networking via an Equinix Metal Gateway.

The VLAN is connected to an Equinix Metal Gateway, which is connected to an instance of Equinix Metal Virtual Routing and Forwarding (VRF). VRF is connected to an Equinix Fabric Virtual Connection (VC). The VC is connected on the Metal end to the VRF, and on the Azure end to an ExpressRoute circuit, which connects to the Virtual Network Gateway.

Prerequisites

The prerequisites for connecting a Metal VLAN to an Azure vnet are:

• An Azure account
• A vnet created in your Azure account
• An Equinix Metal account, with a project in it
• A VLAN deployed to your Equinix Metal project
• Configuration information

The configuration information is the following:

Item	Purpose	Example Values
Azure vnet	CIDR ranges for the vnet	10.40.0.0/16
Azure Subnets	CIDR ranges for the vnet subnets	10.40.10.0/24, 10.40.20.0/24, 10.40.30.0/24
Metal VRF	CIDR range for the VRF	10.50.0.0/16
Metal VLAN	CIDR range for the VLAN, which must be within the VRF range	10.50.10.0/24
Azure ASN	The Autonomous System Number for the Azure side, fixed by Microsoft	12076
Metal ASN	The Autonomous System Number for the Metal side	64600
Primary VC subnet	Subnet for the primary virtual connection; this should be very small, as we will use just two addresses	192.168.15.8/30
Primary VC IP	IPs within the primary VC subnet to assign to each side	192.168.15.9 Metal and 192.168.15.10 Azure
Secondary VC subnet	Subnet for the secondary virtual connection; this should be very small, as we will use just two addresses	192.168.15.12/30
Secondary VC IP	IPs within the secondary VC subnet to assign to each side	192.168.15.13 Metal and 192.168.15.14 Azure
MD5 password	A password to use for BGP peering, MD5-hashed	Clear text of "WhatAGreatGuide", hashed to 8cfe43d779a9096c0ecc901ad0649a0f

See the Microsoft Azure documentation for help setting up the Azure components. When complete, you should have your account, a vnet with a Virtual Network Gateway, and subnets in that vnet.

For Equinix Metal, we have guides to help with setting up your account, organization and project.

Once you have your project, in the Console, select Networking:

Then select VLANs:

Then click the "Add a VLAN" button:

In the dialog that appears, pick a Metro and any VLAN ID that is convenient for you, or let Metal pick it automatically. For our example, we will use VLAN ID 200 in Washington. We are using Washington, because it is closest to where Azure's useast region is located.

Deploy VRF

With the VLAN in place, you now need a VRF that will be used to connect to Equinix Fabric. The VRF is under "Networking", like "VLANs". Click on "Virtual Routing and Forwarding":

Then click "Create Virtual Router":

In order to create the Virtual Router, you need a few pieces of information:

• A name for the VRF. It doesn't have any inherent meaning; it just needs to be useful to you. We will call it "Cloud".
• The Metro. This must be the same metro as the VLAN you are connecting to. In our case, it is Washington.
• The ASN. This is the Metal-side ASN we selected earlier, 64600.
• The allowed IP ranges. These are CIDR ranges that will be "behind" the VRF on the Metal side. Whatever ranges you pick here, these are the ranges that the VRF will tell Fabric, "send those addresses to me, I can handle them." In our case, that includes:
  • CIDRs used for Metal Gateways. We used the range we reserved earlier 10.50.0.0/16.
  • CIDRs used for subnets for the Fabric VCs. We selected the primary and secondary ranges of 192.168.15.8/30 and 192.168.15.12/30, but VRF only allows up to /29, so use 192.168.15.8/29, which covers both.

Click the Create Virtual Router button when you're finished.

Once the VRF is created, you need to reserve IP ranges to use in the Gateway and VLAN, from within the VRF allowed IP ranges. For this particular VLAN we use the range we described above, 10.50.10.0/24, which is within the VRF allowed IP range of 10.50.0.0/16.

In the console, click on the VRF:

This brings up the VRF details. Click "Add IP Reservation":

Then enter the range 10.50.10.0/24 and click "Submit Request":

Deploy Equinix Metal Gateway

We now have a VLAN and a VRF. Let's create a Metal Gateway to link them together. "Metal Gateway" is under "Networking", like "VLANs" and "Virtual Routing and Forwarding". Click on "Metal Gateways":

Then click "Create a Metal Gateway":

In order to create the Metal Gateway, you need a few pieces of information:

• The Metro. This must be the same metro as the VLAN you are connecting to. In our case, it is Washington.
• The VLAN. This is the VLAN that the Metal Gateway will connect to. We will use VLAN 200 which we created earlier.
• The IP block. This is one of a reserved public IPv4 range, a private IPv4 range, or a VRF IP range. Since we are connecting to a VRF, we will pick "VRF IP", and then the IP address reservation from the VRF.

Deploy Fabric Virtual Connection

There are several steps to deploying the Fabric Virtual Connection (VC):

1. Create the connection in Equinix Metal to Fabric
2. Create the ExpressRoute circuit in Azure
3. Create the Fabric VC
4. Configure BGP peering in Azure
5. Create the Virtual Network Gateway in Azure
6. Configure BGP peering in Metal

Create the connection in Equinix Metal to Fabric

From the left navigation bar, select Interconnections:

Then click the "Request a new interconnection" button:

There are two kinds of interconnection you can request, AWS Direct Connect and Request Interconnection. Since we are not connecting to AWS, select the "Request Interconnection" button:

The next screen has two options for interconnections, Fabric VC and Dedicated Port. We won't cover dedicated ports in this guide, as we are focused on using Equinix Fabric. If you are interested in learning more about dedicated ports, please see our dedicated ports documentation.

There are two kinds of Fabric VC, Metal-billed and Fabric-billed. Either can be useful for this kind of interconnection, but we will use the Metal-billed one. Read the interconnections documentation for more information on the differences.

To create the connection, you need a few pieces of information:

• A name for the connection. It doesn't have any inherent meaning; it just needs to be useful to you. We will call it "Azure."
• The Metro. This must be the same metro as the VRF you are connecting to. In our case, it is Washington.
• The connection speed. Higher speeds have a higher cost. Since we are just creating a basic connection, we will use the lowest speed, 50Mbps. You can select a higher one if you need it.
• Single vs redundant. Each connection is exactly that, a single connection. If there are issues anywhere along the line, whether hardware issues such as a port on the Azure or Equinix end, or software problems, your connection will be out of service, at least until it is fixed. Like all good networking, you should plan for redundancy. We will use a redundant connection for this example.
• Connect to VLAN or VRF. The VC is a layer 2 connection between layer 3 routers with BGP support on each size. On the cloud provider side, the layer 3 routing is provided for you. On the Metal side, you can set up the layer 3 routing and BGP manually with software on Metal devices and additional VLANs connected directly to the VC, or you can let Metal do all of that for you with a VRF. We will use a VRF connection for this example.
• Primary and secondary connections. Since we selected redundant connections, we need to specify where each of the two connections attaches on the Equinix Metal side. In the case of a VLAN, that would require two distinct VLANs. Since we are using VRF, which is Virtual Routing and Forwarding, we can use the same VRF for both connections. This is a key advantage of VRF connections over VLAN connections. We will use our "Azure" VRF for both.

Click the "Submit Request" button.

Once the request is submitted, you need two things:

1. An email to your address telling you that the service token is ready to use. You should get two such emails, one for each of the redundant connections. You know it is ready by the "Service Token Status" line.
2. The service token itself. This is a long string of characters that you will need to create the Fabric VC to Equinix Metal. It will appear in the emails, as well as in the console after you submit your request.

If you click Interconnections, you now will see the connection in the list under "Fabric VCs," with a status of "Pending," which eventually will change to "Ready" when all of the other steps are done.

Create the ExpressRoute circuit in Azure

In the Azure console, type "ExpressRoute" in the search bar, and select "ExpressRoute circuits".

Click "Create":

Enter the configuration information for your ExpressRoute circuit:

• Subscription: the billing subscription you are using
• Resource group: the resource group for this circuit, in our example, "Equinix"
• Region: the region in which to create the route, which must be the same as our Metal-side VRF; since the VRF is in Washington, we use "East US"
• Circuit name: a name for the circuit, in our example, "VRF-Metal"
• Port type: select "Provider," as we will be connecting to a known provider
• Provider: search through the list and find "Equinix"
• Peering location: "Washington"
• Bandwidth: select the bandwidth you want for the circuit; we will use 50Mbps
• SKU: select the SKU you want for the circuit; we will use "Standard"
• Billing model: select metered or unlimited; for our example, we will use "Metered"

Click "Review+Create" and then "Create" to create a new circuit:

The status screen shows the deployment progress:

Wait for the deployment to complete, then click the "Go to resource" button:

This brings up the resource details. The piece of information we need is the Service Key, which we will use for Fabric:

Create the Fabric VC

In this section, you'll create a Fabric VC between the VRF on Metal and the ExpressRoute circuit at Azure.

Log in to the Fabric console at https://fabric.equinix.com.

Once you are logged in, you will see the Equinix Fabric portal:

On the top menu bar, go to "Connections" and "Create Connection":

There are three main options for a connection: A Service Provider, An Equinix Fabric Customer, and My Own Assets. Click A Service Provider, then in the search box, type "Azure". This will filter the options below to just a few. Select "Azure" and click Select Service.

This opens up the options for Azure. Select "Azure ExpressRoute" and click Create Connection. You won't use "Quick Connect" in this case, as you need service keys on both the Metal and Azure ends:

Click "Create a Connection to Azure ExpressRoute":

Enter the Azure service key, which you retrieved when the ExpressRoute circuit setup was complete, in the Destination section. Fabric will validate the key with Microsoft. Once that is complete, it will return the location and connection speed.

Next, enter the origin, specifically Metal. You have primary and secondary service tokens from when you created the interconnection.

In the Primary Origin section, click Service Token, and then enter the primary service token from the Metal interconnection. Fabric will validate the location.

Repeat the process for the Redundant Origin, entering the Metal secondary token:

Click Next.

Enter the following configuration information:

• Names for the primary and secondary connections. These can be anything that has meaning to you. For our example, we will call them "VRF-Azure-primary" and "VRF-Azure-secondary".
• Peering type: select Private Peering. Read the Microsoft Azure documentation to understand the difference between the two.

Click Next.

Then review the details and click Submit Order.

At this point, the order is submitted. It may take a few minutes to complete. You can check the status by clicking on the connection:

The connections are ready when the status for them shows Pending BGP when you hover over them:

Configure BGP peering in Azure

In Azure, you need to configure BGP peering for the ExpressRoute circuit. Return to the ExpressRoute details page, the same one that gave you the service key. The service status should be Enabled, and the provider status should be Provisioned:

Under Peerings, click on Azure private:

Enter the configuration information:

• Peer ASN: the Metal-side ASN selected earlier, 64600
• Subnets: IPv4
• IPv4 Primary subnet: the primary VC subnet from the configuration table above, 192.168.15.8/30
• IPv4 Secondary subnet: the secondary VC subnet from the configuration table above, 192.168.15.12/30
• Enable IPv4 peering: checked
• VLAN ID: 300
• Shared key: the MD5 password from our configuration table above, WhatAGreatGuide

When you're done, click Save:

Create the Virtual Network Gateway in Azure

In Azure, a Virtual Network Gateway is a virtual router that connects a vnet to the rest of the world, including ExpressRoute circuits.

You may already have a Virtual Network Gateway created for your vnet. If not, create one now.

In the search box, enter "Virtual Network Gateway" and select it:

Then click Create:

Enter the configuration information for your Virtual Network Gateway:

• Subscription: the billing subscription you are using
• Resource group: the resource group for this gateway, in our example, "equinix:
• Name: a name for the gateway, in our example, "equinix-gateway"
• Region: the region for the gateway, in our example, "East US"
• Gateway type: select ExpressRoute since we are connecting to ExpressRoute circuits
• SKU: this depends on your intended performance; we will use "Standard"
• Virtual network: select the vnet you are connecting to, in our example, "Equinix"
• Subnet: this must be a Virtual Network Gateway subnet, which is created for you; in our example, "GatewaySubnet"
• Public IP address: either use an existing one, if you have, or create a new one

Click the Review + Create button.

Then click "Create".

You will see a page indicating that deployment is in progress:

When deployment is complete, the message will update:

Note that it can take anywhere from several minutes to an hour for a Virtual Network Gateway to be created.

With the Virtual Network Gateway in place, you can link the ExpressRoute circuits with the Gateway. From within the Virtual Network Gateway, under Settings, click Connections:

Then click the "Add" button:

On the Create Connection page, enter the following information:

• Subscription: Azure subscription that will handle the billing of this connection
• Resource group: Resource Group for this connection; in our example, "Equinix"
• Connection type: select "ExpressRoute"
• Name: Any representative name that is meaningful to you; in our example, "equinix-connection"

Click "Next: Settings".

On the settings page, enter the following information:

• Virtual network gateway: select the Virtual Network Gateway you created earlier
• ExpressRoute circuit: select the Express Route circuit you created earlier

Leave the rest as defaults, and click the "Review + create" button, and then "Create".

Wait a few minutes for the connection to be created.

Configure BGP peering in Metal

In the Metal console, click the Azure connection:

Select the Primary Port tab:

At the bottom of the page, you will see the Virtual Circuits listed. Click the three bars button on the right to edit it:

This opens the "Manage Peering Details" window. Enter the following information from our configuration table above.

• "Peer ASN": Azure-side ASN 12076
• "Subnet": Primary VC subnet 192.168.15.8/30
• "Metal IP": IP address of the Metal side of the connection 192.168.15.9
• "Customer IP": IP address of the Azure side of the connection 192.168.15.10
• "MD5 Password": WhatAGreatGuide

Click the "Update Virtual Circuit" button.

Repeat the process with Second Port, using the secondary VC subnet and IP addresses.

• "Peer ASN": Azure-side ASN 12076
• "Subnet": Secondary VC subnet 192.168.15.12/30
• "Metal IP": IP address of the Metal side of the connection 192.168.15.13
• "Customer IP": IP address of the Azure side of the connection 192.168.15.14
• "MD5 Password": WhatAGreatGuide

Click the "Update Virtual Circuit" button again.

When done, if you wait a few moments and return to the main interconnection page, you should see both ports in the Active state:

You also can check the learned routes. Return to the Networking menu and click Virtual Routing and Forwarding:

Click the Cloud VRF:

Then click the "BGP Table" tab:

The details of the BGP table are laid out. The important parts to note:

• The neighbors have a status of "Up" and a Peer ASN of 12076, exactly as we expected from Azure.
• The learned routes include:
  • 10.50.10.0/24, as expected for our VLAN with Metal Gateway attached
  • 10.40.0.0/16, which is the exact range for the Azure vnet

Test the Connection

The setup is complete, and now it's time to test the connection. We will deploy one server on each end, a Virtual Machine (VM) in Azure, and a Metal server in our VLAN.

Since this is a guide on the connections, and not on deploying Azure VMs or Metal servers, we won't give detailed descriptions here. There are a few things to keep in mind.

For the Azure VM, ensure:

• It is in a subnet in the vnet that has the defined Virtual Private Gateway
• It has a public IP address
• It allows ICMP traffic from both of the private ranges: 10.40.0.0/16 for the vnet, and 10.50.0.0/16 for the Metal VLAN
• Save the private IP of the server. In this example, Azure assigned 10.40.10.4

For the Metal server:

1. Deploy the server normally in the Washington metro.
2. Switch the server networking type to hybrid bonded, so you can SSH to it from the Internet while also connecting to the VLAN, then attach it to the VLAN and assign an IP address. Use the Equinix Metal hybrid bonded networking documentation for detailed instructions.
3. Add the route to the Azure vnet via the Metal Gateway, via ip route add <vnet CIDR> via <Metal Gateway>. In this example, that is ip route add 10.40.0.0/16 via 10.50.10.1.

root@c3-small-x86-01:~# apt-get install vlan
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
  grub-pc-bin
Use 'apt autoremove' to remove it.
The following NEW packages will be installed:
  vlan
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.4 kB of archives.
After this operation, 51.2 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 vlan all 2.0.5ubuntu5 [10.4 kB]
Fetched 10.4 kB in 0s (41.5 kB/s)
Selecting previously unselected package vlan.
(Reading database ... 74606 files and directories currently installed.)
Preparing to unpack .../vlan_2.0.5ubuntu5_all.deb ...
Unpacking vlan (2.0.5ubuntu5) ...
Setting up vlan (2.0.5ubuntu5) ...
Processing triggers for man-db (2.10.2-1) ...
Scanning processes...
Scanning processor microcode...
Scanning linux images...

Running kernel seems to be up-to-date.

The processor microcode seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

root@c3-small-x86-01:~# modprobe 8021q

root@c3-small-x86-01:~# echo "8021q" >> /etc/modules

root@c3-small-x86-01:~# ip link add link bond0 name bond0.200 type vlan id 200

root@c3-small-x86-01:~# ip addr add 10.50.10.2/24 dev bond0.200

root@c3-small-x86-01:~# ip ro add 10.40.0.0/16 via 10.50.10.1

With everything set up, first try to ping the Metal Gateway:

root@c3-small-x86-01:~# ping 10.50.10.1
PING 10.50.10.1 (10.50.10.1) 56(84) bytes of data.
64 bytes from 10.50.10.1: icmp_seq=1 ttl=64 time=0.240 ms
64 bytes from 10.50.10.1: icmp_seq=2 ttl=64 time=0.220 ms
64 bytes from 10.50.10.1: icmp_seq=3 ttl=64 time=0.224 ms
^C
--- 10.50.10.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2053ms
rtt min/avg/max/mdev = 0.220/0.228/0.240/0.008 ms

Finally, ping the Azure instance, using the IP from earlier:

root@c3-small-x86-01:~# ping 10.40.10.4
PING 10.40.10.4 (10.40.10.4) 56(84) bytes of data.
64 bytes from 10.40.10.4: icmp_seq=1 ttl=61 time=2.03 ms
64 bytes from 10.40.10.4: icmp_seq=2 ttl=61 time=2.27 ms
64 bytes from 10.40.10.4: icmp_seq=3 ttl=61 time=2.84 ms
^C
--- 10.40.10.4 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 2.031/2.379/2.840/0.339 ms

Conclusion

You've now connected you private VLAN on Equinix Metal to an external Azure virtual network. You've configured gateways on either side of the connection, and you have a connection between them via Equinix Fabric Virtual Connection. This setup means you can communicate securely between the two networks.