Layer 2 Only Unbonded Mode¶
Equinix Metal™ allows users to change the networking mode of a server from the default Layer 3 Bonded mode to modes that only use Layer 2 networking over VLANs. This means all access to the public Internet is lost, and the host can only be reached by the Serial Over SSH (SOS) console or from other servers attached to the same VLAN.
This option will destroy the LACP bond and allow you to place each interface into individual Layer 2 networks. This does introduce a single point of failure on the upstream switch or the network interface. An outage, maintenance, or reboot of one of the ports will cause network interruptions.
If you have high-availability concerns, Layer 2 Bonded Mode supports a Layer 2 only configuration while maintaining the bonded networking interface that spans 2 diverse upstream switches.
General Overview¶
- Layer 2 Unbonded mode breaks the bond, allowing you to configure the two interfaces separately, but does expose a single point of failure to your networking if one of the ports or switches fails.
- You can then add VLANs to
eth1
andeth0
for your Layer 2 traffic. If you are only adding one VLAN per interface toeth1
oreth0
, traffic can not be tagged. - If you are adding more than one VLAN to
eth1
oreth0
, you will need to create subinterfaces to handle tagged traffic for each VLAN and/or you have to set a Native VLAN to handle untagged traffic.
Converting to Layer 2 Unbonded Mode¶
In the Equinix Metal console, navigate to the server's Network page, click Convert To Other Network Type and choose Layer 2. Select the Unbonded mode. Click Convert to Layer 2 to start the conversion.
When it is finished, you will be back on the server's Network page.
To assign a VLAN to one of the ports, click Add New VLAN, and select either the eth0
or eth1
interface. Click Add to start the changes.
Convert the bonded interface to Layer 2 while removing both port interfaces from the bond with the metal port convert command, specifying the UUID of bond0
for the --port-id
and using the --unbonded
flag.
metal port convert --port-id <bond0_id> --layer2 --unbonded
Then you can add VLANs to either eth0
or eth1
(or both) with the metal port vlan
command. Specify the UUID of eth0
or eth1
for the --port-id
and the VLAN that you are assigning to --assign
.
metal port vlan --port-id <eth1_id> --assign <VLAN_id>
The VLAN_id
can be either the VLAN's UUID or the VLAN ID number that is in the console.
In the API, converting to Layer 2 Unbonded mode is a two-step process. You have to first break the bond for all ports by sending a POST
request to the /ports/{id}/disbond
endpoint.
You have to specify the port ID of eth1
in the path, and it needs to be the UUID for that port as returned by the /devices/{id}
endpoint.
Set the bulk_disable
field to false
in the body of the request.
curl -X POST -H \
"Content-Type: application/json" \
-H "X-Auth-Token: <API_TOKEN>" \
"https://api.equinix.com/metal/v1/ports/{id}/disbond" \
-d '{
"bulk_disable": false
}'
Then, you need to assign the VLAN to eth1
. Send a POST
request to the /ports/{id}/assign
endpoint.
You have to specify the port ID of eth1
, and it needs to be the UUID for that port as returned by the /devices/{id}
endpoint.
curl -X POST -H \
"Content-Type: application/json" \
-H "X-Auth-Token: <API_TOKEN>" \
"https://api.equinix.com/metal/v1/ports/{id}/assign" \
-d '{
"vnid": "<vlan_ID>"
}'
The 'vnid' is the ID of the VLAN that is sent in the body of the request. The ID an be either the VLAN's UUID as returned by the /projects/{id}/virtual-networks
endpoint, or the VLAN ID that is in the console.
Attaching Multiple VLANs - Unbonded¶
Adding multiple VLANs to the bonded interface is supported. This does affect the server configuration as you will have to make sure there is a subinterface that will receive packets destined for each VLAN and/or you set a Native VLAN to handle untagged traffic.
To assign multiple VLANs to a port, navigate to the server's Network page. In the Layer 2 section, click Add New VLAN, which will allow you to assign more VLANs to eth0
or eth1
.
Click Add to start the changes. Note that if you assign multiple VLANs at once, they are added through an asynchronous batch process, which begins immediately, but may take some time to complete.
To assign another or multiple more VLANs to either eth0
or eth1
, use the metal port vlan
command, specify each VLAN you are assigning, and with port you are assigning it to.
metal port vlan --port-id <eth0_id> --assign <vlan> --assign <vlan>
The procedure for assigning a single additional VLAN to a port is the same as assigning the first VLAN. Send a POST
request to the /ports/{id}/assign
endpoint.
You have to specify the port ID in the path, and it needs to be the UUID for that port as returned by the /devices/{id}
endpoint.
The ID of the VLAN is sent in the body of the request, and it can be either the VLAN's UUID as returned by the /projects/{id}/virtual-networks
endpoint or the VLAN ID that is in the console.
curl -X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: <API_TOKEN> " \
"https://api.equinix.com/metal/v1/ports/{id}/assign" \
-d '{
"vnid": "c4032b18-5494-451b-a779-a7d3c536bfd7"
}'
It is also possible to assign VLANs to a port in bulk, as part of an asynchronous batch process. Send a POST
request to the /ports/{id}/vlan-assignments/batches
endpoint an array of VLAN assignments in the body of the request.
curl -X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: <API_TOKEN> " \
"https://api.equinix.com/metal/v1/ports/{id}/vlan-assignments/batches" \
-d '{
"vlan_assignments": [
{
"vlan": "string",
"state": "assigned"
},
{
"vlan": "string",
"state": "assigned"
},
]
}'
Configuring Your Servers for Layer 2 Unbonded¶
Once you have converted the server to Layer 2 Unbonded mode and assigned the VLAN to an interface, you will need to configure the networking on the server's operating system and assign it an IP address on the VLAN.
Note - Since all Equinix Metal assigned IP addresses were removed when you converted this server to Layer 2, you are free to configure and use whatever subnets you need, but you may have to remove any existing configuration in the server's operating system to prevent conflicts.
There are two example configurations, the first example is a configuration for assigning a single VLAN, and the second example is for assigning multiple VLANs.
Remember you will need to use the SOS/OOB Console to SSH into your server.
For a Single VLAN per Interface¶
If you have only one VLAN, do not tag the traffic, and assign the VLAN IP Address directly to the interface. This example uses eth1
, but the procedure is the same for eth0
or eth1
.
Note - in our recent Ubuntu images, the interfaces are no longer aliased to eth1
and eth0
, eth1
is enp1s0f1
and eth0
is enp1s0f0
.
-
Install and configure the prerequisites for VLANs.
apt-get install vlan modprobe 8021q echo "8021q" >> /etc/modules
-
Make sure
eth1
andeth0
have been removed frombond0
.ip -d link show enp1s0f1 ip -d link show enp1s0f0
If they haven't been removed, remove it.
ip link set enp1s0f1 nomaster ip link set enp1s0f0 nomaster
-
Configure
enp1s0f1
with an IP address for the VLAN. The example uses IP address192.168.1.2/29
.ip addr add 192.168.1.2/29 dev enp1s0f1
To make the networking configuration permanent and survive server reboots, edit
enp1s0f1
in the/etc/network/interfaces
file.auto enp1s0f1 iface enp1s0f1 inet manual address 192.168.1.2 netmask 255.255.255.248 pre-up sleep 4
-
Bring up the interface, and check that it is back up.
ip link set dev enp1s0f1 up ip -d link show enp1s0f1
For Multiple VLANs on an Interface¶
If you are using multiple VLANs on eth1
or eth0
, IP packets will have the to be tagged, and you will need to setup subinterfaces that will receive packets destined for each VLAN.
This example uses eth1
, but the procedure is the same for eth0
or eth1
.
Note: If you need support for untagged packets, set the VLAN that handles the untagged packets as the Native VLAN.
Note - in our recent Ubuntu images, the interfaces are no longer aliased to eth1
and eth0
, eth1
is enp1s0f1
and eth0
is enp1s0f0
.
-
Install and configure the prerequisites for VLANs.
apt-get install vlan modprobe 8021q echo "8021q" >> /etc/modules
-
Make sure
eth1
andeth0
have been removed frombond0
.ip -d link show enp1s0f1 ip -d link show enp1s0f0
If they haven't been removed, remove it.
ip link set enp1s0f1 nomaster ip link set enp1s0f0 nomaster
-
Add new subinterfaces on
enp1s0f1
to handle tagged traffic, one for each VLAN, and assign them IP addresses. Use your VLAN IDs from the Equinix Metal VLAN as in theVLAN_ID
fields. The example uses VLAN IDs1000
and1001
and IP addresses192.168.1.2/29
and192.168.100.2/29
.ip link add link enp1s0f1 name enp1s0f1.1000 type vlan id 1000 ip addr add 192.168.1.2/29 dev enp1s0f1.1000 ip link add link enp1s0f1 name enp1s0f1.1001 type vlan id 1001 ip addr add 192.168.100.2/29 dev enp1s0f1.1001
To make the networking configuration permanent and survive server reboots, add the new subinterfaces to the
/etc/network/interfaces
file.auto enp1s0f1.1000 iface enp1s0f1.1000 inet static address 192.168.1.2 netmask 255.255.255.248 vlan-raw-device enp1s0f1 auto enp1s0f1.1001 iface enp1s0f1.1001 inet static address 192.168.100.2 netmask 255.255.255.248 vlan-raw-device enp1s0f1
-
Bring up the interfaces, and check that they came up.
ip link set dev enp1s0f1.1000 up ip -d link show enp1s0f1.1000 ip link set dev enp1s0f1.1001 up ip -d link show enp1s0f1.1001
You will need to run through the same steps on all the servers that you want to attach to the VLANs, assigning different IP addresses to each.
Converting Back to Layer 3¶
If you are in one of the Layer 2 modes and want to go back to Layer 3, you must first remove any attached VLANs.
To detach a VLAN in the console, navigate to the server's Network page. In the Layer 2 section, click Remove next to the VLAN you are detaching from the server. Confirm that you wish to remove it by clicking Yes.
Detaching the VLAN from this server does NOT delete it from your project. The VLAN will continue to exist after detaching it from the server.
Then, to convert back to Layer 3, click Convert To Other Network Type, select Layer 3. Click Convert to Layer 3 to start the process.
Note - Being in Layer Bonded Mode removed the server's public IP addresses. If the server is later converted back to Layer 3, new public IP addresses are assigned and the server is accessible to the public Internet