Hybrid Bonded Mode¶
Equinix Metal™ allows users to change the networking mode of servers from the default Layer 3 Bonded mode to a Hybrid Bonded Layer 3 and Layer 2 mode.
Hybrid Bonded mode enables a highly available “bonded” setup of 2 networking interfaces that supports both Layer 2 and Layer 3 modes at the same time. This keeps the functionality of supporting both Layer 2 and Layer 3, but does so while maintaining a highly available bonded networking interface that spans 2 diverse upstream switches.
This is a way to implement common hybrid cloud networking models such as running firewalls, custom gateways, ingress controllers and other types of proxies that face the Internet on one side and private Layer 2 infrastructure on the other side.
Availability and Compatibility¶
Hybrid Bonded mode is available in all Equinix IBX locations on Equinix Metal 3rd generation servers.
Servers in Equinix Metal's other data centers from previous generations can still use Hybrid Unbonded mode.
Enabling Hybrid Bonded Mode¶
Enabling Hybrid Bonded mode can be done by assigning a VLAN to the bonded interface on your server.
In the Equinix Metal console, navigate to the server's Network tab, click Convert To Other Network Type, select Hybrid, and choose Bonded.
Then, select the VLAN from the drop-down, which will allow you to assign a VLAN to the bond0
interface. Click Convert to Hybrid to start the changes.
If you are in an Equinix Metal data center that does not support Hybrid Bonded mode, you will only have the option to convert to Hybrid Unbonded mode.
In the CLI, assign a VLAN to the bonded interface, bond0
, with the metal port vlan
command.
metal ports vlan --port-id <bond0_id> --assign <VLAN_id>
Specify the UUID of bond0
for the --port-id
and the VLAN that you are assigning to --assign
.
In the API, you assign a VLAN to a port by sending a POST
to the /ports/{id}/assign
endpoint.
You specify the UUID of a port in the path. For Hybrid Bonded mode, it needs to be the UUID for bond0
as returned by the /devices/{id}
endpoint.
The ID of the VLAN is sent in the body of the request, and it can be either the VLAN's UUID as returned by the /projects/{id}/virtual-networks
endpoint or the VLAN ID that is in the console.
curl -X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: <API_TOKEN> " \
"https://api.equinix.com/metal/v1/ports/c4032b18-5494-451b-a779-a7d3c536bfd7/assign" \
-d '{
"vnid": "1001"
}'
Configuring Your Servers¶
Once you have assigned the VLAN to the port, you will need to configure the networking on the server's operating system to use the VLAN on bond0
. Because traffic from both Layer 3 and the VLAN are going through bond0
, IP packets that arrive at the host will have the VLAN ID populated.
-
Enable VLAN support.
modprobe 8021q lsmod | grep 8021q echo "8021q" >> /etc/modules-load.d/networking.conf
-
Add the VLAN to
bond0
.VLAN_ID
should match the VLAN ID found on the console.ip link add link bond0 name bond0.<VLAN_ID> type vlan id <VLAN_ID>
For example,
ip link add link bond0 name bond0.1036 type vlan id 1036
-
Add
192.168.100.1
IP address to the VLAN. IP addresses in the 192.168 range are recommended, as the10.0.0.0/8
range is used internally by Equinix Metal. For example,ip addr add 192.168.100.1/24 brd 192.168.100.255 dev bond0.1036 ip link set dev bond0.1036 up
To make the changes permanent, configure
/etc/network/interfaces
with the IP address to the desired IP from your chosen block. For example,auto bond0.1036 iface bond0.1036 inet static pre-up sleep 5 address 192.168.100.1 netmask 255.255.255.0 vlan-raw-device bond0
Note: The line
pre-up sleep 5
helps to prevent conflicts onbond0
when the server boots. -
Ensure the IP address is configured.
ip -d link show bond0.1036
-
Enable VLAN support.
echo 8021q > /etc/modules-load.d/8021q.conf
-
Add the VLAN to
bond0
.VLAN_ID
should match the VLAN ID found on the console.nmcli connection add type vlan con-name bond0.<VLAN_ID> ifname bond0.<VLAN_ID> vlan.parent bond0 vlan.id <VLAN_ID>
For example,
nmcli connection add type vlan con-name bond0.1036 ifname bond0.1036 vlan.parent bond0 vlan.id 1036
-
Add
192.168.100.1
IP address to the VLAN. IP addresses in the 192.168 range are recommended, as the10.0.0.0/8
range is used internally by Equinix Metal. For example,nmcli connection modify bond0.1036 ipv4.addresses '192.168.100.1/24' nmcli connection modify bond0.1036 ipv4.gateway '192.168.100.1' nmcli connection modify bond0.1036 ipv4.method manual nmcli con up bond0.1036
You need to run through the same steps on all the servers that you want to attach to the VLAN, assigning a different IP address to each.
Adding Multiple VLANs¶
Adding multiple VLANs to the bond is supported, you just have to make sure that the interface will receive packets destined for each VLAN.
To assign another VLAN, or to assign multiple VLANs at once, navigate to the server's Network page. In the Layer 2 section, click Add New VLAN, to assign the VLAN to the bond0
port.
Click Add to start the changes. Note that if you assign multiple VLANs at once, they are added through an asynchronous batch process, which begins immediately, but may take some time to complete.
To assign another or multiple more VLANs, use the metal port vlan
command, and specify each VLAN you are assigning to bond0
.
metal port vlan --port-id <bond0_id> --assign <vlan> --assign <vlan>
The procedure for assigning one more VLAN to the port in the API is the same assigning the first VLAN. Send a POST
request to the /ports/{id}/assign
endpoint, where the UUID of bond0
is the port ID in the path and the VLAN you want to assign is specified in the body of the request.
curl -X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: <API_TOKEN> " \
"https://api.equinix.com/metal/v1/ports/{id}/assign" \
-d '{
"vnid": "<vlan_id>"
}'
It is also possible to assign VLANs to a port in bulk, as part of an asynchronous batch process. Send a POST
request to the /ports/{id}/vlan-assignments/batches
endpoint an array of VLAN assignments in the body of the request.
curl -X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: <API_TOKEN> " \
"https://api.equinix.com/metal/v1/ports/{id}/vlan-assignments/batches" \
-d '{
"vlan_assignments": [
{
"vlan": "string",
"state": "assigned"
},
{
"vlan": "string",
"state": "assigned"
},
]
}'
The VLAN can be identified by either the VLAN's UUID or the VLAN ID that is in the console. Since you are assigning the VLAN to the ports, the "state"
field should be "assigned".
Once the VLANs are assigned to bond0
, repeat the process for configuring your servers to create the additional tagged interfaces.
Testing the VLAN Connection¶
You should now be able to communicate between hosts over your VLAN.
root@layer2:~# ping -I eth1 192.168.1.2
PING 192.168.1.3 (192.168.1.3) from 192.168.1.4 eth1: 56(84) bytes of data.
64 bytes from 192.168.1.3: icmp\_seq=1 ttl=64 time=0.106 ms
64 bytes from 192.168.1.3: icmp\_seq=2 ttl=64 time=0.110 ms
64 bytes from 192.168.1.3: icmp\_seq=3 ttl=64 time=0.115 ms
^C
--- 192.168.1.3 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.106/0.110/0.115/0.009 ms
Converting Back to Layer 3¶
To go back to the default Layer 3 networking configuration, remove any and all VLANs from bond0
.
To unassign a VLAN from a port, navigate to the server's Network tab. In the Layer 2 section, select the VLAN or VLANs you are detaching from the server and click Remove.
Note that detaching the VLAN from this server does NOT delete it from your project. The VLAN will continue to exist after detaching it from the server.
In the CLI, unassign a VLAN from bond0
, with the metal port vlan
command.
metal ports vlan --port-id <bond0_id> --unassign <vlan>
Specify the UUID of bond0
for the --port-id
and the VLAN that you are unassigning to --unassign
.
If you are removing multiple VLANs, specify each VLAN you are removing.
metal port vlan --port-id <bond0_id> --unassign <vlan> --unassign <vlan>
In the API, you unassign a VLAN from a port by sending a POST
to the /ports/{id}/unassign
endpoint, where the UUID of bond0
is the port ID in the path and the VLAN you want to unassign is specified in the body of the request.
curl -X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: <API_TOKEN> " \
"https://api.equinix.com/metal/v1/ports/{id}/unassign" \
-d '{
"vnid": "<vlan_id>"
}'
You can also unassign multiple VLANs from a port in bulk, as part of an asynchronous batch process. Send a POST
request to the /ports/{id}/vlan-assignments/batches
endpoint an array of VLAN assignments in the body of the request, and their state set to "unassigned"
.
curl -X POST \
-H "Content-Type: application/json" \
-H "X-Auth-Token: API_TOKEN" \
"https://api.equinix.com/metal/v1/ports/{id}/vlan-assignments/batches" \
-d '{
"vlan_assignments": [
{
"vlan": "string",
"state": "unassigned"
},
{
"vlan": "string",
"state": "unassigned"
}
]
}'
Once all the VLANs are removed from the port, bond0
will be back on Layer 3 mode. From there you can use any other of the existing network modes.