Skip to main content

Kubernetes Workload Cluster with k3s

Set up a streamlined Kubernetes (k3s) cluster on Equinix Metal, integrating Traefik for efficient ingress handling and kube-vip for BGP-based load balancing, suitable for both private and public network implementations.

Kubernetes Workload Cluster with k3s

This guide will show you how to deploy a Kubernetes cluster using k3s, with ingress for running workloads; this will be provided by Traefik as the ingress controller and kube-vip to advertise the BGP address.

Equinix Metal Metadata

As we're going to use the Equinix Metal metadata a fair bit during this guide, we're going to use curl to fetch the data and store it locally, to prevent future requests from going over the network.

curl -fsSL > /tmp/metadata.json


The first thing you need to determine is whether you need your k3s API available outside of Equinix Metal. We strongly encourage you to think this constraint through thoroughly. If you're using GitOps to deploy your workloads, we'd recommend binding k3s to the private IPv4 management interface. You can grab this IP address with the following command.

export API_IP=$(jq -r '.network.addresses | map(select(.public==false and .management==true)) | first | .address' /tmp/metadata.json)

If you decide that it should be public, you'll want to use the public IPv4 management address.

export API_IP=$(jq -r '.network.addresses | map(select(.public==true and .management==true)) | first | .address' /tmp/metadata.json)

You can then install k3s with the following snippet. This configures k3s to bind to the correct IP address, based on the above, as well as disabling the cloud controller and service load balancer. We disable the service load balancer because Equinix Metal doesn't provide that functionality and we'll need to deploy kube-vip instead.

export INSTALL_K3S_EXEC="\
    --bind-address ${API_IP} \
    --advertise-address ${API_IP} \
    --node-ip ${API_IP} \
    --tls-san ${API_IP} \
    --no-deploy servicelb \

curl -sfL | sh -


BGP Routes

Now that k3s is running, we need to configure the routing table on the machine to understand how to communicate with the BGP peers for advertising any load balancer services with kube-vip.

We can fetch the gateway address from the metadata API and use ip route to add.

GATEWAY_IP=$(jq -r ".network.addresses[] | select(.public == false) | .gateway" /tmp/metadata.json)
ip route add via $GATEWAY_IP
ip route add via $GATEWAY_IP


We now need to apply the kube-vip service account and role to our k3s control plane. This will apply just enough permissions for kube-vip to understand the services running within.

kubectl apply -f

This applies the following rules to our service account:

  - apiGroups: [""]
    resources: ["services", "services/status", "nodes"]
    verbs: ["list","get","watch", "update"]
  - apiGroups: [""]
    resources: ["leases"]
    verbs: ["list", "get", "watch", "update", "create"]


k3s supports running workloads via the manifests directory, available at /var/lib/rancher/k3s/server/manifests. We're going to use ctr to pull the kube-vip image and generate a manifest.

crictl pull

ctr run --rm --net-host kube-vip /kube-vip manifest daemonset \
  --interface lo \
  --vip ${API_IP} \
  --metal \
  --metalKey YOUR_METAL_API_KEY \
  --metalProjectID YOUR_METAL_PROJECT_ID \
  --services \
  --inCluster \
  --taint \
  --bgp \
   | tee /var/lib/rancher/k3s/server/manifests/kube-vip.yaml


k3s automatically ships with Traefik on our cluster as the ingress controller, so we don't need to deploy anything. However, as we're not using a Service Load Balancer, we need to configure Traefik with the EIP.

kubectl --kubeconfig /etc/rancher/k3s/k3s.yaml patch svc traefik -n kube-system -p '{"spec": {"type": "LoadBalancer", "loadBalancerIP":"${API_IP}"}}'

Last updated

03 June, 2024



Subscribe to our newsletter

A monthly digest of the latest news, articles, and resources.