Encrypt data in transit
Big picture
Enable WireGuard to secure on the wire in-cluster pod traffic in a Calico Enterprise cluster.
Value
When this feature is enabled, Calico Enterprise automatically creates and manages WireGuard tunnels between nodes providing transport-level security for on-the-wire, in-cluster pod traffic. WireGuard provides formally verified secure and performant tunnels without any specialized hardware. For a deep dive in to WireGuard implementation, see this whitepaper.
Features
This how-to guide uses the following Calico Enterprise features:
- Felix configuration resource with WireGuard configuration parameters
Before you begin
Supported
The following platforms using only IPv4:
- Kubernetes, on-premises
- EKS using Calico CNI
- EKS using AWS CNI
- AKS using Azure CNI
All platforms listed above will encrypt pod-to-pod traffic. Additionally, when using AKS or EKS, host-to-host traffic will also be encrypted, including host-networked pods.
Required
- Operating system(s) of nodes running in the cluster must support WireGuard
- IP addresses for every node in the cluster. This is required to establish secure tunnels between the nodes. Calico Enterprise can automatically do this using IP Setting and IP autodetection methods available under calico/node resource.
- Under installation, set the autodetection method (
nodeAddressAutodetectionV4
and/ornodeAddressAutodetectionV6
) for your cluster.
- Under installation, set the autodetection method (
How to
- Install WireGuard
- Enable WireGuard for a cluster
- Verify encryption is enabled
- Disable WireGuard for an individual node
- Disable WireGuard for a cluster
Install WireGuard
WireGuard is included in Linux 5.6+ kernels, and has been backported to earlier Linux kernels in some Linux distributions.
Install WireGuard on cluster nodes using instructions for your operating system. Note that you may need to reboot your nodes after installing WireGuard to make the kernel modules available on your system.
Use the following instructions for these platforms that are not listed on the WireGuard installation page.
To install WireGuard on the default Amazon Machine Image (AMI):
sudo yum install kernel-devel-`uname -r` -y
sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm -y
sudo curl -o /etc/yum.repos.d/jdoss-wireguard-epel-7.repo https://copr.fedorainfracloud.org/coprs/jdoss/wireguard/repo/epel-7/jdoss-wireguard-epel-7.repo
sudo yum install wireguard-dkms wireguard-tools -y
Additionally, you may optionally enable host-to-host encryption mode for WireGuard using the following command.
kubectl patch felixconfiguration default --type='merge' -p '{"spec": {"wireguardHostEncryptionEnabled": true}}'
Warning:
wireguardHostEncryptionEnabled
is an experimental flag that extends WireGuard encryption to host-network IP addresses. It is currently only supported on managed clusters deployed on EKS and AKS, where WireGuard cannot be enabled on the cluster’s control-plane node. Enabling this flag while WireGuard is enabled on the control-plane node can lead to a broken cluster, and neworking deadlock.
AKS cluster nodes run Ubuntu with a kernel that has WireGuard installed already, so there is no manual installation required.
However, you will need to enable host-to-host encryption mode for WireGuard using the following command:
kubectl patch felixconfiguration default –type=’merge’ -p ‘{“spec”: {“wireguardHostEncryptionEnabled”: true}}’
Warning:
wireguardHostEncryptionEnabled
is an experimental flag that extends WireGuard encryption to host-network IP addresses. It is currently only supported on managed clusters deployed on EKS and AKS, where WireGuard cannot be enabled on the cluster’s control-plane node. Enabling this flag while WireGuard is enabled on the control-plane node can lead to a broken cluster, and neworking deadlock.
To install WireGuard for OpenShift v4.8:
- Install requirements:
- Download and configure the tools needed for kmods.
FAKEROOT=$(mktemp -d) git clone https://github.com/tigera/kmods-via-containers cd kmods-via-containers make install FAKEROOT=${FAKEROOT} cd .. git clone https://github.com/tigera/kvc-wireguard-kmod cd kvc-wireguard-kmod make install FAKEROOT=${FAKEROOT} cd ..
-
Configure/edit
${FAKEROOT}/root/etc/kvc/wireguard-kmod.conf
.a. You must then set the URLs for the
KERNEL_CORE_RPM
,KERNEL_DEVEL_RPM
andKERNEL_MODULES_RPM
packages in the conf file$FAKEROOT/etc/kvc/wireguard-kmod.conf
. Obtain copies forkernel-core
,kernel-devel
, andkernel-modules
rpms from RedHat Access and host it in an http file server that is reachable by your OCP workers.b. For more details and help about configuring
kvc-wireguard-kmod/wireguard-kmod.conf
, see the kvc-wireguard-kmod README file. Notes about wireguard version to kernel version compatibility is also available there. - Get RHEL Entitlement data from your own RHEL8 system from a host in your cluster.
tar -czf subs.tar.gz /etc/pki/entitlement/ /etc/rhsm/ /etc/yum.repos.d/redhat.repo
Please refer to Openshift documentation for more information about these entitlement files.
- Copy the
subs.tar.gz
file to your workspace and then extract the contents using the following command.tar -x -C ${FAKEROOT}/root -f subs.tar.gz
- Transpile your machine config using CoreOS Butane.
cd kvc-wireguard-kmod make ignition FAKEROOT=${FAKEROOT} > mc-wg.yaml
- With the KUBECONFIG set for your cluster, run the following command to apply the MachineConfig which will install WireGuard across your cluster.
oc create -f mc-wg.yaml
Enable WireGuard for a cluster
Note: Nodes that do not support WireGuard will not be secured by WireGuard tunnels, even if traffic running on the node to and from the pods goes to nodes that do support WireGuard.
Enable WireGuard encryption across all the nodes using the following command.
kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabled":true}}'
For OpenShift, add the Felix configuration with WireGuard enabled under custom resources. To enable WireGuard at the time of installation using operator, add the felixconfiguration to custom-resources.
Note: The above command can be used to change other WireGuard attributes. For a list of other WireGuard parameters and configuration evaluation, see the Felix configuration.
We recommend that you review and modify the MTU used by Calico networking when WireGuard is enabled to increase network performance. Follow the instructions in the Configure MTU to maximize network performance guide to set the MTU to a value appropriate for your network.
Verify encryption is enabled
To verify that the nodes are configured for WireGuard encryption, check the node status set by Felix using kubectl
. For example:
kubectl get node <NODE-NAME> -o yaml
...
kind: Node
metadata:
annotations:
projectcalico.org/WireguardPublicKey: jlkVyQYooZYzI2wFfNhSZez5eWh44yfq1wKVjLvSXgY=
...
Enable WireGuard statistics
To access wireguard statistics, prometheus stats in Felix configuration should be turned on. A quick way to do this is to apply the following command and manifest:
- Enable
nodeMetricsPort
:
kubectl patch installation.operator.tigera.io default --type merge -p '{"spec":{"nodeMetricsPort":9091}}'
- Apply Service, ServiceMonitor, NetworkPolicy manifests:
cat <<EOF | kubectl apply -f -
---
apiVersion: v1
kind: Service
metadata:
name: calico-prometheus-metrics
namespace: calico-system
labels:
k8s-app: calico-node
spec:
ports:
- name: calico-prometheus-metrics-port
port: 9091
protocol: TCP
selector:
k8s-app: calico-node
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
generation: 1
labels:
team: network-operators
name: calico-node-monitor-additional
namespace: tigera-prometheus
spec:
endpoints:
- bearerTokenSecret:
key: ""
honorLabels: true
interval: 5s
port: calico-prometheus-metrics-port
scrapeTimeout: 5s
namespaceSelector:
matchNames:
- calico-system
selector:
matchLabels:
k8s-app: calico-node
---
apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
labels:
projectcalico.org/tier: allow-tigera
name: allow-tigera.prometheus-calico-node-prometheus-metrics-egress
namespace: tigera-prometheus
spec:
egress:
- action: Allow
destination:
ports:
- 9091
protocol: TCP
source: {}
selector: app == 'prometheus' && prometheus == 'calico-node-prometheus'
tier: allow-tigera
types:
- Egress
---
apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
labels:
projectcalico.org/tier: allow-tigera
name: allow-tigera.calico-node-prometheus-metrics-ingress
namespace: calico-system
spec:
tier: allow-tigera
selector: k8s-app == 'calico-node'
types:
- Ingress
ingress:
- action: Allow
protocol: TCP
source:
selector: app == 'prometheus' && prometheus == 'calico-node-prometheus'
destination:
ports:
- 9091
EOF
Disable WireGuard for an individual node
To disable WireGuard on a specific node with WireGuard installed, modify the node-specific Felix configuration. e.g., to turn off encryption for pod traffic on node my-node
, use the following command:
cat <<EOF | kubectl apply -f -
apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
name: node.my-node
spec:
logSeverityScreen: Info
reportingInterval: 0s
wireguardEnabled: false
EOF
With the above command, Calico will not encrypt any of the pod traffic to or from node my-node
.
To enable encryption for pod traffic on node my-node
again:
calicoctl patch felixconfiguration node.my-node --type='merge' -p '{"spec":{"wireguardEnabled":true}}'
Disable WireGuard for a cluster
To disable WireGuard on all nodes modify the default Felix configuration. For example:
kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabled":false}}'