diff --git a/images/552_OpenShift_slb_mode_0625.png b/images/552_OpenShift_slb_mode_0625.png new file mode 100644 index 0000000000..4b51b9c1ff Binary files /dev/null and b/images/552_OpenShift_slb_mode_0625.png differ diff --git a/installing/installing_bare_metal_ipi/ipi-install-installation-workflow.adoc b/installing/installing_bare_metal_ipi/ipi-install-installation-workflow.adoc index 1169df5e83..e87aece304 100644 --- a/installing/installing_bare_metal_ipi/ipi-install-installation-workflow.adoc +++ b/installing/installing_bare_metal_ipi/ipi-install-installation-workflow.adoc @@ -38,6 +38,9 @@ include::modules/creating-manifest-file-customized-br-ex-bridge.adoc[leveloffset // Scale each machine set to compute nodes include::modules/creating-scaling-machine-sets-compute-nodes-networking.adoc[leveloffset=+2] +// Enabling OVS balance-slb mode for your cluster +include::modules/enabling-OVS-balance-slb-mode.adoc[leveloffset=+1] + // Establishing communication between subnets include::modules/ipi-install-establishing-communication-between-subnets.adoc[leveloffset=+1] diff --git a/modules/enabling-OVS-balance-slb-mode.adoc b/modules/enabling-OVS-balance-slb-mode.adoc new file mode 100644 index 0000000000..d5f9c767c0 --- /dev/null +++ b/modules/enabling-OVS-balance-slb-mode.adoc @@ -0,0 +1,206 @@ +// Module included in the following assemblies: +// +// IPI +// * installing/installing_bare_metal/ipi/ipi-install-installation-workflow.adoc +// UPI +// * installing/installing_bare_metal/upi/installing-bare-metal-network-customizations.adoc +// * installing/installing_bare_metal/upi/installing-restricted-networks-bare-metal.adoc +// * installing/installing_bare_metal/upi/installing-bare-metal.adoc + +:_mod-docs-content-type: PROCEDURE +[id="enabling-OVS-balance-slb-mode_{context}"] += Enabling OVS balance-slb mode for your cluster + +You can enable the Open vSwitch (OVS) `balance-slb` mode so that two or more physical interfaces can share their network traffic. A `balance-slb` mode interface can give source load balancing (SLB) capabilities to a cluster that runs virtualization workloads, without requiring load balancing negotiation with the network switch. + +Currently, source load balancing runs on a bond interface, where the interface connects to an auxiliary bridge, such as `br-phy`. Source load balancing balances only across different Media Access Control (MAC) address and virtual local area network (VLAN) combinations. Note that all OVN-Kubernetes pod traffic uses the same MAC address and VLAN, so this traffic cannot be load balanced across many physical interfaces. + +The following diagram shows `balance-slb` mode on a simple cluster infrastructure layout. Virtual machines (VMs) connect to specific localnet `NetworkAttachmentDefinition` (NAD) custom resource definition (CRDs), `NAD 0` or `NAD 1`. Each NAD provides VMs with access to the underlying physical network, supporting VLAN-tagged or untagged traffic. A `br-ex` OVS bridge receives traffic from VMs and passes the traffic to the next OVS bridge, `br-phy`. The `br-phy` bridge functions as the controller for the SLB bond. The SLB bond balances traffic from different VM ports over the physical interface links, such as `eno0` and `eno1`. Additionally, ingress traffic from either physical interface can pass through the set of OVS bridges to reach the VMs. + +.OVS `balance-slb` mode operating on a localnet with two NAD CRDs +image::552_OpenShift_slb_mode_0625.png[OVS `balance-slb` mode operating on a localnet with two NAD CRDs] + +You can integrate the `balance-slb` mode interface into primary or secondary network types by using OVS bonding. Note the following points about OVS bonding: + +* Supports the OVN-Kubernetes CNI plugin and easily integrates with the plugin. +* Natively supports `balance-slb` mode. + +.Prerequisites + +* You have more than one physical interface attached to your primary network and you defined the interfaces in a `MachineConfig` file. +* You created a manifest object and defined a customized `br-ex` bridge in the object configuration file. +* You have more than one physical interfaces attached to your primary network and you defined the interfaces in a NAD CRD file. + +.Procedure + +. For each bare-metal host that exists in a cluster, in the `install-config.yaml` file for your cluster define a `networkConfig` section similar to the following example: ++ +[source,yaml] +---- +apiVersion: v1 +kind: InstallConfig +metadata: + name: +# ... +networkConfig: + interfaces: + - name: enp1s0 <1> + type: ethernet + state: up + ipv4: + dhcp: true + enabled: true + ipv6: + enabled: false + - name: enp2s0 <2> + type: ethernet + state: up + mtu: 1500 <3> + ipv4: + dhcp: true + enabled: true + ipv6: + dhcp: true + enabled: true + - name: enp3s0 <4> + type: ethernet + state: up + mtu: 1500 + ipv4: + enabled: false + ipv6: + enabled: false +# ... +---- +<1> The interface for the provisioned network interface controller (NIC). +<2> The first bonded interface that pulls in the Ignition config file for the bond interface. +<3> Manually set the `br-ex` maximum transmission unit (MTU) on the bond ports. +<4> The second bonded interface is part of a minimal configuration that pulls ignition during cluster installation. + +. Define each network interface in an NMState configuration file: ++ +.Example NMState configuration file that defines many network interfaces +[source,yaml] +---- +apiVersion: nmstate.io/v1 +kind: NodeNetworkConfigurationPolicy +metadata: +# ... +ovn: + bridge-mappings: + - localnet: localnet-network + bridge: br-ex + state: present +interfaces: + - name: br-ex + type: ovs-bridge + state: up + bridge: + allow-extra-patch-ports: true + port: + - name: br-ex + - name: patch-ex-to-phy + ovs-db: + external_ids: + bridge-uplink: "patch-ex-to-phy" + - name: br-ex + type: ovs-interface + state: up + mtu: 1500 <1> + ipv4: + enabled: true + dhcp: true + auto-route-metric: 48 + ipv6: + enabled: false + dhcp: false + auto-route-metric: 48 + - name: br-phy + type: ovs-bridge + state: up + bridge: + allow-extra-patch-ports: true + port: + - name: patch-phy-to-ex + - name: ovs-bond + link-aggregation: + mode: balance-slb + port: + - name: enp2s0 + - name: enp3s0 + - name: patch-ex-to-phy + type: ovs-interface + state: up + patch: + peer: patch-phy-to-ex + - name: patch-phy-to-ex + type: ovs-interface + state: up + patch: + peer: patch-ex-to-phy + - name: enp1s0 + type: ethernet + state: up + ipv4: + dhcp: true + enabled: true + ipv6: + enabled: false + - name: enp2s0 + type: ethernet + state: up + mtu: 1500 + ipv4: + enabled: false + ipv6: + enabled: false + - name: enp3s0 + type: ethernet + state: up + mtu: 1500 + ipv4: + enabled: false + ipv6: + enabled: false +# ... +---- +<1> Manually set the `br-ex` MTU on the bond ports. + +. Use the `base64` command to encode the interface content of the NMState configuration file: ++ +[source,terminal] +---- +$ base64 -w0 .yml <1> +---- +<1> Where the `-w0` option prevents line wrapping during the base64 encoding operation. + +. Create `MachineConfig` manifest files for the `master` role and the `worker` role. Ensure that you embed the base64-encoded string from an earlier command into each `MachineConfig` manifest file. The following example manifest file configures the `master` role for all nodes that exist in a cluster. You can also create a manifest file for `master` and `worker` roles specific to a node. ++ +[source,yaml] +---- +apiVersion: machineconfiguration.openshift.io/v1 +kind: MachineConfig +metadata: + labels: + machineconfiguration.openshift.io/role: master + name: 10-br-ex-master <1> +spec: + config: + ignition: + version: 3.2.0 + storage: + files: + - contents: + source: data:text/plain;charset=utf-8;base64, <2> + mode: 0644 + overwrite: true + path: /etc/nmstate/openshift/cluster.yml <3> +---- +<1> The name of the policy. +<2> Writes the encoded base64 information to the specified path. +<3> Specify the path to the `cluster.yml` file. For each node in your cluster, you can specify the short hostname path to your node, such as ``.yml. + +. Save each `MachineConfig` manifest file to the `.//manifests` directory, where `` is the directory in which the installation program creates files. ++ +The Machine Config Operator (MCO) takes the content from each manifest file and consistently applies the content to all selected nodes during a rolling update. + diff --git a/modules/virt-example-slb-secondary-bond.adoc b/modules/virt-example-slb-secondary-bond.adoc new file mode 100644 index 0000000000..c68a3f9875 --- /dev/null +++ b/modules/virt-example-slb-secondary-bond.adoc @@ -0,0 +1,67 @@ +// Module included in the following assemblies: +// +// * networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc + +:_mod-docs-content-type: PROCEDURE +[id="virt-example-slb-secondary-bond_{context}"] += Example: Source load balancing (SLB) on a secondary bond interface + +You can run the Open vSwitch (OVS) `balance-slb` mode on a secondary bond interface so to isolate traffic from the primary network. This configuration for managing traffic provides your cluster with high availability and traffic segmentation capabilities. + +The following YAML file shows an example of a manifest for a secondary bond interface. The file includes samples values that you must replace with your own information. + +[source,yaml] +---- +apiVersion: nmstate.io/v1 +kind: NodeNetworkConfigurationPolicy +metadata: + name: bond1-policy <1> +spec: + nodeSelector: + kubernetes.io/hostname: + desiredState: + interfaces: + - name: br1 + type: ovs-interface <2> + state: up + copy-mac-from: enp2s0 <3> + ipv4: + enabled: true + dhcp: true + - name: br1 + type: ovs-bridge + state: up + ipv4: + enabled: false + dhcp: false + bridge: + port: + - name: br1 <4> + - name: bond0 + link-aggregation: + mode: balance-slb <5> + port: <6> + - name: enp2s0 + - name: enp3s0 + - name: enp2s0 + type: ethernet + state: up + ipv4: + enabled: false + ipv6: + enabled: false + - name: enp3s0 + type: ethernet + state: up + ipv4: + enabled: false + ipv6: + enabled: false +# ... +---- +<1> Name of the policy. +<2> Specifies the virtual interface managed by Open vSwitch (OVS). +<3> Ensures the virtual interface inherits the MAC address of the physical network interface controller (NIC), `enp2s0`. +<4> The name of the OVS bridge. Note that the `br0` interface name is reserved so you cannot use that name with NMstate configurations. +<5> Sets the bonding mode to SLB, so that distribution of traffic happens based on the source MAC address and the VLAN. +<6> Lists physical interfaces that get included in the secondary bond interface. diff --git a/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc b/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc index f780481808..87209c3ef5 100644 --- a/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc +++ b/networking/k8s_nmstate/k8s-nmstate-updating-node-network-config.adoc @@ -91,6 +91,9 @@ include::modules/virt-example-vlan-nncp.adoc[leveloffset=+2] include::modules/virt-example-bond-nncp.adoc[leveloffset=+2] +// Example: Source load balancing (SLB) on a secondary bond interface +include::modules/virt-example-slb-secondary-bond.adoc[leveloffset=+2] + // Example: Multiple interfaces in the same node network configuration policy include::modules/virt-example-nmstate-multiple-interfaces.adoc[leveloffset=+2]