From 05eaeeb3073726f4330b9e167efcd0720838508b Mon Sep 17 00:00:00 2001 From: dfitzmau Date: Mon, 3 Mar 2025 17:26:03 +0000 Subject: [PATCH] OSDOCS-11692: Added a note for OVNK, NIC, JUMBO packets, MTU value --- modules/ipsec-impact-networking.adoc | 3 ++- modules/optimizing-mtu-networking.adoc | 11 +++++++++-- modules/recommended-install-practices.adoc | 11 +++++------ .../optimization/optimizing-networking.adoc | 7 ++++--- 4 files changed, 20 insertions(+), 12 deletions(-) diff --git a/modules/ipsec-impact-networking.adoc b/modules/ipsec-impact-networking.adoc index e735553f96..f0925b7b09 100644 --- a/modules/ipsec-impact-networking.adoc +++ b/modules/ipsec-impact-networking.adoc @@ -2,9 +2,10 @@ // // * scalability_and_performance/optimization/optimizing-networking.adoc +:_mod-docs-content-type: CONCEPT [id="ipsec-impact_{context}"] = Impact of IPsec Because encrypting and decrypting node hosts uses CPU power, performance is affected both in throughput and CPU usage on the nodes when encryption is enabled, regardless of the IP security system being used. -IPSec encrypts traffic at the IP payload level, before it hits the NIC, protecting fields that would otherwise be used for NIC offloading. This means that some NIC acceleration features might not be usable when IPSec is enabled and will lead to decreased throughput and increased CPU usage. +IPSec encrypts traffic at the IP payload level, before it hits the NIC, protecting fields that would otherwise be used for NIC offloading. This means that some NIC acceleration features might not be usable when IPSec is enabled and leads to decreased throughput and increased CPU usage. diff --git a/modules/optimizing-mtu-networking.adoc b/modules/optimizing-mtu-networking.adoc index 1db3018222..f945dbd556 100644 --- a/modules/optimizing-mtu-networking.adoc +++ b/modules/optimizing-mtu-networking.adoc @@ -2,11 +2,18 @@ // // * scalability_and_performance/optimization/optimizing-networking.adoc +:_mod-docs-content-type: CONCEPT [id="optimizing-mtu_{context}"] = Optimizing the MTU for your network There are two important maximum transmission units (MTUs): the network interface controller (NIC) MTU and the cluster network MTU. -The NIC MTU is configured at the time of {product-title} installation, and you can also change the cluster's MTU as a Day 2 operation. See "Changing cluster network MTU" for more information. The MTU must be less than or equal to the maximum supported value of the NIC of your network. If you are optimizing for throughput, choose the largest possible value. If you are optimizing for lowest latency, choose a lower value. +The NIC MTU is configured at the time of {product-title} installation, and you can also change the MTU of a cluster as a postinstallation task. For more information, see "Changing cluster network MTU". + +For a cluster that uses the OVN-Kubernetes plugin, the MTU must be less than `100` bytes to the maximum supported value of the NIC of your network. If you are optimizing for throughput, choose the largest possible value, such as `8900`. If you are optimizing for lowest latency, choose a lower value. + +[IMPORTANT] +==== +If your cluster uses the OVN-Kubernetes plugin and the network uses a NIC to send and receive unfragmented jumbo frame packets over the network, you must specify `9000` bytes as the MTU value for the NIC so that pods do not fail. +==== -For OVN and Geneve, the MTU must be less than the NIC MTU by 100 bytes at a minimum. diff --git a/modules/recommended-install-practices.adoc b/modules/recommended-install-practices.adoc index bc8d0f1103..4ec05bbef1 100644 --- a/modules/recommended-install-practices.adoc +++ b/modules/recommended-install-practices.adoc @@ -2,12 +2,13 @@ // // * scalability_and_performance/recommended-install-practices.adoc +:_mod-docs-content-type: REFERENCE [id="recommended-install-practices_{context}"] = Recommended practices for installing large scale clusters -When installing large clusters or scaling the cluster to larger node counts, -set the cluster network `cidr` accordingly in your `install-config.yaml` -file before you install the cluster: +When installing large clusters or scaling the cluster to larger node counts, set the cluster network `cidr` accordingly in your `install-config.yaml` file before you install the cluster. + +.Example `install-config.yaml` file with a network configuration for a cluster with a large node count [source,yaml] ---- @@ -22,6 +23,4 @@ networking: - 172.30.0.0/16 ---- -The default cluster network `cidr` `10.128.0.0/14` cannot be used if the cluster -size is more than 500 nodes. It must be set to `10.128.0.0/12` or -`10.128.0.0/10` to get to larger node counts beyond 500 nodes. +The default cluster network `cidr` `10.128.0.0/14` cannot be used if the cluster size is more than 500 nodes. The `cidr` must be set to `10.128.0.0/12` or `10.128.0.0/10` to get to larger node counts beyond 500 nodes. diff --git a/scalability_and_performance/optimization/optimizing-networking.adoc b/scalability_and_performance/optimization/optimizing-networking.adoc index ba9c44499e..4e79a1b971 100644 --- a/scalability_and_performance/optimization/optimizing-networking.adoc +++ b/scalability_and_performance/optimization/optimizing-networking.adoc @@ -10,9 +10,7 @@ xref:../../networking/ovn_kubernetes_network_provider/about-ovn-kubernetes.adoc# Geneve provides benefits over VLANs, such as an increase in networks from 4096 to over 16 million, and layer 2 connectivity across physical networks. This allows for all pods behind a service to communicate with each other, even if they are running on different systems. -Geneve encapsulates all tunneled traffic in user datagram protocol (UDP) packets. However, this leads to increased CPU utilization. Both these outer- and -inner-packets are subject to normal checksumming rules to guarantee data is not corrupted during transit. Depending on CPU performance, this additional -processing overhead can cause a reduction in throughput and increased latency when compared to traditional, non-overlay networks. +Geneve encapsulates all tunneled traffic in user datagram protocol (UDP) packets. However, this leads to increased CPU utilization. Both these outer- and inner-packets are subject to normal checksumming rules to guarantee data is not corrupted during transit. Depending on CPU performance, this additional processing overhead can cause a reduction in throughput and increased latency when compared to traditional, non-overlay networks. Cloud, VM, and bare metal CPU performance can be capable of handling much more than one Gbps network throughput. When using higher bandwidth links such as 10 or 40 Gbps, reduced performance can occur. This is a known issue in Geneve-based environments and is not specific to containers or {product-title}. Any network that relies on Geneve or VXLAN tunnels will perform similarly because of the tunnel implementation. @@ -23,6 +21,7 @@ If you are looking to push beyond one Gbps, you can: Geneve-offload does not reduce latency. However, CPU utilization is reduced even in latency tests. +// Optimizing the MTU for your network include::modules/optimizing-mtu-networking.adoc[leveloffset=+1] [role="_additional-resources"] @@ -30,8 +29,10 @@ include::modules/optimizing-mtu-networking.adoc[leveloffset=+1] * xref:../../networking/changing-cluster-network-mtu.adoc#changing-cluster-network-mtu[Changing cluster network MTU] +// Recommended practices for installing large scale clusters include::modules/recommended-install-practices.adoc[leveloffset=+1] +// Impact of IPsec include::modules/ipsec-impact-networking.adoc[leveloffset=+1] [role="_additional-resources"]