1
0
mirror of https://github.com/openshift/openshift-docs.git synced 2026-02-05 12:46:18 +01:00

OSDOCS-8895: introduce network config options and limits

This commit is contained in:
“Shauna Diaz”
2023-12-04 12:20:15 -05:00
committed by openshift-cherrypick-robot
parent dbec27a145
commit 83efe065ee
8 changed files with 238 additions and 134 deletions

View File

@@ -10,8 +10,12 @@ A YAML file customizes {microshift-short} instances with your preferences, setti
include::snippets/microshift-greenboot-status-snip.adoc[leveloffset=+2]
include::modules/microshift-default-settings.adoc[leveloffset=+1]
include::modules/microshift-config-yaml.adoc[leveloffset=+1]
include::modules/microshift-nw-advertise-address.adoc[leveloffset=+2]
include::modules/microshift-config-nodeport-limits.adoc[leveloffset=+2]
[id="additional-resources_microshift-using-config-tools_{context}"]

View File

@@ -6,49 +6,50 @@ include::_attributes/attributes-microshift.adoc[]
toc::[]
OVN-Kubernetes is the default networking solution for {microshift-short} deployments. OVN-Kubernetes is a virtualized network for pods and services that is based on Open Virtual Network (OVN). The OVN-Kubernetes Container Network Interface (CNI) plugin is the network plugin for the cluster. A cluster that uses the OVN-Kubernetes network plugin also runs Open vSwitch (OVS) on the node. OVN configures OVS on the node to implement the declared network configuration.
The OVN-Kubernetes Container Network Interface (CNI) plugin is the default networking solution for {microshift-short} clusters. OVN-Kubernetes is a virtualized network for pods and services that is based on Open Virtual Network (OVN).
[id="microshift-network-topology_{context}"]
== Network topology
OVN-Kubernetes provides an overlay-based networking implementation. This overlay includes an OVS-based implementation of Service and NetworkPolicy. The overlay network uses the Geneve (Generic Network Virtualization Encapsulation) tunnel protocol. The pod maximum transmission unit (MTU) for the Geneve tunnel is set to a smaller value than the MTU of the physical interface on the host. This smaller MTU makes room for the required information that is added to the tunnel header before it is transmitted.
* Default network configuration and connections are applied automatically in {microshift-short} with the `microshift-networking` RPM during installation.
* A cluster that uses the OVN-Kubernetes network plugin also runs Open vSwitch (OVS) on the node.
* OVN-K configures OVS on the node to implement the declared network configuration.
* Host physical interfaces are not bound by default to the OVN-K gateway bridge, `br-ex`. You can use standard tools on the host for managing the default gateway, such as the Network Manager CLI (`nmcli`).
* Changing the CNI is not supported on {microshift-short}.
OVS runs as a systemd service on the {microshift-short} node. The OVS RPM package is installed as a dependency to the `microshift-networking` RPM package. OVS is started immediately when the `microshift-networking` RPM is installed.
Using configuration files or custom scripts, you can configure the following networking settings:
.{product-title} network topology
image:317_RHbM_OVN_topology_0923.png[title="{microshift-short} uses an overlay-based networking implementation, details follow."]
* You can use subnet CIDR ranges to allocate IP addresses to pods.
* You can change the maximum transmission unit (MTU) value.
* You can configure firewall ingress and egress.
* You can define network policies in the {microshift-short} cluster, including ingress and egress rules.
[id="microshift-description-ovn-logical-components_{context}"]
=== Description of the OVN logical components of the virtualized network
OVN node switch::
A virtual switch named `<node-name>`. The OVN node switch is named according to the hostname of the node.
** In this example, the `node-name` is `microshift-dev`.
include::modules/microshift-cni-customization-matrix.adoc[leveloffset=+1]
OVN cluster router::
A virtual router named `ovn_cluster_router`, also known as the distributed router.
** In this example, the cluster network is `10.42.0.0/16`.
include::modules/microshift-default-settings.adoc[leveloffset=+2]
OVN join switch::
A virtual switch named `join`.
[id="microshift-network-features_{context}"]
== Network features
Networking features available with {microshift-short} {product-version} include:
OVN gateway router::
A virtual router named `GR_<node-name>`, also known as the external gateway router.
* Kubernetes network policy
* Dynamic node IP
* Custom gateway interface
* Second gateway interface
* Cluster network on specified host interface
* Blocking external access to NodePort service on specific host interfaces
OVN external switch::
A virtual switch named `ext_<node-name>.`
Networking features not available with {microshift-short} {product-version}:
[id="microshift-description-connections-network-topology_{context}"]
=== Description of the connections in the network topology figure
* The north-south traffic between the network service and the OVN external switch `ext_microshift-dev`, is provided through the host kernel by the gateway bridge `br-ex`.
* The OVN gateway router `GR_microshift-dev` is connected to the external network switch `ext_microshift-dev` through the logical router port 4. Port 4 is attached with the node IP address 192.168.122.14.
* The join switch `join` connects the OVN gateway router `GR_microshift-dev` to the OVN cluster router `ovn_cluster_router`. The IP address range is 100.62.0.0/16.
** The OVN gateway router `GR_microshift-dev` connects to the OVN join switch `join` through the logical router port 3. Port 3 attaches with the internal IP address 100.64.0.2.
** The OVN cluster router `ovn_cluster_router` connects to the join switch `join` through the logical router port 2. Port 2 attaches with the internal IP address 100.64.0.1.
* The OVN cluster router `ovn_cluster_router` connects to the node switch `microshift-dev` through the logical router port 1. Port 1 is attached with the OVN cluster network IP address 10.42.0.1.
* The east-west traffic between the pods and the network service is provided by the OVN cluster router `ovn_cluster_router` and the node switch `microshift-dev`. The IP address range is 10.42.0.0/24.
* The east-west traffic between pods is provided by the node switch `microshift-dev` without network address translation (NAT).
* The north-south traffic between the pods and the external network is provided by the OVN cluster router `ovn_cluster_router` and the host network. This router is connected through the `ovn-kubernetes` management port `ovn-k8s-mp0`, with the IP address 10.42.0.2.
* All the pods are connected to the OVN node switch through their interfaces.
** In this example, Pod 1 and Pod 2 are connected to the node switch through `Interface 1` and `Interface 2`.
* Egress IP/firewall/QoS: disabled
* Hybrid networking: not supported
* IPsec: not supported
* Hardware offload: not supported
[id="_additional-resources_microshift-cni_{context}"]
[role="_additional-resources"]
.Additional resources
* xref:../microshift_configuring/microshift-using-config-tools.adoc#microshift-using-config-tools_microshift-config-yaml[Using a YAML configuration file]
* xref:../microshift_networking/microshift-networking-settings.adoc#microshift-config-OVN-K_microshift-networking[Understanding networking settings]
[id="microshift-ip-forward_{context}"]
== IP forward
@@ -62,53 +63,10 @@ By default, three performance optimizations are applied to OVS services to minim
* `no-mlockall` to `openvswitch.service`
* Limit handler and `revalidator` threads to `ovs-vswitchd.service`
[id="microshift-network-features_{context}"]
== Network features
Networking features available with {product-title} {product-version} include:
* Kubernetes network policy
* Dynamic node IP
* Cluster network on specified host interface
Networking features not available with {product-title} {product-version}:
* Egress IP/firewall/qos: disabled
* Hybrid networking: not supported
* IPsec: not supported
* Hardware offload: not supported
//Q: are there immutable network settings we should tell users about?
[id="microshift-network-comps-svcs_{context}"]
== {microshift-short} networking components and services
This brief overview describes networking components and their operation in {microshift-short}. The `microshift-networking` RPM is a package that automatically pulls in any networking-related dependencies and systemd services to initialize networking, for example, the `microshift-ovs-init` systemd service.
NetworkManager::
NetworkManager is required to set up the initial gateway bridge on the {microshift-short} node. The NetworkManager and `NetworkManager-ovs` RPM packages are installed as dependencies to the `microshift-networking` RPM package, which contains the necessary configuration files. NetworkManager in {microshift-short} uses the `keyfile` plugin and is restarted after installation of the `microshift-networking` RPM package.
microshift-ovs-init::
The `microshift-ovs-init.service` is installed by the `microshift-networking` RPM package as a dependent systemd service to microshift.service. It is responsible for setting up the OVS gateway bridge.
OVN containers::
Two OVN-Kubernetes daemon sets are rendered and applied by {microshift-short}.
* *ovnkube-master*
Includes the `northd`, `nbdb`, `sbdb` and `ovnkube-master` containers.
* *ovnkube-node*
The ovnkube-node includes the OVN-Controller container.
+
After {microshift-short} boots, the OVN-Kubernetes daemon sets are deployed in the `openshift-ovn-kubernetes` namespace.
Packaging::
OVN-Kubernetes manifests and startup logic are built into {microshift-short}. The systemd services and configurations included in `microshift-networking` RPM are:
* `/etc/NetworkManager/conf.d/microshift-nm.conf` for NetworkManager.service
* `/etc/systemd/system/ovs-vswitchd.service.d/microshift-cpuaffinity.conf` for ovs-vswitchd.service
* `/etc/systemd/system/ovsdb-server.service.d/microshift-cpuaffinity.conf`for ovs-server.service
* `/usr/bin/configure-ovs-microshift.sh` for microshift-ovs-init.service
* `/usr/bin/configure-ovs.sh` for microshift-ovs-init.service
* `/etc/crio/crio.conf.d/microshift-ovn.conf` for CRI-O service
include::modules/microshift-nw-components-svcs.adoc[leveloffset=+1]
[id="microshift-bridge-mapping_{context}"]
== Bridge mappings
Bridge mappings allow provider network traffic to reach the physical network. Traffic leaves the provider network and arrives at the `br-int` bridge. A patch port between `br-int` and `br-ex` then allows the traffic to traverse to and from the provider network and the edge network. Kubernetes pods are connected to the `br-int` bridge through virtual ethernet pair: one end of the virtual ethernet pair is attached to the pod namespace, and the other end is attached to the `br-int` bridge.
include::modules/microshift-nw-topology.adoc[leveloffset=+1]

View File

@@ -0,0 +1,48 @@
:_mod-docs-content-type: REFERENCE
[id="microshift-nw-customization-matrix_{context}"]
= {microshift-short} networking customization matrix
The following table summarizes the status of networking features and capabilities that are either present as defaults, supported for configuration, or not available with the {microshift-short} service:
.{microshift-short} networking capabilities and customization status
[cols="50%,20%,30%",options="header"]
|===
|Network feature|Availability|Customization supported
|Advertise address|Yes|Yes ^[1]^
|Kubernetes network policy|Yes|Yes
|Kubernetes network policy logs|Not available|N/A
|Load balancing|Yes|Yes
|Multicast DNS|Yes|Yes ^[2]^
|Network proxies|Yes ^[3]^|CRI-O
|Network performance|Yes|MTU configuration
|Egress IPs|Not available|N/A
|Egress firewall|Not available|N/A
|Egress router|Not available|N/A
|Firewall|No ^[4]^|Yes
|Hardware offloading|Not available|N/A
|Hybrid networking|Not available|N/A
|IPsec encryption for intra-cluster communication|Not available|N/A
|IPv6|Not available ^[5]^|N/A
|===
1. If unset, the default value is set to the next immediate subnet after the service network. For example, when the service network is `10.43.0.0/16`, the `advertiseAddress` is set to `10.44.0.0/32`.
2. You can use the multicast DNS protocol (mDNS) to allow name resolution and service discovery within a Local Area Network (LAN) using multicast exposed on the `5353/UDP` port.
3. There is no built-in transparent proxying of egress traffic in {microshift-short}. Egress must be manually configured.
4. Setting up the firewalld service is supported by {op-system-ostree}.
5. IPv6 is not available in any configuration.

View File

@@ -6,65 +6,13 @@
[id="microshift-config-yaml_{context}"]
= Using a YAML configuration file
On start up, {microshift-short} searches the system-wide `/etc/microshift/` directory for a configuration file named `config.yaml`. To use custom configurations, you must create the configuration file and specify any settings that should override the defaults before starting {microshift-short}.
[id="microshift-yaml-default_{context}"]
== Default settings
If you do not create a `config.yaml` file, the default values are used. The following example shows the default configuration settings.
* You can use the following command to see the default values:
+
[source,terminal]
----
$ microshift show-config
----
+
.Default values example output in YAML form
[source,yaml]
----
dns:
baseDomain: microshift.example.com <1>
network:
clusterNetwork:
- 10.42.0.0/16 <2>
serviceNetwork:
- 10.43.0.0/16 <3>
serviceNodePortRange: 30000-32767 <4>
node:
hostnameOverride: "" <5>
nodeIP: "" <6>
apiServer:
advertiseAddress: 10.44.0.0 <7>
subjectAltNames: [] <8>
debugging:
logLevel: "Normal" <9>
----
<1> Base domain of the cluster. All managed DNS records will be subdomains of this base.
<2> A block of IP addresses from which Pod IP addresses are allocated.
<3> A block of virtual IP addresses for Kubernetes services.
<4> The port range allowed for Kubernetes services of type NodePort.
<5> The name of the node. The default value is the hostname.
<6> The IP address of the node. The default value is the IP address of the default route.
<7> A string that specifies the IP address from which the API server is advertised to members of the cluster. The default value is calculated based on the address of the service network.
<8> Subject Alternative Names for API server certificates.
<9> Log verbosity. Valid values for this field are `Normal`, `Debug`, `Trace`, or `TraceAll`.
On start up, {microshift-short} searches the system-wide `/etc/microshift/` directory for a configuration file named `config.yaml`. To use custom configurations, you must create the configuration file and specify any settings that are expected to override the defaults before starting {microshift-short}.
[id="microshift-yaml-custom_{context}"]
== Custom settings
To create custom configurations, you must create a `config.yaml` file in the `/etc/microshift/` directory, then change any settings that should override the defaults before starting or restarting {microshift-short}.
To create custom configurations, you must create a `config.yaml` file in the `/etc/microshift/` directory, and then change any settings that are expected to override the defaults before starting or restarting {microshift-short}.
[IMPORTANT]
====
Restart {microshift-short} after changing any configuration settings to have them take effect. The `config.yaml` file is read only when {microshift-short} starts.
====
[id="microshift-yaml-advertiseAddress_{context}"]
=== advertiseAddress
The `apiserver.advertiseAddress` flag specifies the IP address on which to advertise the API server to members of the cluster. This address must be reachable by the cluster. You can set a custom IP address here, but you must also add the IP address to a host interface. Customizing this parameter preempts {microshift-short} from adding a default IP address to the `br-ex` network interface.
[IMPORTANT]
====
If you customize the `advertiseAddress` IP address, make sure it is reachable by the cluster when {microshift-short} starts by adding the IP address to a host interface.
====
If unset, the default value is `10.44.0.0/32`. It will be set to the next immediate subnet after the service network. For example, when the service network is `10.43.0.0/16`, the `advertiseAddress` is set to `10.44.0.0/32`.

View File

@@ -0,0 +1,47 @@
// Module included in the following assemblies:
//
// * microshift_configuring/microshift-using-config-tools.adoc
// * microshift_networking/
:_mod-docs-content-type: CONCEPT
[id="microshift-yaml-default_{context}"]
= Default settings
If you do not create a `config.yaml` file, default values are used. The following example shows the default configuration settings.
* To see the default values, run the following command:
+
[source,terminal]
----
$ microshift show-config
----
+
.Default values example output in YAML form
[source,yaml]
----
dns:
baseDomain: microshift.example.com <1>
network:
clusterNetwork:
- 10.42.0.0/16 <2>
serviceNetwork:
- 10.43.0.0/16 <3>
serviceNodePortRange: 30000-32767 <4>
node:
hostnameOverride: "" <5>
nodeIP: "" <6>
apiServer:
advertiseAddress: 10.44.0.0/32 <7>
subjectAltNames: [] <8>
debugging:
logLevel: "Normal" <9>
----
<1> Base domain of the cluster. All managed DNS records will be subdomains of this base.
<2> A block of IP addresses from which Pod IP addresses are allocated.
<3> A block of virtual IP addresses for Kubernetes services.
<4> The port range allowed for Kubernetes services of type NodePort.
<5> The name of the node. The default value is the hostname.
<6> The IP address of the node. The default value is the IP address of the default route.
<7> A string that specifies the IP address from which the API server is advertised to members of the cluster. The default value is calculated based on the address of the service network.
<8> Subject Alternative Names for API server certificates.
<9> Log verbosity. Valid values for this field are `Normal`, `Debug`, `Trace`, or `TraceAll`.

View File

@@ -0,0 +1,16 @@
// Module included in the following assemblies:
//
// * microshift/using-config-tools.adoc
:_mod-docs-content-type: CONCEPT
[id="microshift-yaml-advertiseAddress_{context}"]
= Configuring the advertise address network flag
The `apiserver.advertiseAddress` flag specifies the IP address on which to advertise the API server to members of the cluster. This address must be reachable by the cluster. You can set a custom IP address here, but you must also add the IP address to a host interface. Customizing this parameter preempts {microshift-short} from adding a default IP address to the `br-ex` network interface.
[IMPORTANT]
====
If you customize the `advertiseAddress` IP address, make sure it is reachable by the cluster when {microshift-short} starts by adding the IP address to a host interface.
====
If unset, the default value is set to the next immediate subnet after the service network. For example, when the service network is `10.43.0.0/16`, the `advertiseAddress` is set to `10.44.0.0/32`.

View File

@@ -0,0 +1,35 @@
// Module included in the following assemblies:
//
// * microshift_networking/microshift-cni.adoc
:_mod-docs-content-type: CONCEPT
[id="microshift-network-comps-svcs_{context}"]
= {microshift-short} networking components and services
This brief overview describes networking components and their operation in {microshift-short}. The `microshift-networking` RPM is a package that automatically pulls in any networking-related dependencies and systemd services to initialize networking, for example, the `microshift-ovs-init` systemd service.
NetworkManager::
NetworkManager is required to set up the initial gateway bridge on the {microshift-short} node. The NetworkManager and `NetworkManager-ovs` RPM packages are installed as dependencies to the `microshift-networking` RPM package, which contains the necessary configuration files. NetworkManager in {microshift-short} uses the `keyfile` plugin and is restarted after installation of the `microshift-networking` RPM package.
microshift-ovs-init::
The `microshift-ovs-init.service` is installed by the `microshift-networking` RPM package as a dependent systemd service to `microshift.service`. It is responsible for setting up the OVS gateway bridge.
OVN containers::
Two OVN-Kubernetes daemon sets are rendered and applied by {microshift-short}.
* *ovnkube-master*
Includes the `northd`, `nbdb`, `sbdb` and `ovnkube-master` containers.
* *ovnkube-node*
The ovnkube-node includes the OVN-Controller container.
+
After {microshift-short} starts, the OVN-Kubernetes daemon sets are deployed in the `openshift-ovn-kubernetes` namespace.
Packaging::
OVN-Kubernetes manifests and startup logic are built into {microshift-short}. The systemd services and configurations included in the `microshift-networking` RPM are:
* `/etc/NetworkManager/conf.d/microshift-nm.conf` for `NetworkManager.service`
* `/etc/systemd/system/ovs-vswitchd.service.d/microshift-cpuaffinity.conf` for `ovs-vswitchd.service`
* `/etc/systemd/system/ovsdb-server.service.d/microshift-cpuaffinity.conf` for `ovs-server.service`
* `/usr/bin/configure-ovs-microshift.sh` for `microshift-ovs-init.service`
* `/usr/bin/configure-ovs.sh` for `microshift-ovs-init.service`
* `/etc/crio/crio.conf.d/microshift-ovn.conf` for the CRI-O service

View File

@@ -0,0 +1,48 @@
// Module included in the following assemblies:
//
// * microshift_networking/microshift-cni.adoc
:_mod-docs-content-type: CONCEPT
[id="microshift-network-topology_{context}"]
= Network topology
OVN-Kubernetes provides an overlay-based networking implementation. This overlay includes an OVS-based implementation of Service and NetworkPolicy. The overlay network uses the Geneve (Generic Network Virtualization Encapsulation) tunnel protocol. The pod maximum transmission unit (MTU) for the Geneve tunnel is set to the default route MTU if it is not configured.
To configure the MTU, you must must set an equal-to or less-than value than the MTU of the physical interface on the host. A less-than value for the MTU makes room for the required information that is added to the tunnel header before it is transmitted.
OVS runs as a systemd service on the {microshift-short} node. The OVS RPM package is installed as a dependency to the `microshift-networking` RPM package. OVS is started immediately when the `microshift-networking` RPM is installed.
.{product-title} network topology
image:317_RHbM_OVN_topology_0923.png[title="{microshift-short} uses an overlay-based networking implementation, details follow."]
[id="microshift-description-ovn-logical-components_{context}"]
== Description of the OVN logical components of the virtualized network
OVN node switch::
A virtual switch named `<node-name>`. The OVN node switch is named according to the hostname of the node.
** In this example, the `node-name` is `microshift-dev`.
OVN cluster router::
A virtual router named `ovn_cluster_router`, also known as the distributed router.
** In this example, the cluster network is `10.42.0.0/16`.
OVN join switch::
A virtual switch named `join`.
OVN gateway router::
A virtual router named `GR_<node-name>`, also known as the external gateway router.
OVN external switch::
A virtual switch named `ext_<node-name>.`
[id="microshift-description-connections-network-topology_{context}"]
== Description of the connections in the network topology figure
* The north-south traffic between the network service and the OVN external switch `ext_microshift-dev` is provided through the host kernel by the gateway bridge `br-ex`.
* The OVN gateway router `GR_microshift-dev` is connected to the external network switch `ext_microshift-dev` through the logical router port 4. Port 4 is attached with the node IP address 192.168.122.14.
* The join switch `join` connects the OVN gateway router `GR_microshift-dev` to the OVN cluster router `ovn_cluster_router`. The IP address range is 100.62.0.0/16.
** The OVN gateway router `GR_microshift-dev` connects to the OVN join switch `join` through the logical router port 3. Port 3 attaches with the internal IP address 100.64.0.2.
** The OVN cluster router `ovn_cluster_router` connects to the join switch `join` through the logical router port 2. Port 2 attaches with the internal IP address 100.64.0.1.
* The OVN cluster router `ovn_cluster_router` connects to the node switch `microshift-dev` through the logical router port 1. Port 1 is attached with the OVN cluster network IP address 10.42.0.1.
* The east-west traffic between the pods and the network service is provided by the OVN cluster router `ovn_cluster_router` and the node switch `microshift-dev`. The IP address range is 10.42.0.0/24.
* The east-west traffic between pods is provided by the node switch `microshift-dev` without network address translation (NAT).
* The north-south traffic between the pods and the external network is provided by the OVN cluster router `ovn_cluster_router` and the host network. This router is connected through the `ovn-kubernetes` management port `ovn-k8s-mp0`, with the IP address 10.42.0.2.
* All the pods are connected to the OVN node switch through their interfaces.
** In this example, Pod 1 and Pod 2 are connected to the node switch through `Interface 1` and `Interface 2`.