Merge pull request #20552 from openshift-cherrypick-robot/cherry-pick-20039-to-enterprise-4.4

[enterprise-4.4] CNV3936 - nmstate report and configure

_topic_map.yml

@@ -1451,6 +1451,14 @@ Topics:
     File: cnv-setting-node-maintenance
   - Name: Resuming a node from maintenance mode
     File: cnv-resuming-node
+# Node Networking
+- Name: Node networking
+  Dir: cnv_node_network
+  Topics:
+  - Name: Observing node network state
+    File: cnv-observing-node-network-state
+  - Name: Updating node network configuration
+    File: cnv-updating-node-network-config
 # Logging, events, and monitoring
 - Name: Logging, events, and monitoring
   Dir: cnv_logging_events_monitoring

cnv/cnv_node_network/cnv-observing-node-network-state.adoc

@@ -0,0 +1,14 @@
+[id="cnv-observing-node-network-state"]
+= Observing node network state
+include::modules/cnv-document-attributes.adoc[]
+:context: cnv-observing-node-network-state
+toc::[]
+
+Node network state is the network configuration for all nodes in the cluster. 
+
+include::modules/cnv-about-nmstate.adoc[leveloffset=+1]
+
+include::modules/cnv-viewing-network-state-of-node.adoc[leveloffset=+1]
+
+
+

cnv/cnv_node_network/cnv-updating-node-network-config.adoc

@@ -0,0 +1,25 @@
+[id="cnv-updating-node-network"]
+= Updating node network configuration
+include::modules/cnv-document-attributes.adoc[]
+:context: cnv-updating-node-network-config 
+toc::[]
+
+You can update the node network configuration, such as adding or removing interfaces
+from nodes, by applying `NodeNetworkConfigurationPolicy` manifests to the cluster.
+
+include::modules/cnv-about-nmstate.adoc[leveloffset=+1]
+
+include::modules/cnv-creating-interface-on-nodes.adoc[leveloffset=+1]
+
+include::modules/cnv-confirming-policy-updates-on-nodes.adoc[leveloffset=+1]
+
+include::modules/cnv-removing-interface-from-nodes.adoc[leveloffset=+1]
+
+include::modules/cnv-restoring-node-network-configuration.adoc[leveloffset=+1]
+
+include::modules/cnv-example-bridge-nncp.adoc[leveloffset=+1]
+
+include::modules/cnv-example-vlan-nncp.adoc[leveloffset=+1]
+
+include::modules/cnv-example-bond-nncp.adoc[leveloffset=+1]
+

cnv/cnv_node_network/images

@@ -0,0 +1 @@
+../../images/

cnv/cnv_node_network/modules

@@ -0,0 +1 @@
+../../modules/

cnv/cnv_virtual_machines/cnv_vm_networking/cnv-attaching-vm-multiple-networks.adoc

@@ -16,6 +16,11 @@ include::modules/cnv-networking-glossary.adoc[leveloffset=+1]
 
 == Creating a NetworkAttachmentDefinition
 
+.Prerequisites
+
+* A Linux bridge must be configured and attached on every node. 
+See the xref:../../../cnv/cnv_node_network/cnv-updating-node-network-config.adoc#cnv-about-nmstate_cnv-updating-node-network-config[node networking] section for more information.
+
 include::modules/cnv-creating-bridge-nad-web.adoc[leveloffset=+2]
 
 include::modules/cnv-creating-bridge-nad-cli.adoc[leveloffset=+2]

modules/cnv-about-nmstate.adoc

@@ -0,0 +1,14 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-observing-node-network-state.adoc
+
+[id="cnv-about-nmstate_{context}"]
+= About nmstate
+
+{CNVProductNameStart} uses link:https://nmstate.github.io/[`nmstate`] to report on and configure the state of the node network. This makes it possible to modify network policy configuration, such as by creating a Linux bridge on all nodes, by applying a single configuration manifest to the cluster.
+
+Node networking is monitored and updated by the following objects:
+
+`NodeNetworkState`:: Reports the state of the network on that node.
+`NodeNetworkConfigurationPolicy`:: Describes the requested network configuration on nodes. You update the node network configuration, including adding and removing interfaces, by applying a `NodeNetworkConfigurationPolicy` manifest to the cluster. 
+`NodeNetworkConfigurationEnactment`:: Reports the network policies enacted upon each node.

modules/cnv-confirming-policy-updates-on-nodes.adoc

@@ -0,0 +1,40 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-configuring-node-network-policy
+
+[id="cnv-confirming-policy-updates-on-nodes_{context}"]
+= Confirming Policy updates on nodes
+
+A `NodeNetworkConfigurationPolicy` manifest describes your requested network configuration for nodes in the cluster.
+The Policy object includes your requestd network configuration and the status of execution of the Policy on the cluster as a whole.
+
+
+When you apply a Policy, a `NodeNetworkConfigurationEnactment` is created for every node in the cluster. The Enactment is a read-only object that represents the status of execution of the Policy on that node. 
+If the Policy fails to be applied on the node, the Enactment for that node includes a traceback for troubleshooting.
+
+.Procedure
+
+. To confirm that a Policy has been applied to the cluster, list the Policies and their status:
++
+----
+$ oc get nncp
+----
+
+. (Optional) If a Policy is taking longer than expected to successfully configure, you can inspect the requested state and status conditions of a particular Policy:
++
+----
+$ oc get nncp <policy> -o yaml
+----
+
+. (Optional) If a policy is taking longer than expected to successfully configure on all nodes, you can list the status of the Enactments on the cluster:
++
+----
+$ oc get nnce
+----
+
+. (Optional) To view the configuration of a particular Enactment, including any error reporting for a failed configuration:
++
+----
+$ oc get nnce <node>.<policy> -o yaml
+----
+

modules/cnv-creating-bridge-nad-cli.adoc

@@ -8,15 +8,6 @@
 As a network administrator, you can configure a NetworkAttachmentDefinition
 of type `cnv-bridge` to provide Layer-2 networking to Pods and virtual machines.
 
-.Prerequisites
-
-* {CNVProductNameStart} 2.0 or newer
-* A Linux bridge must be configured and attached to the correct
-Network Interface Card on every node.
-* If you use VLANs, `vlan_filtering` must be enabled on the bridge.
-* The NIC must be tagged to all relevant VLANs.
-** For example: `bridge vlan add dev bond0 vid 1-4095 master`
-
 .Procedure
 
 . Create a new file for the NetworkAttachmentDefinition in any local directory.

modules/cnv-creating-bridge-nad-web.adoc

@@ -1,6 +1,6 @@
 // Module included in the following assemblies:
 //
-// * cnv-attaching-multiple-networks.adoc
+// * cnv/cnv_virtual_machines/cnv_vm_networking/cnv-attaching-vm-multiple-networks.adoc
 
 [id="cnv-creating-bridge-nad-web_{context}"]
 = Creating a Linux bridge NetworkAttachmentDefinition in the web console
@@ -11,15 +11,6 @@ to a specific namespace in your {CNVProductName} cluster.
 Network administrators can create NetworkAttachmentDefinitions 
 to provide existing layer-2 networking to Pods and virtual machines. 
 
-.Prerequisites
-
-* Container-native virtualization 2.2 or above installed on your cluster.
-* A Linux bridge must be configured and attached to the correct
-Network Interface Card (NIC) on every node.
-* If you use VLANs, `vlan_filtering` must be enabled on the bridge.
-* The NIC must be tagged to all relevant VLANs.
-** For example: `bridge vlan add dev bond0 vid 1-4095 master`
-
 .Procedure
 
 . In the web console, click *Networking* -> *Network Attachment Definitions*.

modules/cnv-creating-interface-on-nodes.adoc

@@ -0,0 +1,52 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-updating-node-network-config.adoc
+
+[id="cnv-creating-interface-on-nodes_{context}"]
+= Creating an interface on nodes
+
+Create an interface on nodes in the cluster by applying a `NodeNetworkConfigurationPolicy` manifest to the cluster. The manifest details the requested configuration for the interface. 
+
+By default, the manifest applies to all nodes in the cluster. To add the interface to specific nodes, add the `spec: nodeSelector` parameter and the appropriate `<key>:<value>` for your node selector.
+
+.Procedure
+
+. Create the `NodeNetworkConfigurationPolicy` manifest. The following example configures a Linux bridge on all worker nodes:
++
+[source,yaml]
+----
+apiVersion: nmstate.io/v1alpha1
+kind: NodeNetworkConfigurationPolicy
+metadata:
+  name: <br1-eth1-policy> <1>
+spec:
+  nodeSelector: <2>
+    node-role.kubernetes.io/worker: "" <3>
+  desiredState:
+    interfaces:
+      - name: br1
+        description: Linux bridge with eth1 as a port <4>
+        type: linux-bridge
+        state: up
+        ipv4:
+          dhcp: true
+          enabled: true
+        bridge:
+          options:
+            stp:
+              enabled: false
+          port:
+            - name: eth1
+----
+<1> Name of the Policy. 
+<2> Optional. If you do not include the `nodeSelector`, the Policy applies to all nodes in the cluster.
+<3> This example uses the `node-role.kubernetes.io/worker: ""` node selector to select all worker nodes in the cluster.
+<4> Optional. Human-readable description for the interface.
+
+. Create the Policy:
++
+----
+$ oc apply -f <br1-eth1-policy.yaml> <1>
+----
+<1> File name of the Policy manifest.
+

modules/cnv-example-bond-nncp.adoc

@@ -0,0 +1,53 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-updating-node-network-config.adoc
+
+[id="cnv-example-bond-nncp_{context}"]
+= Example: Bond interface NodeNetworkConfigurationPolicy
+
+Create a bond interface on nodes in the cluster by applying a `NodeNetworkConfigurationPolicy` manifest
+to the cluster. 
+
+The following YAML file is an example of a manifest for a bond interface.
+It includes samples values that you must replace with your own information.
+
+[source,yaml]
+----
+apiVersion: nmstate.io/v1alpha1
+kind: NodeNetworkConfigurationPolicy
+metadata:
+  name: bond0-eth1-eth2-policy <1>
+spec:
+  nodeSelector: <2>
+    kubernetes.io/hostname: <node01> <3>
+  desiredState:
+    interfaces:
+    - name: bond0 <4>
+      description: Bond enslaving eth1 and eth2 <5>
+      type: bond <6>
+      state: up <7>
+      ipv4:
+        dhcp: true <8>
+        enabled: true <9>
+      link-aggregation:
+        mode: balance-rr <10>
+        options:
+          miimon: '140' <11>
+        slaves: <12>
+        - eth1 
+        - eth2
+      mtu: 1450 <13>
+----
+<1> Name of the Policy. 
+<2> Optional. If you do not include the `nodeSelector`, the Policy applies to all nodes in the cluster.
+<3> This example uses a `hostname` node selector.
+<4> Name of the interface.
+<5> Optional. Human-readable description of the interface.
+<6> The type of interface. This example creates a bond. 
+<7> The requested state for the interface after creation.
+<8> Optional. If you do not use `dhcp` you must specify static IP address for the interface.
+<9> Enables `ipv4` in this example.
+<10> The driver mode for the bond. This example uses a round-robin mode.
+<11> Optional. This example uses miimon to inspect the bond link every 140ms.
+<12> The subordinate node NICs in the bond. 
+<13> Optional. The maximum transmission unit (MTU) for the bond. If not specified, this value is set to `1500` by default. 

modules/cnv-example-bridge-nncp.adoc

@@ -0,0 +1,49 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-updating-node-network-config.adoc
+
+[id="cnv-example-bridge-nncp_{context}"]
+= Example: Linux bridge interface NodeNetworkConfigurationPolicy
+
+Create a Linux bridge interface on nodes in the cluster by applying a `NodeNetworkConfigurationPolicy` manifest
+to the cluster. 
+
+The following YAML file is an example of a manifest for a Linux bridge interface.
+It includes samples values that you must replace with your own information.
+
+[source,yaml]
+----
+apiVersion: nmstate.io/v1alpha1
+kind: NodeNetworkConfigurationPolicy
+metadata:
+  name: br1-eth1-policy <1>
+spec:
+  nodeSelector: <2>
+    kubernetes.io/hostname: <node01> <3>
+  desiredState:
+    interfaces:
+      - name: br1 <4>
+        description: Linux bridge with eth1 as a port <5>
+        type: linux-bridge <6>
+        state: up <7>
+        ipv4:
+          dhcp: true <8>
+          enabled: true <9>
+        bridge:
+          options:
+            stp:
+              enabled: false <10>
+          port:
+            - name: eth1 <11>
+----
+<1> Name of the Policy. 
+<2> Optional. If you do not include the `nodeSelector`, the Policy applies to all nodes in the cluster.
+<3> This example uses a `hostname` node selector.
+<4> Name of the interface.
+<5> Optional. Human-readable description of the interface.
+<6> The type of interface. This example creates a bridge.
+<7> The requested state for the interface after creation.
+<8> Optional. If you do not use `dhcp` you must specify static IP address for the interface.
+<9> Enables `ipv4` in this example.
+<10> Disables `stp` in this example.
+<11> The node NIC to which the bridge attaches.

modules/cnv-example-vlan-nncp.adoc

@@ -0,0 +1,41 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-updating-node-network-config.adoc
+
+[id="cnv-example-vlan-nncp_{context}"]
+= Example: VLAN interface NodeNetworkConfigurationPolicy
+
+Create a VLAN interface on nodes in the cluster by applying a `NodeNetworkConfigurationPolicy` manifest
+to the cluster. 
+
+The following YAML file is an example of a manifest for a VLAN interface.
+It includes samples values that you must replace with your own information.
+
+[source,yaml]
+----
+apiVersion: nmstate.io/v1alpha1
+kind: NodeNetworkConfigurationPolicy
+metadata:
+  name: vlan-eth1-policy <1>
+spec:
+  nodeSelector: <2>
+    kubernetes.io/hostname: <node01> <3>
+  desiredState:
+    interfaces:
+    - name: eth1.102 <4>
+      description: VLAN using eth1 <5>
+      type: vlan <6>
+      state: up <7>
+      vlan:
+        base-iface: eth1 <8>
+        id: 102 <9>
+----
+<1> Name of the Policy. 
+<2> Optional. If you do not include the `nodeSelector`, the Policy applies to all nodes in the cluster.
+<3> This example uses a `hostname` node selector.
+<4> Name of the interface.
+<5> Optional. Human-readable description of the interface.
+<6> The type of interface. This example creates a VLAN.
+<7> The requested state for the interface after creation.
+<8> The node NIC to which the VLAN is attached. 
+<9> The VLAN tag. 

modules/cnv-removing-interface-from-nodes.adoc

@@ -0,0 +1,48 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-updating-node-network-config.adoc
+
+[id="cnv-removing-interface-from-nodes_{context}"]
+= Removing an interface from nodes
+
+Remove an interface from nodes by editing the `NodeNetworkConfigurationPolicy` object and set
+the `state` of the interface to `absent`.
+
+[NOTE]
+====
+Deleting the Policy that added an interface does not change the configuration of the network policy on the node.
+Although a `NodeNetworkConfigurationPolicy` is an object in the cluster, it only represents the requested configuration. +
+Similarly, removing an interface does not delete the Policy.
+====
+
+.Procedure
+
+. Update the `NodeNetworkConfigurationPolicy` manifest used to create the interface. The following example removes a Linux bridge:
++
+[source,yaml]
+----
+apiVersion: nmstate.io/v1alpha1
+kind: NodeNetworkConfigurationPolicy
+metadata:
+  name: <br1-eth1-policy> <1>
+spec:
+  nodeSelector: <2>
+    node-role.kubernetes.io/worker: "" <3>
+  desiredState:
+    interfaces:
+      - name: br1
+        type: linux-bridge
+        state: absent <4>
+----
+<1> Name of the Policy. 
+<2> Optional. If you do not include the `nodeSelector`, the Policy applies to all nodes in the cluster.
+<3> This example uses the `node-role.kubernetes.io/worker: ""` node selector to select all worker nodes in the cluster.
+<4> Changing the state to `absent` removes the interface.
+
+. Update the  Policy on the node and remove the interface:
++
+----
+$ oc apply -f <br1-eth1-policy.yaml> <1>
+----
+<1> File name of the Policy manifest.
+

modules/cnv-restoring-node-network-configuration.adoc

@@ -0,0 +1,38 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-updating-node-network-config.adoc
+
+[id="cnv-restoring-node-network-configuration_{context}"]
+= Restoring node network configuration after removing an interface
+
+Removing an interface from a node does not automatically restore the node network configuration to a previous state. After you remove an interface, any of the node NICs throughout the cluster that were previously attached or subordinate to the interface are placed in a `down` state. Restore the NICs by applying a new `NodeNetworkConfigurationPolicy` manifest to the cluster.
+
+.Procedure
+
+. Create a `NodeNetworkConfigurationPolicy` manifest that specifies the NIC and the desired state of `up`:
++
+[source,yaml]
+----
+apiVersion: nmstate.io/v1alpha1
+kind: NodeNetworkConfigurationPolicy
+metadata:
+  name: eth1
+spec:
+  desiredState:
+    interfaces:
+    - name: eth1
+      type: ethernet
+      state: up
+      ipv4:
+        dhcp: true
+        enabled: true
+----
+
+. Apply the manifest to the cluster:
++
+----
+$ oc apply -f <eth1.yaml> <1>
+----
+<1> File name of the Policy manifest.
+
+

modules/cnv-viewing-network-state-of-node.adoc

@@ -0,0 +1,45 @@
+// Module included in the following assemblies:
+//
+// * cnv/cnv_node_network/cnv-observing-node-network-state.adoc
+
+[id="cnv-viewing-network-state-of-node_{context}"]
+= Viewing the network state of a node
+
+A `NodeNetworkState` object exists on every node in the cluster. This object is periodically updated and captures the state of the network for that node. 
+
+.Procedure
+
+. List all the `NodeNetworkState` objects in the cluster:
++
+----
+$ oc get nns
+----
+
+. Inspect a `NodeNetworkState` to view the network on that node. The output in this example has been redacted for clarity:
++
+----
+$ oc get nns node01 -o yaml
+----
++
+[source,yaml]
+----
+apiVersion: nmstate.io/v1alpha1
+kind: NodeNetworkState
+metadata:
+  name: node01 <1>
+status:
+  currentState: <2>
+    dns-resolver:
+...
+    interfaces:
+...
+    route-rules:
+...
+    routes:
+...
+  lastSuccessfulUpdateTime: "2020-01-31T12:14:00Z" <3>
+----
+<1> The name of the `NodeNetworkState` is taken from the node.
+<2> The `currentState` contains the complete network configuration for the node, including DNS, interfaces, and routes.
+<3> Timestamp of the last successful update. This is updated periodically as long as the node is reachable and can be used to evalute the freshness of the report. 
+