mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
132 lines
6.0 KiB
Plaintext
132 lines
6.0 KiB
Plaintext
// Module included in the following assemblies:
|
|
//
|
|
// * networking/hardware_networks/configuring-sriov-device.adoc
|
|
// * virt/vm_networking/virt-connecting-vm-to-sriov.adoc
|
|
// * virt/post_installation_configuration/virt-post-install-network-config.adoc
|
|
|
|
ifeval::["{context}" == "configuring-sriov-device"]
|
|
:ocp-sriov:
|
|
endif::[]
|
|
|
|
ifeval::["{context}" == "virt-connecting-vm-to-sriov"]
|
|
:virt-sriov:
|
|
endif::[]
|
|
ifeval::["{context}" == "virt-post-install-network-config"]
|
|
:virt-sriov:
|
|
endif::[]
|
|
|
|
:_mod-docs-content-type: PROCEDURE
|
|
[id="nw-sriov-configuring-device_{context}"]
|
|
= Configuring SR-IOV network devices
|
|
|
|
The SR-IOV Network Operator adds the `SriovNetworkNodePolicy.sriovnetwork.openshift.io` CustomResourceDefinition to {product-title}.
|
|
You can configure an SR-IOV network device by creating a SriovNetworkNodePolicy custom resource (CR).
|
|
|
|
[NOTE]
|
|
=====
|
|
When applying the configuration specified in a `SriovNetworkNodePolicy` object, the SR-IOV Operator might drain the nodes, and in some cases, reboot nodes.
|
|
Reboot only happens in the following cases:
|
|
|
|
* With Mellanox NICs (`mlx5` driver) a node reboot happens every time the number of virtual functions (VFs) increase on a physical function (PF).
|
|
* With Intel NICs, a reboot only happens if the kernel parameters do not include `intel_iommu=on` and `iommu=pt`.
|
|
|
|
It might take several minutes for a configuration change to apply.
|
|
=====
|
|
|
|
.Prerequisites
|
|
|
|
* You installed the OpenShift CLI (`oc`).
|
|
* You have access to the cluster as a user with the `cluster-admin` role.
|
|
* You have installed the SR-IOV Network Operator.
|
|
* You have enough available nodes in your cluster to handle the evicted workload from drained nodes.
|
|
* You have not selected any control plane nodes for SR-IOV network device configuration.
|
|
|
|
.Procedure
|
|
|
|
. Create an `SriovNetworkNodePolicy` object, and then save the YAML in the `<name>-sriov-node-network.yaml` file. Replace `<name>` with the name for this configuration.
|
|
ifdef::virt-sriov[]
|
|
+
|
|
[source,yaml]
|
|
----
|
|
apiVersion: sriovnetwork.openshift.io/v1
|
|
kind: SriovNetworkNodePolicy
|
|
metadata:
|
|
name: <name> <1>
|
|
namespace: openshift-sriov-network-operator <2>
|
|
spec:
|
|
resourceName: <sriov_resource_name> <3>
|
|
nodeSelector:
|
|
feature.node.kubernetes.io/network-sriov.capable: "true" <4>
|
|
priority: <priority> <5>
|
|
mtu: <mtu> <6>
|
|
numVfs: <num> <7>
|
|
nicSelector: <8>
|
|
vendor: "<vendor_code>" <9>
|
|
deviceID: "<device_id>" <10>
|
|
pfNames: ["<pf_name>", ...] <11>
|
|
rootDevices: ["<pci_bus_id>", "..."] <12>
|
|
deviceType: vfio-pci <13>
|
|
isRdma: false <14>
|
|
----
|
|
+
|
|
--
|
|
<1> Specify a name for the CR object.
|
|
<2> Specify the namespace where the SR-IOV Operator is installed.
|
|
<3> Specify the resource name of the SR-IOV device plugin. You can create multiple `SriovNetworkNodePolicy` objects for a resource name.
|
|
<4> Specify the node selector to select which nodes are configured.
|
|
Only SR-IOV network devices on selected nodes are configured. The SR-IOV
|
|
Container Network Interface (CNI) plugin and device plugin are deployed only on selected nodes.
|
|
<5> Optional: Specify an integer value between `0` and `99`. A smaller number gets higher priority, so a priority of `10` is higher than a priority of `99`. The default value is `99`.
|
|
<6> Optional: Specify a value for the maximum transmission unit (MTU) of the virtual function. The maximum MTU value can vary for different NIC models.
|
|
<7> Specify the number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel network interface controller (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than `127`.
|
|
<8> The `nicSelector` mapping selects the Ethernet device for the Operator to configure. You do not need to specify values for all the parameters.
|
|
+
|
|
[NOTE]
|
|
====
|
|
It is recommended to identify the Ethernet adapter with enough precision to minimize the possibility of selecting an Ethernet device unintentionally.
|
|
If you specify `rootDevices`, you must also specify a value for `vendor`, `deviceID`, or `pfNames`.
|
|
====
|
|
If you specify both `pfNames` and `rootDevices` at the same time, ensure that they point to an identical device.
|
|
<9> Optional: Specify the vendor hex code of the SR-IOV network device. The only allowed values are either `8086` or `15b3`.
|
|
<10> Optional: Specify the device hex code of SR-IOV network device. The only allowed values are `158b`, `1015`, `1017`.
|
|
<11> Optional: The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.
|
|
<12> The parameter accepts an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format: `0000:02:00.1`.
|
|
<13> The `vfio-pci` driver type is required for virtual functions in {VirtProductName}.
|
|
<14> Optional: Specify whether to enable remote direct memory access (RDMA) mode. For a Mellanox card, set `isRdma` to `false`. The default value is `false`.
|
|
+
|
|
[NOTE]
|
|
====
|
|
If `isRDMA` flag is set to `true`, you can continue to use the RDMA enabled VF as a normal network device.
|
|
A device can be used in either mode.
|
|
====
|
|
--
|
|
endif::virt-sriov[]
|
|
|
|
. Optional: Label the SR-IOV capable cluster nodes with `SriovNetworkNodePolicy.Spec.NodeSelector` if they are not already labeled. For more information about labeling nodes, see "Understanding how to update labels on nodes".
|
|
|
|
. Create the `SriovNetworkNodePolicy` object:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc create -f <name>-sriov-node-network.yaml
|
|
----
|
|
+
|
|
where `<name>` specifies the name for this configuration.
|
|
+
|
|
After applying the configuration update, all the pods in `sriov-network-operator` namespace transition to the `Running` status.
|
|
|
|
. To verify that the SR-IOV network device is configured, enter the following command. Replace `<node_name>` with the name of a node with the SR-IOV network device that you just configured.
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'
|
|
----
|
|
|
|
ifdef::virt-sriov[]
|
|
:!virt-sriov:
|
|
endif::[]
|
|
|
|
ifdef::ocp-sriov[]
|
|
:!ocp-sriov:
|
|
endif::[]
|