openshift-docs/modules/nw-sriov-configuring-device.adoc

// Module included in the following assemblies:
//
// * networking/hardware_networks/configuring-sriov-device.adoc
// * virt/vm_networking/virt-connecting-vm-to-sriov.adoc
// * virt/post_installation_configuration/virt-post-install-network-config.adoc

ifeval::["{context}" == "configuring-sriov-device"]
:ocp-sriov:
endif::[]

ifeval::["{context}" == "virt-connecting-vm-to-sriov"]
:virt-sriov:
endif::[]
ifeval::["{context}" == "virt-post-install-network-config"]
:virt-sriov:
endif::[]

:_mod-docs-content-type: PROCEDURE
[id="nw-sriov-configuring-device_{context}"]
= Configuring SR-IOV network devices

[role="_abstract"]
The SR-IOV Network Operator adds the `SriovNetworkNodePolicy.sriovnetwork.openshift.io` custom resource definition (CRD) to {product-title}.
You can configure an SR-IOV network device by creating a `SriovNetworkNodePolicy` custom resource (CR).

[NOTE]
=====
When applying the configuration specified in a `SriovNetworkNodePolicy` object, the SR-IOV Operator might drain the nodes, and in some cases, reboot nodes.
Reboot only happens in the following cases:

* With Mellanox NICs (`mlx5` driver) a node reboot happens every time the number of virtual functions (VFs) increase on a physical function (PF).
* With Intel NICs, a reboot only happens if the kernel parameters do not include `intel_iommu=on` and `iommu=pt`.

It might take several minutes for a configuration change to apply.
=====

.Prerequisites

* You installed the {oc-first}.
* You have access to the cluster as a user with the `cluster-admin` role.
* You have installed the SR-IOV Network Operator.
* You have enough available nodes in your cluster to handle the evicted workload from drained nodes.
* You have not selected any control plane nodes for SR-IOV network device configuration.

.Procedure

. Create an `SriovNetworkNodePolicy` object, and then save the YAML in the `<name>-sriov-node-network.yaml` file. Replace `<name>` with the name for this configuration.
ifdef::virt-sriov[]
+
[source,yaml]
----
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: <name>
  namespace: openshift-sriov-network-operator
spec:
  resourceName: <sriov_resource_name>
  nodeSelector:
    feature.node.kubernetes.io/network-sriov.capable: "true"
  priority: <priority>
  mtu: <mtu>
  numVfs: <num>
  nicSelector:
    vendor: "<vendor_code>"
    deviceID: "<device_id>"
    pfNames: ["<pf_name>", ...]
    rootDevices: ["<pci_bus_id>", "..."]
  deviceType: vfio-pci
  isRdma: false
----
+
`metadata.name`:: Specify a name for the `SriovNetworkNodePolicy` object.
`metadata.namespace`:: Specify the namespace where the SR-IOV Network Operator is installed.
`spec.resourceName`:: Specify the resource name of the SR-IOV device plugin. You can create multiple `SriovNetworkNodePolicy` objects for a resource name.
`spec.nodeSelector.feature.node.kubernetes.io/network-sriov.capable`:: Specify the node selector to select which nodes are configured. Only SR-IOV network devices on selected nodes are configured. The SR-IOV Container Network Interface (CNI) plugin and device plugin are deployed only on selected nodes.
`spec.priority`:: Optional: Specify an integer value between `0` and `99`. A smaller number gets higher priority, so a priority of `10` is higher than a priority of `99`. The default value is `99`.
`spec.mtu`:: Optional: Specify a value for the maximum transmission unit (MTU) of the virtual function. The maximum MTU value can vary for different NIC models.
`spec.numVfs`:: Specify the number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel network interface controller (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than `127`.
`spec.nicSelector`:: The `nicSelector` mapping selects the Ethernet device for the Operator to configure. You do not need to specify values for all the parameters.
+
[NOTE]
====
It is recommended to identify the Ethernet adapter with enough precision to minimize the possibility of selecting an Ethernet device unintentionally.
If you specify `rootDevices`, you must also specify a value for `vendor`, `deviceID`, or `pfNames`.
====
+
If you specify both `pfNames` and `rootDevices` at the same time, ensure that they point to an identical device.
`spec.nicSelector.vendor`:: Optional: Specify the vendor hex code of the SR-IOV network device. The only allowed values are either `8086` or `15b3`.
`spec.nicSelector.deviceID`:: Optional: Specify the device hex code of SR-IOV network device. The only allowed values are `158b`, `1015`, `1017`.
`spec.nicSelector.pfNames`:: Optional: The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.
`spec.nicSelector.rootDevices`:: The parameter accepts an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format: `0000:02:00.1`.
`spec.deviceType`:: The `vfio-pci` driver type is required for virtual functions in {VirtProductName}.
`spec.isRdma`:: Optional: Specify whether to enable remote direct memory access (RDMA) mode. For a Mellanox card, set `isRdma` to `false`. The default value is `false`.
+
[NOTE]
====
If `isRDMA` flag is set to `true`, you can continue to use the RDMA enabled VF as a normal network device.
A device can be used in either mode.
====

endif::virt-sriov[]

. Optional: Label the SR-IOV capable cluster nodes with `SriovNetworkNodePolicy.Spec.NodeSelector` if they are not already labeled. For more information about labeling nodes, see "Understanding how to update labels on nodes".

. Create the `SriovNetworkNodePolicy` object:
+
[source,terminal]
----
$ oc create -f <name>-sriov-node-network.yaml
----
+
where `<name>` specifies the name for this configuration.
+
After applying the configuration update, all the pods in `sriov-network-operator` namespace transition to the `Running` status.

. To verify that the SR-IOV network device is configured, enter the following command. Replace `<node_name>` with the name of a node with the SR-IOV network device that you just configured.
+
[source,terminal]
----
$ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'
----

ifdef::virt-sriov[]
:!virt-sriov:
endif::[]

ifdef::ocp-sriov[]
:!ocp-sriov:
endif::[]