mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-07 00:48:01 +01:00
Apply feedback to SR-IOV related documentation 3/3
This commit is contained in:
Jason Boxman
@@ -1,6 +1,6 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/multiple_networks/configuring-sr-iov.adoc
|
||||
// * networking/hardware_networks/configuring-sriov-net-attach.adoc
|
||||
|
||||
// Because of an existing issue in go-yaml, the strings 'on' and 'off'
|
||||
// are interpreted as booleans, not strings. The SR-IOV admission controller
|
||||
@@ -8,15 +8,15 @@
|
||||
// So these values must be explicitly quoted in the YAML.
|
||||
// https://github.com/go-yaml/yaml/issues/214
|
||||
|
||||
[id="configuring-sr-iov-networks_{context}"]
|
||||
[id="nw-sriov-network-attachment_{context}"]
|
||||
= Configuring SR-IOV additional network
|
||||
|
||||
You can configure an additional network that uses SR-IOV hardware by creating a SriovNetwork Custom Resource (CR).
|
||||
You can configure an additional network that uses SR-IOV hardware by creating a SriovNetwork custom resource (CR).
|
||||
When you create a SriovNetwork CR, the SR-IOV Operator automatically creates a NetworkAttachmentDefinition CR.
|
||||
|
||||
[NOTE]
|
||||
=====
|
||||
Do not modify or delete a SriovNetwork Custom Resource (CR) if it is attached to any Pods in the `running` state.
|
||||
Do not modify or delete a SriovNetwork CR if it is attached to any Pods in the `running` state.
|
||||
=====
|
||||
|
||||
.Prerequisites
|
||||
@@ -28,9 +28,7 @@ Do not modify or delete a SriovNetwork Custom Resource (CR) if it is attached to
|
||||
|
||||
. Create the following SriovNetwork CR, and then save the YAML in the `<name>-sriov-network.yaml` file. Replace `<name>` with a name for this additional network.
|
||||
|
||||
//To get the notes nexted in the annotations following this code block
|
||||
//I've removed the item from the procedure and started the next step of
|
||||
//the procedure mid-list.
|
||||
// The list breaks because of the [NOTE]
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: sriovnetwork.openshift.io/v1
|
||||
@@ -50,15 +48,14 @@ spec:
|
||||
spoofChk: "<spoof_check>" <11>
|
||||
trust: "<trust_vf>" <12>
|
||||
capabilities: <capabilities> <13>
|
||||
|
||||
----
|
||||
<1> Replace `<name>` with a name for the CR. The Operator will create a NetworkAttachmentDefinition CR with same name.
|
||||
<1> Replace `<name>` with a name for the CR. The SR-IOV Network Operator creates a NetworkAttachmentDefinition CR with same name.
|
||||
<2> Specify the namespace where the SR-IOV Operator is installed.
|
||||
<3> Optional: Replace `<target_namespace>` with the namespace where the NetworkAttachmentDefinition CR will be created. The default value is `openshift-sriov-network-operator`.
|
||||
<3> Optional: Replace `<target_namespace>` with the namespace where the NetworkAttachmentDefinition CR is created. The default value is `openshift-sriov-network-operator`.
|
||||
<4> Optional: Replace `<ipam>` a configuration object for the ipam CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.
|
||||
<5> Optional: Replace `<vlan>` with a Virtual LAN (VLAN) ID for the additional network. The integer value must be from `0` to `4095`. The default value is `0`.
|
||||
<6> Replace `<sriov_resource_name>` with the value for the `.spec.resourceName` parameter from the SriovNetworkNodePolicy CR that defines the SR-IOV hardware for this additional network.
|
||||
<7> Optional: Replace `<link_state>` with the link state of Virtual Function (VF). Allowed value are `enable`, `disable` and `auto`.
|
||||
<7> Optional: Replace `<link_state>` with the link state of virtual function (VF). Allowed value are `enable`, `disable` and `auto`.
|
||||
<8> Optional: Replace `<max_tx_rate>` with a maximum transmission rate, in Mbps, for the VF.
|
||||
<9> Optional: Replace `<min_tx_rate>` with a minimum transmission rate, in Mbps, for the VF. This value should always be less than or equal to Maximum transmission rate.
|
||||
+
|
||||
@@ -71,26 +68,25 @@ Intel NICs do not support the `minTxRate` parameter. For more information, see l
|
||||
+
|
||||
[IMPORTANT]
|
||||
====
|
||||
You must enclose the value you specify in quotes or the CR will be rejected by the SR-IOV Network Operator.
|
||||
You must enclose the value you specify in quotes or the CR is rejected by the SR-IOV Network Operator.
|
||||
====
|
||||
<12> Optional: Replace `<trust_vf>` with the trust mode of the VF. The allowed values are the strings `"on"` and `"off"`.
|
||||
+
|
||||
[IMPORTANT]
|
||||
====
|
||||
You must enclose the value you specify in quotes or the CR will be rejected by the SR-IOV Network Operator.
|
||||
You must enclose the value you specify in quotes or the CR is rejected by the SR-IOV Network Operator.
|
||||
====
|
||||
<13> Optional: Replace `<capabilities>` with the capabilities to configure for this network. You can specify `"{ "ips": true }"` to enable IP address support or `"{ "mac": true }"` to enable MAC address support.
|
||||
|
||||
[start=2]
|
||||
. Create the CR by running the following command:
|
||||
. To create the CR object, enter the following command. Replace `<name>` with a name for this additional network.
|
||||
+
|
||||
----
|
||||
$ oc create -f <filename> <1>
|
||||
$ oc create -f <name>-sriov-network.yaml
|
||||
----
|
||||
<1> Replace `<filename>` with the name of the file you created in the previous step.
|
||||
|
||||
. Optional: Confirm that the NetworkAttachmentDefinition CR associated with the SriovNetwork CR that you created in the previous step exists by running the following command. Replace `<namespace>` with the namespace you specified in the SriovNetwork CR.
|
||||
. Optional: To confirm that the NetworkAttachmentDefinition CR associated with the SriovNetwork CR that you created in the previous step exists, enter the following command. Replace `<namespace>` with the namespace you specified in the SriovNetwork CR.
|
||||
+
|
||||
----
|
||||
oc get net-attach-def -n <namespace>
|
||||
$ oc get net-attach-def -n <namespace>
|
||||
----
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/multiple_networks/configuring-sr-iov.adoc
|
||||
// * networking/hardware_networks/using-dpdk-and-rdma.adoc
|
||||
|
||||
[id="example-vf-use-in-rdma-mode-mellanox_{context}"]
|
||||
= Example of a virtual function in RDMA mode with Mellanox NICs
|
||||
|
||||
@@ -1,12 +1,11 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/multiple_networks/configuring-sr-iov.adoc
|
||||
// * networking/hardware_networks/about-sriov.adoc
|
||||
|
||||
[id="supported-devices_{context}"]
|
||||
= Supported devices
|
||||
|
||||
The following Network Interface Card (NIC) models are supported in
|
||||
{product-title}:
|
||||
{product-title} supports the following Network Interface Card (NIC) models:
|
||||
|
||||
* Intel XXV710 25GbE SFP28 with vendor ID `0x8086` and device ID `0x158b`
|
||||
* Mellanox MT27710 Family [ConnectX-4 Lx] 25GbE dual-port SFP28 with vendor ID `0x15b3` and device ID `0x1015`
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/multiple_networks/configuring-sr-iov.adoc
|
||||
// * networking/hardware_networks/add-pod.adoc
|
||||
|
||||
[id="nw-sriov-topology-manager_{context}"]
|
||||
= Creating a non-uniform memory access (NUMA) aligned SR-IOV pod
|
||||
@@ -51,20 +51,20 @@ $ oc create -f <filename> <1>
|
||||
----
|
||||
<1> Replace `<filename>` with the name of the file you created in the previous step.
|
||||
|
||||
. Optional: Confirm that the `sample-pod` is configured with guaranteed QoS.
|
||||
. Confirm that the `sample-pod` is configured with guaranteed QoS.
|
||||
+
|
||||
----
|
||||
oc describe pod sample-pod
|
||||
$ oc describe pod sample-pod
|
||||
----
|
||||
|
||||
. Optional: Confirm that the `sample-pod` is allocated with exclusive CPUs.
|
||||
. Confirm that the `sample-pod` is allocated with exclusive CPUs.
|
||||
+
|
||||
----
|
||||
oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus
|
||||
$ oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus
|
||||
----
|
||||
|
||||
. Optional: Confirm that the SR-IOV device and CPUs that are allocated for the `sample-pod` are on the same NUMA node.
|
||||
. Confirm that the SR-IOV device and CPUs that are allocated for the `sample-pod` are on the same NUMA node.
|
||||
+
|
||||
----
|
||||
oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus
|
||||
$ oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus
|
||||
----
|
||||
|
||||
@@ -5,63 +5,49 @@ include::modules/common-attributes.adoc[]
|
||||
|
||||
toc::[]
|
||||
|
||||
You can use Single Root I/O Virtualization (SR-IOV) network devices with additional networks on your {product-title} cluster for high performance applications.
|
||||
The Single Root I/O Virtualization (SR-IOV) specification is a standard for a type of PCI device assignment that can share a single device with multiple pods.
|
||||
|
||||
[id="about-using-sr-iov-hardware"]
|
||||
== About SR-IOV hardware on {product-title}
|
||||
SR-IOV enables you to segment a compliant network device, recognized on the host node as a physical function (PF), into multiple virtual functions (VFs).
|
||||
The VF is used like any other network device.
|
||||
The SR-IOV device driver for the device determines how the VF is exposed in the container:
|
||||
|
||||
{product-title} includes the capability to use SR-IOV hardware on your nodes.
|
||||
You can attach SR-IOV virtual function (VF) interfaces to Pods on nodes with
|
||||
SR-IOV hardware.
|
||||
* `netdevice` driver: A regular kernel network device in the `netns` of the container
|
||||
* `vfio-pci` driver: A character device mounted in the container
|
||||
|
||||
You can use the {product-title} console to install SR-IOV by deploying the
|
||||
SR-IOV Network Operator. The SR-IOV Network Operator creates and manages the
|
||||
components of the SR-IOV stack. The Operator provisions the following
|
||||
components:
|
||||
You can use SR-IOV network devices with additional networks on your {product-title} cluster for application that require high bandwidth or low latency.
|
||||
|
||||
* Provision the SR-IOV network operator deployment on master nodes.
|
||||
* Provision the SR-IOV network config daemon on worker nodes.
|
||||
* Provision the Operator webhook on master nodes.
|
||||
* Provision the Network resources injector on master nodes.
|
||||
* Provision the SR-IOV network device plug-in on worker nodes.
|
||||
* Provision the SR-IOV CNI plug-in executable on worker nodes.
|
||||
[id="components-sr-iov-network-devices"]
|
||||
== Components that manage SR-IOV network devices
|
||||
|
||||
Here's the function of each above mentioned SR-IOV components.
|
||||
The SR-IOV Network Operator creates and manages the components of the SR-IOV stack.
|
||||
It performs the following functions:
|
||||
|
||||
* The SR-IOV Operator is a Kubernetes Deployment that manages all SR-IOV
|
||||
components in a cluster. It watches creation, update and deletion of Operator
|
||||
Custom Resources and takes corresponding actions such as generating
|
||||
NetworkAttachmentDefinition Custom Resources for SR-IOV CNI, creating and
|
||||
updating configuration of SR-IOV network device plug-in, creating node
|
||||
specific SriovNetworkNodeState Custom Resources and updating Spec.Interfaces
|
||||
field in each SriovNetworkNodeState Custom Resource, etc.
|
||||
- Orchestrates discovery and management of SR-IOV network devices
|
||||
- Generates NetworkAttachmentDefinition custom resources for the SR-IOV Container Network Interface (CNI)
|
||||
- Creates and updates the configuration of the SR-IOV network device plug-in
|
||||
- Creates node specific SriovNetworkNodeState custom resources
|
||||
- Updates the `spec.interfaces` field in each SriovNetworkNodeState custom resource
|
||||
|
||||
* The SR-IOV network config daemon is a Kubernetes DaemonSet deployed on
|
||||
worker nodes when SR-IOV Operator is launched. It is responsible for
|
||||
discovering and initializing SR-IOV network devices in cluster.
|
||||
The Operator provisions the following components:
|
||||
|
||||
* The Operator webhook is a Kubernetes Dynamic Admission Controller Webhook
|
||||
that validates correctness of Operator Custom Resource and sets default values
|
||||
for fields in Operator Custom Resource that are not configured by user.
|
||||
SR-IOV network configuration daemon::
|
||||
A DaemonSet that is deployed on worker nodes when the SR-IOV Operator starts. The daemon is responsible for discovering and initializing SR-IOV network devices in the cluster.
|
||||
|
||||
* The Network resources injector is a Kubernetes Dynamic Admission Controller
|
||||
Webhook that provides functionality for patching Kubernetes Pod specifications
|
||||
with requests and limits for custom network resources such as SR-IOV VFs.
|
||||
SR-IOV Operator webhook::
|
||||
A dynamic admission controller webhook that validates the Operator custom resource and sets appropriate default values for unset fields.
|
||||
|
||||
* The SR-IOV network device plug-in is a Kubernetes device plug-in for
|
||||
discovering, advertising, and allocating SR-IOV network virtual function (VF)
|
||||
resources. Device plug-ins are used in Kubernetes to enable the use of limited
|
||||
resources, typically in physical devices. Device plug-ins give the Kubernetes
|
||||
scheduler awareness of resource availability, so the scheduler can schedule
|
||||
Pods on nodes with sufficient resources.
|
||||
SR-IOV Network resources injector::
|
||||
A dynamic admission controller webhook that provides functionality for patching Kubernetes Pod specifications with requests and limits for custom network resources such as SR-IOV VFs.
|
||||
|
||||
* The SR-IOV CNI plug-in plumbs VF interfaces allocated from the SR-IOV device
|
||||
plug-in directly into a Pod.
|
||||
SR-IOV network device plug-in::
|
||||
A device plug-in that discovers, advertises, and allocates SR-IOV network virtual function (VF) resources. Device plug-ins are used in Kubernetes to enable the use of limited resources, typically in physical devices. Device plug-ins give the Kubernetes scheduler awareness of resource availability, so that the scheduler can schedule Pods on nodes with sufficient resources.
|
||||
|
||||
SR-IOV CNI plug-in::
|
||||
A CNI plug-in that attaches VF interfaces allocated from the SR-IOV device plug-in directly into a Pod.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The Network resources injector and Operator webhook are enabled by default and
|
||||
can be disabled by editing the default SriovOperatorConfig CR.
|
||||
The SR-IOV Network resources injector and SR-IOV Network Operator webhook are enabled by default and can be disabled by editing the `default` SriovOperatorConfig CR.
|
||||
====
|
||||
|
||||
include::modules/nw-sriov-supported-devices.adoc[leveloffset=+2]
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
[id="using-dpdk-and-rdma"]
|
||||
= Using Virtual Functions (VFs) with DPDK and RDMA modes
|
||||
= Using virtual functions (VFs) with DPDK and RDMA modes
|
||||
include::modules/common-attributes.adoc[]
|
||||
:context: using-dpdk-and-rdma
|
||||
|
||||
@@ -20,12 +20,12 @@ include::modules/technology-preview.adoc[leveloffset=+0]
|
||||
|
||||
* Install the OpenShift Command-line Interface (CLI), commonly known as `oc`.
|
||||
* Log in as a user with `cluster-admin` privileges.
|
||||
* You must have installed the SR-IOV Operator.
|
||||
* You must have installed the SR-IOV Network Operator.
|
||||
|
||||
include::modules/nw-sriov-dpdk-example-intel.adoc[leveloffset=+2]
|
||||
include::modules/nw-sriov-dpdk-example-intel.adoc[leveloffset=+1]
|
||||
|
||||
include::modules/nw-sriov-dpdk-example-mellanox.adoc[leveloffset=+2]
|
||||
include::modules/nw-sriov-dpdk-example-mellanox.adoc[leveloffset=+1]
|
||||
|
||||
include::modules/nw-sriov-rdma-example-mellanox.adoc[leveloffset=+2]
|
||||
include::modules/nw-sriov-rdma-example-mellanox.adoc[leveloffset=+1]
|
||||
|
||||
:!FeatureName:
|
||||
|
||||
Reference in New Issue
Block a user