mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
Add SR-IOV InfiniBand network configuration content
- https://issues.redhat.com/browse/OSDOCS-1281 Co-authored-by: Kathryn Alexander <37149781+kalexand-rh@users.noreply.github.com>
This commit is contained in:
committed by
openshift-cherrypick-robot
parent
6c6123f1d9
commit
6ce2caedd0
@@ -620,8 +620,10 @@ Topics:
|
||||
File: configuring-sriov-operator
|
||||
- Name: Configuring an SR-IOV network device
|
||||
File: configuring-sriov-device
|
||||
- Name: Configuring an SR-IOV network attachment
|
||||
- Name: Configuring an SR-IOV Ethernet network attachment
|
||||
File: configuring-sriov-net-attach
|
||||
- Name: Configuring an SR-IOV InfiniBand network attachment
|
||||
File: configuring-sriov-ib-attach
|
||||
- Name: Adding Pod to an SR-IOV network
|
||||
File: add-pod
|
||||
- Name: Using high performance multicast
|
||||
|
||||
@@ -24,6 +24,7 @@ You can add a Pod to an additional network. The Pod continues to send normal clu
|
||||
|
||||
When a Pod is created additional networks are attached to it. However, if a Pod already exists, you cannot attach additional networks to it.
|
||||
|
||||
The Pod must be in the same namespace as the additional network.
|
||||
|
||||
ifdef::sriov[]
|
||||
[NOTE]
|
||||
@@ -41,11 +42,11 @@ endif::sriov[]
|
||||
|
||||
.Prerequisites
|
||||
|
||||
* The Pod must be in the same namespace as the additional network.
|
||||
* Install the OpenShift CLI (`oc`).
|
||||
* You must log in to the cluster.
|
||||
* Log in to the cluster.
|
||||
ifdef::sriov[]
|
||||
* You must have the SR-IOV Operator installed and a SriovNetwork CR defined.
|
||||
* Install the SR-IOV Operator.
|
||||
* Create either an `SriovNetwork` object or an `SriovIBNetwork` object to attach the Pod to.
|
||||
endif::sriov[]
|
||||
|
||||
.Procedure
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/hardware_networks/configuring-sriov-net-attach.adoc
|
||||
// * virt/node_network/virt-configuring-sr-iov-network.adoc
|
||||
// * virt/virtual_machines/vm_networking/virt-defining-an-sriov-network.adoc
|
||||
|
||||
// Deprecating in OCP; This is identical in practice to adding a pod
|
||||
// to an additional network.
|
||||
|
||||
[id="nw-sriov-add-pod-runtimeconfig_{context}"]
|
||||
= Configuring static MAC and IP addresses on additional SR-IOV networks
|
||||
|
||||
|
||||
@@ -32,8 +32,9 @@ Ensure that there are enough available nodes in your cluster to handle the evict
|
||||
|
||||
.Procedure
|
||||
|
||||
. Create the following SriovNetworkNodePolicy CR, and then save the YAML in the `<name>-sriov-node-network.yaml` file. Replace `<name>` with the name for this configuration.
|
||||
. Create an SriovNetworkNodePolicy object, and then save the YAML in the `<name>-sriov-node-network.yaml` file. Replace `<name>` with the name for this configuration.
|
||||
|
||||
ifdef::virt-sriov[]
|
||||
// The list breaks because of the [NOTE]
|
||||
[source,yaml]
|
||||
----
|
||||
@@ -54,12 +55,7 @@ spec:
|
||||
deviceID: "<device_id>" <10>
|
||||
pfNames: ["<pf_name>", ...] <11>
|
||||
rootDevices: ["<pci_bus_id>", "..."] <12>
|
||||
ifdef::ocp-sriov[]
|
||||
deviceType: <device_type> <13>
|
||||
endif::ocp-sriov[]
|
||||
ifdef::virt-sriov[]
|
||||
deviceType: vfio-pci <13>
|
||||
endif::virt-sriov[]
|
||||
isRdma: false <14>
|
||||
----
|
||||
<1> Specify a name for the CR object.
|
||||
@@ -78,34 +74,26 @@ If you specify both `pfNames` and `rootDevices` at the same time, ensure that th
|
||||
<10> Optional: Specify the device hex code of SR-IOV network device. The only allowed values are `158b`, `1015`, `1017`.
|
||||
<11> Optional: The parameter accepts an array of one or more physical function (PF) names for the Ethernet device.
|
||||
<12> The parameter accepts an array of one or more PCI bus addresses for the physical function of the Ethernet device. Provide the address in the following format: `0000:02:00.1`.
|
||||
ifdef::ocp-sriov[]
|
||||
<13> Optional: Specify the driver type for the virtual functions. You can specify one of the following values: `netdevice` or `vfio-pci`. The default value is `netdevice`.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
For a Mellanox card to work in Data Plane Development Kit (DPDK) mode on bare metal nodes, use the `netdevice` driver type and set `isRdma` to `true`.
|
||||
====
|
||||
<14> Optional: Specify whether to enable remote direct memory access (RDMA) mode. The default value is `false`. Only RDMA over Converged Ethernet (RoCE) mode is supported on Mellanox Ethernet adapters.
|
||||
endif::ocp-sriov[]
|
||||
ifdef::virt-sriov[]
|
||||
<13> The `vfio-pci` driver type is required for virtual functions in {VirtProductName}.
|
||||
<14> Optional: Specify whether to enable remote direct memory access (RDMA) mode. For a Mellanox card, set `isRdma` to `false`. The default value is `false`.
|
||||
endif::virt-sriov[]
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
If `isRDMA` flag is set to `true`, you can continue to use the RDMA enabled VF as a normal network device.
|
||||
A device can be used in either mode.
|
||||
====
|
||||
endif::virt-sriov[]
|
||||
|
||||
[start=2]
|
||||
. Create the SriovNetworkNodePolicy CR. Replace `<name>` with the name for this configuration.
|
||||
. Create the SriovNetworkNodePolicy CR:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc create -f <name>-sriov-node-network.yaml
|
||||
----
|
||||
+
|
||||
where `<name>` specifies the name for this configuration.
|
||||
+
|
||||
After applying the configuration update, all the Pods in `sriov-network-operator` namespace transition to the `Running` status.
|
||||
|
||||
. To verify that the SR-IOV network device is configured, enter the following command. Replace `<node_name>` with the name of a node with the SR-IOV network device that you just configured.
|
||||
|
||||
39
modules/nw-sriov-ibnetwork-object.adoc
Normal file
39
modules/nw-sriov-ibnetwork-object.adoc
Normal file
@@ -0,0 +1,39 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/hardware_networks/configuring-sriov-ib-attach.adoc
|
||||
|
||||
[id="nw-sriov-ibnetwork-object_{context}"]
|
||||
= InfiniBand device configuration object
|
||||
|
||||
You can configure an InfiniBand (IB) network device by defining an `SriovIBNetwork` object.
|
||||
|
||||
The following YAML describes an `SriovIBNetwork` object:
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: sriovnetwork.openshift.io/v1
|
||||
kind: SriovIBNetwork
|
||||
metadata:
|
||||
name: <name> <1>
|
||||
namespace: openshift-sriov-network-operator <2>
|
||||
spec:
|
||||
resourceName: <sriov_resource_name> <3>
|
||||
networkNamespace: <target_namespace> <4>
|
||||
ipam: |- <5>
|
||||
{}
|
||||
linkState: <link_state> <6>
|
||||
capabilities: <capabilities> <7>
|
||||
----
|
||||
<1> A name for the object. The SR-IOV Network Operator creates a NetworkAttachmentDefinition CR with same name.
|
||||
|
||||
<2> The namespace where the SR-IOV Operator is installed.
|
||||
|
||||
<3> The value for the `spec.resourceName` parameter from the `SriovNetworkNodePolicy` object that defines the SR-IOV hardware for this additional network.
|
||||
|
||||
<4> The target namespace for the `SriovIBNetwork`. Only pods in the target namespace can attach to the `SriovIBNetwork`.
|
||||
|
||||
<5> Optional: A configuration object for the IPAM CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.
|
||||
|
||||
<6> Optional: The link state of virtual function (VF). Allowed values are `enable`, `disable` and `auto`.
|
||||
|
||||
<7> Optional: The capabilities to configure for this network. You can specify `"{ "ips": true }"` to enable IP address support or `"{ "infinibandGUID": true }"` to enable IB Global Unique Identifier (GUID) support.
|
||||
@@ -3,23 +3,92 @@
|
||||
// * networking/hardware_networks/configuring-sriov-net-attach.adoc
|
||||
// * virt/virtual_machines/vm_networking/virt-defining-an-sriov-network.adoc
|
||||
|
||||
// Because of an existing issue in go-yaml, the strings 'on' and 'off'
|
||||
// are interpreted as booleans, not strings. The SR-IOV admission controller
|
||||
// will reject 'spoofCheck' and 'trust' if the values are not strings.
|
||||
// So these values must be explicitly quoted in the YAML.
|
||||
// https://github.com/go-yaml/yaml/issues/214
|
||||
// Note: IB does not support ipam with `type=dhcp`.
|
||||
|
||||
ifeval::["{context}" == "configuring-sriov-net-attach"]
|
||||
:rs: SriovNetwork
|
||||
:ocp-sriov-net:
|
||||
:object: Pods
|
||||
endif::[]
|
||||
|
||||
ifeval::["{context}" == "configuring-sriov-ib-attach"]
|
||||
:rs: SriovIBNetwork
|
||||
:ocp-sriov-net:
|
||||
:object: Pods
|
||||
endif::[]
|
||||
|
||||
ifeval::["{context}" == "virt-defining-an-sriov-network"]
|
||||
:rs: SriovNetwork
|
||||
:virt-sriov-net:
|
||||
:object: Pods or virtual machines
|
||||
endif::[]
|
||||
|
||||
ifdef::ocp-sriov-net[]
|
||||
[id="nw-sriov-network-attachment_{context}"]
|
||||
= Configuring SR-IOV additional network
|
||||
|
||||
You can configure an additional network that uses SR-IOV hardware by creating a `{rs}` custom resource (CR).
|
||||
When you create a `{rs}` CR, the SR-IOV Operator automatically creates a NetworkAttachmentDefinition CR.
|
||||
|
||||
ifdef::virt-sriov-net[]
|
||||
Users can then attach virtual machines to the SR-IOV network by specifying the network in the virtual machine configurations.
|
||||
endif::virt-sriov-net[]
|
||||
|
||||
[NOTE]
|
||||
=====
|
||||
Do not modify or delete a `{rs}` object if it is attached to any {object} in the `running` state.
|
||||
=====
|
||||
|
||||
.Prerequisites
|
||||
|
||||
* Install the OpenShift CLI (`oc`).
|
||||
* Log in as a user with `cluster-admin` privileges.
|
||||
|
||||
.Procedure
|
||||
|
||||
. Create a `{rs}` CR, and then save the YAML in the `<name>.yaml` file, where `<name>` is a name for this additional network. The object specification might resemble the following example:
|
||||
+
|
||||
[source,yaml,subs="attributes+"]
|
||||
----
|
||||
apiVersion: sriovnetwork.openshift.io/v1
|
||||
kind: {rs}
|
||||
metadata:
|
||||
name: attach1
|
||||
namespace: openshift-sriov-network-operator
|
||||
spec:
|
||||
resourceName: net1
|
||||
networkNamespace: project2
|
||||
ipam: |-
|
||||
{
|
||||
"ipam": {
|
||||
"type": "host-local",
|
||||
"subnet": "10.56.217.0/24",
|
||||
"rangeStart": "10.56.217.171",
|
||||
"rangeEnd": "10.56.217.181",
|
||||
"gateway": "10.56.217.1"
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
. To create the CR object, enter the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc create -f <name>.yaml
|
||||
----
|
||||
+
|
||||
where `<name>` specifies the name of the additional network.
|
||||
|
||||
. Optional: To confirm that the NetworkAttachmentDefinition CR that is associated with the `{rs}` CR that you created in the previous step exists, enter the following command. Replace `<namespace>` with the networkNamespace you specified in the `{rs}` CR.
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc get net-attach-def -n <namespace>
|
||||
----
|
||||
endif::ocp-sriov-net[]
|
||||
|
||||
// LEGACY
|
||||
ifdef::virt-sriov-net[]
|
||||
[id="nw-sriov-network-attachment_{context}"]
|
||||
= Configuring SR-IOV additional network
|
||||
|
||||
@@ -112,13 +181,21 @@ $ oc create -f <name>-sriov-network.yaml
|
||||
----
|
||||
$ oc get net-attach-def -n <namespace>
|
||||
----
|
||||
// LEGACY
|
||||
endif::virt-sriov-net[]
|
||||
|
||||
ifeval::["{context}" == "virt-defining-an-sriov-network"]
|
||||
:virt-sriov-net!:
|
||||
:object!:
|
||||
ifdef::object[]
|
||||
:!object:
|
||||
endif::[]
|
||||
|
||||
ifeval::["{context}" == "configuring-sriov-net-attach"]
|
||||
:ocp-sriov-net!:
|
||||
:object!:
|
||||
ifdef::rs[]
|
||||
:!rs:
|
||||
endif::[]
|
||||
|
||||
ifdef::virt-sriov-net[]
|
||||
:!virt-sriov-net:
|
||||
endif::[]
|
||||
|
||||
ifdef::ocp-sriov-net[]
|
||||
:!ocp-sriov-net:
|
||||
endif::[]
|
||||
|
||||
93
modules/nw-sriov-network-object.adoc
Normal file
93
modules/nw-sriov-network-object.adoc
Normal file
@@ -0,0 +1,93 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/hardware_networks/configuring-sriov-net-attach.adoc
|
||||
|
||||
// Because of an existing issue in go-yaml, the strings 'on' and 'off'
|
||||
// are interpreted as booleans, not strings. The SR-IOV admission controller
|
||||
// will reject 'spoofCheck' and 'trust' if the values are not strings.
|
||||
// So these values must be explicitly quoted in the YAML.
|
||||
// https://github.com/go-yaml/yaml/issues/214
|
||||
|
||||
ifeval::["{context}" == "configuring-sriov-net-attach"]
|
||||
:ocp-sriov-net:
|
||||
:object: Pods
|
||||
endif::[]
|
||||
|
||||
ifeval::["{context}" == "virt-defining-an-sriov-network"]
|
||||
:virt-sriov-net:
|
||||
:object: Pods or virtual machines
|
||||
endif::[]
|
||||
|
||||
[id="nw-sriov-network-object_{context}"]
|
||||
= Ethernet device configuration object
|
||||
|
||||
You can configure an Ethernet network device by defining an `SriovNetwork` object.
|
||||
|
||||
The following YAML describes an `SriovNetwork` object:
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: sriovnetwork.openshift.io/v1
|
||||
kind: SriovNetwork
|
||||
metadata:
|
||||
name: <name> <1>
|
||||
namespace: openshift-sriov-network-operator <2>
|
||||
spec:
|
||||
resourceName: <sriov_resource_name> <3>
|
||||
networkNamespace: <target_namespace> <4>
|
||||
vlan: <vlan> <5>
|
||||
spoofChk: "<spoof_check>" <6>
|
||||
ifdef::ocp-sriov-net[]
|
||||
ipam: |- <7>
|
||||
{}
|
||||
linkState: <link_state> <8>
|
||||
maxTxRate: <max_tx_rate> <9>
|
||||
minTxRate: <min_rx_rate> <10>
|
||||
vlanQoS: <vlan_qos> <11>
|
||||
trust: "<trust_vf>" <12>
|
||||
capabilities: <capabilities> <13>
|
||||
endif::ocp-sriov-net[]
|
||||
----
|
||||
<1> A name for the CR. The SR-IOV Network Operator creates a NetworkAttachmentDefinition CR with same name.
|
||||
<2> The namespace where the SR-IOV Operator is installed.
|
||||
<3> The value for the `spec.resourceName` parameter from the SriovNetworkNodePolicy CR that defines the SR-IOV hardware for this additional network.
|
||||
<4> The target namespace for the SriovNetwork. Only {object} in the target namespace can attach to the SriovNetwork.
|
||||
<5> Optional: A Virtual LAN (VLAN) ID for the additional network. The integer value must be from `0` to `4095`. The default value is `0`.
|
||||
<6> Optional: The spoof check mode of the VF. The allowed values are the strings `"on"` and `"off"`.
|
||||
+
|
||||
[IMPORTANT]
|
||||
====
|
||||
You must enclose the value you specify in quotes or the CR is rejected by the SR-IOV Network Operator.
|
||||
====
|
||||
+
|
||||
ifdef::ocp-sriov-net[]
|
||||
<7> A configuration object for the IPAM CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.
|
||||
<8> Optional: The link state of virtual function (VF). Allowed value are `enable`, `disable` and `auto`.
|
||||
<9> Optional: A maximum transmission rate, in Mbps, for the VF.
|
||||
<10> Optional: A minimum transmission rate, in Mbps, for the VF. This value must be less than or equal to the maximum transmission rate.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
Intel NICs do not support the `minTxRate` parameter. For more information, see link:https://bugzilla.redhat.com/show_bug.cgi?id=1772847[BZ#1772847].
|
||||
====
|
||||
+
|
||||
<11> Optional: An IEEE 802.1p priority level for the VF. The default value is `0`.
|
||||
<12> Optional: The trust mode of the VF. The allowed values are the strings `"on"` and `"off"`.
|
||||
+
|
||||
[IMPORTANT]
|
||||
====
|
||||
You must enclose the value that you specify in quotes, or the SR-IOV Network Operator rejects the CR.
|
||||
====
|
||||
+
|
||||
<13> Optional: The capabilities to configure for this network. You can specify `"{ "ips": true }"` to enable IP address support or `"{ "mac": true }"` to enable MAC address support.
|
||||
endif::ocp-sriov-net[]
|
||||
|
||||
ifdef::object[]
|
||||
:object!:
|
||||
endif::[]
|
||||
ifdef::ocp-sriov-net[]
|
||||
:ocp-sriov-net!:
|
||||
endif::[]
|
||||
ifdef::virt-sriov-net[]
|
||||
:virt-sriov-net!:
|
||||
endif::[]
|
||||
105
modules/nw-sriov-networknodepolicy-object.adoc
Normal file
105
modules/nw-sriov-networknodepolicy-object.adoc
Normal file
@@ -0,0 +1,105 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/hardware_networks/configuring-sriov-device.adoc
|
||||
|
||||
[id="nw-sriov-networknodepolicy-object_{context}"]
|
||||
= SR-IOV network node configuration object
|
||||
|
||||
You specify the SR-IOV network device configuration for a node by defining an `SriovNetworkNodePolicy` object. The object is part of the `sriovnetwork.openshift.io` API group.
|
||||
|
||||
The following YAML describes an `SriovNetworkNodePolicy` object:
|
||||
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: sriovnetwork.openshift.io/v1
|
||||
kind: SriovNetworkNodePolicy
|
||||
metadata:
|
||||
name: <name> <1>
|
||||
namespace: openshift-sriov-network-operator <2>
|
||||
spec:
|
||||
resourceName: <sriov_resource_name> <3>
|
||||
nodeSelector:
|
||||
feature.node.kubernetes.io/network-sriov.capable: "true" <4>
|
||||
priority: <priority> <5>
|
||||
mtu: <mtu> <6>
|
||||
numVfs: <num> <7>
|
||||
nicSelector: <8>
|
||||
vendor: "<vendor_code>" <9>
|
||||
deviceID: "<device_id>" <10>
|
||||
pfNames: ["<pf_name>", ...] <11>
|
||||
rootDevices: ["<pci_bus_id>", "..."] <12>
|
||||
deviceType: <device_type> <13>
|
||||
isRdma: false <14>
|
||||
linkType: <link_type> <15>
|
||||
----
|
||||
<1> The name for the CR object.
|
||||
|
||||
<2> The namespace where the SR-IOV Operator is installed.
|
||||
|
||||
<3> The resource name of the SR-IOV device plug-in. You can create multiple SriovNetworkNodePolicy CRs for a resource name.
|
||||
|
||||
<4> The node selector to select which nodes are configured.
|
||||
Only SR-IOV network devices on selected nodes are configured. The SR-IOV
|
||||
Container Network Interface (CNI) plug-in and device plug-in are deployed on only selected nodes.
|
||||
|
||||
<5> Optional: An integer value between `0` and `99`. A smaller number gets higher priority, so a priority of `10` is higher than a priority of `99`. The default value is `99`.
|
||||
|
||||
<6> Optional: The maximum transmission unit (MTU) of the virtual function. The maximum MTU value can vary for different NIC models.
|
||||
|
||||
<7> The number of the virtual functions (VF) to create for the SR-IOV physical network device. For an Intel Network Interface Card (NIC), the number of VFs cannot be larger than the total VFs supported by the device. For a Mellanox NIC, the number of VFs cannot be larger than `128`.
|
||||
|
||||
<8> The `nicSelector` mapping selects the device for the Operator to configure. You do not have to specify values for all the parameters. It is recommended to identify the network device with enough precision to avoid selecting a device unintentionally.
|
||||
If you specify `rootDevices`, you must also specify a value for `vendor`, `deviceID`, or `pfNames`.
|
||||
If you specify both `pfNames` and `rootDevices` at the same time, ensure that they point to the same device.
|
||||
|
||||
<9> Optional: The vendor hex code of the SR-IOV network device. The only allowed values are `8086` and `15b3`.
|
||||
|
||||
<10> Optional: The device hex code of SR-IOV network device. The only allowed values are `158b`, `1015`, and `1017`.
|
||||
|
||||
<11> Optional: An array of one or more physical function (PF) names for the device.
|
||||
|
||||
<12> An array of one or more PCI bus addresses for the PF of the device. Provide the address in the following format: `0000:02:00.1`.
|
||||
|
||||
<13> Optional: The driver type for the virtual functions. The only allowed values are `netdevice` and `vfio-pci`. The default value is `netdevice`.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
For a Mellanox card to work in Data Plane Development Kit (DPDK) mode on bare metal nodes, use the `netdevice` driver type and set `isRdma` to `true`.
|
||||
====
|
||||
|
||||
<14> Optional: Whether to enable remote direct memory access (RDMA) mode. The default value is `false`.
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
If the `isRDMA` parameter is set to `true`, you can continue to use the RDMA enabled VF as a normal network device.
|
||||
A device can be used in either mode.
|
||||
====
|
||||
|
||||
<15> Optional: The link type for the VFs. You can specify one of the following values: `eth` or `ib`. `eth` is ethernet and `ib` is InfiniBand. The default value is `eth` if it is not explicitly set. When `linkType` is set to `ib`, `isRdma` will be set to `true` by SR-IOV Network Operator webhook automatically. When `linkType` is set to `ib`, `deviceType` should not be set to `vfio-pci`.
|
||||
|
||||
[id="sr-iov-network-node-configuration-examples_{context}"]
|
||||
== SR-IOV network node configuration examples
|
||||
|
||||
The following example describes the configuration for an IB device:
|
||||
|
||||
.Example configuration for an IB device
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: sriovnetwork.openshift.io/v1
|
||||
kind: SriovNetworkNodePolicy
|
||||
metadata:
|
||||
name: policy-ib-net-1
|
||||
namespace: openshift-sriov-network-operator
|
||||
spec:
|
||||
resourceName: ibnic1
|
||||
nodeSelector:
|
||||
feature.node.kubernetes.io/network-sriov.capable: "true"
|
||||
numVfs: 4
|
||||
nicSelector:
|
||||
vendor: "15b3"
|
||||
deviceID: "101b"
|
||||
rootDevices:
|
||||
- "0000:19:00.0"
|
||||
linkType: ib
|
||||
isRdma: true
|
||||
----
|
||||
96
modules/nw-sriov-runtime-config.adoc
Normal file
96
modules/nw-sriov-runtime-config.adoc
Normal file
@@ -0,0 +1,96 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/hardware_networks/add-pod.adoc
|
||||
|
||||
[id="nw-sriov-runtime-config_{context}"]
|
||||
= Runtime configuration for a network attachment
|
||||
|
||||
When attaching a Pod to an additional network, you can specify a runtime configuration to make specific customizations for the Pod. For example, you can request a specific MAC hardware address.
|
||||
|
||||
You specify the runtime configuration by setting an annotation in the Pod specification. The annotation key is `k8s.v1.cni.cncf.io/networks`, and it accepts a JSON object that describes the runtime configuration.
|
||||
|
||||
[id="runtime-config-ethernet_{context}"]
|
||||
== Runtime configuration for an Ethernet-based SR-IOV attachment
|
||||
|
||||
The following JSON describes the runtime configuration options for an Ethernet-based SR-IOV network attachment.
|
||||
|
||||
[source,json]
|
||||
----
|
||||
[
|
||||
{
|
||||
"name": "<name>", <1>
|
||||
"mac": "<mac_address>", <2>
|
||||
"ips": ["<cidr_range>"] <3>
|
||||
}
|
||||
]
|
||||
----
|
||||
<1> The name of the SR-IOV network attachment definition CR.
|
||||
<2> Optional: The MAC address for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. To use this feature, you also must specify `{ "mac": true }` in the `SriovNetwork` object.
|
||||
<3> Optional: IP addresses for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. Both IPv4 and IPv6 addresses are supported. To use this feature, you also must specify `{ "ips": true }` in the `SriovNetwork` object.
|
||||
|
||||
.Example runtime configuration
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: sample-pod
|
||||
annotations:
|
||||
k8s.v1.cni.cncf.io/networks: |-
|
||||
[
|
||||
{
|
||||
"name": "net1",
|
||||
"mac": "20:04:0f:f1:88:01",
|
||||
"ips": ["192.168.10.1/24", "2001::1/64"]
|
||||
}
|
||||
]
|
||||
spec:
|
||||
containers:
|
||||
- name: sample-container
|
||||
image: <image>
|
||||
imagePullPolicy: IfNotPresent
|
||||
command: ["sleep", "infinity"]
|
||||
----
|
||||
|
||||
[id="runtime-config-infiniband_{context}"]
|
||||
== Runtime configuration for an InfiniBand-based SR-IOV attachment
|
||||
|
||||
The following JSON describes the runtime configuration options for an InfiniBand-based SR-IOV network attachment.
|
||||
|
||||
[source,json]
|
||||
----
|
||||
[
|
||||
{
|
||||
"name": "<network_attachment>", <1>
|
||||
"infiniband-guid": "<guid>", <2>
|
||||
"ips": ["<cidr_range>"] <3>
|
||||
}
|
||||
]
|
||||
----
|
||||
<1> The name of the SR-IOV network attachment definition CR.
|
||||
<2> The InfiniBand GUID for the SR-IOV device. To use this feature, you also must specify `{ "infinibandGUID": true }` in the `SriovIBNetwork` object.
|
||||
<3> The IP addresses for the SR-IOV device that is allocated from the resource type defined in the SR-IOV network attachment definition CR. Both IPv4 and IPv6 addresses are supported. To use this feature, you also must specify `{ "ips": true }` in the `SriovIBNetwork` object.
|
||||
|
||||
.Example runtime configuration
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: sample-pod
|
||||
annotations:
|
||||
k8s.v1.cni.cncf.io/networks: |-
|
||||
[
|
||||
{
|
||||
"name": "ib1",
|
||||
"infiniband-guid": "c2:11:22:33:44:55:66:77",
|
||||
"ips": ["192.168.10.1/24", "2001::1/64"]
|
||||
}
|
||||
]
|
||||
spec:
|
||||
containers:
|
||||
- name: sample-container
|
||||
image: <image>
|
||||
imagePullPolicy: IfNotPresent
|
||||
command: ["sleep", "infinity"]
|
||||
----
|
||||
@@ -11,3 +11,7 @@
|
||||
* Mellanox MT27710 Family [ConnectX-4 Lx] 25GbE dual-port SFP28 with vendor ID `0x15b3` and device ID `0x1015`
|
||||
* Mellanox MT27800 Family [ConnectX-5] 25GbE dual-port SFP28 with vendor ID `0x15b3` and device ID `0x1017`
|
||||
* Mellanox MT27800 Family [ConnectX-5] 100GbE with vendor ID `0x15b3` and device ID `0x1017`
|
||||
* Mellanox MT27700 Family [ConnectX-4] VPI adapter card, EDR IB (100Gb/s), single-port QSFP28 with vendor ID `0x15b3` and device ID `0x1013`
|
||||
* Mellanox MT27800 Family [ConnectX-5] VPI adapter card, EDR IB (100Gb/s), single-port QSFP28 with vendor ID `0x15b3` and device ID `0x1017`
|
||||
* Mellanox MT28908 Family [ConnectX-6] VPI adapter card, 100Gb/s (HDR100, EDR IB), single-port QSFP56 with vendor ID `0x15b3` and device ID `0x101b`
|
||||
* Mellanox MT28908 Family [ConnectX-6] VPI adapter card, HDR200 IB (200Gb/s), single-port QSFP56 with vendor ID `0x15b3` and device ID `0x101b`
|
||||
|
||||
@@ -45,12 +45,16 @@ A device plug-in that discovers, advertises, and allocates SR-IOV network virtua
|
||||
SR-IOV CNI plug-in::
|
||||
A CNI plug-in that attaches VF interfaces allocated from the SR-IOV device plug-in directly into a Pod.
|
||||
|
||||
SR-IOV InfiniBand CNI plug-in::
|
||||
A CNI plug-in that attaches InfiniBand (IB) VF interfaces allocated from the SR-IOV device plug-in directly into a Pod.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
The SR-IOV Network resources injector and SR-IOV Network Operator webhook are enabled by default and can be disabled by editing the `default` SriovOperatorConfig CR.
|
||||
====
|
||||
|
||||
include::modules/nw-sriov-supported-devices.adoc[leveloffset=+2]
|
||||
include::modules/nw-sriov-device-discovery.adoc[leveloffset=+2]
|
||||
include::modules/nw-sriov-example-vf-function-in-pod.adoc[leveloffset=+2]
|
||||
|
||||
[id="about-sriov-next-steps"]
|
||||
|
||||
@@ -7,7 +7,14 @@ toc::[]
|
||||
|
||||
You can add a Pod to an existing Single Root I/O Virtualization (SR-IOV) network.
|
||||
|
||||
include::modules/nw-sriov-runtime-config.adoc[leveloffset=+1]
|
||||
|
||||
include::modules/nw-multus-add-pod.adoc[leveloffset=+1]
|
||||
|
||||
include::modules/nw-sriov-topology-manager.adoc[leveloffset=+1]
|
||||
|
||||
[id="add-pod-additional-resources"]
|
||||
== Additional resources
|
||||
|
||||
- xref:../../networking/hardware_networks/configuring-sriov-device.adoc#configuring-sriov-device[Configuring an SR-IOV Ethernet network attachment]
|
||||
- xref:../../networking/hardware_networks/configuring-sriov-ib-attach.adoc#configuring-sriov-ib-attach[Configuring an SR-IOV InfiniBand network attachment]
|
||||
|
||||
@@ -7,9 +7,11 @@ toc::[]
|
||||
|
||||
You can configure a Single Root I/O Virtualization (SR-IOV) device in your cluster.
|
||||
|
||||
include::modules/nw-sriov-device-discovery.adoc[leveloffset=+1]
|
||||
include::modules/nw-sriov-networknodepolicy-object.adoc[leveloffset=+1]
|
||||
// A direct companion to nw-sriov-networknodepolicy-object
|
||||
include::modules/nw-sriov-nic-partitioning.adoc[leveloffset=+2]
|
||||
|
||||
include::modules/nw-sriov-configuring-device.adoc[leveloffset=+1]
|
||||
include::modules/nw-sriov-nic-partitioning.adoc[leveloffset=+1]
|
||||
|
||||
[id="configuring-sriov-device-next-steps"]
|
||||
== Next steps
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
[id="configuring-sriov-ib-attach"]
|
||||
= Configuring an SR-IOV InfiniBand network attachment
|
||||
include::modules/common-attributes.adoc[]
|
||||
:context: configuring-sriov-ib-attach
|
||||
|
||||
toc::[]
|
||||
|
||||
You can configure an InfiniBand (IB) network attachment for an Single Root I/O Virtualization (SR-IOV) device in the cluster.
|
||||
|
||||
include::modules/nw-sriov-ibnetwork-object.adoc[leveloffset=+1]
|
||||
include::modules/nw-multus-ipam-object.adoc[leveloffset=+2]
|
||||
include::modules/nw-sriov-network-attachment.adoc[leveloffset=+1]
|
||||
|
||||
[id="configuring-sriov-ib-attach-next-steps"]
|
||||
== Next steps
|
||||
|
||||
* xref:../../networking/hardware_networks/add-pod.adoc#add-pod[Adding a Pod to an SR-IOV additional network]
|
||||
|
||||
[id="configuring-sriov-ib-attach-additional-resources"]
|
||||
== Additional resources
|
||||
|
||||
* xref:../../networking/hardware_networks/configuring-sriov-device.adoc#configuring-sriov-device[Configuring an SR-IOV network device]
|
||||
@@ -1,17 +1,22 @@
|
||||
[id="configuring-sriov-net-attach"]
|
||||
= Configuring an SR-IOV network attachment
|
||||
= Configuring an SR-IOV Ethernet network attachment
|
||||
include::modules/common-attributes.adoc[]
|
||||
:context: configuring-sriov-net-attach
|
||||
|
||||
toc::[]
|
||||
|
||||
You can configure a network attachment for an Single Root I/O Virtualization (SR-IOV) device in the cluster.
|
||||
You can configure an Ethernet network attachment for an Single Root I/O Virtualization (SR-IOV) device in the cluster.
|
||||
|
||||
include::modules/nw-sriov-network-attachment.adoc[leveloffset=+1]
|
||||
include::modules/nw-sriov-network-object.adoc[leveloffset=+1]
|
||||
include::modules/nw-multus-ipam-object.adoc[leveloffset=+2]
|
||||
include::modules/nw-sriov-add-pod-runtimeconfig.adoc[leveloffset=+2]
|
||||
include::modules/nw-sriov-network-attachment.adoc[leveloffset=+1]
|
||||
|
||||
[id="configuring-sriov-net-attach-next-steps"]
|
||||
== Next steps
|
||||
|
||||
* xref:../../networking/hardware_networks/add-pod.adoc#add-pod[Adding a Pod to an SR-IOV additional network]
|
||||
|
||||
[id="configuring-sriov-net-attach-additional-resources"]
|
||||
== Additional resources
|
||||
|
||||
* xref:../../networking/hardware_networks/configuring-sriov-device.adoc#configuring-sriov-device[Configuring an SR-IOV network device]
|
||||
|
||||
Reference in New Issue
Block a user