mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
152 lines
4.8 KiB
Plaintext
152 lines
4.8 KiB
Plaintext
// Module included in the following assemblies:
|
|
//
|
|
// * virt/managing_vms/advanced_vm_management/virt-configuring-virtual-gpus.adoc
|
|
|
|
:_mod-docs-content-type: PROCEDURE
|
|
[id="virt-creating-exposing-mediated-devices_{context}"]
|
|
= Creating and exposing mediated devices
|
|
|
|
[role="_abstract"]
|
|
As an administrator, you can create mediated devices and expose them to the cluster by editing the `HyperConverged` custom resource (CR). Before you edit the CR, explore a worker node to find the configuration values that are specific to your hardware devices.
|
|
|
|
.Prerequisites
|
|
|
|
* You installed the {oc-first}.
|
|
* You enabled the Input-Output Memory Management Unit (IOMMU) driver.
|
|
* If your hardware vendor provides drivers, you installed them on the nodes where you want to create mediated devices.
|
|
** If you use NVIDIA cards, you link:https://docs.nvidia.com/datacenter/cloud-native/openshift/latest/openshift-virtualization.html[installed the NVIDIA GRID driver].
|
|
|
|
// [IMPORTANT]
|
|
// ====
|
|
// Before {VirtProductName} 4.14, the `mediatedDeviceTypes` field was named `mediatedDevicesTypes`. Ensure that you use the correct field name when configuring mediated devices.
|
|
// ====
|
|
|
|
.Procedure
|
|
|
|
. Identify the name selector and resource name values for the mediated devices by exploring a worker node:
|
|
|
|
.. Start a debugging session with the worker node by using the `oc debug` command. For example:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc debug node/node-11.redhat.com
|
|
----
|
|
|
|
.. Change the root directory of the shell process to the file system of the host node by running the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
# chroot /host
|
|
----
|
|
|
|
.. Navigate to the `mdev_bus` directory and view its contents. Each subdirectory name is a PCI address of a physical GPU. For example:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
# cd sys/class/mdev_bus && ls
|
|
----
|
|
+
|
|
Example output:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
0000:4b:00.4
|
|
----
|
|
|
|
.. Go to the directory for your physical device and list the supported mediated device types as defined by the hardware vendor. For example:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
# cd 0000:4b:00.4 && ls mdev_supported_types
|
|
----
|
|
+
|
|
Example output:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
nvidia-742 nvidia-744 nvidia-746 nvidia-748 nvidia-750 nvidia-752
|
|
nvidia-743 nvidia-745 nvidia-747 nvidia-749 nvidia-751 nvidia-753
|
|
----
|
|
|
|
.. Select the mediated device type that you want to use and identify its name selector value by viewing the contents of its `name` file. For example:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
# cat nvidia-745/name
|
|
----
|
|
+
|
|
Example output:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
NVIDIA A2-2Q
|
|
----
|
|
|
|
. Open the `HyperConverged` CR in your default editor by running the following command:
|
|
+
|
|
[source,terminal,subs="attributes+"]
|
|
----
|
|
$ oc edit hyperconverged kubevirt-hyperconverged -n {CNVNamespace}
|
|
----
|
|
|
|
. Create and expose the mediated devices by updating the configuration:
|
|
|
|
.. Create mediated devices by adding them to the `spec.mediatedDevicesConfiguration` stanza.
|
|
|
|
.. Expose the mediated devices to the cluster by adding the `mdevNameSelector` and `resourceName` values to the `spec.permittedHostDevices.mediatedDevices` stanza. The `resourceName` value is based on the `mdevNameSelector` value, but you use underscores instead of spaces.
|
|
+
|
|
Example `HyperConverged` CR:
|
|
+
|
|
[source,yaml,subs="attributes+"]
|
|
----
|
|
apiVersion: hco.kubevirt.io/v1
|
|
kind: HyperConverged
|
|
metadata:
|
|
name: kubevirt-hyperconverged
|
|
namespace: {CNVNamespace}
|
|
spec:
|
|
mediatedDevicesConfiguration:
|
|
mediatedDeviceTypes:
|
|
- nvidia-745
|
|
nodeMediatedDeviceTypes:
|
|
- mediatedDeviceTypes:
|
|
- nvidia-746
|
|
nodeSelector:
|
|
kubernetes.io/hostname: node-11.redhat.com
|
|
permittedHostDevices:
|
|
mediatedDevices:
|
|
- mdevNameSelector: NVIDIA A2-2Q
|
|
resourceName: nvidia.com/NVIDIA_A2-2Q
|
|
- mdevNameSelector: NVIDIA A2-4Q
|
|
resourceName: nvidia.com/NVIDIA_A2-4Q
|
|
# ...
|
|
----
|
|
+
|
|
where:
|
|
|
|
`mediatedDeviceTypes`:: Specifies global settings for the cluster and is required.
|
|
|
|
`nodeMediatedDeviceTypes`:: Specifies global configuration overrides for a specific node or group of nodes and is optional. Must be used with the global `mediatedDeviceTypes` configuration.
|
|
|
|
`mediatedDeviceTypes`:: Specifies an override to the global `mediatedDeviceTypes` configuration for the specified nodes. Required if you use `nodeMediatedDeviceTypes`.
|
|
|
|
`nodeSelector`:: Specifies the node selector and must include a `key:value` pair. Required if you use `nodeMediatedDeviceTypes`.
|
|
|
|
`mdevNameSelector`:: Specifies the mediated devices that map to this value on the host.
|
|
|
|
`resourceName`:: Specifies the matching resource name that is allocated on the node.
|
|
|
|
. Save your changes and exit the editor.
|
|
|
|
.Verification
|
|
|
|
* Confirm that the virtual GPU is attached to the node by running the following command:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc get node <node_name> -o json \
|
|
| jq '.status.allocatable \
|
|
| with_entries(select(.key | startswith("nvidia.com/"))) \
|
|
| with_entries(select(.value != "0"))'
|
|
----
|