1
0
mirror of https://github.com/openshift/openshift-docs.git synced 2026-02-05 12:46:18 +01:00

[OSDOCS-3593]: Azure machineset support for ultra disks

This commit is contained in:
Jeana Routh
2022-06-07 14:41:15 -04:00
parent b130aa7715
commit 325f95a4cb
6 changed files with 481 additions and 6 deletions

View File

@@ -24,13 +24,28 @@ include::modules/machineset-creating.adoc[leveloffset=+1]
include::modules/machineset-non-guaranteed-instance.adoc[leveloffset=+1]
//Creating Spot VMs by using machine sets
include::modules/machineset-creating-non-guaranteed-instances.adoc[leveloffset=+1]
include::modules/machineset-creating-non-guaranteed-instances.adoc[leveloffset=+2]
//Machine sets that deploy machines on Ephemeral OS disks
include::modules/machineset-azure-ephemeral-os.adoc[leveloffset=+1]
//Creating machines on Ephemeral OS disks by using machine sets
include::modules/machineset-creating-azure-ephemeral-os.adoc[leveloffset=+1]
include::modules/machineset-creating-azure-ephemeral-os.adoc[leveloffset=+2]
//Machine sets that deploy machines on ultra disks as data disks
include::modules/machineset-azure-ultra-disk.adoc[leveloffset=+1]
[role="_additional-resources"]
.Additional resources
* link:https://docs.microsoft.com/en-us/azure/virtual-machines/disks-types#ultra-disks[Microsoft Azure ultra disks documentation]
* xref:../../storage/container_storage_interface/persistent-storage-csi-azure.adoc#machineset-azure-ultra-disk_persistent-storage-csi-azure[Machine sets that deploy machines on ultra disks using CSI PVCs]
* xref:../../storage/persistent_storage/persistent-storage-azure.adoc#machineset-azure-ultra-disk_persistent-storage-azure[Machine sets that deploy machines on ultra disks using in-tree PVCs]
//Creating machines on ultra disks by using machine sets
include::modules/machineset-creating-azure-ultra-disk.adoc[leveloffset=+2]
//Troubleshooting resources for machine sets that enable ultra disks
include::modules/machineset-troubleshooting-azure-ultra-disk.adoc[leveloffset=+2]
//Enabling customer-managed encryption keys for a machine set
include::modules/machineset-customer-managed-encryption-azure.adoc[leveloffset=+1]
@@ -41,12 +56,11 @@ include::modules/machineset-azure-accelerated-networking.adoc[leveloffset=+1]
[role="_additional-resources"]
.Additional resources
* For information about enabling Accelerated Networking during installation, see xref:../../installing/installing_azure/installing-azure-customizations.adoc#machineset-azure-enabling-accelerated-networking-new-install_installing-azure-customizations[Enabling Accelerated Networking during installation].
* xref:../../installing/installing_azure/installing-azure-customizations.adoc#machineset-azure-enabling-accelerated-networking-new-install_installing-azure-customizations[Enabling Accelerated Networking during installation]
// Enabling Accelerated Networking on an existing Microsoft Azure cluster
include::modules/machineset-azure-enabling-accelerated-networking-existing.adoc[leveloffset=+1]
include::modules/machineset-azure-enabling-accelerated-networking-existing.adoc[leveloffset=+2]
[role="_additional-resources"]
.Additional resources
* For more details about scaling a machine set, see xref:../../machine_management/manually-scaling-machineset.adoc#manually-scaling-machineset[Manually scaling a machine set].
* xref:../../machine_management/manually-scaling-machineset.adoc#manually-scaling-machineset[Manually scaling a machine set]

View File

@@ -0,0 +1,45 @@
// Module included in the following assemblies:
//
// * machine_management/creating_machinesets/creating-machineset-azure.adoc
// * storage/persistent_storage/persistent-storage-azure.adoc
// * storage/persistent_storage/persistent-storage-csi-azure.adoc
ifeval::["{context}" == "creating-machineset-azure"]
:mapi:
endif::[]
ifeval::["{context}" == "persistent-storage-azure"]
:pvc:
endif::[]
ifeval::["{context}" == "persistent-storage-csi-azure"]
:pvc:
endif::[]
:_content-type: CONCEPT
[id="machineset-azure-ultra-disk_{context}"]
ifdef::mapi[= Machine sets that deploy machines on ultra disks as data disks]
ifdef::pvc[= Machine sets that deploy machines on ultra disks using PVCs]
You can create a machine set running on Azure that deploys machines on ultra disks. Ultra disks are high-performance storage that are intended for use with the most demanding data workloads.
ifdef::mapi[]
You can also create a persistent volume claim (PVC) that dynamically binds to a storage class backed by Azure ultra disks and mounts them to pods.
[NOTE]
====
Data disks do not support the ability to specify disk throughput or disk IOPS. You can configure these properties by using PVCs.
====
endif::mapi[]
ifdef::pvc[]
Both the in-tree plug-in and CSI driver support using PVCs to enable ultra disks. You can also deploy machines on ultra disks as data disks without creating a PVC.
endif::pvc[]
ifeval::["{context}" == "creating-machineset-azure"]
:!mapi:
endif::[]
ifeval::["{context}" == "persistent-storage-azure"]
:!pvc:
endif::[]
ifeval::["{context}" == "persistent-storage-csi-azure"]
:!pvc:
endif::[]

View File

@@ -0,0 +1,300 @@
// Module included in the following assemblies:
//
// * machine_management/creating_machinesets/creating-machineset-azure.adoc
// * storage/persistent_storage/persistent-storage-azure.adoc
// * storage/persistent_storage/persistent-storage-csi-azure.adoc
ifeval::["{context}" == "creating-machineset-azure"]
:mapi:
endif::[]
ifeval::["{context}" == "persistent-storage-azure"]
:pvc:
endif::[]
ifeval::["{context}" == "persistent-storage-csi-azure"]
:pvc:
endif::[]
:_content-type: PROCEDURE
[id="machineset-creating-azure-ultra-disk_{context}"]
= Creating machines on ultra disks by using machine sets
You can deploy machines on ultra disks on Azure by editing your machine set YAML file.
.Prerequisites
* Have an existing Microsoft Azure cluster.
.Procedure
ifdef::mapi[]
. Create a custom secret in the `openshift-machine-api` namespace using the worker data secret by running the following command:
+
[source,terminal]
----
$ oc -n openshift-machine-api \
get secret worker-user-data \
--template='{{index .data.userData | base64decode}}' | jq > userData.txt
----
+
where `userData.txt` is the name of the new custom secret.
. In a text editor, open the `userData.txt` file and locate the final `}` character in the file.
.. On the immediately preceding line, add a `,`.
.. Create a new line after the `,` and add the following configuration details:
+
[source,json]
----
"storage": {
"disks": [ <1>
{
"device": "/dev/disk/azure/scsi1/lun0", <2>
"partitions": [ <3>
{
"label": "lun0p1", <4>
"sizeMiB": 1024, <5>
"startMiB": 0
}
]
}
],
"filesystems": [ <6>
{
"device": "/dev/disk/by-partlabel/lun0p1",
"format": "xfs",
"path": "/var/lib/lun0p1"
}
]
},
"systemd": {
"units": [ <7>
{
"contents": "[Unit]\nBefore=local-fs.target\n[Mount]\nWhere=/var/lib/lun0p1\nWhat=/dev/disk/by-partlabel/lun0p1\nOptions=defaults,pquota\n[Install]\nWantedBy=local-fs.target\n", <8>
"enabled": true,
"name": "var-lib-lun0p1.mount"
}
]
}
----
<1> The configuration details for the disk that you want to attach to a node as an ultra disk.
<2> Specify the `lun` value that is defined in the `dataDisks` stanza of the machine set you are using. For example, if the machine set contains `lun: 0`, specify `lun0`. You can initialize multiple data disks by specifying multiple `"disks"` entries in this configuration file. If you specify multiple `"disks"` entries, ensure that the `lun` value for each matches the value in the machine set.
<3> The configuration details for a new partition on the disk.
<4> Specify a label for the partition. You might find it helpful to use hierarchical names, such as `lun0p1` for the first partition of `lun0`.
<5> Specify the total size in MiB of the partition.
<6> Specify the filesystem to use when formatting a partition. Use the partition label to specify the partition.
<7> Specify a `systemd` unit to mount the partition at boot. Use the partition label to specify the partition. You can create multiple partitions by specifying multiple `"partitions"` entries in this configuration file. If you specify multiple `"partitions"` entries, you must specify a `systemd` unit for each.
<8> For `Where`, specify the value of `storage.filesystems.path`. For `What`, specify the value of `storage.filesystems.device`.
. Extract the disabling template value to a file called `disableTemplating.txt` by running the following command:
+
[source,terminal]
----
$ oc -n openshift-machine-api get secret worker-user-data \
--template='{{index .data.disableTemplating | base64decode}}' | jq > disableTemplating.txt
----
. Combine the `userData.txt` file and `disableTemplating.txt` file to create a data secret file by running the following command:
+
[source,terminal]
----
$ oc -n openshift-machine-api create secret generic worker-user-data-x5 \
--from-file=userData=userData.txt \
--from-file=disableTemplating=disableTemplating.txt
----
+
where `worker-user-data-x5` is the name of the secret.
endif::mapi[]
. Copy an existing Azure `MachineSet` custom resource (CR) and edit it by running the following command:
+
[source,terminal]
----
$ oc edit machineset <machine-set-name>
----
+
where `<machine-set-name>` is the machine set that you want to provision machines on ultra disks.
. Add the following lines in the positions indicated:
+
[source,yaml]
----
apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
...
spec:
...
template:
...
spec:
metadata:
...
labels:
...
disk: ultrassd <1>
...
providerSpec:
value:
...
ultraSSDCapability: Enabled <2>
ifdef::mapi[]
dataDisks: <2>
- nameSuffix: ultrassd
lun: 0
diskSizeGB: 4
deletionPolicy: Delete
cachingType: None
managedDisk:
storageAccountType: UltraSSD_LRS
userDataSecret:
name: worker-user-data-x5 <3>
endif::mapi[]
...
----
+
<1> Specify a label to use to select a node that is created by this machine set. This procedure uses `disk.ultrassd` for this value.
<2> These lines enable the use of ultra disks.
ifdef::mapi[]
For `dataDisks`, include the entire stanza.
<3> Specify the user data secret created earlier.
endif::mapi[]
. Create a machine set using the updated configuration by running the following command:
+
[source,terminal]
----
$ oc create -f <machine-set-name>.yaml
----
ifdef::pvc[]
. Create a storage class that contains the following YAML definition:
+
[source,yaml]
----
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ultra-disk-sc <1>
parameters:
cachingMode: None
diskIopsReadWrite: "2000" <2>
diskMbpsReadWrite: "320" <3>
kind: managed
skuname: UltraSSD_LRS
provisioner: disk.csi.azure.com <4>
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer <5>
----
<1> Specify the name of the storage class. This procedure uses `ultra-disk-sc` for this value.
<2> Specify the number of IOPS for the storage class.
<3> Specify the throughput in MBps for the storage class.
<4> For Azure Kubernetes Service (AKS) version 1.21 or later, use `disk.csi.azure.com`. For earlier versions of AKS, use `kubernetes.io/azure-disk`.
<5> Optional: Specify this parameter to wait for the creation of the pod that will use the disk.
. Create a persistent volume claim (PVC) to reference the `ultra-disk-sc` storage class that contains the following YAML definition:
+
[source,yaml]
----
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ultra-disk <1>
spec:
accessModes:
- ReadWriteOnce
storageClassName: ultra-disk-sc <2>
resources:
requests:
storage: 4Gi <3>
----
<1> Specify the name of the PVC. This procedure uses `ultra-disk` for this value.
<2> This PVC references the `ultra-disk-sc` storage class.
<3> Specify the size for the storage class. The minimum value is `4Gi`.
. Create a pod that contains the following YAML definition:
+
[source,yaml]
----
apiVersion: v1
kind: Pod
metadata:
name: nginx-ultra
spec:
nodeSelector:
disk: ultrassd <1>
containers:
- name: nginx-ultra
image: alpine:latest
command:
- "sleep"
- "infinity"
volumeMounts:
- mountPath: "/mnt/azure"
name: volume
volumes:
- name: volume
persistentVolumeClaim:
claimName: ultra-disk <2>
----
<1> Specify the label of the machine set that enables the use of ultra disks. This procedure uses `disk.ultrassd` for this value.
<2> This pod references the `ultra-disk` PVC.
endif::pvc[]
.Verification
. Validate that the machines are created by running the following command:
+
[source,terminal]
----
$ oc get machines
----
+
The machines should be in the `Running` state.
. For a machine that is running and has a node attached, validate the partition by running the following command:
+
[source,terminal]
----
$ oc debug node/<node-name> -- chroot /host lsblk
----
+
In this command, `oc debug node/<node-name>` starts a debugging shell on the node `<node-name>` and passes a command with `--`. The passed command `chroot /host` provides access to the underlying host OS binaries, and `lsblk` shows the block devices that are attached to the host OS machine.
.Next steps
* To use an ultra disk from within a pod, create workload that uses the mount point. Create a YAML file similar to the following example:
+
[source,yaml]
----
apiVersion: v1
kind: Pod
metadata:
name: ssd-benchmark1
spec:
containers:
- name: ssd-benchmark1
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- name: lun0p1
mountPath: "/tmp"
volumes:
- name: lun0p1
hostPath:
path: /var/lib/lun0p1
type: DirectoryOrCreate
nodeSelector:
disktype: ultrassd
----
ifeval::["{context}" == "creating-machineset-azure"]
:!mapi:
endif::[]
ifeval::["{context}" == "persistent-storage-azure"]
:!pvc:
endif::[]
ifeval::["{context}" == "persistent-storage-csi-azure"]
:!pvc:
endif::[]

View File

@@ -0,0 +1,86 @@
// Module included in the following assemblies:
//
// * machine_management/creating_machinesets/creating-machineset-azure.adoc
// * storage/persistent_storage/persistent-storage-azure.adoc
// * storage/persistent_storage/persistent-storage-csi-azure.adoc
ifeval::["{context}" == "creating-machineset-azure"]
:mapi:
endif::[]
ifeval::["{context}" == "persistent-storage-azure"]
:pvc:
endif::[]
ifeval::["{context}" == "persistent-storage-csi-azure"]
:pvc:
endif::[]
:_content-type: REFERENCE
[id="machineset-troubleshooting-azure-ultra-disk_{context}"]
= Troubleshooting resources for machine sets that enable ultra disks
Use the information in this section to understand and recover from issues you might encounter.
ifdef::pvc[]
[id="ts-pvc-mounting-ultra_{context}"]
== Unable to mount a persistent volume claim backed by an ultra disk
If there is an issue mounting a persistent volume claim backed by an ultra disk, the pod becomes stuck in the `ContainerCreating` state and an alert is triggered.
For example, if the `additionalCapabilities.ultraSSDEnabled` parameter is not set on the machine that backs the node that hosts the pod, the following error message appears:
[source,terminal]
----
StorageAccountType UltraSSD_LRS can be used only when additionalCapabilities.ultraSSDEnabled is set.
----
* To resolve this issue, describe the pod by running the following command:
+
[source,terminal]
----
$ oc -n <stuck_pod_namespace> describe pod <stuck_pod_name>
----
endif::pvc[]
ifdef::mapi[]
[id="ts-mapi-attach-misconfigure_{context}"]
== Incorrect ultra disk configuration
If an incorrect configuration of the `ultraSSDCapability` parameter is specified in the machine set, the machine provisioning fails.
For example, if the `ultraSSDCapability` parameter is set to `Disabled`, but an ultra disk is specified in the `dataDisks` parameter, the following error message appears:
[source,terminal]
----
StorageAccountType UltraSSD_LRS can be used only when additionalCapabilities.ultraSSDEnabled is set.
----
* To resolve this issue, verify that your machine set configuration is correct.
[id="ts-mapi-attach-unsupported_{context}"]
== Unsupported disk parameters
If a region, availability zone, or instance size that is not compatible with ultra disks is specified in the machine set, the machine provisioning fails. Check the logs for the following error message:
[source,terminal]
----
failed to create vm <machine_name>: failure sending request for machine <machine_name>: cannot create vm: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="BadRequest" Message="Storage Account type 'UltraSSD_LRS' is not supported <more_information_about_why>."
----
* To resolve this issue, verify that you are using this feature in a supported environment and that your machine set configuration is correct.
[id="ts-mapi-delete_{context}"]
== Unable to delete disks
If the deletion of ultra disks as data disks is not working as expected, the machines are deleted and the data disks are orphaned. You must delete the orphaned disks manually if desired.
endif::mapi[]
ifeval::["{context}" == "creating-machineset-azure"]
:!mapi:
endif::[]
ifeval::["{context}" == "persistent-storage-azure"]
:!pvc:
endif::[]
ifeval::["{context}" == "persistent-storage-csi-azure"]
:!pvc:
endif::[]

View File

@@ -29,8 +29,23 @@ In future {product-title} versions, volumes provisioned using existing in-tree p
After full migration, in-tree plug-ins will eventually be removed in later versions of {product-title}.
====
//Machine sets that deploy machines on ultra disks using PVCs
include::modules/machineset-azure-ultra-disk.adoc[leveloffset=+1]
[role="_additional-resources"]
.Additional resources
* link:https://docs.microsoft.com/en-us/azure/virtual-machines/disks-types#ultra-disks[Microsoft Azure ultra disks documentation]
* xref:../../storage/persistent_storage/persistent-storage-azure.adoc#machineset-azure-ultra-disk_persistent-storage-azure[Machine sets that deploy machines on ultra disks using in-tree PVCs]
* xref:../../machine_management/creating_machinesets/creating-machineset-azure.adoc#machineset-azure-ultra-disk_creating-machineset-azure[Machine sets that deploy machines on ultra disks as data disks]
//Creating machines on ultra disks by using machine sets
include::modules/machineset-creating-azure-ultra-disk.adoc[leveloffset=+2]
//Troubleshooting resources for machine sets that enable ultra disks
include::modules/machineset-troubleshooting-azure-ultra-disk.adoc[leveloffset=+2]
[id="additional-resources_persistent-storage-csi-azure"]
[role="_additional-resources"]
== Additional resources
* xref:../../storage/persistent_storage/persistent-storage-azure.adoc#persistent-storage-using-azure[Persistent storage using Azure Disk]
* xref:../../storage/container_storage_interface/persistent-storage-csi.adoc#persistent-storage-csi[Configuring CSI volumes]

View File

@@ -43,3 +43,18 @@ include::modules/storage-azure-create-storage-class.adoc[leveloffset=+1]
include::modules/storage-persistent-storage-creating-volume-claim.adoc[leveloffset=+1]
include::modules/storage-persistent-storage-azure-volume-format.adoc[leveloffset=+1]
//Machine sets that deploy machines on ultra disks using PVCs
include::modules/machineset-azure-ultra-disk.adoc[leveloffset=+1]
[role="_additional-resources"]
.Additional resources
* link:https://docs.microsoft.com/en-us/azure/virtual-machines/disks-types#ultra-disks[Microsoft Azure ultra disks documentation]
* xref:../../storage/container_storage_interface/persistent-storage-csi-azure.adoc#machineset-azure-ultra-disk_persistent-storage-csi-azure[Machine sets that deploy machines on ultra disks using CSI PVCs]
* xref:../../machine_management/creating_machinesets/creating-machineset-azure.adoc#machineset-azure-ultra-disk_creating-machineset-azure[Machine sets that deploy machines on ultra disks as data disks]
//Creating machines on ultra disks by using machine sets
include::modules/machineset-creating-azure-ultra-disk.adoc[leveloffset=+2]
//Troubleshooting resources for machine sets that enable ultra disks
include::modules/machineset-troubleshooting-azure-ultra-disk.adoc[leveloffset=+2]