diff --git a/_topic_maps/_topic_map.yml b/_topic_maps/_topic_map.yml index 3b92fe33e9..d897dff7bf 100644 --- a/_topic_maps/_topic_map.yml +++ b/_topic_maps/_topic_map.yml @@ -492,6 +492,8 @@ Topics: File: configuring-private-cluster - Name: Bare metal configuration File: bare-metal-configuration +- Name: Configuring a heterogeneous cluster + File: deploy-heterogeneous-configuration - Name: Machine configuration tasks File: machine-configuration-tasks - Name: Cluster tasks diff --git a/modules/mixed-arch-creating-arm64-bootimage.adoc b/modules/mixed-arch-creating-arm64-bootimage.adoc new file mode 100644 index 0000000000..c069ea86e6 --- /dev/null +++ b/modules/mixed-arch-creating-arm64-bootimage.adoc @@ -0,0 +1,133 @@ +//Module included in the following assemblies +// +//post_installation_configuration/cluster-tasks.adoc + +:_content-type: PROCEDURE +[id="mixed-arch-creating-arm64-bootimage_{context}"] + += Creating an `arm64` boot image using the Azure image gallery + +To configure your heterogeneous cluster, you must create an `arm64` boot image and add it to your Azure machine set. The following procedure describes how to manually generate an `arm64` boot image. + +.Prerequisites + +* You installed the Azure CLI (`az`). +* You created a single-architecture Azure installer-provisioned cluster with the heterogeneous installer binary. + +.Procedure +. Log in to your Azure account: ++ +[source,terminal] +---- +$ az login +---- +. Create a storage account and upload the `arm64` virtual hard disk (VHD) to your storage account. The {product-title} installation program creates a resource group, however, the boot image can also be uploaded to a custom named resource group: ++ +[source,terminal] +---- +$ az storage account create -n ${STORAGE_ACCOUNT_NAME} -g ${RESOURCE_GROUP} -l westus --sku Standard_LRS <1> +---- ++ +<1> The `westus` object is an example region. ++ +. Create a storage container using the storage account you generated: ++ +[source,terminal] ++ +---- +$ az storage container create -n ${CONTAINER_NAME} --account-name ${STORAGE_ACCOUNT_NAME} +---- +. You must use the {product-title} installation program JSON file to extract the URL and `arch64` VHD name: +.. Extract the `URL` field and set it to `RHCOS_VHD_ORIGIN_URL` as the file name by running the following command: ++ +[source,terminal] +---- +$ RHCOS_VHD_ORIGIN_URL=$(./openshift-install coreos print-stream-json | jq -r '.architectures.aarch64."rhel-coreos-extensions"."azure-disk".url') +---- +.. Extract the `aarch64` VHD name and set it to `BLOB_NAME` as the file name by running the following command: ++ +[source,terminal] +---- +$ BLOB_NAME=rhcos-$(./openshift-install coreos print-stream-json | jq -r '.architectures.aarch64."rhel-coreos-extensions"."azure-disk".release')-azure.aarch64.vhd +---- +. Generate a shared access signature (SAS) token. Use this token to upload the {op-system} VHD to your storage container with the following commands: ++ +[source,terminal] +---- +$ end=`date -u -d "30 minutes" '+%Y-%m-%dT%H:%MZ'` +---- ++ +[source,terminal] +---- +$ sas=`az storage container generate-sas -n ${CONTAINER_NAME} --account-name ${STORAGE_ACCOUNT_NAME} --https-only --permissions dlrw --expiry $end -o tsv` +---- +. Copy the {op-system} VHD into the storage container: ++ +[source, terminal] +---- +$ az storage blob copy start --account-name ${STORAGE_ACCOUNT_NAME} --sas-token "$sas" \ + --source-uri "${RHCOS_VHD_ORIGIN_URL}" \ + --destination-blob "${BLOB_NAME}" --destination-container ${CONTAINER_NAME} +---- ++ +You can check the status of the copying process with the following command: ++ +[source,terminal] +---- +$ az storage blob show -c ${CONTAINER_NAME} -n ${BLOB_NAME} --account-name ${STORAGE_ACCOUNT_NAME} | jq .properties.copy +---- ++ +.Example output +[source,terminal] +---- +{ + "completionTime": null, + "destinationSnapshot": null, + "id": "1fd97630-03ca-489a-8c4e-cfe839c9627d", + "incrementalCopy": null, + "progress": "17179869696/17179869696", + "source": "https://rhcos.blob.core.windows.net/imagebucket/rhcos-411.86.202207130959-0-azure.aarch64.vhd", + "status": "success", <1> + "statusDescription": null +} +---- ++ +<1> If the status parameter displays the `success` object, the copying process is complete. + +. Create an image gallery using the following command: ++ +[source,terminal] +---- +$ az sig create --resource-group ${RESOURCE_GROUP} --gallery-name ${GALLERY_NAME} +---- +Use the image gallery to create an image definition. In the following example command, `rhcos-arm64` is the name of the image definition. ++ +[source,terminal] +---- +$ az sig image-definition create --resource-group ${RESOURCE_GROUP} --gallery-name ${GALLERY_NAME} --gallery-image-definition rhcos-arm64 --publisher RedHat --offer arm --sku arm64 --os-type linux --architecture Arm64 --hyper-v-generation V2 +---- +. To get the URL of the VHD and set it to `RHCOS_VHD_URL` as the file name, run the following command: ++ +[source,terminal] +---- +$ RHCOS_VHD_URL=$(az storage blob url --account-name ${STORAGE_ACCOUNT_NAME} -c ${CONTAINER_NAME} -n "${BLOB_NAME}" -o tsv) +---- +. Use the `RHCOS_VHD_URL` file, your storage account, resource group, and image gallery to create an image version. In the following example, `1.0.0` is the image version. ++ +[source,terminal] +---- +$ az sig image-version create --resource-group ${RESOURCE_GROUP} --gallery-name ${GALLERY_NAME} --gallery-image-definition rhcos-arm64 --gallery-image-version 1.0.0 --os-vhd-storage-account ${STORAGE_ACCOUNT_NAME} --os-vhd-uri ${RHCOS_VHD_URL} +---- +. Your `arm64` boot image is now generated. You can access the ID of your image with the following command: ++ +[source,terminal] +---- +$ az sig image-version show -r $GALLERY_NAME -g $RESOURCE_GROUP -i rhcos-arm64 -e 1.0.0 +---- +The following example image ID is used in the `recourseID` parameter of the machine set: ++ +.Example `resourceID` +[source,terminal] +---- +/resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.Compute/galleries/${GALLERY_NAME}/images/rhcos-arm64/versions/1.0.0 +---- \ No newline at end of file diff --git a/modules/mixed-arch-modify-machine-set.adoc b/modules/mixed-arch-modify-machine-set.adoc new file mode 100644 index 0000000000..62e7a166e2 --- /dev/null +++ b/modules/mixed-arch-modify-machine-set.adoc @@ -0,0 +1,107 @@ +//Module included in the following assembly +// +//post_installation_configuration/cluster-tasks.adoc + +:_content-type: PROCEDURE +[id="mixed-arch-modify-machine-set_{context}"] + += Adding a machine set to your cluster using the `arm64` boot image + +To add `arm64` worker nodes to your heterogeneous cluster, you must create an Azure machine set that uses the `arm64` boot image. To create your own custom machine set on Azure, see "Creating a machine set on Azure". + +.Prerequisites + +* You installed the OpenShift CLI (`oc`). + +.Procedure +* Create a machine set and modify the `resourceID` and `vmSize` parameters with the following command. This machine set will control the `arm64` worker nodes in your cluster: ++ +[source,terminal] +---- +$ oc create -f arm64-machine-set-0.yaml +---- +.Sample YAML machine set with `arm64` boot image ++ +[source,yaml] +---- +apiVersion: machine.openshift.io/v1beta1 +kind: MachineSet +metadata: + labels: + machine.openshift.io/cluster-api-cluster: + machine.openshift.io/cluster-api-machine-role: worker + machine.openshift.io/cluster-api-machine-type: worker + name: -arm64-machine-set-0 + namespace: openshift-machine-api +spec: + replicas: 2 + selector: + matchLabels: + machine.openshift.io/cluster-api-cluster: + machine.openshift.io/cluster-api-machineset: -arm64-machine-set-0 + template: + metadata: + labels: + machine.openshift.io/cluster-api-cluster: + machine.openshift.io/cluster-api-machine-role: worker + machine.openshift.io/cluster-api-machine-type: worker + machine.openshift.io/cluster-api-machineset: -arm64-machine-set-0 + spec: + lifecycleHooks: {} + metadata: {} + providerSpec: + value: + acceleratedNetworking: true + apiVersion: machine.openshift.io/v1beta1 + credentialsSecret: + name: azure-cloud-credentials + namespace: openshift-machine-api + image: + offer: "" + publisher: "" + resourceID: /resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.Compute/galleries/${GALLERY_NAME}/images/rhcos-arm64/versions/1.0.0 <1> + sku: "" + version: "" + kind: AzureMachineProviderSpec + location: + managedIdentity: -identity + networkResourceGroup: -rg + osDisk: + diskSettings: {} + diskSizeGB: 128 + managedDisk: + storageAccountType: Premium_LRS + osType: Linux + publicIP: false + publicLoadBalancer: + resourceGroup: -rg + subnet: -worker-subnet + userDataSecret: + name: worker-user-data + vmSize: Standard_D4ps_v5 <2> + vnet: -vnet + zone: "" +---- +<1> Set the `resourceID` parameter to the `arm64` boot image. +<2> Set the `vmSize` parameter to the instance type used in your installation. Some example instance types are `Standard_D4ps_v5` or `D8ps`. + +.Verification +. Verify that the new ARM64 machines are running by entering the following command: ++ +[source,terminal] +---- +$ oc get machineset -n openshift-machine-api +---- ++ +.Example output +[source,terminal] +---- +NAME DESIRED CURRENT READY AVAILABLE AGE +-arm64-machine-set-0 2 2 2 2 10m +---- +. You can check that the nodes are ready and scheduable with the following command: ++ +[source, terminal] +---- +$ oc get nodes +---- \ No newline at end of file diff --git a/modules/mixed-arch-upgrade-mirrors.adoc b/modules/mixed-arch-upgrade-mirrors.adoc new file mode 100644 index 0000000000..acb994e7f9 --- /dev/null +++ b/modules/mixed-arch-upgrade-mirrors.adoc @@ -0,0 +1,22 @@ +//Module included in the following assemblies +// +//post_installation_configuration/cluster-tasks.adoc + +:_content-type: PROCEDURE +[id="mixed-arch-upgrade-mirrors_{context}"] + += Upgrading your heterogeneous cluster + +You must perform an explicit upgrade command to upgrade your existing cluster to a heterogeneous cluster. + +.Prerequisites + +* You installed the OpenShift CLI (`oc`). + +.Procedure +* To manually upgrade your cluster, use the following command: +[source, terminal] +---- +$ oc adm upgrade --allow-explicit-upgrade --to-image <1> +---- +<1> You can access the `image-pullspec` object from the link:https://mirror.openshift.com/pub/openshift-v4/multi/clients/ocp/4.11.0-multi-rc.6/[mixed-arch mirrors page] in the `release.txt` file. \ No newline at end of file diff --git a/post_installation_configuration/deploy-heterogeneous-configuration.adoc b/post_installation_configuration/deploy-heterogeneous-configuration.adoc new file mode 100644 index 0000000000..d3fb0da103 --- /dev/null +++ b/post_installation_configuration/deploy-heterogeneous-configuration.adoc @@ -0,0 +1,29 @@ +:_content-type: ASSEMBLY +:context: deploy-heterogeneous-configuration +[id="post-install-deploy-heterogeneous-configuration"] += Configuring a heterogeneous cluster +include::_attributes/common-attributes.adoc[] + +toc::[] + +A heterogeneous cluster is a cluster that supports worker machines with different architectures. You can deploy a heterogeneous cluster by creating an Azure installer-provisioned cluster using the heterogeneous installer binary. For Azure installation, see xref:../installing/installing_azure/installing-azure-customizations.adoc[Installing on Azure with customizations]. + +[WARNING] +==== +The heterogeneous clusters Technology Preview feature has limited usability with installing, upgrading, and running payloads. +==== + +The following procedures explain how to generate an `arm64` boot image and create an Azure machine set with the `arm64` boot image. This will add `arm64` worker nodes to your heterogeneous cluster and deploy the desired amount of ARM64 virtual machines (VM). This section also shows how to upgrade your existing cluster to a heterogeneous cluster. Heterogeneous clusters are only available on Azure installer-provisioned infrastructures with `x86_64` control planes. + +:FeatureName: Heterogeneous clusters for {product-title} on Azure installer-provisioned infrastructure installations +include::snippets/technology-preview.adoc[leveloffset=+1] + +include::modules/mixed-arch-creating-arm64-bootimage.adoc[leveloffset=+1] + +include::modules/mixed-arch-modify-machine-set.adoc[leveloffset=+1] + +[role="_additional-resources"] +.Additional resources +* xref:../machine_management/creating_machinesets/creating-machineset-azure.adoc[Creating a machine set on Azure] + +include::modules/mixed-arch-upgrade-mirrors.adoc[leveloffset=+1] \ No newline at end of file