From ea97fba1f26be4e79a2dbe6843f252dc1cdb8499 Mon Sep 17 00:00:00 2001 From: srir Date: Tue, 25 Feb 2025 18:33:37 +0530 Subject: [PATCH] OSDOCS#13057: Cluster-wide architecture preferred/weighted affinity --- ...ch-creating-podplacment-config-using-cli.adoc | 10 +++++++++- .../multi-arch-creating-podplacment-config.adoc | 16 +++++++++++++++- .../multi-arch-gather-info-about-workloads.adoc | 3 ++- modules/multi-arch-installing-using-cli.adoc | 6 +++--- modules/multi-arch-uninstalling-using-cli.adoc | 6 +++--- .../multiarch-tuning-operator.adoc | 10 +++++++++- 6 files changed, 41 insertions(+), 10 deletions(-) diff --git a/modules/multi-arch-creating-podplacment-config-using-cli.adoc b/modules/multi-arch-creating-podplacment-config-using-cli.adoc index e4d0ff8bf1..faedc9a3e2 100644 --- a/modules/multi-arch-creating-podplacment-config-using-cli.adoc +++ b/modules/multi-arch-creating-podplacment-config-using-cli.adoc @@ -30,7 +30,15 @@ spec: namespaceSelector: matchExpressions: - key: multiarch.openshift.io/exclude-pod-placement - operator: DoesNotExist + operator: DoesNotExist + plugins: + nodeAffinityScoring: + enabled: true + platforms: + - architecture: amd64 + weight: 100 + - architecture: arm64 + weight: 50 ---- . Create the `ClusterPodPlacementConfig` object by running the following command: diff --git a/modules/multi-arch-creating-podplacment-config.adoc b/modules/multi-arch-creating-podplacment-config.adoc index 324b2937e1..be68140122 100644 --- a/modules/multi-arch-creating-podplacment-config.adoc +++ b/modules/multi-arch-creating-podplacment-config.adoc @@ -25,11 +25,25 @@ spec: namespaceSelector: <3> matchExpressions: - key: multiarch.openshift.io/exclude-pod-placement - operator: DoesNotExist + operator: DoesNotExist + plugins: <4> + nodeAffinityScoring: <5> + enabled: true <6> + platforms: <7> + - architecture: amd64 <8> + weight: 100 <9> + - architecture: arm64 + weight: 50 ---- <1> You must set this field value to `cluster`. <2> Optional: You can set the field value to `Normal`, `Debug`, `Trace`, or `TraceAll`. The value is set to `Normal` by default. <3> Optional: You can configure the `namespaceSelector` to select the namespaces in which the Multiarch Tuning Operator's pod placement operand must process the `nodeAffinity` of the pods. All namespaces are considered by default. +<4> Optional: Includes a list of plugins for architecture-aware workload scheduling. +<5> Optional: You can use this plugin to set architecture preferences for pod placement. When enabled, the scheduler first filters out nodes that do not meet the pod’s requirements. Then, it prioritizes the remaining nodes based on the architecture scores defined in the `nodeAffinityScoring.platforms` field. +<6> Optional: Set this field to `true` to enable the `nodeAffinityScoring` plugin. The default value is `false`. +<7> Optional: Defines a list of architectures and their corresponding scores. +<8> Specify the node architecture to score. The scheduler prioritizes nodes for pod placement based on the architecture scores that you set and the scheduling requirements defined in the pod specification. Accepted values are `arm64`, `amd64`, `ppc64le`, or `s390x`. +<9> Assign a score to the architecture. The value for this field must be configured in the range of `1` (lowest priority) to `100` (highest priority). The scheduler uses this score to prioritize nodes for pod placement, favoring nodes with architectures that have higher scores. In this example, the `operator` field value is set to `DoesNotExist`. Therefore, if the `key` field value (`multiarch.openshift.io/exclude-pod-placement`) is set as a label in a namespace, the operand does not process the `nodeAffinity` of the pods in that namespace. Instead, the operand processes the `nodeAffinity` of the pods in namespaces that do not contain the label. diff --git a/modules/multi-arch-gather-info-about-workloads.adoc b/modules/multi-arch-gather-info-about-workloads.adoc index 914584f6ba..0d4c3d0573 100644 --- a/modules/multi-arch-gather-info-about-workloads.adoc +++ b/modules/multi-arch-gather-info-about-workloads.adoc @@ -27,5 +27,6 @@ The following table describes the labels that the Multiarch Tuning Operator adds |`multiarch.openshift.io/scheduling-gate: gated` |The pod is gated. |`multiarch.openshift.io/scheduling-gate: removed` |The pod gate has been removed. |`multiarch.openshift.io/inspection-error: ""` |An error has occurred while building the node affinity requirements. - +|`multiarch.openshift.io/preferred-node-affinity: set` |The Operator has set the architecture preferences in the pod. +|`multiarch.openshift.io/preferred-node-affinity: not-set` |The Operator did not set the architecture preferences in the pod because the user had already set them in the `preferredDuringSchedulingIgnoredDuringExecution` node affinity. |==== \ No newline at end of file diff --git a/modules/multi-arch-installing-using-cli.adoc b/modules/multi-arch-installing-using-cli.adoc index 22b89a91d8..1c4e0d32ec 100644 --- a/modules/multi-arch-installing-using-cli.adoc +++ b/modules/multi-arch-installing-using-cli.adoc @@ -63,7 +63,7 @@ spec: source: redhat-operators sourceNamespace: openshift-marketplace installPlanApproval: Automatic - startingCSV: multiarch-tuning-operator.v1.0.0 + startingCSV: multiarch-tuning-operator. ---- .. Create the `Subscription` object by running the following command: @@ -91,8 +91,8 @@ $ oc get csv -n openshift-multiarch-tuning-operator .Example output [source,terminal] ---- -NAME DISPLAY VERSION REPLACES PHASE -multiarch-tuning-operator.v1.0.0 Multiarch Tuning Operator 1.0.0 multiarch-tuning-operator.v0.9.0 Succeeded +NAME DISPLAY VERSION REPLACES PHASE +multiarch-tuning-operator. Multiarch Tuning Operator multiarch-tuning-operator.1.0.0 Succeeded ---- + The installation is successful if the Operator is in `Succeeded` phase. diff --git a/modules/multi-arch-uninstalling-using-cli.adoc b/modules/multi-arch-uninstalling-using-cli.adoc index 2df2a74b35..ca5724682a 100644 --- a/modules/multi-arch-uninstalling-using-cli.adoc +++ b/modules/multi-arch-uninstalling-using-cli.adoc @@ -47,7 +47,7 @@ $ oc get subscription.operators.coreos.com -n -o .Example output [source,terminal] ---- -currentCSV: multiarch-tuning-operator.v1.0.0 +currentCSV: multiarch-tuning-operator. ---- . Delete the `Subscription` object by running the following command: @@ -70,12 +70,12 @@ subscription.operators.coreos.com "openshift-multiarch-tuning-operator" deleted ---- $ oc delete clusterserviceversion -n <1> ---- -<1> Replace `` with the `currentCSV` value for the Multiarch Tuning Operator. For example: `multiarch-tuning-operator.v1.0.0`. Replace `` with the name of the namespace where you want to uninstall the Multiarch Tuning Operator. +<1> Replace `` with the `currentCSV` value for the Multiarch Tuning Operator. For example: `multiarch-tuning-operator.`. Replace `` with the name of the namespace where you want to uninstall the Multiarch Tuning Operator. + .Example output [source,terminal] ---- -clusterserviceversion.operators.coreos.com "multiarch-tuning-operator.v1.0.0" deleted +clusterserviceversion.operators.coreos.com "multiarch-tuning-operator." deleted ---- .Verification diff --git a/post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator.adoc b/post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator.adoc index 60b97a6e51..53fec285de 100644 --- a/post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator.adoc +++ b/post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator.adoc @@ -12,7 +12,7 @@ Architecture-aware workload scheduling allows the scheduler to place pods onto n By default, the scheduler does not consider the architecture of a pod's container images when determining the placement of new pods onto nodes. -To enable architecture-aware workload scheduling, you must create the `ClusterPodPlacementConfig` object. When you create the `ClusterPodPlacementConfig` object, the Multiarch Tuning Operator deploys the necessary operands to support architecture-aware workload scheduling. +To enable architecture-aware workload scheduling, you must create the `ClusterPodPlacementConfig` object. When you create the `ClusterPodPlacementConfig` object, the Multiarch Tuning Operator deploys the necessary operands to support architecture-aware workload scheduling. You can also use the `nodeAffinityScoring` plugin in the `ClusterPodPlacementConfig` object to set cluster-wide scores for node architectures. If you enable the `nodeAffinityScoring` plugin, the scheduler first filters nodes with compatible architectures and then places the pod on the node with the highest score. When a pod is created, the operands perform the following actions: @@ -30,6 +30,14 @@ Note the following operand behaviors: * If the `nodeSelector` field is not configured with the `kubernetes.io/arch` label for a workload, the operand updates the `nodeAffinity` field for that workload. However, in that `nodeAffinity` field, the operand updates only the node selector terms that are not configured with the `kubernetes.io/arch` label. * If the `nodeName` field is already set, the Multiarch Tuning Operator does not process the pod. + +* If the pod is owned by a DaemonSet, the operand does not update the the `nodeAffinity` field. + +* If both `nodeSelector` or `nodeAffinity` and `preferredAffinity` fields are set for the `kubernetes.io/arch` label, the operand does not update the `nodeAffinity` field. + +* If only `nodeSelector` or `nodeAffinity` field is set for the `kubernetes.io/arch` label and the `nodeAffinityScoring` plugin is disabled, the operand does not update the `nodeAffinity` field. + +* If the `nodeAffinity.preferredDuringSchedulingIgnoredDuringExecution` field already contains terms that score nodes based on the `kubernetes.io/arch` label, the operand ignores the configuration in the `nodeAffinityScoring` plugin. ==== //Installing Multiarch Tuning Operator