Merge pull request #87350 from openshift-cherrypick-robot/cherry-pick-85563-to-enterprise-4.18

_topic_maps/_topic_map.yml

@@ -3278,6 +3278,22 @@ Topics:
       File: telco-core-ref-crs
     - Name: Telco core software specifications
       File: telco-core-ref-software-artifacts
+- Name: Comparing cluster configurations
+  Dir: cluster-compare
+  Distros: openshift-origin,openshift-enterprise
+  Topics:
+  - Name: Understanding the cluster-compare plugin
+    File: understanding-the-cluster-compare-plugin
+  - Name: Installing the cluster-compare plugin
+    File: installing-cluster-compare-plugin
+  - Name: Using the cluster-compare plugin
+    File: using-the-cluster-compare-plugin
+  - Name: Creating a reference configuration
+    File: creating-a-reference-configuration
+  - Name: Performing advanced reference configuration customization
+    File: advanced-ref-config-customization
+  - Name: Troubleshooting cluster comparisons
+    File: troubleshooting-cluster-comparisons
 - Name: Planning your environment according to object maximums
   File: planning-your-environment-according-to-object-maximums
   Distros: openshift-origin,openshift-enterprise

modules/cluster-compare-configure-inline-diff.adoc

@@ -0,0 +1,96 @@
+// Module included in the following assembly:
+//
+// * scalability_and_performance/cluster-compare/creating-a-reference-configuration.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+[id="cluster-compare-configure-inline-diff_{context}"]
+= Configuring inline validation for template fields
+
+You can enable inline regular expressions to validate template fields, especially in scenarios where Golang templating syntax is difficult to maintain or overly complex. Using inline regular expressions simplifies templates, improves readability, and allows for more advanced validation logic.
+
+The `cluster-compare` plugin provides two functions for inline validation:
+
+* `regex`: Validates content in a field using a regular expression.
+* `capturegroups`: Enhances multi-line text comparisons by processing non-capture group text as exact matches, applying regular expression matching only within named capture groups, and ensuring consistency for repeated capture groups.
+
+[id="cluster-compare-configure-regex_{context}"]
+== Configuring inline validation with the regex function
+
+Use the `regex` inline function to validate fields using regular expressions.
+
+.Procedure
+
+. Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+[source,yaml]
+----
+apiVersion: v2
+parts:
+- name: Part1
+  components:
+  - name: Example
+    allOf:
+    - path: example.yaml
+      config:
+        perField:
+        - pathToKey: spec.bigTextBlock <1>
+          inlineDiffFunc: regex <2>
+----
+<1> Specifies the field for inline validation.
+<2> Enables inline validation using regular expressions.
+
+. Use a regular expression to validate the field in the associated template:
++
+[source,yaml]
+----
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  namespace: dashboard
+data:
+  bigTextBlock: |-
+    This is a big text block with some static content, like this line.
+    It also has a place where (?<username>[a-z0-9]+) would put in their own name. (?<username>[a-z0-9]+) would put in their own name.
+----
+
+[id="cluster-compare-configure-capturegroups_{context}"]
+== Configuring inline validation with the capturegroups function
+
+Use the `capturegroups` inline function for more precise validation of fields featuring multi-line strings. 
+
+.Procedure
+
+. Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+[source,yaml]
+----
+apiVersion: v2
+parts:
+- name: Part1
+  components:
+  - name: Example
+    allOf:
+    - path: example.yaml
+      config:
+        perField:
+        - pathToKey: spec.bigTextBlock <1>
+          inlineDiffFunc: capturegroups <2>
+----
+<1> Specifies the field for inline validation.
+<2> Enables inline validation using capture groups.
+
+. Use a regular expression to validate the field in the associated template:
++
+[source,yaml]
+----
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  namespace: dashboard
+data:
+  bigTextBlock: |-
+    This static content outside of a capture group should match exactly.
+    Here is a username capture group: (?<username>[a-z0-9]+). 
+    It should match this capture group: (?<username>[a-z0-9]+).
+----

modules/cluster-compare-exclude-metadata.adoc

@@ -0,0 +1,142 @@
+// Module included in the following assembly:
+//
+// * scalability_and_performance/cluster-compare/creating-a-reference-configuration.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+[id="cluster-compare-exclude-metadata_{context}"]
+= Configuring the metadata.yaml file to exclude template fields
+
+You can configure the `metadata.yaml` file to exclude fields from a comparison. Exclude fields that are irrelevant to a comparison, for example annotations or labels that are inconsequential to a cluster configuration. 
+
+You can configure exclusions in the `metadata.yaml` file in the following ways:
+
+* Exclude all fields in a custom resource not specified in a template.
+
+* Exclude specific fields that you define using the `pathToKey` field.
++
+[NOTE]
+====
+`pathToKey` is a dot separated path. Use quotes to escape key values featuring a period.
+====
+
+[id="cluster-compare-ignore-all-fields_{context}"]
+== Excluding all fields not specified in a template
+
+During the comparison process, the `cluster-compare` plugin renders a template by merging fields from the corresponding custom resource (CR). If you configure the `ignore-unspecified-fields` to `true`, all fields that are present in the CR, but not in the template, are excluded from the merge. Use this approach when you want to focus the comparison on the fields specified in the template only.
+
+.Procedure
+
+* Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+[source,yaml]
+----
+apiVersion: v2
+parts:
+  - name: Part1
+    components:
+      - name: Namespace
+        allOf:
+          - path: namespace.yaml
+            config:
+              ignore-unspecified-fields: true <1>
+#...
+----
+<1> Specify `true` to exclude from the comparison all fields in a CR that are not explicitly configured in the corresponding `namespace.yaml` template.
+
+[id="cluster-compare-ignore-default-fields_{context}"]
+== Excluding specific fields by setting default exclusion fields
+
+You can exclude fields by defining a default value for `fieldsToOmitRefs` in the `defaultOmitRef` field. This default exclusion applies to all templates, unless overridden by the `config.fieldsToOmitRefs` field for a specific template.
+
+.Procedure
+
+* Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+.Example metadata.yaml file
+[source,yaml]
+----
+apiVersion: v2
+parts:
+
+#...
+
+fieldsToOmit:
+   defaultOmitRef: default <1>
+   items:
+      default:
+         - pathToKey: a.custom.default."k8s.io" <2>
+----
+<1> Sets the default exclusion for all templates, unless overridden by the `config.fieldsToOmitRefs` field for a specific template.
+<2> The value is excluded for all templates.
+
+[id="cluster-compare-ignore-specified-fields_{context}"]
+== Excluding specific fields
+
+You can specify fields to exclude by defining the path to the field, and then referencing the definition in the `config` section for a template. 
+
+.Procedure
+
+* Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+.Example metadata.yaml file
+[source,yaml]
+----
+apiVersion: v2
+parts:
+  - name: Part1
+    components:
+      - name: Component1
+        - path: deployment.yaml
+          config:
+            fieldsToOmitRefs:
+              - deployments <1>
+
+#...
+
+fieldsToOmit:
+   items:
+      deployments:
+         - pathToKey: spec.selector.matchLabels.k8s-app <2>
+----
+<1> References the `fieldsToOmit.items.deployments` item for the `deployment.yaml` template.
+<2> Excludes the `spec.selector.matchLabels.k8s-app` field from the comparison. 
++
+[NOTE]
+====
+Setting `fieldsToOmitRefs` replaces the default value.
+====
+
+[id="cluster-compare-ignore-default-groups_{context}"]
+== Excluding specific fields by setting default exclusion groups
+
+You can create default groups of fields to exclude. A group of exclusions can reference another group to avoid duplication when defining exclusions. 
+
+.Procedure
+
+* Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+.Example metadata.yaml file
+[source,yaml]
+----
+apiVersion: v2
+parts:
+
+#...
+
+fieldsToOmit:
+   defaultOmitRef: default
+   items:
+    common:
+      - pathToKey: metadata.annotations."kubernetes.io/metadata.name"
+      - pathToKey: metadata.annotations."kubernetes.io/metadata.name"
+      - pathToKey: metadata.annotations."kubectl.kubernetes.io/last-applied-configuration"
+      - pathToKey: metadata.creationTimestamp
+      - pathToKey: metadata.generation
+      - pathToKey: spec.ownerReferences
+      - pathToKey: metadata.ownerReferences
+    default:
+      - include: common <1>
+      - pathToKey: status
+----
+<1> The `common` group is included in the default group.

modules/cluster-compare-manual-match.adoc

@@ -0,0 +1,34 @@
+// Module included in the following assembly:
+//
+// * scalability_and_performance/cluster-compare/advanced-ref-config-customization.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+[id="cluster-compare-manual-match_{context}"]
+= Configuring manual matching between CRs and templates
+
+For scenarios where the `cluster-compare` plugin's default matching does not work as expected, you can manually match a custom resource (CR) to a template. 
+
+For example, if there is more than one CR in the cluster with the same `apiversion`, `kind`, `name`, and `namespace` fields, the plugin's default matching compares the CR that features the least differences. To control what CR the plugin chooses, you can create a user configuration YAML file with the manual matching configuration, then pass this configuration file to the `cluster-compare` command.
+
+.Procedure
+
+. Create a user configuration file to define the manual matching criteria:
++
+.Example `user-config.yaml` file
+[source,yaml]
+----
+correlationSettings:
+   manualCorrelation:
+      correlationPairs:
+         apps.v1.DaemonSet.kube-system.kindnet.yaml: "template_example.yaml" <1>
+----
+<1> Specifies the CR and template pair to match. The CR specification uses the following format: `<apiversion>.<kind>.<namespace>.<name>`. For cluster scoped CRs that do not have a namespace, use the format `<apiversion>.<kind>.<name>`.
+
+. Reference the user configuration file in a `cluster-compare` command by running the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -r <path_to_reference_config>/metadata.yaml -c <path_to_user_config>/user-config.yaml <1>
+----
+<1> Specify the `user-config.yaml` file by using the `-c` option.

modules/cluster-compare-metadata-structure.adoc

@@ -0,0 +1,30 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/creating-a-reference-configuration.adoc
+
+:_mod-docs-content-type: CONCEPT
+
+[id="cluster-compare-metadata-structure_{context}"]
+= Structure of the metadata.yaml file
+
+The `metadata.yaml` file provides a central configuration point to define and configure the templates in a reference configuration. 
+The file features a hierarchy of `parts` and `components`. `parts` are groups of `components` and `components` are groups of templates.
+Under each component, you can configure template dependencies, validation rules, and add descriptive metadata.
+
+.Example metadata.yaml file
+[source,yaml]
+----
+apiVersion: v2
+parts: <1>
+  - name: Part1 <2>
+    components:
+      - name: Component1 <3>
+        <component1_configuration> <4>
+  - name: Part2
+      - name: Component2
+        <component2_configuration>
+----
+<1> Every `part` typically describes a workload or a set of workloads.
+<2> Specify a `part` name.
+<3> Specify a `component` name.
+<4> Specify the configuration for a template. For example, define template relationships or configure what fields to use in a comparison.

modules/cluster-compare-output-description.adoc

@@ -0,0 +1,43 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/creating-a-reference-configuration.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+[id="cluster-compare-output-description_{context}"]
+= Configuring descriptions for the output
+
+Each part, component, or template can include descriptions to provide additional context, instructions, or documentation links. These descriptions are helpful to convey why a specific template or structure is required.
+
+.Procedure
+
+* Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+[source,yaml]
+----
+apiVersion: v2
+parts:
+  - name: Part1
+    description: |-
+      General text for every template under this part, unless overridden.
+    components:
+      - name: Component1
+        # With no description set, this inherits the description from the part above.
+        OneOf:
+          - path: Template1.yaml
+            # This inherits the component description, if set.
+          - path: Template2.yaml
+          - path: Template3.yaml
+            description: |-
+              This template has special instructions that don't apply to the others.
+      - name: Component2
+        description: |-
+          This overrides the part text with something more specific.
+          Multi-line text is supported, at all levels.
+        allOf:
+          - path: RequiredTemplate1.yaml
+          - path: RequiredTemplate2.yaml
+            description: |-
+              Required for important reasons.
+          - path: RequiredTemplate3.yaml
+----

modules/cluster-compare-overview.adoc

@@ -0,0 +1,27 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/understanding-the-cluster-compare-plugin.adoc
+
+:_mod-docs-content-type: CONCEPT
+
+[id="cluster-compare-overview_{context}"]
+= Overview of the cluster-compare plugin
+
+Clusters deployed at scale typically use a validated set of baseline custom resources (CRs) to configure clusters to meet use-case requirements and ensure consistency when deploying across different environments. 
+
+In live clusters, some variation from the validated set of CRs is expected. For example, configurations might differ because of variable substitution, optional components, or hardware-specific fields. This variation makes it difficult to accurately assess if a cluster is compliant with the baseline configuration.
+
+Using the `cluster-compare` plugin with the `oc` command, you can compare the configuration from a live cluster with a reference configuration. A reference configuration represents the baseline configuration but uses the various plugin features to suppresses expected variation during a comparison. For example, you can apply validation rules, specify optional and required resources, and define relationships between resources. By reducing irrelevant differences, the plugin makes it easier to assess cluster compliance with baseline configurations, and across environments.
+
+The ability to intelligently compare a configuration from a cluster with a reference configuration has the following example use-cases:
+
+*Production:* Ensure compliance with a reference configuration across service updates, upgrades and changes to the reference configuration.
+
+*Development:* Ensure compliance with a reference configuration in test pipelines.
+
+*Design:* Compare configurations with a partner lab reference configuration to ensure consistency.
+
+*Support:* Compare the reference configuration to `must-gather` data from a live cluster to troubleshoot configuration issues.
+
+.Cluster-compare plugin overview
+image::493-OpenShift-0125.png[Cluster-compare plugin overview]

modules/cluster-compare-patching.adoc

@@ -0,0 +1,202 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/advanced-ref-config-customization.adoc
+
+:_mod-docs-content-type: PROCEDURE
+[id="cluster-compare-patching_{context}"]
+= Patching a reference configuration
+
+In certain scenarios, you might need to patch the reference configuration to handle expected deviations in a cluster configuration. The plugin applies the patch during the comparison process, modifying the specified resource fields as defined in the patch file.
+
+For example, you might need to temporarily patch a template because a cluster uses a deprecated field that is out-of-date with the latest reference configuration. Patched files are reported in the comparison output summary.
+
+You can create a patch file in two ways:
+
+* Use the `cluster-compare` plugin to generate a patch YAML file. 
+
+* Create your own patch file.
+
+== Using the cluster-compare plugin to generate a patch
+
+You can use the `cluster-compare` plugin to generate a patch for specific template files. The plugin adjusts the template to ensure it matches with the cluster custom resource (CR). Any previously valid differences in the patched template are not reported. The plugin highlights the patched files in the output.
+
+.Procedure
+
+. Generate patches for templates by running the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -r <path_to_reference_config>/metadata.yaml -o 'generate-patches' --override-reason "A valid reason for the override" --generate-override-for "<template1_path>" --generate-override-for "<template2_path>" > <path_to_patches_file>
+----
++
+* `-r` specifies the path to the metadata.yaml file of the reference configuration. 
+* `-o` specifies the output format. To generate a patch output, you must use the `generate-patches` value.
+* `--override-reason` describes the reason for the patch.
+* `--generate-override-for` specifies a path to the template that requires a patch.
++
+[NOTE]
+====
+You must use a file path for the target template that is relative to the `metadata.yaml` file. For example, if the file path for the `metadata.yaml` file is `./compare/metadata.yaml`, a relative file path for the template might be `optional/my-template.yaml`.
+====
++
+* `<path_to_patches_file>` specifies the filename and path for your patch.
+
+. Optional: Review the patch file before applying to the reference configuration:
++
+.Example `patch-config` file
+[source,yaml]
+----
+- apiVersion: storage.k8s.io/v1
+  kind: StorageClass
+  name: crc-csi-hostpath-provisioner
+  patch: '{"provisioner":"kubevirt.io.hostpath-provisioner"}' <1>
+  reason: A valid reason for the override
+  templatePath: optional/local-storage-operator/StorageClass.yaml <2>
+  type: mergepatch <3>
+----
+<1> The plugin patches the fields in the template to match the CR.
+<2> The path to the template.
+<3> The `mergepath` option merges the JSON into the target template. Unspecified fields remain unchanged.
+
+. Apply the patch to the reference configuration by running the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -r <referenceConfigurationDirectory> -p <path_to_patches_file>
+----
++
+* `-r` specifies the path to the metadata.yaml file of the reference configuration. 
+* `-p` specifies the path to the patch file.
++
+.Example output
+[source,terminal]
+----
+...
+
+Cluster CR: storage.k8s.io/v1_StorageClass_crc-csi-hostpath-provisioner
+Reference File: optional/local-storage-operator/StorageClass.yaml
+Description: Component description
+Diff Output: None
+Patched with patch
+Patch Reasons:
+- A valid reason for the override
+
+...
+
+No CRs are unmatched to reference CRs
+Metadata Hash: bb2165004c496b32e0c8509428fb99c653c3cf4fba41196ea6821bd05c3083ab
+Cluster CRs with patches applied: 1
+----
+
+
+== Creating a patch file manually
+
+You can write a patch file to handle expected deviations in a cluster configuration. 
+
+[NOTE]
+====
+Patches have three possible values for the `type` field: 
+
+* `mergepatch` - Merges the JSON into the target template. Unspecified fields remain unchanged.
+* `rfc6902` - Merges the JSON in the target template using `add`, `remove`, `replace`, `move`, and `copy` operations. Each operation targets a specific path.
+* `go-template` - Defines a Golang template. The plugin renders the template using the cluster custom resource (CR) as input and generates either a `mergepatch` or `rfc6902` patch for the target template.
+
+The following example shows the same patch using all three different formats.
+====
+
+.Procedure
+
+. Create a patch file to match your use case. Use the following structure as an example:
++
+.Example `patch-config`
+[source,yaml]
+----
+- apiVersion: v1 <1>
+  kind: Namespace
+  name: openshift-storage
+  reason: known deviation
+  templatePath: namespace.yaml
+  type: mergepatch
+  patch: '{"metadata":{"annotations":{"openshift.io/sa.scc.mcs":"s0:c29,c14","openshift.io/sa.scc.supplemental-groups":"1000840000/10000","openshift.io/sa.scc.uid-range":"1000840000/10000","reclaimspace.csiaddons.openshift.io/schedule":"@weekly","workload.openshift.io/allowed":null},"labels":{"kubernetes.io/metadata.name":"openshift-storage","olm.operatorgroup.uid/ffcf3f2d-3e37-4772-97bc-983cdfce128b":"","openshift.io/cluster-monitoring":"false","pod-security.kubernetes.io/audit":"privileged","pod-security.kubernetes.io/audit-version":"v1.24","pod-security.kubernetes.io/warn":"privileged","pod-security.kubernetes.io/warn-version":"v1.24","security.openshift.io/scc.podSecurityLabelSync":"true"}},"spec":{"finalizers":["kubernetes"]}}'
+- name: openshift-storage
+  apiVersion: v1
+  kind: Namespace
+  templatePath: namespace.yaml
+  type: rfc6902
+  reason: known deviation
+  patch: '[
+    {"op": "add", "path": "/metadata/annotations/openshift.io~1sa.scc.mcs", "value": "s0:c29,c14"}, 
+    {"op": "add", "path": "/metadata/annotations/openshift.io~1sa.scc.supplemental-groups", "value": "1000840000/10000"}, 
+    {"op": "add", "path": "/metadata/annotations/openshift.io~1sa.scc.uid-range", "value": "1000840000/10000"}, 
+    {"op": "add", "path": "/metadata/annotations/reclaimspace.csiaddons.openshift.io~1schedule", "value": "@weekly"}, 
+    {"op": "remove", "path": "/metadata/annotations/workload.openshift.io~1allowed"},
+    {"op": "add", "path": "/metadata/labels/kubernetes.io~1metadata.name", "value": "openshift-storage"}, 
+    {"op": "add", "path": "/metadata/labels/olm.operatorgroup.uid~1ffcf3f2d-3e37-4772-97bc-983cdfce128b", "value": ""}, 
+    {"op": "add", "path": "/metadata/labels/openshift.io~1cluster-monitoring", "value": "false"}, 
+    {"op": "add", "path": "/metadata/labels/pod-security.kubernetes.io~1audit", "value": "privileged"}, 
+    {"op": "add", "path": "/metadata/labels/pod-security.kubernetes.io~1audit-version", "value": "v1.24"}, 
+    {"op": "add", "path": "/metadata/labels/pod-security.kubernetes.io~1warn", "value": "privileged"}, 
+    {"op": "add", "path": "/metadata/labels/pod-security.kubernetes.io~1warn-version", "value": "v1.24"}, 
+    {"op": "add", "path": "/metadata/labels/security.openshift.io~1scc.podSecurityLabelSync", "value": "true"}, 
+    {"op": "add", "path": "/spec", "value": {"finalizers": ["kubernetes"]}}
+    ]'
+- apiVersion: v1
+  kind: Namespace
+  name: openshift-storage
+  reason: "known deviation"
+  templatePath: namespace.yaml
+  type: go-template
+  patch: |
+    {
+        "type": "rfc6902",
+        "patch": '[
+            {"op": "add", "path": "/metadata/annotations/openshift.io~1sa.scc.mcs", "value": "s0:c29,c14"},
+            {"op": "add", "path": "/metadata/annotations/openshift.io~1sa.scc.supplemental-groups", "value": "1000840000/10000"},
+            {"op": "add", "path": "/metadata/annotations/openshift.io~1sa.scc.uid-range", "value": "1000840000/10000"},
+            {"op": "add", "path": "/metadata/annotations/reclaimspace.csiaddons.openshift.io~1schedule", "value": "@weekly"},
+            {"op": "remove", "path": "/metadata/annotations/workload.openshift.io~1allowed"},
+            {"op": "add", "path": "/metadata/labels/kubernetes.io~1metadata.name", "value": "openshift-storage"},
+            {"op": "add", "path": "/metadata/labels/olm.operatorgroup.uid~1ffcf3f2d-3e37-4772-97bc-983cdfce128b", "value": ""},
+            {"op": "add", "path": "/metadata/labels/openshift.io~1cluster-monitoring", "value": "false"},
+            {"op": "add", "path": "/metadata/labels/pod-security.kubernetes.io~1audit", "value": "privileged"},
+            {"op": "add", "path": "/metadata/labels/pod-security.kubernetes.io~1audit-version", "value": "v1.24"},
+            {"op": "add", "path": "/metadata/labels/pod-security.kubernetes.io~1warn", "value": "privileged"},
+            {"op": "add", "path": "/metadata/labels/pod-security.kubernetes.io~1warn-version", "value": "v1.24"},
+            {"op": "add", "path": "/metadata/labels/security.openshift.io~1scc.podSecurityLabelSync", "value": "true"},
+            {"op": "add", "path": "/spec", "value": {"finalizers": {{ .spec.finalizers | toJson }} }}
+        ]'
+    }
+----
+<1> The patches uses the `kind`, `apiVersion`, `name`, and `namespace` fields to match the patch with the correct cluster CR.
+
+. Apply the patch to the reference configuration by running the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -r <referenceConfigurationDirectory> -p <path_to_patches_file>
+----
++
+* `-r` specifies the path to the metadata.yaml file of the reference configuration. 
+* `p` specifies the path to the patch file.
++
+.Example output
+[source,terminal]
+----
+...
+
+Cluster CR: storage.k8s.io/v1_StorageClass_crc-csi-hostpath-provisioner
+Reference File: namespace.yaml
+Description: Component description
+Diff Output: None
+Patched with patch
+Patch Reasons:
+- known deviation
+- known deviation
+- known deviation
+
+...
+
+No CRs are unmatched to reference CRs
+Metadata Hash: bb2165004c496b32e0c8509428fb99c653c3cf4fba41196ea6821bd05c3083ab
+Cluster CRs with patches applied: 1
+----

modules/cluster-compare-reference-args.adoc

@@ -0,0 +1,65 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/using-the-cluster-compare-plugin.adoc
+
+:_mod-docs-content-type: REFERENCE
+
+[id="cluster-compare-reference-args_{context}"]
+= Reference cluster-compare plugin options
+
+The following content describes the options for the `cluster-compare` plugin. 
+
+.Cluster-compare plugin options
+[options="header"]
+[cols="2a,3a"]
+|====
+| Option | Description
+| `-A`, `--all-resources` 
+| When used with a live cluster, attempts to match all resources in the cluster that match a type in the reference configuration. When used with local files, attempts to match all resources in the local files that match a type in the reference configuration.
+
+| `--concurrency` 
+| Specify an integer value for the number of templates to process in parallel when comparing with resources from the live version. A larger number increases speed but also memory, I/O, and CPU usage during that period. The default value is `4`.
+
+| `-c`, `--diff-config` 
+| Specify the path to the user configuration file.
+
+| `-f`, `--filename` 
+| Specify a filename, directory, or URL for the configuration custom resources that you want to use for a comparison with a reference configuration.
+
+| `--generate-override-for` 
+| Specify the path for templates that requires a patch.
+
+[NOTE]
+====
+You must use a file path for the target template that is relative to the `metadata.yaml` file. For example, if the file path for the `metadata.yaml` file is `./compare/metadata.yaml`, a relative file path for the template might be `optional/my-template.yaml`.
+====
+
+| `-h`, `--help` 
+| Display help information.
+
+| `-k`, `--kustomize` 
+| Specify a path to process the `kustomization` directory. This flag cannot be used together with `-f` or `-R`.
+
+| `-o`, `--output` 
+| Specify the output format. Options include `json`, `yaml`, or `generate-patches`.
+
+| `--override-reason` 
+| Specify a reason for generating the override.
+
+| `-p`, `--overrides` 
+| Specify a path to a patch override file for the reference configuration.
+
+| `-R`, `--recursive` 
+| Processes the directory specified in `-f`, `--filename` recursively. 
+
+| `-r`, `--reference` 
+| Specify the path to the reference configuration `metadata.yaml` file.
+
+| `--show-managed-fields` 
+| Specify `true` to include managed fields in the comparison.
+
+| `-v`, `--verbose` 
+| Increases the verbosity of the plugin output.
+
+|====
+

modules/cluster-compare-template-groupings.adoc

@@ -0,0 +1,55 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/creating-a-reference-configuration.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+[id="cluster-compare-template-groupings_{context}"]
+= Configuring template relationships
+
+By defining relationships between templates in your reference configuration, you can support use-cases with complex dependencies. For example, you can configure a component to require specific templates, require one template from a group, or allow any template from a group, and so on. 
+
+.Procedure
+
+* Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+.Example metadata.yaml file
+[source,yaml]
+----
+apiVersion: v2
+parts:
+  - name: Part1
+    components:
+      - name: Component1
+        allOf: <1>
+          - path: RequiredTemplate1.yaml
+          - path: RequiredTemplate2.yaml
+      - name: Component2
+        allOrNoneOf: <2>
+          - path: OptionalBlockTemplate1.yaml
+          - path: OptionalBlockTemplate2.yaml
+      - name: Component3
+        anyOf: <3>
+          - path: OptionalTemplate1.yaml
+          - path: OptionalTemplate2.yaml
+      - name: Component4
+        noneOf: <4>
+          - path: BannedTemplate1.yaml
+          - path: BannedTemplate2.yaml
+      - name: Component5
+        oneOf: <5>
+          - path: RequiredExclusiveTemplate1.yaml
+          - path: RequiredExclusiveTemplate2.yaml
+      - name: Component6
+        anyOneOf: <6>
+          - path: OptionalExclusiveTemplate1.yaml
+          - path: OptionalExclusiveTemplate2.yaml
+#...
+
+----
+<1> Specifies required templates.
+<2> Specifies a group of templates that are either all required or all optional. If one corresponding custom resource (CR) is present in the cluster, then all corresponding CRs must be present in the cluster.
+<3> Specifies optional templates.
+<4> Specifies templates to exclude. If a corresponding CR is present in the cluster, the plugin returns a validation error.
+<5> Specifies templates where only one can be present. If none, or more than one of the corresponding CRs are present in the cluster, the plugin returns a validation error .
+<6> Specifies templates where only one can be present in the cluster. If more than one of the corresponding CRs are present in the cluster, the plugin returns a validation error.

modules/cluster-compare-templating.adoc

@@ -0,0 +1,50 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/creating-a-reference-configuration.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+[id="cluster-compare-templating_{context}"]
+= Configuring expected variation in a template
+
+You can handle variable content within a template by using Golang templating syntax. Using this syntax, you can configure validation logic that handles optional, required, and conditional content within the template.  
+
+[NOTE]
+====
+* The `cluster-compare` plugin requires all templates to render as valid YAML. To avoid parsing errors for missing fields, use conditional templating syntax such as `{{- if .spec.<optional_field> }}` when implementing templating syntax. This conditional logic ensures templates process missing fields gracefully and maintains valid YAML formatting.
+
+* You can use the Golang templating syntax with custom and built-in functions for complex use cases. All Golang built-in functions are supported including the functions in the Sprig library.
+====
+
+.Procedure
+
+* Create a `metadata.yaml` file to match your use case. Use the following structure as an example:
++
+[source,yaml]
+----
+apiVersion: v2
+kind: Service
+metadata:
+  name: frontend <1>
+  namespace: {{ .metadata.namespace }}  <2>
+  labels:
+    app: guestbook
+    tier: frontend
+spec:
+  {{- if and .spec.type (eq (.spec.type) "NodePort" "LoadBalancer") }}
+  type: {{.spec.type }} <3>
+  {{- else }}
+  type: should be NodePort or LoadBalancer
+  {{- end }}
+  ports:
+  - port: 80
+  selector:
+    app: guestbook
+    {{- if .spec.selector.tier }} <4>
+    tier: frontend
+    {{- end }}
+----
+<1> Configures a required field that must match the specified value.
+<2> Configures a required field that can have any value.
+<3> Configures validation for the `.spec.type` field.
+<4> Configures an optional field.

modules/installing-cluster-compare.adoc

@@ -0,0 +1,77 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/installing-cluster-compare-plugin.adoc
+
+:_mod-docs-content-type: CONCEPT
+
+[id="installing-cluster-compare_{context}"]
+= Installing the cluster-compare plugin
+
+Install the `cluster-compare` plugin to compare a reference configuration with a cluster configuration from a live cluster or `must-gather` data.
+
+.Prerequisites
+
+. You have installed the OpenShift CLI (`oc`).
+
+. You installed `podman`.
+
+. You have access to the Red Hat container catalog.
+
+.Procedure
+
+. Log in to the Red Hat container catalog by running the following command:
++
+[source,terminal]
+----
+$ podman login registry.access.redhat.com
+----
+
+. Create a container for the `cluster-compare` image by running the following command:
++
+[source,terminal]
+----
+$ podman create --name cca registry.redhat.io/openshift4/kube-compare-artifacts-rhel9:latest
+----
+
+. Copy the `cluster-compare` plugin to a directory that is included in your `PATH` environment variable by running the following command:
++
+[source,terminal]
+----
+$ podman cp cca:/usr/share/openshift/<arch>/kube-compare.<rhel_version> <directory_on_path>/kubectl-cluster_compare
+----
++
+* `arch` is the architecture for your machine. Valid values are:
+** `linux_amd64`
+** `linux_arm64`
+** `linux_ppc64le`
+** `linux_s390x`
+* `<rhel_version>` is the version of {op-system-base} on your machine. Valid values are `rhel8` or `rhel9`.
+* `<directory_on_path>` is the path to a directory included in your `PATH` environment variable.
+
+
+.Verification
+
+* View the help for the plugin by running the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -h
+----
++
+.Example output
+[source,terminal]
+----
+Compare a known valid reference configuration and a set of specific cluster configuration CRs.
+
+...
+
+Usage:
+  compare -r <Reference File>
+
+Examples:
+  # Compare a known valid reference configuration with a live cluster:
+  kubectl cluster-compare -r ./reference/metadata.yaml
+
+ ...
+----
+

modules/telco-core-rds-container.adoc

@@ -29,7 +29,7 @@ $ podman run -it registry.redhat.io/openshift4/openshift-telco-core-rds-rhel9:v4
 
 .Verification
 
-* The `out` directory has the following folder structure. You can view the telco core CRs in the `out/telco-core-rds/` directory.
+* The `out` directory has the following directory structure. You can view the telco core CRs in the `out/telco-core-rds/` directory.
 +
 .Example output
 [source,text]

modules/understanding-a-reference-config.adoc

@@ -0,0 +1,38 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/understanding-the-cluster-compare-plugin.adoc
+
+:_mod-docs-content-type: CONCEPT
+
+[id="understanding-a-reference-config_{context}"]
+= Understanding a reference configuration
+
+The `cluster-compare` plugin uses a reference configuration to validate a configuration from a live cluster.
+The reference configuration consists of a YAML file called `metadata.yaml`, which references a set of templates that represent the baseline configuration.
+
+.Example directory structure for a reference configuration
+[source,text]
+----
+├── metadata.yaml <1>
+├── optional <2>
+│   ├── optionalTemplate1.yaml
+│   └── optionalTemplate2.yaml 
+├── required
+│   ├── requiredTemplate3.yaml
+│   └── requiredTemplate4.yaml
+└── baselineClusterResources <3>
+    ├── clusterResource1.yaml
+    ├── clusterResource2.yaml
+    ├── clusterResource3.yaml
+    └── clusterResource4.yaml
+----
+<1> The reference configuration consists of the `metadata.yaml` file and a set of templates.
+<2> This example uses an optional and required directory structure for templates that are referenced by the `metadata.yaml` file.
+<3> The configuration CRs to use as a baseline configuration for clusters.
+
+During a comparison, the plugin matches each template to a configuration resource from the cluster. 
+The plugin evaluates optional or required fields in the template using features such as Golang templating syntax and inline regular expression validation. The `metadata.yaml` file applies additional validation rules to decide whether a template is optional or required and assesses template dependency relationships.
+
+Using these features, the plugin identifies relevant configuration differences between the cluster and the reference configuration. For example, the plugin can highlight mismatched field values, missing resources, extra resources, field type mismatches, or version discrepancies.
+
+For further information about configuring a reference configuration, see "Creating a reference configuration".

modules/using-cluster-compare-live-cluster.adoc

@@ -0,0 +1,86 @@
+
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/using-the-cluster-compare-plugin.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+[id="using-cluster-compare-live-cluster_{context}"]
+= Using the cluster-compare plugin with a live cluster
+
+You can use the `cluster-compare` plugin to compare a reference configuration with configuration custom resources (CRs) from a live cluster. 
+
+Validate live cluster configurations to ensure compliance with reference configurations during design, development, or testing scenarios.
+
+[NOTE]
+====
+Use the `cluster-compare` plugin with live clusters in non-production environments only. For production environments, use the plugin with `must-gather` data.
+====
+
+.Prerequisites
+
+* You installed the OpenShift CLI (`oc`).
+
+* You have access to the cluster as a user with the `cluster-admin` role.
+
+* You downloaded the `cluster-compare` plugin and include it in your `PATH` environment variable.
+
+* You have access to a reference configuration. 
+
+.Procedure
+
+* Run the `cluster-compare` plugin by using the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -r <path_to_reference_config>/metadata.yaml
+----
++
+** `-r` specifies a path to the `metadata.yaml` file of the reference configuration. You can specify a local directory or a URI.
++
+.Example output
+[source,terminal]
+----
+
+...
+
+**********************************
+
+Cluster CR: operator.openshift.io/v1_Console_cluster <1>
+Reference File: optional/console-disable/ConsoleOperatorDisable.yaml <2>
+Diff Output: diff -u -N /tmp/MERGED-622469311/operator-openshift-io-v1_console_cluster /tmp/LIVE-2358803347/operator-openshift-io-v1_console_cluster
+/tmp/MERGED-622469311/operator-openshift-io-v1_console_cluster	2024-11-20 15:43:42.888633602 +0000
++++ /tmp/LIVE-2358803347/operator-openshift-io-v1_console_cluster	2024-11-20 15:43:42.888633602 +0000
+@@ -4,5 +4,5 @@
+   name: cluster
+ spec:
+   logLevel: Normal
+-  managementState: Removed <3>
++  managementState: Managed
+   operatorLogLevel: Normal
+
+**********************************
+
+…
+
+Summary <4>
+CRs with diffs: 5/49 <5> 
+CRs in reference missing from the cluster: 1 <6> 
+required-cluster-tuning:
+  cluster-tuning:
+    Missing CRs: <7> 
+    - required/cluster-tuning/disabling-network-diagnostics/DisableSnoNetworkDiag.yaml
+No CRs are unmatched to reference CRs <8> 
+Metadata Hash: 512a9bf2e57fd5a5c44bbdea7abb3ffd7739d4a1f14ef9021f6793d5cdf868f0 <9>
+No patched CRs <10>
+----
+<1> The CR under comparison. The plugin displays each CR with a difference from the corresponding template.
+<2> The template matching with the CR for comparison.
+<3> The output in Linux diff format shows the difference between the template and the cluster CR.
+<4> After the plugin reports the line diffs for each CR, the summary of differences are reported.
+<5> The number of CRs in the comparison with differences from the corresponding templates.
+<6> The number of CRs represented in the reference configuration, but missing from the live cluster.
+<7> The list of CRs represented in the reference configuration, but missing from the live cluster.
+<8> The CRs that did not match to a corresponding template in the reference configuration.
+<9> The metadata hash identifies the reference configuration.
+<10> The list of patched CRs.

modules/using-cluster-compare-must-gather.adoc

@@ -0,0 +1,84 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/cluster-compare/using-the-cluster-compare-plugin.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+[id="using-cluster-compare-must-gather_{context}"]
+= Using the cluster-compare plugin with must-gather data
+
+You can use the `cluster-compare` plugin to compare a reference configuration with configuration custom resources (CRs) from `must-gather` data. 
+
+Validate cluster configurations by using `must-gather` data to troubleshoot configuration issues in production environments. 
+
+[NOTE]
+====
+For production environments, use the `cluster-compare` plugin with `must-gather` data only.
+====
+
+* You have access to `must-gather` data from a target cluster.
+
+* You installed the OpenShift CLI (`oc`).
+
+* You have downloaded the `cluster-compare` plugin and included it in your `PATH` environment variable.
+
+* You have access to a reference configuration. 
+
+.Procedure
+
+* Compare the `must-gather` data to a reference configuration by running the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -r <path_to_reference_config>/metadata.yaml -f "must-gather*/*/cluster-scoped-resources","must-gather*/*/namespaces" -R
+----
++
+** `-r` specifies a path to the `metadata.yaml` file of the reference configuration. You can specify a local directory or a URI.
+** `-f` specifies the path to the `must-gather` data directory. You can specify a local directory or a URI. This example restricts the comparison to the relevant cluster configuration directories. 
+** `-R` searches the target directories recursively.
++
+.Example output
+[source,terminal]
+----
+...
+
+**********************************
+
+Cluster CR: operator.openshift.io/v1_Console_cluster <1>
+Reference File: optional/console-disable/ConsoleOperatorDisable.yaml <2>
+Diff Output: diff -u -N /tmp/MERGED-622469311/operator-openshift-io-v1_console_cluster /tmp/LIVE-2358803347/operator-openshift-io-v1_console_cluster
+/tmp/MERGED-622469311/operator-openshift-io-v1_console_cluster	2024-11-20 15:43:42.888633602 +0000
++++ /tmp/LIVE-2358803347/operator-openshift-io-v1_console_cluster	2024-11-20 15:43:42.888633602 +0000
+@@ -4,5 +4,5 @@
+   name: cluster
+ spec:
+   logLevel: Normal
+-  managementState: Removed <3>
++  managementState: Managed
+   operatorLogLevel: Normal
+
+**********************************
+
+…
+
+Summary <4>
+CRs with diffs: 5/49 <5>
+CRs in reference missing from the cluster: 1 <6>
+required-cluster-tuning:
+  cluster-tuning:
+    Missing CRs: <7>
+    - required/cluster-tuning/disabling-network-diagnostics/DisableSnoNetworkDiag.yaml
+No CRs are unmatched to reference CRs <8>
+Metadata Hash: 512a9bf2e57fd5a5c44bbdea7abb3ffd7739d4a1f14ef9021f6793d5cdf868f0 <9>
+No patched CRs <10>
+----
+<1> The CR under comparison. The plugin displays each CR with a difference from the corresponding template.
+<2> The template matching with the CR for comparison.
+<3> The output in Linux diff format shows the difference between the template and the cluster CR.
+<4> After the plugin reports the line diffs for each CR, the summary of differences are reported.
+<5> The number of CRs in the comparison with differences from the corresponding templates.
+<6> The number of CRs represented in the reference configuration, but missing from the live cluster.
+<7> The list of CRs represented in the reference configuration, but missing from the live cluster.
+<8> The CRs that did not match to a corresponding template in the reference configuration.
+<9> The metadata hash identifies the reference configuration.
+<10> The list of patched CRs.

modules/using-cluster-compare-telco-ref.adoc

@@ -0,0 +1,296 @@
+// Module included in the following assemblies:
+
+// *scalability_and_performance/telco_ref_design_specs/core/telco-core-ref-crs.adoc
+// *scalability_and_performance/telco_ref_design_specs/ran/telco-ran-ref-du-crs.adoc
+// *scalability_and_performance/cluster-compare/using-the-cluster-compare-plugin.adoc
+
+:_mod-docs-content-type: PROCEDURE
+
+ifeval::["{context}" == "using-cluster-compare-plugin"]
+:cluster-compare:
+endif::[]
+
+ifeval::["{context}" == "ran-core-ref-design-crs"]
+:cluster-compare-core:
+endif::[]
+
+ifeval::["{context}" == "ran-ref-design-crs"]
+:cluster-compare-ran:
+endif::[]
+
+[id="using-cluster-compare-telco_ref_{context}"]
+ifdef::cluster-compare[= Example: Comparing a cluster with the telco core reference configuration]
+ifdef::cluster-compare-core,cluster-compare-ran[= Comparing a cluster with the {rds} reference configuration]
+
+//Intro for procedure in telco core/RAN RDS assembly
+ifdef::cluster-compare-core,cluster-compare-ran[]
+After you deploy a {rds} cluster, you can use the `cluster-compare` plugin to assess the cluster's compliance with the {rds} reference design specification (RDS). The `cluster-compare` plugin is an OpenShift CLI (`oc`) plugin. The plugin uses a {rds} reference configuration to validate the cluster with the {rds} custom resources (CRs). 
+
+The plugin-specific reference configuration for {rds} is packaged in a container image with the {rds} CRs. 
+
+For further information about the `cluster-compare` plugin, see "Understanding the cluster-compare plugin".
+endif::cluster-compare-core,cluster-compare-ran[]
+
+//Intro for procedure in cluster-compare assembly
+ifdef::cluster-compare[]
+You can use the `cluster-compare` plugin to compare a reference configuration with a configuration from a live cluster or `must-gather` data. 
+
+This example compares a configuration from a live cluster with the telco core reference configuration. The telco core reference configuration is derived from the telco core reference design specifications (RDS). The telco core RDS is designed for clusters to support large scale telco applications including control plane and some centralized data plane functions.
+
+The reference configuration is packaged in a container image with the telco core RDS. 
+
+For further examples of using the `cluster-compare` plugin with the telco core and telco RAN distributed unit (DU) profiles, see the "Additional resources" section.
+endif::cluster-compare[]
+
+.Prerequisites
+
+* You have access to the cluster as a user with the `cluster-admin` role.
+
+* You have credentials to access the `registry.redhat.io` container image registry.
+
+* You installed the `cluster-compare` plugin.
+
+.Procedure
+
+. Login to the container image registry with your credentials by running the following command:
++
+[source,terminal]
+----
+$ podman login registry.redhat.io
+----
+
+ifdef::cluster-compare-core,cluster-compare[]
+. Extract the content from the `telco-core-rds-rhel9` container image by running the following commands:
++
+[source,terminal]
+----
+$ mkdir -p ./out
+----
++
+[source,terminal]
+----
+$ podman run -it registry.redhat.io/openshift4/openshift-telco-core-rds-rhel9:v4.17 | base64 -d | tar xv -C out
+----
++
+You can view the reference configuration in the `reference-crs-kube-compare/` directory.
++
+[source,text]
+----
+out/telco-core-rds/configuration/reference-crs-kube-compare/
+├── metadata.yaml <1>
+├── optional <2>
+│   ├── logging
+│   ├── networking
+│   ├── other
+│   └── tuning
+└── required <3>
+    ├── networking
+    ├── other
+    ├── performance
+    ├── scheduling
+    └── storage
+----
+<1> Configuration file for the reference configuration.
+<2> Directory for optional templates.
+<3> Directory for required templates.
+
+. Compare the configuration for your cluster to the telco core reference configuration by running the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -r out/telco-core-rds/configuration/reference-crs-kube-compare/metadata.yaml
+----
++
+.Example output
+[source,terminal]
+----
+W1212 14:13:06.281590   36629 compare.go:425] Reference Contains Templates With Types (kind) Not Supported By Cluster: BFDProfile, BGPAdvertisement, BGPPeer, ClusterLogForwarder, Community, IPAddressPool, MetalLB, MultiNetworkPolicy, NMState, NUMAResourcesOperator, NUMAResourcesScheduler, NodeNetworkConfigurationPolicy, SriovNetwork, SriovNetworkNodePolicy, SriovOperatorConfig, StorageCluster
+
+...
+
+**********************************
+
+Cluster CR: config.openshift.io/v1_OperatorHub_cluster <1>
+Reference File: required/other/operator-hub.yaml <2>
+Diff Output: diff -u -N /tmp/MERGED-2801470219/config-openshift-io-v1_operatorhub_cluster /tmp/LIVE-2569768241/config-openshift-io-v1_operatorhub_cluster
+--- /tmp/MERGED-2801470219/config-openshift-io-v1_operatorhub_cluster	2024-12-12 14:13:22.898756462 +0000
++++ /tmp/LIVE-2569768241/config-openshift-io-v1_operatorhub_cluster	2024-12-12 14:13:22.898756462 +0000
+@@ -1,6 +1,6 @@
+ apiVersion: config.openshift.io/v1
+ kind: OperatorHub
+ metadata:
++  annotations: <3>
++    include.release.openshift.io/hypershift: "true"
+   name: cluster
+-spec:
+-  disableAllDefaultSources: true
+
+**********************************
+
+Summary <4>
+CRs with diffs: 3/4 <5>
+CRs in reference missing from the cluster: 22 <6>
+other:
+  other:
+    Missing CRs: <7>
+    - optional/other/control-plane-load-kernel-modules.yaml
+    - optional/other/worker-load-kernel-modules.yaml
+required-networking:
+  networking-root:
+    Missing CRs:
+    - required/networking/nodeNetworkConfigurationPolicy.yaml
+  networking-sriov:
+    Missing CRs:
+    - required/networking/sriov/sriovNetwork.yaml
+    - required/networking/sriov/sriovNetworkNodePolicy.yaml
+    - required/networking/sriov/SriovOperatorConfig.yaml
+    - required/networking/sriov/SriovSubscription.yaml
+    - required/networking/sriov/SriovSubscriptionNS.yaml
+    - required/networking/sriov/SriovSubscriptionOperGroup.yaml
+required-other:
+  scheduling:
+    Missing CRs:
+    - required/other/catalog-source.yaml
+    - required/other/icsp.yaml
+required-performance:
+  performance:
+    Missing CRs:
+    - required/performance/PerformanceProfile.yaml
+required-scheduling:
+  scheduling:
+    Missing CRs:
+    - required/scheduling/nrop.yaml
+    - required/scheduling/NROPSubscription.yaml
+    - required/scheduling/NROPSubscriptionNS.yaml
+    - required/scheduling/NROPSubscriptionOperGroup.yaml
+    - required/scheduling/sched.yaml
+required-storage:
+  storage-odf:
+    Missing CRs:
+    - required/storage/odf-external/01-rook-ceph-external-cluster-details.secret.yaml
+    - required/storage/odf-external/02-ocs-external-storagecluster.yaml
+    - required/storage/odf-external/odfNS.yaml
+    - required/storage/odf-external/odfOperGroup.yaml
+    - required/storage/odf-external/odfSubscription.yaml
+No CRs are unmatched to reference CRs <8>
+Metadata Hash: fe41066bac56517be02053d436c815661c9fa35eec5922af25a1be359818f297 <9>
+No patched CRs <10>
+----
+<1> The CR under comparison. The plugin displays each CR with a difference from the corresponding template.
+<2> The template matching with the CR for comparison.
+<3> The output in Linux diff format shows the difference between the template and the cluster CR.
+<4> After the plugin reports the line diffs for each CR, the summary of differences are reported.
+<5> The number of CRs in the comparison with differences from the corresponding templates.
+<6> The number of CRs represented in the reference configuration, but missing from the live cluster.
+<7> The list of CRs represented in the reference configuration, but missing from the live cluster.
+<8> The CRs that did not match to a corresponding template in the reference configuration.
+<9> The metadata hash identifies the reference configuration.
+<10> The list of patched CRs.
+endif::cluster-compare-core,cluster-compare[]
+ifdef::cluster-compare-ran[]
+. Extract the content from the `ztp-site-generate-rhel8` container image by running the following commands::
++
+[source,terminal,subs="attributes+"]
+----
+$ podman pull registry.redhat.io/openshift4/ztp-site-generate-rhel8:v{product-version}
+----
++
+[source,terminal]
+----
+$ mkdir -p ./out
+----
++
+[source,terminal,subs="attributes+"]
+----
+$ podman run --log-driver=none --rm registry.redhat.io/openshift4/ztp-site-generate-rhel8:v{product-version} extract /home/ztp --tar | tar x -C ./out
+----
+
+. Compare the configuration for your cluster to the reference configuration by running the following command:
++
+[source,terminal]
+----
+$ oc cluster-compare -r out/reference/metadata.yaml
+----
++
+.Example output
+[source,terminal]
+----
+...
+
+**********************************
+
+Cluster CR: config.openshift.io/v1_OperatorHub_cluster <1>
+Reference File: required/other/operator-hub.yaml <2>
+Diff Output: diff -u -N /tmp/MERGED-2801470219/config-openshift-io-v1_operatorhub_cluster /tmp/LIVE-2569768241/config-openshift-io-v1_operatorhub_cluster
+--- /tmp/MERGED-2801470219/config-openshift-io-v1_operatorhub_cluster	2024-12-12 14:13:22.898756462 +0000
++++ /tmp/LIVE-2569768241/config-openshift-io-v1_operatorhub_cluster	2024-12-12 14:13:22.898756462 +0000
+@@ -1,6 +1,6 @@
+ apiVersion: config.openshift.io/v1
+ kind: OperatorHub
+ metadata:
++  annotations: <3>
++    include.release.openshift.io/hypershift: "true"
+   name: cluster
+-spec:
+-  disableAllDefaultSources: true
+
+**********************************
+
+Summary <4>
+CRs with diffs: 11/12 <5>
+CRs in reference missing from the cluster: 40 <6>
+optional-image-registry:
+  image-registry:
+    Missing CRs: <7>
+    - optional/image-registry/ImageRegistryPV.yaml
+optional-ptp-config:
+  ptp-config:
+    One of the following is required:
+    - optional/ptp-config/PtpConfigBoundary.yaml
+    - optional/ptp-config/PtpConfigGmWpc.yaml
+    - optional/ptp-config/PtpConfigDualCardGmWpc.yaml
+    - optional/ptp-config/PtpConfigForHA.yaml
+    - optional/ptp-config/PtpConfigMaster.yaml
+    - optional/ptp-config/PtpConfigSlave.yaml
+    - optional/ptp-config/PtpConfigSlaveForEvent.yaml
+    - optional/ptp-config/PtpConfigForHAForEvent.yaml
+    - optional/ptp-config/PtpConfigMasterForEvent.yaml
+    - optional/ptp-config/PtpConfigBoundaryForEvent.yaml
+  ptp-operator-config:
+    One of the following is required:
+    - optional/ptp-config/PtpOperatorConfig.yaml
+    - optional/ptp-config/PtpOperatorConfigForEvent.yaml
+optional-storage:
+  storage:
+    Missing CRs:
+    - optional/local-storage-operator/StorageLV.yaml
+
+...
+
+No CRs are unmatched to reference CRs <8>
+Metadata Hash: 09650c31212be9a44b99315ec14d2e7715ee194a5d68fb6d24f65fd5ddbe3c3c <9>
+No patched CRs <10>
+----
+<1> The CR under comparison. The plugin displays each CR with a difference from the corresponding template.
+<2> The template matching with the CR for comparison.
+<3> The output in Linux diff format shows the difference between the template and the cluster CR.
+<4> After the plugin reports the line diffs for each CR, the summary of differences are reported.
+<5> The number of CRs in the comparison with differences from the corresponding templates.
+<6> The number of CRs represented in the reference configuration, but missing from the live cluster.
+<7> The list of CRs represented in the reference configuration, but missing from the live cluster.
+<8> The CRs that did not match to a corresponding template in the reference configuration.
+<9> The metadata hash identifies the reference configuration.
+<10> The list of patched CRs.
+endif::cluster-compare-ran[]
+
+ifeval::["{context}" == "cluster-compare"]
+:!cluster-compare:
+endif::[]
+
+ifeval::["{context}" == "cluster-compare-core"]
+:!cluster-compare-core:
+endif::[]
+
+ifeval::["{context}" == "cluster-compare-ran"]
+:!cluster-compare-ran:
+endif::[]

scalability_and_performance/cluster-compare/_attributes

@@ -0,0 +1 @@
+../../_attributes/

scalability_and_performance/cluster-compare/advanced-ref-config-customization.adoc

@@ -0,0 +1,24 @@
+:_mod-docs-content-type: ASSEMBLY
+[id="advanced-ref-config-customization"]
+= Performing advanced reference configuration customization
+include::_attributes/common-attributes.adoc[]
+:context: advanced-ref-config-customization
+
+toc::[]
+
+For scenarios where you want to allow temporary deviations from the reference design, you can apply more advanced customizations. 
+
+[WARNING]
+====
+These customizations override the default matching process that the `cluster-compare` plugin uses during a comparison. Use caution when applying these advanced customizations as it can lead to unintended consequences, such as excluding consequential information from a cluster comparison.
+====
+
+Some advanced tasks to dynamically customize your reference configuration include the following:
+
+* *Manual matching*: Configure a user configuration file to manually match a custom resource from the cluster to a template in the reference configuration.
+
+* *Patching the reference*: Patch a reference to configure a reference configuration by using a patch option with the `cluster-compare` command.
+
+include::modules/cluster-compare-manual-match.adoc[leveloffset=+1]
+
+include::modules/cluster-compare-patching.adoc[leveloffset=+1]

scalability_and_performance/cluster-compare/creating-a-reference-configuration.adoc

@@ -0,0 +1,21 @@
+:_mod-docs-content-type: ASSEMBLY
+[id="creating-a-reference-configuration"]
+= Creating a reference configuration
+include::_attributes/common-attributes.adoc[]
+:context: creating-a-reference-configuration
+
+toc::[]
+
+Configure a reference configuration to validate configuration resources from a cluster.
+
+include::modules/cluster-compare-metadata-structure.adoc[leveloffset=+1]
+
+include::modules/cluster-compare-template-groupings.adoc[leveloffset=+1]
+
+include::modules/cluster-compare-templating.adoc[leveloffset=+1]
+
+include::modules/cluster-compare-exclude-metadata.adoc[leveloffset=+1]
+
+include::modules/cluster-compare-configure-inline-diff.adoc[leveloffset=+1]
+
+include::modules/cluster-compare-output-description.adoc[leveloffset=+1]

scalability_and_performance/cluster-compare/images

@@ -0,0 +1 @@
+../../images/

scalability_and_performance/cluster-compare/installing-cluster-compare-plugin.adoc

@@ -0,0 +1,16 @@
+:_mod-docs-content-type: ASSEMBLY
+[id="installing-cluster-compare-plugin"]
+= Installing the cluster-compare plugin
+include::_attributes/common-attributes.adoc[]
+:context: installing-cluster-compare-plugin
+
+toc::[]
+
+You can extract the `cluster-compare` plugin from a container image in the Red Hat container catalog and use it as a plugin to the `oc` command.
+
+include::modules/installing-cluster-compare.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+== Additional resources
+
+* xref:../../cli_reference/openshift_cli/extending-cli-plugins.adoc#cli-extend-plugins[Extending the OpenShift CLI with plugins]

scalability_and_performance/cluster-compare/modules

@@ -0,0 +1 @@
+../../modules/

scalability_and_performance/cluster-compare/snippets

@@ -0,0 +1 @@
+../../snippets/

scalability_and_performance/cluster-compare/troubleshooting-cluster-comparisons.adoc

@@ -0,0 +1,42 @@
+:_mod-docs-content-type: ASSEMBLY
+[id="troubleshooting-cluster-comparisons"]
+= Troubleshooting cluster comparisons
+include::_attributes/common-attributes.adoc[]
+:context: troubleshooting-cluster-comparisons
+
+toc::[]
+
+When using the `cluster-compare` plugin, you might see unexpected results, such as false positives or conflicts when multiple cluster custom resources (CRs) exist. 
+
+[id="troubleshooting-cc-false-positives_{context}"]
+== Troubleshooting false positives for missing resources
+
+The plugin might report a missing resource even though the cluster custom resource (CR) is present in the cluster.
+
+.Procedure
+
+. Ensure you are using the latest version of the `cluster-compare` plugin. For more information, see "Installing the cluster-compare plugin".
+
+. Ensure you are using the most up-to-date version of the reference configuration.
+
+. Ensure that template has the same `apiVersion`, `kind`, `name`, and `namespace` fields as the cluster CR.
+
+[id="troubleshooting-cc-multiple-matches_{context}"]
+== Troubleshooting multiple template matches for the same CR
+
+In some cases, more than one cluster CR can match a template because they feature the same `apiVersion`, `namespace`, and `kind`. The plugin's default matching compares the CR that features the least differences.
+
+You can optionally configure your reference configuration to avoid this situation.
+
+.Procedure
+
+. Ensure the templates feature distinct `apiVersion`, `namespace`, and `kind` values to ensure no duplicate template matching.
+
+. Use a user configuration file to manually match a template to a CR. For more information, see "Configuring manual matching between CRs and templates".
+
+[role="_additional-resources"]
+== Additional resources
+
+* xref:../../scalability_and_performance/cluster-compare/installing-cluster-compare-plugin.adoc#installing-cluster-compare_installing-cluster-compare-plugin[Installing the cluster-compare plugin]
+
+* xref:../../scalability_and_performance/cluster-compare/advanced-ref-config-customization.adoc#cluster-compare-manual-match_advanced-ref-config-customization[Configuring manual matching between CRs and templates]

scalability_and_performance/cluster-compare/understanding-the-cluster-compare-plugin.adoc

@@ -0,0 +1,23 @@
+:_mod-docs-content-type: ASSEMBLY
+[id="understanding-the-cluster-compare-plugin"]
+= Understanding the cluster-compare plugin
+include::_attributes/common-attributes.adoc[]
+:context: understanding-cluster-compare
+
+toc::[]
+
+The `cluster-compare` plugin is an OpenShift CLI (`oc`) plugin that compares a cluster configuration with a reference configuration. The plugin reports configuration differences while suppressing expected variations by using configurable validation rules and templates.
+
+Use the `cluster-compare` plugin in development, production, and support scenarios to ensure cluster compliance with a reference configuration, and to quickly identify and troubleshoot relevant configuration differences.
+
+include::modules/cluster-compare-overview.adoc[leveloffset=+1]
+
+include::modules/understanding-a-reference-config.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+== Additional resources
+
+* xref:../../scalability_and_performance/telco_ref_design_specs/telco-ref-design-specs-overview.adoc#telco-ref-design-overview_telco_ref_design_specs[Reference design specifications for telco 5G deployments]
+
+
+

scalability_and_performance/cluster-compare/using-the-cluster-compare-plugin.adoc

@@ -0,0 +1,28 @@
+:_mod-docs-content-type: ASSEMBLY
+[id="using-the-cluster-compare-plugin"]
+= Using the cluster-compare plugin
+include::_attributes/common-attributes.adoc[]
+:context: using-cluster-compare-plugin
+
+toc::[]
+
+You can use the `cluster-compare` plugin to compare a reference configuration with a configuration from a live cluster or `must-gather` data.
+
+include::modules/using-cluster-compare-live-cluster.adoc[leveloffset=+1]
+
+include::modules/using-cluster-compare-must-gather.adoc[leveloffset=+1]
+
+[discrete]
+[role="_additional-resources"]
+.Additional resources
+* xref:../../support/gathering-cluster-data.adoc#about-must-gather_gathering-cluster-data[Gathering data about your cluster]
+
+include::modules/cluster-compare-reference-args.adoc[leveloffset=+1]
+
+include::modules/using-cluster-compare-telco-ref.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+[id="additional-resources_{context}"]
+== Additional resources
+* xref:../../scalability_and_performance/telco_ref_design_specs/ran/telco-ran-ref-du-crs.adoc#using-cluster-compare-telco_ref_ran-ref-design-crs[Comparing a cluster with the telco RAN DU reference configuration]
+* xref:../../scalability_and_performance/telco_ref_design_specs/core/telco-core-ref-crs.adoc#using-cluster-compare-telco_ref_ran-core-ref-design-crs[Comparing a cluster with the telco core reference configuration]

scalability_and_performance/telco_ref_design_specs/core/telco-core-ref-crs.adoc

@@ -12,6 +12,12 @@ Use the CRs to form the common baseline used in all the specific use models unle
 
 include::modules/telco-core-rds-container.adoc[leveloffset=+1]
 
+include::modules/using-cluster-compare-telco-ref.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+.Additional resources
+* xref:../../../scalability_and_performance/cluster-compare/understanding-the-cluster-compare-plugin.adoc#understanding-the-cluster-compare-plugin[Understanding the cluster-compare plugin]
+
 include::modules/telco-core-crs-networking.adoc[leveloffset=+1]
 
 include::modules/telco-core-crs-node-configuration.adoc[leveloffset=+1]

scalability_and_performance/telco_ref_design_specs/ran/telco-ran-ref-du-crs.adoc

@@ -17,6 +17,12 @@ You can extract the complete set of RAN DU CRs from the `ztp-site-generate` cont
 See xref:../../../edge_computing/ztp-preparing-the-hub-cluster.adoc#ztp-preparing-the-ztp-git-repository_ztp-preparing-the-hub-cluster[Preparing the {ztp} site configuration repository] for more information.
 ====
 
+include::modules/using-cluster-compare-telco-ref.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+.Additional resources
+* xref:../../../scalability_and_performance/cluster-compare/understanding-the-cluster-compare-plugin.adoc#understanding-the-cluster-compare-plugin[Understanding the cluster-compare plugin]
+
 include::modules/telco-ran-crs-day-2-operators.adoc[leveloffset=+1]
 
 include::modules/telco-ran-crs-cluster-tuning.adoc[leveloffset=+1]