openshift-docs/machine_management/creating-infrastructure-machinesets.adoc

:_mod-docs-content-type: ASSEMBLY
[id="creating-infrastructure-machinesets"]
= Creating infrastructure machine sets
include::_attributes/common-attributes.adoc[]
:context: creating-infrastructure-machinesets

toc::[]

include::snippets/machine-user-provisioned-limitations.adoc[leveloffset=+1]

You can use infrastructure machine sets to create machines that host only infrastructure components, such as the default router, the integrated container image registry, and the components for cluster metrics and monitoring. These infrastructure machines are not counted toward the total number of subscriptions that are required to run the environment.

include::modules/infrastructure-components.adoc[leveloffset=+1]

For information about infrastructure nodes and which components can run on infrastructure nodes, see the "Red Hat OpenShift control plane and infrastructure nodes" section in the link:https://www.redhat.com/en/resources/openshift-subscription-sizing-guide[OpenShift sizing and subscription guide for enterprise Kubernetes] document.

To create an infrastructure node, you can xref:../machine_management/creating-infrastructure-machinesets.adoc#machineset-creating_creating-infrastructure-machinesets[use a machine set], xref:../machine_management/creating-infrastructure-machinesets.adoc#creating-an-infra-node_creating-infrastructure-machinesets[label the node], or xref:../machine_management/creating-infrastructure-machinesets.adoc#creating-infra-machines_creating-infrastructure-machinesets[use a machine config pool].

[id="creating-infrastructure-machinesets-clouds"]
=== Creating infrastructure machine sets for different clouds

Use the sample compute machine set for your cloud.

include::modules/machineset-yaml-aws.adoc[leveloffset=+3]

Machine sets running on AWS support non-guaranteed xref:../machine_management/creating_machinesets/creating-machineset-aws.adoc#machineset-non-guaranteed-instance_creating-machineset-aws[Spot Instances]. You can save on costs by using Spot Instances at a lower price compared to
On-Demand Instances on AWS. xref:../machine_management/creating_machinesets/creating-machineset-aws.adoc#machineset-creating-non-guaranteed-instance_creating-machineset-aws[Configure Spot Instances] by adding `spotMarketOptions` to the `MachineSet` YAML file.

include::modules/machineset-yaml-azure.adoc[leveloffset=+3]

Machine sets running on Azure support non-guaranteed xref:../machine_management/creating_machinesets/creating-machineset-azure.adoc#machineset-non-guaranteed-instance_creating-machineset-azure[Spot VMs]. You can save on costs by using Spot VMs at a lower price compared to standard VMs on Azure. You can xref:../machine_management/creating_machinesets/creating-machineset-azure.adoc#machineset-creating-non-guaranteed-instance_creating-machineset-azure[configure Spot VMs] by adding `spotVMOptions` to the `MachineSet` YAML file.

[role="_additional-resources"]
.Additional resources
* xref:../machine_management/creating_machinesets/creating-machineset-azure.adoc#installation-azure-marketplace-subscribe_creating-machineset-azure[Using the Azure Marketplace offering]

include::modules/machineset-yaml-azure-stack-hub.adoc[leveloffset=+3]

[NOTE]
====
Machine sets running on Azure Stack Hub do not support non-guaranteed Spot VMs.
====

include::modules/machineset-yaml-ibm-cloud.adoc[leveloffset=+3]

include::modules/machineset-yaml-gcp.adoc[leveloffset=+3]

Machine sets running on {gcp-short} support non-guaranteed xref:../machine_management/creating_machinesets/creating-machineset-gcp.adoc#machineset-non-guaranteed-instance_creating-machineset-gcp[preemptible VM instances]. You can save on costs by using preemptible VM instances at a lower price
compared to normal instances on {gcp-short}. You can xref:../machine_management/creating_machinesets/creating-machineset-gcp.adoc#machineset-creating-non-guaranteed-instance_creating-machineset-gcp[configure preemptible VM instances] by adding `preemptible` to the `MachineSet` YAML file.

include::modules/machineset-yaml-nutanix.adoc[leveloffset=+3]

include::modules/machineset-yaml-osp.adoc[leveloffset=+3]

include::modules/machineset-yaml-vsphere.adoc[leveloffset=+3]

include::modules/machineset-creating.adoc[leveloffset=+2]

include::modules/creating-an-infra-node.adoc[leveloffset=+2]

[role="_additional-resources"]
.Additional resources

* xref:moving-resources-to-infrastructure-machinesets[Moving resources to infrastructure machine sets]

include::modules/creating-infra-machines.adoc[leveloffset=+2]

[role="_additional-resources"]
.Additional resources

* xref:../architecture/control-plane.adoc#architecture-machine-config-pools_control-plane[Node configuration management with machine config pools].

[id="assigning-machineset-resources-to-infra-nodes"]
== Assigning machine set resources to infrastructure nodes

After creating an infrastructure machine set, the `worker` and `infra` roles are applied to new infra nodes. Nodes with the `infra` role applied are not counted toward the total number of subscriptions that are required to run the environment, even when the `worker` role is also applied.

However, with an infra node being assigned as a worker, there is a chance user workloads could get inadvertently assigned to an infra node. To avoid this, you can apply a taint to the infra node and tolerations for the pods you want to control.

include::modules/binding-infra-node-workloads-using-taints-tolerations.adoc[leveloffset=+2]

[role="_additional-resources"]
.Additional resources

* xref:../nodes/scheduling/nodes-scheduler-about.adoc#nodes-scheduler-about[Controlling pod placement using the scheduler].
* xref:moving-resources-to-infrastructure-machinesets[Moving resources to infrastructure machine sets].
* xref:../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations-about_nodes-scheduler-taints-tolerations[Understanding taints and tolerations].

[id="moving-resources-to-infrastructure-machinesets"]
== Moving resources to infrastructure machine sets

Some of the infrastructure resources are deployed in your cluster by default. You can move them to the infrastructure machine sets that you created by adding the infrastructure node selector, as shown:

[source,yaml]
----
apiVersion: imageregistry.operator.openshift.io/v1
kind: Config
metadata:
  name: cluster
# ...
spec:
  nodePlacement: <1>
    nodeSelector:
      matchLabels:
        node-role.kubernetes.io/infra: ""
    tolerations:
    - effect: NoSchedule
      key: node-role.kubernetes.io/infra
      value: reserved
    - effect: NoExecute
      key: node-role.kubernetes.io/infra
      value: reserved
----
<1> Add a `nodeSelector` parameter with the appropriate value to the component you want to move. You can use a `nodeSelector` in the format shown or use `<key>: <value>` pairs, based on the value specified for the node.  If you added a taint to the infrasructure node, also add a matching toleration.

Applying a specific node selector to all infrastructure components causes {product-title} to xref:../machine_management/creating-infrastructure-machinesets.adoc#moving-resources-to-infrastructure-machinesets[schedule those workloads on nodes with that label].

include::modules/infrastructure-moving-router.adoc[leveloffset=+2]

include::modules/infrastructure-moving-registry.adoc[leveloffset=+2]

include::modules/infrastructure-moving-monitoring.adoc[leveloffset=+2]

include::modules/nodes-pods-vertical-autoscaler-moving-vpa.adoc[leveloffset=+2]

include::modules/nodes-cluster-resource-override-move-infra.adoc[leveloffset=+2]

[role="_additional-resources"]
.Additional resources
* link:https://docs.redhat.com/en/documentation/monitoring_stack_for_red_hat_openshift/4.21/html/configuring_core_platform_monitoring/configuring-performance-and-scalability#moving-monitoring-components-to-different-nodes_configuring-performance-and-scalability[Moving monitoring components to different nodes]