openshift-docs/modules/machine-config-overview.adoc

// Module included in the following assemblies:
//
// * operators/operator-reference.adoc
// * post_installation_configuration/machine-configuration-tasks.adoc

:_content-type: CONCEPT
[id="machine-config-overview-{context}"]
= Machine config overview

The Machine Config Operator (MCO) manages updates to systemd, CRI-O and Kubelet, the kernel, Network Manager and other system features. It also offers a `MachineConfig` CRD that can write configuration files onto the host (see link:https://github.com/openshift/machine-config-operator#machine-config-operator[machine-config-operator]). Understanding what MCO does and how it interacts with other components is critical to making advanced, system-level changes to an {product-title} cluster. Here are some things you should know about MCO, machine configs, and how they are used:

* A machine config can make a specific change to a file or service on the operating system of each system representing a pool of {product-title} nodes.

* MCO applies changes to operating systems in pools of machines. All {product-title} clusters start with worker and control plane node pools. By adding more role labels, you can configure custom pools of nodes. For example, you can set up a custom pool of worker nodes that includes particular hardware features needed by an application. However, examples in this section focus on changes to the default pool types.
+
[IMPORTANT]
====
A node can have multiple labels applied that indicate its type, such as `master` or `worker`, however it can be a member of only a *single* machine config pool.
====

* After a machine config change, the MCO updates the affected nodes alphabetically by zone, based on the `topology.kubernetes.io/zone` label. If a zone has more than one node, the oldest nodes are updated first. For nodes that do not use zones, such as in bare metal deployments, the nodes are upgraded by age, with the oldest nodes updated first. The MCO updates the number of nodes as specified by the `maxUnavailable` field on the machine configuration pool at a time.

* Some machine configuration must be in place before {product-title} is installed to disk. In most cases, this can be accomplished by creating
a machine config that is injected directly into the {product-title} installer process, instead of running as a post-installation machine config. In other cases, you might need to do bare metal installation where you pass kernel arguments at {product-title} installer startup, to do such things as setting per-node individual IP addresses or advanced disk partitioning.

* MCO manages items that are set in machine configs. Manual changes you do to your systems will not be overwritten by MCO, unless MCO is explicitly told to manage a conflicting file. In other words, MCO only makes specific updates you request, it does not claim control over the whole node.

* Manual changes to nodes are strongly discouraged. If you need to decommission a node and start a new one, those direct changes would be lost.

* MCO is only supported for writing to files in `/etc` and `/var` directories, although there are symbolic links to some directories that can be writeable by being symbolically linked to one of those areas. The `/opt` and `/usr/local` directories are examples.

* Ignition is the configuration format used in MachineConfigs. See the link:https://coreos.github.io/ignition/configuration-v3_2/[Ignition Configuration Specification v3.2.0] for details.

* Although Ignition config settings can be delivered directly at {product-title} installation time, and are formatted in the same way that MCO delivers Ignition configs, MCO has no way of seeing what those original Ignition configs are. Therefore, you should wrap Ignition config settings into a machine config before deploying them.

* When a file managed by MCO changes outside of MCO, the Machine Config Daemon (MCD) sets the node as `degraded`. It will not overwrite the
offending file, however, and should continue to operate in a `degraded` state.

* A key reason for using a machine config is that it will be applied when you spin up new nodes for a pool in your {product-title} cluster. The `machine-api-operator` provisions a new machine and MCO configures it.

MCO uses link:https://coreos.github.io/ignition/[Ignition] as the configuration format. {product-title} 4.6 moved from Ignition config specification version 2 to version 3.

== What can you change with machine configs?
The kinds of components that MCO can change include:

* **config**: Create Ignition config objects (see the link:https://coreos.github.io/ignition/configuration-v3_2/[Ignition configuration specification]) to do things like modify files, systemd services, and other features on {product-title} machines, including:
- **Configuration files**: Create or overwrite files in the `/var` or `/etc` directory.
- **systemd units**: Create and set the status of a systemd service or add to an existing systemd service by dropping in additional settings.
- **users and groups**: Change SSH keys in the passwd section post-installation.

[IMPORTANT]
====
Changing SSH keys via machine configs is only supported for the `core` user.
====
* **kernelArguments**: Add arguments to the kernel command line when {product-title} nodes boot.
* **kernelType**: Optionally identify a non-standard kernel to use instead of the standard kernel. Use `realtime` to use the RT kernel (for RAN). This is only supported on select platforms.
ifndef::openshift-origin[]
* **fips**: Enable link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/security_hardening/index#using-the-system-wide-cryptographic-policies_security-hardening[FIPS] mode. FIPS should be set at installation-time setting and not a post-installation procedure.

[IMPORTANT]
====
The use of FIPS Validated / Modules in Process cryptographic libraries is only supported on {product-title} deployments on `x86_64`, `ppc64le`, and `s390x` architectures.
====
endif::openshift-origin[]
* **extensions**: Extend {op-system} features by adding selected pre-packaged software. For this feature, available extensions include link:https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/security_hardening/index#protecting-systems-against-intrusive-usb-devices_security-hardening[usbguard] and kernel modules.
* **Custom resources (for `ContainerRuntime` and `Kubelet`)**: Outside of machine configs, MCO manages two special custom resources for modifying CRI-O container runtime settings (`ContainerRuntime` CR) and the Kubelet service (`Kubelet` CR).

The MCO is not the only Operator that can change operating system components on {product-title} nodes. Other Operators can modify operating system-level features as well. One example is the Node Tuning Operator, which allows you to do node-level tuning through Tuned daemon profiles.

Tasks for the MCO configuration that can be done post-installation are included in the following procedures. See descriptions of {op-system} bare metal installation for system configuration tasks that must be done during or before {product-title} installation.

There might be situations where the configuration on a node does not fully match what the currently-applied machine config specifies. This state is called _configuration drift_. The Machine Config Daemon (MCD) regularly checks the nodes for configuration drift. If the MCD detects configuration drift, the MCO marks the node `degraded` until an administrator corrects the node configuration. A degraded node is online and operational, but, it cannot be updated. For more information on configuration drift, see _Understanding configuration drift detection_.

== Project

See the link:https://github.com/openshift/machine-config-operator[openshift-machine-config-operator] GitHub site for details.