openshift/openshift-docs
1
0
Fork 0
mirror of https://github.com/openshift/openshift-docs.git synced 2026-02-05 12:46:18 +01:00
Code Issues Releases Activity

osdocs-434 install/arch updates

Browse Source
This commit is contained in:
Kathryn Alexander
2019-05-23 15:11:50 -04:00
parent 6528a794da
commit 917fd30c4d
12 changed files with 468 additions and 445 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
_topic_map.yml
View File

@@ -53,10 +53,11 @@ Topics:
File: architecture-installation
- Name: The control plane
File: control-plane
Distros: openshift-enterprise,openshift-origin
Distros: openshift-enterprise,openshift-origin,openshift-dedicated
- Name: Understanding OpenShift development
File: understanding-development
- Name: Red Hat Enterprise Linux CoreOS
File: architecture-rhcos
---
Name: Administering a cluster
Dir: administering_a_cluster

7
architecture/architecture-rhcos.adoc Normal file
View File

@@ -0,0 +1,7 @@
[id="architecture-rhcos"]
= {op-system-first}
include::modules/common-attributes.adoc[]
:context: architecture-rhcos
toc::[]
include::modules/rhcos-about.adoc[leveloffset=+1]

4
installing/installing_aws/installing-aws-customizations.adoc
View File

@@ -6,7 +6,9 @@ include::modules/common-attributes.adoc[]
toc::[]
In {product-title} version {product-version}, you can install a customized
cluster on Amazon Web Services (AWS).
cluster on infrastructure that the installer provisions on
Amazon Web Services (AWS). To customize the installation, you modify
some parameters in the `install-config.yaml` file before you install the cluster.
.Prerequisites

417
modules/following-installation.adoc
View File

@@ -1,417 +0,0 @@
// Module included in the following assemblies:
//
// * TBD
[id="following-installation_{context}"]
= Following an {product-title} Install
As someone installing an {product-title} cluster, you need to choose and set up the installation platform, choose the type of installation, get the openshift-install tool on your local computer, and run the installer. The details of what happens in this process are described in the following sections.
[id="choosing-platform_{context}"]
== Choosing an {product-title} platform and installation type
Currently, the easiest {product-title} installation platform is an Amazon Web Services (AWS) installation, using an installation type where the infrastructure is provisioned by the installer. The other supported installation type is a more manual installation, where the infrastructure is set up manually by the user. The user-provisioned installation is supported on bare metal, VMware vSphere, and AWS platforms.
[id="choosing-installer-installation_{context}"]
== Choosing an installer-provisioned installation
For {product-title} {product-version} installation types where the platform supports provisioning from the installer, the openshift-install process creates all parts of the nodes, operating systems, networking, and other components of the {product-title} cluster by requesting those resources from the platform itself. For {product-title} {product-version}, this type of installation is limited to the following: +
* {op-system-first}({op-system}) is the first supported operating system for {product-title} {product-version}
* AWS is the first supported {product-title} {product-version} platform. Other platforms planned to be supported include Google Cloud Platform (GCP), Microsoft Azure, Red Hat OpenStack, and others.
Instructions for configuring AWS for {product-title} {product-version} are available from the https://docs.openshift.com/container-platform/4.1/installing/installing_aws/installing-aws-account.html[{product-title} Installing on AWS] documentation.
[id="choosing-user-installation_{context}"]
== Choosing a user-provisioned installation
{product-title} {product-version} installations where the users provision their own infrastructure require more setup before you can install an {product-title} cluster. Requirements for installations where the user sets up the infrastructure needed by the installer will include the following:
* Bare metal, AWS, OpenStack and VMware platforms (see {product-title} installation documentation for details)
* Expect these types of installations to be available for any platform where you can take responsibility for provisioning your own servers, setting up networking and configuring DNS.
* {op-system} will be required for the control plane (master) systems, but you will be able to use Red Hat Enterprise Linux systems you install yourself as worker (compute) nodes. Those systems are sometimes referred to as bring-your-own-host systems.
The text that follows demonstrates an installation on an AWS platform where the platform provides the infrastructure needed by {product-title}. Expect a section on user-provisioned infrastructure to be added to this document in the future.
[id="understanding-openshift-install_{context}"]
== Understanding openshift-install
The {product-title} installer is unlike typical Linux installers. In fact, it’s more like a build system, in the spirit of tools like make or the systemd Init system. Understanding these major characteristics of openshift-install should help you see how it differs from other installation and configuration tools:
* Targets and dependencies: Unlike a traditional installer that just steps through a set of actions, openshift-install starts out to achieve a set of targets. Each of those targets has dependencies. So, for example, before you can achieve the target of setting up a master node, dependencies require the installer to have a {op-system} image and an Install Config created. The Install Config, in turn, has dependencies of obtaining information about the platform, domain, an ssh key and a pull secret needed for the installation. +
+
Each dependency is referred to as an asset. The ultimate target is a running cluster. Having dependencies instead of commands lets the installer see if the asset is already in place and just consume it if it is, instead of running through an unneeded action.
* Fast builds: Because each target is only concerned with its own dependencies, the installer can be working on many targets at the same time. While a missing dependency, like a slow response to acquiring a certificate, might slow one target from completing, other targets that don’t have that dependency in their path can continue in parallel to completion. As a result, the entire installation process runs much faster.
* Limited configuration: The most basic supported configuration changes you can make to your {product-title} cluster at install time comes in the form of the six questions you are prompted for when you run openshift-install. Another supported option is to produce an install-config.yaml file, which lets you configure a few additional items, such as number of nodes, IP address ranges, and additional ssh keys. {product-title} {product-version} does include other, unsupported escape hatches into modifying your installation configuration, such as creating your own manifests. In the spirit of understanding the installer, manifests are described here, along with warnings about why, in general, you should not make changes in this way.
* Only the installer and MachineConfigOperator should change nodes: This is critical. The installer uses Ignition config files to set the exact state each node should be in when it is installed. There are a few supported options you can change at install time. Over time, changes to the node configs, such as new certificates or updated keys, are done through the Machine Config Operator. Logging into a node to make manual changes is unsupported and strongly discouraged. More on that later.
[id="starting-openshift-install_{context}"]
== Starting openshift-install
Running the openshift-install command kicks off a process to install the initial set of nodes in the {product-title} cluster, as well as the services running on those nodes. The nodes are deployed as follows:
* The installer gathers the configuration information needed to create the cluster
* Terraform takes over to begin creating the infrastructure needed to support the cluster
* The installer grabs a {op-system-first}({op-system}) operating system image (which is made available by Red Hat, but pulled in automatically by the installer)
* A bootstrap node starts up from an Ignition config file and sets up the master nodes (control plane) using the {op-system} image and an Ignition config file tailored to setting up masters
* After configuring and starting the nodes in the control plane, and after handing off the Ignition configs to the Machine Config Operator in the control plane, the bootstrap node is destroyed
* Using the Ignition config it got from the bootstrap node, the control plane sets up the worker nodes
* Additional services are then added to the control plane in the form of a set of Operators.
Keep in mind that the objective of an {product-title} {product-version} installation is to achieve targets. If you are interested in seeing a graph of all the targets and dependencies for {product-title} {product-version} installation, run this command:
`$ openshift-install graph | dot -Tsvg >/tmp/ocp4_graph.svg`
Open the resulting graph with any SVG reader (such as eog /tmp/ocp4_graph.svg). A partial view of the dependency tree looks as follows:
.{product-title} installation targets and dependencies
image::targets-and-dependencies.png[{product-title} installation targets and dependencies]
The graph contains vertices, and the edges that connect them, of component Operators. The edges in the graph show which versions you can safely update to, and the vertices are update payloads that specify the intended state of the managed cluster components.
The following text describes the components involved in the install process. Keep in mind that these descriptions indicate a general order of activities that occur during the installation of an {product-title} cluster, since the exact order varies based on when each target completes.
[id="providing-configuration-information_{context}"]
== Providing configuration information to the installer
With a https://docs.openshift.com/container-platform/4.1/installing/installing_aws/installing-aws-account.html[properly configured AWS platform] in place, answers to a few prompts is all you need to start up an {product-title} Cluster. However, there are other supported and unsupported ways to change the configuration that is deployed. Here is a rundown of your options:
[IMPORTANT]
====
Although not required, you should create a directory to hold your installation configuration files and identify it when you run the installation. Don’t delete that directory when you have created your cluster, since it can be used to delete your cluster later. Use separate directories for each cluster.
====
[id="running-default-install_{context}"]
=== Running a default install
For the default {product-title} install shown here, you are prompted for a few pieces of information needed by the installer: +
----
$ openshift-install create cluster --dir $HOME/clusterconfig
----
You are asked for:
* An ssh public key to access the cluster
* aws as the platform
* A selected region to run in
* The base domain name for your cluster
* The name of your cluster (which will append to your domain name)
* A pull secret.
This approach is fully supported.
[id="running-install-config_{context}"]
=== Running an install-config install
For this installation type, you create an install-config.yaml file, modify it, and run the installer so it uses that file to start the installation. This type of installation is fully supported.
----
$ openshift-install create install-config --dir $HOME/clusterconfig
----
Here’s an example of the install-config.yaml created in your install directory, with fields highlighted that you can change:
----
apiVersion: v1
baseDomain: devel.example.com (1)
compute:
- hyperthreading: Enabled
name: worker
platform: {}
replicas: 3 (2)
controlPlane:
hyperthreading: Enabled
name: master
platform: {}
replicas: 3 (3)
metadata:
creationTimestamp: null
name: mycluster (4)
networking:
clusterNetwork:
- cidr: 10.128.0.0/14 (5)
hostPrefix: 23
machineCIDR: 10.0.0.0/16 (6)
networkType: OpenShiftSDN
serviceNetwork:
- 172.30.0.0/16 (7)
platform:
aws:
region: us-east-2 (8)
pullSecret: '{"auths":{"x.example.com":{"auth":"b3Blb...}}}' (9)
sshKey: |
ssh-rsa AAAAB...|== joe@example.com (10)
----
1. baseDomain: The DNS domain name configured with your AWS account. It will be added as the base to the name you assign to your cluster.
2. compute.worker.replicas: The number of compute nodes to create initially in your cluster. The default is 3.
3. controlPlane.master.replicas: The number of master nodes to create initially in your cluster. The default is 3.
4. metadata.name: The name assigned to your cluster. For example, if the name were mycluster, and the domain name were devel.example.com, the full DNS name of your cluster would be mycluster.devel.example.com
5. networking.clusterNetworks.cidr: The CIDR formatted IP address range to use for the cluster networks.
6. networking.machineCIDR: The CIDR formatted IP address range to use for the nodes
7. networking.serviceNetwork: The CIDR formatted IP address range to use for the Kubernetes services
Once you have modified the install-config.yaml file, make a backup copy of that file if you intend to use it again. The reason is that the file is deleted when you install from it. If you are ready, run the installer again (identifying the directory holding the install-config.yaml file) to install the cluster with those new settings:
`$ openshift-install create cluster --dir $HOME/clusterconfig`
[id="running-manifest-install_{context}"]
=== Running a manifest install (unsupported)
By default, the fact that an {product-title} cluster installation uses a set of manifest files is invisible. However, the installer gives you the option to drop those manifest files into a directory and halt the install process. It is possible, though not supported, to then change or add to those manifest files and continue the installation with the modified manifests.
The manifest files are saved in two directories. The manifests/ directory contains the Kubernetes manifests, while the openshift/ directory hold {product-title} manifests. Here’s how to get those manifests, followed by some suggestions about using them:
----
$ openshift-install create manifests --dir $HOME/clusterconfig
$ ls $HOME/clusterconfig/manifests/ $HOME/clusterconfig/openshift/
clusterconfig/manifests/:
04-openshift-machine-config-operator.yaml  host-etcd-service-endpoints.yaml
cluster-config.yaml                        host-etcd-service.yaml
cluster-dns-02-config.yml                  kube-cloud-config.yaml
cluster-infrastructure-02-config.yml       kube-system-configmap-etcd-serving-ca.yaml
cluster-ingress-02-config.yml              kube-system-configmap-root-ca.yaml
cluster-network-01-crd.yml                 kube-system-secret-etcd-client.yaml
cluster-network-02-config.yml              machine-config-server-tls-secret.yaml
cvo-overrides.yaml                         pull.json
etcd-service.yaml
clusterconfig/openshift/:
99_binding-discovery.yaml               99_openshift-cluster-api_master-machines-2.yaml
99_cloud-creds-secret.yaml              99_openshift-cluster-api_master-user-data-secret.yaml
99_kubeadmin-password-secret.yaml       99_openshift-cluster-api_worker-machineset.yaml
99_openshift-cluster-api_cluster.yaml   99_openshift-cluster-api_worker-user-data-secret.yaml
99_openshift-cluster-api_master-machines-0.yaml  99_role-cloud-creds-secret-reader.yaml
99_openshift-cluster-api_master-machines-1.yaml
----
At this point, if you were to add or modify files in the manifest/ or openshift/ directories, those would be consumed and used in the cluster installation. As mentioned earlier, changing or adding to these files is not supported, but only described here for informational purposes. Here are some tips on how to handle the manifest files:
* You should make sure to get the install-config.yaml file exactly right before creating the manifest files. Manifest files are generated, in part, from information in install-config.yaml. Because things like domain names are spread across manifest files, changes to those settings in a manifest file could easily become out of sync with other manifest files.
* Think of the manifest file more like intermediate files than configuration files. Changing them would be like editing object files instead of fixing the source code.
* If you do want to change your cluster from manifest files, the safest way is to create a new manifest file and make sure it doesn’t conflict with anything any other manifest file is doing.
Once you have the manifests the way you like, simply continue the install process:
----
$ openshift-install create cluster --dir $HOME/clusterconfig
----
There a couple other things to note about providing information to the installer:
* Ignition config files are also used in the install process, to configure the initial setup of  nodes. Refer to the Ignition section later for details on those configuration files
* When you generate the configuration files, the entire state of the installer configuration is stored in a .openshift_install_state.json file in the cluster directory. Any config file that is deleted can be regenerated from that file by simply running the openshift-installer create command for the configuration file you want. Keep in mind, though, that you can’t rely on the format of this file going forward.
If you want to watch the install take place in detail, add a --log-level debug option to the openshift-install command line. Instead you could also just follow the installation from the .openshift_install.log file as the installation progresses:
----
$ tail -f $HOME/clusterconfig/.openshift_install.log
time="2019-04-08T10:36:09-04:00" level=debug msg="{product-title} Installer v0.14.0"
time="2019-04-08T10:36:09-04:00" level=debug msg="Fetching \"Terraform Variables\"…​"
time="2019-04-08T10:36:09-04:00" level=debug msg="Loading \"Terraform Variables\"…​"
time="2019-04-08T10:36:09-04:00" level=debug msg="  Loading \"Cluster ID\"…​"
…​
time="2019-04-08T10:36:09-04:00" level=debug msg="  Loading \"Install Config\"…​"
time="2019-04-08T10:36:09-04:00" level=debug msg="  Loading \"Image\"…​"
time="2019-04-08T10:36:09-04:00" level=debug msg="        Loading \"Install Config\"…​"
time="2019-04-08T10:36:09-04:00" level=debug msg="  Loading \"Bootstrap Ignition Config\"…​"
time="2019-04-08T10:36:09-04:00" level=debug msg="        Loading \"Install Config\"…​
…​
----
[id="configuring-infrastructure-terraform_{context}"]
== Setting up infrastructure with Terraform
When you run openshift-install create cluster, one of the first things to happen is that Terraform takes over to create the infrastructure. Terraform gathers and loads all the configuration files, then it creates the VMs, the network, and other parts of the infrastructure.
After Terraform has set up the infrastructure, it polls the Kubernetes API endpoint, trying to connect to it. Once it can reach the API endpoint, it queries for some basic information about the cluster and waits for the bootstrap node to finish setting up the control plane. In particular, it is looking to see the status of the Cluster Version Operator, which is responsible for starting all the other Operators in the control plane. The work isn’t done until the CVO status is 100% complete.
Once the bootstrap node’s work is done, Terraform destroys the bootstrap node, deleting the virtual machine and removing AWS resources for that node (the S3 bucket, security group rules, IAM roles, and other resources). At this point, the cluster takes over communications with the AWS API and begins bringing up the worker nodes.
There are a few things to keep in mind about the completed cluster going forward:
* To keep track of the state of the cluster, a set of terraform* files is copied to the installation directory. Don’t delete that directory yet! Those files will be used later when you tear down the cluster with the following command: +
      openshift-install destroy cluster --dir $HOME/clusterconfig
* Other than the terraform* files, the configuration files in the cluster directory are not used again for the current cluster. In other words, any new nodes or management of existing nodes are managed by the cluster itself (see the description of Operators later). You can copy Ignition config, install config, and manifest files to use for later installations, but consider them having been consumed and done with for the current configuration.
The process of creating the cluster, for a default installation, starts by creating a bootstrap node. The boostrap node creates the master nodes that make up the control plane. The worker nodes are then create from the control plane. The following figure shows this process:
.Creating the bootstrap, master, and worker nodes
image::create-nodes.png[Creating bootstrap, master, and worker nodes]
The nodes just alluded to are created by combining a {op-system-first}({op-system}) operating system image with the configuration files just described. Leading the actual initial setup of each system is the Ignition first boot installer. Those components are described next.
[id="about-rhcos_{context}"]
== About {op-system} operating system images
{op-system-first}({op-system}) represents the next generation of single-purpose container operating system technology. Created by the same development teams that created RHEL Atomic Host and CoreOS Container Linux, {op-system} combines the quality standards of RHEL with automated, remote upgrade features from Container Linux.
At the moment, {op-system} is only supported as a component of {product-title} {product-version} for all {product-title} nodes, as the team locks down those features needed to make {op-system} a fully automated operating system for running and managing containers. In fact, {op-system} is currently the only supported operating system for {product-title} {product-version} (with manual installation of RHEL scheduled to be supported on worker nodes in the near future).
{op-system} images are downloaded to the target platform during {product-title} installation and initially set up with Ignition (described next). Here are some of the key features of {op-system}:
* Based on RHEL: The underlying operating system consists primarily of Red Hat Enterprise Linux components. So the same quality, security, and control measures supporting RHEL also are in {op-system}. For example, {op-system} software is in RPM packages and each {op-system} system starts up with a RHEL kernel and a set of services managed by the systemd init system.
* Controlled immutability: Although it contains RHEL components, {op-system} is designed to be managed more tightly than a default RHEL installation, with most of that management done remotely (in this case, from an {product-title} cluster). A limited number of levers are available to change how your {op-system} nodes are set up and modified. This controlled immutability allows {product-title} to store the latest state of {op-system} systems in the cluster so it is always able to spin up new nodes and perform updates based on the latest {op-system} configurations. {op-system} is required for all master nodes, but if you are provisioning your own infrastructure, RHEL 7.6 and RHEL 8 systems are also supported.
* CRI-O container runtime: Although {op-system} contains features for running OCI- and libcontainer-formatted (used by Docker) containers, it incorporates the https://access.redhat.com/documentation/en-us/openshift_container_platform/3.11/html-single/cri-o_runtime/index[CRI-O container engine], instead of the Docker container engine. By focusing on features needed by Kubernetes platforms, such as {product-title}, CRI-O can offer specific compatibility with different Kubernetes versions. CRI-O also offers a smaller footprint and reduced attack surface than is possible with container engines that offer a larger feature set. At the moment, Red Hat’s CRI-O is not offered as a stand-alone container engine outside of {product-title}.
* Set of container tools: For tasks such as building, copying, and otherwise managing containers, {op-system} replaces the docker CLI tool with a compatible set of container tools. The podman CLI tool supports many container runtime features, such as running, starting, stopping, listing, and removing containers and container images. The skopeo CLI tool can copy, authenticate, and sign images. The crictl CLI tool lets you work with containers and pods from the CRI-O container engine. While direct use of these tools in {op-system} is discouraged, they are used under the covers in {op-system} or for debugging purposes. You can use podman and skopeo on RHEL systems, without {product-title}, to work directly with containers.
* rpm-ostree upgrades: {op-system} features transactional upgrades and rollbacks using the http://www.projectatomic.io/docs/os-updates/[rpm]http://www.projectatomic.io/docs/os-updates/[ ]http://www.projectatomic.io/docs/os-updates/[-ostree] upgrade system. In {product-title}, the https://github.com/openshift/machine-config-operator/blob/master/docs/OSUpgrades.md[MachineConfigOperator handles operating system upgrades]. Instead of upgrading individual packages, as is done with yum upgrades, rpm-ostree delivers upgrades as an atomic unit. The downloaded tree goes into effect on the next reboot. If something goes wrong with the upgrade, a single rollback and reboot returns the system to the previous state. {op-system} upgrades in {product-title} {product-version} are done as part of larger cluster upgrades as needed.
* Set up with Ignition, managed by MCO: For {product-title} {product-version}, {op-system} images are set up initially with a feature called  https://github.com/coreos/ignition[Ignition], which runs only on the system’s first boot. After first boot, {op-system} systems are managed by the Machine Config Operator (MCO) running in the {product-title} cluster. Ignition and MCO are described later.
Because {op-system} systems in {product-title} are designed to be fully managed from the {product-title} cluster, directly logging into a {op-system} node is discouraged. Limited direct access to {op-system} nodes in a {product-title} cluster can be done for debugging purposes. Using the oc debug node/<nodename> command (if {product-title} is up) or creating a bastion host (if the nodes are inaccessible through {product-title}) are ways of gaining direct access to a {op-system} node.
Whether during initial installation from the openshift-install tool or when later adding an {op-system} system to {product-title} later from the cluster, the feature that does the initial setup of {op-system} into bootstrap, worker, or master nodes is done by Ignition.
[id="initialize-nodes-ignition_{context}"]
== Using Ignition to initialize nodes
For an {product-title} installation on AWS (where AWS provisions the infrastructure), the https://github.com/coreos/ignition[Ignition] first boot installer and configuration tool takes raw {op-system} images and prepares them to become the master (control plane) nodes that make up the {product-title} cluster. Later, master (control plane) nodes use the machine api/controller to deploy worker (compute) nodes with Ignition. The {product-title} installer begins Ignition’s work using a bootstrap config that spins up a temporary bootstrap node to direct the setup of each worker node slated for the permanent cluster. The bootstrap node:
* Starts up as a separate, and first, node in the cluster’s environment
* Generates and deploys Ignition configs to each of the master machines that will make up the initial cluster. Ignition configs can vary, based on the specific operating system being deployed and the cluster environment (for example, a particular cloud or bare metal environment).
* Shuts down and is deleted from the cluster once all the control plane nodes get to a point where they can serve their own configs
Most of the actual system setup happens on each node itself. For each node, Ignition takes the {op-system} image and boots the {op-system} kernel. On the kernel command line, options identify the type of deployment and the Ignition-enabled initial Ram Disk (initramfs). To configure the worker and master nodes, Ignition does the following during the firstboot (and only during firstboot) of each node:
* Gets its Ignition config. Master nodes get their Ignition configs from the bootstrap node, while worker nodes get Ignition configs from a master.
* Creates disk partitions, filesystems, directories and links. Can do RAID arrays, but does not yet support LVM volumes
* Mounts the root file system to /sysroot inside the initramfs and sets about directing its work to that root filesystem
* Configures all defined filesystems and sets them up to mount appropriately at runtime
* Runs systemd temp files to populate anything needed in /var
* Runs ignition files to set up users, systemd unit files, and other configuration files
* Unmounts anything from the permanent system that was mounted inside the initramfs
* Starts up new node’s init process which, in turn, starts up all other services on the node as if it were carrying out a normal boot on the node
The node is then ready to join the cluster, without requiring a reboot.
[id="about-ignition_{context}"]
=== How Ignition works
The way that Ignition configures nodes is similar to how tools like https://cloud-init.io/[cloud-init] or Linux Anaconda https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/installation_guide/index%23chap-kickstart-installations[kickstart] configure systems, but with some important differences:
* Runs from initramfs: Ignition runs from an initial RAM disk that is separate from the system you are installing to. Because of that, Ignition is free to repartition disks, set up file systems, and do anything else it likes to the node’s image. In contrast, cloud-init runs as part of a node’s init system when the system boots, so making foundational changes to things like disk partitions cannot be done as easily. With cloud-init, it is also difficult to reconfigure the boot process while you are in the middle of the node’s boot process.
* Runs only on firstboot: Ignition is meant to initialize systems, not change them going forward. After a node is setup and the kernel is running from the installed system, future configuration of the node is handled by the Machine Config Operator from the {product-title} cluster.
* Declarative: Ignition does not run a set of actions, but rather it implements a declarative configuration. Unlike an Anaconda kickstart, which runs through a set of steps like installing packages or setting firewall rules, Ignition checks that all partitions, files, services and other items are in place before the resulting node even starts up. It then makes the changes (like copying files to disk) to get the target node set up so it can be in the desired state.
* No half-configured nodes: If a node’s setup fails for some reason, like it can’t get a file because the network goes down, Ignition will not complete successfully and will not pivot to starting up the new node from its installed hard disk. In other words, you will never end up with a node that runs in a partially configured state. If Ignition cannot completely finish its work, the node is never added to the cluster. You would have to kill it and start over. This prevents the difficult case of debugging a node when the results of a failed configuration task are not known until something that depended on it fails at a later date.
* No multiple node failures from the same config: If there is a problem with an Ignition config that causes the setup of a node to fail, Ignition will not try to use the same config to set up another node. For example, a failure could result from an Ignition config made up of a parent and child config that both want to create the same file. A failure in such a case would prevent that Ignition config from being used again to set up an other nodes, until the problem is resolved.
* Same kernel for configuring and running: Once Ignition is done with its configuration, the kernel keeps running, but discards the initial RAM disk and pivots to the installed system on disk. All the newly installed system services and other features are started, without requiring a reboot, as is needed with cloud-init or kickstart configurations.
* Merge Ignition configs:  If there is a set of Ignition configs, you get a union of that set of configs.  Because Ignition is declarative, conflicts between the configs could cause Ignition to fail to set up the node. The order of information in those files doesn’t matter. Ignition will sort and implement each setting in ways that make the most sense. For example, if a file needs a directory several levels deep, if another file needs a directory along that path, the later file is created first. Ignition sorts and creates all files, directories, and links by depth.
* Bare metal, as well as cloud: Because Ignition isn’t limited by running inside the target node, it can do something cloud-init can’t do: set up systems on bare metal from scratch (using features such as PXE boot). In the bare metal case, the Ignition config is injected into the boot partition so Ignition can find it and configure the system correctly.
Due to some quirks in the tooling used to create Ignition, some explanation is needed for its version numbers:
* Ignition version: {product-title} {product-version} uses Ignition v2. The previous versions were all 0.x. There is no Ignition version 1.
* Ignition configs version: {product-title} {product-version} uses v2.3 Ignition configs and only supports that version. Previous Ignition config versions included v1, v2, v2.1, v2.2, and v2.3. If presented with those earlier versions, Ignition upgrades that Ignition config through each version until it reaches v2.3, then runs the resulting Ignition config. +
+
Ignition config version 3 is available, but has not yet been added to {product-title} {product-version}. One of the new features of v3 is that it will allow a child Ignition config that merges with a parent config to overwrite any file on the parent for which there is a conflict. The merge and replace features of the https://github.com/coreos/ignition/blob/master/doc/configuration-v3_0.md[Ignition config v3 spec]provides cleaner ways of managing these conflicts. This feature also allows many system types to share a common Ignition config, while differences, such as specific hardware or cloud features, can be added with child configs. +
+
All v3 versions (v3.1, v3.2, etc.) will be guaranteed to be supported until v4 comes out. At that point, fields deprecated in later v3.x versions could be removed in v4.
[id="viewing-ignition-config-files_{context}"]
=== Viewing Ignition configuration files
To see the Ignition config file used to deploy the bootstrap node, run the following command:
----
$ openshift-install create ignition-configs --dir $HOME/testconfig
----
After you answer a few questions, the bootstrap.ign, master.ign, and worker.ign files appear in the directory you entered.
To see the contents of the bootstrap.ign file, pipe it through the jq filter. Here’s a snippet from that file:
----
$ cat $HOME/testconfig/bootstrap.ign | jq
\\{
  "ignition": \\{
        "config": \\{},
…​
  "storage": \\{
        "files": [
          \\{
            "filesystem": "root",
            "path": "/etc/motd",
            "user": \\{
              "name": "root"
            },
            "append": true,
            "contents": \\{
              "source": "data:text/plain;charset=utf-8;base64,VGhpcyBpcyB0aGUgYm9vdHN0cmFwIG5vZGU7IGl0IHdpbGwgYmUgZGVzdHJveWVkIHdoZW4gdGhlIG1hc3RlciBpcyBmdWxseSB1cC4KClRoZSBwcmltYXJ5IHNlcnZpY2UgaXMgImJvb3RrdWJlLnNlcnZpY2UiLiBUbyB3YXRjaCBpdHMgc3RhdHVzLCBydW4gZS5nLgoKICBqb3VybmFsY3RsIC1iIC1mIC11IGJvb3RrdWJlLnNlcnZpY2UK",
----
To decode the contents of a file listed in the bootstrap.ign file, pipe the base64-encoded data string representing the contents of that file to the base64 -d command. Here’s an example using the contents of the /etc/motd file added to the bootstrap node from the output shown above:
----
$ echo VGhpcyBpcyB0aGUgYm9vdHN0cmFwIG5vZGU7IGl0IHdpbGwgYmUgZGVzdHJveWVkIHdoZW4gdGhlIG1hc3RlciBpcyBmdWxseSB1cC4KClRoZSBwcmltYXJ5IHNlcnZpY2UgaXMgImJvb3RrdWJlLnNlcnZpY2UiLiBUbyB3YXRjaCBpdHMgc3RhdHVzLCBydW4gZS5nLgoKICBqb3VybmFsY3RsIC1iIC1mIC11IGJvb3RrdWJlLnNlcnZpY2UK | base64 -d
----
This is the bootstrap node; it will be destroyed when the master is fully up.
The primary service is "bootkube.service". To watch its status, run, e.g.:
----
          journalctl -b -f -u bootkube.service
----
Repeat those commands on the master.ign and worker.ign files to see the source of Ignition config files for each of those node types.  You should see a line like the following for the worker.ign, identifying how it gets its Ignition config from the bootstrap node:
----
"source": "https://api.myign.develcluster.example.com:22623/config/worker",
----
Here are a few things you can learn from the bootstrap.ign file: +
* Format: The format of the file is defined in the https://github.com/coreos/ignition/tree/spec2x[Ignition config spec]. Files of the same format are used later by the MCO to merge changes into a node’s configuration.
* Contents: Because the bootstrap node serves the Ignition configs for other nodes, both master and worker node Ignition config information is stored in the bootstrap.ign, along with the bootstrap node’s configuration.
* Size: The file is more than 1300 lines long, with path to various types of resources.
* The content of each file that will be copied to the node is actually encoded into data URLs, which tends to make the content a bit clumsy to read. (Use the jq and base64 commands shown previously to make the content more readable.)
* Configuration: The different sections of the Ignition config file are generally meant to contain files that are just dropped into a node’s file system, rather than commands to modify existing files. For example, instead of having a section on NFS that configures that service, you would just add an NFS configuration file, which would then be started by the init process when the system comes up.
* users: A user named core is created, with your ssh key assigned to that user. This will allow you to log into the cluster with that user name and your credentials.
* storage: The storage section identifies files that are added to each node. A few notable files include /root/.docker/config.json (which provides credentials your cluster needs to pull from container image registries) and a bunch of manifest files in /opt/openshift/manifests that are used to configure your cluster.
* systemd: The systemd section holds content used to create systemd unit files. Those files are used to start up services at boot time, as well as manage those services on running systems.
* Primitives: Ignition also exposes low-level primitives that other tools can build on.
After Ignition finishes its work on an individual node, the kernel pivots to the installed system. The initial RAM disk is no longer used and the kernel goes on to run the init service to start up everything on the host from the installed disk. When the last node under the bootstrap node’s control is completed, and the services on those nodes come up, the work of the bootstrap node is over.

109
modules/ignition-config-viewing.adoc Normal file
View File

@@ -0,0 +1,109 @@
// Module included in the following assemblies:
//
// *
[id="ignition-config-viewing_{context}"]
= Viewing Ignition configuration files
To see the Ignition config file used to deploy the bootstrap machine, run the following command:
----
$ openshift-install create ignition-configs --dir $HOME/testconfig
----
After you answer a few questions, the bootstrap.ign, master.ign, and worker.ign
files appear in the directory you entered.
To see the contents of the bootstrap.ign file, pipe it through the jq filter.
Here’s a snippet from that file:
----
$ cat $HOME/testconfig/bootstrap.ign | jq
\\{
  "ignition": \\{
        "config": \\{},
…​
  "storage": \\{
        "files": [
          \\{
            "filesystem": "root",
            "path": "/etc/motd",
            "user": \\{
              "name": "root"
            },
            "append": true,
            "contents": \\{
              "source": "data:text/plain;charset=utf-8;base64,VGhpcyBpcyB0aGUgYm9vdHN0cmFwIG5vZGU7IGl0IHdpbGwgYmUgZGVzdHJveWVkIHdoZW4gdGhlIG1hc3RlciBpcyBmdWxseSB1cC4KClRoZSBwcmltYXJ5IHNlcnZpY2UgaXMgImJvb3RrdWJlLnNlcnZpY2UiLiBUbyB3YXRjaCBpdHMgc3RhdHVzLCBydW4gZS5nLgoKICBqb3VybmFsY3RsIC1iIC1mIC11IGJvb3RrdWJlLnNlcnZpY2UK",
----
To decode the contents of a file listed in the bootstrap.ign file, pipe the
base64-encoded data string representing the contents of that file to the base64
-d command. Here’s an example using the contents of the /etc/motd file added to
the bootstrap machine from the output shown above:
----
$ echo VGhpcyBpcyB0aGUgYm9vdHN0cmFwIG5vZGU7IGl0IHdpbGwgYmUgZGVzdHJveWVkIHdoZW4gdGhlIG1hc3RlciBpcyBmdWxseSB1cC4KClRoZSBwcmltYXJ5IHNlcnZpY2UgaXMgImJvb3RrdWJlLnNlcnZpY2UiLiBUbyB3YXRjaCBpdHMgc3RhdHVzLCBydW4gZS5nLgoKICBqb3VybmFsY3RsIC1iIC1mIC11IGJvb3RrdWJlLnNlcnZpY2UK | base64 -d
----
This is the bootstrap machine; it will be destroyed when the master is fully up.
The primary service is "bootkube.service". To watch its status, run, e.g.:
----
          journalctl -b -f -u bootkube.service
----
Repeat those commands on the master.ign and worker.ign files to see the source
of Ignition config files for each of those machine types.  You should see a line
like the following for the worker.ign, identifying how it gets its Ignition
config from the bootstrap machine:
----
"source": "https://api.myign.develcluster.example.com:22623/config/worker",
----
Here are a few things you can learn from the bootstrap.ign file: +
* Format: The format of the file is defined in the
https://github.com/coreos/ignition/tree/spec2x[Ignition config spec].
Files of the same format are used later by the MCO to merge changes into a
machine’s configuration.
* Contents: Because the bootstrap machine serves the Ignition configs for other
machines, both master and worker machine Ignition config information is stored in the
bootstrap.ign, along with the bootstrap machine’s configuration.
* Size: The file is more than 1300 lines long, with path to various types of resources.
* The content of each file that will be copied to the machine is actually encoded
into data URLs, which tends to make the content a bit clumsy to read. (Use the
jq and base64 commands shown previously to make the content more readable.)
* Configuration: The different sections of the Ignition config file are generally
meant to contain files that are just dropped into a machine’s file system, rather
than commands to modify existing files. For example, instead of having a section
on NFS that configures that service, you would just add an NFS configuration
file, which would then be started by the init process when the system comes up.
* users: A user named core is created, with your ssh key assigned to that user.
This will allow you to log into the cluster with that user name and your
credentials.
* storage: The storage section identifies files that are added to each machine. A
few notable files include /root/.docker/config.json (which provides credentials
your cluster needs to pull from container image registries) and a bunch of
manifest files in /opt/openshift/manifests that are used to configure your cluster.
* systemd: The systemd section holds content used to create systemd unit files.
Those files are used to start up services at boot time, as well as manage those
services on running systems.
* Primitives: Ignition also exposes low-level primitives that other tools can
build on.

4
modules/installation-aws-user-infra-bootstrap.adoc
View File

@@ -25,10 +25,12 @@ you can install the cluster.
following command:
+
----
$ ./openshift-install wait-for bootstrap-complete --dir=<installation_directory> <1>
$ ./openshift-install wait-for bootstrap-complete --dir=<installation_directory> \ <1>
--log-level debug <2>
----
<1> For `<installation_directory>`, specify the path to the directory that you
stored the installation files in.
<2> Optionally, include the `--log-level debug` option to view installation details.
+
If the command exits without a `FATAL` warning, your production control plane
has initialized.

8
modules/installation-configuration-parameters.adoc
View File

@@ -26,7 +26,9 @@ You cannot modify these parameters after installation.
|`baseDomain`
|The base domain of your cloud provider. This value is used to create routes
to your {product-title} cluster components.
to your {product-title} cluster components. The full DNS name for your cluster
is a combination of the `baseDomain` and `metadata.name` parameter values that
uses the `<metadata.name>.<baseDomain>` format.
|A fully-qualified domain or subdomain name, such as `example.com`.
|`controlPlane.platform`
@@ -130,7 +132,7 @@ such as `us-east-1`.
|`compute.replicas`
|The number of compute, or worker, machines to provision.
|A positive integer greater than or equal to `2`.
|A positive integer greater than or equal to `2`. The default value is `3`.
|`controlPlane.hyperthreading`
|Whether to enable or disable simultaneous multithreading, or `hyperthreading`
@@ -161,7 +163,7 @@ such as `us-east-1`.
|`controlPlane.replicas`
|The number of control plane machines to provision.
|A positive integer greater than or equal to `3`.
|A positive integer greater than or equal to `3`. The default value is `3`.
|`platform.aws.userTags`
|A map of keys and values that the installation program adds as tags to all

4
modules/installation-installing-bare-metal.adoc
View File

@@ -23,7 +23,8 @@ cluster.
. Start and monitor the installation process:
+
----
$ ./openshift-install --dir=<installation_directory> wait-for bootstrap-complete <1>
$ ./openshift-install --dir=<installation_directory> wait-for bootstrap-complete \ <1>
--log-level debug <2>
INFO Waiting up to 30m0s for the Kubernetes API at https://api.test.example.com:6443...
INFO API v1.13.4+b626c2fe1 up
@@ -31,6 +32,7 @@ INFO Waiting up to 30m0s for the bootstrap-complete event...
----
<1> For `<installation_directory>`, specify the path to the directory that you
stored the installation files in.
<2> Optionally, include the `--log-level debug` option to view installation details.
+
The command succeeds when the Kubernetes APIServer signals that it has been
bootstrapped on the control plane machines.

6
modules/installation-launching-installer.adoc
View File

@@ -22,10 +22,11 @@ You can run the installation program only once, during initial installation.
.Procedure
. Run the installation program:
. Run the installation program
+
----
$ ./openshift-install create cluster --dir=<installation_directory> <1>
$ ./openshift-install create cluster --dir=<installation_directory> \ <1>
--log-level debug <2>
----
ifeval::["{context}" == "install-customizations-cloud"]
<1> For `<installation_directory>`, specify the location of your customized
@@ -38,6 +39,7 @@ endif::[]
ifeval::["{context}" == "installing-aws-default"]
<1> For `<installation_directory>`, specify the directory name to store the
files that the installation program creates.
<2> Optionally, include the `--log-level debug` option to view installation details.
+
[IMPORTANT]
====

66
modules/installation-overview.adoc
View File

@@ -1,5 +1,6 @@
// Module included in the following assemblies:
//
// * architecture/architecture-installation.adoc
// * installing/installing_aws/installing-aws-default.adoc
// * installing/installing_aws/installing-aws-customizations.adoc
// * installing/installing_aws/installing-aws-network-customizations.adoc
@@ -39,6 +40,20 @@ assets generated by the installation program are the Ignition config files for t
bootstrap, master, and worker machines. With these three configurations and correctly
configured infrastructure, you can start an {product-title} cluster.
The {product-title} installation program uses a set of targets and dependencies to manage
cluster installation. The installation program has a set of targets that it must achieve,
and each target has a set of dependencies. Because each target is only
concerned with its own dependencies, the installation program can act to achieve
multiple targets in parallel. The ultimate target is a running cluster. By
meeting dependencies instead of running commands, the installation program is
able to recognize and use existing components instead of running the commands to
create them again.
The following diagram shows a subset of the installation targets and dependencies:
.{product-title} installation targets and dependencies
image::targets-and-dependencies.png[{product-title} installation targets and dependencies]
After installation, each cluster machine uses {op-system-first} as the operating
system. {op-system} is the immutable container host version of
Red Hat Enterprise Linux (RHEL) and features a RHEL kernel with SELinux enabled
@@ -47,7 +62,7 @@ the CRI-O container runtime, which is optimized for Kubernetes.
Every control plane machine in an {product-title} {product-version} cluster must
use {op-system}, which includes a critical first-boot provisioning tool called
CoreOS Ignition. This tool enables the cluster to configure the machines.
Ignition. This tool enables the cluster to configure the machines.
Operating system updates are delivered as an Atomic OSTree repository that is
embedded in a container image that is rolled out across the cluster by an
Operator. Actual operating system changes are made in-place on each machine as
@@ -56,6 +71,14 @@ an atomic operation by using rpm-ostree. Together, these technologies enable
application on the cluster, via in-place upgrades that keep the entire platform
up-to-date. These in-place updates can reduce the burden on operations teams.
If you use {op-system} as the operating system for all cluster machines, the
cluster manages all aspects of its components and machines, including the
operating system. Because of this, only the installation program and the
Machine Config Operator can change machines. The installation program uses
Ignition config files to set the exact state of each machine, and the Machine
Config Operator completes more changes to the machines, such as the application
of new certificates or keys, after installation.
[id="available-platforms_{context}"]
== Available platforms
@@ -97,11 +120,12 @@ cluster.
* If you provision and manage the infrastructure for your cluster, you must
provide all of the cluster infrastructure and resources, including the
bootstrap node, networking, load balancing, storage, and individual cluster
bootstrap machine, networking, load balancing, storage, and individual cluster
machines.
You use three sets of files during installation: an installation configuration
file, Kubernetes manifests, and Ignition config files for your machine types.
file that is named `install-config.yaml`, Kubernetes manifests, and Ignition
config files for your machine types.
[IMPORTANT]
====
@@ -179,18 +203,24 @@ and maintain the cluster resources yourself, including:
* Storage for the cluster infrastructure and applications
If your cluster uses user-provisioned infrastructure, you have the option of
adding RHEL worker nodes to your cluster.
adding RHEL worker machines to your cluster.
[discrete]
=== Bootstrap process details
=== Installation process details
Because each machine in the cluster requires information about the cluster when
it is provisioned, {product-title} uses a temporary _bootstrap_ machine during
initial configuration. This machine provides the required information. It boots
by using an Ignition config file
that describes how to create the cluster. This temporary machine
is a temporary control plane that launches the rest of the cluster and then is
destroyed. All clusters use the bootstrap process to initialize the
initial configuration to provide the required information to the permanent
control plane. It boots by using an Ignition config file that describes how to
create the cluster. The boostrap machine creates the master machines that
make up the control plane. The control plane machines then create the compute,
or worker, machines. The following figure illustrates this process:
.Creating the bootstrap, master, and worker machines
image::create-nodes.png[Creating bootstrap, master, and worker machines]
After the cluster machines initialize, the bootstrap machine is destroyed.
All clusters use the bootstrap process to initialize the
cluster, but if you provision the infrastructure for your cluster, you must
complete many of the steps manually.
@@ -209,26 +239,32 @@ for the master machines to boot. (Requires manual intervention if you provision
. The master machines fetch the remote resources from the bootstrap machine
and finish booting. (Requires manual intervention if you provision the
infrastructure)
. The master machines use the bootstrap node to form an etcd cluster.
. The bootstrap node starts a temporary Kubernetes control plane using the
. The master machines use the bootstrap machine to form an etcd cluster.
. The bootstrap machine starts a temporary Kubernetes control plane using the
new etcd cluster.
. The temporary control plane schedules the production control plane to the
master machines.
. The temporary control plane shuts down and passes control to the production
control plane.
. The bootstrap node injects {product-title} components into the production
. The bootstrap machine injects {product-title} components into the production
control plane.
. The installation program shuts down the bootstrap node.
. The installation program shuts down the bootstrap machine.
(Requires manual intervention if you provision the infrastructure)
. The control plane sets up the worker nodes.
. The control plane installs additional services in the form of a set
of Operators.
The result of this bootstrapping process is a fully running {product-title}
cluster. The cluster then downloads and configures remaining components
needed for the day-to-day operation, including the creation of worker machines
in supported environments.
[discrete]
=== Installation process details
[discrete]
== Installation scope
The scope of the {product-title} installation program is intentionally narrow.
It is designed for simplicity and ensured success. You can complete many
more configuration tasks after installation completes.
more configuration tasks after installation completes.

263
modules/rhcos-about.adoc Normal file
View File

@@ -0,0 +1,263 @@
// Module included in the following assemblies:
//
// * architecture/architecture_rhcos.adoc
[id="rhcos-about_{context}"]
= About {op-system}
{op-system-first} represents the next generation of single-purpose
container operating system technology. Created by the same development teams
that created Red Hat Enterprise Linux Atomic Host and CoreOS Container Linux,
{op-system} combines the quality standards of Red Hat Enterprise Linux (RHEL)
with the automated, remote upgrade features from Container Linux.
{op-system} is supported only as a component of {product-title}
{product-version} for all {product-title} machines. {op-system} is the only
supported operating system for {product-title} control plane, or master,
machines. While {op-system} is the default operating system for all cluster
machines, you can create compute, or worker, machines that use RHEL as their
operating system.
If you install your cluster on infrastructure that the cluster provisions,
{op-system} images are downloaded to the target platform during installation,
and suitable Ignition config files, which control the {op-system} configuration,
are used to deploy the machines. If you install your cluster on infrastructure
that you manage, you must follow the installation documentation to obtain the
{op-system} images, generate Ignition config files, and use the Ignition config
files to provision your machines.
[id="rhcos-key-features_{context}"]
== Key {op-system} features
The following list describes key features of the {op-system} operating system:
The underlying operating system consists primarily of RHEL components.
The same quality, security, and control measures that support RHEL also support
{op-system}. For example, {op-system} software is in
RPM packages, and each {op-system} system starts up with a RHEL kernel and a set
of services that are managed by the systemd init system.
Although it contains RHEL components, {op-system} is designed to be managed
more tightly than a default RHEL installation. In most cases, management is
performed remotely from the {product-title} cluster. When you set up your
{op-system} machines, you can modify only a few system settings. This controlled
immutability allows {product-title} to
store the latest state of {op-system} systems in the cluster so it is always
able to create additional machines and perform updates based on the latest {op-system}
configurations.
Although {op-system} contains features for running the
OCI- and libcontainer-formatted containers that Docker requires, it incorporates
the CRI-O container engine
instead of the Docker container engine. By focusing on features needed by
Kubernetes platforms, such as {product-title}, CRI-O can offer specific
compatibility with different Kubernetes versions. CRI-O also offers a smaller
footprint and reduced attack surface than is possible with container engines
that offer a larger feature set. At the moment, CRI-O is only available as a
container engine within {product-title} clusters.
////
For tasks such as building, copying, and otherwise
managing containers, {op-system} replaces the Docker CLI tool with a compatible
set of container tools. The podman CLI tool supports many container runtime
features, such as running, starting, stopping, listing, and removing containers
and container images. The skopeo CLI tool can copy, authenticate, and sign
images. You can use the crictl CLI tool to work with containers and pods from the
CRI-O container engine. While direct use of these tools in {op-system} is
discouraged, you can use them for debugging purposes.
////
{op-system} features transactional upgrades and rollbacks using the
http://www.projectatomic.io/docs/os-updates/[rpm]
http://www.projectatomic.io/docs/os-updates/[ ]
http://www.projectatomic.io/docs/os-updates/[-ostree]
upgrade system. In {product-title}, the MachineConfigOperator handles operating
system upgrades. Instead of upgrading individual packages, as is done with yum
upgrades, rpm-ostree delivers upgrades as an atomic unit. The downloaded tree goes
into effect on the next reboot. If something goes wrong with the upgrade, a
single rollback and reboot returns the system to the previous state. {op-system}
upgrades in {product-title} are performed during cluster updates.
[id="rhcos-configuring_{context}"]
== Configuring {op-system} in {product-title}
For {product-title}, {op-system} images are set up initially with a feature called
Ignition, which runs only on the system’s
first boot. After first boot, {op-system} systems are managed by the
Machine Config Operator (MCO) that runs in the {product-title} cluster.
Because {op-system} systems in {product-title} are designed to be fully managed
from the {product-title} cluster, directly logging into a {op-system} machine is
discouraged. Limited direct access to {op-system} machines in a {product-title}
cluster can be completed for debugging purposes.
[id="rhcos-about-ignition_{context}"]
== About Ignition
Ignition is the utility that is used by {op-system} to manipulate disks during
initial configuration. It completes common disk tasks, including partitioning
disks, formatting partitions, writing files, and configuring users. On first
boot, Ignition reads its configuration from the installation media or the
location that you specify and applies the configuration to the machines.
Whether you are installing your cluster or adding machines to it, Ignition
always performs the initial configuration of the {product-title}
cluster machines. Most of the actual system setup happens on each machine
itself. For each machine,
Ignition takes the {op-system} image and boots the {op-system} kernel. On the
kernel command line, options identify the type of deployment and the
Ignition-enabled initial Ram Disk (initramfs).
{product-title} uses Ignition version 2 and Ignition config version 2.3
////
Due to some quirks in the tooling used to create Ignition, some explanation is
needed for its version numbers:
* Ignition version: {product-title} {product-version} uses Ignition v2. The
previous versions were all 0.x. There is no Ignition version 1.
* Ignition configs version: {product-title} {product-version} uses v2.3 Ignition
configs and only supports that version. Previous Ignition config versions
included v1, v2, v2.1, v2.2, and v2.3. If presented with those earlier versions,
Ignition upgrades that Ignition config through each version until it reaches
v2.3, then runs the resulting Ignition config. +
+
Ignition config version 3 is available, but has not yet been added to
{product-title} {product-version}. One of the new features of v3 is that it will
allow a child Ignition config that merges with a parent config to overwrite any
file on the parent for which there is a conflict. The merge and replace features
of the
https://github.com/coreos/ignition/blob/master/doc/configuration-v3_0.md[Ignition config v3 spec]
provides cleaner ways of managing these conflicts. This feature also allows many
system types to share a common Ignition config, while differences, such as specific
hardware or cloud features, can be added with child configs. +
+
All v3 versions (v3.1, v3.2, etc.) will be guaranteed to be supported until v4
comes out. At that point, fields deprecated in later v3.x versions could be removed in v4.
////
[id="about-ignition_{context}"]
=== How Ignition works
////
The way that Ignition configures machines is similar to how tools like
https://cloud-init.io/[cloud-init] or Linux Anaconda
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/installation_guide/index%23chap-kickstart-installations[kickstart]
configure systems, but with some important differences:
////
To create machines by using Ignition, you need Ignition config files. The
{product-title} installation program creates the Ignition config files that you
need to deploy your cluster. These files are based on the information that you
provide to the installation program directly or through an `install-config.yaml`
file.
In some installation scenarios, you must use the ability to join Ignition
config files. If you use a set of Ignition config files, Ignition applies the
union of that set of config files.
[NOTE]
====
Because Ignition is declarative, conflicts between the config files might cause
Ignition to fail to set up the machine. Ignition sorts the data in the files
and implements each setting in the best way.
====
////
The order
of information in those files doesn’t matter. For example, if a file needs a
directory several levels deep, if another file needs a directory along that
path, the later file is created first. Ignition sorts and creates all files,
directories, and links by depth.
////
Ignition runs from an initial RAM disk that is separate
from the system you are installing to. Because of that, Ignition can
repartition disks, set up file systems, and perform other changes to the
machine’s image.
////
In contrast, cloud-init runs as part of a machine’s init system when
the system boots, so making foundational changes to things like disk partitions
cannot be done as easily. With cloud-init, it is also difficult to reconfigure
the boot process while you are in the middle of the machine’s boot process.
////
Ignition is meant to initialize systems, not change existing systems. After a
machine initializes and the kernel is running from the installed system, the
Machine Config Operator from the {product-title} cluster completes all necessary
machine configuration.
Instead of completing a defined set of actions, Ignition implements
a declarative configuration.
////
Unlike an Anaconda kickstart, which runs through a
set of steps like installing packages or setting firewall rules,
////
It checks that all partitions, files, services, and other items are in place before
the new machine starts. It then makes the changes, like copying files to disk.
that are necessary for the new machine to meet the specified configuration.
After Ignition finishes configuring a machine, the kernel keeps running but
discards the initial RAM disk and pivots to the installed system on disk. All of
the new system services and other features start without requiring a system
reboot.
////
as is needed with cloud-init or kickstart configurations.
////
Because Ignition confirms that all new machines meet the declared configuration,
you cannot have a partially-configured machine. If a machine’s setup fails,
the initialization process does not finish, and Ignition does not start the new
machine. Your cluster will never contain partially-configured machines. If
Ignition cannot complete, the machine is not added to the cluster. You must add
a new machine instead. This behavior prevents the difficult case of debugging a machine when the results of a
failed configuration task are not known until something that depended on it
fails at a later date.
////
* No multiple machine failures from the same config: If there is a problem with an
Ignition config that causes the setup of a machine to fail, Ignition will not try
to use the same config to set up another machine. For example, a failure could
result from an Ignition config made up of a parent and child config that both
want to create the same file. A failure in such a case would prevent that
Ignition config from being used again to set up an other machines, until the
problem is resolved.
* Bare metal, as well as cloud: Because Ignition isn’t limited by running
inside the target machine, it can do something cloud-init can’t do: set up systems
on bare metal from scratch (using features such as PXE boot). In the bare metal
case, the Ignition config is injected into the boot partition so Ignition can
find it and configure the system correctly.
////
[id="ignition-sequence_{context}"]
=== The Ignition sequence
The Ignition process for an {op-system} machine in a {product-title} cluster
involves the following steps:
* The machine gets its Ignition config file. Master machines get their Ignition
config files from the bootstrap machine, and worker machines get Ignition config
files from a master.
* Ignition creates disk partitions, file systems, directories, and links on the
machine. It supports RAID arrays but does not support LVM volumes
* Ignition mounts the root file system to the `/sysroot` directory in the
initramfs and starts working in the root file system.
* Ignition configures all defined file systems and sets them up to mount appropriately
at runtime.
* Ignition runs `systemd` temporary files to populate required files in the
`/var` directory.
* Ignition runs the Ignition config files to set up users, systemd unit files,
and other configuration files.
* Ignition unmounts all components in the permanent system that were mounted in
the initramfs.
* Ignition starts up new machine’s init process which, in turn, starts up all other
services on the machine that run during system boot.
The machine is then ready to join the cluster and does not require a reboot.
////
After Ignition finishes its work on an individual machine, the kernel pivots to the
installed system. The initial RAM disk is no longer used and the kernel goes on
to run the init service to start up everything on the host from the installed
disk. When the last machine under the bootstrap machine’s control is completed, and
the services on those machines come up, the work of the bootstrap machine is over.
////

22
modules/update-service-overview.adoc
View File

@@ -1,17 +1,17 @@
// Module included in the following assemblies:
//
// * architecture/architecture.adoc
// * updating/updating-cluster.adoc
// * upgrading/upgrading-cluster.adoc
[id="update-service-overview_{context}"]
= About the {product-title} update service and Cluster Version Operator
= About the {product-title} update service
The {product-title} update service is the hosted service that provides over-the-air
updates to both {product-title} and {op-system-first}. It provides a graph,
or diagram that contain _vertices_ and the _edges_ that connect them, of
component Operators. The edges in the graph show which versions you can safely
update to, and the vertices are update payloads that specify the intended state
of the managed cluster components.
of the managed cluster components.
The Cluster Version Operator (CVO) in your cluster checks with the
{product-title} update service to see the valid updates and update paths based
@@ -19,6 +19,10 @@ on current component versions and information in the graph. When you request an
update, the {product-title} CVO uses the release image for that update to
upgrade your cluster. The release artifacts are hosted in Quay as container
images.
////
By accepting automatic updates, you can automatically
keep your cluster up to date with the most recent compatible components.
////
To allow the {product-title} update service to provide only compatible updates,
a release verification pipeline exists to drive automation. Each release
@@ -27,8 +31,18 @@ architectures as well as other component packages. After the pipeline confirms
the suitability of a release, the {product-title} update service can apply the
update to your cluster or notify you that it is available.
////
The interaction between the registry and the {product-title} update service is different during
bootstrap and continuous update modes. When you bootstrap the initial
infrastructure, the Cluster Version Operator finds
the fully qualified image name for the shortname of the images that it needs to
apply to the server during installation. It looks at the imagestream that it needs
to apply and renders it to disk. It calls bootkube and waits for a temporary minimal control
plane to come up and load the Cluster Version Operator.
////
During continuous update mode, two controllers run. One continuously updates
the payload manifests, applies them to the cluster, and outputs the status of
the controlled rollout of the Operators, whether they are available, upgrading,
or failed. The second controller polls the {product-title} update service to
determine if updates are available.
determine if updates are available.