1
0
mirror of https://github.com/openshift/openshift-docs.git synced 2026-02-05 21:46:22 +01:00

Create conneted to disconnected cluster document

This commit is contained in:
Michael Burke
2021-09-14 15:49:27 -04:00
parent a424f4bbf9
commit ac9f6b9dff
10 changed files with 627 additions and 0 deletions

View File

@@ -459,6 +459,8 @@ Topics:
File: preparing-for-users
- Name: Configuring alert notifications
File: configuring-alert-notifications
- Name: Converting a connected cluster to a disconnected cluster
File: connected-to-disconnected
---
Name: Updating clusters
Dir: updating

BIN
images/create-nodes-okd.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

View File

@@ -0,0 +1,190 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/connected-to-disconnected.adoc
[id="connected-to-disconnected-config-registry_{context}"]
= Configuring the cluster for the mirror registry
After creating and mirroring the images to the mirror registry, you must modify your cluster so that pods can pull images from the mirror registry.
You must:
* Add the mirror registry credentials to the global pull secret.
* Add the mirror registry server certificate to the cluster.
* Create an `ImageContentSourcePolicy` custom resource (ICSP), which associates the mirror registry with the source registry.
. Add mirror registry credential to the cluster global pull-secret:
+
[source,terminal]
----
$ oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson=<pull_secret_location> <1>
----
<1> Provide the path to the new pull secret file.
+
For example:
+
[source,terminal]
----
$ oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson=.mirrorsecretconfigjson
----
. Add the CA-signed mirror registry server certificate to the nodes in the cluster:
.. Create a config map that includes the server certificate for the mirror registry
+
[source,terminal]
----
$ oc create configmap <config_map_name> --from-file=<mirror_address_host>..<port>=$path/ca.crt -n openshift-config
----
+
For example:
+
[source,terminal]
----
S oc create configmap registry-config --from-file=mirror.registry.com..443=/root/certs/ca-chain.cert.pem -n openshift-config
----
.. Use the config map to update the `image.config.openshift.io/cluster` custom resource (CR). {product-title} applies the changes to this CR to all nodes in the cluster:
+
[source,terminal]
----
$ oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"<config_map_name>"}}}' --type=merge
----
+
For example:
+
[source,terminal]
----
$ oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"registry-config"}}}' --type=merge
----
. Create an ICSP to redirect container pull requests from the online registries to the mirror registry:
.. Create the `ImageContentSourcePolicy` custom resource:
+
[source,yaml]
----
apiVersion: operator.openshift.io/v1alpha1
kind: ImageContentSourcePolicy
metadata:
name: mirror-ocp
spec:
repositoryDigestMirrors:
- mirrors:
- mirror.registry.com:443/ocp/release <1>
source: quay.io/openshift-release-dev/ocp-release <2>
- mirrors:
- mirror.registry.com:443/ocp/release
source: quay.io/openshift-release-dev/ocp-v4.0-art-dev
----
<1> Specifies the name of the mirror image registry and repository.
<2> Specifies the online registry and repository containing the content that is mirrored.
.. Create the ICSP object:
+
[source,terminal]
----
$ oc create -f registryrepomirror.yaml
----
+
.Example output
[source,terminal]
----
imagecontentsourcepolicy.operator.openshift.io/mirror-ocp created
----
+
{product-title} applies the changes to this CR to all nodes in the cluster.
. Verify that the credentials, CA, and ICSP for mirror registry were added:
.. Log into a node:
+
[source,terminal]
----
$ oc debug node/<node_name>
----
.. Set `/host` as the root directory within the debug shell:
+
[source,terminal]
----
sh-4.4# chroot /host
----
.. Check the `config.json` file for the credentials:
+
[source,terminal]
----
sh-4.4# cat /var/lib/kubelet/config.json
----
+
.Example output
[source,terminal]
----
{"auths":{"brew.registry.redhat.io":{"xx=="},"brewregistry.stage.redhat.io":{"auth":"xxx=="},"mirror.registry.com:443":{"auth":"xx="}}} <1>
----
<1> Ensure that the mirror registry and credentials are present.
.. Change to the `certs.d` directory
+
[source,terminal]
----
sh-4.4# cd /etc/docker/certs.d/
----
.. List the certificates in the `certs.d` directory:
+
[source,terminal]
----
sh-4.4# ls
----
+
.Example output
----
image-registry.openshift-image-registry.svc.cluster.local:5000
image-registry.openshift-image-registry.svc:5000
mirror.registry.com:443 <1>
----
<1> Ensure that the mirror registry is in the list.
.. Check that the ICSP added the mirror registry to the `registries.conf` file:
+
[source,terminal]
----
sh-4.4# cat /etc/containers/registries.conf
----
+
.Example output
+
[source,terminal]
----
unqualified-search-registries = ["registry.access.redhat.com", "docker.io"]
[[registry]]
prefix = ""
location = "quay.io/openshift-release-dev/ocp-release"
mirror-by-digest-only = true
[[registry.mirror]]
location = "mirror.registry.com:443/ocp/release"
[[registry]]
prefix = ""
location = "quay.io/openshift-release-dev/ocp-v4.0-art-dev"
mirror-by-digest-only = true
[[registry.mirror]]
location = "mirror.registry.com:443/ocp/release"
----
+
The `registry.mirror` parameters indicate that the mirror registry is searched before the original registry.
.. Exit the node.
+
[source,terminal]
----
sh-4.4# exit
----

View File

@@ -0,0 +1,11 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/connected-to-disconnected.adoc
[id="connected-to-disconnected-disconnect_{context}"]
= Disconnect the cluster from the network
After mirroring all the required repositories and configuring your cluster to work as a disconnected cluster, you can disconnect the cluster from the network.

View File

@@ -0,0 +1,110 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/connected-to-disconnected.adoc
[id="connected-to-disconnected-mirror-images_{context}"]
= Mirroring the images
After the cluster is properly configured, you can mirror the images from your external repositories to the mirror repository.
.Procedure
. Mirror the Operator Lifecycle Manager (OLM) images:
// copied from olm-mirroring-catalog.adoc
+
[source,terminal]
----
$ oc adm catalog mirror registry.redhat.io/redhat/redhat-operator-index:v{product-version} <mirror_registry>:<port>/olm -a <reg_creds>
----
+
--
where:
`product-version`:: Specifies the tag that corresponds to the version of {product-title} to install, such as `4.8`.
`mirror_registry`:: Specifies the target registry and namespace to mirror the Operator content to, where `<namespace>` is any existing namespace on the registry.
`reg_creds`:: Specifies the location of your modified `.dockerconfigjson` file.
--
+
For example:
+
[source,terminal]
----
$ oc adm catalog mirror registry.redhat.io/redhat/redhat-operator-index:v4.8 mirror.registry.com:443/olm -a ./.dockerconfigjson --index-filter-by-os='.*'
----
. Mirror the content for any other Red Hat-provided Operator:
+
[source,terminal]
----
$ oc adm catalog mirror <index_image> <mirror_registry>:<port>/<namespace> -a <reg_creds>
----
+
--
where:
`index_image`:: Specifies the index image for the catalog you want to mirror. For example, this might be a pruned index image that you created previously, or one of the source index images for the default catalogs, such as `{index-image-pullspec}`.
`mirror_registry`:: Specifies the target registry and namespace to mirror the Operator content to, where `<namespace>` is any existing namespace on the registry.
`reg_creds`:: Optional: Specifies the location of your registry credentials file, if required.
--
+
For example:
+
[source,terminal]
----
$ oc adm catalog mirror registry.redhat.io/redhat/community-operator-index:v4.8 mirror.registry.com:443/olm -a ./.dockerconfigjson --index-filter-by-os='.*'
----
. Mirror the {product-title} image repository:
+
[source,terminal]
----
$ oc adm release mirror -a .dockerconfigjson --from=quay.io/openshift-release-dev/ocp-release:v<product-version>-<architecture> --to=<local_registry>/<local_repository> --to-release-image=<local_registry>/<local_repository>:v<product-version>-<architecture>
----
+
--
where:
`product-version`:: Specifies the tag that corresponds to the version of {product-title} to install, such as `4.8.15-x86_64`.
`architecture`:: Specifies the type of architecture for your server, such as `x86_64`.
`local_registry`:: Specifies the registry domain name for your mirror repository.
`local_repository`:: Specifies the name of the repository to create in your registry, such as `ocp4/openshift4`.
--
+
For example:
+
[source,terminal]
----
$ oc adm release mirror -a .dockerconfigjson --from=quay.io/openshift-release-dev/ocp-release:4.8.15-x86_64 --to=mirror.registry.com:443/ocp/release --to-release-image=mirror.registry.com:443/ocp/release:4.8.15-x86_64
----
+
.Example output
+
[source,terminal]
+
----
info: Mirroring 109 images to mirror.registry.com/ocp/release ...
mirror.registry.com:443/
ocp/release
manifests:
sha256:086224cadce475029065a0efc5244923f43fb9bb3bb47637e0aaf1f32b9cad47 -> 4.8.15-x86_64-thanos
sha256:0a214f12737cb1cfbec473cc301aa2c289d4837224c9603e99d1e90fc00328db -> 4.8.15-x86_64-kuryr-controller
sha256:0cf5fd36ac4b95f9de506623b902118a90ff17a07b663aad5d57c425ca44038c -> 4.8.15-x86_64-pod
sha256:0d1c356c26d6e5945a488ab2b050b75a8b838fc948a75c0fa13a9084974680cb -> 4.8.15-x86_64-kube-client-agent
…..
sha256:66e37d2532607e6c91eedf23b9600b4db904ce68e92b43c43d5b417ca6c8e63c mirror.registry.com:443/ocp/release:4.5.41-multus-admission-controller
sha256:d36efdbf8d5b2cbc4dcdbd64297107d88a31ef6b0ec4a39695915c10db4973f1 mirror.registry.com:443/ocp/release:4.5.41-cluster-kube-scheduler-operator
sha256:bd1baa5c8239b23ecdf76819ddb63cd1cd6091119fecdbf1a0db1fb3760321a2 mirror.registry.com:443/ocp/release:4.5.41-aws-machine-controllers
info: Mirroring completed in 2.02s (0B/s)
Success
Update image: mirror.registry.com:443/ocp/release:4.5.41-x86_64
Mirror prefix: mirror.registry.com:443/ocp/release
----
. Mirror any other registries, as needed:
+
[source,terminal]
----
$ oc image mirror <online_registry>/my/image:latest <mirror_registry>
----

View File

@@ -0,0 +1,74 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/connected-to-disconnected.adoc
[id="connected-to-disconnected-prepare-mirror_{context}"]
= Preparing the cluster for mirroring
Before disconnecting your cluster, you must mirror, or copy, the images to a mirror registry that is reachable by every node in your disconnected cluster. In order to mirror the images, you must prepare your cluster by:
* Adding the mirror registry certificates to the list of trusted CAs on your host.
* Creating a `.dockerconfigjson` file that contains your image pull secret, which is from the `cloud.openshift.com` token.
.Procedure
. Configuring credentials that allow image mirroring:
.. Add the CA certificate for the mirror registry, in the simple PEM or DER file formats, to the list of trusted CAs. For example:
+
[source,terminal]
----
$ cp </path/to/cert.crt> /usr/share/pki/ca-trust-source/anchors/
----
+
--
where::
+
`</path/to/cert.crt>`:: Specifies the path to the certificate on your local file system.
--
.. Update the CA trust. For example, in Linux:
+
[source,terminal]
----
$ update-ca-trust
----
.. Extract the `.dockerconfigjson` file from the global pull secret:
+
[source,terminal]
----
$ oc extract secret/pull-secret -n openshift-config --confirm --to=.
----
+
.Example output
[source,terminal]
----
.dockerconfigjson
----
.. Edit the `.dockerconfigjson` file to add your mirror registry and authentication credentials and save it as a new file:
// copied from olm-accessing-images-private-registries
+
[source,terminal]
----
{"auths":{"<local_registry>": {"auth": "<credentials>","email": "you@example.com"}}},"<registry>:<port>/<namespace>/":{"auth":"<token>"}}}
----
+
where:
+
`<local_registry>`:: Specifies the registry domain name, and optionally the port, that your mirror registry uses to serve content.
`auth`:: Specifies the base64-encoded user name and password for your mirror registry.
`<registry>:<port>/<namespace>`:: Specifies the mirror registry details.
`<token>`:: Specifies the base64-encoded `username:password` for your mirror registry.
+
For example:
+
[source,terminal]
----
$ {"auths":{"cloud.openshift.com":{"auth":"b3BlbnNoaWZ0Y3UjhGOVZPT0lOMEFaUjdPUzRGTA==","email":"user@example.com"},
"quay.io":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGOVZPT0lOMEFaUGSTd4VGVGVUjdPUzRGTA==","email":"user@example.com"},
"registry.connect.redhat.com"{"auth":"NTE3MTMwNDB8dWhjLTFEZlN3VHkxOSTd4VGVGVU1MdTpleUpoYkdjaUailA==","email":"user@example.com"},
"registry.redhat.io":{"auth":"NTE3MTMwNDB8dWhjLTFEZlN3VH3BGSTd4VGVGVU1MdTpleUpoYkdjaU9fZw==","email":"user@example.com"},
"registry.svc.ci.openshift.org":{"auth":"dXNlcjpyWjAwWVFjSEJiT2RKVW1pSmg4dW92dGp1SXRxQ3RGN1pwajJhN1ZXeTRV"},"my-registry:5000/my-namespace/":{"auth":"dXNlcm5hbWU6cGFzc3dvcmQ="}}}
----

View File

@@ -0,0 +1,44 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/connected-to-disconnected.adoc
[id="connected-to-disconnected-restore-insights_{context}"]
= Restoring a degraded Insights Operator
Disconnecting the cluster from the network necessarily causes the cluster to lose the Internet connection. The Insights Operator becomes degraded because it requires access to link:https://console.redhat.com[Red Hat Insights].
This topic describes how to recover from a degraded Insights Operator.
.Procedure
. Edit your `.dockerconfigjson` file to remove the `cloud.openshift.com` entry, for example:
+
[source,terminal]
----
"cloud.openshift.com":{"auth":"<hash>","email":"user@example.com"}
----
. Save the file.
. Update the cluster secret with the edited `.dockerconfigjson` file:
+
[source,terminal]
----
$ oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson=./.dockerconfigjson
----
. Verify that the Insights Operator is no longer degraded:
+
[source,terminal]
----
$ oc get co insights
----
+
.Example output
[source,terminal]
----
NAME VERSION AVAILABLE PROGRESSING DEGRADED SINCE
insights 4.5.41 True False False 3d
----

View File

@@ -0,0 +1,78 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/connected-to-disconnected.adoc
[id="connected-to-disconnected-restore_{context}"]
= Restoring the network
If you want to reconnect a disconnected cluster and pull images from online registries, delete the cluster's ImageContentSourcePolicy (ICSP) objects. Without the ICSP, pull requests to external registries are no longer redirected to the mirror registry.
.Procedure
. View the ICSP objects in your cluster:
+
[source,terminal]
----
$ oc get imagecontentsourcepolicy
----
+
.Example output
[source,terminal]
----
NAME AGE
mirror-ocp 6d20h
ocp4-index-0 6d18h
qe45-index-0 6d15h
----
. Delete all the ICSP objects you created when disconnecting your cluster:
+
[source,terminal]
----
$ oc delete imagecontentsourcepolicy <icsp_name> <icsp_name> <icsp_name>
----
+
For example:
+
[source,terminal]
----
$ oc delete imagecontentsourcepolicy mirror-ocp ocp4-index-0 qe45-index-0
----
+
.Example output
[source,terminal]
----
imagecontentsourcepolicy.operator.openshift.io "mirror-ocp" deleted
imagecontentsourcepolicy.operator.openshift.io "ocp4-index-0" deleted
imagecontentsourcepolicy.operator.openshift.io "qe45-index-0" deleted
----
. Wait for all the nodes to restart and return to the READY status and verify that the `registries.conf` file is pointing to the original registries and not the mirror registries:
.. Log into a node:
+
[source,terminal]
----
$ oc debug node/<node_name>
----
.. Set `/host` as the root directory within the debug shell:
+
[source,terminal]
----
sh-4.4# chroot /host
----
.. Examine the `registries.conf` file:
+
[source,terminal]
----
sh-4.4# cat /etc/containers/registries.conf
----
+
.Example output
[source,terminal]
----
unqualified-search-registries = ["registry.access.redhat.com", "docker.io"] <1>
----
<1> The `registry` and `registry.mirror` entries created by the ICSPs you deleted are removed.

View File

@@ -0,0 +1,52 @@
// Module included in the following assemblies:
//
// * post_installation_configuration/connected-to-disconnected.adoc
[id="connected-to-disconnected-verify_{context}"]
= Ensure applications continue to work
Before disconnecting the cluster from the network, ensure that your cluster is working as expected and all of your applications are working as expected.
.Procedure
Use the following commands to check the status of your cluster:
* Ensure your pods are running:
+
[source,terminal]
----
$ oc get pods --all-namespaces
----
+
.Example output
[source,terinal]
----
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system apiserver-watcher-ci-ln-47ltxtb-f76d1-mrffg-master-0 1/1 Running 0 39m
kube-system apiserver-watcher-ci-ln-47ltxtb-f76d1-mrffg-master-1 1/1 Running 0 39m
kube-system apiserver-watcher-ci-ln-47ltxtb-f76d1-mrffg-master-2 1/1 Running 0 39m
openshift-apiserver-operator openshift-apiserver-operator-79c7c646fd-5rvr5 1/1 Running 3 45m
openshift-apiserver apiserver-b944c4645-q694g 2/2 Running 0 29m
openshift-apiserver apiserver-b944c4645-shdxb 2/2 Running 0 31m
openshift-apiserver apiserver-b944c4645-x7rf2 2/2 Running 0 33m
...
----
* Ensure your nodes are in the READY status:
+
[source,terminal]
----
$ oc get nodes
----
+
.Example output
[source,terminal]
----
NAME STATUS ROLES AGE VERSION
ci-ln-47ltxtb-f76d1-mrffg-master-0 Ready master 42m v1.21.1+a620f50
ci-ln-47ltxtb-f76d1-mrffg-master-1 Ready master 42m v1.21.1+a620f50
ci-ln-47ltxtb-f76d1-mrffg-master-2 Ready master 42m v1.21.1+a620f50
ci-ln-47ltxtb-f76d1-mrffg-worker-a-gsxbz Ready worker 35m v1.21.1+a620f50
ci-ln-47ltxtb-f76d1-mrffg-worker-b-5qqdx Ready worker 35m v1.21.1+a620f50
ci-ln-47ltxtb-f76d1-mrffg-worker-c-rjkpq Ready worker 34m v1.21.1+a620f50
----

View File

@@ -0,0 +1,66 @@
[id="connected-to-disconnected"]
= Converting a connected cluster to a disconnected cluster
include::modules/common-attributes.adoc[]
:context: connected-to-disconnected
toc::[]
There might be some scenarios where you need to convert your {product-title} cluster from a connected cluster to a disconnected cluster.
A disconnected cluster, also known as a restricted cluster, does not have an active connection to the internet. As such, you must mirror the contents of your registries and installation media. You can create this mirror registry on a host that can access both the internet and your closed network, or copy images to a device that you can move across network boundaries.
This topic describes the general process for converting an existing, connected cluster into a disconnected cluster.
:FeatureName: The process for converting a cluster from connected to disconnected
include::modules/technology-preview.adoc[leveloffset=+1]
include::modules/installation-about-mirror-registry.adoc[leveloffset=+1]
[id="prerequisites_connected-to-disconnected"]
== Prerequisites
* The `oc` client is installed.
* A running cluster.
* An installed mirror registry, which is a container image registry that supports link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2] in the location that will host the {product-title} cluster, such as one of the following registries:
+
--
** link:https://www.redhat.com/en/technologies/cloud-computing/quay[Red Hat Quay]
** link:https://jfrog.com/artifactory/[JFrog Artifactory]
** link:https://www.sonatype.com/products/repository-oss?topnav=true[Sonatype Nexus Repository]
** link:https://goharbor.io/[Harbor]
--
+
If you have an subscription to Red Hat Quay, see the documentation on deploying Red Hat Quay link:https://access.redhat.com/documentation/en-us/red_hat_quay/3.5/html/deploy_red_hat_quay_for_proof-of-concept_non-production_purposes/[for proof-of-concept purposes] or link:https://access.redhat.com/documentation/en-us/red_hat_quay/3.5/html/deploy_red_hat_quay_on_openshift_with_the_quay_operator/[by using the Quay Operator].
* The mirror repository must be configured to share images. For example, a Red Hat Quay repository requires link:https://access.redhat.com/documentation/en-us/red_hat_quay/3.5/html/use_red_hat_quay/user-org-intro#org-create[Organizations] in order to share images.
* Access to the internet to obtain the necessary container images.
include::modules/connected-to-disconnected-prepare-mirror.adoc[leveloffset=+1]
include::modules/connected-to-disconnected-mirror-images.adoc[leveloffset=+1]
.Additional information
* For more information about mirroring Operator catalogs, see xref:../operators/admin/olm-restricted-networks.adoc#olm-mirror-catalog_olm-restricted-networks[Mirroring an Operator catalog].
* For more information about the `oc adm catalog mirror` command, see the xref:../cli_reference/openshift_cli/administrator-cli-commands.adoc#oc-adm-catalog-mirror[OpenShift CLI administrator command reference].
include::modules/connected-to-disconnected-config-registry.adoc[leveloffset=+1]
include::modules/connected-to-disconnected-verify.adoc[leveloffset=+1]
include::modules/connected-to-disconnected-disconnect.adoc[leveloffset=+1]
[NOTE]
====
The Insights Operator is degraded when the cluster loses its Internet connection. You can avoid this problem by temporarily xref:../support/remote_health_monitoring/opting-out-of-remote-health-reporting.adoc[disabling the Insights Operator] until you can restore it.
====
include::modules/connected-to-disconnected-restore-insights.adoc[leveloffset=+1]
include::modules/connected-to-disconnected-restore.adoc[leveloffset=+1]