From 0fa00845ffb6a0951392f1cb9589e2dd6785daf8 Mon Sep 17 00:00:00 2001 From: Alex Dellapenta Date: Thu, 23 Sep 2021 15:22:29 -0600 Subject: [PATCH] Mv OLM mirroring module to Installing guide & reorg --- ...talling-mirroring-installation-images.adoc | 22 +- modules/olm-catalogsource.adoc | 2 +- modules/olm-creating-catalog-from-index.adoc | 13 +- modules/olm-mirroring-catalog-airgapped.adoc | 87 +++++++ modules/olm-mirroring-catalog-colocated.adoc | 65 ++++++ modules/olm-mirroring-catalog-extracting.adoc | 10 + modules/olm-mirroring-catalog-icsp.adoc | 26 +++ modules/olm-mirroring-catalog-manifests.adoc | 48 ++++ modules/olm-mirroring-catalog-post.adoc | 8 + modules/olm-mirroring-catalog-prereqs.adoc | 29 +++ modules/olm-mirroring-catalog.adoc | 214 +----------------- operators/admin/olm-restricted-networks.adoc | 10 +- .../olm/olm-understanding-olm.adoc | 2 +- .../preparing-for-users.adoc | 15 ++ 14 files changed, 329 insertions(+), 222 deletions(-) create mode 100644 modules/olm-mirroring-catalog-airgapped.adoc create mode 100644 modules/olm-mirroring-catalog-colocated.adoc create mode 100644 modules/olm-mirroring-catalog-extracting.adoc create mode 100644 modules/olm-mirroring-catalog-icsp.adoc create mode 100644 modules/olm-mirroring-catalog-manifests.adoc create mode 100644 modules/olm-mirroring-catalog-post.adoc create mode 100644 modules/olm-mirroring-catalog-prereqs.adoc diff --git a/installing/installing-mirroring-installation-images.adoc b/installing/installing-mirroring-installation-images.adoc index dead94989b..68017c297d 100644 --- a/installing/installing-mirroring-installation-images.adoc +++ b/installing/installing-mirroring-installation-images.adoc @@ -73,12 +73,32 @@ In a disconnected environment, you must take additional steps after you install include::modules/installation-images-samples-disconnected-mirroring-assist.adoc[leveloffset=+2] +include::modules/olm-mirroring-catalog.adoc[leveloffset=+1] +.Additional resources +* xref:../operators/admin/olm-restricted-networks.adoc#olm-restricted-networks[Using Operator Lifecycle Manager on restricted networks] + +include::modules/olm-mirroring-catalog-prereqs.adoc[leveloffset=+2] +include::modules/olm-mirroring-catalog-extracting.adoc[leveloffset=+2] +include::modules/olm-mirroring-catalog-colocated.adoc[leveloffset=+3] +.Additional resources +* xref:../operators/operator_sdk/osdk-generating-csvs.adoc#olm-arch-os-support_osdk-generating-csvs[Architecture and operating system support for Operators] + +include::modules/olm-mirroring-catalog-airgapped.adoc[leveloffset=+3] +.Additional resources +* xref:../operators/operator_sdk/osdk-generating-csvs.adoc#olm-arch-os-support_osdk-generating-csvs[Architecture and operating system support for Operators] + +include::modules/olm-mirroring-catalog-manifests.adoc[leveloffset=+2] +include::modules/olm-mirroring-catalog-post.adoc[leveloffset=+2] +.Additional resources + +* xref:../post_installation_configuration/preparing-for-users.adoc#post-install-mirrored-catalogs[Populating OperatorHub from mirrored Operator catalogs] + +[id="next-steps_installing-mirroring-installation-images"] == Next steps //* TODO need to add the registry secret to the machines, which is different -* xref:../operators/admin/olm-restricted-networks.adoc#olm-mirror-catalog_olm-restricted-networks[Mirror] the OperatorHub images for the Operators that you want to install in your cluster. * Install a cluster on infrastructure that you provision in your restricted network, such as on xref:../installing/installing_vsphere/installing-restricted-networks-vsphere.adoc#installing-restricted-networks-vsphere[VMware vSphere], xref:../installing/installing_bare_metal/installing-restricted-networks-bare-metal.adoc#installing-restricted-networks-bare-metal[bare metal], or xref:../installing/installing_aws/installing-restricted-networks-aws.adoc#installing-restricted-networks-aws[Amazon Web Services]. diff --git a/modules/olm-catalogsource.adoc b/modules/olm-catalogsource.adoc index 5315670ef0..16d6c4ba92 100644 --- a/modules/olm-catalogsource.adoc +++ b/modules/olm-catalogsource.adoc @@ -12,7 +12,7 @@ endif::[] [id="olm-catalogsource_{context}"] = Catalog source -A _catalog source_ represents a store of metadata, typically by referencing an _index image_ stored in a container registry. Operator Lifecycle Manager (OLM) queries catalog sources to discover and install Operators and their dependencies. The OperatorHub in the {product-title} web console also displays the Operators provided by catalog sources. +A _catalog source_ represents a store of metadata, typically by referencing an _index image_ stored in a container registry. Operator Lifecycle Manager (OLM) queries catalog sources to discover and install Operators and their dependencies. OperatorHub in the {product-title} web console also displays the Operators provided by catalog sources. [TIP] ==== diff --git a/modules/olm-creating-catalog-from-index.adoc b/modules/olm-creating-catalog-from-index.adoc index 4c12946295..ca295d0f2c 100644 --- a/modules/olm-creating-catalog-from-index.adoc +++ b/modules/olm-creating-catalog-from-index.adoc @@ -1,5 +1,6 @@ // Module included in the following assemblies: // +// * post_installation_configuration/preparing-for-users.adoc // * operators/admin/olm-restricted-networks.adoc // * operators/admin/managing-custom-catalogs.adoc @@ -13,14 +14,17 @@ ifndef::openshift-origin[] :tag: v{product-version} :namespace: openshift-marketplace endif::[] +ifeval::["{context}" == "post-install-preparing-for-users"] +:olm-restricted-networks: +endif::[] ifeval::["{context}" == "olm-restricted-networks"] :olm-restricted-networks: endif::[] [id="olm-creating-catalog-from-index_{context}"] -= Creating a catalog from an index image += Adding a catalog source to a cluster -You can create an Operator catalog from an index image and apply it to an {product-title} cluster for use with Operator Lifecycle Manager (OLM). +Adding a catalog source to an {product-title} cluster enables the discovery and installation of Operators for users. Cluster administrators can create a `CatalogSource` object that references an index image. OperatorHub uses catalog sources to populate the user interface. .Prerequisites @@ -30,7 +34,7 @@ You can create an Operator catalog from an index image and apply it to an {produ . Create a `CatalogSource` object that references your index image. ifdef::olm-restricted-networks[] -If you used the `oc adm catalog mirror` command to mirror your catalog to a target registry, you can use the generated `catalogSource.yaml` file as a starting point. +If you used the `oc adm catalog mirror` command to mirror your catalog to a target registry, you can use the generated `catalogSource.yaml` file in your manifests directory as a starting point. endif::[] .. Modify the following to your specifications and save it as a `catalogSource.yaml` file: @@ -134,6 +138,9 @@ You can now install the Operators from the *OperatorHub* page on your {product-t :!index-image: :!tag: :!namespace: +ifeval::["{context}" == "post-install-preparing-for-users"] +:!olm-restricted-networks: +endif::[] ifeval::["{context}" == "olm-restricted-networks"] :!olm-restricted-networks: endif::[] diff --git a/modules/olm-mirroring-catalog-airgapped.adoc b/modules/olm-mirroring-catalog-airgapped.adoc new file mode 100644 index 0000000000..ca576ddfbf --- /dev/null +++ b/modules/olm-mirroring-catalog-airgapped.adoc @@ -0,0 +1,87 @@ +// Module included in the following assemblies: +// +// * installing/installing-mirroring-installation-images.adoc + +ifdef::openshift-origin[] +:index-image-pullspec: quay.io/operatorhubio/catalog:latest +endif::[] +ifndef::openshift-origin[] +:index-image-pullspec: registry.redhat.io/redhat/redhat-operator-index:v{product-version} +endif::[] + +[id="olm-mirror-catalog-airgapped_{context}"] += Mirroring catalog contents to airgapped registries + +If your mirror registry is on a completely disconnected, or airgapped, host, take the following actions. + +.Procedure + +. Run the following command on your workstation with unrestricted network access to mirror the content to local files: ++ +[source,terminal] +---- +$ oc adm catalog mirror \ + \ <.> + file:///local/index \ <.> + -a ${REG_CREDS} \ <.> + --insecure \ <.> + --index-filter-by-os='/' <.> +---- +<.> Specify the index image for the catalog that you want to mirror. For example, this might be a pruned index image that you created previously, or one of the source index images for the default catalogs, such as `{index-image-pullspec}`. +<.> Specify the content to mirror to local files in your current directory. +<.> Optional: If required, specify the location of your registry credentials file. +<.> Optional: If you do not want to configure trust for the target registry, add the `--insecure` flag. +<.> Optional: Specify which platform and architecture of the index image to select when multiple variants are available. Images are specified as `'/[/]'`. This does not apply to images referenced by the index. Valid values are `linux/amd64`, `linux/ppc64le`, `linux/s390x`, and `.*` ++ +.Example output +[source,terminal] +---- +... +info: Mirroring completed in 5.93s (5.915MB/s) +wrote mirroring manifests to manifests-my-index-1614985528 <1> + +To upload local images to a registry, run: + + oc adm catalog mirror file://local/index/myrepo/my-index:v1 REGISTRY/REPOSITORY <2> +---- +<1> Record the manifests directory name that is generated. This directory is referenced in subsequent procedures. +<2> Record the expanded `file://` path that is based on your provided index image. This path is referenced in a subsequent step. ++ +This command creates a `v2/` directory in your current directory. + +. Copy the `v2/` directory to removable media. + +. Physically remove the media and attach it to a host in the disconnected environment that has access to the mirror registry. + +. If your mirror registry requires authentication, run the following command on your host in the disconnected environment to log in to the registry: ++ +[source,terminal] +---- +$ podman login +---- + +. Run the following command from the parent directory containing the `v2/` directory to upload the images from local files to the mirror registry: ++ +[source,terminal] +---- +$ oc adm catalog mirror \ + file://local/index//: \ <.> + :/ \ <.> + -a ${REG_CREDS} \ <.> + --insecure \ <.> + --index-filter-by-os='/' <.> +---- +<.> Specify the `file://` path from the previous command output. +<.> Specify the target registry and namespace to mirror the Operator contents to, where `` is any existing namespace on the registry. For example, you might create an `olm-mirror` namespace to push all mirrored content to. +<.> Optional: If required, specify the location of your registry credentials file. +<.> Optional: If you do not want to configure trust for the target registry, add the `--insecure` flag. +<.> Optional: Specify which platform and architecture of the index image to select when multiple variants are available. Images are specified as `'/[/]'`. This does not apply to images referenced by the index. Valid values are `linux/amd64`, `linux/ppc64le`, `linux/s390x`, and `.*` ++ +[NOTE] +==== +Red Hat Quay does not support nested repositories. As a result, running the `oc adm catalog mirror` command will fail with a `401` unauthorized error. As a workaround, you can use the `--max-components=2` option when running the `oc adm catalog mirror` command to disable the creation of nested repositories. For more information on this workaround, see the link:https://access.redhat.com/solutions/5440741[Unauthorized error thrown while using catalog mirror command with Quay registry] Knowledgebase Solution. +==== + +After you mirror the catalog, you can continue with the remainder of your cluster installation. After your cluster installation has finished successfully, you must specify the manifests directory from this procedure to create the `ImageContentSourcePolicy` and `CatalogSource` objects. These objects are required to enable installation of Operators from OperatorHub. + +:!index-image-pullspec: diff --git a/modules/olm-mirroring-catalog-colocated.adoc b/modules/olm-mirroring-catalog-colocated.adoc new file mode 100644 index 0000000000..e1f8d89737 --- /dev/null +++ b/modules/olm-mirroring-catalog-colocated.adoc @@ -0,0 +1,65 @@ +// Module included in the following assemblies: +// +// * installing/installing-mirroring-installation-images.adoc + +ifdef::openshift-origin[] +:index-image-pullspec: quay.io/operatorhubio/catalog:latest +:index-image: catalog +endif::[] +ifndef::openshift-origin[] +:index-image-pullspec: registry.redhat.io/redhat/redhat-operator-index:v{product-version} +:index-image: redhat-operator-index +endif::[] + +[id="olm-mirror-catalog-colocated_{context}"] += Mirroring catalog contents to registries on the same network + +If your mirror registry is co-located on the same network as your workstation with unrestricted network access, take the following actions on your workstation. + +.Procedure + +. If your mirror registry requires authentication, run the following command to log in to the registry: ++ +[source,terminal] +---- +$ podman login +---- + +. Run the following command to extract and mirror the content to the mirror registry: ++ +[source,terminal] +---- +$ oc adm catalog mirror \ + \ <.> + :/ \ <.> + [-a ${REG_CREDS}] \ <.> + [--insecure] \ <.> + [--index-filter-by-os='/'] \ <.> + [--manifests-only] <.> +---- +<1> Specify the index image for the catalog that you want to mirror. For example, this might be a pruned index image that you created previously, or one of the source index images for the default catalogs, such as `{index-image-pullspec}`. +<2> Specify the target registry and namespace to mirror the Operator contents to, where `` is any existing namespace on the registry. For example, you might create an `olm-mirror` namespace to push all mirrored content to. +<3> Optional: If required, specify the location of your registry credentials file. +<4> Optional: If you do not want to configure trust for the target registry, add the `--insecure` flag. +<5> Optional: Specify which platform and architecture of the index image to select when multiple variants are available. Images are passed as `'/[/]'`. This does not apply to images referenced by the index. Valid values are `linux/amd64`, `linux/ppc64le`, `linux/s390x`, and `.*` +<6> Optional: Generate only the manifests required for mirroring, and do not actually mirror the image content to a registry. This option can be useful for reviewing what will be mirrored, and it allows you to make any changes to the mapping list if you require only a subset of packages. You can then use the `mapping.txt` file with the `oc image mirror` command to mirror the modified list of images in a later step. This flag is intended for only advanced selective mirroring of content from the catalog; the `opm index prune` command, if you used it previously to prune the index image, is suitable for most catalog management use cases. ++ +.Example output +[source,terminal,subs="attributes+"] +---- +src image has index label for database path: /database/index.db +using database path mapping: /database/index.db:/tmp/153048078 +wrote database to /tmp/153048078 <1> +... +wrote mirroring manifests to manifests-{index-image}-1614211642 <2> +---- +<1> Directory for the temporary `index.db` database generated by the command. +<2> Record the manifests directory name that is generated. This directory is referenced in subsequent procedures. ++ +[NOTE] +==== +Red Hat Quay does not support nested repositories. As a result, running the `oc adm catalog mirror` command will fail with a `401` unauthorized error. As a workaround, you can use the `--max-components=2` option when running the `oc adm catalog mirror` command to disable the creation of nested repositories. For more information on this workaround, see the link:https://access.redhat.com/solutions/5440741[Unauthorized error thrown while using catalog mirror command with Quay registry] Knowledgebase Solution. +==== + +:!index-image-pullspec: +:!index-image: diff --git a/modules/olm-mirroring-catalog-extracting.adoc b/modules/olm-mirroring-catalog-extracting.adoc new file mode 100644 index 0000000000..7ae19c43ff --- /dev/null +++ b/modules/olm-mirroring-catalog-extracting.adoc @@ -0,0 +1,10 @@ +// Module included in the following assemblies: +// +// * installing/installing-mirroring-installation-images.adoc + +[id="olm-mirror-catalog-extracting_{context}"] += Extracting and mirroring catalog contents + +The `oc adm catalog mirror` command extracts the contents of an index image to generate the manifests required for mirroring. The default behavior of the command generates manifests, then automatically mirrors all of the image content from the index image, as well as the index image itself, to your mirror registry. + +Alternatively, if your mirror registry is on a completely disconnected, or _airgapped_, host, you can first mirror the content to removable media, move the media to the disconnected environment, then mirror the content from the media to the registry. diff --git a/modules/olm-mirroring-catalog-icsp.adoc b/modules/olm-mirroring-catalog-icsp.adoc new file mode 100644 index 0000000000..0adafef744 --- /dev/null +++ b/modules/olm-mirroring-catalog-icsp.adoc @@ -0,0 +1,26 @@ +// Module included in the following assemblies: +// +// * post_installation_configuration/preparing-for-users.adoc + +[id="olm-mirror-catalog-icsp_{context}"] += Creating the ImageContentSourcePolicy object + +After mirroring Operator catalog content to your mirror registry, create the required `ImageContentSourcePolicy` (ICSP) object. The ICSP object configures nodes to translate between the image references stored in Operator manifests and the mirrored registry. + +.Procedure + +* On a host with access to the disconnected cluster, create the ICSP by running the following command to specify the `imageContentSourcePolicy.yaml` file in your manifests directory: ++ +[source,terminal,subs="attributes+"] +---- +$ oc create -f /imageContentSourcePolicy.yaml +---- ++ +where `` is the path to the manifests directory for your mirrored content. ++ +[NOTE] +==== +Applying the ICSP causes all worker nodes in the cluster to restart. You must wait for this reboot process to finish cycling through each of your worker nodes before proceeding. +==== + +You can now create a `CatalogSource` object to reference your mirrored index image and Operator content. diff --git a/modules/olm-mirroring-catalog-manifests.adoc b/modules/olm-mirroring-catalog-manifests.adoc new file mode 100644 index 0000000000..b744a4e260 --- /dev/null +++ b/modules/olm-mirroring-catalog-manifests.adoc @@ -0,0 +1,48 @@ +// Module included in the following assemblies: +// +// * installing/installing-mirroring-installation-images.adoc + +[id="olm-mirror-catalog-manifests_{context}"] += Generated manifests + +After mirroring Operator catalog content to your mirror registry, a manifests directory is generated in your current directory. + +If you mirrored content to a registry on the same network, the directory name takes the following pattern: + +[source,text] +---- +manifests-- +---- + +If you mirrored content to a registry on a disconnected host in the previous section, the directory name takes the following pattern: + +[source,text] +---- +manifests-index//- +---- + +[NOTE] +==== +The manifests directory name is referenced in subsequent procedures. +==== + +The manifests directory contains the following files, some of which might require further modification: + +* The `catalogSource.yaml` file is a basic definition for a `CatalogSource` object that is pre-populated with your index image tag and other relevant metadata. This file can be used as is or modified to add the catalog source to your cluster. ++ +[IMPORTANT] +==== +If you mirrored the content to local files, you must modify your `catalogSource.yaml` file to remove any backslash (`/`) characters from the `metadata.name` field. Otherwise, when you attempt to create the object, it fails with an "invalid resource name" error. +==== +* The `imageContentSourcePolicy.yaml` file defines an `ImageContentSourcePolicy` object that can configure nodes to translate between the image references stored in Operator manifests and the mirrored registry. ++ +[NOTE] +==== +If your cluster uses an `ImageContentSourcePolicy` object to configure repository mirroring, you can use only global pull secrets for mirrored registries. You cannot add a pull secret to a project. +==== +* The `mapping.txt` file contains all of the source images and where to map them in the target registry. This file is compatible with the `oc image mirror` command and can be used to further customize the mirroring configuration. ++ +[IMPORTANT] +==== +If you used the `--manifests-only` flag during the mirroring process and want to further trim the subset of packages to mirror, see the steps in the link:https://docs.openshift.com/container-platform/4.7/operators/admin/olm-managing-custom-catalogs.html#olm-mirroring-package-manifest-catalog_olm-managing-custom-catalogs[Mirroring a package manifest format catalog image] procedure of the {product-title} 4.7 documentation about modifying your `mapping.txt` file and using the file with the `oc image mirror` command. +==== diff --git a/modules/olm-mirroring-catalog-post.adoc b/modules/olm-mirroring-catalog-post.adoc new file mode 100644 index 0000000000..cddffe6791 --- /dev/null +++ b/modules/olm-mirroring-catalog-post.adoc @@ -0,0 +1,8 @@ +// Module included in the following assemblies: +// +// * installing/installing-mirroring-installation-images.adoc + +[id="olm-mirror-catalog-post_{context}"] += Post-installation requirements + +After you mirror the catalog, you can continue with the remainder of your cluster installation. After your cluster installation has finished successfully, you must specify the manifests directory from this procedure to create the `ImageContentSourcePolicy` and `CatalogSource` objects. These objects are required to populate and enable installation of Operators from OperatorHub. diff --git a/modules/olm-mirroring-catalog-prereqs.adoc b/modules/olm-mirroring-catalog-prereqs.adoc new file mode 100644 index 0000000000..625b6a487f --- /dev/null +++ b/modules/olm-mirroring-catalog-prereqs.adoc @@ -0,0 +1,29 @@ +// Module included in the following assemblies: +// +// * installing/installing-mirroring-installation-images.adoc + +[id="olm-mirror-catalog-prerequisites_{context}"] += Prerequisites + +Mirroring Operator catalogs for use with disconnected clusters has the following prerequisites: + +* Workstation with unrestricted network access. +* `podman` version 1.9.3 or later. +ifndef::openshift-origin[] +* If you want to mirror a Red Hat-provided catalog, run the following command on your workstation with unrestricted network access to authenticate with `registry.redhat.io`: ++ +[source,terminal] +---- +$ podman login registry.redhat.io +---- +endif::[] +* Access to mirror registry that supports +link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2]. +* Decide which namespace on your mirror registry you will use to store the mirrored Operator content. For example, you might create an `olm-mirror` namespace. +* If your mirror registry does not have internet access, connect removable media to your workstation with unrestricted network access. +* If you are working with private registries, set the `REG_CREDS` environment variable to the file path of your registry credentials for use in later steps. For example, for the `podman` CLI: ++ +[source,terminal] +---- +$ REG_CREDS=${XDG_RUNTIME_DIR}/containers/auth.json +---- diff --git a/modules/olm-mirroring-catalog.adoc b/modules/olm-mirroring-catalog.adoc index 71c42e5c58..ed6c11ac67 100644 --- a/modules/olm-mirroring-catalog.adoc +++ b/modules/olm-mirroring-catalog.adoc @@ -1,218 +1,10 @@ // Module included in the following assemblies: // -// * operators/admin/olm-restricted-networks.adoc - -ifdef::openshift-origin[] -:index-image-pullspec: quay.io/operatorhubio/catalog:latest -:index-image: catalog -:example-registry: example.com -endif::[] -ifndef::openshift-origin[] -:index-image-pullspec: registry.redhat.io/redhat/redhat-operator-index:v{product-version} -:index-image: redhat-operator-index -:example-registry: registry.redhat.io -endif::[] +// * installing/installing-mirroring-installation-images.adoc [id="olm-mirror-catalog_{context}"] -= Mirroring an Operator catalog += Mirroring Operator catalogs for use with disconnected clusters -You can mirror the Operator content of a Red Hat-provided catalog, or a custom catalog, into a container image registry using the `oc adm catalog mirror` command. The target registry must support link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2]. For a cluster on a restricted network, this registry can be one that the cluster has network access to, such as a mirror registry created during a restricted network cluster installation. +You can mirror the Operator contents of a Red Hat-provided catalog, or a custom catalog, into a container image registry using the `oc adm catalog mirror` command. The target registry must support link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2]. For a cluster on a restricted network, this registry can be one that the cluster has network access to, such as a mirror registry created during a restricted network cluster installation. The `oc adm catalog mirror` command also automatically mirrors the index image that is specified during the mirroring process, whether it be a Red Hat-provided index image or your own custom-built index image, to the target registry. You can then use the mirrored index image to create a catalog source that allows Operator Lifecycle Manager (OLM) to load the mirrored catalog onto your {product-title} cluster. - -.Prerequisites - -* Workstation with unrestricted network access. -* `podman` version 1.9.3 or later. -* Access to mirror registry that supports -link:https://docs.docker.com/registry/spec/manifest-v2-2/[Docker v2-2]. -* Decide which namespace on your mirror registry you will use to store the mirrored Operator content. For example, you might create an `olm-mirror` namespace. -* If your mirror registry does not have internet access, connect removable media to your workstation with unrestricted network access. -* If you are working with private registries, set the `REG_CREDS` environment variable to the file path of your registry credentials for use in later steps. For example, for the `podman` CLI: -+ -[source,terminal] ----- -$ REG_CREDS=${XDG_RUNTIME_DIR}/containers/auth.json ----- - -.Procedure - -ifndef::openshift-origin[] -. If you want to mirror a Red Hat-provided catalog, run the following command on your workstation with unrestricted network access to authenticate with `registry.redhat.io`: -+ -[source,terminal] ----- -$ podman login registry.redhat.io ----- -endif::[] - -. The `oc adm catalog mirror` command extracts the contents of an index image to generate the manifests required for mirroring. The default behavior of the command generates manifests, then automatically mirrors all of the image content from the index image, as well as the index image itself, to your mirror registry. Alternatively, if your mirror registry is on a completely disconnected, or _airgapped_, host, you can first mirror the content to removable media, move the media to the disconnected environment, then mirror the content from the media to the registry. - -** *Option A: If your mirror registry is on the same network* as your workstation with unrestricted network access, take the following actions on your workstation: - -.. If your mirror registry requires authentication, run the following command to log in to the registry: -+ -[source,terminal] ----- -$ podman login ----- - -.. Run the following command to mirror the content: -+ -[source,terminal] ----- -$ oc adm catalog mirror \ - \//<1> - :/ \//<2> - [-a ${REG_CREDS}] \//<3> - [--insecure] \//<4> - [--index-filter-by-os='/'] \//<5> - [--manifests-only] <6> ----- -<1> Specify the index image for the catalog you want to mirror. For example, this might be a pruned index image that you created previously, or one of the source index images for the default catalogs, such as `{index-image-pullspec}`. -<2> Specify the target registry and namespace to mirror the Operator content to, where `` is any existing namespace on the registry. For example, you might create an `olm-mirror` namespace to push all mirrored content to. -<3> Optional: If required, specify the location of your registry credentials file. -<4> Optional: If you do not want to configure trust for the target registry, add the `--insecure` flag. -<5> Optional: Specify which platform and architecture of the index image to select when multiple variants are available. Images are passed as `'/[/]'`. This does not apply to images referenced by the index. Valid values are `linux/amd64`, `linux/ppc64le`, `linux/s390x`, and `.*` -<6> Optional: Generate only the manifests required for mirroring, and do not actually mirror the image content to a registry. This option can be useful for reviewing what will be mirrored, and it allows you to make any changes to the mapping list if you require only a subset of packages. You can then use the `mapping.txt` file with the `oc image mirror` command to mirror the modified list of images in a later step. This flag is intended for only advanced selective mirroring of content from the catalog; the `opm index prune` command, if you used it previously to prune the index image, is suitable for most catalog management use cases. -+ -.Example output -[source,terminal,subs="attributes+"] ----- -src image has index label for database path: /database/index.db -using database path mapping: /database/index.db:/tmp/153048078 -wrote database to /tmp/153048078 <1> -... -wrote mirroring manifests to manifests-{index-image}-1614211642 <2> ----- -<1> Directory for the temporary `index.db` database generated by the command. -<2> Be sure to record the manifests directory name that is generated. This directory name is used in a later step. -+ -[NOTE] -==== -Red Hat Quay does not support nested repositories. As a result, running the `oc adm catalog mirror` command will fail with a `401` unauthorized error. As a workaround, you can use the `--max-components=2` option when running the `oc adm catalog mirror` command to disable the creation of nested repositories. For more information on this workaround, see the link:https://access.redhat.com/solutions/5440741[Unauthorized error thrown while using catalog mirror command with Quay registry] Knowledgebase Solution article. -==== - -** *Option B: If your mirror registry is on a disconnected host,* take the following actions. - -.. Run the following command on your workstation with unrestricted network access to mirror the content to local files: -+ -[source,terminal] ----- -$ oc adm catalog mirror \ - \//<1> - file:///local/index \//<2> - [-a ${REG_CREDS}] \ - [--insecure] \ - [--index-filter-by-os='/'] ----- -<1> Specify the index image for the catalog you want to mirror. For example, this might be a pruned index image that you created previously, or one of the source index images for the default catalogs, such as `{index-image-pullspec}`. -<2> Mirrors content to local files in your current directory. -+ -.Example output -[source,terminal] ----- -... -info: Mirroring completed in 5.93s (5.915MB/s) -wrote mirroring manifests to manifests-my-index-1614985528 <1> - -To upload local images to a registry, run: - - oc adm catalog mirror file://local/index/myrepo/my-index:v1 REGISTRY/REPOSITORY <2> ----- -<1> Be sure to record the manifests directory name that is generated. This directory name is used in a later step. -<2> Record the expanded `file://` path that based on your provided index image. This path is used in a later step. - -.. Copy the `v2/` directory that is generated in your current directory to removable media. - -.. Physically remove the media and attach it to a host in the disconnected environment that has access to the mirror registry. - -.. If your mirror registry requires authentication, run the following command on your host in the disconnected environment to log in to the registry: -+ -[source,terminal] ----- -$ podman login ----- - -.. Run the following command from the parent directory containing the `v2/` directory to upload the images from local files to the mirror registry: -+ -[source,terminal] ----- -$ oc adm catalog mirror \ - file://local/index//: \//<1> - :/ \//<2> - [-a ${REG_CREDS}] \ - [--insecure] \ - [--index-filter-by-os='/'] ----- -<1> Specify the `file://` path from the previous command output. -<2> Specify the target registry and namespace to mirror the Operator content to, where `` is any existing namespace on the registry. For example, you might create an `olm-mirror` namespace to push all mirrored content to. -+ -[NOTE] -==== -Red Hat Quay does not support nested repositories. As a result, running the `oc adm catalog mirror` command will fail with a `401` unauthorized error. As a workaround, you can use the `--max-components=2` option when running the `oc adm catalog mirror` command to disable the creation of nested repositories. For more information on this workaround, see the link:https://access.redhat.com/solutions/5440741[Unauthorized error thrown while using catalog mirror command with Quay registry] Knowledgebase Solution article. -==== - -. After mirroring the content to your registry, inspect the manifests directory that is generated in your current directory. -+ -[NOTE] -==== -The manifests directory name is used in a later step. -==== -+ -If you mirrored content to a registry on the same network in the previous step, the directory name takes the following form: -+ -[source,text] ----- -manifests-- ----- -+ -If you mirrored content to a registry on a disconnected host in the previous step, the directory name takes the following form: -+ -[source,text] ----- -manifests-index//- ----- -+ -The manifests directory contains the following files, some of which might require further modification: -+ --- -* The `catalogSource.yaml` file is a basic definition for a `CatalogSource` object that is pre-populated with your index image tag and other relevant metadata. This file can be used as is or modified to add the catalog source to your cluster. -+ -[IMPORTANT] -==== -If you mirrored the content to local files, you must modify your `catalogSource.yaml` file to remove any backslash (`/`) characters from the `metadata.name` field. Otherwise, when you attempt to create the object, it fails with an "invalid resource name" error. -==== -* The `imageContentSourcePolicy.yaml` file defines an `ImageContentSourcePolicy` object that can configure nodes to translate between the image references stored in Operator manifests and the mirrored registry. -+ -[NOTE] -==== -If your cluster uses an `ImageContentSourcePolicy` object to configure repository mirroring, you can use only global pull secrets for mirrored registries. You cannot add a pull secret to a project. -==== -* The `mapping.txt` file contains all of the source images and where to map them in the target registry. This file is compatible with the `oc image mirror` command and can be used to further customize the mirroring configuration. -+ -[IMPORTANT] -==== -If you used the `--manifests-only` flag during the mirroring process and want to further trim the subset of packages to be mirrored, see the steps in the link:https://docs.openshift.com/container-platform/4.7/operators/admin/olm-managing-custom-catalogs.html#olm-mirroring-package-manifest-catalog_olm-managing-custom-catalogs[Mirroring a package manifest format catalog image] procedure of the {product-title} 4.7 documentation about modifying your `mapping.txt` file and using the file with the `oc image mirror` command. After following those further actions, you can continue this procedure. -==== --- - -. On a host with access to the disconnected cluster, create the `ImageContentSourcePolicy` (ICSP) object by running the following command to specify the `imageContentSourcePolicy.yaml` file in your manifests directory: -+ -[source,terminal,subs="attributes+"] ----- -$ oc create -f /imageContentSourcePolicy.yaml ----- -+ -where `` is the path to the manifests directory for your mirrored content. -+ -[NOTE] -==== -Applying the ICSP causes all worker nodes in the cluster to restart. You must wait for this reboot process to finish cycling through each of your worker nodes before proceeding. -==== - -You can now create a `CatalogSource` object to reference your mirrored index image and Operator content. - -:!index-image-pullspec: -:!index-image: -:!example-registry: diff --git a/operators/admin/olm-restricted-networks.adoc b/operators/admin/olm-restricted-networks.adoc index fed2aa3ae9..8928518ec9 100644 --- a/operators/admin/olm-restricted-networks.adoc +++ b/operators/admin/olm-restricted-networks.adoc @@ -49,17 +49,17 @@ If you are using OLM in a restricted network on IBM Z, you must have at least 12 include::modules/olm-restricted-networks-configuring-operatorhub.adoc[leveloffset=+1] include::modules/olm-pruning-index-image.adoc[leveloffset=+1] -include::modules/olm-mirroring-catalog.adoc[leveloffset=+1] -.Additional resources -* xref:../../installing/installing-mirroring-installation-images.adoc#installing-mirroring-installation-images[Mirroring images for a disconnected installation] -* xref:../../operators/operator_sdk/osdk-generating-csvs.adoc#olm-arch-os-support_osdk-generating-csvs[Architecture and operating system support for Operators] +[id="olm-mirror-catalog_olm-restricted-networks"] +== Mirroring an Operator catalog + +For instructions about mirroring Operator catalogs for use with disconnected clusters, see xref:../../installing/installing-mirroring-installation-images.adoc#olm-mirroring-catalog_installing-mirroring-installation-images[Installing -> Mirroring images for a disconnected installation]. include::modules/olm-creating-catalog-from-index.adoc[leveloffset=+1] .Additional resources * If your index image is hosted on a private registry and requires authentication, see xref:../../operators/admin/olm-managing-custom-catalogs.adoc#olm-accessing-images-private-registries_olm-managing-custom-catalogs[Accessing images for Operators from private registries]. -* If you want your catalogs to be able to automatically update their index image version after cluster upgrades by using Kubernetes version-based image tags, see xref:../../operators/understanding/olm/olm-understanding-olm.html#olm-catalogsource-image-template_olm-understanding-olm[Image template for custom catalog sources]. +* If you want your catalogs to be able to automatically update their index image version after cluster upgrades by using Kubernetes version-based image tags, see xref:../../operators/understanding/olm/olm-understanding-olm.adoc#olm-catalogsource-image-template_olm-understanding-olm[Image template for custom catalog sources]. include::modules/olm-updating-index-image.adoc[leveloffset=+1] .Additional resources diff --git a/operators/understanding/olm/olm-understanding-olm.adoc b/operators/understanding/olm/olm-understanding-olm.adoc index ab13247217..51ec917dfd 100644 --- a/operators/understanding/olm/olm-understanding-olm.adoc +++ b/operators/understanding/olm/olm-understanding-olm.adoc @@ -15,7 +15,7 @@ include::modules/olm-catalogsource.adoc[leveloffset=+2] * xref:../../../operators/understanding/olm-understanding-operatorhub.adoc#olm-understanding-operatorhub[Understanding OperatorHub] * xref:../../../operators/understanding/olm-rh-catalogs.adoc#olm-rh-catalogs[Red Hat-provided Operator catalogs] -* xref:../../../operators/admin/olm-managing-custom-catalogs.adoc#olm-creating-catalog-from-index_olm-managing-custom-catalogs[Creating a catalog from an index image] +* xref:../../../operators/admin/olm-managing-custom-catalogs.adoc#olm-creating-catalog-from-index_olm-managing-custom-catalogs[Adding a catalog source to a cluster] * xref:../../../operators/understanding/olm/olm-understanding-dependency-resolution.adoc#olm-dependency-catalog-priority_olm-understanding-dependency-resolution[Catalog priority] * xref:../../../operators/admin/olm-status.adoc#olm-cs-status-cli_olm-status[Viewing Operator catalog source status by using the CLI] diff --git a/post_installation_configuration/preparing-for-users.adoc b/post_installation_configuration/preparing-for-users.adoc index 42e36d7629..b374e9f1fd 100644 --- a/post_installation_configuration/preparing-for-users.adoc +++ b/post_installation_configuration/preparing-for-users.adoc @@ -131,6 +131,21 @@ include::modules/images-configuration-cas.adoc[leveloffset=+3] include::modules/images-configuration-registry-mirror.adoc[leveloffset=+3] +[id="post-install-mirrored-catalogs"] +== Populating OperatorHub from mirrored Operator catalogs + +If you mirrored Operator catalogs for use with disconnected clusters, you can populate OperatorHub with the Operators from your mirrored catalogs. You can use the generated manifests from the mirroring process to create the required `ImageContentSourcePolicy` and `CatalogSource` objects. + +[id="prerequisites_post-install-mirrored-catalogs"] +=== Prerequistes + +* xref:../installing/installing-mirroring-installation-images.html#olm-mirror-catalog_installing-mirroring-installation-images[Mirroring Operator catalogs for use with disconnected clusters] + +include::modules/olm-mirroring-catalog-icsp.adoc[leveloffset=+2] +include::modules/olm-creating-catalog-from-index.adoc[leveloffset=+2] +* If your index image is hosted on a private registry and requires authentication, see xref:../operators/admin/olm-managing-custom-catalogs.adoc#olm-accessing-images-private-registries_olm-managing-custom-catalogs[Accessing images for Operators from private registries]. +* If you want your catalogs to be able to automatically update their index image version after cluster upgrades by using Kubernetes version-based image tags, see xref:../operators/understanding/olm/olm-understanding-olm.adoc#olm-catalogsource-image-template_olm-understanding-olm[Image template for custom catalog sources]. + include::modules/olm-installing-operators-from-operatorhub.adoc[leveloffset=+1] ifdef::openshift-origin[] include::modules/olm-installing-operators-from-operatorhub-configure.adoc[leveloffset=+2]