mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
OSDOCS-12745: Rotating OIDC bound service account signer keys
This commit is contained in:
Jeana Routh
@@ -320,8 +320,6 @@ endif::openshift-origin[]
|
||||
:vmw-first: VMware vSphere
|
||||
:vmw-full: VMware vSphere
|
||||
:vmw-short: vSphere
|
||||
|
||||
|
||||
//Token-based auth products
|
||||
//AWS Security Token Service
|
||||
:sts-first: Security Token Service (STS)
|
||||
@@ -333,8 +331,6 @@ endif::openshift-origin[]
|
||||
//Google Cloud Platform Workload Identity
|
||||
:gcp-wid-first: Google Cloud Platform Workload Identity
|
||||
:gcp-wid-short: GCP Workload Identity
|
||||
|
||||
|
||||
// Cluster API terminology
|
||||
// Cluster CAPI Operator
|
||||
:cluster-capi-operator: Cluster CAPI Operator
|
||||
@@ -365,9 +361,8 @@ endif::openshift-origin[]
|
||||
// Cluster API Provider VMware vSphere
|
||||
:cap-vsphere-first: Cluster API Provider VMware vSphere
|
||||
:cap-vsphere-short: Cluster API Provider vSphere
|
||||
|
||||
// Hosted control planes related attributes
|
||||
:hcp-capital: Hosted control planes
|
||||
:hcp: hosted control planes
|
||||
:mce: multicluster engine for Kubernetes Operator
|
||||
:mce-short: multicluster engine Operator
|
||||
:mce-short: multicluster engine Operator
|
||||
@@ -14,7 +14,7 @@ include::modules/alternatives-to-storing-admin-secrets-in-kube-system.adoc[level
|
||||
.Additional resources
|
||||
|
||||
* xref:../../authentication/managing_cloud_provider_credentials/cco-mode-gcp-workload-identity.adoc#cco-mode-gcp-workload-identity[Using manual mode with GCP Workload Identity]
|
||||
* xref:../../post_installation_configuration/cluster-tasks.adoc#post-install-rotate-remove-cloud-creds[Rotating or removing cloud provider credentials]
|
||||
* xref:../../post_installation_configuration/cluster-tasks.adoc#post-install-remove-cloud-creds[Removing cloud provider credentials]
|
||||
|
||||
For a detailed description of all available CCO credential modes and their supported platforms, see xref:../../authentication/managing_cloud_provider_credentials/about-cloud-credential-operator.adoc#about-cloud-credential-operator[About the Cloud Credential Operator].
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
//Postinstall and update content
|
||||
// * post_installation_configuration/cluster-tasks.adoc
|
||||
// * post_installation_configuration/changing-cloud-credentials-configuration.adoc
|
||||
// * updating/preparing_for_updates/preparing-manual-creds-update.adoc
|
||||
//
|
||||
//Platforms that must use `ccoctl` and update content
|
||||
@@ -169,155 +169,8 @@ ifdef::update[]
|
||||
* You have extracted the `CredentialsRequest` custom resources (CRs) from the {product-title} release image and ensured that a namespace that matches the text in the `spec.secretRef.namespace` field exists in the cluster.
|
||||
endif::update[]
|
||||
|
||||
//AWS permissions needed when running ccoctl during install (I think we can omit from upgrade, since they already have an appropriate AWS account if they are upgrading).
|
||||
ifdef::aws-sts[]
|
||||
* You have created an AWS account for the `ccoctl` utility to use with the following permissions:
|
||||
+
|
||||
.Required AWS permissions
|
||||
[%collapsible]
|
||||
====
|
||||
**Required `iam` permissions**
|
||||
|
||||
* `iam:CreateOpenIDConnectProvider`
|
||||
* `iam:CreateRole`
|
||||
* `iam:DeleteOpenIDConnectProvider`
|
||||
* `iam:DeleteRole`
|
||||
* `iam:DeleteRolePolicy`
|
||||
* `iam:GetOpenIDConnectProvider`
|
||||
* `iam:GetRole`
|
||||
* `iam:GetUser`
|
||||
* `iam:ListOpenIDConnectProviders`
|
||||
* `iam:ListRolePolicies`
|
||||
* `iam:ListRoles`
|
||||
* `iam:PutRolePolicy`
|
||||
* `iam:TagOpenIDConnectProvider`
|
||||
* `iam:TagRole`
|
||||
|
||||
**Required `s3` permissions**
|
||||
|
||||
* `s3:CreateBucket`
|
||||
* `s3:DeleteBucket`
|
||||
* `s3:DeleteObject`
|
||||
* `s3:GetBucketAcl`
|
||||
* `s3:GetBucketTagging`
|
||||
* `s3:GetObject`
|
||||
* `s3:GetObjectAcl`
|
||||
* `s3:GetObjectTagging`
|
||||
* `s3:ListBucket`
|
||||
* `s3:PutBucketAcl`
|
||||
* `s3:PutBucketPolicy`
|
||||
* `s3:PutBucketPublicAccessBlock`
|
||||
* `s3:PutBucketTagging`
|
||||
* `s3:PutObject`
|
||||
* `s3:PutObjectAcl`
|
||||
* `s3:PutObjectTagging`
|
||||
|
||||
**Required `cloudfront` permissions**
|
||||
|
||||
* `cloudfront:ListCloudFrontOriginAccessIdentities`
|
||||
* `cloudfront:ListDistributions`
|
||||
* `cloudfront:ListTagsForResource`
|
||||
====
|
||||
+
|
||||
If you plan to store the OIDC configuration in a private S3 bucket that is accessed by the IAM identity provider through a public CloudFront distribution URL, the AWS account that runs the `ccoctl` utility requires the following additional permissions:
|
||||
+
|
||||
.Additional permissions for a private S3 bucket with CloudFront
|
||||
[%collapsible]
|
||||
====
|
||||
* `cloudfront:CreateCloudFrontOriginAccessIdentity`
|
||||
* `cloudfront:CreateDistribution`
|
||||
* `cloudfront:DeleteCloudFrontOriginAccessIdentity`
|
||||
* `cloudfront:DeleteDistribution`
|
||||
* `cloudfront:GetCloudFrontOriginAccessIdentity`
|
||||
* `cloudfront:GetCloudFrontOriginAccessIdentityConfig`
|
||||
* `cloudfront:GetDistribution`
|
||||
* `cloudfront:TagResource`
|
||||
* `cloudfront:UpdateDistribution`
|
||||
|
||||
[NOTE]
|
||||
=====
|
||||
These additional permissions support the use of the `--create-private-s3-bucket` option when processing credentials requests with the `ccoctl aws create-all` command.
|
||||
=====
|
||||
====
|
||||
endif::aws-sts[]
|
||||
|
||||
//Azure permissions needed when running ccoctl during install.
|
||||
ifdef::azure-workload-id[]
|
||||
* You have created a global Microsoft Azure account for the `ccoctl` utility to use with the following permissions:
|
||||
+
|
||||
.Required Azure permissions
|
||||
[%collapsible]
|
||||
====
|
||||
* Microsoft.Resources/subscriptions/resourceGroups/read
|
||||
* Microsoft.Resources/subscriptions/resourceGroups/write
|
||||
* Microsoft.Resources/subscriptions/resourceGroups/delete
|
||||
* Microsoft.Authorization/roleAssignments/read
|
||||
* Microsoft.Authorization/roleAssignments/delete
|
||||
* Microsoft.Authorization/roleAssignments/write
|
||||
* Microsoft.Authorization/roleDefinitions/read
|
||||
* Microsoft.Authorization/roleDefinitions/write
|
||||
* Microsoft.Authorization/roleDefinitions/delete
|
||||
* Microsoft.Storage/storageAccounts/listkeys/action
|
||||
* Microsoft.Storage/storageAccounts/delete
|
||||
* Microsoft.Storage/storageAccounts/read
|
||||
* Microsoft.Storage/storageAccounts/write
|
||||
* Microsoft.Storage/storageAccounts/blobServices/containers/write
|
||||
* Microsoft.Storage/storageAccounts/blobServices/containers/delete
|
||||
* Microsoft.Storage/storageAccounts/blobServices/containers/read
|
||||
* Microsoft.ManagedIdentity/userAssignedIdentities/delete
|
||||
* Microsoft.ManagedIdentity/userAssignedIdentities/read
|
||||
* Microsoft.ManagedIdentity/userAssignedIdentities/write
|
||||
* Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/read
|
||||
* Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/write
|
||||
* Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/delete
|
||||
* Microsoft.Storage/register/action
|
||||
* Microsoft.ManagedIdentity/register/action
|
||||
====
|
||||
endif::azure-workload-id[]
|
||||
|
||||
//GCP permissions needed when running ccoctl during install.
|
||||
ifdef::google-cloud-platform[]
|
||||
* You have added one of the following authentication options to the GCP account that the installation program uses:
|
||||
|
||||
** The **IAM Workload Identity Pool Admin** role.
|
||||
|
||||
** The following granular permissions:
|
||||
+
|
||||
.Required GCP permissions
|
||||
[%collapsible]
|
||||
====
|
||||
* compute.projects.get
|
||||
* iam.googleapis.com/workloadIdentityPoolProviders.create
|
||||
* iam.googleapis.com/workloadIdentityPoolProviders.get
|
||||
* iam.googleapis.com/workloadIdentityPools.create
|
||||
* iam.googleapis.com/workloadIdentityPools.delete
|
||||
* iam.googleapis.com/workloadIdentityPools.get
|
||||
* iam.googleapis.com/workloadIdentityPools.undelete
|
||||
* iam.roles.create
|
||||
* iam.roles.delete
|
||||
* iam.roles.list
|
||||
* iam.roles.undelete
|
||||
* iam.roles.update
|
||||
* iam.serviceAccounts.create
|
||||
* iam.serviceAccounts.delete
|
||||
* iam.serviceAccounts.getIamPolicy
|
||||
* iam.serviceAccounts.list
|
||||
* iam.serviceAccounts.setIamPolicy
|
||||
* iam.workloadIdentityPoolProviders.get
|
||||
* iam.workloadIdentityPools.delete
|
||||
* resourcemanager.projects.get
|
||||
* resourcemanager.projects.getIamPolicy
|
||||
* resourcemanager.projects.setIamPolicy
|
||||
* storage.buckets.create
|
||||
* storage.buckets.delete
|
||||
* storage.buckets.get
|
||||
* storage.buckets.getIamPolicy
|
||||
* storage.buckets.setIamPolicy
|
||||
* storage.objects.create
|
||||
* storage.objects.delete
|
||||
* storage.objects.list
|
||||
====
|
||||
endif::google-cloud-platform[]
|
||||
//Permissions requirements (per platform, for install and key rotation)
|
||||
include::snippets/ccoctl-provider-permissions-requirements.adoc[]
|
||||
|
||||
.Procedure
|
||||
|
||||
|
||||
@@ -1,27 +1,27 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * post_installation_configuration/cluster-tasks.adoc
|
||||
// * post_installation_configuration/changing-cloud-credentials-configuration.adoc
|
||||
|
||||
:_mod-docs-content-type: PROCEDURE
|
||||
[id="refreshing-service-ids-ibm-cloud_{context}"]
|
||||
= Rotating API keys
|
||||
= Rotating {ibm-cloud-title} credentials
|
||||
|
||||
You can rotate API keys for your existing service IDs and update the corresponding secrets.
|
||||
|
||||
.Prerequisites
|
||||
|
||||
* You have configured the `ccoctl` binary.
|
||||
* You have configured the `ccoctl` utility.
|
||||
* You have existing service IDs in a live {product-title} cluster installed.
|
||||
|
||||
.Procedure
|
||||
|
||||
* Use the `ccoctl` utility to rotate your API keys for the service IDs and update the secrets:
|
||||
* Use the `ccoctl` utility to rotate your API keys for the service IDs and update the secrets by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ ccoctl <provider_name> refresh-keys \ <1>
|
||||
--kubeconfig <openshift_kubeconfig_file> \ <2>
|
||||
--credentials-requests-dir <path_to_credential_requests_directory> \ <3>
|
||||
$ ccoctl <provider_name> refresh-keys \// <1>
|
||||
--kubeconfig <openshift_kubeconfig_file> \// <2>
|
||||
--credentials-requests-dir <path_to_credential_requests_directory> \// <3>
|
||||
--name <name> <4>
|
||||
----
|
||||
<1> The name of the provider. For example: `ibmcloud` or `powervs`.
|
||||
@@ -34,4 +34,4 @@ $ ccoctl <provider_name> refresh-keys \ <1>
|
||||
====
|
||||
If your cluster uses Technology Preview features that are enabled by the `TechPreviewNoUpgrade` feature set, you must include the `--enable-tech-preview` parameter.
|
||||
====
|
||||
--
|
||||
--
|
||||
409
modules/rotating-bound-service-keys.adoc
Normal file
409
modules/rotating-bound-service-keys.adoc
Normal file
@@ -0,0 +1,409 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * post_installation_configuration/changing-cloud-credentials-configuration.adoc
|
||||
|
||||
ifeval::["{context}" == "key-rotation-aws"]
|
||||
:rotate-aws:
|
||||
endif::[]
|
||||
ifeval::["{context}" == "key-rotation-gcp"]
|
||||
:rotate-gcp:
|
||||
endif::[]
|
||||
ifeval::["{context}" == "key-rotation-azure"]
|
||||
:rotate-azure:
|
||||
endif::[]
|
||||
|
||||
:_mod-docs-content-type: PROCEDURE
|
||||
[id="rotating-bound-service-keys_{context}"]
|
||||
ifdef::rotate-aws[= Rotating {aws-short} OIDC bound service account signer keys]
|
||||
ifdef::rotate-gcp[= Rotating {gcp-short} OIDC bound service account signer keys]
|
||||
ifdef::rotate-azure[= Rotating {azure-short} OIDC bound service account signer keys]
|
||||
|
||||
If the Cloud Credential Operator (CCO) for your {product-title} cluster
|
||||
ifdef::rotate-aws[on {aws-first}]
|
||||
ifdef::rotate-gcp[on {gcp-first}]
|
||||
ifdef::rotate-azure[on {azure-first}]
|
||||
is configured to operate in manual mode with
|
||||
ifdef::rotate-aws[{sts-short},]
|
||||
ifdef::rotate-gcp[{gcp-wid-short},]
|
||||
ifdef::rotate-azure[{entra-short},]
|
||||
you can rotate the bound service account signer key.
|
||||
|
||||
To rotate the key, you delete the existing key on your cluster, which causes the Kubernetes API server to create a new key.
|
||||
To reduce authentication failures during this process, you must immediately add the new public key to the existing issuer file.
|
||||
After the cluster is using the new key for authentication, you can remove any remaining keys.
|
||||
|
||||
//Modified version of the disclaimer from enabling Azure WID on an existing cluster, since there are similar concerns:
|
||||
[IMPORTANT]
|
||||
====
|
||||
The process to rotate OIDC bound service account signer keys is disruptive and takes a significant amount of time.
|
||||
Some steps are time-sensitive.
|
||||
Before proceeding, observe the following considerations:
|
||||
|
||||
* Read the following steps and ensure that you understand and accept the time requirement.
|
||||
The exact time requirement varies depending on the individual cluster, but it is likely to require at least one hour.
|
||||
|
||||
* To reduce the risk of authentication failures, ensure that you understand and prepare for the time-sensitive steps.
|
||||
|
||||
* During this process, you must refresh all service accounts and restart all pods on the cluster.
|
||||
These actions are disruptive to workloads.
|
||||
To mitigate this impact, you can temporarily halt these services and then redeploy them when the cluster is ready.
|
||||
====
|
||||
|
||||
.Prerequisites
|
||||
|
||||
* You have access to the {oc-first} as a user with the `cluster-admin` role.
|
||||
//Permissions requirements (per platform, for install and key rotation)
|
||||
include::snippets/ccoctl-provider-permissions-requirements.adoc[]
|
||||
* You have configured the `ccoctl` utility.
|
||||
* Your cluster is in a stable state.
|
||||
You can confirm that the cluster is stable by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc adm wait-for-stable-cluster --minimum-stable-period=5s
|
||||
----
|
||||
|
||||
.Procedure
|
||||
|
||||
. Configure the following environment variables:
|
||||
+
|
||||
[source,text]
|
||||
----
|
||||
ifdef::rotate-aws[]
|
||||
INFRA_ID=$(oc get infrastructures cluster -o jsonpath='{.status.infrastructureName}')
|
||||
CLUSTER_NAME=${INFRA_ID%-*} <1>
|
||||
endif::rotate-aws[]
|
||||
ifdef::rotate-gcp[]
|
||||
CURRENT_ISSUER=$(oc get authentication cluster -o jsonpath='{.spec.serviceAccountIssuer}')
|
||||
GCP_BUCKET=$(echo ${CURRENT_ISSUER} | cut -d "/" -f4)
|
||||
endif::rotate-gcp[]
|
||||
ifdef::rotate-azure[]
|
||||
CURRENT_ISSUER=$(oc get authentication cluster -o jsonpath='{.spec.serviceAccountIssuer}')
|
||||
AZURE_STORAGE_ACCOUNT=$(echo ${CURRENT_ISSUER} | cut -d "/" -f3 | cut -d "." -f1)
|
||||
AZURE_STORAGE_CONTAINER=$(echo ${CURRENT_ISSUER} | cut -d "/" -f4)
|
||||
endif::rotate-azure[]
|
||||
----
|
||||
ifdef::rotate-aws[]
|
||||
<1> This value should match the name of the cluster that was specified in the `metadata.name` field of the `install-config.yaml` file during installation.
|
||||
endif::rotate-aws[]
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
Your cluster might differ from this example, and the resource names might not be derived identically from the cluster name.
|
||||
Ensure that you specify the correct corresponding resource names for your cluster.
|
||||
====
|
||||
ifdef::rotate-aws[]
|
||||
** For {aws-short} clusters that store the OIDC configuration in a public S3 bucket, configure the following environment variable:
|
||||
+
|
||||
[source,text]
|
||||
----
|
||||
AWS_BUCKET=$(oc get authentication cluster -o jsonpath={'.spec.serviceAccountIssuer'} | awk -F'://' '{print$2}' |awk -F'.' '{print$1}')
|
||||
----
|
||||
|
||||
** For {aws-short} clusters that store the OIDC configuration in a private S3 bucket that is accessed by the IAM identity provider through a public CloudFront distribution URL, complete the following steps:
|
||||
|
||||
... Extract the public CloudFront distribution URL by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ basename $(oc get authentication cluster -o jsonpath={'.spec.serviceAccountIssuer'} )
|
||||
----
|
||||
+
|
||||
.Example output
|
||||
[source,text]
|
||||
----
|
||||
<subdomain>.cloudfront.net
|
||||
----
|
||||
+
|
||||
where `<subdomain>` is an alphanumeric string.
|
||||
|
||||
... Determine the private S3 bucket name by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ aws cloudfront list-distributions --query "DistributionList.Items[].{DomainName: DomainName, OriginDomainName: Origins.Items[0].DomainName}[?contains(DomainName, '<subdomain>.cloudfront.net')]"
|
||||
----
|
||||
+
|
||||
.Example output
|
||||
[source,text]
|
||||
----
|
||||
[
|
||||
{
|
||||
"DomainName": "<subdomain>.cloudfront.net",
|
||||
"OriginDomainName": "<s3_bucket>.s3.us-east-2.amazonaws.com"
|
||||
}
|
||||
]
|
||||
----
|
||||
+
|
||||
where `<s3_bucket>` is the private S3 bucket name for your cluster.
|
||||
|
||||
... Configure the following environment variable:
|
||||
+
|
||||
[source,text]
|
||||
----
|
||||
AWS_BUCKET=$<s3_bucket>
|
||||
----
|
||||
+
|
||||
where `<s3_bucket>` is the private S3 bucket name for your cluster.
|
||||
endif::rotate-aws[]
|
||||
|
||||
. Create a temporary directory to use and assign it an environment variable by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ TEMPDIR=$(mktemp -d)
|
||||
----
|
||||
|
||||
. To cause the Kubernetes API server to create a new bound service account signing key, you delete the next bound service account signing key.
|
||||
+
|
||||
[IMPORTANT]
|
||||
====
|
||||
After you complete this step, the Kubernetes API server starts to roll out a new key.
|
||||
To reduce the risk of authentication failures, complete the remaining steps as quickly as possible.
|
||||
The remaining steps might be disruptive to workloads.
|
||||
====
|
||||
+
|
||||
When you are ready, delete the next bound service account signing key by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc delete secrets/next-bound-service-account-signing-key \
|
||||
-n openshift-kube-apiserver-operator
|
||||
----
|
||||
|
||||
. Download the public key from the service account signing key secret that the Kubernetes API server created by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc get secret/next-bound-service-account-signing-key \
|
||||
-n openshift-kube-apiserver-operator \
|
||||
-ojsonpath='{ .data.service-account\.pub }' | base64 \
|
||||
-d > ${TEMPDIR}/serviceaccount-signer.public
|
||||
----
|
||||
|
||||
. Use the public key to create a `keys.json` file by running the following command:
|
||||
ifdef::rotate-aws[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ ccoctl aws create-identity-provider \
|
||||
--dry-run \// <1>
|
||||
--output-dir ${TEMPDIR} \
|
||||
--name fake \// <2>
|
||||
--region us-east-1 <3>
|
||||
----
|
||||
<1> The `--dry-run` option outputs files, including the new `keys.json` file, to the disk without making API calls.
|
||||
<2> Because the `--dry-run` option does not make any API calls, some parameters do not require real values.
|
||||
<3> Specify any valid {aws-short} region, such as `us-east-1`.
|
||||
This value does not need to match the region the cluster is in.
|
||||
endif::rotate-aws[]
|
||||
ifdef::rotate-gcp[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ ccoctl gcp create-workload-identity-provider \
|
||||
--dry-run \// <1>
|
||||
--output-dir=${TEMPDIR} \
|
||||
--name fake \// <2>
|
||||
--project fake \
|
||||
--workload-identity-pool fake
|
||||
----
|
||||
<1> The `--dry-run` option outputs files, including the new `keys.json` file, to the disk without making API calls.
|
||||
<2> Because the `--dry-run` option does not make any API calls, some parameters do not require real values.
|
||||
endif::rotate-gcp[]
|
||||
ifdef::rotate-azure[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ ccoctl aws create-identity-provider \// <1>
|
||||
--dry-run \// <2>
|
||||
--output-dir ${TEMPDIR} \
|
||||
--name fake \// <3>
|
||||
--region us-east-1 <4>
|
||||
----
|
||||
<1> The `ccoctl azure` command does not include a `--dry-run` option.
|
||||
To use the `--dry-run` option, you must specify `aws` for an {azure-short} cluster.
|
||||
<2> The `--dry-run` option outputs files, including the new `keys.json` file, to the disk without making API calls.
|
||||
<3> Because the `--dry-run` option does not make any API calls, some parameters do not require real values.
|
||||
<4> Specify any valid {aws-short} region, such as `us-east-1`.
|
||||
This value does not need to match the region the cluster is in.
|
||||
endif::rotate-azure[]
|
||||
|
||||
. Rename the `keys.json` file by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ cp ${TEMPDIR}/<number>-keys.json ${TEMPDIR}/jwks.new.json
|
||||
----
|
||||
+
|
||||
where `<number>` is a two-digit numerical value that varies depending on your environment.
|
||||
|
||||
. Download the existing `keys.json` file from the cloud provider by running the following command:
|
||||
ifdef::rotate-aws[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ aws s3api get-object \
|
||||
--bucket ${AWS_BUCKET} \
|
||||
--key keys.json ${TEMPDIR}/jwks.current.json
|
||||
----
|
||||
endif::rotate-aws[]
|
||||
ifdef::rotate-gcp[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ gcloud storage cp gs://${GCP_BUCKET}/keys.json ${TEMPDIR}/jwks.current.json
|
||||
----
|
||||
endif::rotate-gcp[]
|
||||
ifdef::rotate-azure[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ az storage blob download \
|
||||
--container-name ${AZURE_STORAGE_CONTAINER} \
|
||||
--account-name ${AZURE_STORAGE_ACCOUNT} \
|
||||
--name 'openid/v1/jwks' \
|
||||
-f ${TEMPDIR}/jwks.current.json
|
||||
----
|
||||
endif::rotate-azure[]
|
||||
|
||||
. Combine the two `keys.json` files by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ jq -s '{ keys: map(.keys[])}' ${TEMPDIR}/jwks.current.json ${TEMPDIR}/jwks.new.json > ${TEMPDIR}/jwks.combined.json
|
||||
----
|
||||
|
||||
. To enable authentication for the old and new keys during the rotation, upload the combined `keys.json` file to the cloud provider by running the following command:
|
||||
ifdef::rotate-aws[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ aws s3api put-object \
|
||||
--bucket ${AWS_BUCKET} \
|
||||
--tagging "openshift.io/cloud-credential-operator/${CLUSTER_NAME}=owned" \
|
||||
--key keys.json \
|
||||
--body ${TEMPDIR}/jwks.combined.json
|
||||
----
|
||||
endif::rotate-aws[]
|
||||
ifdef::rotate-gcp[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ gcloud storage cp ${TEMPDIR}/jwks.combined.json gs://${GCP_BUCKET}/keys.json
|
||||
----
|
||||
endif::rotate-gcp[]
|
||||
ifdef::rotate-azure[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ az storage blob upload \
|
||||
--overwrite \
|
||||
--account-name ${AZURE_STORAGE_ACCOUNT} \
|
||||
--container-name ${AZURE_STORAGE_CONTAINER} \
|
||||
--name 'openid/v1/jwks' \
|
||||
-f ${TEMPDIR}/jwks.combined.json
|
||||
----
|
||||
endif::rotate-azure[]
|
||||
|
||||
. Wait for the Kubernetes API server to update and use the new key.
|
||||
You can monitor the update progress by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc adm wait-for-stable-cluster
|
||||
----
|
||||
+
|
||||
This process might take 15 minutes or longer.
|
||||
The following output indicates that the process is complete:
|
||||
+
|
||||
[source,text]
|
||||
----
|
||||
All clusteroperators are stable
|
||||
----
|
||||
|
||||
. To ensure that all pods on the cluster use the new key, you must restart them.
|
||||
+
|
||||
[IMPORTANT]
|
||||
====
|
||||
This step maintains uptime for services that are configured for high availability across multiple nodes, but might cause downtime for any services that are not.
|
||||
====
|
||||
+
|
||||
Restart all of the pods in the cluster by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc adm reboot-machine-config-pool mcp/worker mcp/master
|
||||
----
|
||||
|
||||
. Monitor the restart and update process by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc adm wait-for-node-reboot nodes --all
|
||||
----
|
||||
+
|
||||
This process might take 15 minutes or longer.
|
||||
The following output indicates that the process is complete:
|
||||
+
|
||||
[source,text]
|
||||
----
|
||||
All nodes rebooted
|
||||
----
|
||||
|
||||
. Monitor the update progress by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc adm wait-for-stable-cluster
|
||||
----
|
||||
+
|
||||
This process might take 15 minutes or longer.
|
||||
The following output indicates that the process is complete:
|
||||
+
|
||||
[source,text]
|
||||
----
|
||||
All clusteroperators are stable
|
||||
----
|
||||
|
||||
. Replace the combined `keys.json` file with the updated `keys.json` file on the cloud provider by running the following command:
|
||||
ifdef::rotate-aws[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ aws s3api put-object \
|
||||
--bucket ${AWS_BUCKET} \
|
||||
--tagging "openshift.io/cloud-credential-operator/${CLUSTER_NAME}=owned" \
|
||||
--key keys.json \
|
||||
--body ${TEMPDIR}/jwks.new.json
|
||||
----
|
||||
endif::rotate-aws[]
|
||||
ifdef::rotate-gcp[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ gcloud storage cp ${TEMPDIR}/jwks.new.json gs://${GCP_BUCKET}/keys.json
|
||||
----
|
||||
endif::rotate-gcp[]
|
||||
ifdef::rotate-azure[]
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ az storage blob upload \
|
||||
--overwrite \
|
||||
--account-name ${AZURE_STORAGE_ACCOUNT} \
|
||||
--container-name ${AZURE_STORAGE_CONTAINER} \
|
||||
--name 'openid/v1/jwks' \
|
||||
-f ${TEMPDIR}/jwks.new.json
|
||||
----
|
||||
endif::rotate-azure[]
|
||||
|
||||
ifeval::["{context}" == "key-rotation-aws"]
|
||||
:!rotate-aws:
|
||||
endif::[]
|
||||
ifeval::["{context}" == "key-rotation-gcp"]
|
||||
:!rotate-gcp:
|
||||
endif::[]
|
||||
ifeval::["{context}" == "key-rotation-azure"]
|
||||
:!rotate-azure:
|
||||
endif::[]
|
||||
@@ -10,22 +10,42 @@ For supported configurations, you can change how {product-title} authenticates w
|
||||
|
||||
To determine which cloud credentials strategy your cluster uses, see xref:../authentication/managing_cloud_provider_credentials/about-cloud-credential-operator.adoc#cco-determine-mode_about-cloud-credential-operator[Determining the Cloud Credential Operator mode].
|
||||
|
||||
[id="post-install-rotate-remove-cloud-creds_{context}"]
|
||||
== Rotating or removing cloud provider credentials
|
||||
[id="ccoctl-rotate-cloud-creds_{context}"]
|
||||
== Rotating cloud provider service keys with the Cloud Credential Operator utility
|
||||
|
||||
After installing {product-title}, some organizations require the rotation or removal of the cloud provider credentials that were used during the initial installation.
|
||||
Some organizations require the rotation of the service keys that authenticate the cluster.
|
||||
You can use the Cloud Credential Operator (CCO) utility (`ccoctl`) to update keys for clusters installed on the following cloud providers:
|
||||
|
||||
* xref:../post_installation_configuration/changing-cloud-credentials-configuration.adoc#rotating-bound-service-keys_key-rotation-aws[{aws-first} with {sts-first}]
|
||||
* xref:../post_installation_configuration/changing-cloud-credentials-configuration.adoc#rotating-bound-service-keys_key-rotation-gcp[{gcp-first} with {gcp-wid-short}]
|
||||
* xref:../post_installation_configuration/changing-cloud-credentials-configuration.adoc#rotating-bound-service-keys_key-rotation-azure[{azure-first} with {entra-short}]
|
||||
* xref:../post_installation_configuration/changing-cloud-credentials-configuration.adoc#refreshing-service-ids-ibm-cloud_changing-cloud-credentials-configuration[{ibm-cloud-title}]
|
||||
|
||||
:context: key-rotation-aws
|
||||
//Rotating OIDC bound service account signer keys
|
||||
include::modules/rotating-bound-service-keys.adoc[leveloffset=+2]
|
||||
:!context: key-rotation-aws
|
||||
|
||||
:context: key-rotation-gcp
|
||||
//Rotating OIDC bound service account signer keys
|
||||
include::modules/rotating-bound-service-keys.adoc[leveloffset=+2]
|
||||
:!context: key-rotation-gcp
|
||||
|
||||
:context: key-rotation-azure
|
||||
//Rotating OIDC bound service account signer keys
|
||||
include::modules/rotating-bound-service-keys.adoc[leveloffset=+2]
|
||||
:!context: key-rotation-azure
|
||||
:context: changing-cloud-credentials-configuration
|
||||
|
||||
//Rotating {ibm-cloud-title} credentials
|
||||
include::modules/refreshing-service-ids-ibm-cloud.adoc[leveloffset=+2]
|
||||
|
||||
[id="post-install-rotate-cloud-creds_{context}"]
|
||||
== Rotating cloud provider credentials
|
||||
|
||||
Some organizations require the rotation of the cloud provider credentials.
|
||||
To allow the cluster to use the new credentials, you must update the secrets that the xref:../operators/operator-reference.adoc#cloud-credential-operator_cluster-operators-ref[Cloud Credential Operator (CCO)] uses to manage cloud provider credentials.
|
||||
|
||||
[id="ccoctl-rotate-remove-cloud-creds_{context}"]
|
||||
=== Rotating cloud provider credentials with the Cloud Credential Operator utility
|
||||
|
||||
// Right now only IBM can do this, but it makes sense to set this up so that other clouds can be added.
|
||||
The Cloud Credential Operator (CCO) utility `ccoctl` supports updating secrets for clusters installed on {ibm-cloud-name}.
|
||||
|
||||
//Rotating {ibm-cloud-title} credentials with ccoctl
|
||||
include::modules/refreshing-service-ids-ibm-cloud.adoc[leveloffset=+3]
|
||||
|
||||
//Rotating cloud provider credentials manually
|
||||
include::modules/manually-rotating-cloud-creds.adoc[leveloffset=+2]
|
||||
|
||||
@@ -35,6 +55,13 @@ include::modules/manually-rotating-cloud-creds.adoc[leveloffset=+2]
|
||||
* xref:../authentication/managing_cloud_provider_credentials/cco-mode-passthrough.html#cco-mode-passthrough[The Cloud Credential Operator in passthrough mode]
|
||||
* xref:../storage/container_storage_interface/persistent-storage-csi-vsphere.adoc#persistent-storage-csi-vsphere[vSphere CSI Driver Operator]
|
||||
|
||||
[id="post-install-remove-cloud-creds_{context}"]
|
||||
== Removing cloud provider credentials
|
||||
//TODO: split out rotate, maintain, and remove and bumpe everything up one level
|
||||
|
||||
After installing {product-title}, some organizations require the removal of the cloud provider credentials that were used during the initial installation.
|
||||
To allow the cluster to use the new credentials, you must update the secrets that the xref:../operators/operator-reference.adoc#cloud-credential-operator_cluster-operators-ref[Cloud Credential Operator (CCO)] uses to manage cloud provider credentials.
|
||||
|
||||
//Removing cloud provider credentials manually
|
||||
include::modules/manually-removing-cloud-creds.adoc[leveloffset=+2]
|
||||
|
||||
|
||||
182
snippets/ccoctl-provider-permissions-requirements.adoc
Normal file
182
snippets/ccoctl-provider-permissions-requirements.adoc
Normal file
@@ -0,0 +1,182 @@
|
||||
// Text snippet included in the following modules:
|
||||
//
|
||||
// * modules/cco-ccoctl-configuring.adoc (ifevals for aws-sts, azure-workload-id, google-cloud-platform)
|
||||
// * modules/rotating-bound-service-keys.adoc (ifevals for rotate-aws, rotate-azure, rotate-gcp)
|
||||
//
|
||||
|
||||
// There is almost certainly a better reuse strategy for the rotation perms but the content needs to go in and this is functional.
|
||||
|
||||
//AWS permissions needed when running ccoctl during installation and key rotation.
|
||||
ifdef::aws-sts[]
|
||||
* You have created an {aws-short} account for the `ccoctl` utility to use with the following permissions:
|
||||
+
|
||||
--
|
||||
**Required `iam` permissions**
|
||||
|
||||
* `iam:CreateOpenIDConnectProvider`
|
||||
* `iam:CreateRole`
|
||||
* `iam:DeleteOpenIDConnectProvider`
|
||||
* `iam:DeleteRole`
|
||||
* `iam:DeleteRolePolicy`
|
||||
* `iam:GetOpenIDConnectProvider`
|
||||
* `iam:GetRole`
|
||||
* `iam:GetUser`
|
||||
* `iam:ListOpenIDConnectProviders`
|
||||
* `iam:ListRolePolicies`
|
||||
* `iam:ListRoles`
|
||||
* `iam:PutRolePolicy`
|
||||
* `iam:TagOpenIDConnectProvider`
|
||||
* `iam:TagRole`
|
||||
|
||||
**Required `s3` permissions**
|
||||
|
||||
* `s3:CreateBucket`
|
||||
* `s3:DeleteBucket`
|
||||
* `s3:DeleteObject`
|
||||
* `s3:GetBucketAcl`
|
||||
* `s3:GetBucketTagging`
|
||||
* `s3:GetObject`
|
||||
* `s3:GetObjectAcl`
|
||||
* `s3:GetObjectTagging`
|
||||
* `s3:ListBucket`
|
||||
* `s3:PutBucketAcl`
|
||||
* `s3:PutBucketPolicy`
|
||||
* `s3:PutBucketPublicAccessBlock`
|
||||
* `s3:PutBucketTagging`
|
||||
* `s3:PutObject`
|
||||
* `s3:PutObjectAcl`
|
||||
* `s3:PutObjectTagging`
|
||||
|
||||
**Required `cloudfront` permissions**
|
||||
|
||||
* `cloudfront:ListCloudFrontOriginAccessIdentities`
|
||||
* `cloudfront:ListDistributions`
|
||||
* `cloudfront:ListTagsForResource`
|
||||
--
|
||||
|
||||
* If you plan to store the OIDC configuration in a private S3 bucket that is accessed by the IAM identity provider through a public CloudFront distribution URL, the {aws-short} account that runs the `ccoctl` utility requires the following additional permissions:
|
||||
+
|
||||
--
|
||||
* `cloudfront:CreateCloudFrontOriginAccessIdentity`
|
||||
* `cloudfront:CreateDistribution`
|
||||
* `cloudfront:DeleteCloudFrontOriginAccessIdentity`
|
||||
* `cloudfront:DeleteDistribution`
|
||||
* `cloudfront:GetCloudFrontOriginAccessIdentity`
|
||||
* `cloudfront:GetCloudFrontOriginAccessIdentityConfig`
|
||||
* `cloudfront:GetDistribution`
|
||||
* `cloudfront:TagResource`
|
||||
* `cloudfront:UpdateDistribution`
|
||||
--
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
These additional permissions support the use of the `--create-private-s3-bucket` option when processing credentials requests with the `ccoctl aws create-all` command.
|
||||
====
|
||||
endif::aws-sts[]
|
||||
ifdef::rotate-aws[]
|
||||
* You have created an {aws-short} account for the `ccoctl` utility to use with the following permissions:
|
||||
+
|
||||
--
|
||||
* `s3:GetObject`
|
||||
* `s3:PutObject`
|
||||
* `s3:PutObjectTagging`
|
||||
* For clusters that store the OIDC configuration in a private S3 bucket that is accessed by the IAM identity provider through a public CloudFront distribution URL, the {aws-short} account that runs the `ccoctl` utility requires the `cloudfront:ListDistributions` permission.
|
||||
--
|
||||
endif::rotate-aws[]
|
||||
|
||||
//Azure permissions needed when running ccoctl during installation and key rotation.
|
||||
ifdef::azure-workload-id[]
|
||||
* You have created a global {azure-short} account for the `ccoctl` utility to use with the following permissions:
|
||||
+
|
||||
--
|
||||
* `Microsoft.Resources/subscriptions/resourceGroups/read`
|
||||
* `Microsoft.Resources/subscriptions/resourceGroups/write`
|
||||
* `Microsoft.Resources/subscriptions/resourceGroups/delete`
|
||||
* `Microsoft.Authorization/roleAssignments/read`
|
||||
* `Microsoft.Authorization/roleAssignments/delete`
|
||||
* `Microsoft.Authorization/roleAssignments/write`
|
||||
* `Microsoft.Authorization/roleDefinitions/read`
|
||||
* `Microsoft.Authorization/roleDefinitions/write`
|
||||
* `Microsoft.Authorization/roleDefinitions/delete`
|
||||
* `Microsoft.Storage/storageAccounts/listkeys/action`
|
||||
* `Microsoft.Storage/storageAccounts/delete`
|
||||
* `Microsoft.Storage/storageAccounts/read`
|
||||
* `Microsoft.Storage/storageAccounts/write`
|
||||
* `Microsoft.Storage/storageAccounts/blobServices/containers/delete`
|
||||
* `Microsoft.Storage/storageAccounts/blobServices/containers/read`
|
||||
* `Microsoft.Storage/storageAccounts/blobServices/containers/write`
|
||||
* `Microsoft.ManagedIdentity/userAssignedIdentities/delete`
|
||||
* `Microsoft.ManagedIdentity/userAssignedIdentities/read`
|
||||
* `Microsoft.ManagedIdentity/userAssignedIdentities/write`
|
||||
* `Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/read`
|
||||
* `Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/write`
|
||||
* `Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/delete`
|
||||
* `Microsoft.Storage/register/action`
|
||||
* `Microsoft.ManagedIdentity/register/action`
|
||||
--
|
||||
endif::azure-workload-id[]
|
||||
ifdef::rotate-azure[]
|
||||
* You have created a global {azure-short} account for the `ccoctl` utility to use with the following permissions:
|
||||
+
|
||||
--
|
||||
* `Microsoft.Storage/storageAccounts/listkeys/action`
|
||||
* `Microsoft.Storage/storageAccounts/read`
|
||||
* `Microsoft.Storage/storageAccounts/write`
|
||||
* `Microsoft.Storage/storageAccounts/blobServices/containers/read`
|
||||
* `Microsoft.Storage/storageAccounts/blobServices/containers/write`
|
||||
--
|
||||
endif::rotate-azure[]
|
||||
|
||||
//GCP permissions needed when running ccoctl during installation and key rotation.
|
||||
ifdef::google-cloud-platform[]
|
||||
* You have added one of the following authentication options to the {gcp-short} account that the `ccoctl` utility uses:
|
||||
|
||||
** The **IAM Workload Identity Pool Admin** role
|
||||
|
||||
** The following granular permissions:
|
||||
+
|
||||
--
|
||||
* `compute.projects.get`
|
||||
* `iam.googleapis.com/workloadIdentityPoolProviders.create`
|
||||
* `iam.googleapis.com/workloadIdentityPoolProviders.get`
|
||||
* `iam.googleapis.com/workloadIdentityPools.create`
|
||||
* `iam.googleapis.com/workloadIdentityPools.delete`
|
||||
* `iam.googleapis.com/workloadIdentityPools.get`
|
||||
* `iam.googleapis.com/workloadIdentityPools.undelete`
|
||||
* `iam.roles.create`
|
||||
* `iam.roles.delete`
|
||||
* `iam.roles.list`
|
||||
* `iam.roles.undelete`
|
||||
* `iam.roles.update`
|
||||
* `iam.serviceAccounts.create`
|
||||
* `iam.serviceAccounts.delete`
|
||||
* `iam.serviceAccounts.getIamPolicy`
|
||||
* `iam.serviceAccounts.list`
|
||||
* `iam.serviceAccounts.setIamPolicy`
|
||||
* `iam.workloadIdentityPoolProviders.get`
|
||||
* `iam.workloadIdentityPools.delete`
|
||||
* `resourcemanager.projects.get`
|
||||
* `resourcemanager.projects.getIamPolicy`
|
||||
* `resourcemanager.projects.setIamPolicy`
|
||||
* `storage.buckets.create`
|
||||
* `storage.buckets.delete`
|
||||
* `storage.buckets.get`
|
||||
* `storage.buckets.getIamPolicy`
|
||||
* `storage.buckets.setIamPolicy`
|
||||
* `storage.objects.create`
|
||||
* `storage.objects.delete`
|
||||
* `storage.objects.list`
|
||||
--
|
||||
endif::google-cloud-platform[]
|
||||
ifdef::rotate-gcp[]
|
||||
* You have added one of the following authentication options to the {gcp-short} account that the `ccoctl` utility uses:
|
||||
|
||||
** The **IAM Workload Identity Pool Admin** role
|
||||
|
||||
** The following granular permissions:
|
||||
+
|
||||
--
|
||||
* `storage.objects.create`
|
||||
* `storage.objects.delete`
|
||||
--
|
||||
endif::rotate-gcp[]
|
||||
Reference in New Issue
Block a user