Azure role granularity

authentication/managing_cloud_provider_credentials/cco-short-term-creds.adoc

@@ -10,7 +10,7 @@ During installation, you can configure the Cloud Credential Operator (CCO) to op
 
 [NOTE]
 ====
-This credentials strategy is supported for Amazon Web Services (AWS) and Google Cloud Platform (GCP) only. The strategy must be configured during installation of a new {product-title} cluster. You cannot configure an existing cluster that uses a different credentials strategy to use this feature.
+This credentials strategy is supported for Amazon Web Services (AWS), Google Cloud Platform (GCP), and global Microsoft Azure only. The strategy must be configured during installation of a new {product-title} cluster. You cannot configure an existing cluster that uses a different credentials strategy to use this feature.
 ====
 
 //todo: Should provide some more info about the benefits of this here as well. Note: Azure is not yet limited-priv, but still gets the benefit of not storing root creds on the cluster and some sort of time-based rotation
@@ -18,7 +18,6 @@ This credentials strategy is supported for Amazon Web Services (AWS) and Google
 Cloud providers use different terms for their implementation of this authentication method.
 
 .Short-term credentials provider terminology
-//[cols="<.^,^.^"]
 |====
 |Cloud provider |Provider nomenclature
 
@@ -28,26 +27,64 @@ Cloud providers use different terms for their implementation of this authenticat
 |Google Cloud Platform (GCP)
 |GCP Workload Identity
 
-//|global Microsoft Azure
-//|Azure AD Workload Identity
-//
+|Global Microsoft Azure
+|Azure AD Workload Identity
+
 |====
 
-//Provider authentication processes
-include::modules/cco-short-term-creds-auth-flows.adoc[leveloffset=+1]
+[id="cco-short-term-creds-aws_{context}"]
+== AWS Security Token Service
 
-[id="cco-short-term-creds-formats_{context}"]
-== Component secret formats
-The content of the component `Secret` object for a cluster that uses short-term credentials managed outside the cluster differs from the secret format used for long-term credentials.
+In manual mode with STS, the individual {product-title} cluster components use the AWS Security Token Service (STS) to assign components IAM roles that provide short-term, limited-privilege security credentials. These credentials are associated with IAM roles that are specific to each component that makes AWS API calls.
+
+[role="_additional-resources"]
+.Additional resources
+* xref:../../installing/installing_aws/installing-aws-customizations.adoc#installing-aws-with-short-term-creds_installing-aws-customizations[Configuring an AWS cluster to use short-term credentials]
+
+//AWS Security Token Service authentication process
+include::modules/cco-short-term-creds-auth-flow-aws.adoc[leveloffset=+2]
 
 //AWS component secret formats
 include::modules/cco-short-term-creds-format-aws.adoc[leveloffset=+2]
 
+//AWS component secret permissions requirements
+include::modules/cco-short-term-creds-component-permissions-aws.adoc[leveloffset=+2]
+
+[id="cco-short-term-creds-gcp_{context}"]
+== GCP Workload Identity
+
+In manual mode with GCP Workload Identity, the individual {product-title} cluster components use the GCP workload identity provider to allow components to impersonate GCP service accounts using short-term, limited-privilege credentials.
+
+[role="_additional-resources"]
+.Additional resources
+* xref:../../installing/installing_gcp/installing-gcp-customizations.adoc#installing-gcp-with-short-term-creds_installing-gcp-customizations[Configuring a GCP cluster to use short-term credentials]
+
+//GCP Workload Identity authentication process
+include::modules/cco-short-term-creds-auth-flow-gcp.adoc[leveloffset=+2]
+
 //GCP component secret formats
 include::modules/cco-short-term-creds-format-gcp.adoc[leveloffset=+2]
 
+//GCP component secret permissions requirements (placeholder)
+//include::modules/cco-short-term-creds-component-permissions-gcp.adoc[leveloffset=+2]
+
+[id="cco-short-term-creds-azure_{context}"]
+== Azure AD Workload Identity
+
+In manual mode with Azure AD Workload Identity, the individual {product-title} cluster components use the Azure AD workload identity provider to assign components short-term security credentials.
+
+[role="_additional-resources"]
+.Additional resources
+//* xr\ef:../../installing/installing_azure/installing-azure-customizations.adoc#installing-azure-with-short-term-creds_installing-azure-customizations[Configuring a global Microsoft Azure cluster to use short-term credentials]
+
+//Azure AD Workload Identity authentication process (placeholder)
+//include::modules/cco-short-term-creds-auth-flow-azure.adoc[leveloffset=+2]
+
 //Azure component secret formats
-//inc\lude::modules/cco-short-term-creds-format-azure.adoc[leveloffset=+2]
+include::modules/cco-short-term-creds-format-azure.adoc[leveloffset=+2]
+
+//Azure component secret permissions requirements
+include::modules/cco-short-term-creds-component-permissions-azure.adoc[leveloffset=+2]
 
 [role="_additional-resources"]
 [id="additional-resources_{context}"]

modules/cco-short-term-creds-auth-flow-aws.adoc

@@ -0,0 +1,12 @@
+// Module included in the following assemblies:
+//
+// * authentication/managing_cloud_provider_credentials/cco-short-term-creds.adoc
+
+:_content-type: REFERENCE
+[id="cco-short-term-creds-auth-flow-aws_{context}"]
+= AWS Security Token Service authentication process
+
+The following diagram details the authentication flow between AWS and the {product-title} cluster when using AWS STS.
+
+.AWS Security Token Service authentication flow
+image::347_OpenShift_credentials_with_STS_updates_0623_AWS.png[Detailed authentication flow between AWS and the cluster when using AWS STS]

modules/cco-short-term-creds-auth-flow-azure.adoc

@@ -0,0 +1,13 @@
+// Module included in the following assemblies:
+//
+// * authentication/managing_cloud_provider_credentials/cco-short-term-creds.adoc
+
+:_content-type: REFERENCE
+[id="cco-short-term-creds-auth-flow-azure_{context}"]
+= Azure AD Workload Identity authentication process
+
+The following diagram details the authentication flow between Azure and the {product-title} cluster when using Azure AD Workload Identity.
+
+//todo: work with dev and diagrams team to get a diagram for Azure
+.Azure AD Workload Identity authentication flow
+//image::azure_ad_workload_identity_flow.png[Detailed authentication flow between Azure and the cluster when using Azure AD Workload Identity]

modules/cco-short-term-creds-auth-flow-gcp.adoc

@@ -0,0 +1,12 @@
+// Module included in the following assemblies:
+//
+// * authentication/managing_cloud_provider_credentials/cco-short-term-creds.adoc
+
+:_content-type: REFERENCE
+[id="cco-short-term-creds-auth-flow-gcp_{context}"]
+= GCP Workload Identity authentication process
+
+The following diagram details the authentication flow between GCP and the {product-title} cluster when using GCP Workload Identity.
+
+.GCP Workload Identity authentication flow
+image::347_OpenShift_credentials_with_STS_updates_0623_GCP.png[Detailed authentication flow between GCP and the cluster when using GCP Workload Identity]

modules/cco-short-term-creds-component-permissions-aws.adoc

@@ -0,0 +1,209 @@
+// Module included in the following assemblies:
+//
+// * authentication/managing_cloud_provider_credentials/cco-short-term-creds.adoc
+
+:_content-type: REFERENCE
+[id="cco-short-term-creds-component-permissions-aws_{context}"]
+= AWS component secret permissions requirements
+
+{product-title} components require the following permissions. These values are in the `CredentialsRequest` custom resource (CR) for each component.
+
+[NOTE]
+====
+These permissions apply to all resources. Unless specified, there are no request conditions on these permissions.
+====
+
+[cols="a,a,a"]
+|====
+|Component |Custom resource |Required permissions for services
+
+|Cluster CAPI Operator
+|`openshift-cluster-api-aws`
+|**EC2**
+
+* `ec2:CreateTags`
+* `ec2:DescribeAvailabilityZones`
+* `ec2:DescribeDhcpOptions`
+* `ec2:DescribeImages`
+* `ec2:DescribeInstances`
+* `ec2:DescribeInternetGateways`
+* `ec2:DescribeSecurityGroups`
+* `ec2:DescribeSubnets`
+* `ec2:DescribeVpcs`
+* `ec2:DescribeNetworkInterfaces`
+* `ec2:DescribeNetworkInterfaceAttribute`
+* `ec2:ModifyNetworkInterfaceAttribute`
+* `ec2:RunInstances`
+* `ec2:TerminateInstances`
+
+**Elastic load balancing**
+
+* `elasticloadbalancing:DescribeLoadBalancers`
+* `elasticloadbalancing:DescribeTargetGroups`
+* `elasticloadbalancing:DescribeTargetHealth`
+* `elasticloadbalancing:RegisterInstancesWithLoadBalancer`
+* `elasticloadbalancing:RegisterTargets`
+* `elasticloadbalancing:DeregisterTargets`
+
+**Identity and Access Management (IAM)**
+
+* `iam:PassRole`
+* `iam:CreateServiceLinkedRole`
+
+**Key Management Service (KMS)**
+
+* `kms:Decrypt`
+* `kms:Encrypt`
+* `kms:GenerateDataKey`
+* `kms:GenerateDataKeyWithoutPlainText`
+* `kms:DescribeKey`
+* `kms:RevokeGrant`^[1]^
+* `kms:CreateGrant` ^[1]^
+* `kms:ListGrants` ^[1]^
+
+|Machine API Operator
+|`openshift-machine-api-aws`
+|**EC2**
+
+* `ec2:CreateTags`
+* `ec2:DescribeAvailabilityZones`
+* `ec2:DescribeDhcpOptions`
+* `ec2:DescribeImages`
+* `ec2:DescribeInstances`
+* `ec2:DescribeInstanceTypes`
+* `ec2:DescribeInternetGateways`
+* `ec2:DescribeSecurityGroups`
+* `ec2:DescribeRegions`
+* `ec2:DescribeSubnets`
+* `ec2:DescribeVpcs`
+* `ec2:RunInstances`
+* `ec2:TerminateInstances`
+
+**Elastic load balancing**
+
+* `elasticloadbalancing:DescribeLoadBalancers`
+* `elasticloadbalancing:DescribeTargetGroups`
+* `elasticloadbalancing:DescribeTargetHealth`
+* `elasticloadbalancing:RegisterInstancesWithLoadBalancer`
+* `elasticloadbalancing:RegisterTargets`
+* `elasticloadbalancing:DeregisterTargets`
+
+**Identity and Access Management (IAM)**
+
+* `iam:PassRole`
+* `iam:CreateServiceLinkedRole`
+
+**Key Management Service (KMS)**
+
+* `kms:Decrypt`
+* `kms:Encrypt`
+* `kms:GenerateDataKey`
+* `kms:GenerateDataKeyWithoutPlainText`
+* `kms:DescribeKey`
+* `kms:RevokeGrant`^[1]^
+* `kms:CreateGrant` ^[1]^
+* `kms:ListGrants` ^[1]^
+
+|Cloud Credential Operator
+|`cloud-credential-operator-iam-ro`
+|**Identity and Access Management (IAM)**
+
+* `iam:GetUser`
+* `iam:GetUserPolicy`
+* `iam:ListAccessKeys`
+
+|Cluster Image Registry Operator
+|`openshift-image-registry`
+|**S3**
+
+* `s3:CreateBucket`
+* `s3:DeleteBucket`
+* `s3:PutBucketTagging`
+* `s3:GetBucketTagging`
+* `s3:PutBucketPublicAccessBlock`
+* `s3:GetBucketPublicAccessBlock`
+* `s3:PutEncryptionConfiguration`
+* `s3:GetEncryptionConfiguration`
+* `s3:PutLifecycleConfiguration`
+* `s3:GetLifecycleConfiguration`
+* `s3:GetBucketLocation`
+* `s3:ListBucket`
+* `s3:GetObject`
+* `s3:PutObject`
+* `s3:DeleteObject`
+* `s3:ListBucketMultipartUploads`
+* `s3:AbortMultipartUpload`
+* `s3:ListMultipartUploadParts`
+
+|Ingress Operator
+|`openshift-ingress`
+|**Elastic load balancing**
+
+* `elasticloadbalancing:DescribeLoadBalancers`
+
+**Route 53**
+
+* `route53:ListHostedZones`
+* `route53:ListTagsForResources`
+* `route53:ChangeResourceRecordSets`
+
+**Tag**
+
+* `tag:GetResources`
+
+**Security Token Service (STS)**
+
+* `sts:AssumeRole`
+
+|Cluster Network Operator
+|`openshift-cloud-network-config-controller-aws`
+|**EC2**
+
+* `ec2:DescribeInstances`
+* `ec2:DescribeInstanceStatus`
+* `ec2:DescribeInstanceTypes`
+* `ec2:UnassignPrivateIpAddresses`
+* `ec2:AssignPrivateIpAddresses`
+* `ec2:UnassignIpv6Addresses`
+* `ec2:AssignIpv6Addresses`
+* `ec2:DescribeSubnets`
+* `ec2:DescribeNetworkInterfaces`
+
+|AWS Elastic Block Store CSI Driver Operator
+|`aws-ebs-csi-driver-operator`
+|**EC2**
+
+* `ec2:AttachVolume`
+* `ec2:CreateSnapshot`
+* `ec2:CreateTags`
+* `ec2:CreateVolume`
+* `ec2:DeleteSnapshot`
+* `ec2:DeleteTags`
+* `ec2:DeleteVolume`
+* `ec2:DescribeInstances`
+* `ec2:DescribeSnapshots`
+* `ec2:DescribeTags`
+* `ec2:DescribeVolumes`
+* `ec2:DescribeVolumesModifications`
+* `ec2:DetachVolume`
+* `ec2:ModifyVolume`
+* `ec2:DescribeAvailabilityZones`
+* `ec2:EnableFastSnapshotRestores`
+
+**Key Management Service (KMS)**
+
+* `kms:ReEncrypt*`
+* `kms:Decrypt`
+* `kms:Encrypt`
+* `kms:GenerateDataKey`
+* `kms:GenerateDataKeyWithoutPlainText`
+* `kms:DescribeKey`
+* `kms:RevokeGrant`^[1]^
+* `kms:CreateGrant` ^[1]^
+* `kms:ListGrants` ^[1]^
+
+|====
+[.small]
+--
+1. Request condition: `kms:GrantIsForAWSResource: true`
+--

modules/cco-short-term-creds-component-permissions-azure.adoc

@@ -0,0 +1,136 @@
+// Module included in the following assemblies:
+//
+// * authentication/managing_cloud_provider_credentials/cco-short-term-creds.adoc
+
+:_content-type: REFERENCE
+[id="cco-short-term-creds-component-permissions-azure_{context}"]
+= Azure component secret permissions requirements
+
+{product-title} components require the following permissions. These values are in the `CredentialsRequest` custom resource (CR) for each component.
+
+[cols="a,a,a"]
+|====
+|Component |Custom resource |Required permissions for services
+
+|Cloud Controller Manager Operator
+|`openshift-azure-cloud-controller-manager`
+|* `Microsoft.Compute/virtualMachines/read`
+* `Microsoft.Network/loadBalancers/read`
+* `Microsoft.Network/loadBalancers/write`
+* `Microsoft.Network/networkInterfaces/read`
+* `Microsoft.Network/networkSecurityGroups/read`
+* `Microsoft.Network/networkSecurityGroups/write`
+* `Microsoft.Network/publicIPAddresses/join/action`
+* `Microsoft.Network/publicIPAddresses/read`
+* `Microsoft.Network/publicIPAddresses/write`
+
+|Cluster CAPI Operator
+|`openshift-cluster-api-azure`
+|role: `Contributor` ^[1]^
+
+|Machine API Operator
+|`openshift-machine-api-azure`
+|* `Microsoft.Compute/availabilitySets/delete`
+* `Microsoft.Compute/availabilitySets/read`
+* `Microsoft.Compute/availabilitySets/write`
+* `Microsoft.Compute/diskEncryptionSets/read`
+* `Microsoft.Compute/disks/delete`
+* `Microsoft.Compute/galleries/images/versions/read`
+* `Microsoft.Compute/skus/read`
+* `Microsoft.Compute/virtualMachines/delete`
+* `Microsoft.Compute/virtualMachines/extensions/delete`
+* `Microsoft.Compute/virtualMachines/extensions/read`
+* `Microsoft.Compute/virtualMachines/extensions/write`
+* `Microsoft.Compute/virtualMachines/read`
+* `Microsoft.Compute/virtualMachines/write`
+* `Microsoft.ManagedIdentity/userAssignedIdentities/assign/action`
+* `Microsoft.Network/applicationSecurityGroups/read`
+* `Microsoft.Network/loadBalancers/backendAddressPools/join/action`
+* `Microsoft.Network/loadBalancers/read`
+* `Microsoft.Network/loadBalancers/write`
+* `Microsoft.Network/networkInterfaces/delete`
+* `Microsoft.Network/networkInterfaces/join/action`
+* `Microsoft.Network/networkInterfaces/loadBalancers/read`
+* `Microsoft.Network/networkInterfaces/read`
+* `Microsoft.Network/networkInterfaces/write`
+* `Microsoft.Network/networkSecurityGroups/read`
+* `Microsoft.Network/networkSecurityGroups/write`
+* `Microsoft.Network/publicIPAddresses/delete`
+* `Microsoft.Network/publicIPAddresses/join/action`
+* `Microsoft.Network/publicIPAddresses/read`
+* `Microsoft.Network/publicIPAddresses/write`
+* `Microsoft.Network/routeTables/read`
+* `Microsoft.Network/virtualNetworks/delete`
+* `Microsoft.Network/virtualNetworks/read`
+* `Microsoft.Network/virtualNetworks/subnets/join/action`
+* `Microsoft.Network/virtualNetworks/subnets/read`
+* `Microsoft.Resources/subscriptions/resourceGroups/read`
+
+|Cluster Image Registry Operator
+|`openshift-image-registry-azure`
+|**Data permissions**
+
+* `Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete`
+* `Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write`
+* `Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read`
+* `Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action`
+* `Microsoft.Storage/storageAccounts/blobServices/containers/blobs/move/action`
+
+**General permissions**
+
+* `Microsoft.Storage/storageAccounts/blobServices/read`
+* `Microsoft.Storage/storageAccounts/blobServices/containers/read`
+* `Microsoft.Storage/storageAccounts/blobServices/containers/write`
+* `Microsoft.Storage/storageAccounts/blobServices/generateUserDelegationKey/action`
+* `Microsoft.Storage/storageAccounts/read`
+* `Microsoft.Storage/storageAccounts/write`
+* `Microsoft.Storage/storageAccounts/delete`
+* `Microsoft.Storage/storageAccounts/listKeys/action`
+* `Microsoft.Resources/tags/write`
+
+|Ingress Operator
+|`openshift-ingress-azure`
+|* `Microsoft.Network/dnsZones/A/delete`
+* `Microsoft.Network/dnsZones/A/write`
+* `Microsoft.Network/privateDnsZones/A/delete`
+* `Microsoft.Network/privateDnsZones/A/write`
+
+|Cluster Network Operator
+|`openshift-cloud-network-config-controller-azure`
+|* `Microsoft.Network/networkInterfaces/read`
+* `Microsoft.Network/networkInterfaces/write`
+* `Microsoft.Compute/virtualMachines/read`
+* `Microsoft.Network/virtualNetworks/read`
+* `Microsoft.Network/virtualNetworks/subnets/join/action`
+* `Microsoft.Network/loadBalancers/backendAddressPools/join/action`
+
+|Azure File CSI Driver Operator
+|`azure-file-csi-driver-operator`
+|* `Microsoft.Network/networkSecurityGroups/join/action`
+* `Microsoft.Network/virtualNetworks/subnets/read`
+* `Microsoft.Network/virtualNetworks/subnets/write`
+* `Microsoft.Storage/storageAccounts/delete`
+* `Microsoft.Storage/storageAccounts/fileServices/read`
+* `Microsoft.Storage/storageAccounts/fileServices/shares/delete`
+* `Microsoft.Storage/storageAccounts/fileServices/shares/read`
+* `Microsoft.Storage/storageAccounts/fileServices/shares/write`
+* `Microsoft.Storage/storageAccounts/listKeys/action`
+* `Microsoft.Storage/storageAccounts/read`
+* `Microsoft.Storage/storageAccounts/write`
+
+|Azure Disk CSI Driver Operator
+|`azure-disk-csi-driver-operator`
+|* `Microsoft.Compute/disks/*`
+* `Microsoft.Compute/snapshots/*`
+* `Microsoft.Compute/virtualMachineScaleSets/*/read`
+* `Microsoft.Compute/virtualMachineScaleSets/read`
+* `Microsoft.Compute/virtualMachineScaleSets/virtualMachines/write`
+* `Microsoft.Compute/virtualMachines/*/read`
+* `Microsoft.Compute/virtualMachines/write`
+* `Microsoft.Resources/subscriptions/resourceGroups/read`
+
+|====
+[.small]
+--
+1. This component requires a role rather than a set of permissions.
+--

modules/cco-short-term-creds-component-permissions-gcp.adoc

@@ -0,0 +1,9 @@
+// Module included in the following assemblies:
+//
+// * authentication/managing_cloud_provider_credentials/cco-short-term-creds.adoc
+
+:_content-type: REFERENCE
+[id="cco-short-term-creds-component-permissions-gcp_{context}"]
+= GCP component secret permissions requirements
+
+//This topic is a placeholder for when GCP role granularity can bbe documented

modules/cco-short-term-creds-format-azure.adoc

@@ -0,0 +1,57 @@
+// Module included in the following assemblies:
+//
+// * authentication/managing_cloud_provider_credentials/cco-short-term-creds.adoc
+
+:_content-type: REFERENCE
+[id="cco-short-term-creds-format-azure_{context}"]
+= Azure component secret formats
+
+Using manual mode with AD Workload Identity changes the content of the Azure credentials that are provided to individual {product-title} components. Compare the following secret formats:
+
+.Azure secret format using long-term credentials
+
+[source,yaml]
+----
+apiVersion: v1
+kind: Secret
+metadata:
+  namespace: <target_namespace> <1>
+  name: <target_secret_name> <2>
+data:
+  azure_client_id: <client_id> <3>
+  azure_client_secret: <client_secret> <4>
+  azure_region: <region>
+  azure_resource_prefix: <resource_group_prefix> <5>
+  azure_resourcegroup: <resource_group_prefix>-rg <6>
+  azure_subscription_id: <subscription_id>
+  azure_tenant_id: <tenant_id>
+type: Opaque
+----
+<1> The namespace for the component.
+<2> The name of the component secret.
+<3> The client ID of the Azure AD identity that the component uses to authenticate.
+<4> The component secret that is used to authenticate with Azure AD for the `<client_id>` identity.
+<5> The resource group prefix.
+<6> The resource group. This value is formed by the `<resource_group_prefix>` and the suffix `-rg`.
+
+.Azure secret format using AD Workload Identity
+
+[source,yaml]
+----
+apiVersion: v1
+kind: Secret
+metadata:
+  namespace: <target_namespace> <1>
+  name: <target_secret_name> <2>
+data:
+  azure_client_id: <client_id> <3>
+  azure_federated_token_file: <path_to_token_file> <4>
+  azure_region: <region>
+  azure_subscription_id: <subscription_id>
+  azure_tenant_id: <tenant_id>
+type: Opaque
+----
+<1> The namespace for the component.
+<2> The name of the component secret.
+<3> The client ID of the user-assigned managed identity that the component uses to authenticate.
+<4> The path to the mounted service account token file.