openshift/openshift-docs
1
0
Fork 0
mirror of https://github.com/openshift/openshift-docs.git synced 2026-02-05 12:46:18 +01:00
Code Issues Releases Activity

Merge pull request #100400 from openshift-cherrypick-robot/cherry-pick-97085-to-enterprise-4.19

Browse Source 
[enterprise-4.19] OSDOCS-15638#AWS EFS cross account procedure rewrite
This commit is contained in:
Lisa Pettyjohn
2025-10-13 10:06:41 -04:00
committed by GitHub
parent 618a2c5fea 549c109d70
commit 62ca988c83
2 changed files with 356 additions and 448 deletions
Download Patch File Download Diff File Expand all files Collapse all files

796
modules/persistent-storage-csi-efs-cross-account.adoc
View File

@@ -16,11 +16,6 @@ ifndef::openshift-rosa,openshift-rosa-hcp[]
endif::openshift-rosa,openshift-rosa-hcp[]
in one AWS account and mount your file system in another AWS account by using the AWS Elastic File System (EFS) Container Storage Interface (CSI) driver.
[NOTE]
====
Both the {product-title} cluster and EFS file system must be in the same region.
====
.Prerequisites
* Access to
@@ -29,228 +24,180 @@ ifdef::openshift-rosa,openshift-rosa-hcp[]
endif::openshift-rosa,openshift-rosa-hcp[]
ifndef::openshift-rosa,openshift-rosa-hcp[]
an {product-title} cluster
endif::openshift-rosa,openshift-rosa-hcp[] cluster with administrator rights
endif::openshift-rosa,openshift-rosa-hcp[]
with administrator rights
* Two valid AWS accounts
* The EFS CSI Operator has been installed. For information about installing the EFS CSI Operator, see the _Installing the AWS EFS CSI Driver Operator_ section.
* Both the {product-title} cluster and EFS file system must be located in the same AWS region.
* Ensure that the two virtual private clouds (VPCs) used in the following procedure use different network Classless Inter-Domain Routing (CIDR) ranges.
* Access to {product-title} CLI (`oc`).
* Access to AWS CLI.
* Access to `jq` command-line JSON processor.
.Procedure
The following procedure demonstrates how to set up:
The following procedure explains how to set up:
* {product-title} cluster in AWS account A
* {product-title} AWS Account A: Contains a Red Hat {product-title} cluster v4.16, or later, deployed within a VPC
* Mount an AWS EFS file system in account B
* AWS Account B: Contains a VPC (including subnets, route tables, and network connectivity). The EFS filesystem will be created in this VPC.
To use AWS EFS across accounts:
. Install {product-title} cluster with AWS account A and install the EFS CSI Driver Operator.
. Set up the environment:
. Create an EFS volume in AWS account B:
.. Create a virtual private cloud (VPC) called, for example, "my-efs-vpc” with CIDR, for example, “172.20.0.0/16” and subnet for the AWS EFS volume.
.. On the AWS console, go to https://console.aws.amazon.com/efs.
.. Click *Create new filesystem*:
... Create a filesystem named, for example, "my-filesystem”.
... Select the VPC created earlier (“my-efs-vpc”).
... Accept the default for the remaining settings.
.. Ensure that the volume and Mount Targets have been created:
... Check https://console.aws.amazon.com/efs#/file-systems.
... Click your volume, and on the *Network* tab wait for all Mount Targets to be available (approximately 1-2 minutes).
.. On the *Network* tab, copy the Security Group ID. You will need it for the next step.
. Configure networking access to the AWS EFS volume on AWS account B:
.. Go to https://console.aws.amazon.com/ec2/v2/home#SecurityGroups.
.. Find the Security Group used by the AWS EFS volume by filtering for the group ID copied earlier.
.. On the *Inbound rules* tab, click *Edit inbound rules*, and then add a new rule to allow {product-title} nodes to access the AWS EFS volumes (that is, use NFS ports from the cluster):
+
* *Type*: NFS
* *Protocol*: TCP
* *Port range*: 2049
* *Source*: Custom/IP address range of your {product-title} cluster nodes (for example, “10.0.0.0/16”)
.. Save the rule.
+
[NOTE]
====
If you encounter mounting issues, re-check the port number, IP address range, and verify that the AWS EFS volume uses the expected security group.
====
. Create VPC peering between the {product-title} cluster VPC in AWS account A and the AWS EFS VPC in AWS account B:
+
Ensure the two VPCs are using different network CIDRs, and after creating the VPC peering, add routes in each VPC to connect the two VPC networks.
.. Create a peering connection called, for example, “my-efs-crossaccount-peering-connection” in account B. For the local VPC ID, use the EFS-located VPC. To peer with the VPC for account A, for the VPC ID use the {product-title} cluster VPC ID.
.. Accept the peer connection in AWS account A.
.. Modify the route table of each subnet (EFS-volume used subnets) in AWS account B:
... On the left pane, under *Virtual private cloud*, click the down arrow to expand the available options.
... Under *Virtual private cloud*, click *Route tables"*.
... Click the *Routes* tab.
... Under *Destination*, enter 10.0.0.0/16.
... Under *Target*, use the peer connection type point from the created peer connection.
.. Modify the route table of each subnet ({product-title} cluster nodes used subnets) in AWS account A:
... On the left pane, under *Virtual private cloud*, click the down arrow to expand the available options.
... Under *Virtual private cloud*, click *Route tables"*.
... Click the *Routes* tab.
... Under *Destination*, enter the CIDR for the VPC in account B, which for this example is 172.20.0.0/16.
... Under *Target*, use the peer connection type point from the created peer connection.
// this is where the changes for ROSA starts to line 396
ifdef::openshift-rosa,openshift-rosa-hcp[]
. Prepare the **AWS account A** IAM roles and policies.
+
[NOTE]
====
This process requires **two separate** AWS accounts.
====
.. Create an IAM policy for the EFS CSI driver.
+
[NOTE]
====
This has additional permission's compared to a single account EFS CSI policy.
====
+
[source, json]
----
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"elasticfilesystem:DescribeAccessPoints",
"elasticfilesystem:DescribeFileSystems",
"elasticfilesystem:DescribeMountTargets",
"elasticfilesystem:TagResource",
"elasticfilesystem:ClientMount",
"elasticfilesystem:ClientRootAccess",
"elasticfilesystem:ClientWrite",
"elasticfilesystem:DescribeMountTargets",
"ec2:DescribeAvailabilityZones"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"elasticfilesystem:CreateAccessPoint"
],
"Resource": "*",
"Condition": {
"StringLike": {
"aws:RequestTag/efs.csi.aws.com/cluster": "true"
}
}
},
{
"Effect": "Allow",
"Action": "elasticfilesystem:DeleteAccessPoint",
"Resource": "*",
"Condition": {
"StringEquals": {
"aws:ResourceTag/efs.csi.aws.com/cluster": "true"
}
}
},
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::${AWS_ACCOUNT_B_ID}:role/cross-account-efs-role"
}
]
}
----
.. Create the policy.
.. Configure environment variables by running the following commands:
+
[source,terminal]
----
$ aws iam create-policy --policy-name "${CLUSTER_NAME}-rosa-efs-csi" \
--policy-document file://<path-to-file>.json \
--query 'Policy.Arn' --output text
export CLUSTER_NAME="<CLUSTER_NAME>" <1>
export AWS_REGION="<AWS_REGION>" <2>
export AWS_ACCOUNT_A_ID="<ACCOUNT_A_ID>" <3>
export AWS_ACCOUNT_B_ID="<ACCOUNT_B_ID>" <4>
export AWS_ACCOUNT_A_VPC_CIDR="<VPC_A_CIDR>" <5>
export AWS_ACCOUNT_B_VPC_CIDR="<VPC_B_CIDR>" <6>
export AWS_ACCOUNT_A_VPC_ID="<VPC_A_ID>" <7>
export AWS_ACCOUNT_B_VPC_ID="<VPC_B_ID>" <8>
export SCRATCH_DIR="<WORKING_DIRECTORY>" <9>
export CSI_DRIVER_NAMESPACE="openshift-cluster-csi-drivers" <10>
export AWS_PAGER="" <11>
----
<1> Cluster name of choice.
<2> AWS region of choice.
<3> AWS Account A ID.
<4> AWS Account B ID.
<5> CIDR range of VPC in Account A.
<6> CIDR range of VPC in Account B.
<7> VPC ID in Account A (cluster)
<8> VPC ID in Account B (EFS cross account)
<9> Any writeable directory of choice to use to store temporary files.
<10> If your driver is installed in a non-default namespace, change this value.
<11> Makes AWS CLI output everything directly to stdout.
.. Create the working directory by running the following command:
+
[source,terminal]
----
mkdir -p $SCRATCH_DIR
----
.. Create a trust policy.
.. Verify cluster connectivity by running the following command in the {product-title} CLI:
+
[source, json]
[source,terminal]
----
$ oc whoami
----
.. Determine the {product-title} cluster type and set node selector:
+
The EFS cross account feature requires assigning AWS IAM policies to nodes running EFS CSI controller pods. However, this is
not consistent for every {product-title} type.
+
* If your cluster is deployed as a Hosted Control Plane (HyperShift), set the `NODE_SELECTOR` environment variable to hold the worker node label by running the following command:
+
[source,terminal]
----
export NODE_SELECTOR=node-role.kubernetes.io/worker
----
+
* For all other {product-title} types, set the `NODE_SELECTOR` environment variable to hold the master node label by running the following command:
+
[source,terminal]
----
export NODE_SELECTOR=node-role.kubernetes.io/master
----
.. Configure AWS CLI profiles as environment variables for account switching by running the following commands:
+
[source,terminal]
----
export AWS_ACCOUNT_A="<ACCOUNT_A_NAME>"
export AWS_ACCOUNT_B="<ACCOUNT_B_NAME>"
----
.. Ensure that your AWS CLI is configured with JSON output format as the default for both accounts by running the following commands:
+
[source,terminal]
----
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_A}
aws configure get output
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_B}
aws configure get output
----
+
If the preceding commands return:
+
* *No value*: The default output format is already set to JSON and no changes are required.
+
* *Any value*: Reconfigure your AWS CLI to use JSON format. For information about changing output formats, see _Setting the output format in the AWS CLI_ in the AWS documentation.
.. Unset `AWS_PROFILE` in your shell to prevent conflicts with `AWS_DEFAULT_PROFILE` by running the following command:
+
[source,terminal]
----
unset AWS_PROFILE
----
. Configure the AWS Account B IAM roles and policies:
.. Switch to your Account B profile by running the following command:
+
[source,terminal]
----
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_B}
----
.. Define the IAM role name for the EFS CSI Driver Operator by running the following command:
+
[source,terminal]
----
export ACCOUNT_B_ROLE_NAME=${CLUSTER_NAME}-cross-account-aws-efs-csi-operator
----
.. Create the IAM trust policy file by running the following command:
+
[source,terminal]
----
cat <<EOF > $SCRATCH_DIR/AssumeRolePolicyInAccountB.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${AWS_ACCOUNT_A_ID}:oidc-provider/${OIDC_PROVIDER}"
"AWS": "arn:aws:iam::${AWS_ACCOUNT_A_ID}:root"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"${OIDC_PROVIDER}:sub": [
"system:serviceaccount:openshift-cluster-csi-drivers:aws-efs-csi-driver-operator",
"system:serviceaccount:openshift-cluster-csi-drivers:aws-efs-csi-driver-controller-sa"
]
}
}
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
EOF
----
.. Create a role for the EFS CSI Driver Operator.
.. Create the IAM role for the EFS CSI Driver Operator by running the following command:
+
[source, terminal]
[source,terminal]
----
$ aws iam create-role \
--role-name "${CLUSTER_NAME}-aws-efs-csi-operator" \
--assume-role-policy-document file://<path-to-file>.json \
--query "Role.Arn" --output text
ACCOUNT_B_ROLE_ARN=$(aws iam create-role \
--role-name "${ACCOUNT_B_ROLE_NAME}" \
--assume-role-policy-document file://$SCRATCH_DIR/AssumeRolePolicyInAccountB.json \
--query "Role.Arn" --output text) \
&& echo $ACCOUNT_B_ROLE_ARN
----
.. Attach the policies to the role.
.. Create the IAM policy file by running the following command:
+
[source, terminal]
----
$ aws iam attach-role-policy \
--role-name "${CLUSTER_NAME}-aws-efs-csi-operator" \
--policy-arn ${ACCOUNT_A_POLICY_ARN}
----
+
This role that the EFS CSI controller uses can now assume a role inside account B.
. Prepare the **AWS account B** IAM roles and policies.
.. Create an IAM policy.
+
[source, json]
[source,terminal]
----
cat << EOF > $SCRATCH_DIR/EfsPolicyInAccountB.json
{
"Version": "2012-10-17",
"Statement": [
@@ -281,315 +228,276 @@ This role that the EFS CSI controller uses can now assume a role inside account
}
]
}
EOF
----
.. Create the policy.
.. Create the IAM policy by running the following command:
+
[source,terminal]
----
$ aws iam create-policy --policy-name "cross-account-rosa-efs-csi" \
--policy-document file://<path-to-file>.json \
--query 'Policy.Arn' --output text
ACCOUNT_B_POLICY_ARN=$(aws iam create-policy --policy-name "${CLUSTER_NAME}-efs-csi-policy" \
--policy-document file://$SCRATCH_DIR/EfsPolicyInAccountB.json \
--query 'Policy.Arn' --output text) \
&& echo ${ACCOUNT_B_POLICY_ARN}
----
.. Create a trust policy.
.. Attach the policy to the role by running the following command:
+
[source, json]
[source,terminal]
----
aws iam attach-role-policy \
--role-name "${ACCOUNT_B_ROLE_NAME}" \
--policy-arn "${ACCOUNT_B_POLICY_ARN}"
----
. Configure the AWS Account A IAM roles and policies:
.. Switch to your Account A profile by running the following command:
+
[source,terminal]
----
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_A}
----
.. Create the IAM policy document by running the following command:
+
[source,terminal]
----
cat << EOF > $SCRATCH_DIR/AssumeRoleInlinePolicyPolicyInAccountA.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${AWS_ACCOUNT_A_ID}:root"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "${ACCOUNT_B_ROLE_ARN}"
}
]
}
EOF
----
.. Create the role for the EFS CSI Driver Operator to assume.
.. In AWS Account A, attach the AWS-managed policy "AmazonElasticFileSystemClientFullAccess" to the {product-title} cluster master role by running the following command:
+
[source,terminal]
----
$ aws iam create-role \
--role-name "cross-account-efs-role" \
--assume-role-policy-document file://<path-to-file>.json \
--query "Role.Arn" --output text
EFS_CLIENT_FULL_ACCESS_BUILTIN_POLICY_ARN=arn:aws:iam::aws:policy/AmazonElasticFileSystemClientFullAccess
declare -A ROLE_SEEN
for NODE in $(oc get nodes --selector="${NODE_SELECTOR}" -o jsonpath='{.items[*].metadata.name}'); do
INSTANCE_PROFILE=$(aws ec2 describe-instances \
--filters "Name=private-dns-name,Values=${NODE}" \
--query 'Reservations[].Instances[].IamInstanceProfile.Arn' \
--output text | awk -F'/' '{print $NF}' | xargs)
MASTER_ROLE_ARN=$(aws iam get-instance-profile \
--instance-profile-name "${INSTANCE_PROFILE}" \
--query 'InstanceProfile.Roles[0].Arn' \
--output text | xargs)
MASTER_ROLE_NAME=$(echo "${MASTER_ROLE_ARN}" | awk -F'/' '{print $NF}' | xargs)
echo "Checking role: '${MASTER_ROLE_NAME}'"
if [[ -n "${ROLE_SEEN[$MASTER_ROLE_NAME]:-}" ]]; then
echo "Already processed role: '${MASTER_ROLE_NAME}', skipping."
continue
fi
ROLE_SEEN["$MASTER_ROLE_NAME"]=1
echo "Assigning policy ${EFS_CLIENT_FULL_ACCESS_BUILTIN_POLICY_ARN} to role ${MASTER_ROLE_NAME}"
aws iam attach-role-policy --role-name "${MASTER_ROLE_NAME}" --policy-arn "${EEFS_CLIENT_FULL_ACCESS_BUILTIN_POLICY_ARN}"
done
----
.. Attach the policies to the role.
. Attach the policy to the IAM entity to allow role assumption:
+
[source, terminal]
----
$ aws iam attach-role-policy \
--role-name "cross-account-efs-role" \
--policy-arn ${ACCOUNT_B_POLICY_ARN}
----
. Deploy the AWS EFS CSI Operator in **AWS account A**.
.. Create a `secret` to tell the AWS EFS Operator which IAM role to request.
This step depends on your cluster configuration. In both of the following scenarios, the EFS CSI Driver Operator uses an entity to authenticate to AWS, and this entity must be granted permission to assume roles in Account B.
+
[source, yaml]
If your cluster:
+
* *Does not have STS enabled*: The EFS CSI Driver Operator uses an IAM User entity for AWS authentication. Continue with the step "Attach policy to IAM User to allow role assumption".
+
* *Has STS enabled*: The EFS CSI Driver Operator uses an IAM role entity for AWS authentication. Continue with the step "Attach policy to IAM Role to allow role assumption".
. Attach policy to IAM User to allow role assumption
.. Identify the IAM User used by the EFS CSI Driver Operator by running the following command:
+
[source,terminal]
----
apiVersion: v1
kind: Secret
metadata:
name: aws-efs-cloud-credentials
namespace: openshift-cluster-csi-drivers
stringData:
credentials: |-
[default]
role_arn = ${ACCOUNT_A_ROLE_ARN}
web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token
EFS_CSI_DRIVER_OPERATOR_USER=$(oc -n openshift-cloud-credential-operator get credentialsrequest/openshift-aws-efs-csi-driver -o json | jq -r '.status.providerStatus.user')
----
.. Install the EFS Operator.
.. Attach the policy to the IAM user by running the following command:
+
[source, yaml]
[source,terminal]
----
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
generateName: openshift-cluster-csi-drivers-
namespace: openshift-cluster-csi-drivers
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
labels:
operators.coreos.com/aws-efs-csi-driver-operator.openshift-cluster-csi-drivers: ""
name: aws-efs-csi-driver-operator
namespace: openshift-cluster-csi-drivers
spec:
channel: stable
installPlanApproval: Automatic
name: aws-efs-csi-driver-operator
source: redhat-operators
sourceNamespace: openshift-marketplace
aws iam put-user-policy \
--user-name "${EFS_CSI_DRIVER_OPERATOR_USER}" \
--policy-name efs-cross-account-inline-policy \
--policy-document file://$SCRATCH_DIR/AssumeRoleInlinePolicyPolicyInAccountA.json
----
.. Check if the operator is running.
. Attach the policy to the IAM role to allow role assumption:
.. Identify the IAM role name currently used by the EFS CSI Driver Operator by running the following command:
+
[source, terminal]
[source,terminal]
----
$ oc get deployment aws-efs-csi-driver-operator -n openshift-cluster-csi-drivers
EFS_CSI_DRIVER_OPERATOR_ROLE=$(oc -n ${CSI_DRIVER_NAMESPACE} get secret/aws-efs-cloud-credentials -o jsonpath='{.data.credentials}' | base64 -d | grep role_arn | cut -d'/' -f2) && echo ${EFS_CSI_DRIVER_OPERATOR_ROLE}
----
.. Install the AWS EFS CSI driver.
.. Attach the policy to the IAM role used by the EFS CSI Driver Operator by running the following command:
+
[source, yaml]
[source,terminal]
----
apiVersion: operator.openshift.io/v1
kind: ClusterCSIDriver
metadata:
name: efs.csi.aws.com
spec:
managementState: Managed
aws iam put-role-policy \
--role-name "${EFS_CSI_DRIVER_OPERATOR_ROLE}" \
--policy-name efs-cross-account-inline-policy \
--policy-document file://$SCRATCH_DIR/AssumeRoleInlinePolicyPolicyInAccountA.json
----
.. Check if the CSI driver is running.
. Configure VPC peering:
.. Initiate a peering request from Account A to Account B by running the following command:
+
[source, terminal]
[source,terminal]
----
$ oc get daemonset aws-efs-csi-driver-node -n openshift-cluster-csi-drivers
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_A}
PEER_REQUEST_ID=$(aws ec2 create-vpc-peering-connection --vpc-id "${AWS_ACCOUNT_A_VPC_ID}" --peer-vpc-id "${AWS_ACCOUNT_B_VPC_ID}" --peer-owner-id "${AWS_ACCOUNT_B_ID}" --query VpcPeeringConnection.VpcPeeringConnectionId --output text)
----
.. Create a new secret that will tell the CSI driver the role name in Account B to assume.
.. Accept the peering request from Account B by running the following command:
+
[source, terminal]
[source,terminal]
----
$ oc create secret generic cross-account-arn \
-n openshift-cluster-csi-drivers \
--from-literal=awsRoleArn="arn:aws:iam::${AWS_ACCOUNT_B_ID}:role/cross-account-efs-role"
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_B}
aws ec2 accept-vpc-peering-connection --vpc-peering-connection-id "${PEER_REQUEST_ID}"
----
.. Allow the EFS CSI controller to read this secret.
.. Retrieve the route table IDs for Account A and add routes to the Account B VPC by running the following command:
+
[source, terminal]
[source,terminal]
----
$ oc -n openshift-cluster-csi-drivers create role access-secrets --verb=get,list,watch --resource=secrets
$ oc -n openshift-cluster-csi-drivers create rolebinding --role=access-secrets default-to-secrets --serviceaccount=openshift-cluster-csi-drivers:aws-efs-csi-driver-controller-sa
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_A}
for NODE in $(oc get nodes --selector=node-role.kubernetes.io/worker | tail -n +2 | awk '{print $1}')
do
SUBNET=$(aws ec2 describe-instances --filters "Name=private-dns-name,Values=$NODE" --query 'Reservations[*].Instances[*].NetworkInterfaces[*].SubnetId' | jq -r '.[0][0][0]')
echo SUBNET is ${SUBNET}
ROUTE_TABLE_ID=$(aws ec2 describe-route-tables --filters "Name=association.subnet-id,Values=${SUBNET}" --query 'RouteTables[*].RouteTableId' | jq -r '.[0]')
echo Route table ID is $ROUTE_TABLE_ID
aws ec2 create-route --route-table-id ${ROUTE_TABLE_ID} --destination-cidr-block ${AWS_ACCOUNT_B_VPC_CIDR} --vpc-peering-connection-id ${PEER_REQUEST_ID}
done
----
.. Create a storage class for the EFS volume.
.. Retrieve the route table IDs for Account B and add routes to the Account A VPC by running the following command:
+
[source, yaml]
[source,terminal]
----
# The cross account efs volume storageClass
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_B}
for ROUTE_TABLE_ID in $(aws ec2 describe-route-tables --filters "Name=vpc-id,Values=${AWS_ACCOUNT_B_VPC_ID}" --query "RouteTables[].RouteTableId" | jq -r '.[]')
do
echo Route table ID is $ROUTE_TABLE_ID
aws ec2 create-route --route-table-id ${ROUTE_TABLE_ID} --destination-cidr-block ${AWS_ACCOUNT_A_VPC_CIDR} --vpc-peering-connection-id ${PEER_REQUEST_ID}
done
----
. Configure security groups in Account B to allow NFS traffic from Account A to EFS:
.. Switch to your Account B profile by running the following command:
+
[source,terminal]
----
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_B}
----
.. Configure the VPC security groups for EFS access by running the following command:
+
[source,terminal]
----
SECURITY_GROUP_ID=$(aws ec2 describe-security-groups --filters Name=vpc-id,Values="${AWS_ACCOUNT_B_VPC_ID}" | jq -r '.SecurityGroups[].GroupId')
aws ec2 authorize-security-group-ingress \
--group-id "${SECURITY_GROUP_ID}" \
--protocol tcp \
--port 2049 \
--cidr "${AWS_ACCOUNT_A_VPC_CIDR}" | jq .
----
. Create a region-wide EFS filesystem in Account B:
.. Switch to your Account B profile by running the following command:
+
[source,terminal]
----
export AWS_DEFAULT_PROFILE=${AWS_ACCOUNT_B}
----
.. Create a region-wide EFS file system by running the following command:
+
[source,terminal]
----
CROSS_ACCOUNT_FS_ID=$(aws efs create-file-system --creation-token efs-token-1 \
--region ${AWS_REGION} \
--encrypted | jq -r '.FileSystemId') \
&& echo $CROSS_ACCOUNT_FS_ID
----
.. Configure region-wide mount targets for EFS by running the following command:
+
[source,terminal]
----
for SUBNET in $(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=${AWS_ACCOUNT_B_VPC_ID}" \
--region ${AWS_REGION} \
| jq -r '.Subnets.[].SubnetId'); do \
MOUNT_TARGET=$(aws efs create-mount-target --file-system-id ${CROSS_ACCOUNT_FS_ID} \
--subnet-id ${SUBNET} \
--region ${AWS_REGION} \
| jq -r '.MountTargetId'); \
echo ${MOUNT_TARGET}; \
done
----
+
This creates a mount point in each subnet of your VPC.
. Configure the EFS Operator for cross-account access:
.. Define custom names for the secret and storage class that you will create in subsequent steps by running the following command:
+
[source,terminal]
----
export SECRET_NAME=my-efs-cross-account
export STORAGE_CLASS_NAME=efs-sc-cross
----
.. Create a secret that references the role ARN in Account B by running the following command in the {product-title} CLI:
+
[source,terminal]
----
oc create secret generic ${SECRET_NAME} -n ${CSI_DRIVER_NAMESPACE} --from-literal=awsRoleArn="${ACCOUNT_B_ROLE_ARN}"
----
.. Grant the CSI driver controller access to the newly created secret by running the following commands in the {product-title} CLI:
+
[source,terminal]
----
oc -n ${CSI_DRIVER_NAMESPACE} create role access-secrets --verb=get,list,watch --resource=secrets
oc -n ${CSI_DRIVER_NAMESPACE} create rolebinding --role=access-secrets default-to-secrets --serviceaccount=${CSI_DRIVER_NAMESPACE}:aws-efs-csi-driver-controller-sa
----
.. Create a new storage class that references the EFS ID from Account B and the secret created previously by running the following command in the {product-title} CLI:
+
[source,terminal]
----
cat << EOF | oc apply -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: efs-sc
name: ${STORAGE_CLASS_NAME}
provisioner: efs.csi.aws.com
parameters:
provisioningMode: efs-ap
fileSystemId: ${EFS_FILESYSTEM_ID}
fileSystemId: ${CROSS_ACCOUNT_FS_ID}
directoryPerms: "700"
gidRangeStart: "1000"
gidRangeEnd: "2000"
basePath: "/dynamic_provisioning"
csi.storage.k8s.io/provisioner-secret-name: cross-account-arn
csi.storage.k8s.io/provisioner-secret-namespace: openshift-cluster-csi-drivers
----
endif::openshift-rosa,openshift-rosa-hcp[]
// ROSA solution ends here
ifdef::openshift-enterprise[]
. Create an IAM role, for example, “my-efs-acrossaccount-role” in AWS account B, which has a trust relationship with AWS account A, and add an inline AWS EFS policy with permissions to call “my-efs-acrossaccount-driver-policy”.
+
This role is used by the CSI driver's controller service running on the {product-title} cluster in AWS account A to determine the mount targets for your file system in AWS account B.
+
[source, json]
----
# Trust relationships trusted entity trusted account A configuration on my-efs-acrossaccount-role in account B
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeSubnets"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"elasticfilesystem:DescribeMountTargets",
"elasticfilesystem:DeleteAccessPoint",
"elasticfilesystem:ClientMount",
"elasticfilesystem:DescribeAccessPoints",
"elasticfilesystem:ClientWrite",
"elasticfilesystem:ClientRootAccess",
"elasticfilesystem:DescribeFileSystems",
"elasticfilesystem:CreateAccessPoint"
],
"Resource": [
"arn:aws:elasticfilesystem:*:589722580343:access-point/*",
"arn:aws:elasticfilesystem:*:589722580343:file-system/*"
]
}
]
}
----
. In AWS account A, attach an inline policy to the IAM role of the AWS EFS CSI driver's controller service account with the necessary permissions to perform Security Token Service (STS) assume role on the IAM role created earlier.
+
[source, json]
----
# my-cross-account-assume-policy policy attached to Openshift cluster efs csi driver user in account A
{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::589722580343:role/my-efs-acrossaccount-role"
}
}
----
. In AWS account A, attach the AWS-managed policy “AmazonElasticFileSystemClientFullAccess” to {product-title} cluster master role. The role name is in the form `<clusterID>-master-role` (for example, `my-0120ef-czjrl-master-role`).
. Create a Kubernetes secret with `awsRoleArn` as the key and the role created earlier as the value:
+
[source, cli]
----
$ oc -n openshift-cluster-csi-drivers create secret generic my-efs-cross-account --from-literal=awsRoleArn='arn:aws:iam::589722580343:role/my-efs-acrossaccount-role'
----
+
Since the driver controller needs to get the cross account role information from the secret, you need to add the secret role binding to the AWS EFS CSI driver controller ServiceAccount (SA):
+
[source, cli]
----
$ oc -n openshift-cluster-csi-drivers create role access-secrets --verb=get,list,watch --resource=secrets
$ oc -n openshift-cluster-csi-drivers create rolebinding --role=access-secrets default-to-secrets --serviceaccount=openshift-cluster-csi-drivers:aws-efs-csi-driver-controller-sa
----
. Create a `filesystem` policy for the file system (AWS EFS volume) in account B, which allows AWS account A to perform a mount on it.
+
[NOTE]
====
This step is not mandatory, but can be safer for AWS EFS volume usage.
====
+
[source, json]
----
# EFS volume filesystem policy in account B
{
"Version": "2012-10-17",
"Id": "efs-policy-wizard-8089bf4a-9787-40f0-958e-bc2363012ace",
"Statement": [
{
"Sid": "efs-statement-bd285549-cfa2-4f8b-861e-c372399fd238",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"elasticfilesystem:ClientRootAccess",
"elasticfilesystem:ClientWrite",
"elasticfilesystem:ClientMount"
],
"Resource": "arn:aws:elasticfilesystem:us-east-2:589722580343:file-system/fs-091066a9bf9becbd5",
"Condition": {
"Bool": {
"elasticfilesystem:AccessedViaMountTarget": "true"
}
}
},
{
"Sid": "efs-statement-03646e39-d80f-4daf-b396-281be1e43bab",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::589722580343:role/my-efs-acrossaccount-role"
},
"Action": [
"elasticfilesystem:ClientRootAccess",
"elasticfilesystem:ClientWrite",
"elasticfilesystem:ClientMount"
],
"Resource": "arn:aws:elasticfilesystem:us-east-2:589722580343:file-system/fs-091066a9bf9becbd5"
}
]
}
----
. Create an AWS EFS volume storage class using a similar configuration to the following:
+
[source, yaml]
----
# The cross account efs volume storageClass
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: efs-cross-account-mount-sc
provisioner: efs.csi.aws.com
mountOptions:
- tls
parameters:
provisioningMode: efs-ap
fileSystemId: fs-00f6c3ae6f06388bb
directoryPerms: "700"
gidRangeStart: "1000"
gidRangeEnd: "2000"
basePath: "/account-a-data"
csi.storage.k8s.io/provisioner-secret-name: my-efs-cross-account
csi.storage.k8s.io/provisioner-secret-namespace: openshift-cluster-csi-drivers
volumeBindingMode: Immediate
----
endif::openshift-enterprise[]
csi.storage.k8s.io/provisioner-secret-name: ${SECRET_NAME}
csi.storage.k8s.io/provisioner-secret-namespace: ${CSI_DRIVER_NAMESPACE}
EOF
----

8
storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc
View File

@@ -71,17 +71,17 @@ include::modules/storage-create-storage-class.adoc[leveloffset=+1]
include::modules/storage-create-storage-class-console.adoc[leveloffset=+2]
include::modules/storage-create-storage-class-cli.adoc[leveloffset=+2]
ifdef::openshift-rosa,openshift-rosa-hcp[]
include::modules/persistent-storage-csi-efs-cross-account.adoc[leveloffset=+1]
endif::openshift-rosa,openshift-rosa-hcp[]
include::modules/persistent-storage-csi-efs-create-volume.adoc[leveloffset=+1]
[role="_additional-resources"]
.Additional resources
* link:https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-output-format.html[Setting the output format in the AWS CLI]
include::modules/persistent-storage-csi-dynamic-provisioning-aws-efs.adoc[leveloffset=+1]
If you have problems setting up dynamic provisioning, see xref:../../storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc#efs-troubleshooting_persistent-storage-csi-aws-efs[AWS EFS troubleshooting].
[role="_additional-resources"]
.Additional resources
* xref:../../storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc#efs-create-volume_persistent-storage-csi-aws-efs[Creating and configuring access to AWS EFS volume(s)]
* xref:../../storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc#storage-create-storage-class_persistent-storage-csi-aws-efs[Creating the AWS EFS storage class]
// Undefine {StorageClass} attribute, so that any mistakes are easily spotted