Merge pull request #36603 from openshift-cherrypick-robot/cherry-pick-34426-to-enterprise-4.9

[enterprise-4.9] OSDOCS-2189: AWS EFS CSI Operator (TP)
2026-02-05 12:46:18 +01:00 · 2021-09-21 23:50:40 +10:00
parent 2878b9e91d b2f6fc2298
commit b22be7aec7
12 changed files with 438 additions and 8 deletions
--- a/_topic_map.yml
+++ b/_topic_map.yml
@@ -1150,6 +1150,8 @@ Topics:
    File: persistent-storage-csi-migration
  - Name: AWS Elastic Block Store CSI Driver Operator
    File: persistent-storage-csi-ebs
+  - Name: AWS Elastic File Service CSI Driver Operator
+    File: persistent-storage-csi-aws-efs
  - Name: Azure Disk CSI Driver Operator
    File: persistent-storage-csi-azure
  - Name: Azure Stack Hub CSI Driver Operator
--- a/modules/persistent-storage-csi-drivers-supported.adoc
+++ b/modules/persistent-storage-csi-drivers-supported.adoc
@@ -17,6 +17,7 @@ The following table describes the CSI drivers that are installed with {product-t
 |CSI driver  |CSI volume snapshots  |CSI cloning  |CSI resize

 |AWS EBS | ✅ | - | ✅
+|AWS EFS (Tech Preview) | - | - | -
 |Google Cloud Platform (GCP) persistent disk (PD) (Tech Preview)| ✅ | - | ✅
 |Microsoft Azure Disk (Tech Preview) | ✅ | ✅ | ✅
 |Microsoft Azure Stack Hub | - | - | -
--- a/modules/persistent-storage-csi-dynamic-provisioning-aws-efs.adoc
+++ b/modules/persistent-storage-csi-dynamic-provisioning-aws-efs.adoc
@@ -0,0 +1,71 @@
+// Module included in the following assemblies:
+//
+// * storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc
+
+[id="csi-dynamic-provisioning-aws-efs_{context}"]
+= Dynamic provisioning for AWS EFS
+
+The AWS EFS CSI Driver supports a different form of dynamic provisioning than other CSI drivers. It provisions new PVs as subdirectories of a pre-existing EFS volume. The PVs are independent of each other. However, they all share the same EFS volume. When the volume is deleted, all PVs provisioned out of it are deleted too.
+The EFS CSI driver creates an AWS Access Point for each such subdirectory. Due to AWS AccessPoint limits, you can only dynamically provision 120 PVs from a single `StorageClass`/EFS volume.
+
+[IMPORTANT]
+====
+Note that `PVC.spec.resources` is not enforced by EFS.
+
+In the example below, you request 5 GiB of space. However, the created PV is limitless and can store any amount of data (like petabytes). A broken application, or even a rogue application, can cause significant expenses when it stores too much data on the volume.
+
+Using monitoring of EFS volume sizes in AWS is strongly recommended.
+====
+
+.Prerequisites
+
+* xref:../../storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc#efs-create-volume_persistent-storage-csi-aws-efs[Created AWS EFS volume(s).]
+
+.Procedure
+
+To enable dynamic provisioning:
+
+. Create a `StorageClass` as follows:
+
+[source,yaml]
+----
+kind: StorageClass
+apiVersion: storage.k8s.io/v1
+metadata:
+  name: efs-sc
+provisioner: efs.csi.aws.com
+parameters:
+  provisioningMode: efs-ap <1>
+  fileSystemId: fs-a5324911 <2>
+  directoryPerms: "700" <3>
+  gidRangeStart: "1000" <4>
+  gidRangeEnd: "2000" <4>
+  basePath: "/dynamic_provisioning" <5>
+----
+<1> `provisioningMode` must be `efs-ap` to enable dynamic provisioning.
+<2> `fileSystemId` must be the ID of the EFS volume created manually above.
+<3> `directoryPerms` is the default permission of the root directory of the volume. In this case, the volume is accessible only by the owner.
+<4> `gidRangeStart` and `gidRangeEnd` set the range of POSIX Group IDs (GIDs) that are used to set the GID of the AWS access point. If not specified, the default range is 50000-7000000. Each provisioned volume, and thus AWS access point, is assigned a unique GID from this range.
+<5> `basePath` is the directory on the EFS volume that is used to create dynamically provisioned volumes. In this case, a PV is provisioned as “/dynamic_provisioning/<random uuid>” on the EFS volume. Only the subdirectory is mounted to pods that use the PV.
+
+[NOTE]
+====
+A cluster admin can create several `StorageClasses`, each using a different EFS volume.
+====
+
+. Create a PVC (or StatefulSet or Template) as usual, referring to the `StorageClass` created above.
+
+[source,yaml]
+----
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: test
+spec:
+  storageClassName: efs-sc
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      storage: 5Gi
+----
--- a/modules/persistent-storage-csi-efs-create-volume.adoc
+++ b/modules/persistent-storage-csi-efs-create-volume.adoc
@@ -0,0 +1,50 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent_storage/persistent-storage-csi-aws-efs.adoc
+
+[id="efs-create-volume_{context}"]
+= Creating and configuring access to EFS volumes in AWS
+
+This procedure explains how to create and configure EFS volumes in AWS so that you can use them in {product-title}.
+
+.Prerequisites
+
+* AWS account credentials
+
+.Procedure
+
+To create and configure access to an EFS volume in AWS:
+
+. On the AWS console, open https://console.aws.amazon.com/efs.
+
+. Click *Create file system*:
+
+* Enter a name for the file system.
+
+* For *Virtual Private Cloud (VPC)*, select your {product-title}'s' virtual private cloud (VPC).
+
+* Accept default settings for all other selections.
+
+. Wait for the volume and mount targets to finish being fully created:
+
+.. Go to https://console.aws.amazon.com/efs#/file-systems.
+
+.. Click your volume, and on the *Network* tab wait for all mount targets to become available (~1-2 minutes).
+
+. On the *Network* tab, copy the Security Group ID (you will need this in the next step).
+
+. Go to https://console.aws.amazon.com/ec2/v2/home#SecurityGroups, and find the Security Group used by the EFS volume.
+
+. On the *Inbound rules* tab, click *Edit inbound rules*, and then add a new rule with the following settings to allow {product-title} nodes to access EFS volumes :
+
+* *Type*: NFS
+
+* *Protocol*: TCP
+
+* *Port range*: 2049
+
+* *Source*: Custom/IP address range of your nodes (for example: “10.0.0.0/16”)
+
+This step allows {product-title} to use NFS ports from the cluster.
+
+. Save the rule.
--- a/modules/persistent-storage-csi-efs-security.adoc
+++ b/modules/persistent-storage-csi-efs-security.adoc
@@ -0,0 +1,14 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent_storage/persistent-storage-csi-aws-efs.adoc
+
+[id="efs-security_{context}"]
+= AWS EFS security
+
+The following information is important for AWS EFS security.
+
+When using access points, for example, by using dynamic provisioning as described earlier, Amazon automatically replaces GIDs on files with the GID of the access point. In addition, EFS considers the user ID, group ID, and secondary group IDs of the access point when evaluating file system permissions. EFS ignores the NFS client's IDs. For more information about access points, see https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html.
+
+As a consequence, EFS volumes silently ignore FSGroup; {product-title} is not able to replace the GIDs of files on the volume with FSGroup. Any pod that can access a mounted EFS access point can access any file on it.
+
+Unrelated to this, encryption in transit is enabled by default. For more information, see https://docs.aws.amazon.com/efs/latest/ug/encryption-in-transit.html.
--- a/modules/persistent-storage-csi-efs-static-pv.adoc
+++ b/modules/persistent-storage-csi-efs-static-pv.adoc
@@ -0,0 +1,40 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent_storage/persistent-storage-csi-aws-efs.adoc
+
+[id="efs-create-static-pv_{context}"]
+= Creating static PVs with AWS EFS
+
+It is possible to use an AWS EFS volume as a single PV without any dynamic provisioning. The whole volume is mounted to pods.
+
+.Prerequisites
+
+* xref:../../storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc#efs-create-volume_persistent-storage-csi-aws-efs[Created AWS EFS volume(s).]
+
+.Procedure
+
+* Create the PV using the following YAML file:
+
+[source,yaml]
+----
+apiVersion: v1
+kind: PersistentVolume
+metadata:
+  name: efs-pv
+spec:
+  capacity: <1>
+    storage: 5Gi
+  volumeMode: Filesystem
+  accessModes:
+    - ReadWriteMany
+    - ReadWriteOnce
+  persistentVolumeReclaimPolicy: Retain
+  csi:
+    driver: efs.csi.aws.com
+    volumeHandle: fs-ae66151a <2>
+    volumeAttributes:
+        encryptInTransit: false <3>
+----
+<1> `spec.capacity` does not have any meaning and is ignored by the CSI driver. It is used only when binding to a PVC. Applications can store any amount of data to the volume.
+<2> `volumeHandle` must be the same ID as the EFS volume you created in AWS. If you are providing your own access point, `volumeHandle` should be ``<EFS volume ID>::<access point ID>``. For example: `fs-6e633ada::fsap-081a1d293f0004630`.
+<3> If desired, you can disable encryption in transit. Encryption is enabled by default.
--- a/modules/persistent-storage-csi-efs-troubleshooting.adoc
+++ b/modules/persistent-storage-csi-efs-troubleshooting.adoc
@@ -0,0 +1,52 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent_storage/persistent-storage-csi-aws-efs.adoc
+
+[id="efs-troubleshooting_{context}"]
+= AWS EFS troubleshooting
+
+The following information provides guidance on how to troubleshoot issues with AWS EFS:
+
+* The AWS EFS Operator and CSI driver run in namespace `openshift-cluster-csi-drivers`.
+
+* To initiate gathering of logs of the AWS EFS Operator and CSI driver, run the following command:
+
+[source, terminal]
+----
+$ oc adm must-gather
+[must-gather      ] OUT Using must-gather plugin-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:125f183d13601537ff15b3239df95d47f0a604da2847b561151fedd699f5e3a5
+[must-gather      ] OUT namespace/openshift-must-gather-xm4wq created
+[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-2bd8x created
+[must-gather      ] OUT pod for plug-in image quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:125f183d13601537ff15b3239df95d47f0a604da2847b561151fedd699f5e3a5 created
+----
+
+* To show AWS EFS Operator errors, view the `ClusterCSIDriver` status:
+
+[source, terminal]
+----
+$ oc get clustercsidriver efs.csi.aws.com -o yaml
+----
+
+* If a volume cannot be mounted to a pod (as shown in the output of the following command):
+
+[source, terminal]
+----
+$ oc describe pod
+...
+  Type     Reason       Age    From               Message
+  ----     ------       ----   ----               -------
+  Normal   Scheduled    2m13s  default-scheduler  Successfully assigned default/efs-app to ip-10-0-135-94.ec2.internal
+  Warning  FailedMount  13s    kubelet            MountVolume.SetUp failed for volume "pvc-d7c097e6-67ec-4fae-b968-7e7056796449" : rpc error: code = DeadlineExceeded desc = context deadline exceeded <1>
+  Warning  FailedMount  10s    kubelet            Unable to attach or mount volumes: unmounted volumes=[persistent-storage], unattached volumes=[persistent-storage kube-api-access-9j477]: timed out waiting for the condition
+----
+<1> Warning message indicating volume not mounted.
+
+This error is frequently caused by AWS dropping packets between an {product-title} node and AWS EFS.
+
+Check that the following are correct (see xref:../../storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc#efs-create-volume_persistent-storage-csi-aws-efs[Creating and configuring access to EFS volumes in AWS]):
+
+--
+* AWS firewall and Security Groups
+
+* Networking: port number and IP addresses
+--
--- a/modules/persistent-storage-csi-olm-driver-uninstall.adoc
+++ b/modules/persistent-storage-csi-olm-driver-uninstall.adoc
@@ -0,0 +1,21 @@
+// Module included in the following assemblies:
+//
+// * storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc
+
+[id="persistent-storage-csi-olm-driver-uninstall_{context}"]
+= Uninstalling the {FeatureName} CSI driver
+
+.Prerequisites
+* Access to the {product-title} web console.
+
+To uninstall the {FeatureName} CSI driver:
+
+. Log in to the web console.
+
+. Stop all applications that use {FeatureName} persistent volumes (PVs).
+
+. Click *administration* -> *CustomResourceDefinitions* -> *ClusterCSIDriver*.
+
+. On the *Instances* tab, for *{provisioner}*, on the far left side, click the drop-down menu, and then click *Delete ClusterCSIDriver*.
+
+. When prompted, click *Delete*.
--- a/modules/persistent-storage-csi-olm-operator-install.adoc
+++ b/modules/persistent-storage-csi-olm-operator-install.adoc
@@ -0,0 +1,68 @@
+// Module included in the following assemblies:
+//
+// * storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc
+
+[id="persistent-storage-csi-olm-operator-install_{context}"]
+= Installing the {FeatureName} CSI Driver Operator
+
+The {FeatureName} CSI Driver Operator is not installed in {product-title} by default. Use the following procedure to install and configure the {FeatureName} CSI Driver Operator in your cluster.
+
+.Prerequisites
+* Access to the {product-title} web console.
+
+.Procedure
+To install the {FeatureName} CSI Driver Operator from the web console:
+
+. Log in to the web console.
+
+. Install the {FeatureName} CSI Operator:
+
+.. Click *Operators* -> *OperatorHub*.
+
+.. Locate the {FeatureName} CSI Operator by typing *{FeatureName} CSI* in the filter box.
+
+.. Click the *{FeatureName} CSI Driver Operator* button.
+
+[IMPORTANT]
+====
+Be sure to select the *AWS EFS CSI Driver Operator* and not the *AWS EFS Operator*. The *AWS EFS Operator* is a community Operator and is not supported by Red Hat.
+====
+
+.. On the *{FeatureName} CSI Driver Operator* page, click *Install*.
+
+.. On the *Install Operator* page, ensure that:
+
+* *All namespaces on the cluster (default)* is selected.
+* *Installed Namespace* is set to *openshift-cluster-csi-drivers*.
+
+.. Click *Install*.
+
+After the installation finishes, the {FeatureName} CSI Operator is listed in the *Installed Operators* section of the web console.
+
+. Install the {FeatureName} CSI Driver:
+
+.. Click *administration* -> *CustomResourceDefinitions* -> *ClusterCSIDriver*.
+
+.. On the *Instances* tab, click *Create ClusterCSIDriver*.
+
+.. Use the following YAML file:
+
+[source,yaml]
+----
+apiVersion: operator.openshift.io/v1
+kind: ClusterCSIDriver
+metadata:
+    name: efs.csi.aws.com
+spec:
+  managementState: Managed
+----
+
+.. Click *Create*.
+
+.. Wait for the following Conditions to change to a "true" status:
+
+* AWSEFSDriverCredentialsRequestControllerAvailable
+
+* AWSEFSDriverNodeServiceControllerAvailable
+
+* AWSEFSDriverControllerServiceControllerAvailable
--- a/modules/persistent-storage-csi-olm-operator-uninstall.adoc
+++ b/modules/persistent-storage-csi-olm-operator-uninstall.adoc
@@ -0,0 +1,54 @@
+// Module included in the following assemblies:
+//
+// * storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc
+
+[id="persistent-storage-csi-olm-operator-uninstall_{context}"]
+= Uninstalling the {FeatureName} CSI Driver Operator
+
+All EFS PVs are inaccessible after uninstalling the AWS EFS CSI Driver Operator.
+
+.Prerequisites
+* Access to the {product-title} web console.
+
+.Procedure
+To uninstall the {FeatureName} CSI Driver Operator from the web console:
+
+. Log in to the web console.
+
+. Stop all applications that use {FeatureName} PVs.
+
+. Delete all {FeatureName} PVs:
+
+.. Click *Storage* -> *PersistentVolumeClaims*.
+
+.. Select each PVC that is in use by the {FeatureName} CSI Driver Operator, click the drop-down menu on the far right of the PVC, and then click *Delete PersistentVolumeClaims*.
+
+. Uninstall the {FeatureName} CSI Driver:
+
+[NOTE]
+====
+Before you can uninstall the Operator, you must remove the CSI driver first.
+====
+
+.. Click *administration* -> *CustomResourceDefinitions* -> *ClusterCSIDriver*.
+
+.. On the *Instances* tab, for *{provisioner}*, on the far left side, click the drop-down menu, and then click *Delete ClusterCSIDriver*.
+
+.. When prompted, click *Delete*.
+
+. Uninstall the {FeatureName} CSI Operator:
+
+.. Click *Operators* -> *Installed Operators*.
+
+.. On the *Installed Operators* page, scroll or type {FeatureName} CSI into the *Search by name* box to find the Operator, and then click it.
+
+.. On the upper, right of the *Installed Operators > Operator details* page, click *Actions* -> *Uninstall Operator*.
+
+.. When prompted on the *Uninstall Operator* window, click the *Uninstall* button to remove the Operator from the namespace. Any applications deployed by the Operator on the cluster need to be cleaned up manually.
+
+After uninstalling, the {FeatureName} CSI Driver Operator is no longer listed in the *Installed Operators* section of the web console.
+
+[NOTE]
+====
+Before you can destroy a cluster (`openshift-install destroy cluster`), you must delete the EFS volume in AWS. An {product-title} cluster cannot be destroyed when there is an EFS volume that uses the cluster's VPC. Amazon does not allow deletion of such a VPC.
+====
--- a/modules/storage-create-storage-class.adoc
+++ b/modules/storage-create-storage-class.adoc
@@ -5,24 +5,27 @@
 // :StorageClass: AWS EBS
 // Module included in the following assemblies:
 //
-// * storage/persistent_storage-aws.adoc
+// * storage/persistent_storage/persistent-storage-aws.adoc
+// * storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc

-
-
-[id="storage-create-{StorageClass}-storage-class_{context}"]
+[id="storage-create-storage-class_{context}"]
 = Creating the {StorageClass} storage class

 Storage classes are used to differentiate and delineate storage levels and
 usages. By defining a storage class, users can obtain dynamically provisioned
 persistent volumes.

+ifeval::["{Provisioner}" == "efs.csi.aws.com"]
+The _AWS EFS CSI Driver Operator_, after being installed, does not create a storage class by default. However, you can manually create the AWS EFS `StorageClass`.
+endif::[]
+
 .Procedure

 . In the {product-title} console, click *Storage* -> *Storage Classes*.

-. In the storage class overview, click *Create Storage Class*.
+. On the *StorageClasses* overview page, click *Create Storage Class*.

-. Define the desired options on the page that appears.
+. On the *StorageClasses* create page, enter values as desired:

 .. Enter a name to reference the storage class.

@@ -30,7 +33,7 @@ persistent volumes.

 .. Select the reclaim policy.

-.. Select `{Provisioner}` from the drop-down list.
+.. Select *`{Provisioner}`* from the *Provisioner* drop-down list.
 +
 ifeval::["{Provisioner}" == "kubernetes.io/aws-ebs"]
 [NOTE]
@@ -41,7 +44,7 @@ endif::[]

 .. Enter additional parameters for the storage class as desired.

-. Click *Create* to create the storage class.
+. Click *Create*.

 // Undefine {StorageClass} attribute, so that any mistakes are easily spotted
 :!StorageClass:
--- a/storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc
+++ b/storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc
@@ -0,0 +1,54 @@
+[id="persistent-storage-csi-aws-efs"]
+= AWS Elastic File Service CSI Driver Operator
+include::modules/common-attributes.adoc[]
+:context: persistent-storage-csi-aws-efs
+
+toc::[]
+
+== Overview
+
+{product-title} is capable of provisioning persistent volumes (PVs) using the Container Storage Interface (CSI) driver for AWS Elastic File Service (EFS).
+
+:FeatureName: AWS EFS Driver Operator
+include::modules/technology-preview.adoc[leveloffset=+1]
+
+Familiarity with xref:../../storage/understanding-persistent-storage.adoc#understanding-persistent-storage[persistent storage] and xref:../../storage/container_storage_interface/persistent-storage-csi.adoc#persistent-storage-csi[configuring CSI volumes] is recommended when working with a CSI Operator and driver.
+
+After installing the AWS EFS CSI Driver Operator, {product-title} installs the AWS EFS CSI Operator and the AWS EFS CSI driver by default in the `openshift-cluster-csi-drivers` namespace. This allows the AWS EFS CSI Driver Operator to create CSI-provisioned PVs that mount to AWS EFS assets.
+
+* The _AWS EFS CSI Driver Operator_, after being installed, does not create a storage class by default to use to create persistent volume claims (PVCs). However, you can manually create the AWS EFS `StorageClass`.
+The AWS EFS CSI Driver Operator supports dynamic volume provisioning by allowing storage volumes to be created on-demand, eliminating the need for cluster administrators to pre-provision storage.
+
+* The _AWS EFS CSI driver_ enables you to create and mount AWS EFS PVs. The AWS EFS CSI driver must be manually installed.
+
+[NOTE]
+====
+AWS EFS only supports regional volumes, not zonal volumes.
+====
+
+include::modules/persistent-storage-csi-about.adoc[leveloffset=+1]
+
+:FeatureName: AWS EFS
+include::modules/persistent-storage-csi-olm-operator-install.adoc[leveloffset=+1]
+
+:StorageClass: AWS EFS
+:Provisioner: efs.csi.aws.com
+include::modules/storage-create-storage-class.adoc[leveloffset=+1]
+
+include::modules/persistent-storage-csi-efs-create-volume.adoc[leveloffset=+1]
+
+include::modules/persistent-storage-csi-dynamic-provisioning-aws-efs.adoc[leveloffset=+1]
+If you have problems setting up dynamic provisioning, see xref:../../storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc#efs-troubleshooting_persistent-storage-csi-aws-efs[AWS EFS troubleshooting].
+
+include::modules/persistent-storage-csi-efs-static-pv.adoc[leveloffset=+1]
+If you have problems setting up static PVs, see xref:../../storage/container_storage_interface/persistent-storage-csi-aws-efs.adoc#efs-troubleshooting_persistent-storage-csi-aws-efs[AWS EFS troubleshooting].
+
+include::modules/persistent-storage-csi-efs-security.adoc[leveloffset=+1]
+
+include::modules/persistent-storage-csi-efs-troubleshooting.adoc[leveloffset=+1]
+
+:FeatureName: AWS EFS
+include::modules/persistent-storage-csi-olm-operator-uninstall.adoc[leveloffset=+1]
+
+== Additional resources
+* xref:../../storage/container_storage_interface/persistent-storage-csi.adoc#persistent-storage-csi[Configuring CSI volumes]