OSDOCS-294: Included CSI details.

_topic_map.yml

@@ -343,6 +343,8 @@ Topics:
     File: persistent-storage-aws
   - Name: Persistent Storage using iSCSI
     File: persistent-storage-iscsi
+  - Name: Persistent storage using Container Storage Interface (CSI)
+    File: persistent-storage-csi
 - Name: Dynamic provisioning
   File: dynamic-provisioning
 ---

modules/persistent-storage-csi-architecture.adoc

@@ -0,0 +1,22 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent_storage/persistent-storage-csi.adoc
+
+[id="persistent-storage-csi-architecture-{context}"]
+= CSI Architecture
+
+CSI drivers are typically shipped as container images. These containers 
+are not aware of {product-title} where they run. To use CSI-compatible 
+storage backend in {product-title}, the cluster administrator must deploy 
+several components that serve as a bridge between {product-title} and the 
+storage driver.
+ 
+The following diagram provides a high-level overview about the components 
+running in pods in the {product-title} cluster.
+
+image::../../images/csi-arch.png["Architecture of CSI components"]
+
+It is possible to run multiple CSI drivers for different storage backends. 
+Each driver needs its own external controllers' deployment and DaemonSet 
+with the driver and CSI registrar.
+

modules/persistent-storage-csi-driver-daemonset.adoc

@@ -0,0 +1,22 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent-storage/persistent-storage-csi.adoc
+
+[id="csi-driver-daemonset-{context}"]
+= CSI Driver DaemonSet
+
+The CSI driver DaemonSet runs a pod on every node that allows 
+{product-title} to mount storage provided by the CSI driver to the node 
+and use it in user workloads (pods) as persistent volumes (PVs). The pod 
+with the CSI driver installed contains the following containers: 
+
+* A CSI driver registrar, which registers the CSI driver into the 
+`openshift-node` service running on the node. The `openshift-node` process 
+running on the node then directly connects with the CSI driver using the 
+UNIX Domain Socket available on the node.
+* A CSI driver.
+
+The CSI driver deployed on the node should have as few credentials to the 
+storage backend as possible. {product-title} will only use the node plug-in 
+set of CSI calls such as `NodePublish`/`NodeUnpublish` and 
+`NodeStage`/`NodeUnstage`, if these calls are implemented.

modules/persistent-storage-csi-dynamic-provisioning.adoc

@@ -0,0 +1,34 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent-storage/persistent-storage-csi.adoc
+
+[id="csi-dynamic-provisioning-{context}"]
+= Dynamic Provisioning
+
+Dynamic provisioning of persistent storage depends on the capabilities of 
+the CSI driver and underlying storage backend. The provider of the CSI 
+driver should document how to create a StorageClass in {product-title} and 
+the parameters available for configuration. 
+
+As seen in the OpenStack Cinder example, you can deploy this StorageClass 
+to enable dynamic provisioning. 
+ 
+.Procedure
+
+* Create a default storage class that ensures all PVCs that do not require
+any special storage class are provisioned by the installed CSI driver.
++
+[source,shell]
+----
+# oc create -f - << EOF
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: cinder
+  annotations:
+    storageclass.kubernetes.io/is-default-class: "true"
+provisioner: csi-cinderplugin
+parameters:
+EOF
+----
+

modules/persistent-storage-csi-example-deployment.adoc

@@ -0,0 +1,261 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent-storage/persistent-storage-csi.adoc
+
+[id="csi-example-deployment-{context}"]
+= Example CSI deployment
+
+Since {product-title} does not ship with any CSI driver installed, this 
+example shows how to deploy a community driver for OpenStack Cinder in 
+{product-title}.
+
+.Procedure
+
+. Create a new project where the CSI components will run, and then create
+a new service account to run the components. An explicit node selector is 
+used to run the Daemonset with the CSI driver also on master nodes.
++
+----
+# oc adm new-project csi --node-selector=""
+Now using project "csi" on server "https://example.com:8443".
+
+# oc create serviceaccount cinder-csi
+serviceaccount "cinder-csi" created
+
+# oc adm policy add-scc-to-user privileged system:serviceaccount:csi:cinder-csi
+scc "privileged" added to: ["system:serviceaccount:csi:cinder-csi"]
+----
+
+. Apply this YAML file to create the deployment with the external CSI 
+attacher and provisioner and DaemonSet with the CSI driver.
++
+[source,yaml]
+----
+# This YAML file contains all API objects that are necessary to run Cinder CSI
+# driver.
+#
+# In production, this needs to be in separate files, e.g. service account and
+# role and role binding needs to be created once.
+#
+# It server as an example how to use external attacher and external provisioner
+# images shipped with {product-title} with a community CSI driver.
+
+kind: ClusterRole
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: cinder-csi-role
+rules:
+  - apiGroups: [""]
+    resources: ["persistentvolumes"]
+    verbs: ["create", "delete", "get", "list", "watch", "update", "patch"]
+  - apiGroups: [""]
+    resources: ["events"]
+    verbs: ["create", "get", "list", "watch", "update", "patch"]
+  - apiGroups: [""]
+    resources: ["persistentvolumeclaims"]
+    verbs: ["get", "list", "watch", "update", "patch"]
+  - apiGroups: [""]
+    resources: ["nodes"]
+    verbs: ["get", "list", "watch", "update", "patch"]
+  - apiGroups: ["storage.k8s.io"]
+    resources: ["storageclasses"]
+    verbs: ["get", "list", "watch"]
+  - apiGroups: ["storage.k8s.io"]
+    resources: ["volumeattachments"]
+    verbs: ["get", "list", "watch", "update", "patch"]
+  - apiGroups: [""]
+    resources: ["configmaps"]
+    verbs: ["get", "list", "watch", "create", "update", "patch"]
+
+---
+
+kind: ClusterRoleBinding
+apiVersion: rbac.authorization.k8s.io/v1
+metadata:
+  name: cinder-csi-role
+subjects:
+  - kind: ServiceAccount
+    name: cinder-csi
+    namespace: csi
+roleRef:
+  kind: ClusterRole
+  name: cinder-csi-role
+  apiGroup: rbac.authorization.k8s.io
+
+---
+apiVersion: v1
+data:
+  cloud.conf: W0dsb2JhbF0KYXV0aC11cmwgPSBodHRwczovL2V4YW1wbGUuY29tOjEzMDAwL3YyLjAvCnVzZXJuYW1lID0gYWxhZGRpbgpwYXNzd29yZCA9IG9wZW5zZXNhbWUKdGVuYW50LWlkID0gZTBmYTg1YjZhMDY0NDM5NTlkMmQzYjQ5NzE3NGJlZDYKcmVnaW9uID0gcmVnaW9uT25lCg== <1>
+kind: Secret
+metadata:
+  creationTimestamp: null
+  name: cloudconfig
+---
+kind: Deployment
+apiVersion: apps/v1
+metadata:
+  name: cinder-csi-controller
+spec:
+  replicas: 2
+  selector:
+    matchLabels:
+      app: cinder-csi-controllers
+  template:
+    metadata:
+      labels:
+        app: cinder-csi-controllers
+    spec:
+      serviceAccount: cinder-csi
+      containers:
+        - name: csi-attacher
+          image: registry.redhat.io/openshift/csi-attacher:v4.0
+          args:
+            - "--v=5"
+            - "--csi-address=$(ADDRESS)"
+            - "--leader-election"
+            - "--leader-election-namespace=$(MY_NAMESPACE)"
+            - "--leader-election-identity=$(MY_NAME)"
+          env:
+            - name: MY_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.name
+            - name: MY_NAMESPACE
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.namespace
+            - name: ADDRESS
+              value: /csi/csi.sock
+          volumeMounts:
+            - name: socket-dir
+              mountPath: /csi
+        - name: csi-provisioner
+          image: registry.redhat.io/openshift/csi-provisioner:v4.0
+          args:
+            - "--v=5"
+            - "--provisioner=csi-cinderplugin"
+            - "--csi-address=$(ADDRESS)"
+          env:
+            - name: ADDRESS
+              value: /csi/csi.sock
+          volumeMounts:
+            - name: socket-dir
+              mountPath: /csi
+        - name: cinder-driver
+          image: k8scloudprovider/cinder-csi-plugin:v0.3.0
+          command: [ "/bin/cinder-csi-plugin" ]
+          args:
+            - "--nodeid=$(NODEID)"
+            - "--endpoint=unix://$(ADDRESS)"
+            - "--cloud-config=/etc/cloudconfig/cloud.conf"
+          env:
+            - name: NODEID
+              valueFrom:
+                fieldRef:
+                  fieldPath: spec.nodeName
+            - name: ADDRESS
+              value: /csi/csi.sock
+          volumeMounts:
+            - name: socket-dir
+              mountPath: /csi
+            - name: cloudconfig
+              mountPath: /etc/cloudconfig
+      volumes:
+        - name: socket-dir
+          emptyDir:
+        - name: cloudconfig
+          secret:
+            secretName: cloudconfig
+
+---
+
+kind: DaemonSet
+apiVersion: apps/v1
+metadata:
+  name: cinder-csi-ds
+spec:
+  selector:
+    matchLabels:
+      app: cinder-csi-driver
+  template:
+    metadata:
+      labels:
+        app: cinder-csi-driver
+    spec:
+      <2>
+      serviceAccount: cinder-csi
+      containers:
+        - name: csi-driver-registrar
+          image: registry.redhat.io/openshift/csi-driver-registrar:v4.0
+          securityContext:
+            privileged: true
+          args:
+            - "--v=5"
+            - "--csi-address=$(ADDRESS)"
+          env:
+            - name: ADDRESS
+              value: /csi/csi.sock
+            - name: KUBE_NODE_NAME
+              valueFrom:
+                fieldRef:
+                  fieldPath: spec.nodeName
+          volumeMounts:
+            - name: socket-dir
+              mountPath: /csi
+        - name: cinder-driver
+          securityContext:
+            privileged: true
+            capabilities:
+              add: ["SYS_ADMIN"]
+            allowPrivilegeEscalation: true
+          image: k8scloudprovider/cinder-csi-plugin:v0.3.0
+          command: [ "/bin/cinder-csi-plugin" ]
+          args:
+            - "--nodeid=$(NODEID)"
+            - "--endpoint=unix://$(ADDRESS)"
+            - "--cloud-config=/etc/cloudconfig/cloud.conf"
+          env:
+            - name: NODEID
+              valueFrom:
+                fieldRef:
+                  fieldPath: spec.nodeName
+            - name: ADDRESS
+              value: /csi/csi.sock
+          volumeMounts:
+            - name: socket-dir
+              mountPath: /csi
+            - name: cloudconfig
+              mountPath: /etc/cloudconfig
+            - name: mountpoint-dir
+              mountPath: /var/lib/origin/openshift.local.volumes/pods/
+              mountPropagation: "Bidirectional"
+            - name: cloud-metadata
+              mountPath: /var/lib/cloud/data/
+            - name: dev
+              mountPath: /dev
+      volumes:
+        - name: cloud-metadata
+          hostPath:
+            path: /var/lib/cloud/data/
+        - name: socket-dir
+          hostPath:
+            path: /var/lib/kubelet/plugins/csi-cinderplugin
+            type: DirectoryOrCreate
+        - name: mountpoint-dir
+          hostPath:
+            path: /var/lib/origin/openshift.local.volumes/pods/
+            type: Directory
+        - name: cloudconfig
+          secret:
+            secretName: cloudconfig
+        - name: dev
+          hostPath:
+            path: /dev
+----
+<1> Replace with `cloud.conf` for your OpenStack deployment.
+For example, the Secret can be generated using the `oc create secret 
+generic cloudconfig --from-file cloud.conf --dry-run -o yaml`.
+<2> Optionally, add `nodeSelector` to the CSI driver pod template to 
+configure the nodes on which the CSI driver starts. Only nodes matching 
+the selector run pods that use volumes that are served by the CSI driver. 
+Without `nodeSelector`, the driver runs on all nodes in the cluster.

modules/persistent-storage-csi-external-controllers.adoc

@@ -0,0 +1,41 @@
+// Module included in the following assemblies:
+//
+// * storage/persistent-storage/persistent-storage-csi.adoc
+
+[id="external-csi-contollers-{context}"]
+= External CSI controllers
+
+External CSI Controllers is a deployment that deploys one or more pods 
+with three containers:
+
+* An external CSI attacher container translates `attach` and `detach` 
+calls from {product-title} to respective `ControllerPublish` and 
+`ControllerUnpublish` calls to the CSI driver.
+* An external CSI provisioner container that translates `provision` and 
+`delete` calls from {product-title} to respective `CreateVolume` and 
+`DeleteVolume` calls to the CSI driver.
+* A CSI driver container
+
+The CSI attacher and CSI provisioner containers communicate with the CSI 
+driver container using UNIX Domain Sockets, ensuring that no CSI 
+communication leaves the pod. The CSI driver is not accessible from 
+outside of the pod.
+
+[NOTE]
+====
+`attach`, `detach`, `provision`, and `delete` operations typically require 
+the CSI driver to use credentials to the storage backend. Run the CSI 
+controller pods on infrastructure nodes so the credentials are never leaked
+to user processes, even in the event of a catastrophic security breach 
+on a compute node.
+====
+
+[NOTE]
+====
+The external attacher must also run for CSI drivers that do not support 
+third-party `attach` or `detach` operations. The external attacher will 
+not issue any `ControllerPublish` or `ControllerUnpublish` operations to 
+the CSI driver. However, it still must run to implement the necessary 
+{product-title} attachment API.
+====
+

modules/persistent-storage-csi-mysql-example.adoc

@@ -0,0 +1,31 @@
+// Module included in the following assemblies
+//
+// * storage/persistent-storage/persistent-storage-csi.adoc
+
+[id="csi-example-usage-{context}"]
+= Example using the CSI driver
+
+The following example installs a default MySQL template without any
+changes to the template.
+
+.Prerequisites
+
+* The CSI driver has been deployed.
+* A StorageClass has been created for dynamic provisioning.
+
+.Procedure
+
+* Create the MySQL template:
++
+----
+# oc new-app mysql-persistent
+--> Deploying template "openshift/mysql-persistent" to project default
+...
+
+# oc get pvc
+NAME              STATUS    VOLUME                                   CAPACITY    
+ACCESS MODES   STORAGECLASS   AGE
+mysql             Bound     kubernetes-dynamic-pv-3271ffcb4e1811e8   1Gi         
+RWO            cinder         3s
+----
+

storage/persistent-storage/persistent-storage-csi.adoc

@@ -0,0 +1,36 @@
+[id="persistent-storage-using-csi"]
+= Persistent storage using the Container Storage Interface (CSI)
+include::modules/common-attributes.adoc[]
+:context: persistent-storage-csi
+
+toc::[]
+
+The Container Storage Interface (CSI) allows {product-title} to consume 
+storage from  storage backends that implement the 
+link:https://github.com/container-storage-interface/spec[CSI interface] 
+as persistent storage.
+
+:FeatureName: Container Storage Interface
+include::modules/technology-preview.adoc[leveloffset=+0]
+
+[NOTE]
+====
+{product-title} does not ship with any CSI drivers. It is recommended to 
+use the CSI drivers provided by 
+link:https://kubernetes-csi.github.io/docs/Drivers.html[community or storage vendors].
+
+{product-title} {product-version} supports version 0.4.0 of the 
+link:https://github.com/container-storage-interface/spec[CSI specification].
+====
+
+include::modules/persistent-storage-csi-architecture.adoc[leveloffset=+1]
+
+include::modules/persistent-storage-csi-external-controllers.adoc[leveloffset=+2]
+
+include::modules/persistent-storage-csi-driver-daemonset.adoc[leveloffset=+2]
+
+include::modules/persistent-storage-csi-example-deployment.adoc[leveloffset=+1]
+
+include::modules/persistent-storage-csi-dynamic-provisioning.adoc[leveloffset=+1]
+
+include::modules/persistent-storage-csi-mysql-example.adoc[leveloffset=+1]