diff --git a/modules/op-about-pipelinerun.adoc b/modules/op-about-pipelinerun.adoc new file mode 100644 index 0000000000..9804fcaab1 --- /dev/null +++ b/modules/op-about-pipelinerun.adoc @@ -0,0 +1,38 @@ +// This module is included in the following assembly: +// +// *openshift_pipelines/op-creating-applications-with-cicd-pipelines.adoc + +[id="about-pipelinerun_{context}"] += PipelineRun + +A _PipelineRun_ instantiates a Pipeline for execution with specific inputs, outputs, and execution parameters on a cluster. A corresponding TaskRun is created for each Task automatically in the PipelineRun. + +All the Tasks in the Pipeline are executed in the defined sequence until all Tasks are successful or a Task fails. The `status` field tracks and stores the progress of each TaskRun in the PipelineRun for monitoring and auditing purpose. + +Here is an example of a PipelineRun to run a Pipeline `build-and-deploy` with relevant resources and parameters: +[source, yaml] +---- +apiVersion: tekton.dev/v1beta1 <1> +kind: PipelineRun <2> +metadata: + name: build-deploy-api-pipelinerun <3> +spec: + pipelineRef: + name: build-and-deploy <4> + resources: <5> + - name: git-repo + resourceRef: + name: api-repo + - name: image + resourceRef: + name: api-image + params: <6> + - name: deployment-name + value: vote-api +---- +<1> PipelineRun API version `v1beta1`. +<2> Specifies the type of Kubernetes object. In this example, `PipelineRun`. +<3> Unique name to identify this PipelineRun. +<4> Name of the Pipeline to be run. In this example, `build-and-deploy`. +<5> Name of the resources which will be required to run the Pipeline `build-and-deploy`. +<6> Specifies the list of parameters required to run the Pipeline. diff --git a/modules/op-about-pipelines.adoc b/modules/op-about-pipelines.adoc new file mode 100644 index 0000000000..82ae4a8b6f --- /dev/null +++ b/modules/op-about-pipelines.adoc @@ -0,0 +1,65 @@ +// Ths module is included in the following assembly: +// +// *openshift_pipelines/op-creating-applications-with-cicd-pipelines.adoc + +[id="about-pipelines_{context}"] += Pipelines + +A _Pipeline_ is a collection of Tasks arranged in a specific order of execution. You can define a CI/CD workflow for your application using Pipelines containing one or more Tasks. + +A Pipeline definition consists of a number of fields or attributes, which together enable the Pipeline to accomplish a specific goal. Each Pipeline definition must contain at least one Task, which ingests specific inputs and produces specific outputs. The Pipeline definition can also optionally include Conditions, Workspaces, Parameters, or Resources depending on the application requirements. + +Here is a code snippet of a Pipeline `build-and-deploy`, which builds an application image from a Git repository using `buildah` ClusterTask: + +[source, yaml] +---- +apiVersion: tekton.dev/v1beta1 <1> +kind: Pipeline <2> +metadata: <3> + name: build-and-deploy +spec: <4> + resources: <5> + - name: git-repo + type: git + - name: image + type: image + params: <6> + - name: deployment-name + type: string + description: name of the deployment to be patched + tasks: <7> + - name: build-image <8> + taskRef: + name: buildah + kind: ClusterTask + resources: + inputs: + - name: source + resource: git-repo + outputs: + - name: image + resource: image + params: + - name: TLSVERIFY + value: "false" + - name: apply-manifests <9> + taskRef: + name: apply-manifests + resources: + inputs: + - name: source + resource: git-repo + runAfter: <10> + - build-image +... +---- +<1> Pipeline API version `v1beta1`. +<2> Specifies the type of Kubernetes object. In this example, `Pipeline`. +<3> Unique name of this Pipeline. +<4> Specifies the definition and structure of the Pipeline. +<5> Set of resources required to run the Tasks specified in the Pipeline. +<6> Parameters used across all the Tasks in the Pipeline. +<7> Specifies the list of Tasks used in the Pipeline. In this example, two Tasks are specified: `build-image` and `apply-manifests`. +<8> Task `build-image`, which uses the `buildah` ClusterTask to build application images from a given Git repository. +<9> Task `apply-manifests`, which uses a user-defined Task with the same name. +<10> Specifies the sequence in which Tasks are run in a Pipeline. In this example, the `apply-manifests` Task is run only after the `build-image` Task is completed. diff --git a/modules/op-about-taskrun.adoc b/modules/op-about-taskrun.adoc new file mode 100644 index 0000000000..814289838b --- /dev/null +++ b/modules/op-about-taskrun.adoc @@ -0,0 +1,38 @@ +// Ths module is included in the following assembly: +// +// *openshift_pipelines/op-creating-applications-with-cicd-pipelines.adoc + +[id="about-taskrun_{context}"] += TaskRun + +A _TaskRun_ instantiates a Task for execution with specific inputs, outputs, and execution parameters on a cluster. It can be invoked on its own or as part of a PipelineRun. + +A Task consists of one or more Steps that execute container images, and each container image performs a specific piece of build work. A TaskRun executes the Steps in a Task in the specified order, until all Steps execute successfully or a failure occurs. + +Here is an example of a TaskRun to run a Task `apply-manifests` with relevant input parameters: +[source, yaml] +---- +apiVersion: tekton.dev/v1beta1 <1> +kind: TaskRun <2> +metadata: + name: apply-manifests-taskrun <3> +spec: <4> + resources: <5> + inputs: + - name: source + resourceRef: + name: git-resource-name + params: <6> + - name: manifest_dir + value: directory-name + taskRef: <7> + name: apply-manifests + kind: Task +---- +<1> TaskRun API version `v1beta1`. +<2> Specifies the type of Kubernetes object. In this example, `TaskRun`. +<3> Unique name to identify this TaskRun. +<4> Definition of the TaskRun. For this TaskRun, a set of input resources, parameters, and Tasks are specified. +<5> Resources required to run the Tasks. For this TaskRun, `git-resource-name` is referred as an input resource. +<6> List of parameters required to run the Task. +<7> Name of the Task reference used for this TaskRun. This TaskRun executes the Task `apply-manifests`. diff --git a/modules/op-about-tasks.adoc b/modules/op-about-tasks.adoc index b6c18a28ba..a7a4ae6de3 100644 --- a/modules/op-about-tasks.adoc +++ b/modules/op-about-tasks.adoc @@ -3,11 +3,11 @@ // *openshift_pipelines/creating-applications-with-cicd-pipelines.adoc [id="about-tasks_{context}"] -= About Tasks += Tasks _Tasks_ are the building blocks of a Pipeline and consist of sequentially executed Steps. Steps are a series of commands that achieve a specific goal, such as building an image. -Every Task runs as a Pod and each Step runs in its own container within the same Pod. Because Steps run within the same Pod, they have access to the same volumes for caching files, configmaps, and secrets. +Every Task runs as a Pod and each Step runs in its own container within the same Pod. Because Steps run within the same Pod, they have access to the same volumes for caching files, ConfigMaps, and Secrets. A Task uses `inputs` parameters, such as a Git resource, and `outputs` parameters, such as an image in a registry, to interact with other Tasks. They are reusable and can be used in multiple Pipelines. @@ -15,11 +15,11 @@ Here is an example of a Maven Task with a single Step to build a Maven-based Jav [source,yaml] ---- -apiVersion: tekton.dev/v1beta1 -kind: Task +apiVersion: tekton.dev/v1beta1 <1> +kind: Task <2> metadata: - name: maven-build -spec: + name: maven-build <3> +spec: <4> resources: inputs: - name: workspace-git @@ -33,6 +33,11 @@ spec: args: - install ---- +<1> Task API version `v1beta1`. +<2> Specifies the type of Kubernetes object. In this example, `Task`. +<3> Unique name of this Task. +<4> Lists the Task Steps along with input and output resources. + This Task starts the Pod and runs a container inside that Pod using the `maven:3.6.0-jdk-8-slim` image to run the specified commands. It receives an input directory called `workspace-git` that contains the source code of the application. The Task only declares the placeholder for the Git repository, it does not specify which Git repository to use. This allows Tasks to be reusable for multiple Pipelines and purposes. diff --git a/modules/op-about-triggers.adoc b/modules/op-about-triggers.adoc index 616aaf83de..85f28bfff4 100644 --- a/modules/op-about-triggers.adoc +++ b/modules/op-about-triggers.adoc @@ -5,18 +5,103 @@ [id="about-triggers_{context}"] -= About Triggers += Triggers Use Triggers in conjunction with Pipelines to create a full-fledged CI/CD system where the Kubernetes resources define the entire CI/CD execution. Pipeline Triggers capture the external events and process them to extract key pieces of information. Mapping this event data to a set of predefined parameters triggers a series of tasks that can then create and deploy Kubernetes resources. -For example, you define a CI/CD workflow using {pipelines-title} for your application. The PipelineRun must start for any new changes to take effect in the application repository. Triggers automate this process by capturing and processing any change events, and by triggering a PipelineRun that deploys the new image with the latest changes. +For example, you define a CI/CD workflow using {pipelines-title} for your application. The PipelineRun must start for any new changes to take effect in the application repository. Triggers automate this process by capturing and processing any change events and by triggering a PipelineRun that deploys the new image with the latest changes. Triggers consist of the following main components that work together to form a reusable, decoupled, and self-sustaining CI/CD system: * _EventListeners_ provide endpoints, or an event sink, that listen for incoming HTTP-based events with a JSON payload. The EventListener performs lightweight event processing on the payload using Event Interceptors, which identify the type of payload and optionally modify it. Currently, Pipeline Triggers support four types of Interceptors: Webhook Interceptors, GitHub Interceptors, GitLab Interceptors, and Common Expression Language (CEL) Interceptors. * _TriggerBindings_ extract the fields from an event payload and store them as parameters. -* _TriggerTemplates_ specify how to use the parameterized data from the TriggerBindings. A TriggerTemplate defines a resource template that receives input from the TriggerBindings, while then performing a series of actions that result in creation of new PipelineResources and initiation of a new PipelineRun. +* _TriggerTemplates_ specify how to use the parameterized data from the TriggerBindings. A TriggerTemplate defines a resource template that receives input from the TriggerBindings, and then performs a series of actions that result in creation of new PipelineResources and initiation of a new PipelineRun. EventListeners tie the concepts of TriggerBindings and TriggerTemplates together. The EventListener listens for the incoming event, handles basic filtering using Interceptors, extracts data using TriggerBindings, and then processes this data to create Kubernetes resources using TriggerTemplates. //image::op-triggers.png[] + +Here is a code snippet of the `vote-app-binding` TriggerBinding, which extracts the Git repository information from the received event payload: + +[source,yaml] +---- +apiVersion: triggers.tekton.dev/v1alpha1 <1> +kind: TriggerBinding <2> +metadata: + name: vote-app-binding <3> +spec: + params: <4> + - name: git-repo-url + value: $(body.repository.url) + - name: git-repo-name + value: $(body.repository.name) + - name: git-revision + value: $(body.head_commit.id) +---- + +<1> TriggerBinding API version `v1alpha1`. +<2> Specifies the type of Kubernetes object. In this example, `TriggerBinding`. +<3> Unique name to identify this TriggerBinding. +<4> List of parameters which will be extracted from the received event payload and passed to the TriggerTemplate. In this example, the Git repository URL, name, and revision are extracted from the body of the event payload. + + +Here is a code snippet of a `vote-app-template` TriggerTemplate, which creates Pipeline Resources from the Git repository information received from the TriggerBinding: +[source,yaml] +---- +apiVersion: triggers.tekton.dev/v1alpha1 <1> +kind: TriggerTemplate <2> +metadata: + name: vote-app-template <3> +spec: + params: <4> + - name: git-repo-url + description: The git repository url + - name: git-revision + description: The git revision + default: master + - name: git-repo-name + description: The name of the deployment to be created / patched + resourcetemplates: <5> + - apiVersion: tekton.dev/v1alpha1 + kind: PipelineResource <6> + metadata: + name: $(params.git-repo-name)-git-repo-$(uid) + spec: + type: git + params: + - name: revision + value: $(params.git-revision) + - name: url + value: $(params.git-repo-url) +... +---- + +<1> TriggerTemplate API version `v1alpha1`. +<2> Specifies the type of Kubernetes object. In this example, `TriggerTemplate`. +<3> Unique name to identify this TriggerTemplate. +<4> Parameters supplied by the TriggerBinding or EventListerner. +<5> List of Resource templates created for the Pipeline from the parameters received in the TriggerBinding or EventListener. +<6> A Pipeline Resource of type `git` is created using the parameters `git-repo-name`, `git-repo-url`, and `git-revision`. + +Here is an example of an EventListener which uses `vote-app-binding` TriggerBinding and `vote-app-template` TriggerTemplate to process incoming events. + +[source,yaml] +---- +apiVersion: triggers.tekton.dev/v1alpha1 <1> +kind: EventListener <2> +metadata: + name: vote-app <3> +spec: + serviceAccountName: pipeline <4> + triggers: + - bindings: <5> + - name: vote-app-binding + template: <6> + name: vote-app-template +---- +<1> EventListener API version `v1alpha1`. +<2> Specifies the type of Kubernetes object. In this example, `EventListener`. +<3> Unique name to identify this EventListener. +<4> Service account name to be used. +<5> Name of the TriggerBinding to be used for this EventListener. +<6> Name of the Triggertemplate to be used for this Eventlistener. diff --git a/modules/op-about-workspace.adoc b/modules/op-about-workspace.adoc new file mode 100644 index 0000000000..c74ca8b730 --- /dev/null +++ b/modules/op-about-workspace.adoc @@ -0,0 +1,96 @@ +// This module is included in the following assembly: +// +// *openshift_pipelines/creating-applications-with-cicd-pipelines.adoc + +[id="about-workspaces_{context}"] += Workspaces + +Workspaces declare shared storage volumes that a Task in a Pipeline needs at runtime. Instead of specifying the actual location of the volumes, Workspaces enable you to declare the filesystem or parts of the filesystem that would be required at runtime. You must provide the specific location details of the volume that is mounted into that Workspace in a TaskRun or a PipelineRun. This separation of volume declaration from runtime storage volumes makes the Tasks reusable, flexible, and independent of the user environment. + +Listed below are the various ways in which you can use Workspaces: + +* Store Task inputs and outputs +* Share data among Tasks +* Use it as a mount point for credentials held in Secrets +* Use it as a mount point for configurations held in ConfigMaps +* Use it as a mount point for common tools shared by an organization +* Create a cache of build artifacts that speed up jobs + +You can specify Workspaces in the TaskRun or PipelineRun using: + +* A read-only ConfigMaps or Secret +* An existing PersistentVolumeClaim shared with other Tasks +* A PersistentVolumeClaim from a provided VolumeClaimTemplate +* An emptyDir that is discarded when the TaskRun completes + +Here is a code snippet of the `build-and-deploy` Pipeline which declares a `shared-workspace` Workspace for the Tasks, `build-image` and `apply-manifests`, defined in the Pipeline. + +[source,yaml] +---- +apiVersion: tekton.dev/v1beta1 +kind: Pipeline +metadata: + name: build-and-deploy +spec: + workspaces: <1> + - name: shared-workspace + params: +... + tasks: <2> + - name: build-image + taskRef: + name: buildah + kind: ClusterTask + params: + - name: TLSVERIFY + value: "false" + - name: IMAGE + value: $(params.IMAGE) + workspaces: <3> + - name: source <4> + workspace: shared-workspace <5> + runAfter: + - fetch-repository + - name: apply-manifests + taskRef: + name: apply-manifests + workspaces: <6> + - name: source + workspace: shared-workspace + runAfter: + - build-image +... +---- +<1> List of Workspaces shared between the Tasks defined in the Pipeline. A Pipeline can define as many Workspaces as required. In this example, only one Workspace named `shared-workspace` is declared. +<2> Definition of Tasks used in the Pipeline. This snippet defines two Tasks, `build-image` and `apply-manifests`, which share a common Workspace. +<3> List of Workspaces used in the `build-image` Task. A Task definition can include as many Workspaces as it requires. However, it is recommended that a Task uses at most one writable Workspace. +<4> Name that uniquely identifies the Workspace used in the Task. This Task uses one Workspace named `source`. +<5> Name of the Pipeline Workspace used by the Task. Note that the Workspace `source` in turn uses Pipeline Workspace `shared-workspace`. +<6> List of Workspaces used in `apply-manifests` Task. Note that this Task shares the `source` Workspace with `build-image` Task. + +Here is a code snippet of the `build-deploy-api-pipelinerun` PipelineRun, which uses a PersistentVolumeClaim for defining the storage volume for the `shared-workspace` Workspace used in the `build-and-deploy` Pipeline. + +[source,yaml] +---- +apiVersion: tekton.dev/v1beta1 +kind: PipelineRun +metadata: + name: build-deploy-api-pipelinerun +spec: + pipelineRef: + name: build-and-deploy + resources: + - name: image + resourceRef: + name: api-image + params: +... + + workspaces: <1> + - name: shared-workspace <2> + persistentvolumeclaim: + claimName: source-pvc <3> +---- +<1> Specifies the list of Pipeline Workspaces for which volume binding will be provided in the PipelineRun. +<2> The name of the Workspace in the Pipeline for which the volume is being provided. +<3> Specifies the name of a predefined PersistentVolumeClaim, which will be attached to the Workspace. In this example, an existing `source-pvc` PersistentVolumeClaim is attached with the `shared-workspace` Workspace. diff --git a/modules/op-pipelines-concepts.adoc b/modules/op-pipelines-concepts.adoc index 8f6fa450e0..5be71d93bc 100644 --- a/modules/op-pipelines-concepts.adoc +++ b/modules/op-pipelines-concepts.adoc @@ -17,6 +17,8 @@ TaskRun:: A TaskRun is automatically created by a PipelineRun for each Task in a PipelineResource:: A PipelineResource is an object that is used as an input and output for Pipeline Tasks. For example, if an input is a Git repository and an output is a container image built from that Git repository, these are both classified as PipelineResources. PipelineResources currently support Git resources, Image resources, Cluster resources, Storage Resources and CloudEvent resources. +Workspace:: A Workspace is a storage volume that a Task requires at runtime to receive input or provide output. A Task or Pipeline declares the Workspace, and a TaskRun or PipelineRun provides the actual location of the storage volume, which mounts on the declared Workspace. This makes the Task flexible, reusable, and allows the Workspaces to be shared across multiple Tasks. + Trigger:: A Trigger captures an external event, such as a Git pull request and processes the event payload to extract key pieces of information. This extracted information is then mapped to a set of predefined parameters, which trigger a series of tasks that may involve creation and deployment of Kubernetes resources. You can use Triggers along with Pipelines to create full-fledged CI/CD systems where the execution is defined entirely through Kubernetes resources. Condition:: A Condition refers to a validation or check, which is executed before a Task is run in your Pipeline. Conditions are like `if` statements which perform logical tests, with a return value of `True` or `False`. A Task is executed if all Conditions return `True`, but if any of the Conditions fail, the Task and all subsequent Tasks are skipped. You can use Conditions in your Pipeline to create complex workflows covering multiple scenarios. diff --git a/pipelines/creating-applications-with-cicd-pipelines.adoc b/pipelines/creating-applications-with-cicd-pipelines.adoc index e2893125bc..fee74d35b6 100644 --- a/pipelines/creating-applications-with-cicd-pipelines.adoc +++ b/pipelines/creating-applications-with-cicd-pipelines.adoc @@ -36,8 +36,6 @@ This section uses the `pipelines-tutorial` example to demonstrate the preceding include::modules/op-creating-project-and-checking-pipeline-service-account.adoc[leveloffset=+1] -include::modules/op-about-tasks.adoc[leveloffset=+1] - include::modules/op-creating-pipeline-tasks.adoc[leveloffset=+1] include::modules/op-defining-and-creating-pipelineresources.adoc[leveloffset=+1] @@ -46,8 +44,6 @@ include::modules/op-assembling-a-pipeline.adoc[leveloffset=+1] include::modules/op-running-a-pipeline.adoc[leveloffset=+1] -include::modules/op-about-triggers.adoc[leveloffset=+1] - include::modules/op-adding-triggers.adoc[leveloffset=+1] include::modules/op-creating-webhooks.adoc[leveloffset=+1] diff --git a/pipelines/understanding-openshift-pipelines.adoc b/pipelines/understanding-openshift-pipelines.adoc index 5ca9ac3325..0830bb50ee 100644 --- a/pipelines/understanding-openshift-pipelines.adoc +++ b/pipelines/understanding-openshift-pipelines.adoc @@ -22,8 +22,27 @@ include::modules/technology-preview.adoc[leveloffset=+1] * You can use {pipelines-title} to build images with Kubernetes tools such as Source-to-Image (S2I), Buildah, Buildpacks, and Kaniko that are portable across any Kubernetes platform. * You can use the {product-title} Developer Console to create Tekton resources, view logs of Pipeline runs, and manage pipelines in your {product-title} namespaces. +//Main Concept Topic include::modules/op-pipelines-concepts.adoc[leveloffset=+1] +[id="op-detailed-concepts"] +== Detailed OpenShift Pipeline Concepts +Given below is a detailed view of the various Pipeline concepts. + +//About Tasks +include::modules/op-about-tasks.adoc[leveloffset=+2] +//About TaskRun +include::modules/op-about-taskrun.adoc[leveloffset=+2] +//About Pipelines +include::modules/op-about-pipelines.adoc[leveloffset=+2] +//About PipelineRun +include::modules/op-about-pipelinerun.adoc[leveloffset=+2] +//About Workspace +include::modules/op-about-workspace.adoc[leveloffset=+2] +//About Triggers +include::modules/op-about-triggers.adoc[leveloffset=+2] + + .Additional resources * For information on installing Pipelines, see xref:../pipelines/installing-pipelines.adoc#installing-pipelines[Installing OpenShift Pipelines].