TEP-0087: Custom Tasks Graduation

This change adds the Tekton Enhancement Proposal for `Custom Tasks` Graduation.

Today, `Custom Tasks` shared by the Tekton community are all *experimental*, and we don't have a process to promote
them beyond *experimental*.

As such:
- Users can't depend on the `Custom Tasks` because they can change any time.
- Contributors don't have a process to stabilize their `Custom Tasks` or integrate them to the *Tekton Pipelines API*.

In this TEP, we aim to:
- Define the graduation path for `Custom Tasks` from *experimental* to *integrated*.
- Provide infrastructure for `Custom Tasks` that are officially supported by Tekton.

`Custom Tasks` will have four stability levels in their graduation path:

1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we are iterating on
them.
2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
3. *Packaged*: `Custom Tasks` are shipped with *Tekton Pipelines* releases, so are available for use out of the box.
4. *Integrated*: `Custom Tasks`' functionalities are natively supported in the *Tekton Pipelines API*.

See also [TEP-0002: Enable Custom Tasks](https://github.com/tektoncd/community/blob/main/teps/0002-custom-tasks.md).

[api-policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#alpha-beta-and-ga

teps/0087-custom-tasks-graduation.md

@@ -0,0 +1,351 @@
+---
+status: proposed
+title: Custom Tasks Graduation
+creation-date: '2021-09-28'
+last-updated: '2021-09-30'
+authors:
+- '@jerop'
+see-also:
+- TEP-0002
+---
+
+# TEP-0087: Custom Tasks Graduation
+
+<!-- toc -->
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+  - [Use Cases](#use-cases)
+  - [Requirements](#requirements)
+- [Proposal](#proposal)
+  - [Experimental](#experimental)
+    - [Admission Requirements](#admission-requirements)
+    - [Admission Process](#admission-process)
+  - [Stable](#stable)
+    - [Graduation Requirements](#graduation-requirements)
+    - [Graduation Process](#graduation-process)
+  - [Packaged](#packaged)
+    - [Graduation Requirements](#graduation-requirements-1)
+    - [Graduation Process](#graduation-process-1)
+  - [Integrated](#integrated)
+    - [Graduation Requirements](#graduation-requirements-2)
+    - [Graduation Process](#graduation-process-2)
+- [Design Evaluation](#design-evaluation)
+    - [Simplicity](#simplicity)
+    - [Reusability](#reusability)
+    - [Flexibility](#flexibility)
+- [Alternatives](#alternatives)
+    - [<em>Experimental</em> to <em>Integrated</em>](#experimental-to-integrated)
+        - [Pros](#pros)
+        - [Cons](#cons)
+    - [All <code>Custom Tasks</code> in One Repository](#all--in-one-repository)
+        - [Pros](#pros-1)
+        - [Cons](#cons-1)
+- [Infrastructure Needed](#infrastructure-needed)
+- [References](#references)
+<!-- /toc -->
+
+## Summary
+
+Today, `Custom Tasks` shared by the Tekton community are all *experimental*, and we don't have a process to promote 
+them beyond *experimental*. 
+
+As such:
+- Users can't depend on the `Custom Tasks` because they can change any time.
+- Contributors don't have a process to stabilize their `Custom Tasks` or integrate them to the *Tekton Pipelines API*.
+
+In this TEP, we aim to:
+- Define the graduation path for `Custom Tasks` from *experimental* to *integrated*.
+- Provide infrastructure for `Custom Tasks` that are officially supported by Tekton.
+
+`Custom Tasks` will have four stability levels in their graduation path:
+
+1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we are iterating on 
+them.
+2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
+3. *Packaged*: `Custom Tasks` are shipped with *Tekton Pipelines* releases, so are available for use out of the box.
+4. *Integrated*: `Custom Tasks`' functionalities are natively supported in the *Tekton Pipelines API*.
+
+## Motivation
+
+As described in [TEP-0002: Enable Custom Tasks][TEP-0002], we introduced `Custom Tasks` that are specified as CRDs that 
+are executed using [`Runs`][runs]. These `Custom Tasks` have reconciling controllers that watch for `Runs` referencing 
+their types and updates their status. `Custom Tasks` provide extensibility in Tekton; it allows users to implement 
+functionality that's not supported in the *Tekton Pipelines API*. 
+
+Hitherto, we have implemented several `Custom Tasks` in the [*tektoncd/experimental*][experimental-repo] repository:
+- [Common Expression Language Custom Tasks][cel-ct] - provides support for Common Expression Language.
+- [Wait Custom Tasks][wait-ct] - enables waiting for some amount of time between `Tasks`.
+- [Task Looping Custom Tasks][tl-ct] - enables running a `Task` in a loop with varying `Parameter` values.
+- [Pipelines in Pipelines Custom Tasks][pip-ct] - enables composing and executing `Pipelines` in `Pipelines`.
+
+These `Custom Tasks` implementations are all *experimental*. Notwithstanding, they are in use in critical projects, such 
+as [Kubeflow Pipelines on Tekton][kubeflow]. Providing stability levels and progression for `Custom Tasks` will enable 
+users to rely on them and empower contributors to safely update them. 
+
+We may decide to promote some `Custom Tasks` to the top level such that they are shipped with *Tekton Pipelines* 
+releases, or even more, that their functionality is supported natively in the *Tekton Pipelines API*. On the other hand, 
+we may decide not to natively support some functionalities provided in some `Custom Tasks` but we need to provide 
+stability guarantees. Thus, we need to provide incremental stability levels and graduation path for `Custom Tasks`. 
+
+### Goals
+
+1. Provide incremental stability levels and graduation path for `Custom Tasks`.
+2. Provide infrastructure for `Custom Tasks` that are officially supported by Tekton.
+
+### Non-Goals
+
+1. Promote `Custom Tasks` feature itself from `alpha` to `beta` - we will address this separately soon.
+2. Provide CLI support for `Custom Tasks` that are officially supported by Tekton - we can explore this later.
+
+### Use Cases
+
+1. As a contributor, I want to stabilize my `Custom Tasks`, and possibly integrate them. As such, I need a process that 
+I can follow with clear requirements for graduation to the next stability level.
+2. As a user, I need to use `Custom Tasks` that I can rely on. The stability expectations can vary depending on the 
+requirements based on the use cases. 
+
+### Requirements
+
+1. Define graduation requirements and processes for `Custom Tasks`.
+2. Provide `Custom Tasks` that are official *Tekton Pipelines* extensions, with the necessary infrastructure and 
+processes (e.g. thorough testing and frequent releases).
+
+## Proposal
+
+`Custom Tasks` will have four stability levels in their graduation path:
+
+1. *Experimental*: `Custom Tasks` can change any time, [*alpha* API policy][api-policy] applies, as we are iterating on
+   them.
+2. *Stable*: `Custom Tasks` are stable, [*beta* API policy][api-policy] applies, and have been approved in a TEP.
+3. *Packaged*: `Custom Tasks` are shipped with *Tekton Pipelines* releases, so are available for use out of the box.
+4. *Integrated*: `Custom Tasks` functionalities are natively supported in the *Tekton Pipelines API*.
+
+The above stability levels are in an increasing order, and their requirements additive with increasing stability.
+
+### Experimental
+
+`Custom Tasks` will start as *experimental* to keep the barrier of sharing integrations low. 
+The [*alpha* API policy][api-policy] applies to *experimental* `Custom Tasks`, meaning the contributors can make any 
+changes at any time as they iterate on the `Custom Tasks`.
+
+#### Admission Requirements
+
+We can consider admitting an *experimental* `Custom Task` if:
+
+1. A `Custom Task` is needed to provide a specific functionality to solve common Continuous Delivery use cases.
+2. At least two contributors are interested in owning the `Custom Task` that provides that functionality.
+
+#### Admission Process
+
+As described in [Tekton's community process][proposing-projects] for proposing new projects:
+
+1. Propose the *experimental* `Custom Task` in the [Tekton API Working Group][api-wg] meeting.
+2. File an issue in the [*tektoncd/community*][community-repo] repository that:
+   1. describes the problem the `Custom Task` would solve.
+   2. listing at least two owners of the `Custom Task`.
+3. When at least two *Tekton Pipelines* owners approve the issue, the contributors can add the `Custom Task` to the 
+[*tektoncd/experimental*][experimental-repo] repository.
+
+### Stable
+
+Tekton will add a new repository - *tektoncd/extensions* - that would contain high quality `Custom Tasks`. These are 
+extensions that users can rely on to access functionality that's not provided in *Tekton Pipelines* directly. The
+*tektoncd/pipelines* owners will be the overall owners of the *tektoncd/extensions* repository, and each `Custom Task` 
+will have its own owners. 
+
+The [*beta* API policy][api-policy] applies to the *stable* `Custom Tasks`, meaning that:
+- Any [backwards incompatible][backwards] changes must be introduced in a backwards compatible manner first, with a 
+deprecation warning in the release notes and migration instructions.
+- Users will be given at least 9 months to migrate before a backward incompatible change is made.
+- Backwards incompatible changes have to be approved by more than half of the `Custom Task`'s owners.
+
+The *tektoncd/extensions* repository will provide the testing infrastructure needed by *stable* `Custom Tasks`. We will 
+make monthly releases of `Custom Tasks` that have had changes since their latest release, which we can automate. 
+Moreover, we will have nightly releases of each `Custom Task` to catch any failures. Each *stable* `Custom Task` will 
+have detailed documentation and multiple examples, and be used in dogfooding if we have use cases for it.
+
+#### Graduation Requirements
+
+We can consider graduating a given *experimental* `Custom Task` to *stable* if it meets the following requirements:
+
+1. It is thoroughly tested.
+2. It has nightly releases.
+3. It provides several examples or samples.
+4. It has detailed documentation for installation and usage.
+5. It is used in dogfooding if we have use cases for it.
+6. It has received feedback from the community (solicited in working group, mailing list or other channels).
+
+#### Graduation Process
+
+The graduation process from *experimental* to *stable* for a given `Custom Task` would be:
+
+1. Validate that the *experimental* `Custom Task` meets the above requirements.
+2. Open a Tekton Enhancement Proposal (TEP) for the functionality provided by the `Custom Task` with:
+   1. A nomination for graduation from *experimental* to *stable*.
+   2. The motivation (goals, use cases and requirements).
+   3. The proposal (syntax, design details, design evaluation and alternatives).
+3. When the *Tekton Pipelines* owners have approved the TEP as implementable:
+   4. Migrate the `Custom Task` from the *Tekton Experimental* repository to the *Tekton Custom Tasks* repository.
+   5. Mark the *experimental* to *stable* TEP as *implemented*
+
+### Packaged
+
+When a given *stable* `Custom Task` is actively used and necessary for common Continuous Delivery use cases, we can
+consider making it *packaged* - meaning that when *Tekton Pipelines* is installed, the `Custom Task` is available with 
+no extra step. That is, the `Custom Task` and its functionality is available out of the box with *Tekton Pipelines*. 
+
+The *packaged* `Custom Task` is shipped with *Tekton Pipelines* releases - every major or minor release of 
+*Tekton Pipelines* would include the `Custom Task`. If an urgent fix is required in the `Custom Task`, a new minor 
+release of the entire *Tekton Pipelines* has to be made.
+
+The *packaged* `Custom Tasks` will be moved from the *tektoncd/extensions* repository to a folder in the 
+*tektoncd/pipelines* repository - thereby, the ownership of the `Custom Task` is transferred to the *Tekton Pipelines* 
+owners.
+
+Not all *stable* `Custom Tasks` have to be *packaged*; a *stable* `Custom Task` that improves user experience but
+is not necessary in *Tekton Pipelines* can stay as a *stable* `Custom Task` in the long term.
+
+#### Graduation Requirements
+
+We can consider graduating a given *stable* `Custom Task` to *packaged* if it meets the following requirements:
+
+1. Its functionality is necessary to solve common Continuous Delivery use cases.
+2. It's actively used by the community and in dogfooding (if we have use cases for it).
+
+#### Graduation Process
+
+The graduation process from *stable* to *packaged* for a given `Custom Task` would be:
+
+1. Validate that the *stable* `Custom Task` meets the above requirements.
+2. Open a Tekton Enhancement Proposal (TEP) for the packaging the `Custom Task` with *Tekton Pipelines* releases with:
+    1. A nomination for graduation from *stable* to *packaged*.
+    2. The motivation (goals, use cases and requirements) - a discussion of how it's been useful and why it's necessary
+       to provide the `Custom Task` out of the box with *Tekton Pipelines*.
+3. When the *Tekton Pipelines* owners have approved the TEP as implementable:
+   1. Move the `Custom Task` to the *tektoncd/pipeline*, transferring the ownership to the *Tekton Pipelines* owners. 
+   2. Mark the *stable* to *packaged* TEP as *implemented*.
+   3. Mark the *experimental* to *stable* TEP as *superseded-by* the *stable* to *packaged* TEP.
+
+### Integrated
+
+When a given *packaged* `Custom Task` provides a critical functionality that is essential in the *Tekton Pipelines API*, 
+we can consider making it *integrated* - meaning that we add it directly to the *Tekton Pipelines API* surface. 
+
+Not all *packaged* `Custom Tasks` have to be *integrated*; a *packaged* `Custom Task` that is actively used but its 
+functionality is not absolutely necessary can stay as a *packaged* `Custom Task` in the long term.
+
+#### Graduation Requirements
+
+We can consider graduating a given *packaged* `Custom Task` to *integrated* if it meets the following requirements:
+
+1. Its functionality is essential to solve common Continuous Delivery use cases.
+2. It's widely adopted by users as a *packaged* `Custom Task` that's shipped with *Tekton Pipelines* out of the box 
+
+#### Graduation Process
+
+The graduation process from *packaged* to *integrated* for a given `Custom Task` would be:
+
+1. Validate that the *packaged* `Custom Task` meets the above requirements.
+2. Open a Tekton Enhancement Proposal (TEP) for the integrating the functionality provided by the `Custom Task` 
+directly to the *Tekton Pipelines API* with: 
+   1. A nomination for graduation from *packaged* to *integrated*.
+   2. The motivation (goals, use cases and requirements) - a discussion of how it's been adopted and why it's essential 
+      to natively support the functionality in the *Tekton Pipelines API*.
+   3. The proposal (syntax, design details, design evaluation and alternatives) for its integration to the 
+   *Tekton Pipelines API*. 
+3. When the *Tekton Pipelines* owners have approved the TEP as implementable:
+   1. Add its functionality directly to the *Tekton Pipelines API* as an *alpha* feature.
+   2. Deprecate the *packaged* `Custom Task`.
+   3. Mark the *packaged* to *integrated* TEP as *implemented*.
+   4. Mark the *stable* to *packaged* TEP as *superseded-by* the *packaged* to *integrated* TEP.
+4. When the *alpha* feature is promoted to *beta*, remove the *packaged* `Custom Task`.
+
+## Design Evaluation
+
+#### Simplicity
+
+By providing *stable* and *packaged* `Custom Tasks`, we provide an intermediary stability tiers that provides the 
+reliability needed by users without making unnecessary changes directly in the *Tekton Pipelines API*. This ensures 
+that the *Tekton Pipelines* API has the bare minimum features needed to solve most Continuous Delivery use cases.
+
+#### Reusability
+
+By providing *stable* and *packaged* `Custom Tasks`, we enable users to reuse extensions that they can rely on. 
+Moreover, having *experimental*, *stable* and *packaged* `Custom Tasks` allows contributors to share reusable `
+Custom Tasks` across the community. 
+
+#### Flexibility
+
+The *experimental*, *stable* and *packaged* `Custom Tasks` provide a mechanism to provide functionality that's not 
+directly available in the *Tekton Pipelines API*. It empowers users to extend *Tekton Pipelines* to solve their 
+bespoke Continuous Delivery use cases. 
+
+The `Custom Tasks` graduation path established in this TEP gives us flexibility to safely iterate on functionality 
+provided through the `Custom Tasks` as we make progress through each stage.
+
+## Alternatives
+
+#### *Experimental* to *Integrated*
+
+We could remove the *stable* and *packaged* `Custom Tasks` and promote from *experimental* to *integrated* directly.
+
+###### Pros
+
+- In the best case, speeds up the graduation process without intermediary stages.
+- Simplifies the graduation path.
+
+###### Cons
+
+- In the worst case, slows down the graduation process because of higher bar of approval for graduation from
+  *experimental* to *integrated* directly.
+- Does not provide a way to discover `Custom Tasks` whose quality and reliability are validated by Tekton.
+- Reduces the time to iterate on the functionality provided by `Custom Tasks` before integrating it to *Tekton Pipelines*.
+
+#### All `Custom Tasks` in One Repository
+
+We could have the *experimental*, *stable* and *packaged* `Custom Tasks` in one repository, and indicate their 
+stability levels by other means (such as documentation). 
+
+###### Pros
+
+- Consolidates `Custom Tasks` in one place, making them more easily discoverable.
+- Simplifies the change needed to migrate from *experimental* to *stable* to *packaged*.
+
+###### Cons
+
+- Reduces the separation between the *experimental*, *stable* and *packaged* `Custom Tasks`, taking more effort to 
+distinguish them.
+- Makes it harder to enforce quality requirements for *stable* and *packaged*`Custom Tasks` (we're considering 
+separating official resources in the *Tekton Catalog* for this reason).
+
+## Infrastructure Needed
+
+We need a GitHub repository for *stable* `Custom Tasks`.
+
+## References
+
+- [TEP-0002: Enable Custom Tasks][TEP-0002]
+- [TEP-0056: Pipelines in Pipelines][TEP-0056]
+- [Pipelines in Pipelines Custom Tasks][pip-ct]
+- [Common Expression Language Custom Tasks][cel-ct]
+- [Wait Custom Tasks][wait-ct]
+- [Task Looping Custom Tasks][tl-ct]
+- [Kubeflow Pipelines on Tekton][kubeflow]
+- [Tekton Pipelines API Policy][api-policy]
+
+[experimental-repo]: https://github.com/tektoncd/experimental
+[community-repo]: https://github.com/tektoncd/community
+[api-wg]: https://github.com/tektoncd/community/blob/main/working-groups.md#api
+[TEP-0002]: 0002-custom-tasks.md
+[TEP-0056]: 0056-pipelines-in-pipelines.md
+[pip-ct]: https://github.com/tektoncd/experimental/tree/main/pipelines-in-pipelines
+[cel-ct]: https://github.com/tektoncd/experimental/tree/main/cel
+[wait-ct]: https://github.com/tektoncd/experimental/tree/main/wait-task
+[tl-ct]: https://github.com/tektoncd/experimental/tree/main/task-loops
+[kubeflow]: https://developer.ibm.com/blogs/kubeflow-pipelines-and-tekton-advances-data-workloads
+[proposing-projects]: https://github.com/tektoncd/community/blob/main/process.md#proposing-projects
+[runs]: https://github.com/tektoncd/pipeline/blob/main/docs/runs.md
+[api-policy]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#alpha-beta-and-ga
+[backwards]: https://github.com/tektoncd/pipeline/blob/main/api_compatibility_policy.md#backwards-incompatible-changes

teps/README.md

@@ -223,3 +223,4 @@ This is the complete list of Tekton teps:
 |[TEP-0073](0073-simplify-metrics.md) | Simplify metrics | proposed | 2021-06-23 |
 |[TEP-0080](0080-support-domainscoped-parameterresult-names.md) | Support domain-scoped parameter/result names | implemented | 2021-08-19 |
 |[TEP-0084](0084-endtoend-provenance-collection.md) | end-to-end provenance collection | proposed | 2021-09-16 |
+|[TEP-0087](0087-custom-tasks-graduation.md) | Custom Tasks Graduation | proposed | 2021-09-30 |