Add support for legacy aws plugin

README.md

@@ -25,9 +25,9 @@ Periodic AWS E2E Virtualization Tests in OpenShift 4.17
 [![VM tests](https://prow.ci.openshift.org/badge.svg?jobs=periodic-ci-openshift-oadp-operator-master-4.17-e2e-test-kubevirt-aws-periodic)](https://prow.ci.openshift.org/job-history/gs/origin-ci-test/logs/periodic-ci-openshift-oadp-operator-master-4.17-e2e-test-kubevirt-aws-periodic)
 
 OADP repositories images job
-| OADP | OpenShift Velero plugin | Velero | Velero plugin for AWS | Velero plugin for GCP | Velero plugin for Microsoft Azure | Non Admin |
+| OADP | OpenShift Velero plugin | Velero | Velero plugin for AWS | Velero plugin for Legacy AWS | Velero plugin for GCP | Velero plugin for Microsoft Azure | Non Admin |
 | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- |
-| [![OADP repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-oadp-operator-master-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-oadp-operator-master-images) | [![OpenShift Velero plugin repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-openshift-velero-plugin-master-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-openshift-velero-plugin-master-images) | [![OADP's Velero repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-konveyor-dev-images) | [![OADP's Velero plugin for AWS repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-plugin-for-aws-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-plugin-for-aws-konveyor-dev-images) | [![OADP's Velero plugin for GCP repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-plugin-for-gcp-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-plugin-for-gcp-konveyor-dev-images) | [![OADP's Velero plugin for Microsoft Azure repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-plugin-for-microsoft-azure-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-plugin-for-microsoft-azure-konveyor-dev-images) | [![Non Admin repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-migtools-oadp-non-admin-master-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-migtools-oadp-non-admin-master-images) |
+| [![OADP repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-oadp-operator-master-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-oadp-operator-master-images) | [![OpenShift Velero plugin repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-openshift-velero-plugin-master-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-openshift-velero-plugin-master-images) | [![OADP's Velero repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-konveyor-dev-images) | [![OADP's Velero plugin for AWS repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-plugin-for-aws-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-plugin-for-aws-konveyor-dev-images) | [![OADP's Velero plugin for Legacy AWS repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-plugin-for-legacy-aws-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-plugin-for-legacy-aws-konveyor-dev-images) | [![OADP's Velero plugin for GCP repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-plugin-for-gcp-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-plugin-for-gcp-konveyor-dev-images) | [![OADP's Velero plugin for Microsoft Azure repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-openshift-velero-plugin-for-microsoft-azure-konveyor-dev-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-openshift-velero-plugin-for-microsoft-azure-konveyor-dev-images) | [![Non Admin repository](https://prow.ci.openshift.org/badge.svg?jobs=branch-ci-migtools-oadp-non-admin-master-images)](https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/branch-ci-migtools-oadp-non-admin-master-images) |
 
 Mirroring images to quay.io [![Mirror images](https://prow.ci.openshift.org/badge.svg?jobs=periodic-image-mirroring-konveyor)](https://prow.ci.openshift.org/job-history/gs/origin-ci-test/logs/periodic-image-mirroring-konveyor)
 </div>

api/v1alpha1/oadp_types.go

@@ -35,10 +35,11 @@ const ReconcileCompleteMessage = "Reconcile complete"
 const OadpOperatorLabel = "openshift.io/oadp"
 const RegistryDeploymentLabel = "openshift.io/oadp-registry"
 
-// +kubebuilder:validation:Enum=aws;gcp;azure;csi;vsm;openshift;kubevirt
+// +kubebuilder:validation:Enum=aws;legacy-aws;gcp;azure;csi;vsm;openshift;kubevirt
 type DefaultPlugin string
 
 const DefaultPluginAWS DefaultPlugin = "aws"
+const DefaultPluginLegacyAWS DefaultPlugin = "legacy-aws"
 const DefaultPluginGCP DefaultPlugin = "gcp"
 const DefaultPluginMicrosoftAzure DefaultPlugin = "azure"
 const DefaultPluginCSI DefaultPlugin = "csi"
@@ -56,6 +57,7 @@ type UnsupportedImageKey string
 
 const VeleroImageKey UnsupportedImageKey = "veleroImageFqin"
 const AWSPluginImageKey UnsupportedImageKey = "awsPluginImageFqin"
+const LegacyAWSPluginImageKey UnsupportedImageKey = "legacyAWSPluginImageFqin"
 const OpenShiftPluginImageKey UnsupportedImageKey = "openshiftPluginImageFqin"
 const AzurePluginImageKey UnsupportedImageKey = "azurePluginImageFqin"
 const GCPPluginImageKey UnsupportedImageKey = "gcpPluginImageFqin"
@@ -544,6 +546,7 @@ type DataProtectionApplicationSpec struct {
 	// Available keys are:
 	//   - veleroImageFqin
 	//   - awsPluginImageFqin
+	//   - legacyAWSPluginImageFqin
 	//   - openshiftPluginImageFqin
 	//   - azurePluginImageFqin
 	//   - gcpPluginImageFqin

bundle/manifests/oadp-operator.clusterserviceversion.yaml

@@ -917,6 +917,8 @@ spec:
                   value: quay.io/konveyor/openshift-velero-plugin:latest
                 - name: RELATED_IMAGE_VELERO_PLUGIN_FOR_AWS
                   value: quay.io/konveyor/velero-plugin-for-aws:latest
+                - name: RELATED_IMAGE_VELERO_PLUGIN_FOR_LEGACY_AWS
+                  value: quay.io/konveyor/velero-plugin-for-legacy-aws:latest
                 - name: RELATED_IMAGE_VELERO_PLUGIN_FOR_MICROSOFT_AZURE
                   value: quay.io/konveyor/velero-plugin-for-microsoft-azure:latest
                 - name: RELATED_IMAGE_VELERO_PLUGIN_FOR_GCP
@@ -1068,6 +1070,8 @@ spec:
     name: openshift-velero-plugin
   - image: quay.io/konveyor/velero-plugin-for-aws:latest
     name: velero-plugin-for-aws
+  - image: quay.io/konveyor/velero-plugin-for-legacy-aws:latest
+    name: velero-plugin-for-legacy-aws
   - image: quay.io/konveyor/velero-plugin-for-microsoft-azure:latest
     name: velero-plugin-for-microsoft-azure
   - image: quay.io/konveyor/velero-plugin-for-gcp:latest

bundle/manifests/oadp.openshift.io_dataprotectionapplications.yaml

@@ -824,6 +824,7 @@ spec:
                           items:
                             enum:
                               - aws
+                              - legacy-aws
                               - gcp
                               - azure
                               - csi
@@ -1324,6 +1325,7 @@ spec:
                     Available keys are:
                       - veleroImageFqin
                       - awsPluginImageFqin
+                      - legacyAWSPluginImageFqin
                       - openshiftPluginImageFqin
                       - azurePluginImageFqin
                       - gcpPluginImageFqin

config/crd/bases/oadp.openshift.io_dataprotectionapplications.yaml

@@ -824,6 +824,7 @@ spec:
                           items:
                             enum:
                               - aws
+                              - legacy-aws
                               - gcp
                               - azure
                               - csi
@@ -1324,6 +1325,7 @@ spec:
                     Available keys are:
                       - veleroImageFqin
                       - awsPluginImageFqin
+                      - legacyAWSPluginImageFqin
                       - openshiftPluginImageFqin
                       - azurePluginImageFqin
                       - gcpPluginImageFqin

config/manager/manager.yaml

@@ -50,6 +50,8 @@ spec:
             value: quay.io/konveyor/openshift-velero-plugin:latest
           - name: RELATED_IMAGE_VELERO_PLUGIN_FOR_AWS
             value: quay.io/konveyor/velero-plugin-for-aws:latest
+          - name: RELATED_IMAGE_VELERO_PLUGIN_FOR_LEGACY_AWS
+            value: quay.io/konveyor/velero-plugin-for-legacy-aws:latest
           - name: RELATED_IMAGE_VELERO_PLUGIN_FOR_MICROSOFT_AZURE
             value: quay.io/konveyor/velero-plugin-for-microsoft-azure:latest
           - name: RELATED_IMAGE_VELERO_PLUGIN_FOR_GCP

controllers/bsl.go

@@ -14,6 +14,7 @@ import (
 
 	oadpv1alpha1 "github.com/openshift/oadp-operator/api/v1alpha1"
 	"github.com/openshift/oadp-operator/pkg/common"
+	"github.com/openshift/oadp-operator/pkg/credentials"
 	"github.com/openshift/oadp-operator/pkg/storage/aws"
 )
 
@@ -398,9 +399,9 @@ func (r *DPAReconciler) validateGCPBackupStorageLocation(bslSpec velerov1.Backup
 	return nil
 }
 
-func pluginExistsInVeleroCR(configuredPlugins []oadpv1alpha1.DefaultPlugin, expectedPlugin oadpv1alpha1.DefaultPlugin) bool {
+func pluginExistsInVeleroCR(configuredPlugins []oadpv1alpha1.DefaultPlugin, expectedProvider string) bool {
 	for _, plugin := range configuredPlugins {
-		if plugin == expectedPlugin {
+		if credentials.PluginSpecificFields[plugin].ProviderName == expectedProvider {
 			return true
 		}
 	}
@@ -412,7 +413,7 @@ func (r *DPAReconciler) validateProviderPluginAndSecret(bslSpec velerov1.BackupS
 		return nil
 	}
 	// check for existence of provider plugin and warn if the plugin is absent
-	if !pluginExistsInVeleroCR(r.dpa.Spec.Configuration.Velero.DefaultPlugins, oadpv1alpha1.DefaultPlugin(bslSpec.Provider)) {
+	if !pluginExistsInVeleroCR(r.dpa.Spec.Configuration.Velero.DefaultPlugins, bslSpec.Provider) {
 		r.Log.Info(fmt.Sprintf("%s backupstoragelocation is configured but velero plugin for %s is not present", bslSpec.Provider, bslSpec.Provider))
 		//TODO: set warning condition on Velero CR
 	}

controllers/validator.go

@@ -104,10 +104,20 @@ func (r *DPAReconciler) ValidateVeleroPlugins(log logr.Logger) (bool, error) {
 		}
 	}
 
+	foundAWSPlugin := false
+	foundLegacyAWSPlugin := false
 	for _, plugin := range dpa.Spec.Configuration.Velero.DefaultPlugins {
 		pluginSpecificMap, ok := credentials.PluginSpecificFields[plugin]
 		pluginNeedsCheck, foundInBSLorVSL := providerNeedsDefaultCreds[string(plugin)]
 
+		// "aws" and "legacy-aws" cannot both be specified
+		if plugin == oadpv1alpha1.DefaultPluginAWS {
+			foundAWSPlugin = true
+		}
+		if plugin == oadpv1alpha1.DefaultPluginLegacyAWS {
+			foundLegacyAWSPlugin = true
+		}
+
 		// check for VSM/Volsync DataMover (OADP 1.2 or below) syntax
 		if plugin == oadpv1alpha1.DefaultPluginVSM {
 			return false, errors.New("Delete vsm from spec.configuration.velero.defaultPlugins and dataMover object from spec.features. Use Velero Built-in Data Mover instead")
@@ -155,5 +165,10 @@ func (r *DPAReconciler) ValidateVeleroPlugins(log logr.Logger) (bool, error) {
 			}
 		}
 	}
+
+	if foundAWSPlugin && foundLegacyAWSPlugin {
+		return false, fmt.Errorf("%s and %s can not be both specified in DPA spec.configuration.velero.defaultPlugins", oadpv1alpha1.DefaultPluginAWS, oadpv1alpha1.DefaultPluginLegacyAWS)
+	}
+
 	return true, nil
 }

controllers/validator_test.go

@@ -1396,6 +1396,30 @@ func TestDPAReconciler_ValidateDataProtectionCR(t *testing.T) {
 			wantErr:    true,
 			messageErr: "in order to enable/disable the non-admin feature please set dpa.spec.unsupportedOverrides[tech-preview-ack]: 'true'",
 		},
+		{
+			name: "given invalid DPA CR aws and legacy-aws plugins both specified",
+			dpa: &oadpv1alpha1.DataProtectionApplication{
+				ObjectMeta: metav1.ObjectMeta{
+					Name:      "test-DPA-CR",
+					Namespace: "test-ns",
+				},
+				Spec: oadpv1alpha1.DataProtectionApplicationSpec{
+					Configuration: &oadpv1alpha1.ApplicationConfig{
+						Velero: &oadpv1alpha1.VeleroConfig{
+							DefaultPlugins: []oadpv1alpha1.DefaultPlugin{
+								oadpv1alpha1.DefaultPluginAWS,
+								oadpv1alpha1.DefaultPluginLegacyAWS,
+							},
+							NoDefaultBackupLocation: true,
+						},
+					},
+					BackupImages: pointer.Bool(false),
+				},
+			},
+			objects:    []client.Object{},
+			wantErr:    true,
+			messageErr: "aws and legacy-aws can not be both specified in DPA spec.configuration.velero.defaultPlugins",
+		},
 	}
 	for _, tt := range tests {
 		tt.objects = append(tt.objects, tt.dpa)
@@ -1421,7 +1445,7 @@ func TestDPAReconciler_ValidateDataProtectionCR(t *testing.T) {
 				t.Errorf("ValidateDataProtectionCR() error = %v, wantErr %v", err, tt.wantErr)
 				return
 			}
-			if tt.wantErr && err.Error() != tt.messageErr {
+			if tt.wantErr && err != nil && err.Error() != tt.messageErr {
 				t.Errorf("Error messages are not the same: got %v, expected %v", err.Error(), tt.messageErr)
 				return
 			}

controllers/velero.go

@@ -412,15 +412,15 @@ func (r *DPAReconciler) appendPluginSpecificSpecs(veleroDeployment *appsv1.Deplo
 
 	for _, plugin := range dpa.Spec.Configuration.Velero.DefaultPlugins {
 		if pluginSpecificMap, ok := credentials.PluginSpecificFields[plugin]; ok {
-			imagePullPolicy, err := common.GetImagePullPolicy(dpa.Spec.ImagePullPolicy, credentials.GetPluginImage(pluginSpecificMap.PluginName, dpa))
+			imagePullPolicy, err := common.GetImagePullPolicy(dpa.Spec.ImagePullPolicy, credentials.GetPluginImage(plugin, dpa))
 			if err != nil {
 				r.Log.Error(err, "imagePullPolicy regex failed")
 			}
 
 			veleroDeployment.Spec.Template.Spec.InitContainers = append(
 				veleroDeployment.Spec.Template.Spec.InitContainers,
 				corev1.Container{
-					Image:                    credentials.GetPluginImage(pluginSpecificMap.PluginName, dpa),
+					Image:                    credentials.GetPluginImage(plugin, dpa),
 					Name:                     pluginSpecificMap.PluginName,
 					ImagePullPolicy:          imagePullPolicy,
 					Resources:                init_container_resources,
@@ -434,7 +434,7 @@ func (r *DPAReconciler) appendPluginSpecificSpecs(veleroDeployment *appsv1.Deplo
 					},
 				})
 
-			pluginNeedsCheck, foundInBSLorVSL := providerNeedsDefaultCreds[string(plugin)]
+			pluginNeedsCheck, foundInBSLorVSL := providerNeedsDefaultCreds[pluginSpecificMap.ProviderName]
 
 			if !foundInBSLorVSL && !hasCloudStorage {
 				pluginNeedsCheck = true
@@ -762,7 +762,7 @@ func (r DPAReconciler) noDefaultCredentials() (map[string]bool, bool, error) {
 		}
 		for _, provider := range dpa.Spec.Configuration.Velero.DefaultPlugins {
 			if psf, ok := credentials.PluginSpecificFields[provider]; ok && psf.IsCloudProvider {
-				providerNeedsDefaultCreds[psf.PluginName] = needDefaultCred
+				providerNeedsDefaultCreds[psf.ProviderName] = needDefaultCred
 			}
 		}
 	} else {

controllers/velero_test.go

@@ -1365,6 +1365,37 @@ func TestDPAReconciler_buildVeleroDeployment(t *testing.T) {
 				},
 			}),
 		},
+		{
+			name: "valid DPA CR with legacy aws plugin, Velero Deployment is built with legacy aws plugin",
+			dpa: createTestDpaWith(
+				nil,
+				oadpv1alpha1.DataProtectionApplicationSpec{
+					Configuration: &oadpv1alpha1.ApplicationConfig{
+						Velero: &oadpv1alpha1.VeleroConfig{
+							DefaultPlugins: []oadpv1alpha1.DefaultPlugin{
+								oadpv1alpha1.DefaultPluginLegacyAWS,
+							},
+						},
+					},
+				},
+			),
+			veleroDeployment: testVeleroDeployment.DeepCopy(),
+			wantVeleroDeployment: createTestBuiltVeleroDeployment(TestBuiltVeleroDeploymentOptions{
+				initContainers: []corev1.Container{pluginContainer(common.VeleroPluginForLegacyAWS, common.LegacyAWSPluginImage)},
+				volumes:        []corev1.Volume{deploymentVolumeSecret("cloud-credentials")},
+				volumeMounts: []corev1.VolumeMount{
+					{Name: "cloud-credentials", MountPath: "/credentials"},
+				},
+				env: append(baseEnvVars, []corev1.EnvVar{
+					{Name: common.AWSSharedCredentialsFileEnvKey, Value: "/credentials/cloud"},
+				}...),
+				args: []string{
+					defaultFileSystemBackupTimeout,
+					defaultRestoreResourcePriorities,
+					defaultDisableInformerCache,
+				},
+			}),
+		},
 		{
 			name: "valid DPA CR with aws and kubevirt plugin, Velero Deployment is built with aws and kubevirt plugin",
 			dpa: createTestDpaWith(
@@ -1984,7 +2015,7 @@ func TestDPAReconciler_noDefaultCredentials(t *testing.T) {
 		wantErr             bool
 	}{
 		{
-			name: "dpa with all plugins but with noDefualtBackupLocation should not require default credentials",
+			name: "dpa with all plugins but with noDefaultBackupLocation should not require default credentials",
 			args: args{
 				dpa: oadpv1alpha1.DataProtectionApplication{
 					ObjectMeta: metav1.ObjectMeta{
@@ -2002,9 +2033,9 @@ func TestDPAReconciler_noDefaultCredentials(t *testing.T) {
 				},
 			},
 			want: map[string]bool{
-				"velero-plugin-for-aws":             false,
-				"velero-plugin-for-gcp":             false,
-				"velero-plugin-for-microsoft-azure": false,
+				"aws":   false,
+				"gcp":   false,
+				"azure": false,
 			},
 			wantHasCloudStorage: false,
 			wantErr:             false,

controllers/vsl.go

@@ -276,7 +276,7 @@ func (r *DPAReconciler) ReconcileVolumeSnapshotLocations(log logr.Logger) (bool,
 
 func containsPlugin(d []oadpv1alpha1.DefaultPlugin, value string) bool {
 	for _, elem := range d {
-		if string(elem) == value {
+		if credentials.PluginSpecificFields[elem].ProviderName == value {
 			return true
 		}
 	}

docs/config/plugins.md

@@ -8,8 +8,10 @@ There are mainly two categories of Velero plugins that can be specified while
 installing Velero:
 
 1. `defaultPlugins`:<br>
-   There are six types of default Velero plugins can be installed: 
+   There are several types of default Velero plugins can be installed:
    - `AWS` [Plugins for AWS
+](https://github.com/vmware-tanzu/velero-plugin-for-aws)
+   - `Legacy AWS` [Plugins for Legacy AWS
 ](https://github.com/vmware-tanzu/velero-plugin-for-aws)
    - `GCP` [Plugins for Google Cloud Platform](https://github.com/vmware-tanzu/velero-plugin-for-gcp)
    - `Azure` [Plugins for Microsoft Azure](https://github.com/vmware-tanzu/velero-plugin-for-microsoft-azure)
@@ -18,6 +20,8 @@ installing Velero:
    - `kubevirt` [Plugins for Kubevirt](https://github.com/kubevirt/kubevirt-velero-plugin)
    - `VSM (OADP 1.2 or below)` [Plugin for Volume-Snapshot-Mover](https://github.com/migtools/velero-plugin-for-vsm)
 
+   Note that only one of `AWS` and `Legacy AWS` may be installed at the same time. `Legacy AWS` is intended for use with certain S3 providers that do not support the V2 AWS SDK APIs used in the `AWS` plugin.
+
    For installation, 
    you need to specify them in the `oadp_v1alpha1_dpa.yaml` file 
    during deployment.
@@ -37,7 +41,21 @@ installing Velero:
           - gcp
    ```
    The above specification will install Velero with four of the default plugins.
-
+
+   ```
+    apiVersion: oadp.openshift.io/v1alpha1
+    kind: DataProtectionApplication
+    metadata:
+      name: dpa-sample
+    spec:
+      configuration:
+        velero:
+          defaultPlugins:
+          - openshift
+          - legacy-aws
+   ```
+   The above specification will install Velero with two of the default plugins.
+
 2. `customPlugins`:<br>
    For installation of custom Velero plugins, you need to specify the plugin 
    `image` and plugin `name` in the `oadp_v1alpha1_dpa.yaml` file during