Add kubernetes job

[cedricve]

Description

Pull Request Description

Motivation

This pull request introduces the ability to run the migration as a Kubernetes job within a cluster. This enhancement leverages Kubernetes' internal DNS, eliminating the need to expose services externally and simplifying the deployment process. By running migrations as Kubernetes jobs, users can benefit from the robust orchestration and management capabilities provided by Kubernetes.

Why It Improves the Project

1. Simplified Deployment: Users can now deploy migrations directly in their Kubernetes clusters using a straightforward kubectl apply -f kubernetes-job.yaml command, eliminating the need for complex setup and configuration.

2. Enhanced Security: By using internal Kubernetes DNS, there is no need to expose services externally, reducing the attack surface and enhancing the overall security posture of the deployment.

3. Scalability and Reliability: Kubernetes' built-in features such as automatic restarts, resource management, and scaling ensure that the migration jobs are handled efficiently and reliably, even in large-scale environments.

Changes Introduced

1. README.md:

   • Added instructions on how to run the migration as a Kubernetes job.
   • Updated the example command to include new parameters: -queue and -mode.
   • Documented the new parameters:
     • -queue: Specifies the integration used to transfer events to the hub pipeline.
     • -mode: Allows users to choose between dry-run and live modes.

2. kubernetes-job.yaml:

   • Introduced a Kubernetes job configuration to run the migration.
   • Included parameters such as -mongodb-uri, -mongodb-source-database, -mongodb-destination-database, -username, -queue, -start-timestamp, -end-timestamp, -timezone, -pipeline, -mode, -batch-size, and -batch-delay.
   • Configured the job to use an init-container with the uugai/cli:latest image.
   • Set the restart policy to Never to ensure jobs do not restart automatically upon completion.

Conclusion

This pull request significantly enhances the usability, security, and scalability of the migration tool by integrating Kubernetes job support. Users can now effortlessly deploy and manage migrations within their Kubernetes clusters, benefiting from the robust orchestration capabilities of Kubernetes.

[Copilot]

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 suggestions.