Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue#1785]-Create Data Index(DI)/ Jobs Service(JS) Database(DB) migration image #1791

Closed
wants to merge 12 commits into from

Conversation

rhkp
Copy link

@rhkp rhkp commented Jul 29, 2024

This task is to create a Data Index(DI)/ Jobs Service(JS) Database(DB) migration image, which can be used by Kubernetes/OpenShift job to perform necessary database changes for DI/JS application, before they are started with SonataFlowPlatform. A detailed description can be found here.

  • [Yes] Your feature/bug fix has a testcase that verifies it
  • [Yes] You've tested the new feature/bug fix in an actual OpenShift cluster
  • [Yes] You've added a RELEASE_NOTES.md entry regarding this change

@kie-ci3
Copy link
Contributor

kie-ci3 commented Jul 29, 2024

PR job #32 was: FAILURE
Possible explanation: Pipeline failure or project build failure

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest/job/kogito-images.build-and-test/32/display/redirect
See console log:

Console Logs Starting building: KIE » kogito » main » pullrequest » kogito-images.build-image #110
Starting building: KIE » kogito » main » pullrequest » kogito-images.build-image #116
Starting building: KIE » kogito » main » pullrequest » kogito-images.build-image #115
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-base-builder: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-jit-runner: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-jobs-service-postgresql: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-data-index-postgresql: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Failed in branch kogito-data-index-ephemeral
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-postgres-db-migration: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-jobs-service-ephemeral: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-jobs-service-allinone: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ wget --no-check-certificate -qO - 'https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest/job/kogito-images.build-and-test/32/api/json?depth=0'
[Pipeline] readJSON
[Pipeline] sh

@wmedvede
Copy link
Contributor

@rhkp
One more nitpick regarding the name of the image, considering that the name of the images we have now are like this:

apache/incubator-kie-kogito-jobs-service-postgresql
apache/incubator-kie-kogito-data-index-postgresql

I think this image must have a name like this:

apache/incubator-kie-kogito-service-db-migration-postgresql

@kie-ci3
Copy link
Contributor

kie-ci3 commented Jul 30, 2024

PR job #33 was: FAILURE
Possible explanation: Pipeline failure or project build failure

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest/job/kogito-images.build-and-test/33/display/redirect
See console log:

Console Logs [Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-jit-runner: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-jobs-service-ephemeral: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Failed in branch kogito-data-index-postgresql
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-jobs-service-postgresql: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-jobs-service-allinone: https://github.com//pull/1791 completed: SUCCESS
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-data-index-ephemeral: https://github.com//pull/1791 completed: UNSTABLE
[Pipeline] unstable
WARNING: Tests on kogito-data-index-ephemeral seems to have failed
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
Build KIE » kogito » main » pullrequest » kogito-images.build-image PR #1791 - kogito-postgres-db-migration: https://github.com//pull/1791 completed: UNSTABLE
[Pipeline] unstable
WARNING: Tests on kogito-postgres-db-migration seems to have failed
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ wget --no-check-certificate -qO - 'https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest/job/kogito-images.build-and-test/33/api/json?depth=0'
[Pipeline] readJSON
[Pipeline] sh

@@ -19,6 +19,7 @@

<!-- Keep them in alphabetical order -->
## Enhancements
Added Data Index and Jobs Service database migration image for use by a Kubernetes/OpenShift Job, before the Data Index and Jobs Service are deployed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, but this is unnecessary. You can remove this file, actually. We use GH releases now.


microdnf install --nodocs tar gzip wget unzip

wget https://repository.apache.org/content/groups/snapshots/org/kie/kogito/kogito-ddl/10.0.999-SNAPSHOT/kogito-ddl-10.0.999-20240726.011627-10-db-scripts.zip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This URL should be an ENV VAR that would be set during the image build.

wget https://repository.apache.org/content/groups/snapshots/org/kie/kogito/kogito-ddl/10.0.999-SNAPSHOT/kogito-ddl-10.0.999-20240726.011627-10-db-scripts.zip
unzip kogito-ddl-10.0.999-20240726.011627-10-db-scripts.zip

wget https://repo1.maven.org/maven2/org/flywaydb/flyway-commandline/10.17.0/flyway-commandline-10.17.0-linux-x64.tar.gz
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing here.

path: artifacts/migration.sh
dest: /home/default
execute:
- script: install
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module can set versions/URLs for flyway and kogito DDL scripts.

When container is started with command bash -c '/home/default/migration.sh'
Then container log should contain LISTING SQL DIR
And container log should contain V1.44.0__data_index_definitions.sql
And container log should contain V2.0.1__job_details_increase_job_id_size.sql
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid using version names to avoid errors in the CI testing when this changes.

@rhkp
Copy link
Author

rhkp commented Aug 8, 2024

This PR is being closed as we are pursuing custom sonataflow-db-migrator, which is a different approach from this one.

@rhkp rhkp closed this Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants