Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-35360] support Flink cdc pipeline Yarn application mode #3643

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

MOBIN-F
Copy link
Contributor

@MOBIN-F MOBIN-F commented Oct 14, 2024

hi @Mrart ,I merged your PR and tested it, but found some problem,i fixed it and added some tests.

flink-cdc-pipeline is a great feature, but currently only supports yarn-session in yarn. I think the cdc-community should support application mode as soon as possible so that users can promote the use of pipeline features within the company.

main changes of this PR are:

  1. Added CliExecutor#main method as the program entry of application mode, because in application mode, the main method runs in the cluster
  2. org.apache.flink.core.fs.Path replaces java.nio.file.Path, because additionalJars may be on the remote hdfs or oss
  3. MysqlE2eWithYarnApplicationITCase is the flink yarn application test class, which will start yarnminicluster to submit flink jobs

@yuxiqian @leonardBang @Mrart Can you help review this PR? Thank you~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants