Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create generalized and parameterized e2e pipeline for GitHub PR TTM service #502

Open
chauhankaranraj opened this issue Jun 29, 2022 · 0 comments
Assignees

Comments

@chauhankaranraj
Copy link
Member

Is your feature request related to a problem? Please describe.
Currently the steps to create a github pr ttm prediction service are somewhat "hard-coded" or "customized" for specific repos or orgs (openshift/origin, thoth-station). When we want to extend this service to other users / repos in the open source community, there would be some manual effort / tweaking required for data collection, feature engineering, and model training steps.

IMO this process probably wouldn't scale very well with number of new repos (for example, it took us couple of weeks to update the openshift/origin notebooks to work for thoth-station).

Describe the solution you'd like
A generalized / parameterized end-to-end pipeline (data collection, feature engg, model training, deployment) that can be configured to run for a given repo or set of repos. The inputs could be parameters like repo, thresholds in feature selection methods, names of models to explore, etc. And the output would be a trained model in s3 or even a deployed model, as a stretch goal.

This a pipeline should be reproducible, and should be run every time there's a new request by a user to add this service to their repo.

Describe alternatives you've considered
Manually change repo names and other parameters, and rerun notebooks for data collection, feat engg, model training, model deployment.

/cc @oindrillac

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants