Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: start python NLP workers in embedded mode #1612

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

ClemDoum
Copy link
Contributor

@ClemDoum ClemDoum commented Nov 19, 2024

PR description

Following #1611 and to address #1452

This PR adds the ability to start worker pool of NLP Python workers when in EMBEDDED mode with AMQP support/bus type.

TODO

Changes

datashare-app

Added

  • added a ExecutableExtensionHelper in charge of executing DS executable extension (non JAR)
  • created a ProcessHandler to cleanly handle external processes (dump pids, find running process of some type, kill process etc)

Changed

  • updated WebApp to spawn NLP workers when in EMBEDDED mode with AMQP, if such process are already running the application will ask the user to clean the phantom process (could be otherwise)

@ClemDoum ClemDoum self-assigned this Nov 19, 2024
@ClemDoum ClemDoum force-pushed the feature/start-python-workers branch from 797048f to a5736d0 Compare November 20, 2024 11:55
@ClemDoum ClemDoum marked this pull request as ready for review November 20, 2024 12:04
@ClemDoum ClemDoum mentioned this pull request Nov 20, 2024
13 tasks
@ClemDoum ClemDoum force-pushed the feature/start-python-workers branch 2 times, most recently from 17ca462 to db6f51e Compare November 20, 2024 14:46
@ClemDoum ClemDoum force-pushed the feature/start-python-workers branch 2 times, most recently from 91c7b33 to 1ffe956 Compare November 29, 2024 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant