Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process or Subprocess to spawn offline ingestor process. #66

Open
YooSunYoung opened this issue Aug 7, 2024 · 0 comments
Open

Process or Subprocess to spawn offline ingestor process. #66

YooSunYoung opened this issue Aug 7, 2024 · 0 comments

Comments

@YooSunYoung
Copy link
Contributor

We can either use multiprocessing or subprocess to start an offline ingestor in the different process.

- multiprocessing subprocess
pros - easy to attach callback
- easy to post-process the result of the offline-ingestor
- doesn't need to use arbitrary command
- offline-ingestor will need a command entry point anyways, so don't need to implement another accessible interface
cons - needs a function type interface to run an offline-ingestor - limited access to the result
- not possible to attach a callback function
- needs to wrap the pipe-object with an async call to continue to the next message while offline-ingestor is running

Since we don't use non-python program at all,
I would like to go for the multiprocess.Process...?

In case we keep using subprocess, we need to make sure if it should be asynchronously run or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant