You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We can either use multiprocessing or subprocess to start an offline ingestor in the different process.
-
multiprocessing
subprocess
pros
- easy to attach callback - easy to post-process the result of the offline-ingestor - doesn't need to use arbitrary command
- offline-ingestor will need a command entry point anyways, so don't need to implement another accessible interface
cons
- needs a function type interface to run an offline-ingestor
- limited access to the result - not possible to attach a callback function - needs to wrap the pipe-object with an async call to continue to the next message while offline-ingestor is running
Since we don't use non-python program at all,
I would like to go for the multiprocess.Process...?
In case we keep using subprocess, we need to make sure if it should be asynchronously run or not.
The text was updated successfully, but these errors were encountered:
We can either use
multiprocessing
orsubprocess
to start an offline ingestor in the different process.multiprocessing
subprocess
- easy to post-process the result of the offline-ingestor
- doesn't need to use arbitrary command
function
type interface to run an offline-ingestor- not possible to attach a callback function
- needs to wrap the pipe-object with an async call to continue to the next message while offline-ingestor is running
Since we don't use non-python program at all,
I would like to go for the
multiprocess.Process
...?In case we keep using
subprocess
, we need to make sure if it should be asynchronously run or not.The text was updated successfully, but these errors were encountered: