Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add service manager infrastructure #14150

Draft
wants to merge 1 commit into
base: dev
Choose a base branch
from

Commits on Oct 16, 2024

  1. Add service manager infrastructure

    The changes are (This will be a bit long):
    - A ServiceManager class that spawns a background thread and deals with
      service lifecycle management. The idea is that service lifecycle code
      will run in async functions, so a single thread is enough to manage
      any (reasonable) amount of services.
    
    - A Service class, that offers start() and stop() methods that just
      notify the service manager to... well. Start and stop a service.
    
    (!) Warning: Note that this differs from mp.Process.start/stop in that
      the service commands are sent asynchronously and will complete
      "eventually". This is good because it means that business logic is
      fast when booting up and shutting down, but we need to make sure
      that code does not rely on start() and stop() being instant
      (Mainly pid assignments).
    
      Subclasses of the Service class should use the on_start and on_stop
      methods to monitor for service events. These will be run by the
      service manager thread, so we need to be careful not to block
      execution here. Standard async stuff.
    
    (!) Note on service names: Service names should be unique within a
      ServiceManager. Make sure that you pass the name you want to
      super().__init__(name="...") if you plan to spawn multiple instances
      of a service.
    
    - A ServiceProcess class: A Service that wraps a multiprocessing.Process
      into a Service. It offers a run() method subclasses can override.
    
    And finally, I lied a bit about this whole thing using a single thread.
    I can't find any way to run python multiprocessing in async, so there is
    a MultiprocessingWaiter thread that waits for multiprocessing events and
    notifies any pending futures. This was uhhh... fun? No, not really.
    But it works. Using this part of the code just involves calling the
    provided wait method. See the implementation of ServiceProcess for more
    details.
    gtsiam committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    e187eea View commit details
    Browse the repository at this point in the history