Skip to content
This repository has been archived by the owner on Nov 10, 2024. It is now read-only.

[FEATURE] execute repeat tasks on a single worker #282

Open
TheCodeYoda opened this issue Mar 7, 2024 · 2 comments
Open

[FEATURE] execute repeat tasks on a single worker #282

TheCodeYoda opened this issue Mar 7, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@TheCodeYoda
Copy link

TheCodeYoda commented Mar 7, 2024

Is your feature request related to a problem? Please describe.
execute functions under the repeat_every decorator on a single worker. (when I'm running multiple workers)

Additional context
Add any other context or screenshots about the feature request here.
fastapiutils#230

@TheCodeYoda TheCodeYoda added the enhancement New feature or request label Mar 7, 2024
@TheCodeYoda
Copy link
Author

Don't know if this is the correct way, but leveraged file system locks to pin a repeat task on a single worker(the first worker). locked_pid will only be set for the first worker which manages to create FILE_SYSTEM_LOCK_PATH successfully.

locked_pid=None
FILE_SYSTEM_LOCK_PATH="<path-to-dummy-lock-directory>"
@api.on_event("startup")
async def acquire_lock():
    global locked_pid
    pid = os.getpid()
    try:
        if locked_pid is None:
            os.mkdir(FILE_SYSTEM_LOCK_PATH)
            locked_pid = pid
            logger.info(f"lock acquired by pid={pid}!")
    except FileExistsError:
        logger.warn(f"lock already acquired!")

@api.on_event("startup")
@repeat_every(seconds=20 raise_exceptions=True)
async def periodic_task():
    global locked_pid
    if locked_pid:
        pid = os.getpid()
        do_work()

@api.on_event("shutdown")
async def release_lock():
    os.rmdir(FILE_SYSTEM_LOCK_PATH)
    logger.info("lock released!")

@TheCodeYoda
Copy link
Author

@yuval9313 let me know if the above solution is feasible, if yes I would like to contribute on building this out. Thanks.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant