Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

i852 New worker to run resource intensive jobs #969

Merged
merged 21 commits into from
Feb 14, 2024
Merged

i852 New worker to run resource intensive jobs #969

merged 21 commits into from
Feb 14, 2024

Commits on Feb 13, 2024

  1. new worker to run resource intensive jobs

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    36755a9 View commit details
    Browse the repository at this point in the history

  2. deploy resource intensive worker with 4x the normal resources

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    38f52d0 View commit details
    Browse the repository at this point in the history

  3. no need to specify default file location

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    692f16a View commit details
    Browse the repository at this point in the history

  4. helm deployment of intensive worker

    @orangewolf @bkiahstroud
    orangewolf authored and bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    a97e895 View commit details
    Browse the repository at this point in the history

  5. resource adjustments

    @orangewolf @bkiahstroud
    orangewolf authored and bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    8a8ce54 View commit details
    Browse the repository at this point in the history

  6. undo changes to unused file

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    3c0543f View commit details
    Browse the repository at this point in the history

  7. use ENV var for intensive worker thread count

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    3900d39 View commit details
    Browse the repository at this point in the history

  8. adjust intensive worker prod resources 

    Co-authored-by: Rob Kaufman <[email protected]>
    @bkiahstroud @orangewolf
    bkiahstroud and orangewolf committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    bcdb041 View commit details
    Browse the repository at this point in the history

  9. fix disarranged YAML

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    72410d8 View commit details
    Browse the repository at this point in the history

  10. update prod to Hyrax helm chart v3.5.1

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    8a22e71 View commit details
    Browse the repository at this point in the history

  11. fix demo redis deploy issue 

    Change pulled from:
- samvera/hyku#2145
    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    fee32b2 View commit details
    Browse the repository at this point in the history

  12. env vars need to be strings

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    789a226 View commit details
    Browse the repository at this point in the history

  13. correctly parse Redis host value 

    Because of the special way `extraDeploy` gets interpreted, fetching ENV
values using `$` will not work.
    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    c4b9cc3 View commit details
    Browse the repository at this point in the history

  14. use more generic name for extra worker 

    We will be reusing this pattern across several projects. Not every extra
worker will need increased resource limits, so we should use a more
generic name that can apply to various requirements.
    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    495d807 View commit details
    Browse the repository at this point in the history

  15. add extraDeploy hint for workerAuxiliary

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    ab11894 View commit details
    Browse the repository at this point in the history

  16. add extra worker deploy to prod, update redis conf

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    04bd455 View commit details
    Browse the repository at this point in the history

  17. don't mix JSON and YAML syntax

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    874746e View commit details
    Browse the repository at this point in the history

  18. staging deploy: add workerAuxiliary & update to Hyrax helm chart v3.5.1

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    d704786 View commit details
    Browse the repository at this point in the history

  19. prioritize auxiliary queue 

    Sidekiq queues without explicit weights are prioritized in the order
they are listed. Our auxiliary worker's primary goal is to process jobs
in the auxiliary queue, thus we want it to be the top priority.

[Sidekiq
docs](https://github.com/sidekiq/sidekiq/wiki/Advanced-Options#queues)
    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    e4a57a8 View commit details
    Browse the repository at this point in the history

  20. rename env var to be more generic

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    b26ed4d View commit details
    Browse the repository at this point in the history

  21. update comments to reflect new queue name

    @bkiahstroud
    bkiahstroud committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    d9f5488 View commit details
    Browse the repository at this point in the history