Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New GitHub Action workflow for on demand commit benchmarking #344

Open
rwstauner opened this issue Nov 8, 2024 · 2 comments
Open

New GitHub Action workflow for on demand commit benchmarking #344

rwstauner opened this issue Nov 8, 2024 · 2 comments
Assignees

Comments

@rwstauner
Copy link
Contributor

We should add a new workflow (triggered by a UI button):
It will be very similar to the current "benchmark" interface.

Inputs:

  • repo
  • ref
  • build args?
  • run args?
  • parent/base for comparison
  • x86 and/or arm (default both?)

Tasks:

  • Setup params (resolve shas for specified ref and parent/base to compare to)
  • It should launch a new instance from the AMI
  • compile rubies
  • run the benchmarks
  • upload the json to a bucket
  • then it could DM the user (or post the channel) on slack with urls to the json files to notify when complete
  • shut itself down

We can leave it there for inspection or further testing if desired.
We could then have a periodic cleaner job that would clean up old instances that have been shut down for 7 days or some other amount of time.

@rwstauner rwstauner self-assigned this Nov 8, 2024
@maximecb
Copy link
Contributor

maximecb commented Nov 8, 2024

To simplify the interface, it might be nice if it could just take a commit hash (long or short form), and automatically look for it in ruby/ruby or in Shopify/ruby. People could still optionally specify a repo. Basically just generally have good defaults, and allow starting a run quickly. You only need to specify the full set of parameters if you want to test something specific.

@rwstauner
Copy link
Contributor Author

Yeah it will take a branch/ref/sha and resolve it in either repo (or another if specified). Everything will have useful defaults

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants