[Task]: Automate ballerina performance testing

[heshanpadmasiri]

Description

Ideally we should be able to trigger a workflow in github that will run the performance tests in https://github.com/ballerina-platform/ballerina-performance and give us an report. For this we will need to do the fallowing,

• Setup a self hosted runner.
  These tests will need more that 2 hours to run (and our objective is to run them close to metal as possible) therefore we can't use github actions to run the benchmark

• Setup a private repository
  Using a self hosted runner for the public performance tests repository is a security risk since public forks and perform RCE on our VM by opening a pull request. Note that our runner VM will need permission to allocate cloud resources, so this is very dangerous.

  • This private repository could potentially include both ballerina-performce as well as performance-common (along with any patches we need to add to performance common)

Describe your task(s)

No response

Related area

-> CICD

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

[heshanpadmasiri]

When we trigger a performance test using github action even in a self hosted runner we need to "orphan" the process that is running the test in order to avoid action time out killing it. However still gihub runner sends kill signal to all child processes when the action completes. Therefore I was unable to implement the self hosted runner approach. Instead I decided to do the fallowing

1. In the VM we run a performance test server
2. As part of the action we send a post request to the server which will trigger the performance test
3. When the performance test has been completed server will create a pull request with the performance test results