[EPIC] Create a GitHub Actions trigger to call wf in another project

[mickahell]

What is the expected behavior?

Create an Action who enable project to run their own test inside their own Actions tab and to make their submission directly in their own repo as well.

Tech needs

essentials :

• Having a composite action who can be call using (from any project) :

- name: Add into Ecosystem
  uses: qiskit-community/ecosystem-actions
  with:
    project_name: "My super project"
    test_cmd: tox -epy39
    std_test: true
    stb_test: true
    dev_test: true
    submit_to_ecosystem: true
    ...

• First part of the composite action should be able to :
  • Run given test so call standard tests
  • Run given test with latest qiskit version
  • Run given test with dev qiskit version
  • Give readable logs
• Another part of this composite action should be able to send info about the project as well as test result to us. Some solutions possible :
  • Automatically create issue in Ecosystem repo (as today but probably the best way to do it and to keep clean history)
  • Launch Actions in Ecosystem repo (permission problem for external people)
• Deploy the composite action into GitHub marketplace
• On the Ecosystem repo we will need to create a new wf for update project info from data getting form external to do that we can use the same solution of the previous question (issue to be the best as well).
• Documentations :
  • Actions step to copy and paste
  • Full Actions pipeline file to copy and paste

(optional) :

• Having a docker image creating the perfect qiskit env to run checks inside and to make the actions faster

Why

• Allow people to test their project whenever they want/need
• Help people submit their project into Ecosystem
• We wont force people to have tox or anything special to run their tests
• We wont clone project and host actions processes

---

This would also allow us to keep the actual behaviour with the new one.
Today I don't know how to make all the actual projects migrate from actual process to this new one.

[mickahell]

Hola @frankharkins @Eric-Arellano :)

From the discussion I got with Frank and feedback we got from the Thursday meeting here a proposition.
What do you think about it :) feel free to add more people at the discussion

[frankharkins]

Thanks for this @mickahell! 🚀

Rather than a composite action, what do you think of a template that developers can copy and paste into their projects, and adjust as needed? It might be more flexible, and less maintenance on our end.

Another part of this composite action should be able to send info about the project as well as test result to us.

For now, I think we should remove this goal, as we're stopping the test results aspect of the project.

[mickahell]

Rather than a composite action, what do you think of a template that developers can copy and paste into their projects, and adjust as needed? It might be more flexible, and less maintenance on our end.

Too much text and commande for someone who never use Actions ^^' create our own Action to upload to marketplace is the best. It has to stay as a single process and a single step for them.
And yes in the documentation we will provide full Action to copy and paste.

Another part of this composite action should be able to send info about the project as well as test result to us.

For now, I think we should remove this goal, as we're stopping the test results aspect of the project.

We still need to get back the info about the project to be able to add it to our database, even if we give up the test on our side.

[frankharkins]

Too much text and commande for someone who never use Actions

We can include comments to explain which bits to modify, for example:

  - name: Run tests
    run: |
      # Replace this with your testing commands
      tox -epy39

It should be similar work (i.e. make a new file in .github/workflows and add the correct commands). But the advantage is if they need to add an extra step beforehand (or after), then we don't need to add support for that in our action; they can just edit the workflow file.

We still need to get back the info about the project to be able to add it to our database, even if we give up the test on our side.

Do we? Or are you thinking they should make their initial submission issue through an action?

[mickahell]

Ok i think we misunderstood each other ^^'

What I proposed is not to replace their own test, it's to add a step in // to their Actions to make sure they are running minimal test (with their cmd) and we could put whatever we want inside but it's not gonna replace their own test, lint, coverage ...
If we don't, they can just don't put any test at all and submit whatever they want and we could have a bunch list of unworking project without knowing.

For the 2nd part, what I think is they don't gonna fill any issue by hand, the actions gonna fill it for them. I don't think we can ask them to fill an issue and to create an action, it's too much stuff to do and that could allow us to receive update about their project.

[frankharkins]

...they can just don't put any test at all and submit whatever they want and we could have a bunch list of unworking project without knowing.

IMO the review process should check the project looks "serious" and part of this could include looking at their CI. Even if we require them to run certain commands, they can always put dummy code in the unit tests to "pass".

For the 2nd part, what I think is they don't gonna fill any issue by hand, the actions gonna fill it for them. I don't think we can ask them to fill an issue and to create an action, it's too much stuff to do and that could allow us to receive update about their project.

Are you suggesting projects should report back their test results to us through GitHub issues? If so: We don't plan to continue tracking test results; it doesn't seem like anyone looks at them, it involves a lot of automation, and we'd have to ask lots of projects to modify their CI, which I don't want to do until we have a very good reason.
(Sorry if I misunderstood)

[mickahell]

IMO the review process should check the project looks "serious" and part of this could include looking at their CI. Even if we require them to run certain commands, they can always put dummy code in the unit tests to "pass".

Yes indeed and that's also a problem :/ and we have to establish check in knowing of most of the possibilities.

Are you suggesting projects should report back their test results to us through GitHub issues? If so: We don't plan to continue tracking test results; it doesn't seem like anyone looks at them, it involves a lot of automation, and we'd have to ask lots of projects to modify their CI, which I don't want to do until we have a very good reason.

I was meaning, projects should be submitted through their Actions and those Actions gonna send us every info we need (project name, url, maintainers, ...).

(Sorry if I misunderstood)

No problem at all ;)

With what you told me on slack, I thin we can pause the issue and waiting for the NFC to determine the form we want for the whole process after if you agree :)