Skip to content

Commit

Permalink
[docs] Update test scenario bug form (#683)
Browse files Browse the repository at this point in the history
Update test scenario bug form
  • Loading branch information
BenjaminPelletier authored May 7, 2024
1 parent 03a462d commit 5257005
Showing 1 changed file with 13 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,21 @@ labels: test-scenario-behavior, automated-testing

*Note: remove this note and replace all template instructions below with your content before submitting this issue*

**report.json is not attached because**
If an instance of report.json illustrating the problem is attached to this issue, delete this section. Otherwise, explain why it cannot be attached; troubleshooting test behavior without report.json can be very difficult so almost every issue of this type should have a report.json attached.
**Observed behavior**
Describe the observed behavior in enough detail for contributors to at least unambiguously understand what is happening, and ideally to reproduce. Excellent means to accomplish this are:

* A zip file of a sequence view artifact
* A zip file of report.json

Other acceptable means might include:

* A screenshot of the relevant portions of a sequence view artifact (though it can be difficult to determine what is relevant if the problem is not known) plus an indication of codebase version
* A direct reference to the location of the known problem in code, along with a description of what happens due to that problem

Troubleshooting test behavior without report.json or a sequence view zip can be very difficult so most issues of this type should have a report.json or sequence view zip attached.

**Test check**
Identify the test check that unexpectedly succeeded, failed, or wasn't checked -- try to limit a single GitHub issue to a single test check to investigate. "First failed_check" is fine, or a line number in the attached report.json is ideal.
Identify the test check that unexpectedly succeeded, failed, or wasn't checked -- try to limit a single GitHub issue to a single test check to investigate. "First failed_check" is fine, or a line number in the attached report.json or scenario index + event index in the attached sequence view is ideal.

**Difference from expected behavior**
Explain why you think there is a problem with the observed behavior. For instance, "The test scenario seems to assume that all USSs must do X but my USS does Y instead and I think I'm still compliant with all the standard requirements being tested." Or, "The test report says I didn't do X but I think I actually did do X because of evidence Y."
Expand Down

0 comments on commit 5257005

Please sign in to comment.