From 5257005d1f02b665382bd27e038c02b520bf6ecb Mon Sep 17 00:00:00 2001 From: Benjamin Pelletier Date: Tue, 7 May 2024 13:52:41 +0000 Subject: [PATCH] [docs] Update test scenario bug form (#683) Update test scenario bug form --- ...ug_test_scenariog.md => bug_test_scenario.md} | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) rename .github/ISSUE_TEMPLATE/{bug_test_scenariog.md => bug_test_scenario.md} (50%) diff --git a/.github/ISSUE_TEMPLATE/bug_test_scenariog.md b/.github/ISSUE_TEMPLATE/bug_test_scenario.md similarity index 50% rename from .github/ISSUE_TEMPLATE/bug_test_scenariog.md rename to .github/ISSUE_TEMPLATE/bug_test_scenario.md index 4aa6b4a15c..8798876012 100644 --- a/.github/ISSUE_TEMPLATE/bug_test_scenariog.md +++ b/.github/ISSUE_TEMPLATE/bug_test_scenario.md @@ -6,11 +6,21 @@ labels: test-scenario-behavior, automated-testing *Note: remove this note and replace all template instructions below with your content before submitting this issue* -**report.json is not attached because** -If an instance of report.json illustrating the problem is attached to this issue, delete this section. Otherwise, explain why it cannot be attached; troubleshooting test behavior without report.json can be very difficult so almost every issue of this type should have a report.json attached. +**Observed behavior** +Describe the observed behavior in enough detail for contributors to at least unambiguously understand what is happening, and ideally to reproduce. Excellent means to accomplish this are: + +* A zip file of a sequence view artifact +* A zip file of report.json + +Other acceptable means might include: + +* A screenshot of the relevant portions of a sequence view artifact (though it can be difficult to determine what is relevant if the problem is not known) plus an indication of codebase version +* A direct reference to the location of the known problem in code, along with a description of what happens due to that problem + +Troubleshooting test behavior without report.json or a sequence view zip can be very difficult so most issues of this type should have a report.json or sequence view zip attached. **Test check** -Identify the test check that unexpectedly succeeded, failed, or wasn't checked -- try to limit a single GitHub issue to a single test check to investigate. "First failed_check" is fine, or a line number in the attached report.json is ideal. +Identify the test check that unexpectedly succeeded, failed, or wasn't checked -- try to limit a single GitHub issue to a single test check to investigate. "First failed_check" is fine, or a line number in the attached report.json or scenario index + event index in the attached sequence view is ideal. **Difference from expected behavior** Explain why you think there is a problem with the observed behavior. For instance, "The test scenario seems to assume that all USSs must do X but my USS does Y instead and I think I'm still compliant with all the standard requirements being tested." Or, "The test report says I didn't do X but I think I actually did do X because of evidence Y."