7903935: --verify-exclude existence check misses tests with failing requires

[LudwikJaniuk]

The current implementation relies on
getResultsIterator
which in turn calls into
TestResultTable.getIterator(filters)
passing into jtharness. That seems to currently filter out tests that don't meet the local "requires" criteria. Thus we fail to collect all "existing" tests in to a set to compare with.

The fix seems to be to explicitly pass an empty list of filters into
TestResultTable.getIterator(filters)
as this will make
TRT_Iterator.wouldAccept()
accept every test.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• CODETOOLS-7903935: --verify-exclude existence check misses tests with failing requires (Bug - P4)

Reviewers

• Christian Stein (@sormuras - Committer)
• Iris Clark (@irisclark - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jtreg.git pull/243/head:pull/243
$ git checkout pull/243

Update a local copy of the PR:
$ git checkout pull/243
$ git pull https://git.openjdk.org/jtreg.git pull/243/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 243

View PR using the GUI difftool:
$ git pr show -t 243

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jtreg/pull/243.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back lujaniuk! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@LudwikJaniuk This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

7903935: --verify-exclude existence check misses tests with failing requires

Reviewed-by: cstein, iris

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 3 new commits pushed to the master branch:

• 081b8f7: 7903920: Support building jtreg on FreeBSD
• 7dcd0bc: 7903940: Remove problematic LIBRARY.properties implementation
• 4544d5f: 7903936: Update variable names in build.sh documentation

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[mlbridge]

Webrevs

• 00: Full (3131c353)

[LudwikJaniuk]

As a way to test these changes on real-world data, I ran against the OpenJDK problemlist files. The following lists how many tests in each were listed as problematic:

test/lib-test/ProblemList.txt: 0
test/jaxp/ProblemList.txt: 0
test/docs/ProblemList.txt: 0
test/hotspot/jtreg/ProblemList-AotJdk.txt: 1
test/hotspot/jtreg/ProblemList.txt: 3
test/hotspot/jtreg/ProblemList-Virtual.txt: 0
test/hotspot/jtreg/ProblemList-Xcomp.txt: 0
test/hotspot/jtreg/ProblemList-zgc.txt: 14
test/jdk/ProblemList-AotJdk.txt: 0
test/jdk/ProblemList-zgc.txt: 0
test/jdk/ProblemList-shenandoah.txt: 0
test/jdk/ProblemList-Virtual.txt: 4
test/jdk/ProblemList.txt: 8
test/jdk/ProblemList-Xcomp.txt: 0
test/langtools/ProblemList.txt: 0

After a manual look at test/hotspot/jtreg/ProblemList-zgc.txt, it does look like it references a lot of tests that don't exist. These were:

serviceability/sa/ClhsdbFindPC.java#apa                       8307393   generic-all
serviceability/sa/ClhsdbLauncher.java                         8307393   generic-all
serviceability/sa/ClhsdbPmap.java                             8307393   generic-all
serviceability/sa/ClhsdbScanOops.java                         8307393   generic-all
serviceability/sa/LingeredAppSysProps.java                    8307393   generic-all
serviceability/sa/LingeredAppWithDefaultMethods.java          8307393   generic-all
serviceability/sa/LingeredAppWithEnum.java                    8307393   generic-all
serviceability/sa/LingeredAppWithInterface.java               8307393   generic-all
serviceability/sa/LingeredAppWithInvokeDynamic.java           8307393   generic-all
serviceability/sa/LingeredAppWithLock.java                    8307393   generic-all
serviceability/sa/LingeredAppWithNativeMethod.java            8307393   generic-all
serviceability/sa/LingeredAppWithRecComputation.java          8307393   generic-all
serviceability/sa/jmap-hprof/JMapHProfLargeHeapProc.java      8307393   generic-all
serviceability/sa/sadebugd/DebugdUtils.java                   8307393   generic-all

Indeed, manual sampling reveals these lines are invalid, e.g. because ClhsdbFindPC.java does not have a test with id apa, or because serviceability/sa/LingeredAppWithInterface.java is not a jtreg test at all (no @test comment).

[sormuras]

Looks good to me.