Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Traceability of the test results #11

Open
sidneycadot opened this issue Nov 30, 2024 · 0 comments
Open

Traceability of the test results #11

sidneycadot opened this issue Nov 30, 2024 · 0 comments

Comments

@sidneycadot
Copy link

sidneycadot commented Nov 30, 2024

The tests contained in 65x02 are a valuable resource for 6502 work, however it is somewhat problematic that there is no proper traceability to widely available hardware or an open source emulator. So in case of issues in the tests (see for example issues #8 and #10), there is no ground truth to fall back on.

Currently the README has this short statement:

To generate each test set, an implementation is used that:

    conforms to all available documentation, official and third-party;
    passes all other published test sets; and
    has been verified by usage in an emulated machine.

This is essentially asking that users of the tests just trust the (unsourced) tool that was used to produce the tests, and to trust the process, emulators, and documentation that were used to verify that tool, and to trust that the person who did the work didn't make mistakes.

It would help to raise the standing of this test set to the level of a "gold standard" if users could fully understand the process that was used to produce them, and to have the ability to reproduce them in case of dispute.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant