Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: flexible choices for run_all_single_board_cases #282

Merged
merged 1 commit into from
May 6, 2024

Conversation

horw
Copy link
Member

@horw horw commented Apr 26, 2024

The Unity test framework itself supports different inputs for choice tests. This merge request emulates this behavior to make it possible to control test choices with pytest-embedded.

The current ways to choose tests are:

  • group
  • groups
  • invert group
  • name
  • attributes

For example:

2024-04-26 16:35:17 Here's the test menu, pick your combo:
2024-04-26 16:35:17 (1)	"can use static initializers for non-POD types" [restart_init]
2024-04-26 16:35:17 (2)	"static initialization guards work as expected" [misc]
2024-04-26 16:35:17 (3)	"global initializers run in the correct order" [misc]
2024-04-26 16:35:17 (4)	"before scheduler has started, static initializers work correctly" [misc]
2024-04-26 16:35:17 (5)	"init_priority extension works" [misc]
2024-04-26 16:35:17 (6)	"can use new and delete" [misc]
2024-04-26 16:35:17 (7)	"can call virtual functions" [misc]
2024-04-26 16:35:17 (8)	"can use std::vector" [misc]
2024-04-26 16:35:17 (9)	"stack smashing protection CXX" [stack_smash]
  1. choose all (1-9):
    dut.run_all_single_board_cases()
  2. choose by group(2-8):
    dut.run_all_single_board_cases(group="misc")
  3. choose by inverting group(1,9):
    dut.run_all_single_board_cases(group="!misc")
  4. choose by name(9):
    dut.run_all_single_board_cases(names=["stack smashing protection CXX"])

addition examples:
dut.run_all_single_board_cases(group="heap,mem_prot") # [heap][mem_prot]
dut.run_all_single_board_cases(attributes={"test_env": "noXtal32k"}) # [test_env=noXtal32k]

@horw horw force-pushed the feat/flexible_choises_for_run branch from ffa19cb to bf46653 Compare April 26, 2024 10:12
@horw horw requested a review from hfudev April 26, 2024 10:12
@horw horw force-pushed the feat/flexible_choises_for_run branch from bf46653 to aacd889 Compare April 29, 2024 11:02
@horw
Copy link
Member Author

horw commented Apr 29, 2024

There are some unit test cases that use multiple groups and arguments to filter test cases, for example: [heap][mem_prot] and [test_env=noXtal32k]. So, I have added these options.

@horw horw force-pushed the feat/flexible_choises_for_run branch from aacd889 to ea992d4 Compare April 29, 2024 11:06
Copy link
Member

@hfudev hfudev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some comments regarding api, overall the code LGTM.

pytest-embedded-idf/pytest_embedded_idf/unity_tester.py Outdated Show resolved Hide resolved
pytest-embedded-idf/pytest_embedded_idf/unity_tester.py Outdated Show resolved Hide resolved
pytest-embedded-idf/pytest_embedded_idf/unity_tester.py Outdated Show resolved Hide resolved
pytest-embedded-idf/pytest_embedded_idf/unity_tester.py Outdated Show resolved Hide resolved
@horw horw force-pushed the feat/flexible_choises_for_run branch 7 times, most recently from dc78072 to cbf9f30 Compare May 6, 2024 06:50
Copy link
Member

@hfudev hfudev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

pytest-embedded-idf/pytest_embedded_idf/unity_tester.py Outdated Show resolved Hide resolved
pytest-embedded-idf/pytest_embedded_idf/unity_tester.py Outdated Show resolved Hide resolved
pytest-embedded-idf/pytest_embedded_idf/unity_tester.py Outdated Show resolved Hide resolved
'-s',
'--embedded-services', 'esp,idf',
'--app-path', os.path.join(testdir.tmpdir, 'unit_test_app_esp32'),
'--log-cli-level', 'DEBUG',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're not checking the debug level in the test case, maybe remove the log cli level here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, sure

pytest-embedded-idf/tests/test_idf.py Outdated Show resolved Hide resolved
pytest-embedded-idf/pytest_embedded_idf/unity_tester.py Outdated Show resolved Hide resolved
@horw horw force-pushed the feat/flexible_choises_for_run branch from cbf9f30 to 07dcb19 Compare May 6, 2024 07:58
@hfudev hfudev merged commit 5a86810 into espressif:main May 6, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants