Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a crashy program for SetString test. #4443
base: 25.lts.1+
Are you sure you want to change the base?
Add a crashy program for SetString test. #4443
Changes from all commits
f533d4e
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A downside of adding this
setstring_test
is that it would need to be built and run manually by developers, and won't run automatically in any CI. A cursory look shows thatInitAndRunAllTests()
is unnecessary since it doesn't do anything special. Why not just addsetstring_test.cc
to a larger test-only build target, e.g. "extension_test" in l.15 ?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could alternatively just add this test target to
cobalt/build/cobalt_configuration.py
so that it's run in CI.I think there's limited utility in running it automatically, though, as long as it requires manual steps in order to verify the result (see other comment thread).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yell0wd0g I can suggest rename the executable to 'crashy_process' or alike and remove use of gtest library to avoid confusion with tests, because it is not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I didn't notice the need for manual test.
An interesting pattern to consider then would be the "MANUAL_FooTest" used by Chromium (some doc here, one example here). Test cases go into a larger binary (easier to deploy and conceptualize) but don't run by default unless the test binary is run with the "--run-manual" command line flag.
Perhaps it'd be overkill for this case since it's going to 25.lts, but certainly something to consider for 26+.
Otherwise I'm good with other reviewers' LGTMs. Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yell0wd0g Changed test name to MANUAL_SetString.
I hope, Cobalt team think about adding a test like 'browser_tests' in Chromium.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test doesn't actually do any verifications about the expected annotations in the core dump. Will those be added later? Or is the expectation that the core dump, and its annotations, will be verified manually after this test is run?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Such happened, target part of the test is cross-platform. But verification code, apparently, can be platform-specific only. I've implemented it for PlayStation, but I see no way to implement similar for UWP/Windows. This is current status.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I support getting the code in, if it helps bringing up integration tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could enable platforms to add platform-specific verification code, if we wanted to (and there's a viable way to automatically do the verification locally on the platform, for example by examining an expected crash dump stored on the file system).
One way to do that would be by adding a new method to the CrashHandler Starboard extension itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have had a doubt about possible way to make verification locally on the platform. This way is preferable, as doesn't involve host side. I can consider this way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hlwarriner I see no a viable way to automatically do the verification locally on the platform.