Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a crashy program for SetString test. #4443

Open
wants to merge 1 commit into
base: 25.lts.1+
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions starboard/extension/BUILD.gn
Original file line number Diff line number Diff line change
@@ -29,3 +29,13 @@ target(gtest_target_type, "extension_test") {
deps += cobalt_platform_dependencies
}
}

target(gtest_target_type, "setstring_test") {
v-kryachko marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A downside of adding this setstring_test is that it would need to be built and run manually by developers, and won't run automatically in any CI. A cursory look shows that InitAndRunAllTests() is unnecessary since it doesn't do anything special. Why not just add setstring_test.cc to a larger test-only build target, e.g. "extension_test" in l.15 ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could alternatively just add this test target to cobalt/build/cobalt_configuration.py so that it's run in CI.

I think there's limited utility in running it automatically, though, as long as it requires manual steps in order to verify the result (see other comment thread).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yell0wd0g I can suggest rename the executable to 'crashy_process' or alike and remove use of gtest library to avoid confusion with tests, because it is not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't notice the need for manual test.

An interesting pattern to consider then would be the "MANUAL_FooTest" used by Chromium (some doc here, one example here). Test cases go into a larger binary (easier to deploy and conceptualize) but don't run by default unless the test binary is run with the "--run-manual" command line flag.

Perhaps it'd be overkill for this case since it's going to 25.lts, but certainly something to consider for 26+.

Otherwise I'm good with other reviewers' LGTMs. Thanks!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yell0wd0g Changed test name to MANUAL_SetString.
I hope, Cobalt team think about adding a test like 'browser_tests' in Chromium.

testonly = true
has_pedantic_warnings = true
sources = [ "setstring_test.cc" ]
deps = [ "//testing/gtest" ]
if (sb_is_modular && current_toolchain == cobalt_toolchain) {
deps += cobalt_platform_dependencies
}
}
58 changes: 58 additions & 0 deletions starboard/extension/setstring_test.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
// Copyright 2024 The Cobalt Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// This is a kind of death test, as described in
// https://github.com/google/googletest/blob/main/docs/advanced.md#death-tests,
// except, it doesn't spawn another process, but terminates itself.
// It is intended to be started from a host machine on a target device
// to generate a coredump, which is verified on the host.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test doesn't actually do any verifications about the expected annotations in the core dump. Will those be added later? Or is the expectation that the core dump, and its annotations, will be verified manually after this test is run?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such happened, target part of the test is cross-platform. But verification code, apparently, can be platform-specific only. I've implemented it for PlayStation, but I see no way to implement similar for UWP/Windows. This is current status.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I support getting the code in, if it helps bringing up integration tests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could enable platforms to add platform-specific verification code, if we wanted to (and there's a viable way to automatically do the verification locally on the platform, for example by examining an expected crash dump stored on the file system).

One way to do that would be by adding a new method to the CrashHandler Starboard extension itself.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have had a doubt about possible way to make verification locally on the platform. This way is preferable, as doesn't involve host side. I can consider this way.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hlwarriner I see no a viable way to automatically do the verification locally on the platform.


#include "starboard/client_porting/wrap_main/wrap_main.h"
#include "starboard/event.h"
#include "starboard/extension/crash_handler.h"
#include "starboard/system.h"
#include "testing/gtest/include/gtest/gtest.h"

const char* test_string = "TestString";

TEST(ExtensionTest, MANUAL_SetString) {
typedef CobaltExtensionCrashHandlerApi ExtensionApi;

const ExtensionApi* extension_api = static_cast<const ExtensionApi*>(
SbSystemGetExtension(kCobaltExtensionCrashHandlerName));
if (!extension_api) {
return;
}

extension_api->SetString("Annotation1", test_string);
extension_api->SetString("Annotation2", test_string);
extension_api->SetString("Annotation3", test_string);
extension_api->SetString("Annotation4", test_string);

SbSystemBreakIntoDebugger();
v-kryachko marked this conversation as resolved.
Show resolved Hide resolved
}

namespace {
int InitAndRunAllTests(int argc, char** argv) {
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
} // namespace

// When we are building Evergreen we need to export SbEventHandle so that the
// ELF loader can find and invoke it.
#if SB_IS(MODULAR)
SB_EXPORT
#endif // SB_IS(MODULAR)
STARBOARD_WRAP_SIMPLE_MAIN(InitAndRunAllTests);