Skip to content

Commit

Permalink
corerun supports user defined properties from command line. (dotnet#5…
Browse files Browse the repository at this point in the history
…1774)

* corerun supports user defined properties from command line.

* Update workflow help doc for corerun.

Co-authored-by: Elinor Fung <[email protected]>
  • Loading branch information
AaronRobinsonMSFT and elinor-fung authored Apr 27, 2021
1 parent 08e00be commit 66abf41
Show file tree
Hide file tree
Showing 2 changed files with 176 additions and 85 deletions.
121 changes: 69 additions & 52 deletions docs/workflow/testing/using-corerun.md
Original file line number Diff line number Diff line change
@@ -1,83 +1,100 @@

# Using corerun To Run .NET Application
# Using `corerun` To Run a .NET Application

In page [Using your .NET Runtime Build with dotnet cli](../using-dotnet-cli.md) gives detailed instructions on using the standard
command line host and SDK, dotnet.exe to run an application with the modified build of the
.NET Runtime built here. This is the preferred mechanism for you to officially deploy
your changes to other people since dotnet.exe and Nuget insure that you end up with a consistent
set of DLLs that can work together.
The page [Using your .NET Runtime Build with dotnet cli](../using-dotnet-cli.md) gives detailed instructions on using the standard
command line host (that is, `dotnet.exe` or `dotnet`), and SDK to run an application with a local build of the
.NET Runtime. This is the preferred mechanism for you to officially deploy
your changes to other people since dotnet.exe and NuGet ensure that you end up with a consistent
set of binaries that can work together.

However packing and unpacking the runtime DLLs adds extra steps to the deployment process and when
you are in the tight code-build-debug loop these extra steps are an issue.
However, packing and unpacking the runtime binaries adds extra steps to the deployment process. When
working in a tight edit-build-debug loop, these extra steps become cumbersome.

For this situation there is an alternative host to dotnet.exe called corerun.exe that is well suited
for this. It does not know about Nuget at all, and has very simple rules. It needs to find the
.NET runtime (that is coreclr.dll) and additionally any class library DLLs (e.g. System.Runtime.dll System.IO.dll ...).
For this tight edit-build-debug loop, there is a simplified alternative to `dotnet` called `corerun` which
does not know about NuGet at all. It just needs to find the .NET runtime (for example, `coreclr.dll`)
and any class library assemblies (for example, `System.Runtime.dll`, `System.IO.dll`, etc).

It does this by looking at two environment variables.
It does this using heuristics in the following order:

1. Check if the user passed the `--clr-path` argument.
1. Check if the `CORE_ROOT` environment variable is defined.
1. Check if the .NET runtime binary is in the same directory as the `corerun` binary.

* `CORE_ROOT` - The directory where to find the runtime DLLs itself (e.g. CoreCLR.dll).
Defaults to be next to the corerun.exe host itself.
* `CORE_LIBRARIES` - A directory to look for DLLS to resolve any assembly references.
It defaults CORE_ROOT if it is not specified.
Regardless of which method is used to discover the .NET runtime binary, its location is used to discover
both the .NET runtime binary and all base class library assemblies. Additional directories can be included
in the set of class library assemblies by defining the `CORE_LIBRARIES` environment variable.

These simple rules can be used in a number of ways
The above heuristics can be used in a number of ways.

## Getting the class library from the shared system-wide runtime

Consider that you already have a .NET application DLL called HelloWorld.dll and wish to run it
(You could make such a DLL by using 'dotnet new' 'dotnet restore' 'dotnet build' in a 'HelloWorld' directory).
Consider that you already have a .NET application assembly called `HelloWorld.dll` and wish to run it.
You could make such an assembly by using an officially installed .NET runtime with `dotnet new` and `dotnet build` in a `HelloWorld` directory.

If you execute the following
```cmd
set PATH=%PATH%;%CoreCLR%\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\
set CORE_LIBRARIES=%ProgramFiles%\dotnet\shared\Microsoft.NETCore.App\1.0.0
If you execute the following on Windows, the `HelloWorld` assembly will be run.

```cmd
set PATH=%PATH%;<repo_root>\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root\
set CORE_LIBRARIES=%ProgramFiles%\dotnet\shared\Microsoft.NETCore.App\1.0.0
corerun HelloWorld.dll
corerun HelloWorld.dll
```

for Linux use /usr/share for %Program Files%
On non-Windows platforms, setting environment variables is different but the logic is identical. For example, on macOS use `/usr/local/share` for `%ProgramFiles%`.

Where %CoreCLR% is the base of your CoreCLR repository, then it will run your HelloWorld. application.
You can see why this works. The first line puts build output directory (Your OS, architecture, and buildType
may be different) and thus corerun.exe you just built is on your path.
The second line tells corerun.exe where to find class library files, in this case we tell it
to find them where the installation of dotnet.exe placed its copy. (Note that version number in the path above may change)
The `<repo_root>` represents the base of your dotnet/runtime repository. The first line puts the build output directory
(your OS, architecture, and buildType may be different) and thus the `corerun` binary on your path.
The second line tells `corerun` where to find class library assemblies. In this case we tell it to find them where
the installation of `dotnet` placed its copy. The version number in the path may be different depending on what
is currently installed on your system.

Thus when you run 'corerun HelloWorld.dll' Corerun knows where to get the DLLs it needs. Notice that once
you set up the path and CORE_LIBRARIES environment, after a rebuild you can simply use corerun to run your
application (you don't have to move DLLs around)
Thus when you run `corerun HelloWorld.dll`, `corerun` knows where to get the assemblies it needs.
Once you set the path and `CORE_LIBRARIES` environment variable, after a rebuild you can simply use
`corerun` to run your application &ndash; you don't have to move any binaries around.

## Using corerun.exe to Execute a Published Application
## Using `corerun` to Execute a Published Application

When 'dotnet publish' publishes an application it deploys all the class libraries needed as well.
Thus if you simply change the CORE_LIBRARIES definition in the previous instructions to point at
that publication directory but RUN the corerun from your build output the effect will be that you
run your new runtime getting all the other code needed from that deployed application. This is
When `dotnet publish` publishes an application, it deploys all the class libraries needed as well.
Thus if you simply change the `CORE_LIBRARIES` definition in the previous instructions to point at
that publication directory, but run the `corerun` from your build output, the effect will be that you
run your new runtime getting all the other code needed from that deployed application. This is
very convenient because you don't need to modify the deployed application in order to test
your new runtime.

## How CoreCLR Tests use corerun.exe
## How CoreCLR Tests use `corerun`

When you execute 'runtime/src/tests/build.cmd' one of the things that it does is set up a directory where it
The test build script (`src/tests/build.cmd` or `src/tests/build.sh`) sets up a directory where it
gathers the CoreCLR that has just been built with the pieces of the class library that tests need.
It places this runtime in the directory
```cmd
runtime\artifacts\tests\coreclr\<OS>.<Arch>.<BuildType>\Tests\Core_Root
```
off the CoreCLR Repository. The way the tests are expected to work is that you set the environment
variable CORE_ROOT to this directory
(you don't have to set CORE_LIBRARIES) and you can run any tests. For example after building the tests
(running src\tests\build from the repository base) and running 'src\tests\run') you can do the following
`artifacts\tests\coreclr\<OS>.<Arch>.<BuildType>\Tests\Core_Root`
starting at the repository root. The way the tests are expected to work is that you can set the environment
variable `CORE_ROOT` to this directory &ndash; you don't have to set `CORE_LIBRARIES` since the test environment has copied all base class libraries assemblies to this `Core_Root` directory &ndash; and you can run any test. For example, after building the tests
(running `src\tests\build` from the repository base), you can do the following on Windows to set up an environment where `corerun` can run any test.

```cmd
set PATH=%PATH%;%CoreCLR%\artifacts\Product\windows.x64.Debug
set CORE_ROOT=%CoreCLR%\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root
set PATH=%PATH%;<repo_root>\artifacts\Product\windows.x64.Debug
set CORE_ROOT=<repo_root>\artifacts\tests\coreclr\windows.x64.Debug\Tests\Core_Root
```
sets you up so that corerun can run any of the test. For example
For example, the following runs the finalizerio test on Windows.

```cmd
corerun artifacts\tests\coreclr\windows.X64.Debug\GC\Features\Finalizer\finalizeio\finalizeio\finalizeio.exe
corerun artifacts\tests\coreclr\windows.x64.Debug\GC\Features\Finalizer\finalizeio\finalizeio\finalizeio.dll
```
runs the finalizerio test.

## Additional `corerun` options

The `corerun` binary is designed to be a platform agnostic tool for quick testing of a locally built .NET runtime.
This means the `corerun` binary must be able to feasibly exercise any scenario the official `dotnet` binary is capable
of. It must also be able to help facilitate .NET runtime development and investigation of test failures.
See `corerun --help` for additional details.

**Options**

`--clr-path <PATH>` - Pass the location of Core Root on the command line.
- For example, `corerun --clr-path /usr/scratch/private_build HelloWorld.dll`

`--property <PROPERTY>` - Supply a property to pass to the .NET runtime during initialization.
- For example, `corerun --property System.GC.Concurrent=true HelloWorld.dll`

`--debug` - Wait for a debugger to attach prior to loading the .NET runtime.
- For example, `corerun --debug HelloWorld.dll`
140 changes: 107 additions & 33 deletions src/coreclr/hosts/corerun/corerun.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ struct configuration
int entry_assembly_argc;
const char_t** entry_assembly_argv;

// Collection of user-defined key/value pairs that will be appended
// to the initialization of the runtime.
std::vector<string_t> user_defined_keys;
std::vector<string_t> user_defined_values;

// Wait for debugger to be attached.
bool wait_to_debug;

Expand Down Expand Up @@ -288,40 +293,48 @@ static int run(const configuration& config)
pal::string_utf8_t app_path_ni_utf8 = pal::convert_to_utf8(std::move(app_path_ni));
pal::string_utf8_t native_search_dirs_utf8 = pal::convert_to_utf8(native_search_dirs.str());

// Allowed property names:
//
std::vector<pal::string_utf8_t> user_defined_keys_utf8;
std::vector<pal::string_utf8_t> user_defined_values_utf8;
for (const string_t& str : config.user_defined_keys)
user_defined_keys_utf8.push_back(pal::convert_to_utf8(str.c_str()));
for (const string_t& str : config.user_defined_values)
user_defined_values_utf8.push_back(pal::convert_to_utf8(str.c_str()));

// Set base initialization properties.
std::vector<const char*> propertyKeys;
std::vector<const char*> propertyValues;

// TRUSTED_PLATFORM_ASSEMBLIES
// - The list of complete paths to each of the fully trusted assemblies
//
propertyKeys.push_back("TRUSTED_PLATFORM_ASSEMBLIES");
propertyValues.push_back(tpa_list_utf8.c_str());

// APP_PATHS
// - The list of paths which will be probed by the assembly loader
//
propertyKeys.push_back("APP_PATHS");
propertyValues.push_back(app_path_utf8.c_str());

// APP_NI_PATHS
// - The list of additional paths that the assembly loader will probe for ngen images
//
propertyKeys.push_back("APP_NI_PATHS");
propertyValues.push_back(app_path_ni_utf8.c_str());

// NATIVE_DLL_SEARCH_DIRECTORIES
// - The list of paths that will be probed for native DLLs called by PInvoke
const char* propertyKeys[] =
{
"TRUSTED_PLATFORM_ASSEMBLIES",
"APP_PATHS",
"APP_NI_PATHS",
"NATIVE_DLL_SEARCH_DIRECTORIES",
};
propertyKeys.push_back("NATIVE_DLL_SEARCH_DIRECTORIES");
propertyValues.push_back(native_search_dirs_utf8.c_str());

const char* propertyValues[] =
{
// TRUSTED_PLATFORM_ASSEMBLIES
tpa_list_utf8.c_str(),
// APP_PATHS
app_path_utf8.c_str(),
// APP_NI_PATHS
app_path_ni_utf8.c_str(),
// NATIVE_DLL_SEARCH_DIRECTORIES
native_search_dirs_utf8.c_str(),
};
// Sanity check before adding user-defined properties
assert(propertyKeys.size() == propertyValues.size());

int propertyCount = (int)(sizeof(propertyKeys) / sizeof(propertyKeys[0]));
// Insert user defined properties
for (const pal::string_utf8_t& str : user_defined_keys_utf8)
propertyKeys.push_back(str.c_str());
for (const pal::string_utf8_t& str : user_defined_values_utf8)
propertyValues.push_back(str.c_str());

assert(propertyKeys.size() == propertyValues.size());
int propertyCount = (int)propertyKeys.size();

// Construct arguments
pal::string_utf8_t exe_path_utf8 = pal::convert_to_utf8(std::move(exe_path));
Expand All @@ -331,16 +344,16 @@ static int run(const configuration& config)

logger_t logger{
exe_path_utf8.c_str(),
propertyCount, propertyKeys, propertyValues,
propertyCount, propertyKeys.data(), propertyValues.data(),
entry_assembly_utf8.c_str(), config.entry_assembly_argc, argv_utf8.get() };

int result;
result = coreclr_init_func(
exe_path_utf8.c_str(),
"corerun",
propertyCount,
propertyKeys,
propertyValues,
propertyKeys.data(),
propertyValues.data(),
&CurrentClrInstance,
&CurrentAppDomainId);
if (FAILED(result))
Expand Down Expand Up @@ -397,12 +410,21 @@ static void display_usage()
W("Execute the managed assembly with the passed in arguments\n")
W("\n")
W("Options:\n")
W(" -c, --clr-path - path to CoreCLR binary and managed CLR assemblies\n")
W(" -d, --debug - causes corerun to wait for a debugger to attach before executing\n")
W(" -?, -h, --help - show this help\n")
W(" -c, --clr-path - path to CoreCLR binary and managed CLR assemblies.\n")
W(" -p, --property - Property to pass to runtime during initialization.\n")
W(" If a property value contains spaces, quote the entire argument.\n")
W(" May be supplied multiple times. Format: <key>=<value>.\n")
W(" -d, --debug - causes corerun to wait for a debugger to attach before executing.\n")
W(" -?, -h, --help - show this help.\n")
W("\n")
W("The runtime binary is searched for in --clr-path, CORE_ROOT environment variable, then\n")
W("in the directory the corerun binary is located.\n")
W("\n")
W("CoreCLR is searched for in %%CORE_ROOT%%, then in the directory\n")
W("the corerun binary is located.\n"));
W("Example:\n")
W("Wait for a debugger to attach, provide 2 additional properties for .NET\n")
W("runtime initialization, and pass an argument to the HelloWorld.dll assembly.\n")
W(" corerun -d -p System.GC.Concurrent=true -p \"FancyProp=/usr/first last/root\" HelloWorld.dll arg1\n")
);
}

// Parse the command line arguments
Expand Down Expand Up @@ -466,7 +488,29 @@ static bool parse_args(
break;
}
}
else if ((pal::strcmp(option, W("d")) == 0 || (pal::strcmp(option, W("debug")) == 0)))
else if (pal::strcmp(option, W("p")) == 0 || (pal::strcmp(option, W("property")) == 0))
{
i++;
if (i >= argc)
{
pal::fprintf(stderr, W("Option %s: missing property\n"), arg);
break;
}

string_t prop = argv[i];
size_t delim_maybe = prop.find(W('='));
if (delim_maybe == string_t::npos)
{
pal::fprintf(stderr, W("Option %s: '%s' missing property value\n"), arg, prop.c_str());
break;
}

string_t key = prop.substr(0, delim_maybe);
string_t value = prop.substr(delim_maybe + 1);
config.user_defined_keys.push_back(std::move(key));
config.user_defined_values.push_back(std::move(value));
}
else if (pal::strcmp(option, W("d")) == 0 || (pal::strcmp(option, W("debug")) == 0))
{
config.wait_to_debug = true;
}
Expand Down Expand Up @@ -526,6 +570,7 @@ extern "C" __declspec(dllexport) HRESULT __cdecl GetCurrentClrDetails(void** clr
//

#define THROW_IF_FALSE(stmt) if (!(stmt)) throw W(#stmt);
#define THROW_IF_TRUE(stmt) if (stmt) throw W(#stmt);
static int self_test()
{
try
Expand Down Expand Up @@ -556,6 +601,35 @@ static int self_test()
THROW_IF_FALSE(!config.entry_assembly_fullpath.empty());
THROW_IF_FALSE(config.entry_assembly_argc == 1);
}
{
configuration config{};
const char_t* args[] = { W(""), W("-p"), W("invalid"), W("foo") };
THROW_IF_TRUE(parse_args(4, args, config));
}
{
configuration config{};
const char_t* args[] = { W(""), W("-p"), W("empty="), W("foo") };
THROW_IF_FALSE(parse_args(4, args, config));
THROW_IF_FALSE(config.user_defined_keys.size() == 1);
THROW_IF_FALSE(config.user_defined_values.size() == 1);
THROW_IF_FALSE(!config.entry_assembly_fullpath.empty());
}
{
configuration config{};
const char_t* args[] = { W(""), W("-p"), W("one=1"), W("foo") };
THROW_IF_FALSE(parse_args(4, args, config));
THROW_IF_FALSE(config.user_defined_keys.size() == 1);
THROW_IF_FALSE(config.user_defined_values.size() == 1);
THROW_IF_FALSE(!config.entry_assembly_fullpath.empty());
}
{
configuration config{};
const char_t* args[] = { W(""), W("-p"), W("one=1"), W("--property"), W("System.GC.Concurrent=true"), W("foo") };
THROW_IF_FALSE(parse_args(6, args, config));
THROW_IF_FALSE(config.user_defined_keys.size() == 2);
THROW_IF_FALSE(config.user_defined_values.size() == 2);
THROW_IF_FALSE(!config.entry_assembly_fullpath.empty());
}
{
string_t path;
path = W("path");
Expand Down

0 comments on commit 66abf41

Please sign in to comment.