AutoML tests writing test output outside of a test which causes test process to fail with no failing test cases

[ericstj]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=601237
Build error leg or test failing: Microsoft.ML.AutoML.Tests.WorkItemExecution
Pull request: #7064

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": ["System.InvalidOperationException: There is no currently active test.", "at Microsoft.ML.AutoML.CancellationTokenStopTrainingManager"],
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Sample log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-machinelearning-refs-pull-7064-merge-fb45c2e048d6480197/Microsoft.ML.AutoML.Tests/1/console.02bc281b.log?helixlogtype=result

Unhandled exception: System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: There is no currently active test.
   at Xunit.Sdk.TestOutputHelper.GuardInitialized() in /_/src/xunit.execution/Sdk/Frameworks/TestOutputHelper.cs:line 52
   at Xunit.Sdk.TestOutputHelper.QueueTestOutput(String output) in /_/src/xunit.execution/Sdk/Frameworks/TestOutputHelper.cs:line 63
   at Microsoft.ML.AutoML.Test.AutoMLExperimentTests.<AutoMLExperiment_finish_training_when_time_is_up_Async>b__5_0(Object o, LoggingEventArgs e) in D:\a\_work\1\s\test\Microsoft.ML.AutoML.Tests\AutoMLExperimentTests.cs:line 159
   at Microsoft.ML.MLContext.ProcessMessage(IMessageSource source, ChannelMessage message) in D:\a\_work\1\s\src\Microsoft.ML.Data\MLContext.cs:line 162
   at Microsoft.ML.Runtime.HostEnvironmentBase`1.Dispatcher`1.DispatchCore(IMessageSource sender, TMessage message) in D:\a\_work\1\s\src\Microsoft.ML.Core\Environment\HostEnvironmentBase.cs:line 313
   at Microsoft.ML.Runtime.HostEnvironmentBase`1.ChannelBase.Info(MessageSensitivity sensitivity, String msg) in D:\a\_work\1\s\src\Microsoft.ML.Core\Environment\HostEnvironmentBase.cs:line 256
   at Microsoft.ML.Runtime.HostExtensions.Info(IChannel ch, String fmt) in D:\a\_work\1\s\src\Microsoft.ML.Core\Data\IHostEnvironment.cs:line 314
   at Microsoft.ML.AutoML.CancellationTokenStopTrainingManager.<.ctor>b__5_0() in D:\a\_work\1\s\src\Microsoft.ML.AutoML\AutoMLExperiment\IStopTrainingManager.cs:line 36
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.CancellationCallbackInfo.ExecuteCallback()
   at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean throwOnFirstException)
   --- End of inner exception stack trace ---
   at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean throwOnFirstException)
   at System.Threading.CancellationTokenSource.NotifyCancellation(Boolean throwOnFirstException)
   at System.Threading.CancellationTokenSource.TimerCallbackLogic(Object obj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.TimerQueueTimer.CallCallback()
   at System.Threading.TimerQueueTimer.Fire()
   at System.Threading.TimerQueue.FireNextTimers()
---> (Inner Exception #0) System.InvalidOperationException: There is no currently active test.
   at Xunit.Sdk.TestOutputHelper.GuardInitialized() in /_/src/xunit.execution/Sdk/Frameworks/TestOutputHelper.cs:line 52
   at Xunit.Sdk.TestOutputHelper.QueueTestOutput(String output) in /_/src/xunit.execution/Sdk/Frameworks/TestOutputHelper.cs:line 63
   at Microsoft.ML.AutoML.Test.AutoMLExperimentTests.<AutoMLExperiment_finish_training_when_time_is_up_Async>b__5_0(Object o, LoggingEventArgs e) in D:\a\_work\1\s\test\Microsoft.ML.AutoML.Tests\AutoMLExperimentTests.cs:line 159
   at Microsoft.ML.MLContext.ProcessMessage(IMessageSource source, ChannelMessage message) in D:\a\_work\1\s\src\Microsoft.ML.Data\MLContext.cs:line 162
   at Microsoft.ML.Runtime.HostEnvironmentBase`1.Dispatcher`1.DispatchCore(IMessageSource sender, TMessage message) in D:\a\_work\1\s\src\Microsoft.ML.Core\Environment\HostEnvironmentBase.cs:line 313
   at Microsoft.ML.Runtime.HostEnvironmentBase`1.ChannelBase.Info(MessageSensitivity sensitivity, String msg) in D:\a\_work\1\s\src\Microsoft.ML.Core\Environment\HostEnvironmentBase.cs:line 256
   at Microsoft.ML.Runtime.HostExtensions.Info(IChannel ch, String fmt) in D:\a\_work\1\s\src\Microsoft.ML.Core\Data\IHostEnvironment.cs:line 314
   at Microsoft.ML.AutoML.CancellationTokenStopTrainingManager.<.ctor>b__5_0() in D:\a\_work\1\s\src\Microsoft.ML.AutoML\AutoMLExperiment\IStopTrainingManager.cs:line 36
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.CancellationCallbackInfo.ExecuteCallback()
   at System.Threading.CancellationTokenSource.ExecuteCallbackHandlers(Boolean throwOnFirstException)<---

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=601237
Error message validated: [System.InvalidOperationException: There is no currently active test. at Microsoft.ML.AutoML.CancellationTokenStopTrainingManager]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 3/13/2024 7:06:40 PM UTC

Report

Build	Definition	Test	Pull Request
866886	dotnet/machinelearning	Microsoft.ML.AutoML.Tests.WorkItemExecution	#7298

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	1