JITServer client does not acquire correct monitors for local compilation following remote compilation

[cjjdespres]

While looking at #14755, I managed to build JDK21 in debug mode (after commenting out various asserts that would cause the JDK to fail to build). I got one assert to fire that should be fixed, though it's not the cause of that test failure.

The failing assert is:

Assertion failed at /home/despresc/dev/testing/openj9-openjdk-jdk21/openj9/runtime/compiler/infra/J9MonitorTable.cpp:255: false
VMState: 0x00050080
	comp thread 1 does not have classUnloadMonitor

What happens is here:

openj9/runtime/compiler/control/JITClientCompilationThread.cpp

Line 3752 in 7c8005e

if ((rtn = compiler->compile()) != COMPILATION_SUCCEEDED)

we compile locally following a remote compilation (since I enabled enableJITServerFollowRemoteCompileWithLocalCompile for a particular method), and this eventually calls some frontend method (TR_J9VMBase::isInvokeCacheEntryAnArray in this case) during IL generation that tries to enter a TR::VMAccessCriticalSection::VMAccessCriticalSection, which eventually asserts if it finds that the thread hasn't acquired a class unload monitor.

Judging from this bit of the code just before a local compilation starts:

openj9/runtime/compiler/control/CompilationThread.cpp

Lines 9736 to 9757 in 7c8005e

	// If we want to compile without VM access, now it's the time to release it
	// For the JITClient we must not enter this path. The class unload monitor
	// will not be acquired/released and we'll only release VMaccess when
	// waiting for a reply from the server
	if (!compiler->getOption(TR_DisableNoVMAccess) &&
	!_methodBeingCompiled->_aotCodeToBeRelocated &&
	!_methodBeingCompiled->isRemoteCompReq())
	{
	#if defined(J9VM_GC_DYNAMIC_CLASS_UNLOADING)
	bool doAcquireClassUnloadMonitor = true;
	#else
	bool doAcquireClassUnloadMonitor = compiler->getOption(TR_EnableHCR) || compiler->getOption(TR_FullSpeedDebug);
	#endif
	if (doAcquireClassUnloadMonitor)
	{
	_compInfo.debugPrint(NULL, "\tcompilation thread acquiring class unload monitor\n");
	TR::MonitorTable::get()->readAcquireClassUnloadMonitor(getCompThreadId());
	haveLockedClassUnloadMonitor = true;
	}
	releaseVMAccess(vmThread);
	// GC can go ahead now.
	}

I think we should be acquiring the class unload monitor just before we release vm access before our local-following-remote compilation in the JIT client compilation thread code.

@mpirvu Does that seem right to you?