Error. compiler crashed (exit code 4) - cannot access unnamed package #20786 #21000

JasonFengJ9 · 2025-01-22T22:14:38Z

Failure link

From internal Test_openjdk23_j9_sanity.openjdk_x86-64_windows_testList_2 (win22x86-svl-rt7-1)

openjdk version "23.0.1-beta" 2024-10-15
IBM Semeru Runtime Open Edition 23.0.1+11-202501210331 (build 23.0.1-beta+11-202501210331)
Eclipse OpenJ9 VM 23.0.1+11-202501210331 (build master-63f23a3112, JRE 23 Windows Server 2022 amd64-64-Bit Compressed References 20250121_91 (JIT enabled, AOT enabled)
OpenJ9   - 63f23a3112
OMR      - 6bfb20f78
JCL      - d98479bbe based on jdk-23.0.1+11)

Rerun in Grinder - Change TARGET to run only the failed test targets

Optional info

Failure output (captured from console output)

[2025-01-21T10:11:47.319Z] variation: -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-JITServerTechPreviewMessage Mode650
[2025-01-21T10:11:47.319Z] JVM_OPTIONS:  -Xdump:system:none -Xdump:heap:none -Xdump:system:events=gpf+abort+traceassert+corruptcache -XX:-JITServerTechPreviewMessage -XX:-UseCompressedOops -Xverbosegclog 

[2025-01-21T10:12:57.570Z] TEST: java/lang/IntegralPrimitiveToString.java

[2025-01-21T10:12:57.571Z] C:\Users\jenkins\workspace\Test_openjdk23_j9_sanity.openjdk_x86-64_windows_testList_2\aqa-tests\openjdk\openjdk-jdk\test\jdk\java\lang\IntegralPrimitiveToString.java:24: error: cannot access unnamed package
[2025-01-21T10:12:57.571Z] import org.testng.annotations.DataProvider;
[2025-01-21T10:12:57.571Z] ^
[2025-01-21T10:12:57.571Z]   /com/google/inject/internal/util/StackTraceElements$InMemoryStackTraceElement.class
[2025-01-21T10:12:57.571Z] 1 error
[2025-01-21T10:12:57.571Z] printing javac parameters to: C:\temp\javac.20250121_021254.args
[2025-01-21T10:12:57.571Z] 
[2025-01-21T10:12:57.571Z] TEST RESULT: Error. compiler crashed (exit code 4)

[2025-01-21T11:55:31.319Z] C:\Users\jenkins\workspace\Test_openjdk23_j9_sanity.openjdk_x86-64_windows_testList_2\aqa-tests\openjdk\openjdk-jdk\test\jdk\sun\reflect\ReflectionFactory\ReflectionFactoryTest.java:24: error: cannot access unnamed package
[2025-01-21T11:55:31.319Z] import java.io.ByteArrayInputStream;
[2025-01-21T11:55:31.319Z] ^
[2025-01-21T11:55:31.319Z]   /com/beust/jcommander/ParameterDescription$SubParameterIndex.class
[2025-01-21T11:55:31.319Z] 1 error
[2025-01-21T11:55:31.319Z] printing javac parameters to: C:\temp\javac.20250121_035530.args
[2025-01-21T11:55:31.319Z] 
[2025-01-21T11:55:31.319Z] TEST RESULT: Error. compiler crashed (exit code 4)
[2025-01-21T11:55:31.319Z] --------------------------------------------------
[2025-01-21T11:55:59.869Z] Test results: passed: 633; failed: 5; error: 293
[2025-01-21T11:56:34.377Z] Report written to C:\Users\jenkins\workspace\Test_openjdk23_j9_sanity.openjdk_x86-64_windows_testList_2\aqa-tests\TKG\output_17374542963639\jdk_lang_1\report\html\report.html
[2025-01-21T11:56:34.377Z] Results written to C:\Users\jenkins\workspace\Test_openjdk23_j9_sanity.openjdk_x86-64_windows_testList_2\aqa-tests\TKG\output_17374542963639\jdk_lang_1\work
[2025-01-21T11:56:34.377Z] Error: Some tests failed or other problems occurred.
[2025-01-21T11:56:34.377Z] -----------------------------------
[2025-01-21T11:56:34.377Z] jdk_lang_1_FAILED

openjdk23_j9_sanity.openjdk_x86-64_linux
openjdk21_j9_extended.openjdk_x86-64_linux
openjdk21_j9_extended.openjdk_x86-64_linux

There was a similar issue

Error. compiler crashed (exit code 4) - cannot access unnamed package #20786

FYI @hzongaro

The text was updated successfully, but these errors were encountered:

hzongaro · 2025-01-23T16:54:31Z

@a7ehuo, may I ask you to take a look at this problem?

a7ehuo · 2025-01-23T21:26:22Z

I'm trying to reproduce this failure locally with JDK21 on x86-64 linux machine. With jdk11_tier1_buffer_1, 120 runs all passed. I'm gonna try jdk_other_1 as well since jdk_other_1 seems to have more failures.

llxia · 2025-01-23T21:47:13Z

We also see this issue on RHEL10. In our case, it was jdk_vector_1 with a high failure rate. For details, see link

hzongaro · 2025-01-24T02:42:27Z

I was also able to reproduce that compiler crash in an internal grinder run running jdk_security1_1 and jdk_security2_1.

By the way, the previous problem with a similar symptom — #20786 — was caused by unbalanced Start internal control flow and End internal control flow labels. On the off chance that this is caused by the same problem, you might try picking up this draft OMR pull request eclipse-omr/omr#7597 to see whether it hits any assertions running sanity.openjdk. I was successful running testing with it on x86.

a7ehuo · 2025-01-28T20:36:58Z

Just an update. I'm making some progress on narrowing the problem down, which is kind of challenging because there are no extra error messages except cannot access unnamed package and compiler crashed (exit code 4). Once I started limiting the number of compiled methods, the problem also seems going away.

A few observations so far about this failure:

I tried Henry's PR #7597 that has the assert on mismatched start and end internal control label. I can reproduce the issue in Grinder ID 47031 but not hitting the assert. FR: 56/300 runs
The issue so far looks like only happen on Skylake
- I checked a few reported failures, all of which happened on Skylake
- Therefore I started using a Skylake test machine and I can reliably reproduce the issue although the FR is not 100%
The problem seems to go away if I disable inlining of vectorizedHashCode
- This failure is intermittent, by default I can reproduce the failure five out of the six subtests from the TESTLIST [1] by running the TESTLIST once. After disabling vectorizedHashCode, I'm not able to reproduce the failure when running the TESTLIST once.

[1] TESTLIST=jdk11_tier1_buffer_1,jdk_other_1,jdk_vector_1,jdk_security1_1,jdk_security2_1,jdk_foreign_1

a7ehuo · 2025-01-29T23:57:50Z

Continuing to narrow it down, I'm able to reproduce the error by limiting the compiled methods to only one method: java/util/Arrays.hashCode([B)I which calls ArraysSupport.vectorizedHashCode for byte arrays.

The problem seems to happen with VectorLength512 and unrollCount=4. I wasn't able to reproduce the error by choosing VectorLength256, or VectorLength128, or unrollCount=1, or unrollCount=2.
I adapted the functional test TestArraysSupport.java that Brad created to run as a standalone test for byte arrays on Skylake. It passes.
So far the generated inlined ArraysSupport.vectorizedHashCode instructions look fine to me. The obvious things such as matching internal control flow labels and the number of register dependencies look correct too.

===============================================
Running test jdk11_tier1_buffer_1 ...
===============================================
...
...
rerun:
cd /home/jenkins/ahuo_issue_21000/aqa-tests/TKG/output_17381748712070/jdk11_tier1_buffer_1/work/scratch/0 && \
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus \
HOME=/home/jenkins \
LANG=en_US.UTF-8 \
PATH=/bin:/usr/bin:/usr/sbin \
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop \
XDG_RUNTIME_DIR=/run/user/1000 \
XDG_SESSION_CLASS=user \
XDG_SESSION_ID=46230 \
XDG_SESSION_TYPE=tty \
    /home/jenkins/ahuo_issue_21000/sdks/jdk/bin/javac \
        -J-ea \
        -J-esa \
        -J-Xmx512m \
        -J-XX:-UseCompressedOops \
        -J-Xverbosegclog \
        -J-Xshareclasses:none \
        -J-Xnoaot \
        -J-Xjit:limit={java/util/Arrays.hashCode([B)I},disableVectorHashCodeSecondLoop,disableInlining,verbose,vlog=/home/jenkins/ahuo_issue_21000/logs/vlogs/vlog3,{java/util/Arrays.hashCode([B)I}(optLevel=cold,traceFull,traceCG,log=/home/jenkins/ahuo_issue_21000/logs/jitlogs/jitlog3) \
        -J-Dtest.vm.opts='-ea -esa -Xmx512m -XX:-UseCompressedOops -Xverbosegclog -Xshareclasses:none -Xnoaot -Xjit:limit={java/util/Arrays.hashCode([B)I},disableVectorHashCodeSecondLoop,disableInlining,verbose,vlog=/home/jenkins/ahuo_issue_21000/logs/vlogs/vlog3,{java/util/Arrays.hashCode([B)I}(optLevel=cold,traceFull,traceCG,log=/home/jenkins/ahuo_issue_21000/logs/jitlogs/jitlog3)' \
        -J-Dtest.tool.vm.opts='-J-ea -J-esa -J-Xmx512m -J-XX:-UseCompressedOops -J-Xverbosegclog -J-Xshareclasses:none -J-Xnoaot -J-Xjit:limit={java/util/Arrays.hashCode([B)I},disableVectorHashCodeSecondLoop,disableInlining,verbose,vlog=/home/jenkins/ahuo_issue_21000/logs/vlogs/vlog3,{java/util/Arrays.hashCode([B)I}(optLevel=cold,traceFull,traceCG,log=/home/jenkins/ahuo_issue_21000/logs/jitlogs/jitlog3)' \
        -J-Dtest.compiler.opts= \
        -J-Dtest.java.opts= \
        -J-Dtest.jdk=/home/jenkins/ahuo_issue_21000/sdks/jdk \
        -J-Dcompile.jdk=/home/jenkins/ahuo_issue_21000/sdks/jdk \
        -J-Dtest.timeout.factor=8.0 \
        -J-Dtest.root=/home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk \
        -J-Dtest.name=java/nio/Buffer/ByteBufferViews.java \
        -J-Dtest.file=/home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/nio/Buffer/ByteBufferViews.java \
        -J-Dtest.src=/home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/nio/Buffer \
        -J-Dtest.src.path=/home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/nio/Buffer \
        -J-Dtest.classes=/home/jenkins/ahuo_issue_21000/aqa-tests/TKG/output_17381748712070/jdk11_tier1_buffer_1/work/classes/0/java/nio/Buffer/ByteBufferViews.d \
        -J-Dtest.class.path=/home/jenkins/ahuo_issue_21000/aqa-tests/TKG/output_17381748712070/jdk11_tier1_buffer_1/work/classes/0/java/nio/Buffer/ByteBufferViews.d \
        -J-Dtest.class.path.prefix=/home/jenkins/ahuo_issue_21000/aqa-tests/TKG/output_17381748712070/jdk11_tier1_buffer_1/work/classes/0/java/nio/Buffer/ByteBufferViews.d:/home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/nio/Buffer \
        -d /home/jenkins/ahuo_issue_21000/aqa-tests/TKG/output_17381748712070/jdk11_tier1_buffer_1/work/classes/0/java/nio/Buffer/ByteBufferViews.d \
        -sourcepath /home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/nio/Buffer \
        -classpath /home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/nio/Buffer:/home/jenkins/ahuo_issue_21000/aqa-tests/TKG/output_17381748712070/jdk11_tier1_buffer_1/work/classes/0/java/nio/Buffer/ByteBufferViews.d:/home/jenkins/ahuo_issue_21000/jvmtest/openjdk/jtreg/lib/testng-7.3.0.jar:/home/jenkins/ahuo_issue_21000/jvmtest/openjdk/jtreg/lib/guice-5.1.0.jar:/home/jenkins/ahuo_issue_21000/jvmtest/openjdk/jtreg/lib/jcommander-1.82.jar /home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/nio/Buffer/ByteBufferViews.java
direct:
/home/jenkins/ahuo_issue_21000/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/nio/Buffer/ByteBufferViews.java:30: error: cannot access unnamed package
import org.testng.annotations.DataProvider;
^
  /com/beust/jcommander/ParameterDescription$SubParameterIndex.class
1 error
printing javac parameters to: /home/jenkins/ahuo_issue_21000/aqa-tests/TKG/output_17381748712070/jdk11_tier1_buffer_1/work/scratch/0/javac.20250129_102135.args

TEST RESULT: Error. compiler crashed (exit code 4)
--------------------------------------------------

hzongaro · 2025-01-30T14:07:43Z

@BradleyWood, it looks like this problem could be related to ArraysSupport.vectorizedHashCode. May I ask you to look into it?

BradleyWood · 2025-01-31T15:12:51Z

@a7ehuo Do you have a log from one of the failing cases?

a7ehuo · 2025-01-31T16:11:20Z

@a7ehuo Do you have a log from one of the failing cases?

There are a few jitlogs generated: 21000_jitlogs.zip

a7ehuo · 2025-01-31T16:38:37Z

Attached logs from another reproduced case: 21000_logs_2.zip. This time four compilation logs are generated. The attachment also includes vlogs and a console log for reference.

BradleyWood · 2025-01-31T17:01:06Z

I spotted something suspicious in the log, but I don't know if its the cause of this particular issue.

mov	rcx, qword ptr [rsp+0x40]		# L8RegMem, SymRef  a<parm 0 [B>[#428  Parm +64] [flags 0xc0000107 0x0 ]
mov	r8d, dword ptr [rcx+0x8]		# L4RegMem, SymRef  <contiguous-array-size>[#327  Shadow +8] [flags 0x603 0x0 ]
mov	esi, 0x00000001	# MOV4RegImm4
xor	edx, edx		# XOR4RegReg
mov	rax, rsi		# MOV8RegReg
Label L0064:			# label	# (Start of internal control flow)
mov	rdi, r8		# MOV8RegReg
and	edi, 0xffffffffffffffc0	# AND4RegImm4
cmp	edx, edi		# CMP4RegReg
jge	Label L0065			# JGE4
movd	xmm6, esi		# MOVDRegReg4     ;<------- Possible bug movd might not zero out high bits; but zmm6 is used later for simd.
pxor	zmm7, zmm7		# PXORRegReg
pxor	zmm8, zmm8		# PXORRegReg
pxor	zmm9, zmm9		# PXORRegReg
movdqu	zmm1, zmmword ptr [0x00007da406200200]		# MOVDQURegReg
movdqu	zmm2, zmmword ptr [0x00007da406200240]		# MOVDQURegReg
movdqu	zmm3, zmmword ptr [0x00007da406200280]		# MOVDQURegReg
movdqu	zmm4, zmmword ptr [0x00007da4062002c0]		# MOVDQURegReg
movdqu	zmm5, zmmword ptr [0x00007da406200300]		# MOVDQURegReg
Label L0066:			# label
pmovsxbd	zmm0, zmmword ptr [rcx+1*rdx+0x10]		# PMOVSXBDRegReg, SymRef [#434 +16]
pmulld	zmm0, zmm2		# PMULLDRegReg
pmulld	zmm6, zmm1		# PMULLDRegReg
paddd	zmm6, zmm0		# PADDDRegReg
pmovsxbd	zmm0, zmmword ptr [rcx+1*rdx+0x20]		# PMOVSXBDRegReg, SymRef [#435 +32]
pmulld	zmm0, zmm3		# PMULLDRegReg
pmulld	zmm7, zmm1		# PMULLDRegReg
paddd	zmm7, zmm0		# PADDDRegReg
pmovsxbd	zmm0, zmmword ptr [rcx+1*rdx+0x30]		# PMOVSXBDRegReg, SymRef [#436 +48]
pmulld	zmm0, zmm4		# PMULLDRegReg
pmulld	zmm8, zmm1		# PMULLDRegReg
paddd	zmm8, zmm0		# PADDDRegReg
pmovsxbd	zmm0, zmmword ptr [rcx+1*rdx+0x40]		# PMOVSXBDRegReg, SymRef [#437 +64]
pmulld	zmm0, zmm5		# PMULLDRegReg
pmulld	zmm9, zmm1		# PMULLDRegReg
paddd	zmm9, zmm0		# PADDDRegReg
add	edx, 0x00000040	# ADD4RegImms
cmp	edx, edi		# CMP4RegReg
jl	Label L0066			# JL1
paddd	zmm6, zmm7		# PADDDRegReg
paddd	zmm6, zmm8		# PADDDRegReg
paddd	zmm6, zmm9		# PADDDRegReg
vextractf64x4	zmm0, zmm6, 0xff 	# VEXTRACTF64X4YmmZmmImm1
paddd	ymm6, ymm0		# PADDDRegReg
vextractf128	ymm0, ymm6, 0xff 	# VEXTRACTF128RegRegImm1
paddd	xmm6, xmm0		# PADDDRegReg
pshufd	xmm0, xmm6, 0x0e 	# PSHUFDRegRegImm1
paddd	xmm6, xmm0		# PADDDRegReg
pshufd	xmm0, xmm6, 0x01 	# PSHUFDRegRegImm1
paddd	xmm6, xmm0		# PADDDRegReg
movd	eax, xmm6		# MOVDReg4Reg

JasonFengJ9 added comp:jit test failure labels Jan 22, 2025

JasonFengJ9 added this to the Release 0.51 (Java 8, 11, 17, 21, 24) Apr refresh milestone Jan 22, 2025

pshipton added the blocker label Jan 23, 2025

hzongaro assigned a7ehuo Jan 23, 2025

hzongaro mentioned this issue Jan 28, 2025

JDK24 AIX build StackOverflowError at jdk.compiler.interim/com.sun.tools.javac.tree.JCTree$JCBinary.accept(JCTree.java:2221) #20823

Open

hzongaro assigned BradleyWood Jan 30, 2025

luke-li-2003 mentioned this issue Jan 30, 2025

Internal JITServer test failures #14538

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error. compiler crashed (exit code 4) - cannot access unnamed package #20786 #21000

Error. compiler crashed (exit code 4) - cannot access unnamed package #20786 #21000

JasonFengJ9 commented Jan 22, 2025

hzongaro commented Jan 23, 2025

a7ehuo commented Jan 23, 2025

llxia commented Jan 23, 2025

hzongaro commented Jan 24, 2025

a7ehuo commented Jan 28, 2025 •

edited

Loading

a7ehuo commented Jan 29, 2025 •

edited

Loading

hzongaro commented Jan 30, 2025

BradleyWood commented Jan 31, 2025

a7ehuo commented Jan 31, 2025

a7ehuo commented Jan 31, 2025

BradleyWood commented Jan 31, 2025

Error. compiler crashed (exit code 4) - cannot access unnamed package #20786 #21000

Error. compiler crashed (exit code 4) - cannot access unnamed package #20786 #21000

Comments

JasonFengJ9 commented Jan 22, 2025

Failure link

Optional info

Failure output (captured from console output)

hzongaro commented Jan 23, 2025

a7ehuo commented Jan 23, 2025

llxia commented Jan 23, 2025

hzongaro commented Jan 24, 2025

a7ehuo commented Jan 28, 2025 • edited Loading

a7ehuo commented Jan 29, 2025 • edited Loading

hzongaro commented Jan 30, 2025

BradleyWood commented Jan 31, 2025

a7ehuo commented Jan 31, 2025

a7ehuo commented Jan 31, 2025

BradleyWood commented Jan 31, 2025

a7ehuo commented Jan 28, 2025 •

edited

Loading

a7ehuo commented Jan 29, 2025 •

edited

Loading