Dialyzer incremental analysis crashed!

[williamweicode]

Describe the bug
Dialyzer incremental analysis crashed!

1. first, run analysis with incremental mode, it is ok
   
2. then, change a file, run incremental analysis again, it crashed!
   

To Reproduce
crash dump content，omit the binary value.

Expected behavior
No Crash

Affected versions
Operating System: x86_64-pc-linux-gnu
ERTS: Erlang/OTP 26 [erts-14.2] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns]
Dialyzer version v5.1.2

Additional context
I tried to parse plt file. ifile_plt‘s "types" field parsed failed! Other ifile_plt fileds parsed ok.

[jhogberg]

Thanks for your report! Unfortunately, we cannot do much with this and your other report, #8681, because they lack steps to reproduce the issue. For issues relating to the compiler and dialyzer we need the source code you provide as input (in this case your entire project, basically). We are not going to make much progress with any of these issues until you provide that (or a minimized version thereof).

[williamweicode]

I reproduced the problem locally by adding more and more analysis modules. I added some output logs in dialyzer_iplt.erl.

I ran two analyses and got the logs respectively. The second analysis adds 20 new modules to the code from the first analysis.

Normal Analysis Output:
to file ets types info [{id,#Ref<0.1790361082.685637633.231597>},
{decentralized_counters,false},
{read_concurrency,false},
{write_concurrency,false},
{compressed,true},
{memory,290670482},
{owner,<0.9.0>},
{heir,none},
{name,plt_types},
{size,1561},
{node,nonode@nohost},
{named_table,false},
{type,set},
{keypos,1},
{protection,public}]

Record types 121947007

Abnormal Analysis output
Proceeding with incremental analysis...
compile (+0.00s): 4.65s ( 174 modules)
clean (+0.00s): 0.47s
remote (+4.29s):3280.48s
order (+0.16s): 0.09s
typesig (+0.00s): 82.31s ( 3858 SCCs)
order (+0.00s): 0.10s
refine (+0.00s): 10.63s ( 174 modules)
order (+0.00s): 0.03s
typesig (+0.00s): 30.61s ( 943 SCCs)
order (+0.00s): 0.01s
refine (+0.00s): 9.10s ( 130 modules)
warning (+0.13s): 9.95s ( 174 modules)
(+40.65s)
to file ets types info [{id,#Ref<0.1140431130.2039611393.209923>},
{decentralized_counters,false},
{read_concurrency,false},
{write_concurrency,false},
{compressed,true},
{memory,305916734},
{owner,<0.9.0>},
{heir,none},
{name,plt_types},
{size,1581},
{node,nonode@nohost},
{named_table,false},
{type,set},
{keypos,1},
{protection,public}]

Record types 6152010

the ETS seems normal, but after term_to_binary(dialyzer_utils:ets_tab2list(ETSTypes), [{compressed,9}]). The binary looks truncated.
Is this information useful for you? If not, I have to close this issule. It really took me a lot of time, thank you!
@jhogberg

[jhogberg]

Thank you, can you reproduce the issue with {compressed,9} removed?

[williamweicode]

{compressed,9} removed, the binary is not truncated, unfortunately, it crashed when term_to_binary(Record = #ifile_plt{}). It seems, the record is too large.I'll run it again on a machine with more memory(128G).

{compressed,9} removed Analysis output:
Proceeding with incremental analysis...
compile (+0.05s): 45.18s (1712 modules)
clean (+0.01s): 0.87s
remote (+0.75s):1557.55s
order (+0.29s): 7.56s
typesig (+0.00s): 725.98s ( 90189 SCCs)
order (+0.00s): 8.05s
refine (+0.00s): 31.12s (1712 modules)
order (+0.00s): 3.43s
typesig (+0.00s): 169.71s ( 36958 SCCs)
order (+0.00s): 1.46s
refine (+0.00s): 30.33s (1659 modules)
order (+0.00s): 0.09s
typesig (+0.00s): 18.41s ( 2451 SCCs)
order (+0.00s): 0.02s
refine (+0.00s): 7.35s ( 209 modules)
warning (+0.81s): 35.68s (1712 modules)
(+53.80s)
to file ets types info [{id,#Ref<0.1247172929.1811546113.92810>},
{decentralized_counters,false},
{read_concurrency,false},
{write_concurrency,false},
{compressed,true},
{memory,310638775},
{owner,<0.9.0>},
{heir,none},
{name,plt_types},
{size,1581},
{node,nonode@nohost},
{named_table,false},
{type,set},
{keypos,1},
{protection,public}]
Record types 4515823319

Runtime terminating during boot ({{case_clause,{system_limit,[{erlang,term_to_binary,[{ifile_plt,"v5.1.3",[{'OTP-PUB-KEY',<<251,161,106,43,154,44,61,162,75,78,224,224,134,92,239,96>>},{'PKCS-FRAME',<<22,48,139,3,155,234,96,145,210,208,2,93,254,128,223,182>>},{account_cli,<<156,33,29,148,107,242,102,88,209,21,212,0,54,220,169,22>>},{account_priviledge,<<95,103,178,56,101,139,169,171,219,107,232,88,222,19,149,246>>},{account_settings,<<26,175,240,200,29,121,33,244,76,239,121,222,18,245,119,233>>},{action,<<117,11,53,142,169,14,216,206,2,35,17,2,130,34,239,113>>},{activity,<<91,181,237,170,186,121,174,137,238,248,15,170,185,94,85,64>>},{activity_amend,<<1,5,9,113,154,103,61,117,30,108,175,195,208,91,41,91>>},{activity_anniversary_creation,<<175,76,100,126,241,224,170,193,247,147,254,153,117,45,251,168>>},{activity_anniversary_legend_benefit,<<42,4,199,70,107,30,219,204,96,85,97,171,148,26,140,121>>},{activity_anniversary_mail,<<212,79,186,173,126,243,197,250,89,231,177,190,196,24,1,235>>},{activity_anniversary_

Crash dump is being written to: erl_crash.dump...done

the erl_crash.dump size is 5.7G
@jhogberg

[williamweicode]

[root@k8s-wgn-test Server]# dialyzer --incremental --statistics --verbose

{compressed,9} removed Analysis output:
PLT does not yet exist at /root/.cache/erlang/.dialyzer_iplt, so an analysis must be run for 2052 modules to populate it
Proceeding with incremental analysis...
compile (+0.07s): 100.03s (2052 modules)
clean (+0.02s): 2.65s
remote (+3.31s):4735.31s
order (+0.37s): 13.84s
typesig (+0.00s): 797.04s ( 98547 SCCs)
order (+0.00s): 14.51s
refine (+0.00s): 93.58s (2052 modules)
order (+0.00s): 5.90s
typesig (+0.00s): 270.50s ( 42790 SCCs)
order (+0.00s): 4.12s
refine (+0.00s): 90.07s (1982 modules)
order (+0.00s): 0.27s
typesig (+0.00s): 118.46s ( 4680 SCCs)
order (+0.00s): 0.20s
refine (+0.00s): 33.46s ( 502 modules)
warning (+2.50s): 81.81s (2052 modules)
(+*****s)
to file ets types info [{id,#Ref<0.1822072438.2821586945.135145>},
{decentralized_counters,false},
{read_concurrency,false},
{write_concurrency,false},
{compressed,true},
{memory,783014501},
{owner,<0.9.0>},
{heir,none},
{name,plt_types},
{size,1915},
{node,nonode@nohost},
{named_table,false},
{type,set},
{keypos,1},
{protection,public}]
Record types 11388243954

Runtime terminating during boot ({{case_clause,{system_limit,[{erlang,term_to_binary,[{ifile_plt,"v5.1.3",[{'OTP-PUB-KEY',<<190,210,80,65,92,105,94,225,124,136,78,34,98,224,146,128>>},{'PKCS-FRAME',<<176,237,179,236,60,99,74,146,127,103,121,242,15,127,106,252>>},{account_cli,<<119,13,90,7,148,217,125,134,85,87,64,220,88,242,48,16>>},{account_priviledge,<<128,64,217,149,64,191,132,66,219,194,177,235,114,46,8,104>>},{account_settings,<<12,53,76,165,43,124,29,166,236,72,228,105,127,224,87,12>>},{action,<<117,11,53,142,169,14,216,206,2,35,17,2,130,34,239,113>>},{activity,<<218,225,207,9,135,95,141,80,203,246,245,100,68,179,39,59>>},{activity_amend,<<18,18,79,190,245,74,211,236,252,16,16,210,89,210,108,209>>},{activity_anniversary_creation,<<219,29,47,246,109,149,77,148,90,7,179,24,126,228,19,103>>},{activity_anniversary_legend_benefit,<<167,166,47,163,201,158,59,241,125,24,240,239,147,190,241,41>>},{activity_anniversary_mail,<<127,188,195,14,191,14,59,25,50,64,119,178,253,158,0,92>>},{activity_anniversary_mini_h

Crash dump is being written to: erl_crash.dump...done

@jhogberg After {compressed,9} removed , the analysis crashed again, I cant figure out what the system_limit is related to

erl_crash.dump first 100 lines:
=erl_crash_dump:0.5
Fri Aug 2 11:38:35 2024
Slogan: Runtime terminating during boot ({{case_clause,{system_limit,[{erlang,term_to_binary,[{ifile_plt,"v5.1.3",[{'OTP-PUB-KEY',<<190,210,80,65,92,105,94,225,124,136,78,34,98,224,146,128>>},{'PKCS-FRAME',<<176,237,179,236,60,99,74,146,127,103,121,242,15,127,106,252>>},{account_cli,<<119,13,90,7,148,217,125,134,85,87,64,220,88,242,48,16>>},{account_priviledge,<<128,64,217,149,64,191,132,66,219,194,177,235,114,46,8,104>>},{account_settings,<<12,53,76,165,43,124,29,166,236,72,228,105,127,224,87,12>>},{action,<<117,11,53,142,169,14,216,206,2,35,17,2,130,34,239,113>>},{activity,<<218,225,207,9,135,95,141,80,203,246,245,100,68,179,39,59>>},{activity_amend,<<18,18,79,190,245,74,211,236,252,16,16,210,89,210,108,209>>},{activity_anniversary_creation,<<219,29,47,246,109,149,77,148,90,7,179,24,126,228,19,103>>},{activity_anniversary_legend_benefit,<<167,166,47,163,201,158,59,241,125,24,240,239,147,190,241,41>>},{activity_anniversary_mail,<<127,188,195,14,191,14,59,25,50,64,119,178,253,158,0,92>>},{activity_anniversary_mini_h
System version: Erlang/OTP 26 [erts-14.2] [source] [64-bit] [smp:32:32] [ds:32:32:10] [async-threads:1] [jit:ns]
Taints:
Atoms: 165776
Calling Thread: scheduler:1
=scheduler:1
Scheduler Sleep Info Flags:
Scheduler Sleep Info Aux Work: DD | DD_THR_PRGR
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 1
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: NONEMPTY_NORMAL | OUT_OF_WORK | HALFTIME_OUT_OF_WORK | NONEMPTY | EXEC
Current Process: <0.0.0>
Current Process State: Running
Current Process Internal State: ACT_PRIO_NORMAL | USR_PRIO_NORMAL | PRQ_PRIO_NORMAL | ACTIVE | RUNNING
Current Process Program counter: 0x00007f6742d4700c (init:halt_string/2 + 92)
Current Process Limited Stack Trace:
0x00007f5dbbba0760:SReturn addr 0x42DB24A0 (erlang:halt/1 + 136)
0x00007f5dbbba0778:SReturn addr 0x42D3FF88 ()
=scheduler:2
Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING
Scheduler Sleep Info Aux Work:
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | INACTIVE
Current Process:
=scheduler:3
Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING
Scheduler Sleep Info Aux Work:
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | INACTIVE
Current Process:
=scheduler:4
Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING
Scheduler Sleep Info Aux Work: DD_THR_PRGR
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | INACTIVE
Current Process:
=scheduler:5
Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING
Scheduler Sleep Info Aux Work:
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | INACTIVE
Current Process:
=scheduler:6
Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING
Scheduler Sleep Info Aux Work:
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | INACTIVE
Current Process:
=scheduler:7
Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING
Scheduler Sleep Info Aux Work:
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | INACTIVE
Current Process:
=scheduler:8
Scheduler Sleep Info Flags: SLEEPING | TSE_SLEEPING | WAITING
Scheduler Sleep Info Aux Work:
Current Port:
Run Queue Max Length: 0
Run Queue High Length: 0
Run Queue Normal Length: 0
Run Queue Low Length: 0
Run Queue Port Length: 0
Run Queue Flags: OUT_OF_WORK | INACTIVE

erl_crash.dump memory info:
=memory
total: 11503469144
processes: 28370800
processes_used: 28368768
system: 11475098344
atom: 5358433
atom_used: 5347822
binary: 11412391376
code: 8776534
ets: 485280
=hash_table:atom_tab
size: 131072
used: 94232
objs: 165776
depth: 8
=index_table:atom_tab
size: 165888
limit: 1048576
entries: 165776
=hash_table:module_code
size: 128
used: 97
objs: 162
depth: 5
=index_table:module_code
size: 1024
limit: 65536
entries: 162
=hash_table:export_list
:

[williamweicode]

Hi, Will this issue move forward? @jhogberg

[jhogberg]

Not in the near term, I'm afraid. We'll add it to our backlog.

[williamweicode]

Not in the near term, I'm afraid. We'll add it to our backlog.

OK, Thanks!