Different errors when calling a single scalarized large matrix muliplication runtime-generated function

[bowenszhu]

• It has been reported in issues
  mul_float: types of a and b must match error in ModelingToolkit SciML/ModelingToolkit.jl#1515
  and
  Scary error when generating large number of RuntimeGeneratedFunctions SciML/RuntimeGeneratedFunctions.jl#13
  that some errors involving mul_float are thrown when one generates many functions at runtime.

The following is an example that generates just one single function but can still trigger the error.
The function is a simple dense matrix multiplication.
$$f(\underset{n\times 1}{u})=\underset{m\times n}A\quad\underset{n\times 1}u$$

using Symbolics
const m = 1000 # 600, 700
const n = 100
A = rand(m, n)
@variables u[1:n]
u = collect(u)
expr = A * u
f! = build_function(expr, u; expression=Val{false})[2]
du = Vector{Float64}(undef, m)
f!(du, rand(n))

• When m = 600, this piece of code works fine without trouble.

• When m = 700 or m = 800, the runtime-generated function call throws a very weird error:

Internal error: encountered unexpected error in runtime:
MethodError(f=Core.Compiler.widenconst, args=(0.277297,), world=0x000000000000133e)
jl_method_error_bare at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:1879
jl_method_error at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:1897
jl_lookup_generic_ at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2530 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2545
widen_all_consts! at ./compiler/typeinfer.jl:550
ir_to_codeinf! at ./compiler/optimize.jl:147 [inlined]
transform_result_for_cache at ./compiler/typeinfer.jl:364 [inlined]
cache_result! at ./compiler/typeinfer.jl:395
_typeinf at ./compiler/typeinfer.jl:277
typeinf at ./compiler/typeinfer.jl:213
typeinf_ext at ./compiler/typeinfer.jl:967
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1000
typeinf_ext_toplevel at ./compiler/typeinfer.jl:996
jfptr_typeinf_ext_toplevel_18807.clone_1 at /opt/hostedtoolcache/julia/1.8.1/x64/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
jl_apply at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/julia.h:1838 [inlined]
jl_type_infer at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:319
jl_generate_fptr_impl at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/jitlayers.cpp:319
jl_compile_method_internal at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2081 [inlined]
jl_compile_method_internal at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2025
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2359 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
jl_apply at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/julia.h:1838 [inlined]
do_call at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:126
eval_value at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:215
eval_stmt_value at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:166 [inlined]
eval_body at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:612
jl_interpret_toplevel_thunk at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/interpreter.c:750
top-level scope at none:11
jl_toplevel_eval_flex at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/toplevel.c:906
jl_toplevel_eval_flex at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/toplevel.c:850
ijl_toplevel_eval_in at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/toplevel.c:965
eval at ./boot.jl:368 [inlined]
exec_options at ./client.jl:276
_start at ./client.jl:522
jfptr__start_61720.clone_1 at /opt/hostedtoolcache/julia/1.8.1/x64/lib/julia/sys.so (unknown line)
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
jl_apply at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/julia.h:1838 [inlined]
true_main at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/jlapi.c:575
jl_repl_entrypoint at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/jlapi.c:719
main at julia (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x401098)
Unreachable reached at 0x7f0e1b9e5cb0

signal (4): Illegal instruction
in expression starting at none:11
getindex at ./tuple.jl:29 [inlined]
macro expansion at ./H�f���:128 [inlined]
macro expansion at ./none:0 [inlined]
generated_callfunc at ./none:0
unknown function (ip: 0x7f0e1b9e5d18)
_jl_invoke at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2367 [inlined]
ijl_apply_generic at /cache/build/default-amdci5-0/julialang/julia-release-1-dot-8/src/gf.c:2549
unknown function (ip: 0x1206024212060241)
Allocations: 131475969 (Pool: 131385625; Big: 90344); GC: 59
/home/runner/work/_temp/f762cb4a-9d0d-44f7-965b-2ff4ffc3e275.sh: line 13:  2075 Illegal instruction     (core dumped)

• When m = 1000, the same error is thrown as in mul_float: types of a and b must match error in ModelingToolkit SciML/ModelingToolkit.jl#1515.

ERROR: mul_float: types of a and b must match
Stacktrace:
  [1] *
    @ ./float.jl:385 [inlined]
  [2] macro expansion
    @ ~/.julia/packages/SymbolicUtils/qulQp/src/code.jl:394 [inlined]
  [3] macro expansion
    @ ~/.julia/packages/Symbolics/J8IHJ/src/build_function.jl:504 [inlined]
  [4] macro expansion
    @ ~/.julia/packages/SymbolicUtils/qulQp/src/code.jl:351 [inlined]
  [5] macro expansion
    @ ~/.julia/packages/RuntimeGeneratedFunctions/KrkGo/src/RuntimeGeneratedFunctions.jl:129 [inlined]
  [6] macro expansion
    @ ./none:0 [inlined]
  [7] generated_callfunc
    @ ./none:0 [inlined]
  [8] (::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1), Symbolics.var"#_RGF_ModTag", Symbolics.var"#_RGF_ModTag", (0x65cfcde1, 0x5b714b1f, 0x506f5aac, 0x6e8222ec, 0xa13dd40b)})(::Vector{Float64}, ::Vector{Float64})
    @ RuntimeGeneratedFunctions ~/.julia/packages/RuntimeGeneratedFunctions/KrkGo/src/RuntimeGeneratedFunctions.jl:117
  [9] macro expansion
    @ ~/.julia/packages/Symbolics/J8IHJ/src/build_function.jl:367 [inlined]
 [10] macro expansion
    @ ~/.julia/packages/SymbolicUtils/qulQp/src/code.jl:351 [inlined]
 [11] macro expansion
    @ ~/.julia/packages/RuntimeGeneratedFunctions/KrkGo/src/RuntimeGeneratedFunctions.jl:129 [inlined]
 [12] macro expansion
    @ ./none:0 [inlined]
 [13] generated_callfunc(::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1), Symbolics.var"#_RGF_ModTag", Symbolics.var"#_RGF_ModTag", (0xafbfc150, 0x3b7d376e, 0xd5c0bd21, 0x375b174e, 0xefe1569d)}, ::Vector{Float64}, ::Vector{Float64})
    @ Symbolics ./none:0
 [14] RuntimeGeneratedFunction
    @ ~/.julia/packages/RuntimeGeneratedFunctions/KrkGo/src/RuntimeGeneratedFunctions.jl:117 [inlined]
 [15] macro expansion
    @ ~/.julia/packages/Symbolics/J8IHJ/src/build_function.jl:367 [inlined]
 [16] macro expansion
    @ ~/.julia/packages/SymbolicUtils/qulQp/src/code.jl:351 [inlined]
 [17] macro expansion
    @ ~/.julia/packages/RuntimeGeneratedFunctions/KrkGo/src/RuntimeGeneratedFunctions.jl:129 [inlined]
 [18] macro expansion
    @ ./none:0 [inlined]
 [19] generated_callfunc
    @ ./none:0 [inlined]
 [20] (::RuntimeGeneratedFunctions.RuntimeGeneratedFunction{(:ˍ₋out, :ˍ₋arg1), Symbolics.var"#_RGF_ModTag", Symbolics.var"#_RGF_ModTag", (0xca3239b2, 0xbaf11099, 0x6c3b0ac9, 0x954b7770, 0xfc951b06)})(::Vector{Float64}, ::Vector{Float64})
    @ RuntimeGeneratedFunctions ~/.julia/packages/RuntimeGeneratedFunctions/KrkGo/src/RuntimeGeneratedFunctions.jl:117
 [21] top-level scope
    @ none:11

The computations are run by GitHub Actions. See here for full logs.