installation of Rakudo Star fails on the JVM backend (trying to install modules with zef)

[fabrice-ducos]

$ uname -a
Darwin macbook-pro.home 18.5.0 Darwin Kernel Version 18.5.0: Mon Mar 11 20:40:32 PDT 2019; root:xnu-4903.251.3~3/RELEASE_X86_64 x86_64
$ java -XshowSettings:properties -version 2>&1 | grep java.runtime.version
java.runtime.version = 12.0.1+12
$ tar xvf rakudo-star-2019.03.tar
$ cd rakudo-star-2019.03
$ perl Configure.pl --help
$ mkdir $HOME/local
$ perl Configure.pl --backends=jvm --gen-nqp --prefix=$HOME/local/perl6
$ make && make test && make install

WARNING: Illegal reflective access by org.perl6.nqp.runtime.Ops (file:/Users/ducos/local/perl6/share/nqp/runtime/nqp-runtime.jar) to field sun.management.RuntimeImpl.jvm
WARNING: Please consider reporting this to the maintainers of org.perl6.nqp.runtime.Ops
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Unhandled exception: Wrong number of arguments passed; expected 1..1, but got 1
in dispatch:<::> (gen/jvm/CORE.setting:1765)
in clone (gen/jvm/CORE.setting:43297)
in (gen/jvm/CORE.setting:39248)
in EXCEPTION (gen/jvm/CORE.setting:39242)
in print_exception (gen/jvm/CORE.setting:39319)
in (gen/jvm/CORE.setting:39426)
in command_eval (gen/jvm/stage2/NQPHLL.nqp:1366)
in command_eval (src/Perl6/Compiler.nqp:48)
in command_line (gen/jvm/stage2/NQPHLL.nqp:1288)
in MAIN (gen/jvm/main.nqp:54)
in (gen/jvm/main.nqp:42)
in (gen/jvm/main.nqp)

make: *** [modules-install-j] Error 1

[usev6]

I can reproduce this error on HEAD with Java 8 on Linux and FreeBSD. (The warnings about illegal reflective access don't happen there.)

The error happens when Zef is used the first time in order to install modules. A simple use Zef::Client dies with the above error. Unfortunately, Zef hasn't worked on the JVM backend for a long time. That also means, that Rakudo Star didn't really work on the JVM backend for a long time.

The underlying problem for the above exception seems to be with NativeCall. The following code (which is used in Zef::Client: https://github.com/ugexe/zef/blob/e73897378b/lib/Zef/Client.pm6#L663) dies with the same error message:

$ ./perl6-j -Ilib -e 'use NativeCall'
===SORRY!=== Error while compiling -e
Cannot import the following symbols from NativeCall, because they already exist in this lexical scope:  void, ulonglong, ulong, ssize_t, size_t, longlong, long, bool, Pointer, OpaquePointer, CArray, &trait_mod:<is>, &refresh, &postcircumfix:<[ ]>, &nativesizeof, &nativecast, &explicitly-manage, &cglobal
at -e:1
------> use NativeCall⏏<EOL>
Unhandled exception: Wrong number of arguments passed; expected 1..1, but got 1
  in dispatch:<::> (gen/jvm/CORE.setting:1768)
  in clone (gen/jvm/CORE.setting:43457)
  in <anon> (gen/jvm/CORE.setting:39392)
  in EXCEPTION (gen/jvm/CORE.setting:39386)
  in <anon> (gen/jvm/CORE.setting)
  in <anon> (gen/jvm/CORE.setting)
  in dispatch:<::> (gen/jvm/CORE.setting:1768)
  in clone (gen/jvm/CORE.setting:43457)
  in <anon> (gen/jvm/CORE.setting:39392)
  in EXCEPTION (gen/jvm/CORE.setting:39386)
  in <anon> (gen/jvm/CORE.setting:4884)
  in <anon> (gen/jvm/CORE.setting)
  in <anon> (gen/jvm/CORE.setting:1837)
  in <anon> (gen/jvm/CORE.setting:4747)
  in pull-one (gen/jvm/CORE.setting:4734)
  in push-all (gen/jvm/CORE.setting:3015)
  in coremap (gen/jvm/CORE.setting:4788)
  in coremap (gen/jvm/CORE.setting:4706)
  in deepmap (gen/jvm/CORE.setting:55416)
  in deepmap (gen/jvm/CORE.setting:55415)
  in infix (gen/jvm/CORE.setting:54689)
  in infix (gen/jvm/CORE.setting:54468)
  in HYPER (gen/jvm/CORE.setting:55411)
  in dispatch:<hyper> (gen/jvm/CORE.setting:1838)
  in report (gen/jvm/CORE.setting:55706)
  in report (gen/jvm/CORE.setting:55700)
  in <anon> (gen/jvm/CORE.setting:55799)
  in <anon> (gen/jvm/CORE.setting:55798)
  in <anon> (gen/jvm/CORE.setting:4881)
  in <anon> (gen/jvm/CORE.setting:4880)
  in <anon> (gen/jvm/CORE.setting:4879)
  in <anon> (gen/jvm/CORE.setting:4876)
  in protect (gen/jvm/CORE.setting:56002)
  in protect (gen/jvm/CORE.setting:55998)
  in <anon> (gen/jvm/CORE.setting:4875)
  in <anon> (gen/jvm/CORE.setting:4874)
  in <anon> (gen/jvm/CORE.setting:39509)
  in print_exception (gen/jvm/CORE.setting:39490)
  in <anon> (gen/jvm/CORE.setting:39570)
  in command_eval (gen/jvm/stage2/NQPHLL.nqp:1366)
  in command_eval (src/Perl6/Compiler.nqp:48)
  in command_line (gen/jvm/stage2/NQPHLL.nqp:1288)
  in MAIN (gen/jvm/main.nqp:101)
  in <mainline> (gen/jvm/main.nqp:89)
  in <anon> (gen/jvm/main.nqp)

There were severe problems with NativeCall on the JVM backend at least since September 2017: https://rt.perl.org/Ticket/Display.html?id=132102

I'm going to change the title of the issue, though, since an installation of Rakudo doesn't explode (only an installation of Rakudo Star).

[usev6]

The error happens when Zef is used the first time in order to install modules. A simple use Zef::Client dies with the above error. Unfortunately, Zef hasn't worked on the JVM backend for a long time. That also means, that Rakudo Star didn't really work on the JVM backend for a long time.

I want to add that it's not Zef that is to blame. The JVM backend is in a bad shape.

[fabrice-ducos]

Dear Christian,

thank you for your answer. It was announced at https://rakudo.org/post/announce-rakudo-star-release-2019-03 that "this release of Rakudo Star is not fully functional with the JVM backend from the Rakudo compiler", so I am fine with it. I am however considering to switch to Perl 6 for some projects when it becomes ready for the JVM as well. Do you know if the JVM backend is still actively developed and if there is a timeline for a JVM-supported release? Is there a need for contributors?

Regards

[usev6]

Hi Fabrice,

I can't give you an authoritative answer regarding the future of the JVM backend, but can only describe my subjective view. Since a couple of years (I'd say since 2015 at latest) most of the backend-specific work goes into MoarVM. (Though @pmurias has done a lot for the JS backend.) I think it was (and is) a good decision to focus on MoarVM as the primary backend. Developer time is scarce, and there aren't that many devs that are familiar with the difficult parts of the compiler.

The JVM backend has been kind of dragged along. The one or the other old bug has been fixed, most of the new NQP operators introduced on the MoarVM backend have been ported to the JVM backend (but not always with full functionality) and broken builds on the JVM backend have been fixed (or worked around). Some people say that there is no future for the JVM backend -- at least not in its present form. In this discussion from 2018 @pmurias mentioned the potential of graal/truffle as a backend and that having a working JVM backend might help to get that going.

I think that finding a (or more than one) new champion for the JVM backend would be the only way to get a stable rakudo-j. So, yes there is a need for contributors. (There always is, isn't it? :))

All that said, a great part of the tests from https://github.com/perl6/roast are passing on the JVM backend. And from the looks of it, switching to Java 11 even makes it a bit faster :)

I hope that helps you to get an idea. Maybe someone else will add a different view.

Cheers