[Perf] Parallel sync and update

[mrcjkb]

I think I have an idea for this:

Option 1

• First, install (shared?) dependencies sequentially
• Then update rocks in parallel

Option 2

Implement a per-package locking mechanism in luarocks

[vhyrro]

I'm inclined to start with option 1 and refactor in case option 2 ever becomes a reality. We could pester hisham a bit to try and make that work :p (hell, maybe I can try to contribute, seems like a simplish thing to implement).

The processing time of checking shared dependencies is outweighed by the speed gains of parallel updates.

[mrcjkb]

Played around with this a bit more on the parallel-installs branch.
The luarocks --force-lock option is completely broken when doing parallel installs.

luarocks needs either a way to lock installs per directory or an --ignore-lock option.

[mrcjkb]

Added an --ignore-lock flag to my fork of luarocks. Some packages seem to update fine, but others fail with

Warning: Failed searching manifest: Failed loading manifest for https://nvim-neorocks.github.io/rocks-binaries-dev/: Error loading file: [string "/tmp/luarocks_local_cache-7694486/https___nvi..."]:2887: ']' expected near '}'

error: no results matching query were found for lua 5.1.
to check if it is available for other lua versions, use --check-lua-versions.

It looks like we may have to improve the luarocks locking mechanism.

[Freed-Wu]

Something maybe helpful: Before rocks sync support parallel, we can use parallel luarocks install ::: package1 package2 ... to do same work.

[mrcjkb]

That would add a dependency on GNU parallel, and would not solve the luarocks locking mechanism issue.

[Freed-Wu]

Oh, I means before rocks.nvim support it. We can install the packages in command line.

[mrcjkb]

The issue is in luarocks, not rocks.nvim, so I don't think parallel will help.