running opam-builder as per install-builder.sh generates empty HTML output

[gasche]

Salut Fabrice,

As we previously discussed, I'm trying to setup and run my own instance of opam-builder. I followed the install-builder.sh instruction, which worked remarkably well, but the step of ${OPAMBUILDER} json --watch .. is generating empty HTML files (valid, but with no switch/packages in them).

My impression is that there is something I don't understand about the various opam-builder-launched daemons and how they interact, and I'm opening this issue to get the clarification I need and suggest changes to the documentation.

Eyeballing the sources of the json --watch command suggest that it is looking for *.dump files to turn into web-facing data. However, the only daemon I was running at this step in the process is the ${OPAMBUILDER} switch daemon in a ${ROOTDIR}/4.05.0 switch, and it is not clear to me whether this daemon is ever supposed to generate .dump files. The sources mention that the build command is supposed to generate them after it tried to build the whole installable subset of opam-repository. Does the switch command run the build command implicitly?

My guess is that yes, and that I'm not seeing any output because the whole build has not completed yet. Is this correct? (I can see with tail nohup.out in ${ROOTDIR}/4.05.0 that it seems to be still building new packages.)

So my questions are:

1. the install-builder.sh script only mentions the commands switch, json --watch and opam. Am I correct that switch actually also runs the build command, and that this will only generate the .dump files I want after each round of compiling the whole opam-repository?

2. is there a way to know how far in the build round we are? (I guess that the first round is taking the most time, as incrementality helps for the next ones?)

2bis: Am I correct in assuming that killing the opam switch daemon and restarting it will correctly reuse previous build information (the builder.cache directory` is growing) and is therefore safe? I would like to play with parallelization settings, but I'm worried of losing work.

3. Would it be possible to incrementally generate .dump files, so that we start showing HTML output while the first build progresses?

4. Would it be possible to have a documentation of the input, output, preconditions/postconditions of each opam-builder command? Currently the only documentation I found is --help which is cryptic, and install-builder which only details three of them and makes me guess what they do ("switch", watch a switch and build .dump files, "json --watch", find switches build directory, watch them and build webpages, "opam", watch opam-repository and notifies watched switches (how?)).