Dependencies from GitLab

[robertdj]

When installing internal packages from GitLab the info seems to be stored correctly in renv.lock, but the package installation fails when running renv::restore().

Looking at the source this is also noticed at https://github.com/rstudio/renv/blob/master/R/retrieve.R#L171

There is an open PR in the remotes repo to solve this issue: r-lib/remotes#308

If you are interested, I'd be happy to make a PR that fixes renv until remotes is fixed.

Edit: The reason I am running into problems is that my GitLab repo is located at RemoteUsername / RemoteRepo / RemoteSubdir. All of these are saved in renv.lock, but RemoteSubdir is not used.

[kevinushey]

I'll try to take a look at this today -- thanks for reporting!

[kevinushey]

I've just pushed a few commits to master that should ensure this is handled -- let me know if you're still seeing issues. (Use renv::upgrade(version = "master") to upgrade to the latest version of renv in your project)

[robertdj]

Thanks! I'll try it out :-)

[robertdj]

It doesn't seem to work :-(

It looks like you have made a number of changes in the commit you refer to.
It works for me if I make a small change to the function renv_retrieve_gitlab on https://github.com/rstudio/renv/blob/master/R/retrieve.R#L169

In this block

user <- record$RemoteUsername
repo <- record$RemoteRepo
id <- URLencode(paste(user, repo, sep = "/"), reserved = TRUE)

I also include the subdir in id, e.g.

subdir <- record$RemoteSubdir
id <- URLencode(paste(user, repo, subdir, sep = "/"), reserved = TRUE)

[kevinushey]

Hmm, perhaps I misunderstood a bit. Based on remotes, subdir is documented as

#' subdir: subdirectory within repo that contains the R package.

In other words, it's normally used for sub-directories within the repository, as opposed to sub-repositories or sub-groups themselves. Since these are really semantically different things, I think they need to be expressed separately.

In other words, I think the full repository name should be expressed as e.g.

group/subgroup/repository

and the RemoteSubdir argument would otherwise be irrelevant here.

[kevinushey]

I've just made some changes that make it possible for me to download things from GitLab groups / sub-groups. I have a package in group / subgroup here:

https://gitlab.com/renv-group/renv-subgroup/subpackage

I can install that package and snapshot it with e.g. this script:

renv::init()
renv::install("gitlab::renv-group/renv-subgroup/subpackage")
renv::snapshot(type = "simple", library = .libPaths()[1], confirm = FALSE)
writeLines(readLines("renv.lock"))

I see:

> source("~/scratch/script.R")
* Discovering package dependencies ... Done!
* Copying packages into the cache ... Done!
* Lockfile written to '~/scratch/abc/renv.lock'.
* Project '~/scratch/abc' loaded. [renv 0.6.0-71]
* renv activated -- please restart the R session.
Retrieving 'https://gitlab.com/api/v4/projects/renv-group%2Frenv-subgroup%2Fsubpackage/repository/archive.tar.gz' ...
        OK [file is up to date]
Installing subpackage [1.0.0] from GitLab ...
        OK (built from source)
* The lockfile is already up to date.
* Lockfile written to '~/scratch/abc/renv.lock'.
{
  "renv": {
    "Version": "0.6.0-71"
  },
  "R": {
    "Version": "3.6.1",
    "Repositories": [
      {
        "Name": "CRAN",
        "URL": "https://cran.rstudio.com"
      }
    ]
  },
  "Packages": {
    "subpackage": {
      "Package": "subpackage",
      "Version": "1.0.0",
      "Source": "GitLab",
      "RemoteType": "gitlab",
      "RemoteUsername": "renv-group",
      "RemoteRepo": "renv-subgroup/subpackage",
      "RemoteRef": "master",
      "RemoteHost": "gitlab.com",
      "Hash": "2711102240891160d23cd1edde98a803"
    }
  }
}

[robertdj]

I had misunderstood the meaning of "subdir" :-) But I think you're right.

Your output look just right with RemoteRepo :-) Unfortunately, I cannot reproduce:

> renv::install("gitlab::renv-group/renv-subgroup/subpackage")
curl: (22) The requested URL returned error: 404 Not Found

To make things a bit more complicated, renv::install doesn't seem to work for me -- I need to install from a self-hosted GitLab. remotes works for this (with the PR I reference earlier) because I can supply a host argument. Unfortunately, remotes::install_gitlab does not write renv.lock in the format above.

[kevinushey]

Are you sure you're using the latest version of renv? The changes making this work were only committed recently.

If you need a custom host, you can set:

options(renv.config.gitlab.host = "custom.host")

Will need to think about whether it makes sense to expose a host argument in install(), or if there's a way to express a custom host within the remotes spec.

[robertdj]

Let me try installing it again just to be sure :-)

Otherwise I just had an idea that I posted on remotes's repo: r-lib/remotes#420

[robertdj]

I think I'm using the latest version. I install renv with remotes::install_github("rstudio/renv") and devtools::session_info() returns

VP renv 0.6.0-61 2019-08-09 [?] Github (c45d9c3)

I still get the same curl error

[robertdj]

BTW: Nice with the option :-)

[robertdj]

I don't know how I had managed to restart R/Rstudio/computer the wrong way, but after an R restart I suddenly don't get curl errors anymore.

Now renv::install("gitlab::<path>") starts out, but ends with a weird error:

Retrieving '<URL>' ...
	OK [file is up to date]
Installing foo [0.2.2] from GitLab ...
	OK (built from source)
Error in system(command, intern = TRUE) : reached elapsed time limit

[kevinushey]

You could try running traceback() to see where exactly the error occurred.

That error would imply that someone ran setTimeLimit(...) before your code was invoked. We do something like this during snapshot, but we reset the time limit after:

renv/R/snapshot.R

Lines 594 to 601 in 2a39e9c

	# set a timeout (avoid hanging the session during
	# dependency discovery)
	timeout <- renv_config("snapshot.timeout", default = 3L)
	deps <- local({
	setTimeLimit(elapsed = timeout, transient = TRUE)
	on.exit(setTimeLimit(), add = TRUE)
	dependencies(project)
	})

[kevinushey]

I've also added support for an explicit host, so that:

renv::install("gitlab@<host>::<repo>")

should do the right thing.

[robertdj]

I'll try to follow the stacktrace as you suggest as well as the new gitlab trick -- thanks!

[kevinushey]

I believe this should overall be handled now on the renv side -- please feel free to re-open or file a new issue if you run into problems. Thank you for testing!