Installing old package versions generates invalid URLs

[george-hall-ucl]

Hi, thanks for the very useful software!

I'm currently using renv to restore an R environment during the docker build process. I've done this in a couple of projects without any problems, but this time it is not working... The issue seems to be that if a version of a package is not from the most recent release and must be installed from source from the CRAN archive, then the package URL generated by renv is invalid.

Here is an example, with files collapsed for brevity. The renv.lock is available here.

Click to reveal Dockerfile

FROM rocker/rstudio:4.3.3

RUN R -e "install.packages('renv', repos = c(CRAN = 'https://cloud.r-project.org'))"

WORKDIR /project
COPY renv.lock renv.lock
ENV RENV_PATHS_LIBRARY=renv/library

RUN R -e "renv::restore()"

CMD [ "R" ]  

When I build the docker image with docker build -t renv_mwe . (which installs renv v1.0.9), the process crashes at renv::restore() with:

Click to reveal error message

 => [internal] load build definition from Dockerfile                                                  0.0s
 => => transferring dockerfile: 1.59kB                                                                0.0s
 => [internal] load metadata for docker.io/rocker/rstudio:4.3.3                                       0.8s
 => [internal] load .dockerignore                                                                     0.0s
 => => transferring context: 2B                                                                       0.0s
 => [1/5] FROM docker.io/rocker/rstudio:4.3.3@sha256:a69348c03b6863854045fbe9a049d46a7f0ab2d62fcf579  0.0s
 => [internal] load build context                                                                     0.0s
 => => transferring context: 80.73kB                                                                  0.0s
 => CACHED [2/5] RUN R -e "install.packages('renv', repos = c(CRAN = 'https://cloud.r-project.org'))  0.0s
 => CACHED [3/5] WORKDIR /project                                                                     0.0s
 => [4/5] COPY renv.lock renv.lock                                                                    0.0s
 => ERROR [5/5] RUN R -e "renv::restore()"                                                            7.4s
------                                                                                                     
 > [5/5] RUN R -e "renv::restore()":                                                                       
0.124                                                                                                      
0.124 R version 4.3.3 (2024-02-29) -- "Angel Food Cake"                                                    
0.124 Copyright (C) 2024 The R Foundation for Statistical Computing                                        
0.124 Platform: aarch64-unknown-linux-gnu (64-bit)                                                         
0.124 
0.124 R is free software and comes with ABSOLUTELY NO WARRANTY.
0.124 You are welcome to redistribute it under certain conditions.
0.124 Type 'license()' or 'licence()' for distribution details.
0.124 
0.127   Natural language support but running in an English locale
0.127 
0.127 R is a collaborative project with many contributors.
0.127 Type 'contributors()' for more information and
0.127 'citation()' on how to cite R or R packages in publications.
0.127 
0.127 Type 'demo()' for some demos, 'help()' for on-line help, or
0.127 'help.start()' for an HTML browser interface to help.
0.127 Type 'q()' to quit R.
0.127 
0.169 > renv::restore()
2.663 The following package(s) will be updated:
2.663 
2.663 # Bioconductor ---------------------------------------------------------------
2.663 - BiocGenerics       [* -> 0.48.1]
2.663 - BiocParallel       [* -> 1.36.0]
2.663 - BiocVersion        [* -> 3.18.1]
2.663 - ComplexHeatmap     [* -> 2.18.0]
2.663 - fgsea              [* -> 1.28.0]
2.663 - IRanges            [* -> 2.36.0]
2.663 
2.663 # Bioconductor 3.18 ----------------------------------------------------------
2.663 - S4Vectors          [* -> 0.40.2]
2.663 
2.663 # CRAN -----------------------------------------------------------------------
2.663 - BiocManager        [1.30.25 -> 1.30.23]
2.663 - codetools          [0.2-19 -> 0.2-20]
2.663 - KernSmooth         [2.23-22 -> 2.23-24]
2.663 - lattice            [0.22-5 -> 0.22-6]
2.663 - MASS               [7.3-60.0.1 -> 7.3-59]
2.663 - nlme               [3.1-164 -> 3.1-165]
2.663 - renv               [1.0.9 -> 1.0.7]
2.663 - survival           [3.5-8 -> 3.7-0]
2.663 - abind              [* -> 1.4-5]
2.663 - askpass            [* -> 1.2.0]
2.663 - babelgene          [* -> 22.9]
2.663 - backports          [* -> 1.5.0]
2.663 - base64enc          [* -> 0.1-3]
2.663 - BH                 [* -> 1.84.0-0]
2.663 - bit                [* -> 4.0.5]
2.663 - bit64              [* -> 4.0.5]
2.663 - bitops             [* -> 1.0-7]
2.663 - blob               [* -> 1.2.4]
2.663 - brew               [* -> 1.0-10]
2.663 - brio               [* -> 1.1.5]
2.663 - broom              [* -> 1.0.6]
2.663 - bslib              [* -> 0.7.0]
2.663 - cachem             [* -> 1.1.0]
2.663 - callr              [* -> 3.7.6]
2.663 - caTools            [* -> 1.18.2]
2.663 - cellranger         [* -> 1.1.0]
2.663 - checkmate          [* -> 2.3.1]
2.663 - circlize           [* -> 0.4.16]
2.663 - cli                [* -> 3.6.2]
2.663 - clipr              [* -> 0.8.0]
2.663 - clue               [* -> 0.3-65]
2.663 - clustree           [* -> 0.5.1]
2.663 - colorspace         [* -> 2.1-0]
2.663 - commonmark         [* -> 1.9.1]
2.663 - conflicted         [* -> 1.2.0]
2.663 - cowplot            [* -> 1.1.3]
2.663 - cpp11              [* -> 0.4.7]
2.663 - crayon             [* -> 1.5.2]
2.663 - credentials        [* -> 2.0.1]
2.663 - crosstalk          [* -> 1.2.1]
2.663 - curl               [* -> 5.1.0]
2.663 - data.table         [* -> 1.15.4]
2.663 - DBI                [* -> 1.2.3]
2.663 - dbplyr             [* -> 2.5.0]
2.663 - deldir             [* -> 2.0-4]
2.663 - desc               [* -> 1.4.3]
2.663 - devtools           [* -> 2.4.5]
2.663 - diffobj            [* -> 0.3.5]
2.663 - digest             [* -> 0.6.35]
2.663 - doParallel         [* -> 1.0.17]
2.663 - dotCall64          [* -> 1.1-1]
2.663 - downlit            [* -> 0.4.4]
2.663 - dplyr              [* -> 1.1.4]
2.663 - dqrng              [* -> 0.4.1]
2.663 - dtplyr             [* -> 1.3.1]
2.663 - ellipsis           [* -> 0.3.2]
2.663 - evaluate           [* -> 0.24.0]
2.663 - fansi              [* -> 1.0.6]
2.663 - farver             [* -> 2.1.2]
2.663 - fastDummies        [* -> 1.7.3]
2.663 - fastmap            [* -> 1.2.0]
2.663 - fastmatch          [* -> 1.1-4]
2.663 - fitdistrplus       [* -> 1.1-11]
2.663 - FNN                [* -> 1.1.4]
2.663 - fontawesome        [* -> 0.5.2]
2.663 - forcats            [* -> 1.0.0]
2.663 - foreach            [* -> 1.5.2]
2.663 - formatR            [* -> 1.14]
2.663 - fs                 [* -> 1.6.4]
2.663 - futile.logger      [* -> 1.4.3]
2.663 - futile.options     [* -> 1.0.1]
2.663 - future             [* -> 1.33.2]
2.663 - future.apply       [* -> 1.11.2]
2.663 - gargle             [* -> 1.5.2]
2.663 - generics           [* -> 0.1.3]
2.663 - gert               [* -> 2.0.1]
2.663 - GetoptLong         [* -> 1.0.5]
2.663 - ggforce            [* -> 0.4.2]
2.663 - ggplot2            [* -> 3.5.1]
2.663 - ggraph             [* -> 2.2.1]
2.663 - ggrepel            [* -> 0.9.5]
2.663 - ggridges           [* -> 0.5.6]
2.663 - gh                 [* -> 1.4.1]
2.663 - gitcreds           [* -> 0.1.2]
2.663 - GlobalOptions      [* -> 0.1.2]
2.663 - globals            [* -> 0.16.3]
2.663 - glue               [* -> 1.7.0]
2.663 - goftest            [* -> 1.2-3]
2.663 - googledrive        [* -> 2.1.1]
2.663 - googlesheets4      [* -> 1.1.1]
2.663 - gplots             [* -> 3.1.3.1]
2.663 - graphlayouts       [* -> 1.1.1]
2.663 - gridExtra          [* -> 2.3]
2.663 - gtable             [* -> 0.3.5]
2.663 - gtools             [* -> 3.9.5]
2.663 - haven              [* -> 2.5.4]
2.663 - here               [* -> 1.0.1]
2.663 - highr              [* -> 0.11]
2.663 - hms                [* -> 1.1.3]
2.663 - htmltools          [* -> 0.5.8.1]
2.663 - htmlwidgets        [* -> 1.6.4]
2.663 - httpuv             [* -> 1.6.15]
2.663 - httr               [* -> 1.4.7]
2.663 - httr2              [* -> 1.0.1]
2.663 - ica                [* -> 1.0-3]
2.663 - ids                [* -> 1.0.1]
2.663 - igraph             [* -> 2.0.3]
2.663 - ini                [* -> 0.3.1]
2.663 - irlba              [* -> 2.3.5.1]
2.663 - isoband            [* -> 0.2.7]
2.663 - iterators          [* -> 1.0.14]
2.663 - jquerylib          [* -> 0.1.4]
2.663 - jsonlite           [* -> 1.8.8]
2.663 - knitr              [* -> 1.47]
2.663 - labeling           [* -> 0.4.3]
2.663 - lambda.r           [* -> 1.2.4]
2.663 - later              [* -> 1.3.2]
2.663 - lazyeval           [* -> 0.2.2]
2.663 - leiden             [* -> 0.4.3.1]
2.663 - lifecycle          [* -> 1.0.4]
2.663 - listenv            [* -> 0.9.1]
2.663 - lmtest             [* -> 0.9-40]
2.663 - lubridate          [* -> 1.9.3]
2.663 - magrittr           [* -> 2.0.3]
2.663 - matrixStats        [* -> 1.3.0]
2.663 - memoise            [* -> 2.0.1]
2.663 - mime               [* -> 0.12]
2.663 - miniUI             [* -> 0.1.1.1]
2.663 - modelr             [* -> 0.1.11]
2.663 - msigdbr            [* -> 7.5.1]
2.663 - munsell            [* -> 0.5.1]
2.663 - openssl            [* -> 2.2.0]
2.663 - parallelly         [* -> 1.37.1]
2.663 - patchwork          [* -> 1.2.0]
2.663 - pbapply            [* -> 1.7-2]
2.663 - pheatmap           [* -> 1.0.12]
2.663 - pillar             [* -> 1.9.0]
2.663 - pkgbuild           [* -> 1.4.4]
2.663 - pkgconfig          [* -> 2.0.3]
2.663 - pkgdown            [* -> 2.0.9]
2.663 - pkgload            [* -> 1.3.4]
2.663 - plotly             [* -> 4.10.4]
2.663 - plyr               [* -> 1.8.9]
2.663 - png                [* -> 0.1-8]
2.663 - polyclip           [* -> 1.10-6]
2.663 - praise             [* -> 1.0.0]
2.663 - prettyunits        [* -> 1.2.0]
2.663 - processx           [* -> 3.8.4]
2.663 - profvis            [* -> 0.3.8]
2.663 - progress           [* -> 1.2.3]
2.663 - progressr          [* -> 0.14.0]
2.663 - promises           [* -> 1.3.0]
2.663 - ps                 [* -> 1.7.6]
2.663 - purrr              [* -> 1.0.2]
2.663 - R6                 [* -> 2.5.1]
2.663 - ragg               [* -> 1.3.2]
2.663 - RANN               [* -> 2.6.1]
2.663 - rappdirs           [* -> 0.3.3]
2.663 - rcmdcheck          [* -> 1.4.0]
2.663 - RColorBrewer       [* -> 1.1-3]
2.663 - Rcpp               [* -> 1.0.12]
2.663 - RcppAnnoy          [* -> 0.0.22]
2.663 - RcppArmadillo      [* -> 0.12.8.4.0]
2.663 - RcppEigen          [* -> 0.3.4.0.0]
2.663 - RcppHNSW           [* -> 0.6.0]
2.663 - RcppProgress       [* -> 0.4.2]
2.663 - RcppTOML           [* -> 0.2.2]
2.663 - readr              [* -> 2.1.5]
2.663 - readxl             [* -> 1.4.3]
2.663 - rematch            [* -> 2.0.0]
2.663 - rematch2           [* -> 2.1.2]
2.663 - remotes            [* -> 2.5.0]
2.663 - reprex             [* -> 2.1.0]
2.663 - reshape2           [* -> 1.4.4]
2.663 - reticulate         [* -> 1.37.0]
2.663 - rjson              [* -> 0.2.21]
2.663 - rlang              [* -> 1.1.4]
2.663 - rmarkdown          [* -> 2.27]
2.663 - ROCR               [* -> 1.0-11]
2.663 - roxygen2           [* -> 7.3.1]
2.663 - rprojroot          [* -> 2.0.4]
2.663 - RSpectra           [* -> 0.16-1]
2.663 - rstudioapi         [* -> 0.16.0]
2.663 - Rtsne              [* -> 0.17]
2.663 - rversions          [* -> 2.1.2]
2.663 - rvest              [* -> 1.0.4]
2.663 - sass               [* -> 0.4.9]
2.663 - scales             [* -> 1.3.0]
2.663 - scattermore        [* -> 1.2]
2.663 - sctransform        [* -> 0.4.1]
2.663 - selectr            [* -> 0.4-2]
2.663 - sessioninfo        [* -> 1.2.2]
2.663 - Seurat             [* -> 5.1.0]
2.663 - SeuratObject       [* -> 5.0.2]
2.663 - shape              [* -> 1.4.6.1]
2.663 - shiny              [* -> 1.8.1.1]
2.663 - sitmo              [* -> 2.0.2]
2.663 - snow               [* -> 0.4-4]
2.663 - sourcetools        [* -> 0.1.7-1]
2.663 - sp                 [* -> 2.1-4]
2.663 - spam               [* -> 2.10-0]
2.663 - spatstat.data      [* -> 3.0-4]
2.663 - spatstat.explore   [* -> 3.2-7]
2.663 - spatstat.geom      [* -> 3.2-9]
2.663 - spatstat.random    [* -> 3.2-3]
2.663 - spatstat.sparse    [* -> 3.0-3]
2.663 - spatstat.utils     [* -> 3.0-5]
2.663 - stringi            [* -> 1.8.4]
2.663 - stringr            [* -> 1.5.1]
2.663 - sys                [* -> 3.4.2]
2.663 - systemfonts        [* -> 1.1.0]
2.663 - tensor             [* -> 1.5]
2.663 - testthat           [* -> 3.2.1.1]
2.663 - textshaping        [* -> 0.4.0]
2.663 - tibble             [* -> 3.2.1]
2.663 - tidygraph          [* -> 1.3.1]
2.663 - tidyr              [* -> 1.3.1]
2.663 - tidyselect         [* -> 1.2.1]
2.663 - tidyverse          [* -> 2.0.0]
2.663 - timechange         [* -> 0.3.0]
2.663 - tinytex            [* -> 0.51]
2.663 - tweenr             [* -> 2.0.3]
2.663 - tzdb               [* -> 0.4.0]
2.663 - urlchecker         [* -> 1.0.1]
2.663 - usethis            [* -> 2.2.3]
2.663 - utf8               [* -> 1.2.4]
2.663 - uuid               [* -> 1.2-0]
2.663 - uwot               [* -> 0.2.2]
2.663 - vctrs              [* -> 0.6.5]
2.663 - viridis            [* -> 0.6.5]
2.663 - viridisLite        [* -> 0.4.2]
2.663 - vroom              [* -> 1.6.5]
2.663 - waldo              [* -> 0.5.2]
2.663 - whisker            [* -> 0.4.1]
2.663 - withr              [* -> 3.0.0]
2.663 - xfun               [* -> 0.45]
2.663 - xml2               [* -> 1.3.6]
2.663 - xopen              [* -> 1.0.1]
2.663 - xtable             [* -> 1.8-4]
2.663 - yaml               [* -> 2.3.8]
2.663 - zip                [* -> 2.3.1]
2.663 - zoo                [* -> 1.8-12]
2.663 
2.663 # R-Forge --------------------------------------------------------------------
2.663 - Matrix             [repo: CRAN -> R-Forge; ver: 1.6-5 -> 1.7-0]
2.663 
2.668 # Downloading packages -------------------------------------------------------
2.730 - Downloading codetools from CRAN ...           OK [37.8 Kb in 0.11s]
2.856 - Querying repositories for available source packages ... Done!
6.064 - Downloading Matrix from R-Forge ...           OK [2.4 Mb in 0.44s]
6.643 - Downloading lattice from CRAN ...             OK [584.6 Kb in 0.18s]
6.863 - Downloading KernSmooth from CRAN ...          OK [25.5 Kb in 0.11s]
6.992 - Downloading renv from CRAN ...                	ERROR [cannot open URL 'https://cloud.r-project.org/src/contrib/renv_1.0.7.tar.gz']
7.305 Warning: failed to find source for 'renv 1.0.7' in package repositories
7.305 Warning: error downloading 'https://cloud.r-project.org/src/contrib/renv_1.0.7.tar.gz' [cannot open URL 'https://cloud.r-project.org/src/contrib/renv_1.0.7.tar.gz']
7.305 Error: failed to retrieve package '[email protected]'
7.307 Traceback (most recent calls last):
7.307 8: renv::restore()
7.307 7: renv_restore_run_actions(project, diff, current, lockfile, rebuild)
7.307 6: renv_retrieve_impl(packages)
7.307 5: handler(package, renv_retrieve_impl_one(package))
7.307 4: renv_retrieve_impl_one(package)
7.307 3: renv_retrieve_repos(record)
7.307 2: stopf("failed to retrieve package '%s'", remote)
7.307 1: stop(sprintf(fmt, ...), call. = call.)
7.307 Execution halted
------
Dockerfile:9
--------------------
   7 |     ENV RENV_PATHS_LIBRARY=renv/library
   8 |     
   9 | >>> RUN R -e "renv::restore()"
  10 |     
  11 |     CMD [ "R" ]  
--------------------
ERROR: failed to solve: process "/bin/sh -c R -e \"renv::restore()\"" did not complete successfully: exit code: 1

So, it is trying to download an old version of renv from https://cloud.r-project.org/src/contrib/renv_1.0.7.tar.gz, which doesn't exist. Note that it is (I think) just an unfortunate coincidence that it is the installation of renv that is causing the problem in this particular example -- I've seen it crash with other packages too (e.g. nlme), but in this case it happens to crash at renv as you have recently released an update, making it install v1.0.7 from the CRAN archive.

I think this is an issue with this specific renv.lock file, as I am able to generate other ones which are able to install old versions of software completely fine. Like I say, I've used this workflow in several projects without any problems.

I'm at a bit of a dead end about how to fix this, so any insights would be greatly appreciated please!

Many thanks,
George

[kevinushey]

Thanks for the bug report -- I was able to reproduce this as well. Oddly, the issue doesn't seem to occur if renv is the only package present in the lockfile. I'll investigate further.

[kevinushey]

I'm a bit confused from the provided example... the lockfile depends on Matrix 1.7-0, which requires R 4.4.0 or newer -- however, the image uses R 4.3.3. Is this expected? In addition, the version of Matrix provided by R-Forge is now 1.8-0, and since that repository doesn't maintain an archive, I wouldn't expect it to be available there.

https://r-forge.r-project.org/R/?group_id=61

This seems unrelated to the error message provided, but even so, it seems like attempts to restore this lockfile would fail either way.

[george-hall-ucl]

Hi @kevinushey,

Thank you for the quick response!

I'm afraid that upgrading renv to v1.0.9.9000 didn't fix the bug:

Click to reveal output

=> [internal] load build definition from Dockerfile                                                                               0.0s
 => => transferring dockerfile: 1.69kB                                                                                             0.0s
 => [internal] load metadata for docker.io/rocker/rstudio:4.3.3                                                                    1.4s
 => [internal] load .dockerignore                                                                                                  0.0s
 => => transferring context: 2B                                                                                                    0.0s
 => CACHED [1/6] FROM docker.io/rocker/rstudio:4.3.3@sha256:a69348c03b6863854045fbe9a049d46a7f0ab2d62fcf579d3cb8b2a2eac8f3c5       0.0s
 => [internal] load build context                                                                                                  0.0s
 => => transferring context: 1.35kB                                                                                                0.0s
 => [2/6] RUN R -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))"                                  2.2s
 => [3/6] RUN R -e "remotes::install_github('rstudio/renv@bugfix/repository-archive-format-hardening')"                            8.0s
 => [4/6] WORKDIR /project                                                                                                         0.0s
 => [5/6] COPY renv.lock renv.lock                                                                                                 0.0s
 => ERROR [6/6] RUN R -e "renv::restore()"                                                                                         6.0s
------                                                                                                                                  
 > [6/6] RUN R -e "renv::restore()":                                                                                                    
0.121                                                                                                                                   
0.121 R version 4.3.3 (2024-02-29) -- "Angel Food Cake"                                                                                 
0.121 Copyright (C) 2024 The R Foundation for Statistical Computing                                                                     
0.121 Platform: aarch64-unknown-linux-gnu (64-bit)                                                                                      
0.121 
0.121 R is free software and comes with ABSOLUTELY NO WARRANTY.
0.121 You are welcome to redistribute it under certain conditions.
0.121 Type 'license()' or 'licence()' for distribution details.
0.121 
0.121   Natural language support but running in an English locale
0.121 
0.121 R is a collaborative project with many contributors.
0.121 Type 'contributors()' for more information and
0.121 'citation()' on how to cite R or R packages in publications.
0.121 
0.121 Type 'demo()' for some demos, 'help()' for on-line help, or
0.121 'help.start()' for an HTML browser interface to help.
0.121 Type 'q()' to quit R.
0.121 
0.160 > renv::restore()
2.588 The following package(s) will be updated:
2.588 
2.588 # Bioconductor ---------------------------------------------------------------
2.588 - BiocGenerics       [* -> 0.48.1]
2.588 - BiocParallel       [* -> 1.36.0]
2.588 - BiocVersion        [* -> 3.18.1]
2.588 - ComplexHeatmap     [* -> 2.18.0]
2.588 - fgsea              [* -> 1.28.0]
2.588 - IRanges            [* -> 2.36.0]
2.588 
2.588 # Bioconductor 3.18 ----------------------------------------------------------
2.588 - S4Vectors          [* -> 0.40.2]
2.588 
2.588 # CRAN -----------------------------------------------------------------------
2.588 - BiocManager        [1.30.25 -> 1.30.23]
2.588 - codetools          [0.2-19 -> 0.2-20]
2.588 - KernSmooth         [2.23-22 -> 2.23-24]
2.588 - lattice            [0.22-5 -> 0.22-6]
2.588 - MASS               [7.3-60.0.1 -> 7.3-59]
2.588 - nlme               [3.1-164 -> 3.1-165]
2.588 - renv               [rstudio/renv@bugfix/repository-archive-format-hardening -> 1.0.7]
2.588 - survival           [3.5-8 -> 3.7-0]
2.588 - abind              [* -> 1.4-5]
2.588 - askpass            [* -> 1.2.0]
2.588 - babelgene          [* -> 22.9]
2.588 - backports          [* -> 1.5.0]
2.588 - base64enc          [* -> 0.1-3]
2.588 - BH                 [* -> 1.84.0-0]
2.588 - bit                [* -> 4.0.5]
2.588 - bit64              [* -> 4.0.5]
2.588 - bitops             [* -> 1.0-7]
2.588 - blob               [* -> 1.2.4]
2.588 - brew               [* -> 1.0-10]
2.588 - brio               [* -> 1.1.5]
2.588 - broom              [* -> 1.0.6]
2.588 - bslib              [* -> 0.7.0]
2.588 - cachem             [* -> 1.1.0]
2.588 - callr              [* -> 3.7.6]
2.588 - caTools            [* -> 1.18.2]
2.588 - cellranger         [* -> 1.1.0]
2.588 - checkmate          [* -> 2.3.1]
2.588 - circlize           [* -> 0.4.16]
2.588 - cli                [* -> 3.6.2]
2.588 - clipr              [* -> 0.8.0]
2.588 - clue               [* -> 0.3-65]
2.588 - clustree           [* -> 0.5.1]
2.588 - colorspace         [* -> 2.1-0]
2.588 - commonmark         [* -> 1.9.1]
2.588 - conflicted         [* -> 1.2.0]
2.588 - cowplot            [* -> 1.1.3]
2.588 - cpp11              [* -> 0.4.7]
2.588 - crayon             [* -> 1.5.2]
2.588 - credentials        [* -> 2.0.1]
2.588 - crosstalk          [* -> 1.2.1]
2.588 - curl               [* -> 5.1.0]
2.588 - data.table         [* -> 1.15.4]
2.588 - DBI                [* -> 1.2.3]
2.588 - dbplyr             [* -> 2.5.0]
2.588 - deldir             [* -> 2.0-4]
2.588 - desc               [* -> 1.4.3]
2.588 - devtools           [* -> 2.4.5]
2.588 - diffobj            [* -> 0.3.5]
2.588 - digest             [* -> 0.6.35]
2.588 - doParallel         [* -> 1.0.17]
2.588 - dotCall64          [* -> 1.1-1]
2.588 - downlit            [* -> 0.4.4]
2.588 - dplyr              [* -> 1.1.4]
2.588 - dqrng              [* -> 0.4.1]
2.588 - dtplyr             [* -> 1.3.1]
2.588 - ellipsis           [* -> 0.3.2]
2.588 - evaluate           [* -> 0.24.0]
2.588 - fansi              [* -> 1.0.6]
2.588 - farver             [* -> 2.1.2]
2.588 - fastDummies        [* -> 1.7.3]
2.588 - fastmap            [* -> 1.2.0]
2.588 - fastmatch          [* -> 1.1-4]
2.588 - fitdistrplus       [* -> 1.1-11]
2.588 - FNN                [* -> 1.1.4]
2.588 - fontawesome        [* -> 0.5.2]
2.588 - forcats            [* -> 1.0.0]
2.588 - foreach            [* -> 1.5.2]
2.588 - formatR            [* -> 1.14]
2.588 - fs                 [* -> 1.6.4]
2.588 - futile.logger      [* -> 1.4.3]
2.588 - futile.options     [* -> 1.0.1]
2.588 - future             [* -> 1.33.2]
2.588 - future.apply       [* -> 1.11.2]
2.588 - gargle             [* -> 1.5.2]
2.588 - generics           [* -> 0.1.3]
2.588 - gert               [* -> 2.0.1]
2.588 - GetoptLong         [* -> 1.0.5]
2.588 - ggforce            [* -> 0.4.2]
2.588 - ggplot2            [* -> 3.5.1]
2.588 - ggraph             [* -> 2.2.1]
2.588 - ggrepel            [* -> 0.9.5]
2.588 - ggridges           [* -> 0.5.6]
2.588 - gh                 [* -> 1.4.1]
2.588 - gitcreds           [* -> 0.1.2]
2.588 - GlobalOptions      [* -> 0.1.2]
2.588 - globals            [* -> 0.16.3]
2.588 - glue               [* -> 1.7.0]
2.588 - goftest            [* -> 1.2-3]
2.588 - googledrive        [* -> 2.1.1]
2.588 - googlesheets4      [* -> 1.1.1]
2.588 - gplots             [* -> 3.1.3.1]
2.588 - graphlayouts       [* -> 1.1.1]
2.588 - gridExtra          [* -> 2.3]
2.588 - gtable             [* -> 0.3.5]
2.588 - gtools             [* -> 3.9.5]
2.588 - haven              [* -> 2.5.4]
2.588 - here               [* -> 1.0.1]
2.588 - highr              [* -> 0.11]
2.588 - hms                [* -> 1.1.3]
2.588 - htmltools          [* -> 0.5.8.1]
2.588 - htmlwidgets        [* -> 1.6.4]
2.588 - httpuv             [* -> 1.6.15]
2.588 - httr               [* -> 1.4.7]
2.588 - httr2              [* -> 1.0.1]
2.588 - ica                [* -> 1.0-3]
2.588 - ids                [* -> 1.0.1]
2.588 - igraph             [* -> 2.0.3]
2.588 - ini                [* -> 0.3.1]
2.588 - irlba              [* -> 2.3.5.1]
2.588 - isoband            [* -> 0.2.7]
2.588 - iterators          [* -> 1.0.14]
2.588 - jquerylib          [* -> 0.1.4]
2.588 - jsonlite           [* -> 1.8.8]
2.588 - knitr              [* -> 1.47]
2.588 - labeling           [* -> 0.4.3]
2.588 - lambda.r           [* -> 1.2.4]
2.588 - later              [* -> 1.3.2]
2.588 - lazyeval           [* -> 0.2.2]
2.588 - leiden             [* -> 0.4.3.1]
2.588 - lifecycle          [* -> 1.0.4]
2.588 - listenv            [* -> 0.9.1]
2.588 - lmtest             [* -> 0.9-40]
2.588 - lubridate          [* -> 1.9.3]
2.588 - magrittr           [* -> 2.0.3]
2.588 - matrixStats        [* -> 1.3.0]
2.588 - memoise            [* -> 2.0.1]
2.588 - mime               [* -> 0.12]
2.588 - miniUI             [* -> 0.1.1.1]
2.588 - modelr             [* -> 0.1.11]
2.588 - msigdbr            [* -> 7.5.1]
2.588 - munsell            [* -> 0.5.1]
2.588 - openssl            [* -> 2.2.0]
2.588 - parallelly         [* -> 1.37.1]
2.588 - patchwork          [* -> 1.2.0]
2.588 - pbapply            [* -> 1.7-2]
2.588 - pheatmap           [* -> 1.0.12]
2.588 - pillar             [* -> 1.9.0]
2.588 - pkgbuild           [* -> 1.4.4]
2.588 - pkgconfig          [* -> 2.0.3]
2.588 - pkgdown            [* -> 2.0.9]
2.588 - pkgload            [* -> 1.3.4]
2.588 - plotly             [* -> 4.10.4]
2.588 - plyr               [* -> 1.8.9]
2.588 - png                [* -> 0.1-8]
2.588 - polyclip           [* -> 1.10-6]
2.588 - praise             [* -> 1.0.0]
2.588 - prettyunits        [* -> 1.2.0]
2.588 - processx           [* -> 3.8.4]
2.588 - profvis            [* -> 0.3.8]
2.588 - progress           [* -> 1.2.3]
2.588 - progressr          [* -> 0.14.0]
2.588 - promises           [* -> 1.3.0]
2.588 - ps                 [* -> 1.7.6]
2.588 - purrr              [* -> 1.0.2]
2.588 - R6                 [* -> 2.5.1]
2.588 - ragg               [* -> 1.3.2]
2.588 - RANN               [* -> 2.6.1]
2.588 - rappdirs           [* -> 0.3.3]
2.588 - rcmdcheck          [* -> 1.4.0]
2.588 - RColorBrewer       [* -> 1.1-3]
2.588 - Rcpp               [* -> 1.0.12]
2.588 - RcppAnnoy          [* -> 0.0.22]
2.588 - RcppArmadillo      [* -> 0.12.8.4.0]
2.588 - RcppEigen          [* -> 0.3.4.0.0]
2.588 - RcppHNSW           [* -> 0.6.0]
2.588 - RcppProgress       [* -> 0.4.2]
2.588 - RcppTOML           [* -> 0.2.2]
2.588 - readr              [* -> 2.1.5]
2.588 - readxl             [* -> 1.4.3]
2.588 - rematch            [* -> 2.0.0]
2.588 - rematch2           [* -> 2.1.2]
2.588 - reprex             [* -> 2.1.0]
2.588 - reshape2           [* -> 1.4.4]
2.588 - reticulate         [* -> 1.37.0]
2.588 - rjson              [* -> 0.2.21]
2.588 - rlang              [* -> 1.1.4]
2.588 - rmarkdown          [* -> 2.27]
2.588 - ROCR               [* -> 1.0-11]
2.588 - roxygen2           [* -> 7.3.1]
2.588 - rprojroot          [* -> 2.0.4]
2.588 - RSpectra           [* -> 0.16-1]
2.588 - rstudioapi         [* -> 0.16.0]
2.588 - Rtsne              [* -> 0.17]
2.588 - rversions          [* -> 2.1.2]
2.588 - rvest              [* -> 1.0.4]
2.588 - sass               [* -> 0.4.9]
2.588 - scales             [* -> 1.3.0]
2.588 - scattermore        [* -> 1.2]
2.588 - sctransform        [* -> 0.4.1]
2.588 - selectr            [* -> 0.4-2]
2.588 - sessioninfo        [* -> 1.2.2]
2.588 - Seurat             [* -> 5.1.0]
2.588 - SeuratObject       [* -> 5.0.2]
2.588 - shape              [* -> 1.4.6.1]
2.588 - shiny              [* -> 1.8.1.1]
2.588 - sitmo              [* -> 2.0.2]
2.588 - snow               [* -> 0.4-4]
2.588 - sourcetools        [* -> 0.1.7-1]
2.588 - sp                 [* -> 2.1-4]
2.588 - spam               [* -> 2.10-0]
2.588 - spatstat.data      [* -> 3.0-4]
2.588 - spatstat.explore   [* -> 3.2-7]
2.588 - spatstat.geom      [* -> 3.2-9]
2.588 - spatstat.random    [* -> 3.2-3]
2.588 - spatstat.sparse    [* -> 3.0-3]
2.588 - spatstat.utils     [* -> 3.0-5]
2.588 - stringi            [* -> 1.8.4]
2.588 - stringr            [* -> 1.5.1]
2.588 - sys                [* -> 3.4.2]
2.588 - systemfonts        [* -> 1.1.0]
2.588 - tensor             [* -> 1.5]
2.588 - testthat           [* -> 3.2.1.1]
2.588 - textshaping        [* -> 0.4.0]
2.588 - tibble             [* -> 3.2.1]
2.588 - tidygraph          [* -> 1.3.1]
2.588 - tidyr              [* -> 1.3.1]
2.588 - tidyselect         [* -> 1.2.1]
2.588 - tidyverse          [* -> 2.0.0]
2.588 - timechange         [* -> 0.3.0]
2.588 - tinytex            [* -> 0.51]
2.588 - tweenr             [* -> 2.0.3]
2.588 - tzdb               [* -> 0.4.0]
2.588 - urlchecker         [* -> 1.0.1]
2.588 - usethis            [* -> 2.2.3]
2.588 - utf8               [* -> 1.2.4]
2.588 - uuid               [* -> 1.2-0]
2.588 - uwot               [* -> 0.2.2]
2.588 - vctrs              [* -> 0.6.5]
2.588 - viridis            [* -> 0.6.5]
2.588 - viridisLite        [* -> 0.4.2]
2.588 - vroom              [* -> 1.6.5]
2.588 - waldo              [* -> 0.5.2]
2.588 - whisker            [* -> 0.4.1]
2.588 - withr              [* -> 3.0.0]
2.588 - xfun               [* -> 0.45]
2.588 - xml2               [* -> 1.3.6]
2.588 - xopen              [* -> 1.0.1]
2.588 - xtable             [* -> 1.8-4]
2.588 - yaml               [* -> 2.3.8]
2.588 - zip                [* -> 2.3.1]
2.588 - zoo                [* -> 1.8-12]
2.588 
2.588 # R-Forge --------------------------------------------------------------------
2.588 - Matrix             [repo: CRAN -> R-Forge; ver: 1.6-5 -> 1.7-0]
2.588 
2.593 # Downloading packages -------------------------------------------------------
2.647 - Downloading codetools from CRAN ...           OK [37.8 Kb in 0.32s]
2.989 - Querying repositories for available source packages ... Done!
5.013 - Downloading Matrix from R-Forge ...           	ERROR [cannot open URL 'https://cloud.r-project.org/src/contrib/Archive/Matrix/Matrix_1.7-0.tar.gz']
5.372 - Downloading Matrix from R-Forge ...           	ERROR [cannot open URL 'https://R-Forge.R-project.org/src/contrib/Archive/Matrix/Matrix_1.7-0.tar.gz']
5.886 Warning: failed to find source for 'Matrix 1.7-0' in package repositories
5.886 Warning: error downloading 'https://cloud.r-project.org/src/contrib/Archive/Matrix/Matrix_1.7-0.tar.gz' [cannot open URL 'https://cloud.r-project.org/src/contrib/Archive/Matrix/Matrix_1.7-0.tar.gz']
5.886 Warning: error downloading 'https://R-Forge.R-project.org/src/contrib/Archive/Matrix/Matrix_1.7-0.tar.gz' [cannot open URL 'https://R-Forge.R-project.org/src/contrib/Archive/Matrix/Matrix_1.7-0.tar.gz']
5.886 Error: failed to retrieve package '[email protected]'
5.888 Traceback (most recent calls last):
5.888 8: renv::restore()
5.888 7: renv_restore_run_actions(project, diff, current, lockfile, rebuild)
5.888 6: renv_retrieve_impl(packages)
5.888 5: handler(package, renv_retrieve_impl_one(package))
5.888 4: renv_retrieve_impl_one(package)
5.888 3: renv_retrieve_repos(record)
5.888 2: stopf("failed to retrieve package '%s'", remote)
5.888 1: stop(sprintf(fmt, ...), call. = call.)
5.888 Execution halted
------
Dockerfile:10
--------------------
   8 |     ENV RENV_PATHS_LIBRARY=renv/library
   9 |     
  10 | >>> RUN R -e "renv::restore()"
  11 |     
  12 |     CMD [ "R" ]  
--------------------
ERROR: failed to solve: process "/bin/sh -c R -e \"renv::restore()\"" did not complete successfully: exit code: 1

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/jf3blmawdyafur2aage8c1nq0

I also experimented with removing renv itself from renv.lock (in case it was causing some weird conflict by upgrading itself to an older version with a bug), but this crashed with the same error.

Changing the Matrix repository from R-forge to CRAN doesn't seem to help.

However, your comment about R v4.3.3 being incompatible with the Matrix package made me change to R v4.4.0 in both renv.lock and the Dockerfile and now renv::restore() completes!

There still seem to be some download issues (example below), but the downloads eventually work again at least!

Click to reveal

27.74 - Downloading colorspace from CRAN ...          	ERROR [cannot open URL 'https://bioconductor.org/packages/3.18/bioc/src/contrib/Archive/colorspace/colorspace_2.1-0.tar.gz']
28.61 - Downloading colorspace from CRAN ...          	ERROR [cannot open URL 'https://bioconductor.org/packages/3.18/data/annotation/src/contrib/Archive/colorspace/colorspace_2.1-0.tar.gz']
29.48 - Downloading colorspace from CRAN ...          	ERROR [cannot open URL 'https://bioconductor.org/packages/3.18/data/experiment/src/contrib/Archive/colorspace/colorspace_2.1-0.tar.gz']
30.34 - Downloading colorspace from CRAN ...          	ERROR [cannot open URL 'https://bioconductor.org/packages/3.18/workflows/src/contrib/Archive/colorspace/colorspace_2.1-0.tar.gz']
31.23 - Downloading colorspace from CRAN ...          	ERROR [cannot open URL 'https://bioconductor.org/packages/3.18/books/src/contrib/Archive/colorspace/colorspace_2.1-0.tar.gz']
31.50 - Downloading colorspace from CRAN ...          OK [2 Mb in 0.45s]

I'm not really sure why changing the R version fixes the download URL, but all I can think is that the incompatibility in the R versions that you spotted is causing renv to download a version of Matrix that doesn't exist. In any case, it works now!

Many thanks for your help with this,
George

[kevinushey]

Great, glad to hear it!

The issue ultimately comes down to R reporting that Matrix is not available for older versions of R. For example:

> install.packages("Matrix", type = "source")
Installing package into '/Users/kevin/Library/R/arm64/4.3/library'
(as 'lib' is unspecified)
Warning: package 'Matrix' is not available for this version of R
'Matrix' version 1.7-0 is in the repositories but depends on R (>= 4.4.0)
'Matrix' version 1.7-0 is in the repositories but depends on R (>= 4.5)

A version of this package for your version of R might be available elsewhere,
see the ideas at
https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages

It would certainly be helpful if renv also included diagnostic information like this, instead of just quietly failing!

[AlbertRapp]

Hi @kevinushey,

I currently run into a very similar issue. All archived packages from my our company repo are not available because the url is invalid. Here's on example.

renv tries to download <company-url>/cran-remote/src/contrib/Archive/MASS/7.3-60/MASS_7.3-60.tar.gz but it should be <company-url>/cran-remote/src/contrib/Archive/MASS/MASS_7.3-60.tar.gz.

Basically, it's always package/version/package_version.tar.gz instead of package/package_version.tar.gz at the end. For some reason only archived packages are affected like that. I've tried 2 different R versions but the issue still persists. Do you have any workarounds for this?

[kevinushey]

@AlbertRapp -- are you still seeing this with renv 1.0.10? If so, can you share more information about the R package repository you're using -- are you using a public CRAN instance, or PPM, or something else?

[AlbertRapp]

We're using renv 1.0.10 and this issue still persists. And we use a private Artifactory CRAN repository. Does this help or do you need more specific information? 🤔

[kevinushey]

It does! Do you have the curl command line tool available? What is the output of:

url <- contrib.url(getOption("repos"), "source")
system(paste("curl -I", file.path(url, "PACKAGES")))

The most recent release of renv assumed that the "x-artifactory-id" header would be part of the returned headers; maybe that's not the case here?

If you need a more immediate workaround, you should be able to set:

options(renv.repos.formatters = list(
  <url> = "artifactory"
))

where <url> is the repository URL you see in getOption("repos").

[kevinushey]

A more direct test might be:

url <- file.path(getOption("repos")[[1]], "src/contrib/PACKAGES")
str(renv:::renv_download_headers(url))

[AlbertRapp]

Hi Kevin, thanks for the quick reply!

I ran your suggested test

url <- file.path(getOption("repos")[[1]], "src/contrib/PACKAGES")
str(renv:::renv_download_headers(url))

and found the x-artifactory-id entry in there. Consequently, the workaround of setting

options(renv.repos.formatters = list(
  <url> = "artifactory"
))

still generated invalid URLs. Any idea of how the erroneous path package/version/package_version.tar.gz instead of package/package_version.tar.gz is generated? Maybe some other type of repo uses this erroneous path structure and something with our repo makes {renv} think it's that other type of repo? 🤔

[kevinushey]

Looking back a bit more ... I wonder if this changed recently? There's an old renv issue here: #602.

@brianrepko, is this something that changed recently on the Artifactory side?

[kevinushey]

In any event, if Artifactory is now using a regular CRAN-style archive layout, then something like:

options(renv.repos.formatters = list(
  <url> = "cran"
))

should then be an appropriate workaround for now, I believe.

[AlbertRapp]

Hi Kevin, fantastic workaround 👍 Thank you. (Although I almost didn't think it worked but after putting the options in front of the activate call in .Rprofile it worked)

Also, I investigated a bit more and switched to a previous version. With renv 1.0.7 I can confirm that I don't run into any troubles with archived packages.

[AlbertRapp]

Btw where could I have found out about renv.repos.formatters? I was looking for ages to find something that allows me to change the URL "template" but couldn't find it anywhere in the docs 🙈

[kevinushey]

It's not currently documented; I had added it just as an escape hatch in case of an issue like this.

[brianrepko]

yes - Artifactory support for CRAN repositories allows for multiple layouts - I'm just on an older one that doesn't have that. See https://jfrog.com/help/r/jfrog-artifactory-documentation/apply-the-cran-official-specification-to-local-cran-repositories

[kevinushey]

Do you know if the CRAN layout for archived packages is now the default? It sounds like these older installations of Artifactory are no longer supported, at least from https://jfrog.com/help/r/jfrog-release-information/artifactory-end-of-life?tocId=aaNnCiuFIelrQusjfAFfpw, so I wonder if I should start assuming CRAN layout by default in renv.

[brianrepko]

I would guess that is the case but I don’t know for sure