Skip to content

Commit

Permalink
ARROW-10358: [R] Followups to 2.0.0 release
Browse files Browse the repository at this point in the history
Closes apache#8495 from nealrichardson/r-post-2.0.0

Authored-by: Neal Richardson <[email protected]>
Signed-off-by: Neal Richardson <[email protected]>
  • Loading branch information
nealrichardson committed Oct 20, 2020
1 parent 84e4b15 commit 760284f
Show file tree
Hide file tree
Showing 7 changed files with 40 additions and 5 deletions.
1 change: 1 addition & 0 deletions r/.Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Dockerfile
.*\.tar\.gz
^windows
^libarrow
^revdep
clang_format.sh
^cran-comments\.md$
^arrow_.*.tar.gz$
Expand Down
1 change: 1 addition & 0 deletions r/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ src/Makevars
src/Makevars.win
windows/
libarrow/
revdep/
vignettes/nyc-taxi/
arrow_*.tar.gz
arrow_*.tgz
2 changes: 1 addition & 1 deletion r/DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ BugReports: https://issues.apache.org/jira/projects/ARROW/issues
Encoding: UTF-8
Language: en-US
LazyData: true
SystemRequirements: C++11
SystemRequirements: C++11; for AWS S3 support on Linux, libcurl and openssl (optional)
Biarch: true
Imports:
assertthat,
Expand Down
1 change: 1 addition & 0 deletions r/NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ to send and receive data. See `vignette("flight", package = "arrow")` for an ove
* File writers now respect the system umask setting
* `ParquetFileReader` has additional methods for accessing individual columns or row groups from the file
* Various segfaults fixed: invalid input in `ParquetFileWriter`; invalid `ArrowObject` pointer from a saved R object; converting deeply nested structs from Arrow to R
* The `properties` and `arrow_properties` arguments to `write_parquet()` are deprecated

# arrow 1.0.1

Expand Down
22 changes: 19 additions & 3 deletions r/R/parquet.R
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,12 @@ read_parquet <- function(file,
#' @param allow_truncated_timestamps Allow loss of data when coercing timestamps to a
#' particular resolution. E.g. if microsecond or nanosecond data is lost when coercing
#' to "ms", do not raise an exception
#' @param properties A `ParquetWriterProperties` object, used instead of the options
#' enumerated in this function's signature. Providing `properties` as an argument
#' is deprecated; if you need to assemble `ParquetWriterProperties` outside
#' of `write_parquet()`, use `ParquetFileWriter` instead.
#' @param arrow_properties A `ParquetArrowWriterProperties` object. Like
#' `properties`, this argument is deprecated.
#'
#' @details The parameters `compression`, `compression_level`, `use_dictionary` and
#' `write_statistics` support various patterns:
Expand Down Expand Up @@ -140,7 +146,9 @@ write_parquet <- function(x,
# arrow writer properties
use_deprecated_int96_timestamps = FALSE,
coerce_timestamps = NULL,
allow_truncated_timestamps = FALSE) {
allow_truncated_timestamps = FALSE,
properties = NULL,
arrow_properties = NULL) {
x_out <- x
if (is.data.frame(x)) {
x <- Table$create(x)
Expand All @@ -151,10 +159,18 @@ write_parquet <- function(x,
on.exit(sink$close())
}

# Deprecation warnings
if (!is.null(properties)) {
warning("Providing 'properties' is deprecated. If you need to assemble properties outside this function, use ParquetFileWriter instead.")
}
if (!is.null(arrow_properties)) {
warning("Providing 'arrow_properties' is deprecated. If you need to assemble arrow_properties outside this function, use ParquetFileWriter instead.")
}

writer <- ParquetFileWriter$create(
x$schema,
sink,
properties = ParquetWriterProperties$create(
properties = properties %||% ParquetWriterProperties$create(
x,
version = version,
compression = compression,
Expand All @@ -163,7 +179,7 @@ write_parquet <- function(x,
write_statistics = write_statistics,
data_page_size = data_page_size
),
arrow_properties = ParquetArrowWriterProperties$create(
arrow_properties = arrow_properties %||% ParquetArrowWriterProperties$create(
use_deprecated_int96_timestamps = use_deprecated_int96_timestamps,
coerce_timestamps = coerce_timestamps,
allow_truncated_timestamps = allow_truncated_timestamps
Expand Down
12 changes: 11 additions & 1 deletion r/man/write_parquet.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions r/tools/linuxlibs.R
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,12 @@ download_binary <- function(os = identify_os()) {
binary_url <- paste0(arrow_repo, "bin/", os, "/arrow-", VERSION, ".zip")
if (try_download(binary_url, libfile)) {
cat(sprintf("*** Successfully retrieved C++ binaries for %s\n", os))
if (!identical(os, "centos-7")) {
# centos-7 uses gcc 4.8 so the binary doesn't have ARROW_S3=ON but the others do
# TODO: actually check for system requirements?
cat("**** Binary package requires libcurl and openssl\n")
cat("**** If installation fails, retry after installing those system requirements\n")
}
} else {
cat(sprintf("*** No C++ binaries found for %s\n", os))
libfile <- NULL
Expand Down

0 comments on commit 760284f

Please sign in to comment.