Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement argument checking for exported functions #114

Merged
merged 14 commits into from
Dec 20, 2022
Merged

Conversation

ezraporter
Copy link
Collaborator

@ezraporter ezraporter commented Dec 15, 2022

Description

This PR implements error checking for arguments to exported functions with the following exceptions:

  • extract_tibbles(): No checking for tbls since this takes an unevaluated expression and eval_select() does checking on our behalf with cli-generated messages
  • fmt_* format helpers: No checking on arguments since these are all thin wrappers for stringr functions. stringr does a nice job of trying to coerce inputs to character and I don't think we want to override this behavior.

Proposed Changes

  • Add internal check functions check_arg_is_dataframe(), check_arg_is_env(), check_arg_is_character(), check_arg_is_logical(), check_arg_choices() that are wrappers for checkmate checks.
    • These wrappers are made with a function factory, wrap_checkmate(), so we can easily add new checking functions in the future
  • Add tests to test-checks.R to make sure check functions work
  • Add tests to test files for exported functions to check that they error with bad inputs

Screenshots
Example error message:

Screen Shot 2022-12-15 at 11 46 57 AM

The first line is added by REDCapTidieR and the second line is the message from checkmate.

Issue Addressed

Closes #109

PR Checklist

Before submitting this PR, please check and verify below that the submission meets the below criteria:

  • New/revised functions have associated tests
  • New/revised functions that update downstream outputs have associated static testing files (.RDS) updated under inst/testdata/create_test_data.R
  • New tests that make API calls use httptest::with_mock_api and any new mocks were added to tests/testthat/fixtures/create_httptest_mocks.R
  • New/revised functions use appropriate naming conventions
  • New/revised functions don't repeat code
  • Code changes are less than 250 lines total
  • Issues linked to the PR using GitHub's list of keywords
  • The appropriate reviewer is assigned to the PR
  • The appropriate developers are assigned to the PR

Code Review

This section to be used by the reviewer and developers during Code Review after PR submission

Code Review Checklist

  • I checked that new files follow naming conventions and are in the right place
  • I checked that documentation is complete, clear, and without typos
  • I added/edited comments to explain "why" not "how"
  • I checked that all new variable and function names follow naming conventions
  • I checked that new tests have been written for key business logic and/or bugs that this PR fixes
  • I checked that new tests address important edge cases

@ezraporter ezraporter added the enhancement New feature or request label Dec 15, 2022
@ezraporter ezraporter self-assigned this Dec 15, 2022
Copy link
Collaborator

@skadauke skadauke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an amazing start! My main comment is to tweak things to make it possible to create more informative error messages but I think we're close.

At some point we should run the "bad input" tests manually to refine the exact error messages.

R/bind_tibbles.R Outdated Show resolved Hide resolved
R/bind_tibbles.R Outdated Show resolved Hide resolved
R/checks.R Show resolved Hide resolved
R/checks.R Outdated Show resolved Hide resolved
R/checks.R Outdated Show resolved Hide resolved
R/extract_tibble.R Outdated Show resolved Hide resolved
R/extract_tibble.R Outdated Show resolved Hide resolved
R/read_redcap.R Show resolved Hide resolved
R/read_redcap.R Outdated Show resolved Hide resolved
R/read_redcap.R Outdated Show resolved Hide resolved
@skadauke
Copy link
Collaborator

Also, in case you haven't seen it, there is a really good section in the tidyverse style guide on error messages: https://style.tidyverse.org/error-messages.html

@ezraporter
Copy link
Collaborator Author

Made the following updates:

  • Added call to all cli_*() so the correct function name is shown in error messages
  • Added check_arg_is_supertbl() with option to specify required columns
  • Updated default check_arg_*() message to be more verbose
  • Added capability to tack additional error message info onto error messages with info parameter in check_arg_*() functions
    • See below for examples of the error messages for various bad inputs to our exported function. Looking for feedback on where we might need to add additional info.
  • Updated checkmate parameters based on settings in REDCapR to make argument checking a bit stricter
Error Message Examples
devtools::load_all()
#> ℹ Loading REDCapTidieR

options(rlang_backtrace_on_error_report = "none")

# read_redcap

classic_token <- "123456789ABCDEF"
redcap_uri <- "www.google.com"

## redcap_uri

read_redcap(123, classic_token)
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `redcap_uri`
#> ! Must be of type 'character', not 'double'

read_redcap(letters[1:3], classic_token)
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `redcap_uri`
#> ! Must have length 1, but has length 3

## token

read_redcap(redcap_uri, 123)
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `token`
#> ! Must be of type 'character', not 'double'

read_redcap(redcap_uri, letters[1:3])
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `token`
#> ! Must have length 1, but has length 3

## raw_or_label

read_redcap(redcap_uri, classic_token, raw_or_label = "bad option")
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `raw_or_label`
#> ! Must be element of set {'label','raw'}, but is 'bad option'

## forms

read_redcap(redcap_uri, classic_token, forms = 123)
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `forms`
#> ! Must be of type 'character' (or 'NULL'), not 'double'

## export_survey_fields

read_redcap(redcap_uri, classic_token, export_survey_fields = 123)
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `export_survey_fields`
#> ! Must be of type 'logical', not 'double'

read_redcap(redcap_uri, classic_token, export_survey_fields = c(TRUE, TRUE))
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `export_survey_fields`
#> ! Must have length 1, but has length 2

## suppress_redcapr_messages

read_redcap(redcap_uri, classic_token, suppress_redcapr_messages = 123)
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `suppress_redcapr_messages`
#> ! Must be of type 'logical', not 'double'

read_redcap(redcap_uri, classic_token, suppress_redcapr_messages = c(TRUE, TRUE))
#> Error in `read_redcap()`:
#> ✖ You've supplied an invalid value to `suppress_redcapr_messages`
#> ! Must have length 1, but has length 2

# bind_tibbles

bind_tibbles(123)
#> Error in `bind_tibbles()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! Must be of type 'data.frame', not 'double'

supertbl <- tibble(redcap_data = list())
bind_tibbles(supertbl, environment = "abc")
#> Error in `bind_tibbles()`:
#> ✖ You've supplied an invalid value to `environment`
#> ! Must be an environment, not 'character'

bind_tibbles(supertbl, tbls = 123)
#> Error in `bind_tibbles()`:
#> ✖ You've supplied an invalid value to `tbls`
#> ! Must be of type 'character' (or 'NULL'), not 'double'

# extract_tibbles

extract_tibbles(123)
#> Error in `extract_tibbles()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! Must be of type 'data.frame', not 'double'

# extract_tibble

extract_tibble(123, "my_tibble")
#> Error in `extract_tibble()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! Must be of type 'data.frame', not 'double'

supertbl <- tibble(redcap_data = list())
extract_tibble(supertbl, tbl = 123)
#> Error in `extract_tibble()`:
#> ✖ You've supplied an invalid value to `tbl`
#> ! Must be of type 'character', not 'double'

extract_tibble(supertbl, tbl = letters[1:3])
#> Error in `extract_tibble()`:
#> ✖ You've supplied an invalid value to `tbl`
#> ! Must have length 1, but has length 3

# make_labelled

make_labelled(123)
#> Error in `make_labelled()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! Must be of type 'data.frame', not 'double'

missing_col_supertbl <- tibble(redcap_data = list())
make_labelled(missing_col_supertbl)
#> Error in `make_labelled()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! Must contain columns `redcap_data` and `redcap_metadata`
#> ! You are missing `redcap_metadata`

missing_list_col_supertbl <- tibble(redcap_data = list(), redcap_metadata = 123)
make_labelled(missing_list_col_supertbl)
#> Error in `make_labelled()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! `redcap_metadata` must be of type 'list'

Created on 2022-12-16 with reprex v2.0.2

@rsh52
Copy link
Collaborator

rsh52 commented Dec 16, 2022

I think these returns largely look great. I especially like "element set" message for the raw_or_label return.

One thing to note, which may not be implementable here but for the future, is if extract_tibble winds up taking multiple arguments per Stephan's proposed framework in #111 we would need to allow for multiple tbls (actually it probably needs to be updated to be tbls if we go forward with it).

For make_labelled, if and when we make a custom S3 class, it will be nice to provide an error saying something along the lines of "supertbl must be of class 'supertbl'" or something. The current one specifies just that certain colnames are missing:

> make_labelled(data.frame(mtcars))
Error in `check_req_labelled_fields()` at REDCapTidieR/R/labelled.R:58:2:
! `supertbl` must contain `redcap_data` and `redcap_metadata`You are missing `redcap_data` and `redcap_metadata`
Run `rlang::last_error()` to see where the error occurred.

For fun I modified mtcars to have these colnames, and (while I this should never in a million years happen) here's what could happen:

mtcars_mod <- data.frame(mtcars) |> 
  mutate(
    redcap_data = cyl,
    redcap_metadata = hp
  )

> make_labelled(data.frame(mtcars_mod))
Error in `check_req_labelled_metadata_fields()` at REDCapTidieR/R/labelled.R:59:2:
! All elements of `supertbl$redcap_metadata` must contain
  `field_name` and `field_label``supertbl$redcap_metadata[[1]]` is missing `field_name` and
  `field_label``supertbl$redcap_metadata[[2]]` is missing `field_name` and
  `field_label`
....

@skadauke
Copy link
Collaborator

skadauke commented Dec 16, 2022

This looks great! A few comments:

  • The message "You've supplied an invalid value to xxx" should additionally tell the user what that invalid value was. For example, "You've supplied 123 for redcap_uri which is not a valid value." One caveat is that if the value is too complex to display concisely, we should shorten it. For example, only show the first 5 elements of an atomic vector, don't expand lists or tibbles?
  • I think we can make the supertbl checks a bit nicer. I like the idea of creating an S3 class for this, but I would also check for the presence of critical columns (see my comment above).

How about:

missing_col_supertbl <- tibble(redcap_data = list())
make_labelled(missing_col_supertbl)
#> Error in `make_labelled()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! Must be of class `<redcaptidier_supertbl>`
#> i `supertbl` must be a REDCapTidieR supertibble, generated using `read_redcap()`

missing_list_col_supertbl <- tibble(redcap_data = list(), redcap_metadata = 123)
make_labelled(missing_list_col_supertbl)
#> Error in `make_labelled()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! `supertbl$redcap_metadata` must be of type 'list'
#> i `supertbl` must be a REDCapTidieR supertibble, generated using `read_redcap()`
  • We should handle the situation when someone uses make_labelled() but doesn't have labelled installed. I think this should throw an error and suggest running install.packages("labelled").

@ezraporter
Copy link
Collaborator Author

  • The message "You've supplied an invalid value to xxx" should additionally tell the user what that invalid value was unless I guess it's too complex.

I've been messing around with this but haven't quite gotten it.

If we want the actual evaluated value passed by the user it's easy:

"You've supplied {x} for {.arg {arg}} which is not a valid value."

But this won't work for things like check_arg_is_supertbl() where we don't want to print out the actual tibble().

If we want the actual unevaluated text the user entered it doesn't seem like there's an easy way to do this without capturing the value in the function that calls the check_arg_*() function.

Works but ugly
library(rlang)

exported_function <- function(param) {
  check_arg_function(param, val = caller_arg(param))
}

check_arg_function <- function(x, val, arg = caller_arg(x)) {
  cat("The value of", arg, "was", val)
}

x <- 123

exported_function(x)
#> The value of param was x
exported_function(123)
#> The value of param was 123
  • I think we can make the supertbl checks a bit nicer. I like the idea of creating an S3 class for this, but I would also check for the presence of critical columns (see my comment above).

How about:

missing_col_supertbl <- tibble(redcap_data = list())
make_labelled(missing_col_supertbl)
#> Error in `make_labelled()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! Must be of class `<redcaptidier_supertbl>`
#> i `supertbl` must be a REDCapTidieR supertibble, generated using `read_redcap()`

missing_list_col_supertbl <- tibble(redcap_data = list(), redcap_metadata = 123)
make_labelled(missing_list_col_supertbl)
#> Error in `make_labelled()`:
#> ✖ You've supplied an invalid value to `supertbl`
#> ! `supertbl$redcap_metadata` must be of type 'list'
#> i `supertbl` must be a REDCapTidieR supertibble, generated using `read_redcap()`

If we're good to add the S3 class I think this will all work in a modified version of check_arg_is_supertbl()

  • We should handle the situation when someone uses make_labelled() but doesn't have labelled installed. I think this should throw an error and suggest running install.packages("labelled").

check_installed() currently handles this:

check_installed("labelled", reason = "to use `make_labelled()`")

Any reason to change it?

@skadauke
Copy link
Collaborator

If we want the actual evaluated value passed by the user it's easy:

"You've supplied {x} for {.arg {arg}} which is not a valid value."

But this won't work for things like check_arg_is_supertbl() where we don't want to print out the actual tibble().

If we want the actual unevaluated text the user entered it doesn't seem like there's an easy way to do this without capturing the value in the function that calls the check_arg_*() function.

Got it. I think we could handle this like so:

  • If {x} is an atomic vector <= 5 elements, show the whole thing
  • If {x} is an atomic vector >5 element, truncate it at 5 (example here: https://style.tidyverse.org/error-messages.html#error-details).
  • If {x} is any other data structure, don't show the value, and instead use the message "You've supplied an invalid value for {.arg {arg}}."

@ezraporter
Copy link
Collaborator Author

To summarize the additional pieces of this PR in the works today:

  • Add check_arg_is_valid_token() with sanitize_token() from REDCapR
  • Add S3 class to supertibble and update check_arg_is_supertbl() error messages as outlined here
  • Update wrap_checkmate() error messages to include argument values as outlined here
  • Update ability to customize wrap_checkmate() error messages (see here)
    • @skadauke I'm wondering if we can move this to a separate issue and the functionality if/when we need it. Currently I don't think we'll need this level of customization to produce any of the messages we want

@skadauke
Copy link
Collaborator

skadauke commented Dec 19, 2022 via email

@ezraporter
Copy link
Collaborator Author

  • Add check_arg_is_valid_token() with sanitize_token() from REDCapR

Done

  • Add S3 class to supertibble and update check_arg_is_supertbl() error messages as outlined here

Done with the addition of adding an example superheroes_supertbl to reference in examples

  • Update wrap_checkmate() error messages to include argument values as outlined here

Done. Using cli::cli_vec() to format x values for atomic vectors so they get truncated to 5 elements when printed. For non-atomic values I'm using rlang::as_label() to print a label for the value instead of the value itself. This seems to work pretty well. For example, environments become "<env>" and tibbles becomes <tibble[rows,cols]> with numbers of rows and columns shows. See the examples below.

  • Update ability to customize wrap_checkmate() error messages (see here)

Agreed to hold off on this

Error Message Examples
devtools::load_all()
#> ℹ Loading REDCapTidieR

options(rlang_backtrace_on_error_report = "none")

# read_redcap

classic_token <- "123456789ABCDEF123456789ABCDEF01"
redcap_uri <- "www.google.com"

## redcap_uri

read_redcap(123, classic_token)
#> Error in `read_redcap()`:
#> ✖ You've supplied `123` for `redcap_uri` which is not a valid value
#> ! Must be of type 'character', not 'double'

read_redcap(letters[1:3], classic_token)
#> Error in `read_redcap()`:
#> ✖ You've supplied `a`, `b`, `c` for `redcap_uri` which is not a valid
#>   value
#> ! Must have length 1, but has length 3

## token

read_redcap(redcap_uri, 123)
#> Error in `read_redcap()`:
#> ✖ You've supplied `123` for `token` which is not a valid value
#> ! Must be of type 'character', not 'double'

read_redcap(redcap_uri, letters[1:3])
#> Error in `read_redcap()`:
#> ✖ You've supplied `a`, `b`, `c` for `token` which is not a valid value
#> ! Must have length 1, but has length 3

## raw_or_label

read_redcap(redcap_uri, classic_token, raw_or_label = "bad option")
#> Error in `read_redcap()`:
#> ✖ You've supplied `bad option` for `raw_or_label` which is not a valid
#>   value
#> ! Must be element of set {'label','raw'}, but is 'bad option'

## forms

read_redcap(redcap_uri, classic_token, forms = 123)
#> Error in `read_redcap()`:
#> ✖ You've supplied `123` for `forms` which is not a valid value
#> ! Must be of type 'character' (or 'NULL'), not 'double'

## export_survey_fields

read_redcap(redcap_uri, classic_token, export_survey_fields = 123)
#> Error in `read_redcap()`:
#> ✖ You've supplied `123` for `export_survey_fields` which is not a valid
#>   value
#> ! Must be of type 'logical', not 'double'

read_redcap(redcap_uri, classic_token, export_survey_fields = c(TRUE, TRUE))
#> Error in `read_redcap()`:
#> ✖ You've supplied `TRUE`, `TRUE` for `export_survey_fields` which is not
#>   a valid value
#> ! Must have length 1, but has length 2

## suppress_redcapr_messages

read_redcap(redcap_uri, classic_token, suppress_redcapr_messages = 123)
#> Error in `read_redcap()`:
#> ✖ You've supplied `123` for `suppress_redcapr_messages` which is not a
#>   valid value
#> ! Must be of type 'logical', not 'double'

read_redcap(redcap_uri, classic_token, suppress_redcapr_messages = c(TRUE, TRUE))
#> Error in `read_redcap()`:
#> ✖ You've supplied `TRUE`, `TRUE` for `suppress_redcapr_messages` which
#>   is not a valid value
#> ! Must have length 1, but has length 2

# bind_tibbles

bind_tibbles(123)
#> Error in `bind_tibbles()`:
#> ✖ You've supplied `123` for `supertbl` which is not a valid value
#> ! Must be of class <redcaptidier_supertbl>
#> ℹ `supertbl` must be a REDCapTidieR supertibble, generated using
#>   `read_redcap()`

supertbl <- tibble(redcap_data = list())
bind_tibbles(supertbl, environment = "abc")
#> Error in `bind_tibbles()`:
#> ✖ You've supplied `<tibble[,1]>` for `supertbl` which is not a valid
#>   value
#> ! Must be of class <redcaptidier_supertbl>
#> ℹ `supertbl` must be a REDCapTidieR supertibble, generated using
#>   `read_redcap()`

bind_tibbles(supertbl, tbls = 123)
#> Error in `bind_tibbles()`:
#> ✖ You've supplied `<tibble[,1]>` for `supertbl` which is not a valid
#>   value
#> ! Must be of class <redcaptidier_supertbl>
#> ℹ `supertbl` must be a REDCapTidieR supertibble, generated using
#>   `read_redcap()`

# extract_tibbles

extract_tibbles(letters[1:10])
#> Error in `extract_tibbles()`:
#> ✖ You've supplied `a`, `b`, `c`, …, `i`, `j` for `supertbl` which is not
#>   a valid value
#> ! Must be of class <redcaptidier_supertbl>
#> ℹ `supertbl` must be a REDCapTidieR supertibble, generated using
#>   `read_redcap()`

# extract_tibble

extract_tibble(123, "my_tibble")
#> Error in `extract_tibble()`:
#> ✖ You've supplied `123` for `supertbl` which is not a valid value
#> ! Must be of class <redcaptidier_supertbl>
#> ℹ `supertbl` must be a REDCapTidieR supertibble, generated using
#>   `read_redcap()`

supertbl <- tibble(redcap_data = list()) %>%
  as_supertbl()
extract_tibble(supertbl, tbl = 123)
#> Error in `extract_tibble()`:
#> ✖ You've supplied `123` for `tbl` which is not a valid value
#> ! Must be of type 'character', not 'double'

extract_tibble(supertbl, tbl = letters[1:3])
#> Error in `extract_tibble()`:
#> ✖ You've supplied `a`, `b`, `c` for `tbl` which is not a valid value
#> ! Must have length 1, but has length 3

# make_labelled

make_labelled(123)
#> Error in `make_labelled()`:
#> ✖ You've supplied `123` for `supertbl` which is not a valid value
#> ! Must be of class <redcaptidier_supertbl>
#> ℹ `supertbl` must be a REDCapTidieR supertibble, generated using
#>   `read_redcap()`

missing_col_supertbl <- tibble(redcap_data = list()) %>%
  as_supertbl()
make_labelled(missing_col_supertbl)
#> Error in `make_labelled()`:
#> ✖ You've supplied `<redcaptidier_supertbl[,1]>` for `supertbl` which is
#>   not a valid value
#> ! Must contain `supertbl$redcap_metadata`
#> ℹ `supertbl` must be a REDCapTidieR supertibble, generated using
#>   `read_redcap()`

missing_list_col_supertbl <- tibble(redcap_data = list(), redcap_metadata = 123) %>%
  as_supertbl()
make_labelled(missing_list_col_supertbl)
#> Error in `make_labelled()`:
#> ✖ You've supplied `<redcaptidier_supertbl[,2]>` for `supertbl` which is
#>   not a valid value
#> ! `supertbl$redcap_metadata` must be of type 'list'
#> ℹ `supertbl` must be a REDCapTidieR supertibble, generated using
#>   `read_redcap()`

Created on 2022-12-19 with reprex v2.0.2

@ezraporter
Copy link
Collaborator Author

@skadauke @rsh52 This is ready to review once checks pass

Copy link
Collaborator

@rsh52 rsh52 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of comments, once we make a decision on them everything else LGTM

R/data.R Show resolved Hide resolved
data-raw/superheroes_supertbl.R Show resolved Hide resolved
@ezraporter ezraporter merged commit dd35fa2 into main Dec 20, 2022
@ezraporter ezraporter deleted the argument-checking branch December 20, 2022 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] Implement argument checking for all exported functions with cli-generated messages
3 participants