Skip to content

Commit

Permalink
Merge branch 'develop' into hotfix/issue-80-sdm-likelihood-evaluation…
Browse files Browse the repository at this point in the history
…-explodes-for-extremely-high-kappa
  • Loading branch information
venpopov authored Feb 10, 2024
2 parents 8f145ac + c48148e commit 8f49ae0
Show file tree
Hide file tree
Showing 11 changed files with 442 additions and 56 deletions.
62 changes: 60 additions & 2 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,61 @@
# bmm (development version)
# bmm 0.1.1+

* Initial CRAN submission.
### New features

* New model available - The Signal Discrimination Model by Oberauer (2023) for visual working memory continuous reproduction tasks. See ?sdmSimple. The current version does not take into account non-target activation
* Add ability to extract information about the default priors in `bmm` models with `get_model_prior()` (#53)
* Add ability to generate stan code and stan data for each model with `get_model_stancode()` and `get_model_standata()` (#81)
* BREAKING CHANGE: Add distribution functions for likelihood (e.g. `dimm()`) and random variate generation `rimm()`) for all models in the package. Remove deprecated `gen_3p_data()` and `gen_imm_data()` functions (#69)
* Two new datasets available: `ZhangLuck_2008` and `OberauerLin_2017` (#22)

### Documentation

* Website for the development version of the package is now available at https://venpopov.github.io/bmm/dev/ (#18)
* Add vignettes for each model to the website at https://venpopov.github.io/bmm/dev/articles/
* Add a detailed developer's guide to the website at https://venpopov.github.io/bmm/dev/dev-notes (#21)
* Improve README with more detailed information about the package's goals and its models (#21)

### Other changes

* Save `bmm` package version in the `brmsfit` object for reproducibility - e.g. `fit$version$bmm` (#88)


# bmm 0.1.1

### New features

* BREAKING CHANGE: Improve user interface to fit_model() ensures package stability and future development. Model specific arguments are now passed to the model functions as named arguments (e.g. `mixture3p(non_targets, setsize)`). This allows for a more flexible and intuitive way to specify model arguments. Passing model specific arguments directly to the `fit_model()` function is now deprecated (#43).
* Add information about each model such as domain, task, name, version, citation, requirements and parameters (#42)
* Add ability to generate a template file for adding new models to the package with `use_model_template()` (for developers) (#39)

### Other changes

* Improve documentation of model functions. You can now get help on each model by typing `?model_name` into your console. For example, calling the information on the full version of the Interference Measurement Model would look like this: `?IMMfull`


# bmm 0.1.0

A major restructuring of the package to support stable and generalizable development of future models (#41).

### New Features

* Refactor the `fit_model()` function to be generic and independent of the model being fit (#20)
* Transform models to be S3 objects. (#41).
* View currently supported models with new function `supported_models()`. Currently supported models are: `mixture2p()`, `mixture3p()`, `IMMabc()`, `IMMbsc()`, `IMMfull()`
* Add S3 methods for checking the data, formula, model and priors (#41)
* Add distribution functions for the Signal Discrimination Model. See `?SDM` for usage (#27)
* Add softmax and invsoftmax functions

### Bug Fixes

* Change default prior on log(kappa) to Normal(2,1) for the `mixture3p()` model (#15)

### Other changes

* BREAKING CHANGE: deprecate `model_type` argument in `fit_model()`. Models must now be specified with S3 functions passed to argument `model` rather than model names as strings passed to argument `model_type` (#41)
* Add extensive unit testing


# bmm 0.0.1

* Initial release version
4 changes: 2 additions & 2 deletions R/fit_model.R
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@
#'
#' @examples
#' \dontrun{
#' # generate artificial data from the Bays et al (2009) 3-parameter mixture model
#' dat <- rsdm(n=2000)
#' # generate artificial data from the Signal Discrimination Model
#' dat <- data.frame(y=rsdm(n=2000))
#'
#' # define formula
#' ff <- brms::bf(y ~ 1,
Expand Down
2 changes: 1 addition & 1 deletion R/helpers-model.R
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ print_pretty_models_md <- function() {
}
unique_domains <- unique(domains)
for (dom in unique_domains) {
cat('####', dom, '\n\n')
cat('**', dom, '**\n\n', sep="")
dom_models <- unique(models[domains == dom])
for (model in dom_models) {
cat('*', model, '\n')
Expand Down
2 changes: 2 additions & 0 deletions R/helpers-postprocess.R
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ postprocess_brm <- function(model, fit) {

#' @export
postprocess_brm.bmmmodel <- function(model, fit) {
# save bmm version
fit$version$bmm <- utils::packageVersion('bmm')
fit <- NextMethod('postprocess_brm')
return(fit)
}
Expand Down
124 changes: 90 additions & 34 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -11,53 +11,71 @@ knitr::opts_chunk$set(
)
```
# bmm <!-- badges: start -->
[![R-CMD-check](https://github.com/venpopov/bmm/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/venpopov/bmm/actions/workflows/R-CMD-check.yaml)
[![test-coverage](https://github.com/venpopov/bmm/actions/workflows/test-coverage.yaml/badge.svg)](https://github.com/venpopov/bmm/actions/workflows/test-coverage.yaml)
[![R-CMD-check](https://github.com/venpopov/bmm/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/venpopov/bmm/actions/workflows/R-CMD-check.yaml)
[![test-coverage](https://github.com/venpopov/bmm/actions/workflows/test-coverage.yaml/badge.svg)](https://github.com/venpopov/bmm/actions/workflows/test-coverage.yaml)
<!-- badges: end -->

The goal of bmm (Bayesian Measurement Models) is to make it easier to estimate
common cognitive measurement models for behavioral research using Bayesian hierarchical
estimation via the 'brms' package'. Cognitive measurement models provide a more refined representation of the cognitive processes underlying observed behavior, because they decompose observed behavior into several theoretically meaningful parameters that each represent distinct cognitive processes.
## Overview

Currently the bmm package implements mainly models used in the domain of visual working memory research:
The goal of the `bmm` (Bayesian Measurement Models) package is to make it easier
to estimate common cognitive measurement models for behavioral research. It
achieves this by combining the flexibility of the ‘brms’ package for specifying
linear model syntax with custom functions that translate cognitive measurement
model into distributional families that can be estimated using Bayesian
hierarchical estimation. Cognitive measurement models provide a more refined
representation of the cognitive processes underlying observed behavior, because
they decompose observed behavior into several theoretically meaningful
parameters that each represent distinct cognitive processes.


## Available models

Currently the bmm package implements mainly models used in the domain of visual
working memory research:

```{r, results="asis", echo=FALSE}
bmm::print_pretty_models_md()
```


However, the setup of the bmm package provides the foundation for the implementation of a broad range of cognitive measurement models. In fact, we are already working on implementing additional models, such as:
However, the setup of the bmm package provides the foundation for the
implementation of a broad range of cognitive measurement models. In fact, we are
already working on implementing additional models, such as:

- Signal-Detection Models
- Evidence Accumulation Models
- Memory Models for categorical response

If you have suggestions for models that should be added to the package, feel free to create an issue. Ideally this should describe the model, point towards literature that gives details on the model, and if possible link to code that has already implemented the model.
If you have suggestions for models that should be added to the package, feel
free to create an issue. Ideally this should describe the model, point towards
literature that gives details on the model, and if possible link to code that
has already implemented the model.

Given the dynamic nature the bmm package is currently in, you can always view the latest list of supported models by running:
Given the dynamic nature the bmm package is currently in, you can always view
the latest list of supported models by running:

```{r}
bmm::supported_models()
```

## Installation

Currently, we are working on getting the package ready to be submitted to CRAN. For now, you have to install the development version of bmm from [GitHub](https://github.com/) with:
Currently, we are working on getting the package ready to be submitted to CRAN.
For now, you have to install the development version of bmm from
[GitHub](https://github.com/) with:

``` r
# install.packages("devtools")
devtools::install_github("venpopov/bmm")
```

If you want to install the package together with the vignettes, you can use:

``` r
devtools::install_github("venpopov/bmm", build_vignettes = TRUE)
```

All the vignettes are also available on the [bmm website](https://venpopov.github.io/bmm/).
All the vignettes are also available on the [bmm
website](https://venpopov.github.io/bmm/).

The package was significantly updated on Feb 03, 2024. If you are following older versions (earlier than Version 6) of the [Tutorial preprint](https://osf.io/preprints/psyarxiv/umt57), you need to install the 0.0.1 version of the bmm package with:
The package was significantly updated on Feb 03, 2024. If you are following
older versions (earlier than Version 6) of the [Tutorial
preprint](https://osf.io/preprints/psyarxiv/umt57), you need to install the
0.0.1 version of the bmm package with:

``` r
# install.packages("devtools")
Expand All @@ -66,31 +84,48 @@ devtools::install_github("venpopov/[email protected]")

## The general structure of the bmm package

The main building block of the bmm package is that cognitive measurement models can often be specified as distributional models for which the distributional parameters of the generalized linear mixed model are a function of cognitive measurement model parameters. These functions that translate the cognitive measurement model parameters into distributional parameters is what we implement in the bmm package.
The main building block of the bmm package is that cognitive measurement models
can often be specified as distributional models for which the distributional
parameters of the generalized linear mixed model are a function of cognitive
measurement model parameters. These functions that translate the cognitive
measurement model parameters into distributional parameters is what we
implement in the bmm package.


```{r bmm-logic, echo=F, fig.cap="", out.width=600, fig.align = 'center'}
knitr::include_graphics("man/figures/README-bmmLogic.jpg")
```


As these function can become complicated and their implementation changes with differences in experimental designs, the bmm package provides general translation functions that eases the use of the cognitive measurement models for end users. This way researchers that face challenges in writing their own STAN code to implement such models themselves can still use these models in almost any experimental design.
As these function can become complicated and their implementation changes with
differences in experimental designs, the bmm package provides general
translation functions that eases the use of the cognitive measurement models for
end users. This way researchers that face challenges in writing their own STAN
code to implement such models themselves can still use these models in almost
any experimental design.

### Fitting models using the bmm

The core function of the bmm package is the `fit_model` function. This function takes:
The core function of the bmm package is the `fit_model` function. This function
takes:

1. a linear model formula specifying how parameters of the model should vary as a function of experimental conditions
2. data containing the dependent variables, the variables predicting model parameters, and potentially additional variables providing information to identify the model
1. a linear model formula specifying how parameters of the model should vary as
a function of experimental conditions
2. data containing the dependent variables, the variables predicting model
parameters, and potentially additional variables providing information to
identify the model
3. the model that should be fit

You can get more detailed information on the models implemented in bmm by invoking the documentation of each model typing `?bmmmodel` into your console. For example, calling the information on the full version of the Interference Measurement Model would look like this:
You can get more detailed information on the models implemented in bmm by
invoking the documentation of each model typing `?bmmmodel` into your console.
For example, calling the information on the full version of the Interference
Measurement Model would look like this:

``` r
?IMMfull
```

The function will then call the appropriate functions for the specified model and will perform several steps:
The function will then call the appropriate functions for the specified model
and will perform several steps:

1. Configure the Sample (e.g., set up prallelization)
2. Check the information passed to the `fit_model` function:
Expand All @@ -107,14 +142,18 @@ This process is illustrated in the Figure below:
knitr::include_graphics("man/figures/README-fitModel_process.jpg")
```

A complete call to fit a model using bmm could look like this. For this example, we are using the `OberauerLin_2017` data that is provided with the package.
A complete call to fit a model using bmm could look like this. For this example,
we are using the `OberauerLin_2017` data that is provided with the package.

``` r
library(bmm)
data <- OberauerLin_2017
```

For this quick example, we will show hot to setup fitting the Interference Measurement Model to this data. If you want a detailed description of this model and and in depth explanation of the parameters estimated in the model, please have a look at `vignette("IMM")`.
For this quick example, we will show hot to setup fitting the Interference
Measurement Model to this data. If you want a detailed description of this model
and and in depth explanation of the parameters estimated in the model, please
have a look at `vignette("IMM")`.

``` r
model_formula <- brms::bf(dev_rad ~ 1,
Expand All @@ -133,8 +172,9 @@ fit <- fit_model(
)
```

Using this call, the `fit` object will save all the information about the fitted model. As `bmm` calls `brms` to fit the models, these objects can be handled the same way a normal
`brmsfit` object is handled:
Using this call, the `fit` object will save all the information about the fitted
model. As `bmm` calls `brms` to fit the models, these objects can be handled the
same way a normal `brmsfit` object is handled:

``` r
# print summary
Expand All @@ -144,14 +184,26 @@ summary(fit)
brms::pp_check(fit)
```

You can have a look at examples for how to fit all currently implemented models by reading the vignettes for each model [here for the released version of the package](https://venpopov.github.io/bmm/articles/index.html) or [here for the development version](https://venpopov.github.io/bmm/dev/articles/index.html).
You can have a look at examples for how to fit all currently implemented models
by reading the vignettes for each model [here for the released version of the
package](https://venpopov.github.io/bmm/articles/index.html) or [here for the
development version](https://venpopov.github.io/bmm/dev/articles/index.html).


### Exploring cogntive measurement models

To aid users in improving their intuition about what different models predict for observed data given a certain parameter set, the `bmm` package also includes density and random generation function for all implemented models.
To aid users in improving their intuition about what different models predict
for observed data given a certain parameter set, the `bmm` package also includes
density and random generation function for all implemented models.

These function provide an easy way to see what a model predicts for data given a certain set of parameters. For example you can easily plot the probability density function of the data for the Interference Measurement model using the `dIMM` function. In similar fashion the random generation function included for each model, generates random data based on a set of data generating parameters. For the IMM, you can use `rIMM` to generate data given a set of parameters. Plotting the random data against the density illustrates that the data follows the theoretically implied density.
These function provide an easy way to see what a model predicts for data given a
certain set of parameters. For example you can easily plot the probability
density function of the data for the Interference Measurement model using the
`dIMM` function. In similar fashion the random generation function included for
each model, generates random data based on a set of data generating parameters.
For the IMM, you can use `rIMM` to generate data given a set of parameters.
Plotting the random data against the density illustrates that the data follows
the theoretically implied density.

```{r message=FALSE, warning=FALSE, out.width=400}
library(ggplot2)
Expand All @@ -177,5 +229,9 @@ ggplot(data = simData, aes(x = x)) +

## Contributing to the `bmm` package

Should be interested in contributing a model to the `bmm` package, you should first look into the [Developer Notes](https://venpopov.github.io/bmm/dev/dev-notes/index.html). These give a more in depth description of the package architecture and the steps necessary to add your own model to the package.
Should be interested in contributing a model to the `bmm` package, you should
first look into the [Developer
Notes](https://venpopov.github.io/bmm/dev/dev-notes/index.html). These give a
more in depth description of the package architecture and the steps necessary to
add your own model to the package.

30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,25 @@
[![test-coverage](https://github.com/venpopov/bmm/actions/workflows/test-coverage.yaml/badge.svg)](https://github.com/venpopov/bmm/actions/workflows/test-coverage.yaml)
<!-- badges: end -->

The goal of bmm (Bayesian Measurement Models) is to make it easier to
estimate common cognitive measurement models for behavioral research
using Bayesian hierarchical estimation via the ‘brms’ package’.
Cognitive measurement models provide a more refined representation of
the cognitive processes underlying observed behavior, because they
decompose observed behavior into several theoretically meaningful
parameters that each represent distinct cognitive processes.
## Overview

The goal of the `bmm` (Bayesian Measurement Models) package is to make
it easier to estimate common cognitive measurement models for behavioral
research. It achieves this by combining the flexibility of the ‘brms’
package for specifying linear model syntax with custom functions that
translate cognitive measurement model into distributional families that
can be estimated using Bayesian hierarchical estimation. Cognitive
measurement models provide a more refined representation of the
cognitive processes underlying observed behavior, because they decompose
observed behavior into several theoretically meaningful parameters that
each represent distinct cognitive processes.

## Available models

Currently the bmm package implements mainly models used in the domain of
visual working memory research:

#### Visual working memory
**Visual working memory**

- Interference measurement model by Oberauer and Lin (2017).
- Two-parameter mixture model by Zhang and Luck (2008).
Expand Down Expand Up @@ -66,13 +73,6 @@ from [GitHub](https://github.com/) with:
devtools::install_github("venpopov/bmm")
```

If you want to install the package together with the vignettes, you can
use:

``` r
devtools::install_github("venpopov/bmm", build_vignettes = TRUE)
```

All the vignettes are also available on the [bmm
website](https://venpopov.github.io/bmm/).

Expand Down
1 change: 1 addition & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ reference:
- "fit_model"
- title: "Extract model information"
desc: "Utility functions for extracting default priors, generate stan code, stan data, etc."
- contents:
- has_keyword("extract_stan")
- title: "Specifying models"
desc: "Functions for specifying which model to fit"
Expand Down
Loading

0 comments on commit 8f49ae0

Please sign in to comment.