Feature/108 implement sdt models #113
Conversation
|
I just wanted to share my progress so far. The old/new SDT models should be implemented. For now, I added all noise distributions that can be easily accommodated (Normal, EVD, Cauchy, and Logistic). However, we might want to discuss if there that matches the theoretical discussions around SDT models. The main things that still need to be done with respect to the old/new SDT models are:
|
|
ok, I'll try to review what you have so far before we meet next week |
There is no hurry. I just thought you might be interested. As the old/new models are mostly done. If you other things to do, I can also show you what I have done next week when we meet. |
|
yes, I just want to consider more broadly how the various different SDT models such as old/new,2-AFC, m-AFC, confidence ratings might fit together and what we would need to have as separate models |
|
I found this article that should provide a good reference how to generalize the old/new SDT models to m-AFC SDT models: https://www.sciencedirect.com/science/article/pii/S0022249612000260 |
good find! Given his interest and extensive reasearch and writing on SDT, SDT via regression and bayesian implementations, measurement in general (https://www.tc.columbia.edu/faculty/ld208/), maybe at some point we can contact him and chat about the package and get his thoughts on how the SDT is implemented, maybe get him involved if he wants to? He's also written a bunch of programs for other software for SDT models, so I bet his experience and insight will be useful:
|
|
I thought the same. For the moment, I think we can implement plenty of things ourselves, but it would still be interesting to discuss with him about optimizations with respect to the SDT implementations. He has also done some interesting work linking SDT to psychometric models such as IRT that I would find interesting to explore. But then again, there is already plenty of work on my desk. So maybe this is rather something for the future... |
| #' @param crit A vector of numerical values for the decision criterion. | ||
| #' @param stimulus A character vector ("signal"/"noise", "old"/"new") or | ||
| #' integer vector (1/0) coding which stimuli were shown alongside the observed responses, | ||
| #' or for which to generate data for. If no information is provided for `rSDT` then we will |
There was a problem hiding this comment.
the function only returns hits by default. I think this is probably for the best, and would rather remove the description than change the function. Or return a list of two vectors, one for hits and one for FAs
There was a problem hiding this comment.
I added some documentation to clarify this point. My thought was to make the distribution functions consistent with the implementation as bmmmodel.
There was a problem hiding this comment.
Maybe you can have a look if this resolves your concerns.
|
|
||
| #' @rdname SDTdist | ||
| #' @export | ||
| rSDT <- function(n, size, dprime = 1, crit = dprime/2, stimulus = 1, dist_noise = "normal") { |
There was a problem hiding this comment.
could be good to have an option to return proportions instead of counts
There was a problem hiding this comment.
Hmm... do you mean that we return the proportion of hits/FA instead of the count. Why would users need these? If they want they can directly compute them from returnedCounts/size, but they would loose the information on how many trials the proportion was calculated on.
Maybe I am misunderstanding something, but my thought was to make these functions consistent with the model implementation.
R/bmm_model_SDT.R
Outdated
| nTrials <- model$resp_vars$nTrials | ||
| brms_formula <- brms::bf(paste0(response," | ", "trials(",nTrials,")", " ~ dprime*",stimulus," - crit" ), nl = TRUE) | ||
| } else { | ||
| brms_formula <- brms::bf(paste0(response," ~ dprime*",stimulus," - crit"),nl = TRUE) |
There was a problem hiding this comment.
this works for symmetric distributions, but for assymetric ones it gives the wrong part of the cdf. Will explain in person
There was a problem hiding this comment.
As far as I have checked with the parameter recoveries I ran, this implementation does recover the parameters well using the cloglog link for Gumbel noise. But we should test this thoroughly to make sure that I did not miss anything.
- make "normal" default noise distribution - remove unnecessary functions - Improve documentation - move `stimulus` and `nTrials` to other_vars
|
Things that need to be added for finishing up the Old/New Recognition SDT models.
|
|
I will see that I take care of most of the to dos in the next week. |
|
Perfect, and I can finish the review and test it with the code you sent in the meantime |
- update the already implemented SDT models to the current bmm version - add an info `custom_bmf2bf` to all model objects to handle the `bmmformula` to `brmsformula` translation more efficiently.
|
I just found this package implementing EV and UEV SDT models in STAN with a newly proposed link function that allows to fit hierarchical models also for the thresholds: https://github.com/boryspaulewicz/bhsdtr2/tree/master Maybe we should have a look at how they coded this in STAN and see if we can adapt this for our package. Generally, the principle for predicting the model parameters looks pretty similar, with the difference that they do interact with brms at all. |
Summary
bmmmodelclass.Tests
[x] Confirm that all tests passed
[x] Confirm that devtools::check() produces no errors
Release notes