Lesson 3 - extra details

docs/index.html

@@ -272,6 +272,7 @@ <h1 class="title toc-ignore">Bayesian statistical modelling and workflow
 </div>
 
 
+<p>Last updated: 2024-03-29 09:22:51.489594.</p>
 <ul>
 <li>Lesson 1: <a href="lesson1.html">outline</a>, <a
 href="lesson1-tasks.html">tasks</a></li>

docs/lesson3-tasks.html

@@ -296,25 +296,36 @@ <h2>Task 1: Bayesian “t-test”</h2>
 </div>
 <div id="task-2-convert-to-long-format" class="section level2">
 <h2>Task 2: Convert to “long format”</h2>
-<p>If you haven’t already, adapt the model from the previous task so
-that the observed <span class="math inline">\(y\)</span> are all stored
-in a single vector of length <span class="math inline">\(N\)</span> and
-another vector of length <span class="math inline">\(N\)</span> encodes
-group membership. I.e. your <code>data</code> section should look
-something like:</p>
+<p>Adapt the model from the previous task so that a) the standard
+deviation (<span class="math inline">\(\sigma\)</span>) is the same for
+all data and b) the observed <span class="math inline">\(y\)</span> are
+all stored in a single vector of length <span
+class="math inline">\(N\)</span> and another vector of length <span
+class="math inline">\(N\)</span> encodes group membership. I.e. your
+<code>data</code> section should look something like:</p>
 <pre><code>data {
    int&lt;lower=0&gt; N;
    vector[N] y;
    array[N] int&lt;lower=0,upper=1&gt; isB;
 }</code></pre>
+<p>Hint: after converting <code>isB</code> into a <code>vector</code>
+(with <code>to_vector</code>) you can obtain a vector of length
+<code>N</code> of means for all observations by addition and
+multiplication.</p>
+<p>You can then directly pass the vector to <code>normal_lpdf</code>,
+i.e. have</p>
+<pre><code>target += normal_lpdf(y | __COMPUTATION_HERE__, sigma);</code></pre>
+<p>If stuck, see <a
+href="https://mc-stan.org/docs/stan-users-guide/regression.html"
+class="uri">https://mc-stan.org/docs/stan-users-guide/regression.html</a></p>
+<p>For extra credit, you may try to figure out how to keep the distinct
+standard deviations following a similar computation.</p>
 </div>
 <div id="task-3-add-a-continuous-predictor" class="section level2">
 <h2>Task 3: Add a continuous predictor</h2>
 <p>Add a new continuous predictor (a vector of real numbers) for each
-data point and add an extra coefficient to model its influence. To
-simplify, you may (or may not) assume that the standard deviation of the
-outcome is the same for all observations (i.e. no need to keep separate
-<span class="math inline">\(\sigma_A, \sigma_B\)</span>)</p>
+data point and add an extra coefficient (a new variable in the
+<code>parameters</code> block) to model its influence.</p>
 <p>Simulate data and check parameter recovery.</p>
 </div>
 <div id="task-4-matrix-multiplication" class="section level2">
@@ -339,12 +350,14 @@ <h2>Task 5: Compare to user’s guide</h2>
 in Stan’s User’s guide: <a
 href="https://mc-stan.org/docs/stan-users-guide/regression.html#linear-regression"
 class="uri">https://mc-stan.org/docs/stan-users-guide/regression.html#linear-regression</a></p>
-<p>Don’t be afraid to use the User’s guide as a starting point for your
-models, it is a great resource!</p>
+<p>Generally, don’t be afraid to use the User’s guide as a starting
+point for your models, it is a great resource!</p>
 </div>
-<div id="task-6-negative-binomial-regression" class="section level2">
-<h2>Task 6: Negative binomial regression</h2>
-<p>Use dummy coding to extend the model to have 3 groups in the
+<div id="task-6-dummy-coding" class="section level2">
+<h2>Task 6: Dummy coding</h2>
+<p>Use <a
+href="https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faqwhat-is-dummy-coding/">dummy
+coding</a> to extend the model to have 3 groups instead of 2 in the
 categorical predictor. Your Stan code should not need to change.</p>
 <p>Simulate data, fit, check parameter recovery.</p>
 </div>

docs/lesson3.html

@@ -300,20 +300,36 @@ <h2>Linear regression</h2>
 class="uri">https://lindeloev.github.io/tests-as-linear/</a></li>
 <li>Taylor series</li>
 <li>Intercept, coefficients</li>
-<li>Dummy coding
+<li><a
+href="https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faqwhat-is-dummy-coding/">Dummy
+coding</a>
 <ul>
-<li>There are other ways to code (e.g. effect coding)</li>
-<li><code>model.matrix</code> in <code>R</code></li>
+<li>There are other ways to code (e.g. <a
+href="https://stats.oarc.ucla.edu/other/mult-pkg/faq/general/faqwhat-is-effect-coding/">effect
+coding</a>)</li>
+<li><code>model.matrix</code> in <code>R</code> does the coding for
+you</li>
 </ul></li>
 <li>Linear predictors and matrix multiplication
 <ul>
 <li>Intercept in the matrix</li>
 </ul></li>
-<li>Least squares and normal distribution</li>
-<li>Generalized linear models (GLM)
+<li><a href="https://en.wikipedia.org/wiki/Least_squares">Least
+squares</a> and its relatio to normal distribution
+<ul>
+<li>Maximum likelihood estimator is equivalent to least-squares.</li>
+</ul></li>
+<li><a
+href="https://en.wikipedia.org/wiki/Generalized_linear_model">Generalized
+linear models</a> (GLM)
 <ul>
 <li>Link function, inverse link function</li>
-<li>log for positive, logit for [0, 1].</li>
+<li>Most common link functions:
+<ul>
+<li>log for positive outcomes (i.e. exponentiate the predictors),</li>
+<li>logit (i.e. apply inverse logit to the predictor) for outcomes in
+[0, 1] - most notably for probabilities in logistic regression.</li>
+</ul></li>
 </ul></li>
 <li>Posterior predictive checks as a method to determine which
 predictors to add</li>

index.Rmd

@@ -3,6 +3,7 @@ title: "Bayesian statistical modelling and workflow in Stan"
 output: html_document
 ---
 
+Last updated: `r  Sys.time()`.
 
 - Lesson 1: [outline](lesson1.html), [tasks](lesson1-tasks.html)
 - Lesson 2: [outline](lesson2.html), [tasks](lesson2-tasks.html)

lesson3-tasks.Rmd

@@ -29,7 +29,7 @@ How big do the groups need to be, so that your posterior interval for $\delta$ e
 
 ## Task 2: Convert to "long format"
 
-If you haven't already, adapt the model from the previous task so that the observed $y$ are all stored in a single vector of length $N$ and another vector of length $N$ encodes group membership. I.e. your `data` section should look something like:
+Adapt the model from the previous task so that a) the standard deviation ($\sigma$) is the same for all data and b) the observed $y$ are all stored in a single vector of length $N$ and another vector of length $N$ encodes group membership. I.e. your `data` section should look something like:
 
 ```
 data {
@@ -39,10 +39,23 @@ data {
 }
 ```
 
+Hint: after converting `isB` into a `vector` (with `to_vector`) you can obtain 
+a vector of length `N` of means for all observations by addition and multiplication.
+
+You can then directly pass the vector to `normal_lpdf`, i.e. have 
+
+```
+target += normal_lpdf(y | __COMPUTATION_HERE__, sigma);
+```
+
+If stuck, see https://mc-stan.org/docs/stan-users-guide/regression.html
+
+For extra credit, you may try to figure out how to keep the distinct standard deviations following a similar computation.
+
+
 ## Task 3: Add a continuous predictor
 
-Add a new continuous predictor (a vector of real numbers) for each data point and add an extra coefficient to model its influence. 
-To simplify, you may (or may not) assume that the standard deviation of the outcome is the same for all observations (i.e. no need to keep separate $\sigma_A, \sigma_B$)
+Add a new continuous predictor (a vector of real numbers) for each data point and add an extra coefficient (a new variable in the `parameters` block) to model its influence. 
 
 Simulate data and check parameter recovery.
 
@@ -68,11 +81,11 @@ Test that with the same data, you get the same results as in the previous versio
 Compare what you've built with the example linear regression models in Stan's User's guide:
 https://mc-stan.org/docs/stan-users-guide/regression.html#linear-regression
 
-Don't be afraid to use the User's guide as a starting point for your models, it is a great resource!
+Generally, don't be afraid to use the User's guide as a starting point for your models, it is a great resource!
 
-## Task 6: Negative binomial regression
+## Task 6: Dummy coding
 
-Use dummy coding to extend the model to have 3 groups in the categorical predictor. 
+Use [dummy coding](https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faqwhat-is-dummy-coding/) to extend the model to have 3 groups instead of 2 in the categorical predictor. 
 Your Stan code should not need to change.
 
 Simulate data, fit, check parameter recovery.

lesson3.Rmd

@@ -28,15 +28,18 @@ Pick projects!
 - The most ubiquitous type of model --- see e.g. https://lindeloev.github.io/tests-as-linear/
 - Taylor series
 - Intercept, coefficients
-- Dummy coding
-  - There are other ways to code (e.g. effect coding)
-  - `model.matrix` in `R`
+- [Dummy coding](https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faqwhat-is-dummy-coding/)
+  - There are other ways to code (e.g. [effect coding](https://stats.oarc.ucla.edu/other/mult-pkg/faq/general/faqwhat-is-effect-coding/))
+  - `model.matrix` in `R` does the coding for you
 - Linear predictors and matrix multiplication
   - Intercept in the matrix
-- Least squares and normal distribution
-- Generalized linear models (GLM)
+- [Least squares](https://en.wikipedia.org/wiki/Least_squares) and  its relatio to normal distribution
+  - Maximum likelihood estimator is equivalent to least-squares.
+- [Generalized linear models](https://en.wikipedia.org/wiki/Generalized_linear_model) (GLM)
   - Link function, inverse link function
-  - log for positive, logit for [0, 1].
+  - Most common link functions:
+    - log for positive outcomes (i.e. exponentiate the predictors), 
+    - logit (i.e. apply inverse logit to the predictor) for outcomes in [0, 1] - most notably for probabilities in logistic regression.
 - Posterior predictive checks as a method to determine which predictors to add
 
 Now, let's do some tasks. Note that they match workflow (simple stuff first, ...)