Extend Regression module to address first point in issue #67 #113
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… there are normal errors
Added two type classes, 'WeightedNormalRegress' and 'NormalRegress',
which have primary methods 'weightedNormalRegress' and 'normalRegress',
respectively, which run weigihted and ordinary least squares, again
respectively. These methods extend 'olsRegress' and compute regression
coefficient standard errors and an overall model fit test statistic
in addition to estimating the coefficients and returning the R-squared
goodness of fit metric. Both type classes have instances for when
the noise variance is unknown and when it is known. In both instances,
it is assumed that the noise is Gaussian.
Some auxiliary changes were needed to make these extensions to the
Regression module:
**(list below is of important addtions but is not exhaustive)**
- A new error data type, 'TErr', which stores t-distributed errors
was added to Statistics/Types.hs for coefficient standard errors
when the noise variance is unknown.
- A function 'diagOf' in the Statistics/Matrix.hs returns the
diagonal of a square matrix. This is used to retrieve the standard
errors of each regression coefficient in 'varCoeff'.
- 'inv' in Statistics/Regression.hs can take the inverse of upper
or lower triangular matrices (also needed in 'varCoeff'.
Some future-looking changes were made as well:
- In anticipation of implementing generalized least square which
allows for specifcation of arbitrary linear correlation structure
across the noise component, this commit has implemented the
Cholesky factorization 'chol' in Statistics/Matrix/Algorithms.hs.
- 'solve' can now handle lower triangular matrices as well.
Previously extension of `solve` in Regression module to deal with lower triangular matrices incorrectly used `rfor` for a forward for loop. To resolve this issue and to still allow `solve` to work for both with maximum code reusability I wrote a wrapper for `rfor` and `for` in the Functions modules called `for_` that checks the the start and end indices to determine which direction to run the loop.
|
Thank you I'' review it over the weekend |
|
Sorry I forgot about this PR. I really should add to readme that if I don't reapond for a long time one should ping me |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I extended
olsRegressto the following settings with normally distributed errors(1) homoskedastic errors, known variance
(2) homoskedastic errors, unknown variance
(3) heteroskedastic errors, known variance
(4) heteroskedastic errors, unknown variance
In case (4) will still assume that we known the diagonal matrix$W$ in the error covariance $var(\epsilon) = \sigma^2W$ but not $\sigma^2$ . Case (4) addresses the first point specifically. The other cases are equally (or more) important from a statisticians point of view. In each case, standard errors for regression coefficients and overall model fit test statistics are computed along with their corresponding reference distributions.
I also provide example usage and implementation of Cholesky decomposition to use when extending the Regression module to eventually deal with regression error with arbitrary linear correlation structure (known as generalized least squares). This algorithm will also be useful if, in the future, one wanted to implement a version of R's
glm.