-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathREADME.Rmd
97 lines (77 loc) · 4.95 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# categoryEncodings
<!-- badges: start -->
[](https://github.com/JSzitas/categoryEncodings/actions)
[](https://ci.appveyor.com/project/JSzitas/categoryEncodings)
[](https://codecov.io/gh/JSzitas/categoryEncodings?branch=master)
[](https://www.tidyverse.org/lifecycle/#stable)
[](https://www.gnu.org/licenses/gpl-3.0)
[](https://CRAN.R-project.org/package=categoryEncodings)
<!-- badges: end -->
**categoryEncodings** intends to provide a fast way to encode 'factor' or qualitative variables through various methods. The packages uses **data.table** as the backend for speed, with as few other dependencies as possible. Most of the methods are based on the paper of Johannemann et al.(2019) - Sufficient Representations for Categorical Variables (arXiv:1908.09874).
The current version features automatic inference of factors and uses a very simple heuristic for encoding, as well as allowing manual controls.
## Installation
You can install the latest version of **categoryEncodings** from [github](https://github.com/JSzitas/categoryEncodings) using the *devtools* package
``` {r, eval = FALSE}
devtools::install_github("JSzitas/categoryEncodings")
```
**NOTE:** The latest stable version available from **CRAN** contains features deprecated in the current development version - I hope to resolve this soon, and publish the development version.
## Example
Here we want to encode all of the factors in a given data.frame.
````{r, message = FALSE, eval = TRUE}
library(categoryEncodings)
# create some example data
data_fm <- cbind( data.frame(
matrix( rnorm(5*100),ncol = 5)),
sample(sample(letters, 10), 100, replace = TRUE),
sample(sample(letters, 20), 100, replace = TRUE),
sample(sample(1:10, 5), 100, replace = TRUE),
sample(sample(1:50, 35), 100, replace = TRUE ),
sample(1:2, 100, replace = TRUE ))
colnames(data_fm)[6:10] <- c( "few_letters", "many_letters",
"some_numbers", "many_numbers",
"binary" )
# it does not matter how many factor variables there are, whether they are encoded as factors
# and whether you supply a method to encode them by - some simple inference of factors is done
# based on the number of distinct values in every variable - over a certain threshold
# a variable is deemed as essentially a factor, and treated as such for conversion
# you will be notified of which variables are being converted via a warning
result <- encoder(data_fm)
# note that due to the data.table back-end, the result has to be saved to an object to be
# visible: otherwise printing is suppressed.
head(result$encoded)
````
We also recover a **function closure** which we can reuse to fit new data, as long as it conforms to the same format:
```{r}
# to fit to any dataset you can either call it directly - it is a single argument function
data_fm_encoded <- result$fitted_encoder(data_fm)
# or rename it, and stash it away for later use
encoding_function <- result$fitted_encoder
```
You also get a **"de-encoding"** function -
```{r}
deencoder <- result$fitted_deencoder
```
This undoes the "encoding", effectively returning the original data. This can be quite useful for interpretability methods, where the
interpretation becomes easier for un-encoded data.
Note that this sadly does not maintain the order of the data from the original - and some attributes may be lost. Nonetheless, the recovered data is almost the same, and equivalent for all practical purposes:
```{r}
original <- data.table::data.table(data_fm)
deencoded <- deencoder( result$encoded )
all.equal( data.table::setorder(original),
data.table::setorder(deencoded),
check.attributes = FALSE )
```
## Contributing
Please do contribute to the projects, all contributions are welcome, as long as people keep things civil - there is no need for negativity, hatred, and rudeness. Also, please do refrain from adding unnecessary dependencies (*Ex:* pipe) to the package (such pull requests as would add an unnecessary dependency will be denied/ suspended until the code can be made dependency free). This package wants to be as lightweight as possible - even if this means the code is a bit harder to write and maintain.