Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transformer to apply mean normalization #763

Open
solegalli opened this issue May 15, 2024 · 5 comments
Open

transformer to apply mean normalization #763

solegalli opened this issue May 15, 2024 · 5 comments

Comments

@solegalli
Copy link
Collaborator

in mean normalization, we subtract the mean from each value and then divide by the value range. This centres the variables at 0, and scales their values between -1 and 1. It is an alternative to standardization.

sklearn has no transformer to apply mean normalization. but we can combine the standard scaler and the robust scaler to do so. The thing is, that both transformers need to be fit over the raw data, so we can't use them within a pipeline, because the pipeline applies the transformation before the next transformer learns the required parameters.

My idea is to wrap both transformers within a class, so that fit is applied without transform, and then with those parameters, we can transform the data. See for example here: https://github.com/solegalli/Python-Feature-Engineering-Cookbook-Second-Edition/blob/main/ch07-scaling/Recipe-4-mean-normalization.ipynb

@VascoSch92
Copy link
Contributor

Instead of wrapping two transformers within a class, could also be an idea to create a brand new transformer which accomplish mean normalisation?

I understand that this could create some duplicated code, but on the other hand, the new mean normalisation transformer will be easy to understand and to debug.

I'm just thinking :-)

@solegalli
Copy link
Collaborator Author

Yes, it's perhaps a better idea.

@solegalli
Copy link
Collaborator Author

By any chance, would you like to give this one a go? creating a new class? and a new module. We don't have a module for scaling yet. @VascoSch92

@VascoSch92
Copy link
Contributor

Yes I can give it a try :-)

Just a little summary (please correct me if I'm wrong):

  1. the idea is to create a new module scaling which contains transformers to scale columns
  2. the first of these transformers will be the mean_normalization transformer right?

On a side note: there is still this PR on the distance transformer which attend a review :-) I corrected it and now should be better. Of course, if you have time :-D

@solegalli
Copy link
Collaborator Author

Yes, that's correct. Thanks for the reminder. I'll take a look this week :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants