Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Where can I find documentation on calculation of gradient gradientXalpha in word2vec? #18

Open
DamirTenishev opened this issue Aug 5, 2024 · 0 comments

Comments

@DamirTenishev
Copy link

You write "word2vec++ code is simple and well documented". Could you please help me with some link to a document which could help me to understand the math behind the line you inherited from the original code:

File trainThread, line 237:

auto gradientXalpha = (1.0f - static_cast(huffmanData->huffmanCode[i]) - f) * (*m_sharedData.alpha);

I can't grasp the idea behind this "1.0f - ...".

As I understand the gradient intended to mack Huffman code as close as possible to value f we calculated above. If so, what is the reason of this "1.0f - ..."? Especially taking into account that the huffmanCode is either 0 or 1, so this "1-..." just reverses it. What it the intend/logic behind it?

@DamirTenishev DamirTenishev changed the title Where can I find documentation on calculation of gradient g in word2vec? Where can I find documentation on calculation of gradient gradientXalpha in word2vec? Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant