[layer] add pow operation layer @open sesame 12/06 20:13 #2801
Conversation
baek2sm
requested review from
myungjoo,
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
gichan-jang,
anyj0527,
lhs8928,
songgot,
jihochu,
DonghakPark,
SeoHyungjun,
skykongkong8,
djeong20 and
EunjuYang
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2801. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
baek2sm
force-pushed
the
pow_layer
branch
7 times, most recently
from
0653289 to
ac31a83
Compare
|
|
cibot:
nntrainer/layers/pow_layer.cpp
Outdated
| void PowLayer::forwarding_operation(const Tensor &input, Tensor &hidden) { | ||
| float exp = std::get<props::Exponent>(pow_props).get(); | ||
| input.pow(exp, hidden); | ||
| } |
There was a problem hiding this comment.
Quick question! this isn't really has to do with PR but:
Is there any difference with multiplying itself if the given exponent is set to 2? ( or inv_sqrt function <-> pow with -0.5 exponent )
I find almost every pow or pow_i function usage in the nntrainer use 2.0, 0.5, or -0.5 as an exponent, and this is handled with naive loop Tensor member function.
There was a problem hiding this comment.
@skykongkong8 There is no difference in computation between multiplying itself and setting an exponent value to 2. However, for the cases you mentioned, I will handle them by adding square, sqrt and rsqrt functions instead of using the pow function. Thanks!
baek2sm
changed the title
baek2sm
changed the title
baek2sm
changed the title
baek2sm
changed the title
baek2sm
changed the title
| @@ -9,6 +9,7 @@ layer_sources = [ | |||
| 'subtract_layer.cpp', | |||
| 'multiply_layer.cpp', | |||
| 'divide_layer.cpp', | |||
| 'pow_layer.cpp', | |||
There was a problem hiding this comment.
This is a simple suggestion (not directly related to this PR though).
What about using prefix like 'op_' to operation layers?
It might reduce the confusion on the layer types (e.g., add_layer & addition_layer).
There was a problem hiding this comment.
That sounds good idea. I'll reflect it in later pr. thanks!
nntrainer/layers/pow_layer.cpp
Outdated
| } | ||
|
|
||
| void PowLayer::forwarding_operation(const Tensor &input, Tensor &hidden) { | ||
| float exp = std::get<props::Exponent>(pow_props).get(); |
There was a problem hiding this comment.
Can we change the name exp ? (exp function is defined in cmath)
There was a problem hiding this comment.
I've modified it(exp -> exponent). Thanks a lot!
|
|
|
added a pow operation layer. there was an example of a pow layer in the custom layers, so I modified the key value of the custom pow layer to "custom_pow" in order to avoid duplication of layer key value. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Seungbaek Hong <[email protected]>
baek2sm
changed the title
added a pow operation layer.
there was an example of a pow layer in the custom layers, so I modified the key value of the custom pow layer to "custom_pow" in order to avoid duplication of layer key value.
Self evaluation:
Signed-off-by: Seungbaek Hong [email protected]