How can I make the params of the Model improve with the gradient in IDEA?

[blue-me]

I am now learning how to improve the params of the Linear Regression with gradients in the e-book Deep into Deep Java learning on IDEA. But I met a huge problem when training the params.

I use the instruction param.subi(param.getGradient().mul(lr).div(batchSize)); in IDEA and run it., but the console warns me that a leaf Variable that requires grad is being used in an in-place operation.

Searching information about this problem, I know that it is triggered by math operation directly on NDArray, but I don't know how to solve it with other instructions. I have tried the instruction "param.sub()" and "param = param - param.getGradient().mul(lr).div(batchSize)", but both fali.

The full warning message is below:
Exception in thread "main" ai.djl.engine.EngineException: a leaf Variable that requires grad is being used in an in-place operation.
at ai.djl.pytorch.jni.PyTorchLibrary.torchSubi(Native Method)
at ai.djl.pytorch.jni.JniUtils.subi(JniUtils.java:532)
at ai.djl.pytorch.engine.PtNDArray.subi(PtNDArray.java:535)
at ai.djl.pytorch.engine.PtNDArray.subi(PtNDArray.java:37)
at org.example.Test.sgd(Test.java:136)
at org.example.Test.main(Test.java:110)

My djl version:
ai.djl.pytorch pytorch-engine: 0.17.0
ai.djl.pytorch pytorch-native-cpu win-x86_64:1.11.0
ai.djl.pytorch pytorch-jni 1.11.0-0.17.0
ai.djl api 0.17.0
ai.djl basicdataset 0.17.0

[lanking520]

You cannot use subi, you have to use the sub function. What is the error message when you use sub?

[blue-me]

When I use sub, the console doesn't report any error but the params don't change at all. I think the gradients don't make any influence on the params of model.

[zachgk]

Yeah. So, what is happening is that you are building anoptimizer. In the optimizer, you will of course modify the parameters in place and do that with the in-place operations like subi().

However, the error you are getting is because the PyTorch engine seems to think you are building a model and in the model you shouldn't be doing in-place operations because it needs to keep intermediate values for the back-propagation calculation.

Taking a look at it, within the PyTorch JNI engine implementation (written in C) we seem to call a special function torch::autograd::AutoGradMode no_autograd_guard{false}; before the optimizer code. I expect we would also need to make this accessible or automatically called from DJL to make it possible to write custom optimizers.

In the meantime, I recommend either using the MXNet engine which doesn't have this issue or you can use the built-in optimizers like SGD that internally call no_autograd_guard(false)

[blue-me]

Ok, thanks! Your answer solves my puzzles hugely and I will intend to use the build-in members and functions.