How to set hessian approximation alternative?

[baggepinnen]

Hey! 👋 I'm trying to use Hessian approximation, but setting it like (with JuMP)

m = Model(()->MadNLP.Optimizer(hessian_approximation=MadNLP.CompactLBFGS));

has no effect, it still uses the exact hessian. How do I accomplish this?

[frapac]

Thank you for reporting this, this is indeed an unexpected behavior. The bug is in the MOI wrapper, I am working on a fix.

The option MadNLP.CompactLBFGS is forgotten each time we are calling the function MOI.empty!. The function is called if we build the model with JuMP.

A temporary workaround is to bypass JuMP and use directly the MOI interface. If you have defined a JuMP model model, you can call MadNLP as:

optimizer = MadNLP.Optimizer()
MOI.copy_to(optimizer, model)
MOI.set(optimizer, MOI.RawOptimizerAttribute("hessian_approximation"), MadNLP.CompactLBFGS)
MOI.optimize!(optimizer)

[baggepinnen]

Thanks for the workaround, it works well in the meantime :)

[baggepinnen]

I'm trying the branch of the PR above, which might be a bit bold since it's not released :O but I'm hitting an error with the CompactLBFGS method:

ERROR: MethodError: no method matching eval_constraint_jacobian_transpose_product(::OptimizationMOI.MOIOptimizationNLPEvaluator{…}, ::Vector{…}, ::Vector{…}, ::Vector{…})

Closest candidates are:
  eval_constraint_jacobian_transpose_product(::MadNLPMOI._EmptyNLPEvaluator, ::Any, ::Any, ::Any)
   @ MadNLPMOI ~/.julia/dev/MadNLP/ext/MadNLPMOI/MadNLPMOI.jl:102
  eval_constraint_jacobian_transpose_product(::MadNLPMOI.Optimizer, ::Any, ::Any, ::Any)
   @ MadNLPMOI ~/.julia/dev/MadNLP/ext/MadNLPMOI/MadNLPMOI.jl:749
  eval_constraint_jacobian_transpose_product(::MathOptInterface.Test.HS071, ::Any, ::Any, ::Any)
   @ MathOptInterface ~/.julia/packages/MathOptInterface/jqDoD/src/Test/test_nonlinear.jl:145
  ...

Stacktrace:
  [1] eval_constraint_jacobian_transpose_product(model::MadNLPMOI.Optimizer, Jtv::Vector{…}, x::Vector{…}, v::Vector{…})
    @ MadNLPMOI ~/.julia/dev/MadNLP/ext/MadNLPMOI/MadNLPMOI.jl:750
  [2] jtprod!
    @ ~/.julia/dev/MadNLP/ext/MadNLPMOI/MadNLPMOI.jl:794 [inlined]
  [3] _eval_jtprod_wrapper!(cb::MadNLP.SparseCallback{…}, x::Vector{…}, v::Vector{…}, jvt::Vector{…})
    @ MadNLP ~/.julia/dev/MadNLP/src/nlpmodels.jl:620
  [4] eval_lag_hess_wrapper!(solver::MadNLPSolver{…}, kkt::MadNLP.SparseKKTSystem{…}, x::MadNLP.PrimalVector{…}, l::Vector{…}; is_resto::Bool)
    @ MadNLP ~/.julia/dev/MadNLP/src/IPM/callbacks.jl:169
  [5] eval_lag_hess_wrapper!
    @ ~/.julia/dev/MadNLP/src/IPM/callbacks.jl:142 [inlined]
  [6] regular!(solver::MadNLPSolver{…})
    @ MadNLP ~/.julia/dev/MadNLP/src/IPM/solver.jl:262
  [7] solve!(nlp::MadNLPMOI.MOIModel{…}, solver::MadNLPSolver{…}, stats::MadNLP.MadNLPExecutionStats{…}; x::Nothing, y::Nothing, zl::Nothing, zu::Nothing, kwargs::@Kwargs{})
    @ MadNLP ~/.julia/dev/MadNLP/src/IPM/solver.jl:165
  [8] solve!
    @ ~/.julia/dev/MadNLP/src/IPM/solver.jl:128 [inlined]
  [9] solve!
    @ ~/.julia/dev/MadNLP/src/IPM/solver.jl:14 [inlined]
 [10] solve!(solver::MadNLPSolver{…})
    @ MadNLP ~/.julia/dev/MadNLP/src/IPM/solver.jl:17
 [11] optimize!(model::MadNLPMOI.Optimizer)
    @ MadNLPMOI ~/.julia/dev/MadNLP/ext/MadNLPMOI/MadNLPMOI.jl:948
 [12] __solve(cache::OptimizationMOI.MOIOptimizationNLPCache{…})
    @ OptimizationMOI ~/.julia/packages/OptimizationMOI/LnXYV/src/nlp.jl:496
 [13] solve!(cache::OptimizationMOI.MOIOptimizationNLPCache{…})
    @ SciMLBase ~/.julia/packages/SciMLBase/nftrI/src/solve.jl:188
 [14] macro expansion
    @ ./timing.jl:279 [inlined]
 [15] top-level scope
    @ ./REPL[4]:1

[frapac]

Interesting error. I don't think it's related to the PR. Looking at the stack-trace, it looks like you are using OptimizationMOI. Is that correct?

I suspect OptimizationMOI does not implement the transpose-Jacobian product, required by MadNLP to compute the update in BFGS. A possible workaround is to use by default the full Jacobian to deduce the transpose-Jacobian-product, but that would be more expensive.

[baggepinnen]

Thanks for pointing this out! I opened an issue about it here

• Support for MOI.eval_constraint_jacobian_transpose_product SciML/Optimization.jl#808