[Nonlinear] Performance regression of Hessian evaluation in MOI 1.41

[frapac]

I observe a significant performance regression when using PowerModels.jl with the latest version of MOI.

The MWE is using PowerModels to illustrate the issue:

using JuMP
using PowerModels
using Ipopt
using HSL_jll

instance = "pglib_opf_case9241_pegase.m"
pm = instantiate_model(joinpath(PGLIB_DIR, instance), ACPPowerModel, PowerModels.build_opf)
model = pm.model
JuMP.set_optimizer(model, Ipopt.Optimizer)
JuMP.set_optimizer_attribute(model, "hsllib", HSL_jll.libhsl_path)
JuMP.set_optimizer_attribute(model, "linear_solver", "ma57")
JuMP.set_optimizer_attribute(model, "max_wall_time", 900.0)
JuMP.set_optimizer_attribute(model, "max_iter", 10)
JuMP.optimize!(model)

nlp_block = JuMP.MOI.get(JuMP.unsafe_backend(model), JuMP.MOI.NLPBlock())
total_callback_time =
	nlp_block.evaluator.eval_objective_timer +
	nlp_block.evaluator.eval_objective_gradient_timer +
	nlp_block.evaluator.eval_constraint_timer +
	nlp_block.evaluator.eval_constraint_jacobian_timer +
	nlp_block.evaluator.eval_hessian_lagrangian_timer
println("")
println("   callbacks time:")
println("   * obj.....: $(nlp_block.evaluator.eval_objective_timer)")
println("   * grad....: $(nlp_block.evaluator.eval_objective_gradient_timer)")
println("   * cons....: $(nlp_block.evaluator.eval_constraint_timer)")
println("   * jac.....: $(nlp_block.evaluator.eval_constraint_jacobian_timer)")
println("   * hesslag.: $(nlp_block.evaluator.eval_hessian_lagrangian_timer)")

With MOI 1.41, I obtain a significant increase in the time spent evaluating the Hessian:

   callbacks time:
   * obj.....: 0.0
   * grad....: 0.0
   * cons....: 0.8870217800140381
   * jac.....: 0.4554314613342285
   * hesslag.: 43.00013470649719

whereas with MOI 1.39, everything was working as expected:

   callbacks time:
   * obj.....: 0.0
   * grad....: 0.0
   * cons....: 0.8886904716491699
   * jac.....: 0.46123671531677246
   * hesslag.: 2.1943259239196777

I suspect MOI 1.40 has introduced some slowdown in the Hessian evaluation.

cc @frederikgeth

[blegat]

I did a few PRs on the AD so it should be that, we should start bisecting to detect the culprit.

[odow]

I can look today. This is one reason that I was very nervous to go making changes to the AD code.

[odow]

This is surprisingly unusual to debug. Here's the flame graph: