DNMY: add solver-independent(-ish) callbacks

[odow]

PR in JuMP exposing this:
jump-dev/JuMP.jl#1849

Solver-specific implementations:
GLPK: jump-dev/GLPK.jl#91
Gurobi: jump-dev/Gurobi.jl#194

[mlubin]

Who are the intended users of this callback API? If it's meant to help port Pajarito and Pavito to MOI, that makes sense. Let's call it legacy_callbacks.jl and put a big disclaimer on it. If it's meant for new users, I'm not a fan. It's a half-baked API (not to discredit the author, there just isn't a good way to do this). Do we really want to be responsible for supporting this API and answering questions about how to query X or Y solver-specific property from the callbacks?

If this is meant to be the easy API then shouldn't it also work with the caching optimizer and the bridges? Who volunteers to do that? Why is this a good use of developer time when we could make nice solver-specific callback APIs instead?

[mlubin]

What attributes can be queried in the heuristic callback?

[odow]

Only VariablePrimal

[mlubin]

This is a regression from MPB and isn't sufficient to cover Pajarito: https://github.com/JuliaOpt/Pajarito.jl/blob/027cfbf882b4a1ae78f45fb11a6e51c58468a20f/src/conic_algorithm.jl#L1446 (this is from the lazy callback)

[mlubin]

Does the solution have to satisfy all constraints in the model?

[odow]

No, see above

[mlubin]

If a solution does not satisfy constraints, is it guaranteed to be rejected?

[mlubin]

What does this mean? What are the constraints that could possibly have been added and how do they differ from the constraints actually returned?

[odow]

Poor wording on my part. It should just say all constraints that have been added. I'll hold off changing until we have a consensus on the larger issue of whether we want these in the first place.

[codecov-io]

Codecov Report

Merging #670 into master will decrease coverage by 0.05%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #670      +/-   ##
==========================================
- Coverage   94.96%   94.91%   -0.06%     
==========================================
  Files          50       51       +1     
  Lines        5347     5350       +3     
==========================================
  Hits         5078     5078              
- Misses        269      272       +3

Impacted Files	Coverage Δ
src/MathOptInterface.jl	0% <ø> (ø)	⬆️
src/callbacks.jl	0% <0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f3cb556...d33e7fa. Read the comment docs.

[codecov-io]

Codecov Report

Merging #670 into master will decrease coverage by 0.05%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master     #670      +/-   ##
==========================================
- Coverage   94.96%   94.91%   -0.06%     
==========================================
  Files          50       51       +1     
  Lines        5347     5350       +3     
==========================================
  Hits         5078     5078              
- Misses        269      272       +3

Impacted Files	Coverage Δ
src/MathOptInterface.jl	0% <ø> (ø)	⬆️
src/callbacks.jl	0% <0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f3cb556...d33e7fa. Read the comment docs.

[odow]

Who are the intended users of this callback API? If it's meant to help port Pajarito and Pavito to MOI, that makes sense.

Yes, mainly the MINLP solvers.

Let's call it legacy_callbacks.jl and put a big disclaimer on it.

Another option is to put it in a Legacy submodule (and not provide JuMP bindings) without documentation on the main webpage.

If it's meant for new users, I'm not a fan. It's a half-baked API (not to discredit the author, there just isn't a good way to do this).

@juan-pablo-vielma and I discussed this a little. There isn't a good way to do generic callbacks. We agree. But the concept of a lazy constraint is pretty solver-independent, and is useful for new users. To make it practical though, it has to be restrictive.

• Will only be called from integer feasible nodes
• No certainty when (or if) it will be called
• Can only access primal variable solution

Do we really want to be responsible for supporting this API and answering questions about how to query X or Y solver-specific property from the callbacks?

Answer: the functionality of JuMP-provided callbacks is limited by design. For solver-specific functions, use the solver-specific callbacks.

If this is meant to be the easy API then shouldn't it also work with the caching optimizer and the bridges? Who volunteers to do that?

Nope. See limited functionality. Direct mode only.

Why is this a good use of developer time when we could make nice solver-specific callback APIs instead?

Pajarito and Pavito.

[mlubin]

Another option is to put it in a Legacy submodule (and not provide JuMP bindings) without documentation on the main webpage.

If you're happy calling them legacy callbacks and explaining why they exist, I don't see an issue documenting them and having JuMP bindings.

the functionality of JuMP-provided callbacks is limited by design

I wouldn't call these the JuMP-provided callbacks. JuMP can also provide solver-specific callbacks that aren't limited by design.

But the concept of a lazy constraint is pretty solver-independent, and is useful for new users.

I question how useful the solver-independent aspect is for new users. I think users would be just fine given an example of how to do lazy callbacks in the solver of their choice. Especially when it would lead them to the right API instead of being a dead end.

[juan-pablo-vielma]

In think the advantage of solver independent lazy cut callbacks can be seen when thinking about modeling with a large number of constraints that can be efficiently separated. This is a concept that beginners often are not aware of (I have had referees in OR tell me that they don't believe I solved a MIP with an exponential number of constraints) and it is an important part of the power of MIP (and JuMP).

If we consider lazy cuts as a modeling tool it makes sense that they are not dependent on the solver: you do not need to know anything about the solver to model a MIP without a large number of constraints. This modeling tool goal also keeps the interface simple and bounded: questions like how do I access this solver specific feature become irrelevant as you should only be using this feature it you don't want to think of the solver. So I would say this version would not actually have MINLP solvers as the intended users (they should use the solver specific callbacks).

The issue with the caching optimizer and the bridges is worth considering though. Maybe the way to think about it is considering how SOS2 constraints relate to these two features (maybe also PolyJuMP relates and/or JuMPer would relate).

[blegat]

Is a lazy constraint just a constraint that is adding during optimize! ? Then maybe instead of creating a new function we could simple use MOI.add_constraint, that would allow bridges to just work for lazy constraints.

[mlubin]

If we're considering these lazy constraints as a modeling tool and not as a direct way for users to interact with solvers, then I like the suggestion of coming up with an MOI function object to represent them. The MOI function would store the callback that generates the constraints given a point to separate.

However, this doesn't address the Pajarito/Pavito use case. It also doesn't cover the heuristic callback that's proposed in this PR.

[FD-V]

Hi, I like to bring some arguments in favour of developing optimiser independent callbacks in general instead of solver-specific ones (or on top of solver-specific ones), and in particular lazy_cut_callbacks that are special useful to implement a generic Benders decomposition framework as we plan to do in Coluna. Beyond the fact that lazy_cut_callbacks can be seen as being part of the modelling JuMP language, we should see the great feature that offers MOI by being solver independent. In our development of the Coluna Platform, as in many other and futur project that are building over MOI, we wish to hide the MIP optimiser complexity to the user by providing a generic cut-callback. If it is not done in MOI, all the platforms above MOI will have to develop their own implementation with a SWITCH system to act differently depending on the solver being used. Wasn't it the goal of MOI to provide such service to all, rather than each platform having this maintenance duty.

With Issam, we though of what could be a good compromise between the need for generic callbacks and the reality of the different functionalities that are offered by each solver. The JuMP team could select the "best" (most useful) callback options that are offered by the different solvers (not being restricted to those that are offered by all), and to implement a generic callback that supports those best options/parametrisations. At run time, MOI can check whether the demanded generic feature is offered by the selected solver; it returns an explicit error message if the feature is not supported.

This is just a suggestion to contribute to the debate.

[chriscoey]

following up on the brief gitter discussion yesterday, and the discussion at JuMP-dev 3 in mid-march. upgrading Pajarito and Pavito for MOI is going to be a big job and I'm not prepared to do it until callbacks are resolved.
so let's try to decide on a minimal set of TODOs on this PR
@blegat what needs to be done to this PR to incorporate the ideas you had?

[blegat]

Not needed anymore

[blegat]

tl;dr: One change we should consider is replacing the Callbacks struct by a LazyCallback and a HeuristicCallback struct. Then we should add attributes for the variable primal, primal status, ... accessed during the callbacks which takes the callback data as argument.

There was a concern with the current approach which is:
Inside a callback, the model is set in a state where the statuses MOI.PrimalStatus, MOI.VariablePrimal, ... give the values corresponding to the callback.
For instance, in Gurobi:
https://github.com/JuliaOpt/Gurobi.jl/blob/2309d97266f5d41c6c2431cef2040a0f3fa3541a/src/MOI_wrapper.jl#L695-L704
for lazy callbacks, the solution give the integer solution and for heuristic callbacks, the solution give the fractional solution.
IIRC, the concern was that would not work if several callbacks are called simultaneously.

Sandbox

We then discussed if it was possible to create a kind of sandbox for calls inside a callbacks so that getting attributes would either throw an helpful error if it cannot be called inside the callback or give the corresponding solution. This sandbox could be achieved by wrapping the Gurobi MOI model inside an MOI layer tailored for the callback. If the user tries to call the original model, it would get error since this model still has status MOI.OPTIMIZE_NOT_CALLED.
So if two callbacks are called simultaneously, one will get the model wrapped by one layer and the other will get the model wrapped by another layer so their call would be interpreted differently without the need for modifying the state of the Gurobi model.

New attributes

The advantage of the sandbox is that the user can use the classical attributes MOI.VariablePrimal, ... and hence also the JuMP user friendly syntax value, ... on the sandbox and it is correctly interpreted.
If we relax this requirement to use the same attributes, this "state" that we want to have for the callback which basically represent the "callback data" of this PR, instead of being stored in an MOI layer, it could be attached to ever attributes.
So instead of doing MOI.get(sandbox, MOI.VariablePrimal(), x), we would do MOI.get(model, MOI.IntegerNodeVariablePrimal(callback_data), x).
If the user does MOI.get(model, MOI.VariablePrimal(), x), he would get an error because the status is OPTIMIZE_NOT_CALLED.

At the end of the discussion at JuMP-dev, we prefered the "New attributes" approach for its simplicity and genericity. It is not specific to MIP callbacks used today and would work for any kind of callbacks (maybe even NLP callbacks asking for gradients, hessian, ... ?) that would be defined as "just a function being called during where the user can access custom attributes".
So the message to optimizers is simple: If you have a custom callback, just define MyCallback <: MOI.AbstractModelAttribute, MySolutionValue(callback_data) <: MOI.AbstractVariableAttribute (if the MOI wrapper wants to cache the results for a callback as is currently done in the Gurobi wrapper, it can store the cache in a vector or dict and give the index of the cached result in the callback_data) and the user can simply do

model = Model(...)
@variable(model, x)
MOI.set(model, MyCallback(), (cb_data) -> begin
    v = MOI.get(model, MySolutionValue(cb_data), x)
    ...
end)

Then a solver independent callback would simply be : MySolutionValue is defined with the exact same meaning in several solver wrapper, let's move it to MOI. Same thing for MyCallback.
Here we have two examples of MyCallback with same meaning used in several solvers: LazyCallback and HeuristicCallback so it makes sense to move them to MOI.

[blegat]

Another thing to consider is that if users want the user-friendliness of the Sandbox approach, the sandbox approach can be implemented on top of the "new attributes" approach and this can even be done outside of JuMP/MOI, e.g. like a Training Wheels Protocol

[mlubin]

1. I am strongly not in favor of a sandbox, except maybe in a training wheels version. We can't actually prevent people from keeping a reference to the model and variables outside the sandbox.
2. I agree with MySolutionValue(cb_data)/MOI.IntegerNodeVariablePrimal(callback_data) over VariablePrimal. This doesn't require redefining the meaning of VariablePrimal for callbacks and allows for multithreaded callbacks in the future. If users want to access the values of the variables through JuMP.value, we can provide JuMP.load_callback_solution_into_value or something similar.