Model Observer

[michaelbynum]

Summary/Motivation:

This PR extracts the code for detecting changes in models from the persistent solver interfaces to its own, independent functionality. I will have a separate PR shortly that removes the code from the solver interfaces and updates them to use this code.

Changes proposed in this PR:

• add a module to contrib called observer that can be used to inform other classes of what changed in a pyomo model.

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[codecov]

Codecov Report

❌ Patch coverage is 91.39073% with 52 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.33%. Comparing base (7816915) to head (bb60959).

Files with missing lines	Patch %	Lines
pyomo/contrib/observer/model_observer.py	90.79%	50 Missing ⚠️
pyomo/contrib/observer/component_collector.py	96.72%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3695      +/-   ##
==========================================
+ Coverage   89.27%   89.33%   +0.05%     
==========================================
  Files         896      898       +2     
  Lines      103687   104291     +604     
==========================================
+ Hits        92567    93166     +599     
- Misses      11120    11125       +5     

Flag	Coverage Δ
builders	29.06% <21.68%> (?)
default	85.96% <91.39%> (?)
expensive	35.78% <21.68%> (?)
linux	86.99% <91.36%> (-2.06%)	⬇️
linux_other	86.99% <91.36%> (+0.02%)	⬆️
osx	83.14% <91.36%> (+0.04%)	⬆️
win	85.26% <91.36%> (+0.04%)	⬆️
win_other	85.26% <91.36%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jsiirola]

Overall, this is great. I have lots of questions, but very few should prohibit merging this.

[jsiirola]

This is basically reimplementing a version of ComponentSet. Is there a reason not to re-use that object?

[michaelbynum]

I have found that doing this manually is faster. That is the only reason. I'm not sure if the difference is enough to justify or not.

[michaelbynum]

I'll try to get some numbers.

[michaelbynum]

This seems to be about twice as fast.

[jsiirola]

OK - but we ought to see why ComponentSet is slower and try to fix it there.

[michaelbynum]

I don't disagree, but I don't think that needs to be part of the PR. We can always change the underlying implementation of this later. None of this is exposed to the user.

[jsiirola]

Don't you also have to update a constraint if the var is fixed?

[michaelbynum]

I think this is up to the observer... Worth a discussion.

[michaelbynum]

I'm not sure what the best solution is for this. I think we need more use cases. I vote we leave it for now. Here are a couple of examples that make this challenging:

m.x * (m.y + m.z) == 0
m.x.fix(1)
opt.solve(m)
m.x.value = 2

Right now, this works fine for the existing persistent solver interfaces because changing the value of m.x does not change the "structure" of the constraint (i.e., it remains linear). Essentially, we create a "mutable linear coefficient" and update that coefficient when the value of the variable changes. Furthermore, this does not require reconstructing the constraint in any way. However,

(m.y + m.z) ** m.x == 0
m.x.fix(1)

will currently just result in an exception for both gurobi persistent and highs persistent (scip is fine).

[michaelbynum]

I've been thinking about this more. Maybe we can add some public methods to ModelChangeDetector to provide more information if necessary. For example, we could add a method to get all of the constraints used by a variable (the change detector already has that data).

[michaelbynum]

@jsiirola and I discussed this further and agreed that expanding the interface was a good solution. I added to the docstring explaining that we can expand the interface, but we are waiting to do so until we have a need.

[jsiirola]

Additional question: who is responsible for verifying that the model does not contain any active components that the Solver/Observer doesn't know how to handle? This appears to assume that the responsibility lies with the Solver, but does that mean that the solver needs to also implement it's own observer to make sure that "unallowable" active components aren't added between solves? That makes me think that maybe the responsibility needs to lie in this observer...

[michaelbynum]

Okay, should be good to go, @jsiirola (assuming tests pass).

[michaelbynum]

Okay, I did a fairly substantial refactor of this. The ModelChangeDetector now also tells the Observer the reason for the change instead of just the fact that something changed. For example, if a variable was modified, there is an enum to specify what part of the variable changed (e.g., bounds, domain, etc.). This refactor also makes some improvements to the logic so that we don't remove and add a modified constraint (in the past, that could potentially also lead to removing and re-adding variables unnecessarily).

I'm still working on adding more tests, but I think this should be ready for review.

[michaelbynum]

Add a test for nested Expressions (done)

[michaelbynum]

I think I am finally done with this PR.