Overhaul PyROS Preprocessor Subroutine and Subproblem Objects

[shermanjasonaf]

Fixes #2964.

Summary/Motivation:

Driven by recent algorithmic developments of the PyROS preprocessor, this PR implements sweeping changes to the PyROS preprocessor subroutine and subproblem formulations/object structures.

Changes proposed in this PR:

Update PyROS as follows:

• Overhaul the preprocessor
  • Leverage decision rules and equality constraints to identify second-stage variables and state variables that are mathematically nonadjustable
  • More structured methods for reformulating variable domains/bounds to constraints
  • More structured methods for rearranging inequality constraints
  • Rewrite and extend coefficient matching: carefully account for state variable-independent constraints that cannot be reformulated by casting to inequality constraint pairs
• Update the subproblem formulations and modeling objects
  • Modify block structures of the working model, master-like models, and separation model: partition the constraints by stage
  • Change the objective of the decision rule polishing problem to the 1-norm of the nonstatic terms of the decision rule expressions
  • Perform more careful initialization of the auxiliary uncertain parameter variables of the separation problem
• Update the UncertaintySet class and pre-implemented subclasses to facilitate changes to the subproblems
  • Modify return protocol of UncertaintySet.set_as_constraint to make auxiliary uncertain parameters easier to track
  • Add new efficient methods for calculation of auxiliary uncertain parameter values for the CardinalitySet and FactorModelSet classes
  • Restrict range of valid FactorModelSet instances to those for which psi_mat is full column rank
  • Bug fix(es) for the IntersectionSet
• Logging updates
  • Make log of the model component statistics slightly more informative
  • Add DEBUG-level messages for logging of the individual preprocessor steps
• Documentation updates
  • Update "Methodology Overview" section
  • Add section on PyROS installation instructions
• Make tests more rigorous and extensive
  • Address Restore PyROS failed separation problem test #2964
  • Use expression comparison methods (core.expr.compare.assertExpressionsEqual) for more precise model testing
  • Separate testing modules for the:
    • preprocessor
    • subproblem formulations
    • uncertainty set classes
      • more rigorous testing for point_in_set, set_as_constraint
    • PyROS solver as a whole

TODO

• (After PR Add ParameterizedQuadraticRepn and corresponding expression walker #3324 merged) use ParameterizedQuadraticRepnVisitor in lieu of generate_standard_repn
• Update PyROS version number and changelog

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[jsiirola]

Is this test specific to BARON 24.5.8 or does it need at least 24.5.8? I worry that while that version may not be available in any of the CI systems (I know GHA uses the current release - currently 24.10.10), and therefore will never be run. Would baron_version >= (24, 5, 8) be acceptable?

[shermanjasonaf]

On my machine, this test:

• Times out with BARON > 24.5.8
• Passes with SCIP 8.0.3 and 9.1.0 (but, based on prior discussion, appeared to trigger a time out with the SCIP distribution used in the most recent Jenkins tests)

I am considering replacing the nonlinear equality constraint of the model used in this test with a simpler nonlinear equality constraint, as the new preprocessor (correctly) derives from the current equality two nonconvex inequalities that appear to make the current problem difficult to solve. Thoughts?