-
Notifications
You must be signed in to change notification settings - Fork 577
PyROS Add caching for computed uncertain parameter bounds #3877
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jas-yao
wants to merge
40
commits into
Pyomo:main
Choose a base branch
from
jas-yao:pyros-cache-computed-param-bounds
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 24 commits
Commits
Show all changes
40 commits
Select commit
Hold shift + click to select a range
53bbc30
Move bounds calculation to separate, cached method
jas-yao b6a03aa
Clear any cached bounds at set validation
jas-yao 65e9564
Add tests checking caching.
jas-yao a483ab6
Fix _fbbt_parameter_bounds bound value issues
jas-yao 549d5b8
Run black
jas-yao bb7fca5
Fix test comments
jas-yao a6ef4f5
Merge branch 'main' into pyros-cache-computed-param-bounds
jas-yao 3a955f6
Update _solve_bounds_optimization docstring
jas-yao d003145
Merge branch 'main' into pyros-cache-computed-param-bounds
jsiirola c573512
Merge branch 'main' into pyros-cache-computed-param-bounds
jas-yao f0e86b6
Update bounds optimization caching
jas-yao 2af61c8
Add test_solve_exact_bounds_optimization
jas-yao bb53c9d
Add test_fbbt_values
jas-yao be21c04
Move cache clearing to before/after solving
jas-yao 7619f3e
Add tests for PyROS caching
jas-yao 81a98ab
Run black
jas-yao 4f78486
Run updated black
jas-yao 581a9f7
Update CHANGELOG
jas-yao 7aeeef5
Merge branch 'main' into pyros-cache-computed-param-bounds
jsiirola 6de83b7
Merge branch 'main' into pyros-cache-computed-param-bounds
jas-yao 31e96fd
Update caching with custom dict
jas-yao 43f09d8
Simplify cache clearing setup
jas-yao ee87245
Update caching tests
jas-yao c1145cd
Run black
jas-yao d905ab4
Merge branch 'main' into pyros-cache-computed-param-bounds
jas-yao 5518ee8
Use `var.lb` and `var.ub` for numerical bounds
jas-yao d954aba
Add assertion check for empty _cache
jas-yao c242b6a
Update caching tests for assertion error test
jas-yao e0863b0
Run black
jas-yao 4ebca37
Merge branch 'main' into pyros-cache-computed-param-bounds
jsiirola 6259595
Merge branch 'main' into pyros-cache-computed-param-bounds
jas-yao 43079be
Apply suggestion from @shermanjasonaf
jas-yao 32b278b
Apply descriptive TestPyROSCacheUncertaintySetBounds name and docstring
jas-yao 15c0218
Update caching unit tests.
jas-yao d12f066
Merge branch 'main' into pyros-cache-computed-param-bounds
jas-yao acecc15
Update CHANGELOG
jas-yao 60d2491
Update uncertainty set caching documentation
jas-yao f55f97c
Run black
jas-yao c152970
Update uncertaintyset cache manager design to support CartesianProduc…
jsiirola 1632ba1
NFC: fix typo
jsiirola File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3899,7 +3899,7 @@ def solve(self, model, *args, **kwargs): | |
|
|
||
| class TestPyROSUnavailableSubsolvers(unittest.TestCase): | ||
| """ | ||
| Check that appropriate exceptionsa are raised if | ||
| Check that appropriate exceptions are raised if | ||
| PyROS is invoked with unavailable subsolvers. | ||
| """ | ||
|
|
||
|
|
@@ -5146,5 +5146,270 @@ def test_discrete_set_subsolver_error_recovery(self, name, sec_con_UB): | |
| ) | ||
|
|
||
|
|
||
| # @SolverFactory.register("slow_solver") | ||
| class SlowSolver: | ||
| """ | ||
| Solver which sleeps for a specified time before solving. | ||
| """ | ||
|
|
||
| def __init__(self, sleep_time, sub_solver): | ||
| self.sleep_time = sleep_time | ||
| self.sub_solver = sub_solver | ||
|
|
||
| self.options = Bunch() | ||
|
|
||
| def available(self, exception_flag=True): | ||
| return True | ||
|
|
||
| def license_is_valid(self): | ||
| return True | ||
|
|
||
| def __enter__(self): | ||
| return self | ||
|
|
||
| def __exit__(self, et, ev, tb): | ||
| pass | ||
|
|
||
| def solve(self, model, **kwargs): | ||
| """ | ||
| Sleep, then solve a model. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| model : ConcreteModel | ||
| Model of interest. | ||
|
|
||
| Returns | ||
| ------- | ||
| results : SolverResults | ||
| Solver results. | ||
| """ | ||
|
|
||
| # ensure only one active objective | ||
| active_objs = [ | ||
| obj for obj in model.component_data_objects(Objective, active=True) | ||
| ] | ||
| assert len(active_objs) == 1 | ||
|
|
||
| # sleep for specified time | ||
| time.sleep(self.sleep_time) | ||
|
|
||
| print("I slept. Now, I will solve.") | ||
| # invoke subsolver | ||
| results = self.sub_solver.solve(model, **kwargs) | ||
|
|
||
| return results | ||
|
|
||
|
|
||
| class CustomExactBoundsUncertaintySet(BoxSet): | ||
| """ | ||
| Custom uncertainty set that always solves optimization bounding problems. | ||
|
jas-yao marked this conversation as resolved.
Outdated
|
||
| """ | ||
|
|
||
| def __init__(self, bounds, sleep_time, cache): | ||
| super().__init__(bounds) | ||
| self.sleep_time = sleep_time | ||
| self.cache = cache | ||
|
|
||
| @property | ||
| def parameter_bounds(self): | ||
| """ | ||
| Solve bounding problems to calculate exact parameter bounds. | ||
| """ | ||
| solver = SlowSolver( | ||
| sub_solver=SolverFactory("baron"), sleep_time=self.sleep_time | ||
|
jas-yao marked this conversation as resolved.
Outdated
|
||
| ) | ||
| bounds = self._compute_exact_parameter_bounds(solver=solver) | ||
|
|
||
| if not self.cache: | ||
| self._cache.clear() | ||
|
|
||
| return bounds | ||
|
|
||
|
|
||
| @unittest.skipUnless(ipopt_available, "IPOPT is not available.") | ||
| class TestPyROSCache(unittest.TestCase): | ||
| """ | ||
| Test PyROS cache creation and clearing. | ||
| """ | ||
|
jas-yao marked this conversation as resolved.
Outdated
|
||
|
|
||
| def test_pyros_cache_creation(self): | ||
| """ | ||
| Check that PyROS creates a cache for storing computed exact parameter bounds. | ||
| """ | ||
| m = build_leyffer_two_cons() | ||
|
|
||
| # Define the uncertainty set | ||
| interval = CustomExactBoundsUncertaintySet( | ||
| bounds=[(0.25, 2)], sleep_time=0, cache=True | ||
| ) | ||
|
|
||
| # Instantiate the PyROS solver | ||
| pyros_solver = SolverFactory("pyros") | ||
|
|
||
| # Define subsolvers utilized in the algorithm | ||
| local_subsolver = SolverFactory("ipopt") | ||
| global_subsolver = SolverFactory("ipopt") | ||
|
|
||
| # check cache exists | ||
| self.assertTrue(hasattr(interval, "_cache")) | ||
|
|
||
| # Call the PyROS solver | ||
| results = pyros_solver.solve( | ||
|
jas-yao marked this conversation as resolved.
Outdated
|
||
| model=m, | ||
| first_stage_variables=[m.x1], | ||
| second_stage_variables=[m.x2], | ||
| uncertain_params=[m.u], | ||
| uncertainty_set=interval, | ||
| local_solver=local_subsolver, | ||
| global_solver=global_subsolver, | ||
| options={ | ||
| "objective_focus": ObjectiveType.worst_case, | ||
| "solve_master_globally": True, | ||
| }, | ||
| ) | ||
|
|
||
| # check cache has been cleared after solve | ||
| self.assertTrue(hasattr(interval, "_cache")) | ||
| self.assertEqual( | ||
| interval._cache, {}, msg="Did not clear uncertainty set cache after solve." | ||
| ) | ||
|
|
||
| def test_pyros_cache_time(self): | ||
| """ | ||
| Check that caching improves solve time. | ||
|
jas-yao marked this conversation as resolved.
Outdated
|
||
| """ | ||
| m = build_leyffer_two_cons() | ||
|
|
||
| # Define the uncertainty set | ||
| interval_cache = CustomExactBoundsUncertaintySet( | ||
| bounds=[(0.25, 2)], sleep_time=0.1, cache=True | ||
| ) | ||
| interval_no_cache = CustomExactBoundsUncertaintySet( | ||
| bounds=[(0.25, 2)], sleep_time=0.1, cache=False | ||
| ) | ||
|
|
||
| # Instantiate the PyROS solver | ||
| pyros_solver = SolverFactory("pyros") | ||
|
|
||
| # Define subsolvers utilized in the algorithm | ||
| local_subsolver = SolverFactory("ipopt") | ||
| global_subsolver = SolverFactory("ipopt") | ||
|
|
||
| # Call the PyROS solver | ||
| results_cache = pyros_solver.solve( | ||
| model=m, | ||
| first_stage_variables=[m.x1], | ||
| second_stage_variables=[m.x2], | ||
| uncertain_params=[m.u], | ||
| uncertainty_set=interval_cache, | ||
| local_solver=local_subsolver, | ||
| global_solver=global_subsolver, | ||
| options={ | ||
| "objective_focus": ObjectiveType.worst_case, | ||
| "solve_master_globally": True, | ||
| }, | ||
| ) | ||
|
|
||
| results_no_cache = pyros_solver.solve( | ||
| model=m, | ||
| first_stage_variables=[m.x1], | ||
| second_stage_variables=[m.x2], | ||
| uncertain_params=[m.u], | ||
| uncertainty_set=interval_no_cache, | ||
| local_solver=local_subsolver, | ||
| global_solver=global_subsolver, | ||
| options={ | ||
| "objective_focus": ObjectiveType.worst_case, | ||
| "solve_master_globally": True, | ||
| }, | ||
| ) | ||
|
|
||
| # caching should always result in less time, | ||
| # as not caching reruns the slow solver multiple times | ||
| self.assertGreater(results_no_cache.time, results_cache.time) | ||
|
|
||
| def test_pyros_cache_solutions(self): | ||
| """ | ||
| Check that PyROS clears cache before/after and yields accurate results. | ||
| """ | ||
|
jas-yao marked this conversation as resolved.
Outdated
|
||
| m = build_leyffer_two_cons() | ||
|
|
||
| # Define the uncertainty set | ||
| interval = CustomExactBoundsUncertaintySet( | ||
| bounds=[(25, 200)], sleep_time=0, cache=True | ||
| ) | ||
| self.assertEqual(interval.parameter_bounds, [(25, 200)]) | ||
|
|
||
| # change set attributes, leading to outdated parameter bounds | ||
| interval.bounds = [(0.25, 2)] | ||
| self.assertEqual(interval.parameter_bounds, [(25, 200)]) | ||
|
|
||
| # Instantiate the PyROS solver | ||
| pyros_solver = SolverFactory("pyros") | ||
|
|
||
| # Define subsolvers utilized in the algorithm | ||
| local_subsolver = SolverFactory("ipopt") | ||
| global_subsolver = SolverFactory("ipopt") | ||
|
|
||
| # Solve with PyROS | ||
| results = pyros_solver.solve( | ||
| model=m, | ||
| first_stage_variables=[m.x1], | ||
| second_stage_variables=[m.x2], | ||
| uncertain_params=[m.u], | ||
| uncertainty_set=interval, | ||
| local_solver=local_subsolver, | ||
| global_solver=global_subsolver, | ||
| options={ | ||
| "objective_focus": ObjectiveType.worst_case, | ||
| "solve_master_globally": True, | ||
| }, | ||
| ) | ||
|
|
||
| # check results, which should use the correct parameter bounds | ||
| self.assertEqual(results.iterations, 3) | ||
| self.assertAlmostEqual(results.final_objective_value, 0.531, places=2) | ||
| self.assertAlmostEqual(m.x1.value, 3.518, places=2) | ||
| self.assertAlmostEqual(m.x2.value, 1.547, places=2) | ||
| self.assertAlmostEqual(m.x3.value, 9.684, places=2) | ||
| self.assertEqual( | ||
| results.pyros_termination_condition, | ||
| pyrosTerminationCondition.robust_optimal, | ||
| ) | ||
|
|
||
|
jas-yao marked this conversation as resolved.
|
||
| # modify the cache | ||
| interval._cache[0, minimize] = 25 | ||
| interval._cache[0, maximize] = 200 | ||
|
|
||
| self.assertEqual(interval.parameter_bounds, [(25, 200)]) | ||
|
|
||
| # Solve with PyROS | ||
| results = pyros_solver.solve( | ||
| model=m, | ||
| first_stage_variables=[m.x1], | ||
| second_stage_variables=[m.x2], | ||
| uncertain_params=[m.u], | ||
| uncertainty_set=interval, | ||
| local_solver=local_subsolver, | ||
| global_solver=global_subsolver, | ||
| options={ | ||
| "objective_focus": ObjectiveType.worst_case, | ||
| "solve_master_globally": True, | ||
| }, | ||
| ) | ||
|
|
||
| # check results, which should not change | ||
| self.assertEqual(results.iterations, 3) | ||
| self.assertAlmostEqual(results.final_objective_value, 0.531, places=2) | ||
| self.assertAlmostEqual(m.x1.value, 3.518, places=2) | ||
| self.assertAlmostEqual(m.x2.value, 1.547, places=2) | ||
| self.assertAlmostEqual(m.x3.value, 9.684, places=2) | ||
| self.assertEqual( | ||
| results.pyros_termination_condition, | ||
| pyrosTerminationCondition.robust_optimal, | ||
| ) | ||
|
|
||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would add here a test based on a problem with a diff --git a/pyomo/contrib/pyros/tests/test_grcs.py b/pyomo/contrib/pyros/tests/test_grcs.py
index 410593c82..8f577d879 100644
--- a/pyomo/contrib/pyros/tests/test_grcs.py
+++ b/pyomo/contrib/pyros/tests/test_grcs.py
@@ -75,6 +75,7 @@ from pyomo.contrib.pyros.uncertainty_sets import (
FactorModelSet,
Geometry,
IntersectionSet,
+ PolyhedralSet,
UncertaintyQuantification,
UncertaintySet,
)
@@ -5468,6 +5469,71 @@ class TestPyROSCacheUncertaintySetBounds(unittest.TestCase):
self.assertAlmostEqual(interval._cache[0, minimize], 25, places=2)
self.assertAlmostEqual(interval._cache[0, maximize], 200, places=2)
+ def test_solve_cartesian_product_set_bounds_cache(self):
+ """
+ Test management of uncertainty set bounds caches
+ is carried out as expected in the context of a PyROS solve.
+ """
+ # deterministic model
+ m = ConcreteModel()
+ m.q = Param(range(4), initialize=0, mutable=True)
+ m.x = Var(initialize=0, bounds=(0, 100))
+ m.obj = Objective(expr=m.x, sense=minimize)
+ m.con = Constraint(expr=m.x >= sum(m.q.values()))
+
+ # uncertainty set(s)
+ poly_set = PolyhedralSet(
+ # this is just the cube [-1, 1]^3
+ lhs_coefficients_mat=np.vstack([np.eye(3), -np.eye(3)]),
+ rhs_vec=[1] * 6,
+ )
+ # inclusion of PolyhedralSet means bounds caching takes place
+ cpset = CartesianProductSet([BoxSet([[0, 1]]), poly_set])
+ res1 = SolverFactory("pyros").solve(
+ model=m,
+ first_stage_variables=m.x,
+ second_stage_variables=[],
+ uncertain_params=m.q,
+ uncertainty_set=cpset,
+ local_solver=SolverFactory("ipopt"),
+ global_solver=SolverFactory("ipopt"),
+ objective_focus="worst_case",
+ solve_master_globally=True,
+ )
+
+ # check caches cleared
+ self.assertEqual(cpset._cache, {})
+ self.assertEqual(cpset._all_sets[0]._cache, {})
+ self.assertEqual(cpset._all_sets[1]._cache, {})
+ # check results: worst case objective is just sum of the
+ # uncertainty set upper bounds
+ self.assertEqual(res1.iterations, 2)
+ self.assertAlmostEqual(res1.final_objective_value, 4.0, places=6)
+ self.assertAlmostEqual(m.x.value, 4.0, places=6)
+
+ # expand the polyhedralset to the cube [-2, 2]^3
+ poly_set.rhs_vec = [2] * 6
+ # solve again. since caches cleared, PyROS should work normally
+ res2 = SolverFactory("pyros").solve(
+ model=m,
+ first_stage_variables=m.x,
+ second_stage_variables=[],
+ uncertain_params=m.q,
+ uncertainty_set=cpset,
+ local_solver=SolverFactory("ipopt"),
+ global_solver=SolverFactory("ipopt"),
+ objective_focus="worst_case",
+ solve_master_globally=True,
+ )
+ # check caches cleared
+ self.assertEqual(cpset._cache, {})
+ self.assertEqual(cpset._all_sets[0]._cache, {})
+ self.assertEqual(cpset._all_sets[1]._cache, {})
+ # results have changed, since the polyhedral set was expanded
+ self.assertEqual(res2.iterations, 2)
+ self.assertAlmostEqual(res2.final_objective_value, 7.0, places=6)
+ self.assertAlmostEqual(m.x.value, 7.0, places=6)
+
if __name__ == "__main__":
unittest.main() |
||
|
|
||
| if __name__ == "__main__": | ||
| unittest.main() | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After PR #3951 is either merged or closed, you may update the PyROS version number (in
pyros.py) and further update the changelog here according to this comment. In particular, if #3951 is closed rather than merged, then the final version number here will be 1.3.14.