Skip to content

Commit

Permalink
Fixing flaky test (#780)
Browse files Browse the repository at this point in the history
Summary:
Running some tests by removing seeds revealed that tests such as `test_single_shapley_int_batch_scalar_float`, `test_single_shapley_int_batch_scalar_tensor_0d`, `test_single_shapley_int_batch_scalar_tensor_1d`, and `test_single_shapley_int_batch_scalar_tensor_int` are flaky. They almost fail 40-50% of the times.

To fix these tests, I updated the `n_samples` to 2500 to fix the flakiness. The tests now pass consistently. This only affects the execution time of the test by ~1s.

Please let me know if this change seems reasonable. Similar to #775.

Pull Request resolved: #780

Reviewed By: NarineK, Reubend

Differential Revision: D31587778

Pulled By: vivekmig

fbshipit-source-id: 843adda41662e881a6bf81ebc4354d6f304e83ce
  • Loading branch information
crawlingcub authored and facebook-github-bot committed Oct 13, 2021
1 parent b6f6286 commit 9b33f13
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions tests/attr/test_shapley.py
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,7 @@ def _single_int_input_multi_sample_batch_scalar_shapley_assert(
feature_mask=mask,
perturbations_per_eval=(1,),
target=None,
n_samples=2500,
)

def _multi_input_batch_scalar_shapley_assert(self, func: Callable) -> None:
Expand Down

0 comments on commit 9b33f13

Please sign in to comment.