Running tests modifies .pylintrc and causes subsequent failures

[carlosgmartin]

Running tests modifies the file .pylintrc, by appending the line

+disable=unnecessary-lambda-assignment,no-value-for-parameter,use-dict-literal

Running tests a second time yields numerous failures. An example of this is shown below:

(venv) $ JAX_VERSION= sh ./test.sh
+ rm -rf _testing
+ rm -rf dist/
+ rm -rf '*.whl'
+ rm -rf .pytype
+ mkdir -p _testing
+++ pwd
++ mktemp -d /Users/carlos/desktop/optax/_testing/optax-env.XXXXXXXX
+ readonly VENV_DIR=/Users/carlos/desktop/optax/_testing/optax-env.OHDqCZTC
+ VENV_DIR=/Users/carlos/desktop/optax/_testing/optax-env.OHDqCZTC
+ python3 -m venv /Users/carlos/desktop/optax/_testing/optax-env.OHDqCZTC
+ source /Users/carlos/desktop/optax/_testing/optax-env.OHDqCZTC/bin/activate
++ deactivate nondestructive
++ '[' -n '' ']'
++ '[' -n '' ']'
++ hash -r
++ '[' -n '' ']'
++ unset VIRTUAL_ENV
++ unset VIRTUAL_ENV_PROMPT
++ '[' '!' nondestructive = nondestructive ']'
++ '[' darwin23 = cygwin ']'
++ '[' darwin23 = msys ']'
++ export VIRTUAL_ENV=/Users/carlos/desktop/optax/_testing/optax-env.OHDqCZTC
++ VIRTUAL_ENV=/Users/carlos/desktop/optax/_testing/optax-env.OHDqCZTC
++ _OLD_VIRTUAL_PATH=/Users/carlos/venv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
++ PATH=/Users/carlos/desktop/optax/_testing/optax-env.OHDqCZTC/bin:/Users/carlos/venv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
++ export PATH
++ '[' -n '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ PS1='(optax-env.OHDqCZTC) '
++ export PS1
++ VIRTUAL_ENV_PROMPT='(optax-env.OHDqCZTC) '
++ export VIRTUAL_ENV_PROMPT
++ hash -r
+ python --version
Python 3.12.3
+ pip install -q --upgrade pip setuptools wheel
+ pip install -q flake8 pytest-xdist pylint pylint-exit
+ pip install -q -e '.[test, examples]'
+ pip install -q -e '.[dp-accounting]'
+ pip install -q 'dp-accounting>=0.1.1' --no-deps
+ '[' '' = '' ']'
+ :
+ pip uninstall -q -y optax
++ find optax examples -name '*.py'
++ xargs
+ flake8 optax/second_order/_base.py optax/second_order/__init__.py optax/second_order/_hessian.py optax/second_order/_fisher.py optax/second_order/_hessian_test.py optax/losses/_classification.py optax/losses/_smoothing_test.py optax/losses/_regression.py optax/losses/_ranking_test.py optax/losses/__init__.py optax/losses/_self_supervised.py optax/losses/_fenchel_young_test.py optax/losses/_regression_test.py optax/losses/_classification_test.py optax/losses/_ranking.py optax/losses/_fenchel_young.py optax/losses/_self_supervised_test.py optax/losses/_smoothing.py optax/_src/linear_algebra.py optax/_src/update.py optax/_src/base_test.py optax/_src/linesearch.py optax/_src/alias.py optax/_src/numerics_test.py optax/_src/combine.py optax/_src/utils_test.py optax/_src/transform_test.py optax/_src/alias_test.py optax/_src/float64_test.py optax/_src/constrain.py optax/_src/deprecations.py optax/_src/clipping.py optax/_src/linesearch_test.py optax/_src/linear_algebra_test.py optax/_src/utils.py optax/_src/wrappers.py optax/_src/update_test.py optax/_src/transform.py optax/_src/factorized_test.py optax/_src/schedule.py optax/_src/lookahead.py optax/_src/combine_test.py optax/_src/factorized.py optax/_src/base.py optax/_src/numerics.py optax/_src/lookahead_test.py optax/__init__.py optax/schedules/_join_test.py optax/schedules/__init__.py optax/schedules/_inject.py optax/schedules/inject.py optax/schedules/_join.py optax/schedules/_inject_test.py optax/schedules/_schedule_test.py optax/schedules/_schedule.py optax/projections/__init__.py optax/projections/_projections_test.py optax/projections/_projections.py optax/optax_test.py optax/contrib/_mechanic.py optax/contrib/_sam.py optax/contrib/_privacy_test.py optax/contrib/_schedule_free.py optax/contrib/_complex_valued.py optax/contrib/_cocob.py optax/contrib/_schedule_free_test.py optax/contrib/_common_test.py optax/contrib/__init__.py optax/contrib/_dog.py optax/contrib/_reduce_on_plateau_test.py optax/contrib/_prodigy.py optax/contrib/_acprop.py optax/contrib/_complex_valued_test.py optax/contrib/_sam_test.py optax/contrib/_dadapt_adamw.py optax/contrib/_mechanic_test.py optax/contrib/_momo.py optax/contrib/_privacy.py optax/contrib/_reduce_on_plateau.py optax/transforms/_combining_test.py optax/transforms/_conditionality_test.py optax/transforms/_combining.py optax/transforms/_adding_test.py optax/transforms/_masking.py optax/transforms/_constraining.py optax/transforms/_adding.py optax/transforms/__init__.py optax/transforms/_conditionality.py optax/transforms/_clipping_test.py optax/transforms/_clipping.py optax/transforms/_layouts_test.py optax/transforms/_masking_test.py optax/transforms/_layouts.py optax/transforms/_accumulation.py optax/transforms/_constraining_test.py optax/transforms/_accumulation_test.py optax/monte_carlo/control_variates.py optax/monte_carlo/control_variates_test.py optax/monte_carlo/__init__.py optax/monte_carlo/stochastic_gradient_estimators.py optax/monte_carlo/stochastic_gradient_estimators_test.py optax/tree_utils/_random_test.py optax/tree_utils/_casting_test.py optax/tree_utils/_state_utils_test.py optax/tree_utils/_tree_math_test.py optax/tree_utils/__init__.py optax/tree_utils/_tree_math.py optax/tree_utils/_random.py optax/tree_utils/_state_utils.py optax/tree_utils/_casting.py --count --select=E9,F63,F7,F82,E225,E251 --show-source --statistics
0
+ PYLINT_ARGS='-efail -wfail -cfail -rfail'
+ echo disable=unnecessary-lambda-assignment,no-value-for-parameter,use-dict-literal
++ find optax examples -name '*.py'
++ grep -v test.py
++ xargs
+ pylint --rcfile=.pylintrc optax/second_order/_base.py optax/second_order/__init__.py optax/second_order/_hessian.py optax/second_order/_fisher.py optax/losses/_classification.py optax/losses/_regression.py optax/losses/__init__.py optax/losses/_self_supervised.py optax/losses/_ranking.py optax/losses/_fenchel_young.py optax/losses/_smoothing.py optax/_src/linear_algebra.py optax/_src/update.py optax/_src/linesearch.py optax/_src/alias.py optax/_src/combine.py optax/_src/constrain.py optax/_src/deprecations.py optax/_src/clipping.py optax/_src/utils.py optax/_src/wrappers.py optax/_src/transform.py optax/_src/schedule.py optax/_src/lookahead.py optax/_src/factorized.py optax/_src/base.py optax/_src/numerics.py optax/__init__.py optax/schedules/__init__.py optax/schedules/_inject.py optax/schedules/inject.py optax/schedules/_join.py optax/schedules/_schedule.py optax/projections/__init__.py optax/projections/_projections.py optax/contrib/_mechanic.py optax/contrib/_sam.py optax/contrib/_schedule_free.py optax/contrib/_complex_valued.py optax/contrib/_cocob.py optax/contrib/__init__.py optax/contrib/_dog.py optax/contrib/_prodigy.py optax/contrib/_acprop.py optax/contrib/_dadapt_adamw.py optax/contrib/_momo.py optax/contrib/_privacy.py optax/contrib/_reduce_on_plateau.py optax/transforms/_combining.py optax/transforms/_masking.py optax/transforms/_constraining.py optax/transforms/_adding.py optax/transforms/__init__.py optax/transforms/_conditionality.py optax/transforms/_clipping.py optax/transforms/_layouts.py optax/transforms/_accumulation.py optax/monte_carlo/control_variates.py optax/monte_carlo/__init__.py optax/monte_carlo/stochastic_gradient_estimators.py optax/tree_utils/__init__.py optax/tree_utils/_tree_math.py optax/tree_utils/_random.py optax/tree_utils/_state_utils.py optax/tree_utils/_casting.py -d E1102
************* Module optax.transforms._combining
optax/transforms/_combining.py:17:0: W4904: Using deprecated class Hashable of module typing (deprecated-class)

-------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 1.41/10, +8.59)

+ pylint-exit -efail -wfail -cfail -rfail 4
The following messages were raised:

  - warning message issued

Fatal messages detected.  Failing...
(venv) $ git diff
diff --git a/.pylintrc b/.pylintrc
index 5c771e6..b26aeee 100644
--- a/.pylintrc
+++ b/.pylintrc
@@ -397,3 +397,4 @@ valid-classmethod-first-arg=cls,
 
 # List of valid names for the first argument in a metaclass class method.
 valid-metaclass-classmethod-first-arg=mcs
+disable=unnecessary-lambda-assignment,no-value-for-parameter,use-dict-literal
(venv) $ JAX_VERSION= sh ./test.sh
+ rm -rf _testing
+ rm -rf dist/
+ rm -rf '*.whl'
+ rm -rf .pytype
+ mkdir -p _testing
+++ pwd
++ mktemp -d /Users/carlos/desktop/optax/_testing/optax-env.XXXXXXXX
+ readonly VENV_DIR=/Users/carlos/desktop/optax/_testing/optax-env.aQdKRU1K
+ VENV_DIR=/Users/carlos/desktop/optax/_testing/optax-env.aQdKRU1K
+ python3 -m venv /Users/carlos/desktop/optax/_testing/optax-env.aQdKRU1K
+ source /Users/carlos/desktop/optax/_testing/optax-env.aQdKRU1K/bin/activate
++ deactivate nondestructive
++ '[' -n '' ']'
++ '[' -n '' ']'
++ hash -r
++ '[' -n '' ']'
++ unset VIRTUAL_ENV
++ unset VIRTUAL_ENV_PROMPT
++ '[' '!' nondestructive = nondestructive ']'
++ '[' darwin23 = cygwin ']'
++ '[' darwin23 = msys ']'
++ export VIRTUAL_ENV=/Users/carlos/desktop/optax/_testing/optax-env.aQdKRU1K
++ VIRTUAL_ENV=/Users/carlos/desktop/optax/_testing/optax-env.aQdKRU1K
++ _OLD_VIRTUAL_PATH=/Users/carlos/venv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
++ PATH=/Users/carlos/desktop/optax/_testing/optax-env.aQdKRU1K/bin:/Users/carlos/venv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
++ export PATH
++ '[' -n '' ']'
++ '[' -z '' ']'
++ _OLD_VIRTUAL_PS1=
++ PS1='(optax-env.aQdKRU1K) '
++ export PS1
++ VIRTUAL_ENV_PROMPT='(optax-env.aQdKRU1K) '
++ export VIRTUAL_ENV_PROMPT
++ hash -r
+ python --version
Python 3.12.3
+ pip install -q --upgrade pip setuptools wheel
+ pip install -q flake8 pytest-xdist pylint pylint-exit
+ pip install -q -e '.[test, examples]'
+ pip install -q -e '.[dp-accounting]'
+ pip install -q 'dp-accounting>=0.1.1' --no-deps
+ '[' '' = '' ']'
+ :
+ pip uninstall -q -y optax
++ find optax examples -name '*.py'
++ xargs
+ flake8 optax/second_order/_base.py optax/second_order/__init__.py optax/second_order/_hessian.py optax/second_order/_fisher.py optax/second_order/_hessian_test.py optax/losses/_classification.py optax/losses/_smoothing_test.py optax/losses/_regression.py optax/losses/_ranking_test.py optax/losses/__init__.py optax/losses/_self_supervised.py optax/losses/_fenchel_young_test.py optax/losses/_regression_test.py optax/losses/_classification_test.py optax/losses/_ranking.py optax/losses/_fenchel_young.py optax/losses/_self_supervised_test.py optax/losses/_smoothing.py optax/_src/linear_algebra.py optax/_src/update.py optax/_src/base_test.py optax/_src/linesearch.py optax/_src/alias.py optax/_src/numerics_test.py optax/_src/combine.py optax/_src/utils_test.py optax/_src/transform_test.py optax/_src/alias_test.py optax/_src/float64_test.py optax/_src/constrain.py optax/_src/deprecations.py optax/_src/clipping.py optax/_src/linesearch_test.py optax/_src/linear_algebra_test.py optax/_src/utils.py optax/_src/wrappers.py optax/_src/update_test.py optax/_src/transform.py optax/_src/factorized_test.py optax/_src/schedule.py optax/_src/lookahead.py optax/_src/combine_test.py optax/_src/factorized.py optax/_src/base.py optax/_src/numerics.py optax/_src/lookahead_test.py optax/__init__.py optax/schedules/_join_test.py optax/schedules/__init__.py optax/schedules/_inject.py optax/schedules/inject.py optax/schedules/_join.py optax/schedules/_inject_test.py optax/schedules/_schedule_test.py optax/schedules/_schedule.py optax/projections/__init__.py optax/projections/_projections_test.py optax/projections/_projections.py optax/optax_test.py optax/contrib/_mechanic.py optax/contrib/_sam.py optax/contrib/_privacy_test.py optax/contrib/_schedule_free.py optax/contrib/_complex_valued.py optax/contrib/_cocob.py optax/contrib/_schedule_free_test.py optax/contrib/_common_test.py optax/contrib/__init__.py optax/contrib/_dog.py optax/contrib/_reduce_on_plateau_test.py optax/contrib/_prodigy.py optax/contrib/_acprop.py optax/contrib/_complex_valued_test.py optax/contrib/_sam_test.py optax/contrib/_dadapt_adamw.py optax/contrib/_mechanic_test.py optax/contrib/_momo.py optax/contrib/_privacy.py optax/contrib/_reduce_on_plateau.py optax/transforms/_combining_test.py optax/transforms/_conditionality_test.py optax/transforms/_combining.py optax/transforms/_adding_test.py optax/transforms/_masking.py optax/transforms/_constraining.py optax/transforms/_adding.py optax/transforms/__init__.py optax/transforms/_conditionality.py optax/transforms/_clipping_test.py optax/transforms/_clipping.py optax/transforms/_layouts_test.py optax/transforms/_masking_test.py optax/transforms/_layouts.py optax/transforms/_accumulation.py optax/transforms/_constraining_test.py optax/transforms/_accumulation_test.py optax/monte_carlo/control_variates.py optax/monte_carlo/control_variates_test.py optax/monte_carlo/__init__.py optax/monte_carlo/stochastic_gradient_estimators.py optax/monte_carlo/stochastic_gradient_estimators_test.py optax/tree_utils/_random_test.py optax/tree_utils/_casting_test.py optax/tree_utils/_state_utils_test.py optax/tree_utils/_tree_math_test.py optax/tree_utils/__init__.py optax/tree_utils/_tree_math.py optax/tree_utils/_random.py optax/tree_utils/_state_utils.py optax/tree_utils/_casting.py --count --select=E9,F63,F7,F82,E225,E251 --show-source --statistics
0
+ PYLINT_ARGS='-efail -wfail -cfail -rfail'
+ echo disable=unnecessary-lambda-assignment,no-value-for-parameter,use-dict-literal
++ find optax examples -name '*.py'
++ grep -v test.py
++ xargs
+ pylint --rcfile=.pylintrc optax/second_order/_base.py optax/second_order/__init__.py optax/second_order/_hessian.py optax/second_order/_fisher.py optax/losses/_classification.py optax/losses/_regression.py optax/losses/__init__.py optax/losses/_self_supervised.py optax/losses/_ranking.py optax/losses/_fenchel_young.py optax/losses/_smoothing.py optax/_src/linear_algebra.py optax/_src/update.py optax/_src/linesearch.py optax/_src/alias.py optax/_src/combine.py optax/_src/constrain.py optax/_src/deprecations.py optax/_src/clipping.py optax/_src/utils.py optax/_src/wrappers.py optax/_src/transform.py optax/_src/schedule.py optax/_src/lookahead.py optax/_src/factorized.py optax/_src/base.py optax/_src/numerics.py optax/__init__.py optax/schedules/__init__.py optax/schedules/_inject.py optax/schedules/inject.py optax/schedules/_join.py optax/schedules/_schedule.py optax/projections/__init__.py optax/projections/_projections.py optax/contrib/_mechanic.py optax/contrib/_sam.py optax/contrib/_schedule_free.py optax/contrib/_complex_valued.py optax/contrib/_cocob.py optax/contrib/__init__.py optax/contrib/_dog.py optax/contrib/_prodigy.py optax/contrib/_acprop.py optax/contrib/_dadapt_adamw.py optax/contrib/_momo.py optax/contrib/_privacy.py optax/contrib/_reduce_on_plateau.py optax/transforms/_combining.py optax/transforms/_masking.py optax/transforms/_constraining.py optax/transforms/_adding.py optax/transforms/__init__.py optax/transforms/_conditionality.py optax/transforms/_clipping.py optax/transforms/_layouts.py optax/transforms/_accumulation.py optax/monte_carlo/control_variates.py optax/monte_carlo/__init__.py optax/monte_carlo/stochastic_gradient_estimators.py optax/tree_utils/__init__.py optax/tree_utils/_tree_math.py optax/tree_utils/_random.py optax/tree_utils/_state_utils.py optax/tree_utils/_casting.py -d E1102
************* Module .pylintrc
.pylintrc:1:0: F0011: error while parsing the configuration: While reading from '.pylintrc' [line 401]: option 'disable' in section 'CLASSES' already exists (config-parse-error)
************* Module optax.second_order._base
optax/second_order/_base.py:24:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_base.py:26:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_base.py:27:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_base.py:30:0: W0311: Bad indentation. Found 4 spaces, expected 8 (bad-indentation)
optax/second_order/_base.py:23:0: R0903: Too few public methods (1/2) (too-few-public-methods)
************* Module optax.second_order._hessian
optax/second_order/_hessian.py:32:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:42:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:55:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:56:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:57:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:66:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:78:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:79:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:80:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_hessian.py:56:12: C3001: Lambda expression assigned to a variable. Define a function using the "def" keyword instead. (unnecessary-lambda-assignment)
optax/second_order/_hessian.py:79:9: C3001: Lambda expression assigned to a variable. Define a function using the "def" keyword instead. (unnecessary-lambda-assignment)
************* Module optax.second_order._fisher
optax/second_order/_fisher.py:32:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_fisher.py:41:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)
optax/second_order/_fisher.py:54:0: W0311: Bad indentation. Found 2 spaces, expected 4 (bad-indentation)



[a LOT more output along these lines]



------------------------------------------------------------------
Your code has been rated at 1.41/10 (previous run: 1.41/10, +0.00)

+ pylint-exit -efail -wfail -cfail -rfail 31
The following messages were raised:

  - fatal message issued
  - error message issued
  - warning message issued
  - refactor message issued
  - convention message issued

Fatal messages detected.  Failing...
(venv) $ 

The numerous failures appear only after .pylintrc has been modified. Reverting .pylintrc to its original state, and then running tests again, results in the same output as the first run.

It seems to me that running tests should not be modifying any tracked files (which subsequently have to be reverted or manually excluded from commits).

[carlosgmartin]

It looks like this might be caused by the test script terminating due to failures before any cleanup occurs (see #1068).

In any case, the test script should clean up after itself even if failures occur.