Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pylint update to 3.0.0 #5959

Merged
merged 3 commits into from
Jul 24, 2024
Merged

Conversation

richtja
Copy link
Contributor

@richtja richtja commented Jun 17, 2024

This will update the pylint to version 3.0.0 without enabling any new checks. This is needed because, the pylint 2.17.2 is not compatible with python3.12

Reference: #5957

Copy link
Contributor

@clebergnu clebergnu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the version bump, the spell check fails:

./selftests/spell.sh 
** Running spell check...
No files to lint: exiting.

@richtja
Copy link
Contributor Author

richtja commented Jun 25, 2024

With the version bump, the spell check fails:

./selftests/spell.sh 
** Running spell check...
No files to lint: exiting.

Thanks @clebergnu, I will investigate it. I am wondering why it wasn't caught on our CI. See https://github.com/avocado-framework/avocado/actions/runs/9549221339/job/26353906359?pr=5959#step:8:16

@clebergnu
Copy link
Contributor

With the version bump, the spell check fails:

./selftests/spell.sh 
** Running spell check...
No files to lint: exiting.

Thanks @clebergnu, I will investigate it. I am wondering why it wasn't caught on our CI. See https://github.com/avocado-framework/avocado/actions/runs/9549221339/job/26353906359?pr=5959#step:8:16

I looked into this, and I found that pylint-3 uses the version from the system installation:

# pylint-3 --version
pylint 2.17.4
astroid 2.15.4
Python 3.11.8 (main, Feb 28 2024, 00:00:00) [GCC 13.2.1 20231011 (Red Hat 13.2.1-4)]
# rpm -q python3-pylint
python3-pylint-2.17.4-2.fc38.noarch

As opposed to this which uses the "right" version:

# pylint --version
pylint 3.0.0
astroid 3.0.3
Python 3.11.8 (main, Feb 28 2024, 00:00:00) [GCC 13.2.1 20231011 (Red Hat 13.2.1-4)]
# python3 -m pylint --version
pylint 3.0.0
astroid 3.0.3
Python 3.11.8 (main, Feb 28 2024, 00:00:00) [GCC 13.2.1 20231011 (Red Hat 13.2.1-4)]

IMO, pylint-3 is not as standard as python3 -m pylint or simply pylint and we should switch to use that instead.

@PaulYuuu
Copy link

Hello @clebergnu @richtja, I debugged a bit, pylint-3.0 introduced this commit: pylint-dev/pylint@d203de7

Spelling only check C0401 C0402 C0403, but --errors-only means pylint E rules, so the current command line matches len(linter.config.disable) == len(linter.msgs_store._messages_definitions. After removed --errors-only, I can trigger pylint, but some words need to include to the spell dict.

# python3 -m pylint -j 1 --disable=all --enable=spelling --spelling-dict=en_US --spelling-private-dict-file=spell.ignore *
************* Module avocado.core.output
avocado/core/output.py:432:0: C0402: Wrong spelling of a word 'legger' in a docstring:
Separates logger names and legger levels.
                           ^^^^^^
Did you mean: ''leggier' or 'logger' or 'legged' or 'ledger''? (wrong-spelling-in-docstring)
************* Module avocado.core.resolver
avocado/core/resolver.py:220:0: C0401: Wrong spelling of a word 'bitwise' in a comment:
# If the bitwise AND of user permissions and required permissions is not equal to required permissions,
         ^^^^^^^
Did you mean: ''bit wise' or 'bit-wise' or 'leastwise''? (wrong-spelling-in-comment)
************* Module avocado.core.dependencies.dependency
avocado/core/dependencies/dependency.py:13:0: C0401: Wrong spelling of a word 'jarcihte' in a comment:
# Authors: Jan Richter <[email protected]>
                        ^^^^^^^^
Did you mean: ''architect' or 'jarichte''? (wrong-spelling-in-comment)
************* Module avocado.core.nrunner.runnable
avocado/core/nrunner/runnable.py:32:0: C0401: Wrong spelling of a word 'schemas' in a comment:
#: Location used for schemas when packaged (as in RPMs)
                     ^^^^^^^
Did you mean: ''schema' or 'sachems' or 'schemes' or 'schema s''? (wrong-spelling-in-comment)
************* Module avocado.core.task.runtime
avocado/core/task/runtime.py:129:0: C0401: Wrong spelling of a word 'enum' in a comment:
#: The task status, a value from the enum
                                     ^^^^
Did you mean: ''menu' or 'en um' or 'en-um' or 'plenum''? (wrong-spelling-in-comment)
************* Module avocado.plugins.spawners.lxc
avocado/plugins/spawners/lxc.py:85:0: C0402: Wrong spelling of a word 'fn' in a docstring:
    :param fn: function to run with slot reservation
           ^^
Did you mean: ''f' or 'n' or 'fen' or 'fin''? (wrong-spelling-in-docstring)
avocado/plugins/spawners/lxc.py:86:0: C0402: Wrong spelling of a word 'fn' in a docstring:
    :type fn: function
          ^^
Did you mean: ''f' or 'n' or 'fen' or 'fin''? (wrong-spelling-in-docstring)
************* Module avocado.utils.archive
avocado/utils/archive.py:35:0: C0401: Wrong spelling of a word 'zstd' in a comment:
#: The first two bytes that all zstd files start with.  See
                                ^^^^
Did you mean: ''std' or 'z std' or 'str' or 'rtd''? (wrong-spelling-in-comment)
avocado/utils/archive.py:39:0: C0401: Wrong spelling of a word 'zstd' in a comment:
#: A valid zstd archive with "avocado\n" as content.  Created with:
           ^^^^
Did you mean: ''std' or 'z std' or 'str' or 'rtd''? (wrong-spelling-in-comment)
avocado/utils/archive.py:40:0: C0401: Wrong spelling of a word 'zstd' in a comment:
#: echo "avocado" | zstd -c
                    ^^^^
Did you mean: ''std' or 'z std' or 'str' or 'rtd''? (wrong-spelling-in-comment)
avocado/utils/archive.py:113:0: C0402: Wrong spelling of a word 'zstd' in a docstring:
    Checks if file given by path has contents that suggests zstd file
                                                            ^^^^
Did you mean: ''std' or 'z std' or 'str' or 'rtd''? (wrong-spelling-in-docstring)
avocado/utils/archive.py:137:0: C0402: Wrong spelling of a word 'zstd' in a docstring:
    Extracts a zstd compressed file.
               ^^^^
Did you mean: ''std' or 'z std' or 'str' or 'rtd''? (wrong-spelling-in-docstring)
************* Module avocado.utils.cpu
avocado/utils/cpu.py:626:0: C0402: Wrong spelling of a word 'lscpu' in a docstring:
    by executing 'lscpu' command.
                  ^^^^^
Did you mean: ''scapula' or 'scp' or 'cpu''? (wrong-spelling-in-docstring)
************* Module avocado.utils.data_structures
avocado/utils/data_structures.py:18:0: C0401: Wrong spelling of a word 'Maram' in a comment:
#          Maram Srimannarayana Murthy <[email protected]>
           ^^^^^
Did you mean: ''Mara' or 'Marat' or 'Mar am' or 'Mar-am''? (wrong-spelling-in-comment)
avocado/utils/data_structures.py:18:0: C0401: Wrong spelling of a word 'Srimannarayana' in a comment:
#          Maram Srimannarayana Murthy <[email protected]>
                 ^^^^^^^^^^^^^^
Did you mean: ''Vijayanagar''? (wrong-spelling-in-comment)
avocado/utils/data_structures.py:18:0: C0401: Wrong spelling of a word 'Murthy' in a comment:
#          Maram Srimannarayana Murthy <[email protected]>
                                ^^^^^^
Did you mean: ''Murphy' or 'Multi' or 'Meth' or 'Mpath''? (wrong-spelling-in-comment)
avocado/utils/data_structures.py:18:0: C0401: Wrong spelling of a word 'Maram' in a comment:
#          Maram Srimannarayana Murthy <[email protected]>
                                        ^^^^^
Did you mean: ''Mara' or 'Marat' or 'Mar am' or 'Mar-am''? (wrong-spelling-in-comment)
avocado/utils/data_structures.py:18:0: C0401: Wrong spelling of a word 'Srimannarayana' in a comment:
#          Maram Srimannarayana Murthy <[email protected]>
                                              ^^^^^^^^^^^^^^
Did you mean: ''Vijayanagar''? (wrong-spelling-in-comment)
avocado/utils/data_structures.py:18:0: C0401: Wrong spelling of a word 'Murthy' in a comment:
#          Maram Srimannarayana Murthy <[email protected]>
                                                             ^^^^^^
Did you mean: ''Murphy' or 'Multi' or 'Meth' or 'Mpath''? (wrong-spelling-in-comment)
************* Module avocado.utils.disk
avocado/utils/disk.py:333:0: C0402: Wrong spelling of a word 'sfdisk' in a docstring:
    Creates partitions using sfdisk command
                             ^^^^^^
Did you mean: ''sf disk' or 'sf-disk' or 'disk' or 'stdin''? (wrong-spelling-in-docstring)
avocado/utils/disk.py:420:0: C0402: Wrong spelling of a word 'Rescans' in a docstring:
    Rescans disk
    ^^^^^^^
Did you mean: ''Re scans' or 'Re-scans' or 'Res cans' or 'Res-cans''? (wrong-spelling-in-docstring)
************* Module avocado.utils.git
avocado/utils/git.py:51:0: C0402: Wrong spelling of a word 'submodule' in a docstring:
        :type submodule: Boolean
              ^^^^^^^^^
Did you mean: ''sub module' or 'sub-module' or 'module' or 'submodules''? (wrong-spelling-in-docstring)
avocado/utils/git.py:52:0: C0402: Wrong spelling of a word 'submodule' in a docstring:
        :param submodule: to download submodules recursively
               ^^^^^^^^^
Did you mean: ''sub module' or 'sub-module' or 'module' or 'submodules''? (wrong-spelling-in-docstring)
avocado/utils/git.py:226:0: C0402: Wrong spelling of a word 'submodule' in a docstring:
    :type submodule: Boolean
          ^^^^^^^^^
Did you mean: ''sub module' or 'sub-module' or 'module' or 'submodules''? (wrong-spelling-in-docstring)
avocado/utils/git.py:227:0: C0402: Wrong spelling of a word 'submodule' in a docstring:
    :param submodule: to download submodules recursively
           ^^^^^^^^^
Did you mean: ''sub module' or 'sub-module' or 'module' or 'submodules''? (wrong-spelling-in-docstring)
************* Module avocado.utils.multipath
avocado/utils/multipath.py:105:0: C0402: Wrong spelling of a word 'mpathx' in a docstring:
    :return: mpath name like mpathx
                             ^^^^^^
Did you mean: ''empathy' or 'mpath' or 'mpathb' or 'mpatha''? (wrong-spelling-in-docstring)
avocado/utils/multipath.py:238:0: C0402: Wrong spelling of a word 'friedly' in a docstring:
    :param wwid: wwid or user friedly name of mpath.
                              ^^^^^^^
Did you mean: ''friendly' or 'frenziedly''? (wrong-spelling-in-docstring)
************* Module avocado.utils.nvme
avocado/utils/nvme.py:105:0: C0402: Wrong spelling of a word 'rtyp' in a docstring:
    :rtyp: list
     ^^^^
Did you mean: ''party' or 'rtype''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:120:0: C0402: Wrong spelling of a word 'rtyp' in a docstring:
    :rtyp: list
     ^^^^
Did you mean: ''party' or 'rtype''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:181:0: C0402: Wrong spelling of a word 'ns' in a docstring:
                            ns existence
                            ^^
Did you mean: ''NS' or 'ms' or 'n' or 's''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:194:0: C0402: Wrong spelling of a word 'LBA' in a docstring:
    Returns LBA of the namespace. If not found, return defaults to 0.
            ^^^
Did you mean: ''LAB' or 'LBS' or 'LB' or 'LA''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:286:0: C0402: Wrong spelling of a word 'foce' in a docstring:
    :param force: if wants to create the namespace foce, then pass force=True
                                                   ^^^^
Did you mean: ''foe' or 'force' or 'face' or 'fore''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:302:0: C0402: Wrong spelling of a word 'sixe' in a docstring:
    :param ns_count: Number of namespaces you want to create with equal sixe
                                                                        ^^^^
Did you mean: ''size' or 'six' or 'sixes' or 'sine''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:303:0: C0402: Wrong spelling of a word 'thans' in a docstring:
                     it should be less thans or eaqual to max ns supported
                                       ^^^^^
Did you mean: ''than' or 'tans' or 'thanes' or 'thanks''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:303:0: C0402: Wrong spelling of a word 'eaqual' in a docstring:
                     it should be less thans or eaqual to max ns supported
                                                ^^^^^^
Did you mean: ''equal''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:303:0: C0402: Wrong spelling of a word 'ns' in a docstring:
                     it should be less thans or eaqual to max ns supported
                                                              ^^
Did you mean: ''NS' or 'ms' or 'n' or 's''? (wrong-spelling-in-docstring)
avocado/utils/nvme.py:334:0: C0402: Wrong spelling of a word 'eaqual' in a docstring:
    creates eaqual n number of namespaces on the specified controller
            ^^^^^^
Did you mean: ''equal''? (wrong-spelling-in-docstring)
************* Module avocado.utils.pci
avocado/utils/pci.py:393:0: C0402: Wrong spelling of a word 'rescan' in a docstring:
    Check if rescan for full_pci_address is successful
             ^^^^^^
Did you mean: ''re scan' or 're-scan' or 'res can' or 'res-can''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:396:0: C0402: Wrong spelling of a word 'rescan' in a docstring:
    :return: whether rescan for full_pci_address is successful
                     ^^^^^^
Did you mean: ''re scan' or 're-scan' or 'res can' or 'res-can''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:408:0: C0402: Wrong spelling of a word 'dom' in a docstring:
    Check if the domain changed successfully to "dom" for "full_pci_address"
                                                 ^^^
Did you mean: ''mod' or 'dim' or 'don' or 'do''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:410:0: C0402: Wrong spelling of a word 'dom' in a docstring:
    :param dom: domain type
           ^^^
Did you mean: ''mod' or 'dim' or 'don' or 'do''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:413:0: C0402: Wrong spelling of a word 'dom' in a docstring:
    :return: whether domain changed successfully to "dom"
                                                     ^^^
Did you mean: ''mod' or 'dim' or 'don' or 'do''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:442:0: C0402: Wrong spelling of a word 'Rescan' in a docstring:
    Rescan the system and check for full_pci_address
    ^^^^^^
Did you mean: ''Re scan' or 'Re-scan' or 'Res can' or 'Res-can''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:454:0: C0402: Wrong spelling of a word 'iommu' in a docstring:
    Return the iommu group of full_pci_address
               ^^^^^
Did you mean: ''communion' or 'io' or 'ibm' or 'mtu''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:457:0: C0402: Wrong spelling of a word 'iommu' in a docstring:
    :return: iommu group of full_pci_address
             ^^^^^
Did you mean: ''communion' or 'io' or 'ibm' or 'mtu''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:470:0: C0402: Wrong spelling of a word 'dom' in a docstring:
    Change the domain of pci device(full_pci_address) to dom
                                                         ^^^
Did you mean: ''mod' or 'dim' or 'don' or 'do''? (wrong-spelling-in-docstring)
avocado/utils/pci.py:472:0: C0402: Wrong spelling of a word 'dom' in a docstring:
    :param dom: domain type
           ^^^
Did you mean: ''mod' or 'dim' or 'don' or 'do''? (wrong-spelling-in-docstring)
************* Module spawner_remote.avocado_spawner_remote.__init__
optional_plugins/spawner_remote/avocado_spawner_remote/__init__.py:48:0: C0402: Wrong spelling of a word 'fn' in a docstring:
    :param fn: function to run with slot reservation
           ^^
Did you mean: ''f' or 'n' or 'fen' or 'fin''? (wrong-spelling-in-docstring)
optional_plugins/spawner_remote/avocado_spawner_remote/__init__.py:49:0: C0402: Wrong spelling of a word 'fn' in a docstring:
    :type fn: function
          ^^
Did you mean: ''f' or 'n' or 'fen' or 'fin''? (wrong-spelling-in-docstring)
************* Module selftests.functional.basic
selftests/functional/basic.py:821:0: C0402: Wrong spelling of a word 'matplotlib' in a docstring:
        :avocado: dependency={"type": "ansible-module", "uri": "pip", "name": "matplotlib"}
                                                                               ^^^^^^^^^^
Did you mean: ''diplomatist''? (wrong-spelling-in-docstring)
selftests/functional/basic.py:867:0: C0402: Wrong spelling of a word 'matplotlib' in a docstring:
        :avocado: dependency={"type": "ansible-module", "uri": "pip", "name": "matplotlib"}
                                                                               ^^^^^^^^^^
Did you mean: ''diplomatist''? (wrong-spelling-in-docstring)

------------------------------------------------------------------
Your code has been rated at 9.98/10 (previous run: 9.98/10, +0.00)

@richtja
Copy link
Contributor Author

richtja commented Jun 26, 2024

Hi @PaulYuuu, thank you for your help. I will update the spellcheck command and the spell dict.

@richtja
Copy link
Contributor Author

richtja commented Jun 26, 2024

Hi @clebergnu and @PaulYuuu, thank you for your reviews. I have force-pushed the fix for spell check. Please have a look.

@clebergnu clebergnu added this to the 107 - Codename TBD milestone Jul 2, 2024
@richtja richtja force-pushed the pylint_update branch 5 times, most recently from 7db9433 to a50d623 Compare July 10, 2024 08:33
Comment on lines 31 to 33
run: |
export PATH=/github/home/.local/bin:$PATH
python3 setup.py test --select=static-checks

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @richtja, as we can ensure pylint will be installed during make requirements-dev, can we modify those shell scripts to use tools directly like isort.sh/style.sh, without which pylint, so we don't need this change I think.

Copy link

@PaulYuuu PaulYuuu Jul 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or use pylint-3, seems it will be installed to /usr/bin/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @PaulYuuu thanks for your help. I removed the pylint-3 because of this comment.

The reason for export is that those scripts fails in CI without any logs. I am not sure why it fails with python3 -m pylint therefore I'm moving this to draft, and I will add more verbosity to it to find out what is going on in CI environment.

@richtja richtja marked this pull request as draft July 10, 2024 12:37
@richtja richtja force-pushed the pylint_update branch 4 times, most recently from e8026f6 to 613518e Compare July 10, 2024 14:14
@richtja richtja marked this pull request as ready for review July 10, 2024 14:30
@richtja richtja requested a review from PaulYuuu July 10, 2024 14:30
@richtja
Copy link
Contributor Author

richtja commented Jul 10, 2024

Hi @clebergnu and @PaulYuuu, I did a couple of tests and at the end I decided to use only python module of pylint as the most stable solution. Please have a look.

Copy link

@PaulYuuu PaulYuuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but could you please rebase to latest master branch? I think there is a new word "secureboot" need to handle.

This will update the pylint to version 3.0.0 without enabling any new
checks. This is needed because, the pylint 2.17.2 is not compatible with
python3.12

Reference: avocado-framework#5957
Signed-off-by: Jan Richter <[email protected]>
Avocado is using pylint-3 instead of default pylint. This is a leftover
from the time when avocado supported python2. Unfortunately, the
pylint-3 has slightly different behaviour than pylint, therefore we need
to update it to have more accurate testing. Lets use python3 -m pylint

Signed-off-by: Jan Richter <[email protected]>
@richtja
Copy link
Contributor Author

richtja commented Jul 10, 2024

LGTM, but could you please rebase to latest master branch? I think there is a new word "secureboot" need to handle.

I will do. Thanks for catching that.

The pylints spelling only check C0401 C0402 C0403, but `--errors-only`
in the avocado spell check command line means pylint E rules only,
therefore the avocado spell check is not working. This commit removes
the `--errors-only` flag and fixes all spelling issues accumulated in
avocado code.

Signed-off-by: Jan Richter <[email protected]>
Copy link
Contributor

@clebergnu clebergnu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@clebergnu clebergnu merged commit 34b427e into avocado-framework:master Jul 24, 2024
57 checks passed
@richtja richtja deleted the pylint_update branch October 9, 2024 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants