Getting Cannot execute /home/user/.poetry/bin/poetry error when configuring the poetry environment

[michaelwiles]

Describe the bug
When I setup my poetry environment it correctly auto detects the poetry executable but it says it is not executable. The file is there and it is "executable".

To Reproduce
Steps to reproduce the behavior:

1. Add environment
2. Select poetry environment
3. set correct poetry executable
4. See error

Expected behavior
That the poetry executable gets correctly set.

Environments (please complete the following information):

• IDE: PyCharm Community 2021.1 EAP (same issue with 2020.3)
• OS: ubuntu 20.10
• Poetry Version [e.g. 1.0.8 ]
• Plugin version 1.1.5

Additional context
This is the relevant method AFAICT

fun validatePoetryExecutable(poetryExecutable: @SystemDependent String?): ValidationInfo? {
    val message = if (poetryExecutable.isNullOrBlank()) {
        "Poetry executable is not found"
    }
    else {
        val file = File(poetryExecutable)
        when {
            !file.exists() -> "File ${file.absolutePath} is not found"
            !file.canExecute() || !file.isFile -> "Cannot execute ${file.absolutePath}"
            else -> null
        }
    }

    return message?.let { ValidationInfo(it) }
}

[koxudaxi]

@michaelwiles
Thank you for creating this issue.
I will investigate the problem.

[michaelwiles]

@koxudaxi I know its a little off topic but...

When I run with ./gradlew clean runIde it works. I can use the plugin successfully.

But when I do a ./gradlew clean buildPlugin after having commented out the code that causes this problem and then go to pycharm and manually install from file system the "old code" is still present - install from build/distribution.

the other question is how do I log so that I can do trouble shooting?

Currently I've removed the rogue code and am running pycharm and running with the first option so I can work. Thank you for this plugin.

[koxudaxi]

@michaelwiles

But when I do a ./gradlew clean buildPlugin after having commented out the code that causes this problem and then go to pycharm and manually install from file system the "old code" is still present - install from build/distribution.

I don't know why the behavior. I often run ./grealdew clean without buildPlugin. And delete the build directory 😅

How did you install poetry?
And, Would you please tell me the permission on the executable?

[koxudaxi]

I tested the plugin on macOS and Windows10.
I may test it on Ubuntu or other distros.

Also, I guess that the condition is valid.
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/File.html#canExecute()
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/File.html#isFile()

[koxudaxi]

@michaelwiles
Also, This code came from the pipenv integration of PyCharm.
https://github.com/JetBrains/intellij-community/blob/1e7fb15e147145e9ac0526f0bfdb7f028c11fe96/python/src/com/jetbrains/python/sdk/pipenv/pipenv.kt#L124-L135

fun validatePipEnvExecutable(pipEnvExecutable: @SystemDependent String?): ValidationInfo? {
  val message = if (pipEnvExecutable.isNullOrBlank()) {
    PyBundle.message("python.sdk.pipenv.executable.not.found")
  }
  else {
    val file = File(pipEnvExecutable)
    when {
      !file.exists() -> PyBundle.message("python.sdk.file.not.found", file.absolutePath)
      !file.canExecute() || !file.isFile -> PyBundle.message("python.sdk.cannot.execute", file.absolutePath)
      else -> null
    }
  }

[ric2b]

I'm also getting this error.

• Ubuntu 20.10
• Poetry 1.1.5, installed poetry with curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - as recommended in the official docs
• pyenv global is 3.9.0
• PyCharm is version 2020.3.3, installed via snap with classic confinement

I can run /home/ricardo/.poetry/bin/poetry just fine from a shell:

[koxudaxi]

Thank you for giving me the detail.
I'm installing Ubuntu on my Virtualbox.
But, I can't reproduce the problem.
I will continue working on resolving the problem tomorrow.

[koxudaxi]

I might have better ignore the check that the file is executable on Linux 🤔

[erols]

Hi, same problem here:

• Ubuntu 20.04
  

• Poetry 1.1.5, installed poetry with curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - as recommended in the official docs
  

• pyenv local is 3.9.2, system is 3.8.5
  

• PyCharm professional is version 2020.3.4, installed via snap with classic confinement
  

[koxudaxi]

@erols
Thank you for the version information.
I'm installing Ubuntu 20.10 into VirtualBox. (I installed Ubuntu 20.04 last week)
I don't use snap to install PyCharm. I will try it.

[erols]

Corrected Pycharm version to 2020.3.4

[koxudaxi]

Great!! I just reproduced the problem in Ubuntu 20.10.
The problem doesn't happen in Ubuntu 20.04 🤔
I will check the reason tomorrow.

[koxudaxi]

I just understood why the plugin show Cannot execute .../.poetry/bin/poetry
The plugin executes poetry --version to check executable poetry.
But, The execution is used the system python interpreter. it means pyenv is not used.

Also, I can't run poetry with a system Python interpreter.

Would you please test this command?

$ /usr/bin/python3 `which poetry`

[koxudaxi]

I guess Poetry should be called by an interpreter which is what you used when installing poetry.
Otherwise, the interpreter may not execute poetry.
Unfortunately, the system interpreter of Ubuntu 20.10 can't execute poetry without additional installing.

pyenv is set in shell settings like .bashrc
But, the plugin creates a process to execute poetry without shell settings.
I'm thinking about how do we execute the poetry. 🤔

[erols]

I just understood why the plugin show Cannot execute .../.poetry/bin/poetry
The plugin executes poetry --version to check executable poetry.
But, The execution is used the system python interpreter. it means pyenv is not used.

Also, I can't run poetry with a system Python interpreter.

Would you please test this command?

$ /usr/bin/python3 `which poetry`

Hi,

I ran this command and got this error:

rols@erols-tux:~$ /usr/bin/python3 `which poetry`
Traceback (most recent call last):
  File "/home/erols/.poetry/bin/poetry", line 17, in <module>
    from poetry.console import main
  File "/home/erols/.poetry/lib/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "/home/erols/.poetry/lib/poetry/console/application.py", line 7, in <module>
    from .commands.about import AboutCommand
  File "/home/erols/.poetry/lib/poetry/console/commands/__init__.py", line 4, in <module>
    from .check import CheckCommand
  File "/home/erols/.poetry/lib/poetry/console/commands/check.py", line 2, in <module>
    from poetry.factory import Factory
  File "/home/erols/.poetry/lib/poetry/factory.py", line 18, in <module>
    from .repositories.pypi_repository import PyPiRepository
  File "/home/erols/.poetry/lib/poetry/repositories/pypi_repository.py", line 33, in <module>
    from ..inspection.info import PackageInfo
  File "/home/erols/.poetry/lib/poetry/inspection/info.py", line 25, in <module>
    from poetry.utils.env import EnvCommandError
  File "/home/erols/.poetry/lib/poetry/utils/env.py", line 20, in <module>
    import packaging.tags
  File "/home/erols/.poetry/lib/poetry/_vendor/py3.8/poetry/core/_vendor/packaging/tags.py", line 7, in <module>
    import distutils.util
ModuleNotFoundError: No module named 'distutils.util'
erols@erols-tux:~$ 

[erols]

I guess Poetry should be called by an interpreter which is what you used when installing poetry.
Otherwise, the interpreter may not execute poetry.
Unfortunately, the system interpreter of Ubuntu 20.10 can't execute poetry without additional installing.

pyenv is set in shell settings like .bashrc
But, the plugin creates a process to execute poetry without shell settings.
I'm thinking about how do we execute the poetry. thinking

Actually I can't run poetry with system python on Ubuntu 20.04. I get this error:

pyenv: python: command not found

The `python' command exists in these Python versions:
  3.9.2

Note: See 'pyenv help global' for tips on allowing both
      python2 and python3 to be found.

[koxudaxi]

@erols
Thank you for your report!!

Use System Python to execute Poetry

If you don't use Python2.7 then you can make a symbolic link for python
I found a good package https://packages.ubuntu.com/en/groovy/python-is-python3
Also, you need a pip package to execute poetry with the system's Python3.
https://packages.ubuntu.com/en/groovy/python3-pip

Use pyenv to execute Poetry

I don't know the best way.
The plugin doesn't know what shell is used by a user.
In the first place, the plugin can't know users use pyenv mainly. :(

[erols]

Hi! I did the following:

• added a symbolic link from python to python3
• disabled pyenv

Executing python correctly runs the system python 3.8.5
However, executing poetry still fails in PyCharm and in bash

[erols]

Hi. I got it working finally. Steps from fresh ubuntu 20.04:

• If installed, disable pyenv by commenting out pyenv init in .bashrc
• Start new shell
• install pip3 sudo apt install python3-pip
• pip3 install --upgrade keyrings.alt -> This is the fix
• Now install poetry curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3 -
• edit first line of .poetry/bin/poetry. Change shebang to reference python3 #!/usr/bin/env python3
• Now poetry works in the command line
• Re-enable pyenv
• Check in Pycharm. Using poetry in Add Interpreter now working

[michaelwiles]

Thank you @koxudaxi - very helpful.

So in a nutshell poetry must be executable as a "system" command - without the local shell being fully hydrated - for this to happen, python must be executable as a "system" command.

For me this means adjusting my poetry script to actively reference the system python shell - which is python3 in my case. I want to always control the local python executable so I don't want python to resolve on the system. Or if poetry does not run via the system python this location can be adjusted to point to the python executable location that does have poetry correctly installed (could be a pyenv location).

And this can be done by creating a new poetry executable crafted for pycharm with the relevant python shell specified on the first line. So my first line of this new poetry executable is #!/usr/bin/env python3

I think what might be helpful is to differentiate between the poetry file not being executable and the poetry file executing but failing to return successfully from 'poetry --version'. In this regard if we can log the output of this call we could send the user to the log messages in pycharm maybe for them to troubleshoot.

You want poetry to run in a small a shell as possible and requiring it to have a virtual environment causes a bit of a chicken and egg problem as poetry is a virtual environment tool...

So in other words, I don't think the plugin can fix this issue. All I think it needs to do is provide better feedback to the user to indicate that although it found the poetry executable it failed to execute it and it needs a working poetry executable in order to run.

[erols]

Hi Michael,

I missed that I had to edit .poetry/bin/poetry. I have updated my steps.

I agree that this cannot be fixed in the Pycharm plugin. Unfortunately the poetry team should make this work out of the box on latest ubuntu flavours, or Ubuntu should come with a python executable and pip installed.

[koxudaxi]

@erols @michaelwiles
Thank you for your comments.
I agree.

So in other words, I don't think the plugin can fix this issue. All I think it needs to do is provide better feedback to the user to indicate that although it found the poetry executable it failed to execute it and it needs a working poetry executable in order to run.

Users use flavors of different Python interpreters for each user.
The plugin can show the error message of the poetry command.
But, the message may not indicate the problem's direct reason to the user.
I could know the reason by your feedbacks.

Hi, @abn
Would you tell me what your advice on this problem?
We have problems poetry doesn't run with an interpreter of Ubuntu20.10. But, some users usually use pyenv for poetry.
PyCharm plugin can't detect whether users usually use pyenv for poetry. Because the plugin doesn't read shell settings.
Unfortunately, Poetry can't run on the system's python interpreter of Ubuntu20.10. We need to install few packages into a system .#196 (comment)

How should we solve the problem?
Thank you.