Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OSError: could not find or load spatialindex_c-64.dll on Windows #31

Closed
1 task done
martinfleis opened this issue Apr 21, 2021 · 12 comments
Closed
1 task done

OSError: could not find or load spatialindex_c-64.dll on Windows #31

martinfleis opened this issue Apr 21, 2021 · 12 comments

Comments

@martinfleis
Copy link

Issue:

Rtree under Windows has an issue loading libspatialindex, raising OSError: could not find or load spatialindex_c-64.dll. See this CI log as an example: https://github.com/knaaptime/segregation/runs/2395382705?check_suite_focus=true


Environment (conda list): 
$ conda list
# packages in environment at C:\Miniconda3\envs\test:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                     py_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
atomicwrites              1.4.0              pyh9f0ad1d_0    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
aws-requests-auth         0.4.3                      py_0  
babel                     2.9.0              pyhd3deb0d_0    conda-forge
beautifulsoup4            4.9.3              pyhb0f4dca_0    conda-forge
bleach                    3.3.0              pyh44b312d_0    conda-forge
blosc                     1.21.0               h0e60522_0    conda-forge
boost-cpp                 1.74.0               h54f0996_2    conda-forge
boto3                     1.17.54            pyhd8ed1ab_0    conda-forge
botocore                  1.20.54            pyhd8ed1ab_0    conda-forge
brotlipy                  0.7.0           py39hb82d6ee_1001    conda-forge
bs4                       4.9.3                         0    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2021.4.13            haa95532_1  
cairo                     1.16.0            hb19e0ff_1008    conda-forge
certifi                   2020.12.5        py39hcbf5309_1    conda-forge
cffi                      1.14.5           py39h0878f49_0    conda-forge
cfitsio                   3.470                hbbe6aef_7    conda-forge
chardet                   4.0.0            py39hcbf5309_1    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.1              pyhd8ed1ab_0    conda-forge
cmarkgfm                  0.5.3            py39hb82d6ee_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
coverage                  5.5              py39hb82d6ee_0    conda-forge
coveralls                 3.0.1              pyhd8ed1ab_0    conda-forge
cryptography              3.4.7            py39hd8d06c1_0    conda-forge
curl                      7.76.1               hf1763fc_1    conda-forge
cycler                    0.10.0                     py_2    conda-forge
decorator                 4.4.2                      py_0    conda-forge
descartes                 1.1.0                      py_4    conda-forge
dnspython                 2.0.0                      py_0    conda-forge
docopt                    0.6.2                      py_1    conda-forge
docutils                  0.16             py39hcbf5309_3    conda-forge
expat                     2.3.0                h39d44d4_0    conda-forge
fiona                     1.8.19           py39h9f1b043_0    conda-forge
flask                     1.1.2              pyh9f0ad1d_0    conda-forge
flask_cors                3.0.10             pyhd3deb0d_0    conda-forge
flask_json                0.3.4                      py_0    conda-forge
fontconfig                2.13.1            h1989441_1005    conda-forge
freetype                  2.10.4               h546665d_1    conda-forge
freexl                    1.0.6                ha8e266a_0    conda-forge
gdal                      3.2.2            py39h6795fcd_3    conda-forge
geographiclib             1.50                       py_0    conda-forge
geopandas                 0.9.0              pyhd8ed1ab_0    conda-forge
geopy                     2.1.0              pyhd3deb0d_0    conda-forge
geos                      3.9.1                h39d44d4_2    conda-forge
geotiff                   1.6.0                h8e90983_5    conda-forge
gettext                   0.19.8.1          h1a89ca6_1005    conda-forge
hdf4                      4.2.13            h0e5069d_1005    conda-forge
hdf5                      1.10.6          nompi_h5268f04_1114    conda-forge
icu                       68.1                 h0e60522_0    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagesize                 1.2.0                      py_0    conda-forge
importlib-metadata        3.10.1           py39hcbf5309_0    conda-forge
importlib_metadata        3.10.1               hd8ed1ab_0    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
intel-openmp              2021.2.0           h57928b3_616    conda-forge
itsdangerous              1.1.0                      py_0    conda-forge
jinja2                    2.11.3             pyh44b312d_0    conda-forge
jmespath                  0.10.0             pyh9f0ad1d_0    conda-forge
joblib                    1.0.1              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   he774522_0    conda-forge
jsonlines                 1.2.0              pyh9f0ad1d_2    conda-forge
jsonschema                3.2.0              pyhd8ed1ab_3    conda-forge
kealib                    1.4.14               h96bfa42_2    conda-forge
keyring                   23.0.1           py39hcbf5309_0    conda-forge
kiwisolver                1.3.1            py39h2e07f2f_1    conda-forge
krb5                      1.17.2               hbae68bd_0    conda-forge
latexcodec                2.0.1              pyh9f0ad1d_0    conda-forge
libblas                   3.9.0                     8_mkl    conda-forge
libcblas                  3.9.0                     8_mkl    conda-forge
libclang                  11.1.0          default_h5c34c98_0    conda-forge
libcurl                   7.76.1               hf1763fc_1    conda-forge
libffi                    3.3                  h0e60522_2    conda-forge
libgdal                   3.2.2                hade8cfc_3    conda-forge
libglib                   2.68.1               h1e62bf3_0    conda-forge
libiconv                  1.16                 he774522_0    conda-forge
libkml                    1.3.0             h9859afa_1013    conda-forge
liblapack                 3.9.0                     8_mkl    conda-forge
libnetcdf                 4.8.0           nompi_hee75d68_101    conda-forge
libpng                    1.6.37               ha81a0f5_2    conda-forge
libpq                     13.1                 h4f54205_2    conda-forge
libpysal                  4.4.0              pyhd8ed1ab_0    conda-forge
librttopo                 1.1.0                hb340de5_6    conda-forge
libspatialindex           1.9.3                he025d50_3    conda-forge
libspatialite             5.0.1                h6b539a6_4    conda-forge
libssh2                   1.9.0                h680486a_6    conda-forge
libtiff                   4.2.0                hc10be44_0    conda-forge
libwebp-base              1.2.0                h8ffe710_2    conda-forge
libxml2                   2.9.10               hf5bbc77_4    conda-forge
libzip                    1.7.3                hfed4ece_0    conda-forge
lz4-c                     1.9.3                h8ffe710_0    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
mapclassify               2.4.2              pyhd8ed1ab_0    conda-forge
markupsafe                1.1.1            py39hb82d6ee_3    conda-forge
matplotlib                3.4.1            py39hcbf5309_0    conda-forge
matplotlib-base           3.4.1            py39h581301d_0    conda-forge
mkl                       2020.4             hb70f87d_311    conda-forge
mock                      4.0.3            py39hcbf5309_1    conda-forge
more-itertools            8.7.0              pyhd8ed1ab_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
munch                     2.5.0                      py_0    conda-forge
networkx                  2.5.1              pyhd8ed1ab_0    conda-forge
numexpr                   2.7.3            py39h2e25243_0    conda-forge
numpy                     1.20.2           py39h6635163_0    conda-forge
numpydoc                  1.1.0                      py_1    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openjpeg                  2.4.0                h48faf41_0    conda-forge
openssl                   1.1.1k               h8ffe710_0    conda-forge
oset                      0.1.3                      py_1    conda-forge
osmnet                    0.1.6                      py_0    conda-forge
packaging                 20.9               pyh44b312d_0    conda-forge
pandana                   0.6.1            py39h2e25243_0    conda-forge
pandas                    1.2.4            py39h2e25243_0    conda-forge
patsy                     0.5.1                      py_0    conda-forge
pcre                      8.44                 ha925a31_0    conda-forge
pillow                    8.2.0            py39h4fa10fc_0  
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h8ffe710_0    conda-forge
pkginfo                   1.7.0              pyhd8ed1ab_0    conda-forge
pluggy                    0.13.1           py39hcbf5309_4    conda-forge
pockets                   0.9.1                      py_0    conda-forge
poppler                   21.03.0              h9ff6ed8_0    conda-forge
poppler-data              0.4.10                        0    conda-forge
postgresql                13.1                 h0f1a9bc_2    conda-forge
proj                      8.0.0                h1cfcee9_0    conda-forge
py                        1.10.0             pyhd3deb0d_0    conda-forge
pybtex                    0.24.0           py39hcbf5309_0    conda-forge
pybtex-docutils           1.0.0            py39hcbf5309_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pygeos                    0.9              py39haadaec5_4    conda-forge
pygments                  2.8.1              pyhd8ed1ab_0    conda-forge
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyproj                    3.0.1            py39h1007a03_1    conda-forge
pyqt                      5.12.3           py39hcbf5309_7    conda-forge
pyqt-impl                 5.12.3           py39h415ef7b_7    conda-forge
pyqt5-sip                 4.19.18          py39h415ef7b_7    conda-forge
pyqtchart                 5.12             py39h415ef7b_7    conda-forge
pyqtwebengine             5.12.1           py39h415ef7b_7    conda-forge
pyrsistent                0.17.3           py39hb82d6ee_2    conda-forge
pysocks                   1.7.1            py39hcbf5309_3    conda-forge
pytables                  3.6.1            py39h42e6cd8_3    conda-forge
pytest                    6.2.3            py39hcbf5309_0    conda-forge
pytest-cov                2.11.1             pyh44b312d_0    conda-forge
pytest-mpl                0.12               pyhd3deb0d_0    conda-forge
python                    3.9.4                h6244533_0  
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.9                      1_cp39    conda-forge
pytz                      2021.1             pyhd8ed1ab_0    conda-forge
pywin32-ctypes            0.2.0           py39hcbf5309_1003    conda-forge
pyyaml                    5.4.1            py39hb82d6ee_0    conda-forge
qt                        5.12.9               h5909a2a_4    conda-forge
quilt3                    3.4.0              pyhd8ed1ab_0    conda-forge
readme_renderer           27.0               pyh9f0ad1d_0    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
requests-futures          1.0.0              pyh9f0ad1d_2    conda-forge
requests-toolbelt         0.9.1                      py_0    conda-forge
rfc3986                   1.4.0              pyh9f0ad1d_0    conda-forge
rtree                     0.9.7            py39h09fdee3_1    conda-forge
s3transfer                0.4.0              pyhd8ed1ab_0    conda-forge
scikit-learn              0.24.1           py39he931e04_0    conda-forge
scipy                     1.6.2            py39hc0c34ad_0    conda-forge
seaborn                   0.11.1               h57928b3_0    conda-forge
seaborn-base              0.11.1             pyhd8ed1ab_1    conda-forge
setuptools                52.0.0           py39haa95532_0  
shapely                   1.7.1            py39h90c6b7e_4    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snowballstemmer           2.1.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.2.1              pyhd3eb1b0_0  
sphinx                    3.5.4              pyh44b312d_0    conda-forge
sphinx-gallery            0.8.2              pyhd8ed1ab_0    conda-forge
sphinx_bootstrap_theme    0.8.0                      py_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-bibtex      1.0.0                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-napoleon    0.7                        py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
sqlite                    3.35.4               h8ffe710_0    conda-forge
statsmodels               0.12.2           py39h5d4886f_0    conda-forge
tenacity                  7.0.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             2.1.0              pyh5ca1d4c_0    conda-forge
tiledb                    2.2.7                hf84e3da_0    conda-forge
tk                        8.6.10               he774522_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tornado                   6.1              py39hb82d6ee_1    conda-forge
tqdm                      4.60.0             pyhd8ed1ab_0    conda-forge
twine                     3.4.1              pyhd8ed1ab_0    conda-forge
tzdata                    2021a                he74cb21_0    conda-forge
urbanaccess               0.2.2              pyhd3deb0d_0    conda-forge
urllib3                   1.26.4             pyhd8ed1ab_0    conda-forge
vc                        14.2                 hb210afc_4    conda-forge
vs2015_runtime            14.28.29325          h5e1d092_4    conda-forge
webencodings              0.5.1                      py_1    conda-forge
werkzeug                  1.0.1              pyh9f0ad1d_0    conda-forge
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
win_inet_pton             1.1.0            py39hcbf5309_2    conda-forge
wincertstore              0.2             py39hcbf5309_1006    conda-forge
xerces-c                  3.2.3                h0e60522_2    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
yaml                      0.2.5                he774522_0    conda-forge
zipp                      3.4.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h62dcd97_1010    conda-forge
zstd                      1.4.9                h6255e5f_0    conda-forge

Details about conda and system ( conda info ): 
$ conda info

     active environment : test
    active env location : C:\Miniconda3\envs\test
            shell level : 1
       user config file : C:\Users\runneradmin\.condarc
 populated config files : C:\Users\runneradmin\.condarc
          conda version : 4.9.2
    conda-build version : not installed
         python version : 3.8.5.final.0
       virtual packages : __win=0=0
                          __archspec=1=x86_64
       base environment : C:\Miniconda3  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\runneradmin\conda_pkgs_dir
       envs directories : C:\Miniconda3\envs
                          C:\Users\runneradmin\.conda\envs
                          C:\Users\runneradmin\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.9.2 requests/2.24.0 CPython/3.8.5 Windows/10 Windows/10.0.17763
          administrator : True
             netrc file : None
           offline mode : False

# conda environments:
#
base                     C:\Miniconda3
test                  *  C:\Miniconda3\envs\test
@martinfleis martinfleis mentioned this issue Apr 21, 2021
Closed
@ocefpaf
Copy link
Member

ocefpaf commented Apr 21, 2021

I see that you have a few packages that from conda-forge. However, the ones that matter are so I'm not 100% sure this is the culprit here.

Did you try removing the defaults channel and/or activating strict channel priority? If there are missing packages in conda-forge to make that work we can try to add them. I just want to be sure it is not a case of mixed channel before we dive into debugging this.

@martinfleis
Copy link
Author

@knaaptime and @jGaboardi can you try/have you tried @ocefpaf's suggestions on your CI?

@jGaboardi jGaboardi mentioned this issue Apr 21, 2021
Merged
@jGaboardi
Copy link
Member

@ocefpaf & @martinfleis I had tried out using mamba awhile back in my workflow. Although initially it worked, it was actually ignoring my channel priority in conda. That is what seemingly was causing the rtree failures in spaghetti. Windows builds are now passing again. Thanks for the nudge in the right direction!

@jGaboardi jGaboardi mentioned this issue Apr 21, 2021
Closed
@ocefpaf
Copy link
Member

ocefpaf commented Apr 21, 2021

I'm not sure when mamba added the strict channel priority but I'm pretty sure they have it now. Pinging @wolfv who probably knows better.

@wolfv
Copy link
Member

wolfv commented Apr 21, 2021

Channel priority strict usually works well for us. I'd be curious to learn more about your problems.

You could also completely remove the defaults channel, maybe?

@jGaboardi
Copy link
Member

Removing mamba-version: '*' from unittests.yml in segregation also appears to solve the problem on Windows builds.

See pysal/segregation#166

@martinfleis
Copy link
Author

Strict channel priority seems to resolve this issue. However, it also seems that with conda, it passes even with flexible priority while with mamba it does not.

@wolfv
Copy link
Member

wolfv commented Apr 22, 2021

indeed, strict channel priority is strongly recommended by conda-forge :)

There are quite some small incompatibilities between defaults and conda-forge -- often you could even think about completely removing the defaults channel.

@jGaboardi
Copy link
Member

I can confirm that mamba does pass when channel-priority: strict. However, setting the up the environments seems to take about 1.5x-2x as long regardless of OS or Python version.

@wolfv
Copy link
Member

wolfv commented Apr 22, 2021

From quickly looking at your logs, the timings seem to be quite equal :)

If you want you can test out the provision-with-micromamba action. It might give more speed-up. The way conda env is implemented is quite messy, and we're monkey patching it. Therefore no parallel downloading in mamba env .... With micromamba, all of this is cleaner.

@jGaboardi
Copy link
Member

From quickly looking at your logs, the timings seem to be quite equal :)

Thanks for taking a look. Maybe I was looking at the wrong thing.

If you want you can test out the provision-with-micromamba action. It might give more speed-up. The way conda env is implemented is quite messy, and we're monkey patching it. Therefore no parallel downloading in mamba env .... With micromamba, all of this is cleaner.

Super! I am going to it out try out. I wasn't aware of it.

@mwtoews mwtoews mentioned this issue Oct 5, 2021
Closed
@hobu
Copy link
Contributor

hobu commented Apr 5, 2022

Closing this for now. Rtree 1.0.0 was just released that should have wheels for everything.

@hobu hobu closed this as completed Apr 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@hobu @wolfv @ocefpaf @jGaboardi @martinfleis