Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration of sweatpy #5

Open
5 tasks
glemaitre opened this issue May 22, 2018 · 16 comments
Open
5 tasks

Integration of sweatpy #5

glemaitre opened this issue May 22, 2018 · 16 comments

Comments

@glemaitre
Copy link
Contributor

glemaitre commented May 22, 2018 •
edited
Loading

TODO:

  • Integrate the heart-rate model into the sksports.model module. At a first glance, it would be good to not depend on lmfit but only scipy it will avoid a dependency. ([MRG] EHN add a model to fit and predict heart-rate from power #6)
  • Integrate all new metrics in the sksports.metrics module.
  • Not sure where to integrate the critical power (model or metric?)
  • Same for w_prime_balance
  • Integrate the IO temporarily in sksports.io module.
@glemaitre
Copy link
Contributor Author

@AartGoossens @sladkovm I moreless migrating scikit-cycling into scikit-sports.
Now I would like to make the integration of the sweatpy features and we can discuss what can be moved either part. I will make a TODO list but feel free to edit it any time.

@glemaitre glemaitre mentioned this issue May 27, 2018
Open
@AartGoossens
Copy link

Thanks for all the effort you already put in Guillaume.
About your todo's:

  • lmfit to scipy: good plan. Using lmfit was just laziness, no other valid reason.
  • I think both critical power and w'prime balance should be in models. Can you elaborate on the module structure you envision? How do you want to separate/organize the different models/metrics?

@sladkovm
Copy link

Guys, how do you run the test?

The closest I've got is this, but it still fails. I'm a very noob in build tools to decode it.

 sladkovm@MSMBP2015  ~/Dropbox/Projects/scikit-sports   feature/merge_sweatpy ●  /bin/bash build_tools/travis/test_script.sh
Python 3.7.0
numpy 1.15.4
scipy 1.2.0
4 CPUs
usage: py.test [options] [file_or_dir] [file_or_dir] [...]
py.test: error: unrecognized arguments: --cov= --cov-config
  inifile: /Users/sladkovm/Dropbox/Projects/scikit-sports/temp/setup.cfg
  rootdir: /Users/sladkovm/Dropbox/Projects/scikit-sports/temp

@AartGoossens
Copy link

I have issues running the tests too. @glemaitre did the setup of the test scripts so maybe he can help with running the tests locally.
The error you get is because the package pytest-cov is not installed.
And one other thing, you are running the tests with Python 3.7 but I think it hasn't been tested with that version before so that might cause some issues too.

@glemaitre
Copy link
Contributor Author

I don't think python 3.7 will be an issue. pip install pytest-cov or running the tests without coverage should solve the issue.

@sladkovm
Copy link

I've added pytest-cov, but overall it does not seem to do much:

(scikit-sports)  sladkovm@MSMBP2015  ~/Dropbox/Projects/scikit-sports   feature/merge_sweatpy ●  /bin/bash build_tools/travis/test_script.sh
Python 3.6.7
numpy 1.15.4
scipy 1.2.0
4 CPUs
================================================================================ test session starts ================================================================================
platform darwin -- Python 3.6.7, pytest-4.0.2, py-1.7.0, pluggy-0.8.0
rootdir: /Users/sladkovm/Dropbox/Projects/scikit-sports/temp, inifile: setup.cfg
plugins: cov-2.6.0
collected 0 items                                                                                                                                                                   
Coverage.py warning: Module  was never imported. (module-not-imported)
Coverage.py warning: No data was collected. (no-data-collected)
WARNING: Failed to generate report: No data to report.


================================================================================= warnings summary ==================================================================================
Failed to generate report: No data to report.

-- Docs: https://docs.pytest.org/en/latest/warnings.html

---------- coverage: platform darwin, python 3.6.7-final-0 -----------
Name    Stmts   Miss Branch BrPart  Cover
-----------------------------------------

============================================================================ 1 warnings in 0.01 seconds =============================================================================
Coverage.py warning: --include is ignored because --source is set (include-ignored)

@glemaitre
Copy link
Contributor Author

Could you try to run:

pytest -vsl sksports

from the root directory.

From what I tried quickly, I had issue to compile due to an old version of openmp. Let me check if this is still the case.

@glemaitre
Copy link
Contributor Author

OK so with a clean install:

conda create --name sksports python=3
conda activate sksports
conda install scipy numpy pandas scikit-learn six cython pytest
conda install -c conda-forge openmp
pip install pytest-cov fitparse

followed by a dev install

python setup.py build_ext --inplace
pip install -e .

you should be able to get the following running pytest:

22:58 $ pytest sksports
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.7.1, pytest-4.0.2, py-1.7.0, pluggy-0.8.0
rootdir: /tmp/scikit-sports, inifile: setup.cfg
collected 81 items                                                                                                                                                         

sksports/base.py .....                                                                                                                                               [  6%]
sksports/datasets/__init__.py ..                                                                                                                                     [  8%]
sksports/datasets/tests/test_datasets.py .....                                                                                                                       [ 14%]
sksports/extraction/gradient.py ....                                                                                                                                 [ 19%]
sksports/extraction/power_profile.py .                                                                                                                               [ 20%]
sksports/extraction/tests/test_gradient.py ...............                                                                                                           [ 39%]
sksports/extraction/tests/test_power_profile.py .....                                                                                                                [ 45%]
sksports/io/base.py .                                                                                                                                                [ 46%]
sksports/io/tests/test_fit.py ........                                                                                                                               [ 56%]
sksports/metrics/activity.py ......                                                                                                                                  [ 64%]
sksports/metrics/tests/test_activity.py .....                                                                                                                        [ 70%]
sksports/metrics/tests/test_power_profile.py ...                                                                                                                     [ 74%]
sksports/model/power.py .                                                                                                                                            [ 75%]
sksports/model/tests/test_power.py ....                                                                                                                              [ 80%]
sksports/tests/test_base.py .............                                                                                                                            [ 96%]
sksports/utils/validation.py .                                                                                                                                       [ 97%]
sksports/utils/tests/test_validation.py ..                                                                                                                           [100%]

============================================================================= warnings summary =============================================================================
/home/lemaitre/miniconda3/envs/scikit-sport/lib/python3.7/site-packages/pandas/core/dtypes/inference.py:6
  /home/lemaitre/miniconda3/envs/scikit-sport/lib/python3.7/site-packages/pandas/core/dtypes/inference.py:6: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import Iterable

/home/lemaitre/miniconda3/envs/scikit-sport/lib/python3.7/site-packages/pandas/core/tools/datetimes.py:3
  /home/lemaitre/miniconda3/envs/scikit-sport/lib/python3.7/site-packages/pandas/core/tools/datetimes.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import MutableMapping

-- Docs: https://docs.pytest.org/en/latest/warnings.html
================================================================== 81 passed, 2 warnings in 65.38 seconds ==================================================================

@sladkovm
Copy link

sladkovm commented Jan 4, 2019

Is there a way to install openmp without using conda?

@glemaitre
Copy link
Contributor Author

glemaitre commented Jan 4, 2019 via email

@sladkovm
Copy link

sladkovm commented Jan 5, 2019

I have installed gcc, do I need somehow to point a compiler to it? It seems to be trying to use clang

On a different note, are these parallel computation capabilities strictly necessary?

(scikit-sports)  sladkovm@MSMBP2015  ~/Dropbox/Projects/scikit-sports   feature/merge_sweatpy ●  pipenv install --skip-lock -e .
Installing -e .…
Adding e1839a8 to Pipfile's [packages]…
✔ Installation Succeeded 
Installing dependencies from Pipfile…
An error occurred while installing -e .! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 9/9 — 00:00:04
Installing initially failed dependencies…
[pipenv.exceptions.InstallError]:   File "/Users/sladkovm/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pipenv/core.py", line 1992, in do_install
[pipenv.exceptions.InstallError]:       skip_lock=skip_lock,
[pipenv.exceptions.InstallError]:   File "/Users/sladkovm/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pipenv/core.py", line 1253, in do_init
[pipenv.exceptions.InstallError]:       pypi_mirror=pypi_mirror,
[pipenv.exceptions.InstallError]:   File "/Users/sladkovm/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pipenv/core.py", line 859, in do_install_dependencies
[pipenv.exceptions.InstallError]:       retry_list, procs, failed_deps_queue, requirements_dir, **install_kwargs
[pipenv.exceptions.InstallError]:   File "/Users/sladkovm/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pipenv/core.py", line 763, in batch_install
[pipenv.exceptions.InstallError]:       _cleanup_procs(procs, not blocking, failed_deps_queue, retry=retry)
[pipenv.exceptions.InstallError]:   File "/Users/sladkovm/.pyenv/versions/3.6.7/lib/python3.6/site-packages/pipenv/core.py", line 681, in _cleanup_procs
[pipenv.exceptions.InstallError]:       raise exceptions.InstallError(c.dep.name, extra=err_lines)
[pipenv.exceptions.InstallError]: ['Obtaining file:///Users/sladkovm/Dropbox/Projects/scikit-sports', 'Requirement already satisfied, skipping upgrade: numpy in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from scikit-sports==0.1.0.dev0) (1.15.4)', 'Requirement already satisfied, skipping upgrade: scipy in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from scikit-sports==0.1.0.dev0) (1.2.0)', 'Requirement already satisfied, skipping upgrade: pandas in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from scikit-sports==0.1.0.dev0) (0.23.4)', 'Requirement already satisfied, skipping upgrade: scikit-learn in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from scikit-sports==0.1.0.dev0) (0.20.2)', 'Requirement already satisfied, skipping upgrade: six in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from scikit-sports==0.1.0.dev0) (1.12.0)', 'Requirement already satisfied, skipping upgrade: joblib in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from scikit-sports==0.1.0.dev0) (0.13.0)', 'Requirement already satisfied, skipping upgrade: fitparse in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from scikit-sports==0.1.0.dev0) (1.0.1)', 'Requirement already satisfied, skipping upgrade: cython in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from scikit-sports==0.1.0.dev0) (0.29.2)', 'Requirement already satisfied, skipping upgrade: pytz>=2011k in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from pandas->scikit-sports==0.1.0.dev0) (2018.7)', 'Requirement already satisfied, skipping upgrade: python-dateutil>=2.5.0 in /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages (from pandas->scikit-sports==0.1.0.dev0) (2.7.5)', 'Installing collected packages: scikit-sports', '  Running setup.py develop for scikit-sports', '    Complete output from command /Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/bin/python -c "import setuptools, tokenize;__file__=\'/Users/sladkovm/Dropbox/Projects/scikit-sports/setup.py\';f=getattr(tokenize, \'open\', open)(__file__);code=f.read().replace(\'\\r\\n\', \'\\n\');f.close();exec(compile(code, __file__, \'exec\'))" develop --no-deps:', '    sksports', '    sksports/__check_build/_check_build.pyx has not changed', '    sksports/extraction/_power_profile.pyx has not changed', '    Cythonizing sources', '    running develop', '    running build_scripts', '    running egg_info', '    running build_src', '    build_src', '    building extension "sksports.__check_build._check_build" sources', '    building extension "sksports.extraction._power_profile" sources', '    build_src: building npy-pkg config files', '    writing scikit_sports.egg-info/PKG-INFO', '    writing dependency_links to scikit_sports.egg-info/dependency_links.txt', '    writing requirements to scikit_sports.egg-info/requires.txt', '    writing top-level names to scikit_sports.egg-info/top_level.txt', "    reading manifest file 'scikit_sports.egg-info/SOURCES.txt'", "    reading manifest template 'MANIFEST.in'", "    writing manifest file 'scikit_sports.egg-info/SOURCES.txt'", '    running build_ext', '    customize UnixCCompiler', '    customize UnixCCompiler using build_ext', "    building 'sksports.extraction._power_profile' extension", '    compiling C sources', '    C compiler: clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall', '    ', "    compile options: '-I/Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages/numpy/core/include -I/Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages/numpy/core/include -I/Users/sladkovm/.pyenv/versions/3.6.7/include/python3.6m -I/Users/sladkovm/.pyenv/versions/3.6.7/include/python3.6m -c'", "    extra options: '-O3 -fopenmp'", '    clang: sksports/extraction/_power_profile.c', '    error: Command "clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages/numpy/core/include -I/Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/lib/python3.6/site-packages/numpy/core/include -I/Users/sladkovm/.pyenv/versions/3.6.7/include/python3.6m -I/Users/sladkovm/.pyenv/versions/3.6.7/include/python3.6m -c sksports/extraction/_power_profile.c -o build/temp.macosx-10.11-x86_64-3.6/sksports/extraction/_power_profile.o -MMD -MF build/temp.macosx-10.11-x86_64-3.6/sksports/extraction/_power_profile.o.d -O3 -fopenmp" failed with exit status 1', "    clang: error: unsupported option '-fopenmp'", "    clang: error: unsupported option '-fopenmp'", '    ', '    ----------------------------------------']
[pipenv.exceptions.InstallError]: ['Command "/Users/sladkovm/.local/share/virtualenvs/scikit-sports-6F5t9a5D/bin/python -c "import setuptools, tokenize;__file__=\'/Users/sladkovm/Dropbox/Projects/scikit-sports/setup.py\';f=getattr(tokenize, \'open\', open)(__file__);code=f.read().replace(\'\\r\\n\', \'\\n\');f.close();exec(compile(code, __file__, \'exec\'))" develop --no-deps" failed with error code 1 in /Users/sladkovm/Dropbox/Projects/scikit-sports/']
ERROR: ERROR: Package installation failed...

@glemaitre
Copy link
Contributor Author

glemaitre commented Jan 5, 2019 via email

@sladkovm
Copy link

sladkovm commented Jan 6, 2019

Great!

Managed to get it working.

@glemaitre
Copy link
Contributor Author

glemaitre commented Jan 6, 2019 via email

@sladkovm
Copy link

@AartGoossens , @glemaitre

As triggered by a discussion on twitter. Would it be a good idea to formulate a test suite in such a way that we test the C-heavy part of the library only on a need-to basis?

And the python-only part is tested by standard pytest/tox means.

This would really remove an overhead in bringing the sweatpy into the package.

@AartGoossens
Copy link

I just created a PR that adds running the tests in a docker container. Installing the dependencies was a pain outside of docker so at least for me this makes development a bit easier.

OT:
@sladkovm Why do you want to test the C-heavy part only on a need-to basis? Because of the duration of because of the required local dependencies?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@AartGoossens @glemaitre @sladkovm