Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add Tensorflow test #38

Merged
merged 36 commits into from
Jul 31, 2023
Merged

add Tensorflow test #38

merged 36 commits into from
Jul 31, 2023

Commits on May 22, 2023

  1. Initial version of the python files for the TensorFlow test

    Caspar van Leeuwen committed May 22, 2023
    Configuration menu
    Copy the full SHA
    ff57caa View commit details
    Browse the repository at this point in the history

  2. Merge branch 'EESSI:main' into tensorflow

    @casparvl
    casparvl authored May 22, 2023
    Configuration menu
    Copy the full SHA
    5b16306 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2023

  1. Modify list of visible GPU devices to empty when argument is cpu

    Caspar van Leeuwen committed May 23, 2023
    Configuration menu
    Copy the full SHA
    61c29ec View commit details
    Browse the repository at this point in the history

Commits on May 24, 2023

  1. Merge branch 'tensorflow' of github.com:casparvl/test-suite into tens… 

    …orflow
    Caspar van Leeuwen committed May 24, 2023
    Configuration menu
    Copy the full SHA
    b466396 View commit details
    Browse the repository at this point in the history

  2. Moved test files

    Caspar van Leeuwen committed May 24, 2023
    Configuration menu
    Copy the full SHA
    5428ff5 View commit details
    Browse the repository at this point in the history

  3. Reorganized the test code. Less hardcoded, more arguments, more segme… 

    …nted into functions. Also separated the training (which is timed now) and evaluation (which is not timed). Clearly print computational performance and accuracy at the end, to make it easy for a ReFrame test to pick up in sanity and performance functions.
    Caspar van Leeuwen committed May 24, 2023
    Configuration menu
    Copy the full SHA
    b058bc9 View commit details
    Browse the repository at this point in the history

  4. Added ReFrame TensorFlow test. Still a work in progress: number of ta… 

    …sks etc is still hard-coded. Also still a todo: make sure that proper binding is used
    Caspar van Leeuwen committed May 24, 2023
    Configuration menu
    Copy the full SHA
    9d599c5 View commit details
    Browse the repository at this point in the history

  5. Added support for launching one task per socket, useful for hybrid pr… 

    …ograms like TensorFlow
    Caspar van Leeuwen committed May 24, 2023
    Configuration menu
    Copy the full SHA
    85fab0e View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2023

  1. Added binding environment variables

    Caspar van Leeuwen committed Jun 5, 2023
    Configuration menu
    Copy the full SHA
    7aab841 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2023

  1. Set process binding. We commented out thread binding for now, as on m… 

    …y machine that results in all threads being bound to the first core of the allocation, rather than one thread per core
    Caspar van Leeuwen committed Jun 13, 2023
    Configuration menu
    Copy the full SHA
    e9cab6b View commit details
    Browse the repository at this point in the history

  2. merged with main, resolved conflicts due to renaming of namespace

    Caspar van Leeuwen committed Jun 13, 2023
    Configuration menu
    Copy the full SHA
    bf40e9a View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2023

  1. Made separate hooks for binding processes and binding threads. Only d… 

    …o process binding for TF
    Caspar van Leeuwen committed Jun 16, 2023
    Configuration menu
    Copy the full SHA
    09e7704 View commit details
    Browse the repository at this point in the history

  2. Use tf.config.threading api to set number of threads

    Caspar van Leeuwen committed Jun 16, 2023
    Configuration menu
    Copy the full SHA
    c063f64 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2023

  1. Increased default batch size to correspond to the many tests run in E… 

    …ESSI#38 , e.g. EESSI#38 (comment)
    Caspar van Leeuwen committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    27bf9aa View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into tensorflow

    @casparvl
    casparvl authored Jun 19, 2023
    Configuration menu
    Copy the full SHA
    0ab5a39 View commit details
    Browse the repository at this point in the history

  3. Add logging to binding hooks

    Caspar van Leeuwen committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    e682b7a View commit details
    Browse the repository at this point in the history

  4. Make more sensible default behaviour for partial nodes in the SCALES

    Caspar van Leeuwen committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    c359c3a View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2023

  1. Changed optimizer, faster convergence, so we can more sure that the n… 

    …etwork learned something
    Caspar van Leeuwen committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    18a20ae View commit details
    Browse the repository at this point in the history

  2. Sanity check for large node counts were failing since I/O was not coo… 

    …rdinated between workers. This would result in lines being broken off and sanity patterns not matching. All printing is now done by rank 0
    Caspar van Leeuwen committed Jun 20, 2023
    Configuration menu
    Copy the full SHA
    79eebb7 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2023

  1. Fix process binding on hyperthreading enabled systems

    Caspar van Leeuwen committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    8903e40 View commit details
    Browse the repository at this point in the history

  2. Define separate variable, comment, and reuse that

    Caspar van Leeuwen committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    fae59b1 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2023

  1. Add some support for systems with hyperthreading to assign_one_task_p… 

    …er_compute_unit
    Caspar van Leeuwen committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    c6d77ff View commit details
    Browse the repository at this point in the history

  2. Trying to get hyperthreading to do something sensible. Not working yet

    Caspar van Leeuwen committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    d52ae1a View commit details
    Browse the repository at this point in the history

  3. Add some context

    Caspar van Leeuwen committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    dca0a2e View commit details
    Browse the repository at this point in the history

  4. Revert changes to make TF work on hyperthreading systems

    Caspar van Leeuwen committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    1c023d1 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2023

  1. Should not have been in this PR, is part of another

    Caspar van Leeuwen committed Jul 18, 2023
    Configuration menu
    Copy the full SHA
    358966e View commit details
    Browse the repository at this point in the history

  2. Update eessi/testsuite/tests/apps/tensorflow/src/tf_test.py 

    Co-authored-by: Sam Moors <[email protected]>
    @casparvl @smoors
    casparvl and smoors authored Jul 18, 2023
    Configuration menu
    Copy the full SHA
    16ddf31 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2023

  1. Merge remote-tracking branch 'origin/main' into tensorflow

    Caspar van Leeuwen committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    50d49a6 View commit details
    Browse the repository at this point in the history

  2. Update eessi/testsuite/tests/apps/tensorflow/src/tf_test.py 

    More elegant way of retrieving local rank

Co-authored-by: Sam Moors <[email protected]>
    @casparvl @smoors
    casparvl and smoors authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    ea5414b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'tensorflow' of github.com:casparvl/test-suite into tens… 

    …orflow
    Caspar van Leeuwen committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    932b2f3 View commit details
    Browse the repository at this point in the history

  4. Update eessi/testsuite/hooks.py 

    Enable verbosity for SLURM binding

Co-authored-by: Sam Moors <[email protected]>
    @casparvl @smoors
    casparvl and smoors authored Jul 26, 2023
    Configuration menu
    Copy the full SHA
    03f37d0 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2023

  1. Check if ReFrame config specifies the required processor attributes

    Caspar van Leeuwen committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    178d70c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'tensorflow' of github.com:casparvl/test-suite into tens… 

    …orflow
    Caspar van Leeuwen committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    4416d08 View commit details
    Browse the repository at this point in the history

  3. Implemented review comments: logging, calling a hook for the process … 

    …binding, etc
    Caspar van Leeuwen committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    e87e628 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2023

  1. Merged main into this branch, then resolved conflicts

    Caspar van Leeuwen committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    ea561f4 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. Merge branch 'main' into tensorflow

    Caspar van Leeuwen committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    9e1b5a2 View commit details
    Browse the repository at this point in the history