Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Examples not working with local backend #118

Open
Nollde opened this issue Nov 3, 2023 · 4 comments
Open

Examples not working with local backend #118

Nollde opened this issue Nov 3, 2023 · 4 comments
Assignees
@matthewfeickert
Labels
bug Something isn't working

Comments

@Nollde
Copy link

Nollde commented Nov 3, 2023

Dear experts,

I just wanted to try one of the great recast examples with a local workflow.
It appears that this is currently not possible out of the box.

The problem seems to be that yadage expects a metadir to create a YadageSteering object here.
This metadir however is currently not set here.

Please find a minimal example here:

(base) ➜  ~ conda create -n recast-tmp python
(base) ➜  ~ conda activate recast-tmp
(recast-tmp) ➜  ~ python -m pip install recast-atlas
(recast-tmp) ➜  ~ pip install yadage
(recast-tmp) ➜  ~ recast run examples/rome --backend local                     
2023-11-03 11:27:11,370 | packtivity.asyncback |   INFO | configured pool size to 10
2023-11-03 11:27:11,421 | recastatlas.subcomma |  ERROR | caught exception
Traceback (most recent call last):
  File "/Users/dnoll/anaconda3/envs/recast-tmp/lib/python3.12/site-packages/recastatlas/backends/local.py", line 21, in run_workflow
    run_workflow(**spec)
  File "/Users/dnoll/anaconda3/envs/recast-tmp/lib/python3.12/site-packages/yadage/steering_api.py", line 19, in run_workflow
    with steering_ctx(*args, **kwargs):
  File "/Users/dnoll/anaconda3/envs/recast-tmp/lib/python3.12/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/Users/dnoll/anaconda3/envs/recast-tmp/lib/python3.12/site-packages/yadage/steering_api.py", line 87, in steering_ctx
    ys = YadageSteering.create(
         ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dnoll/anaconda3/envs/recast-tmp/lib/python3.12/site-packages/yadage/steering_object.py", line 61, in create
    prepare_meta(
  File "/Users/dnoll/anaconda3/envs/recast-tmp/lib/python3.12/site-packages/yadage/utils.py", line 229, in prepare_meta
    if os.path.exists(metadir):
       ^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen genericpath>", line 19, in exists
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

Is this the expected behavior?
Or am I missing something more fundamental?

Thank you for your comments!
@matthewfeickert matthewfeickert added the bug Something isn't working label Nov 4, 2023
@matthewfeickert
Copy link
Member

matthewfeickert commented Nov 4, 2023 •
edited
Loading

While you would normally want to use the 'local' extra

python -m pip install --upgrade 'recast-atlas[local]'

for the local backend option, I can reproduce your bug even with it. I think this is another case of yadage/yadage#116 that is getting hit, as

recast run examples/rome --backend docker

runs without errors.

Run output: 
$ recast run examples/rome --backend docker
2023-11-04 04:15:18,036 | packtivity.asyncback |   INFO | configured pool size to 16
2023-11-04 04:15:18,371 |      yadage.creators |   INFO | initializing workflow with initdata: {'did': 404958, 'dxaod_file': 'https://recastwww.web.cern.ch/recastwww/data/reana-recast-demo/mc15_13TeV.123456.cap_recast_demo_signal_one.root', 'xsec_in_pb': 0.00122} discover: True relative: True
2023-11-04 04:15:18,371 |    adage.pollingexec |   INFO | preparing adage coroutine.
2023-11-04 04:15:18,371 |                adage |   INFO | starting state loop.
2023-11-04 04:15:18,458 |     yadage.wflowview |   INFO | added </init:0|defined|unknown>
2023-11-04 04:15:19,438 |     yadage.wflowview |   INFO | added </eventselection:0|defined|unknown>
2023-11-04 04:15:20,710 |     yadage.wflowview |   INFO | added </statanalysis:0|defined|unknown>
2023-11-04 04:15:22,162 |    adage.pollingexec |   INFO | submitting nodes [</init:0|defined|known>]
2023-11-04 04:15:22,666 |       pack.init.step |   INFO | publishing data: <TypedLeafs: {'did': 404958, 'dxaod_file': 'https://recastwww.web.cern.ch/recastwww/data/reana-recast-demo/mc15_13TeV.123456.cap_recast_demo_signal_one.root', 'xsec_in_pb': 0.00122}>
2023-11-04 04:15:22,666 |                adage |   INFO | unsubmittable: 0 | submitted: 0 | successful: 0 | failed: 0 | total: 3 | open rules: 0 | applied rules: 3
2023-11-04 04:15:24,248 |           adage.node |   INFO | node ready </init:0|success|known>
2023-11-04 04:15:24,248 |    adage.pollingexec |   INFO | submitting nodes [</eventselection:0|defined|known>]
2023-11-04 04:15:24,250 | pack.eventselection. |   INFO | starting file logging for topic: step
2023-11-04 04:15:36,945 |           adage.node |   INFO | node ready </eventselection:0|success|known>
2023-11-04 04:15:36,945 |    adage.pollingexec |   INFO | submitting nodes [</statanalysis:0|defined|known>]
2023-11-04 04:15:36,947 | pack.statanalysis.st |   INFO | starting file logging for topic: step
2023-11-04 04:15:42,039 |           adage.node |   INFO | node ready </statanalysis:0|success|known>
2023-11-04 04:15:42,059 | adage.controllerutil |   INFO | no nodes can be run anymore and no rules are applicable
2023-11-04 04:15:42,059 | adage.controllerutil |   INFO | no nodes can be run anymore and no rules are applicable
2023-11-04 04:15:42,060 |                adage |   INFO | unsubmittable: 0 | submitted: 0 | successful: 3 | failed: 0 | total: 3 | open rules: 0 | applied rules: 3
2023-11-04 04:15:45,620 |                adage |   INFO | adage state loop done.
2023-11-04 04:15:45,620 |                adage |   INFO | execution valid. (in terms of execution order)
2023-11-04 04:15:45,620 |                adage |   INFO | workflow completed successfully.
2023-11-04 04:15:45,620 |  yadage.steering_api |   INFO | done. dumping workflow to disk.
2023-11-04 04:15:45,622 |  yadage.steering_api |   INFO | visualizing workflow.
2023-11-03 23:15:46,306 | recastatlas.subcomma |   INFO | RECAST run finished.

RECAST result examples/rome recast-c2b6ce7a:
--------------
- name: CLs 95% based upper limit on poi
  value:
    exp: 0.8924846399964371
    exp_m1: 0.6377501820447065
    exp_m2: 0.4731739008380644
    exp_p1: 1.2720762961732819
    exp_p2: 1.7545752712294322
    obs: 1.3352971254860764
- name: CLs 95% at nominal poi
  value:
    exp: 0.25999040745937085
    exp_m1: 0.10547655600578199
    exp_m2: 0.03889040527686523
    exp_p1: 0.5345040672498215
    exp_p2: 0.8276574946063575
    obs: 0.574709475331039

which means that some dependencies have a bug that needs to get fixed. This isn't getting caught properly in our CI at the moment, so this will probably take some debugging early next week.

@matthewfeickert matthewfeickert self-assigned this Nov 4, 2023
@matthewfeickert
Copy link
Member

matthewfeickert commented Nov 4, 2023 •
edited
Loading

Ah, @Nollde no this is actually a simple permissions issue and lack of documentation at the end of it, in addition to yadage/yadage#116. I should have debugged better.

Okay, starting from scratch in a Python 3.11 environment as I can't claim that Python 3.12 will have support for all dependencies yet.

$ mkdir -p /tmp/debug && cd /tmp/debug
$ python -m venv recast-atlas-issue-118 && . recast-atlas-issue-118/bin/activate
(recast-atlas-issue-118) $ python -m pip install --upgrade pip setuptools wheel
(recast-atlas-issue-118) $ python -m pip install 'recast-atlas[local]' 'yadage<0.21.0'  # c.f. https://github.com/yadage/yadage/issues/116

If we then run

recast run examples/rome --backend local --tag debug

the jobs will start but then crash. That's strange as we already saw that running with the docker backend works fine. If we investiage the log files by checking out recast-debug/eventselection/_packtivity/eventselection.run.log, we'll see that we get a permissions error

...
2023-11-04 01:14:50,653 | pack.eventselection. |   INFO | b'/build2/atnight/localbuilds/nightlies/21.2/AnalysisBase/athena/PhysicsAnalysis/D3PDTools/EventLoop/Root/Driver.cxx:107:exception: could not create output directory /tmp/debug/recast-debug/eventselection/submitDir'
...

that seems strange until we remember that recast-atlas currently assumes that you're root for all operations and so if you're not then you're going to need to tell it that you want it to run the containers as root. If you run

recast run examples/rome --backend docker --tag debug-helper

then we see that all the docker run commands have -u root added into them

$ grep "docker run" recast-debug-helper/eventselection/_packtivity/eventselection.step.log
docker run --rm -i  --cidfile /tmp/debug/recast-debug-helper/eventselection/_packtivity/eventselection.cid    -u root  -v /tmp/debug/recast-debug-helper/eventselection:/tmp/debug/recast-debug-helper/eventselection:rw -v /tmp/debug/recast-debug-helper/init:/tmp/debug/recast-debug-helper/init:rw docker.io/reanahub/reana-demo-atlas-recast-eventselection:1.0 sh -c bash
2023-11-04 06:20:29,219 | pack.eventselection. |  DEBUG | moving on from subprocess execution: docker run --rm -i  --cidfile /tmp/debug/recast-debug-helper/eventselection/_packtivity/eventselection.cid    -u root  -v /tmp/debug/recast-debug-helper/eventselection:/tmp/debug/recast-debug-helper/eventselection:rw -v /tmp/debug/recast-debug-helper/init:/tmp/debug/recast-debug-helper/init:rw docker.io/reanahub/reana-demo-atlas-recast-eventselection:1.0 sh -c bash
2023-11-04 06:20:29,219 | pack.eventselection. |  DEBUG | finally for docker run --rm -i  --cidfile /tmp/debug/recast-debug-helper/eventselection/_packtivity/eventselection.cid    -u root  -v /tmp/debug/recast-debug-helper/eventselection:/tmp/debug/recast-debug-helper/eventselection:rw -v /tmp/debug/recast-debug-helper/init:/tmp/debug/recast-debug-helper/init:rw docker.io/reanahub/reana-demo-atlas-recast-eventselection:1.0 sh -c bash

as a result of this packtivity environment setting getting set in the recast-atlas Dockerfile

recast-atlas/docker/Dockerfile

Line 49 in b660442

ENV PACKTIVITY_DOCKER_CMD_MOD "-u root"

(this isn't documented well).

With that in mind we just need to do the same locally with our local backend run

PACKTIVITY_DOCKER_CMD_MOD="-u root" recast run examples/rome --backend local --tag working

then things run as expected.

Run output: 
$ PACKTIVITY_DOCKER_CMD_MOD="-u root" recast run examples/rome --backend local --tag working
2023-11-04 01:26:35,769 | packtivity.asyncback |   INFO | configured pool size to 16
2023-11-04 01:26:36,085 |      yadage.creators |   INFO | initializing workflow with initdata: {'did': 404958, 'dxaod_file': 'https://recastwww.web.cern.ch/recastwww/data/reana-recast-demo/mc15_13TeV.123456.cap_recast_demo_signal_one.root', 'xsec_in_pb': 0.00122} discover: True relative: True
2023-11-04 01:26:36,085 |    adage.pollingexec |   INFO | preparing adage coroutine.
2023-11-04 01:26:36,085 |                adage |   INFO | starting state loop.
2023-11-04 01:26:36,107 |     yadage.wflowview |   INFO | added </init:0|defined|unknown>
2023-11-04 01:26:36,177 |     yadage.wflowview |   INFO | added </eventselection:0|defined|unknown>
2023-11-04 01:26:36,252 |     yadage.wflowview |   INFO | added </statanalysis:0|defined|unknown>
2023-11-04 01:26:36,326 |    adage.pollingexec |   INFO | submitting nodes [</init:0|defined|known>]
2023-11-04 01:26:36,352 |       pack.init.step |   INFO | publishing data: <TypedLeafs: {'did': 404958, 'dxaod_file': 'https://recastwww.web.cern.ch/recastwww/data/reana-recast-demo/mc15_13TeV.123456.cap_recast_demo_signal_one.root', 'xsec_in_pb': 0.00122}>
2023-11-04 01:26:36,353 |                adage |   INFO | unsubmittable: 0 | submitted: 0 | successful: 0 | failed: 0 | total: 3 | open rules: 0 | applied rules: 3
2023-11-04 01:26:36,456 |           adage.node |   INFO | node ready </init:0|success|known>
2023-11-04 01:26:36,456 |    adage.pollingexec |   INFO | submitting nodes [</eventselection:0|defined|known>]
2023-11-04 01:26:36,460 | pack.eventselection. |   INFO | starting file logging for topic: step
2023-11-04 01:26:47,761 |           adage.node |   INFO | node ready </eventselection:0|success|known>
2023-11-04 01:26:47,761 |    adage.pollingexec |   INFO | submitting nodes [</statanalysis:0|defined|known>]
2023-11-04 01:26:47,765 | pack.statanalysis.st |   INFO | starting file logging for topic: step
2023-11-04 01:26:52,524 |           adage.node |   INFO | node ready </statanalysis:0|success|known>
2023-11-04 01:26:52,544 | adage.controllerutil |   INFO | no nodes can be run anymore and no rules are applicable
2023-11-04 01:26:52,544 | adage.controllerutil |   INFO | no nodes can be run anymore and no rules are applicable
2023-11-04 01:26:52,544 |                adage |   INFO | unsubmittable: 0 | submitted: 0 | successful: 3 | failed: 0 | total: 3 | open rules: 0 | applied rules: 3
2023-11-04 01:26:55,982 |                adage |   INFO | adage state loop done.
2023-11-04 01:26:55,982 |                adage |   INFO | execution valid. (in terms of execution order)
2023-11-04 01:26:55,982 |                adage |   INFO | workflow completed successfully.
2023-11-04 01:26:55,982 |  yadage.steering_api |   INFO | done. dumping workflow to disk.
2023-11-04 01:26:55,984 |  yadage.steering_api |   INFO | visualizing workflow.
2023-11-04 01:26:56,073 | recastatlas.subcomma |   INFO | RECAST run finished.

RECAST result examples/rome recast-working:
--------------
- name: CLs 95% based upper limit on poi
  value:
    exp: 0.8924846399964371
    exp_m1: 0.6377501820447065
    exp_m2: 0.4731739008380644
    exp_p1: 1.2720762961732819
    exp_p2: 1.7545752712294322
    obs: 1.3352971254860764
- name: CLs 95% at nominal poi
  value:
    exp: 0.25999040745937085
    exp_m1: 0.10547655600578199
    exp_m2: 0.03889040527686523
    exp_p1: 0.5345040672498215
    exp_p2: 0.8276574946063575
    obs: 0.574709475331039

So:

@Nollde
Copy link
Author

Nollde commented Nov 7, 2023 •
edited
Loading

Hi @matthewfeickert , thanks for mentioning yadage/yadage#116, I was not aware of this issue.
With yadage<0.21.0 I can indeed use the local workflow.

The PACKTIVITY_DOCKER_CMD_MOD is not needed in my case.
You can see this in a small example here:

Example on local workflow
Environment Setup 
(base) ➜  issue_118: conda create --name recast_118 python==3.11

Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 23.7.4
  latest version: 23.10.0

Please update conda by running

    $ conda update -n base -c defaults conda

Or to minimize the number of packages updated during conda update use

     conda install conda=23.10.0



## Package Plan ##

  environment location: /Users/dnoll/anaconda3/envs/recast_118

  added / updated specs:
    - python==3.11


The following NEW packages will be INSTALLED:

  bzip2              pkgs/main/osx-arm64::bzip2-1.0.8-h620ffc9_4 
  ca-certificates    pkgs/main/osx-arm64::ca-certificates-2023.08.22-hca03da5_0 
  libffi             pkgs/main/osx-arm64::libffi-3.4.4-hca03da5_0 
  ncurses            pkgs/main/osx-arm64::ncurses-6.4-h313beb8_0 
  openssl            pkgs/main/osx-arm64::openssl-1.1.1w-h1a28f6b_0 
  pip                pkgs/main/osx-arm64::pip-23.3-py311hca03da5_0 
  python             pkgs/main/osx-arm64::python-3.11.0-hc0d8a6c_3 
  readline           pkgs/main/osx-arm64::readline-8.2-h1a28f6b_0 
  setuptools         pkgs/main/osx-arm64::setuptools-68.0.0-py311hca03da5_0 
  sqlite             pkgs/main/osx-arm64::sqlite-3.41.2-h80987f9_0 
  tk                 pkgs/main/osx-arm64::tk-8.6.12-hb8d0fd4_0 
  tzdata             pkgs/main/noarch::tzdata-2023c-h04d1e81_0 
  wheel              pkgs/main/osx-arm64::wheel-0.41.2-py311hca03da5_0 
  xz                 pkgs/main/osx-arm64::xz-5.4.2-h80987f9_0 
  zlib               pkgs/main/osx-arm64::zlib-1.2.13-h5a0b063_0 


Proceed ([y]/n)? y


Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate recast_118
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) ➜  issue_118: conda activate recast_118
(recast_118) ➜  issue_118: python -m pip install --upgrade pip setuptools wheel       
Requirement already satisfied: pip in /Users/dnoll/anaconda3/envs/recast_118/lib/python3.11/site-packages (23.3)
Collecting pip
  Downloading pip-23.3.1-py3-none-any.whl.metadata (3.5 kB)
Requirement already satisfied: setuptools in /Users/dnoll/anaconda3/envs/recast_118/lib/python3.11/site-packages (68.0.0)
Collecting setuptools
  Using cached setuptools-68.2.2-py3-none-any.whl.metadata (6.3 kB)
Requirement already satisfied: wheel in /Users/dnoll/anaconda3/envs/recast_118/lib/python3.11/site-packages (0.41.2)
Collecting wheel
  Using cached wheel-0.41.3-py3-none-any.whl.metadata (2.2 kB)
Downloading pip-23.3.1-py3-none-any.whl (2.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 10.2 MB/s eta 0:00:00
Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
Using cached wheel-0.41.3-py3-none-any.whl (65 kB)
Installing collected packages: wheel, setuptools, pip
  Attempting uninstall: wheel
    Found existing installation: wheel 0.41.2
    Uninstalling wheel-0.41.2:
      Successfully uninstalled wheel-0.41.2
  Attempting uninstall: setuptools
    Found existing installation: setuptools 68.0.0
    Uninstalling setuptools-68.0.0:
      Successfully uninstalled setuptools-68.0.0
  Attempting uninstall: pip
    Found existing installation: pip 23.3
    Uninstalling pip-23.3:
      Successfully uninstalled pip-23.3
Successfully installed pip-23.3.1 setuptools-68.2.2 wheel-0.41.3
(recast_118) ➜  issue_118: python -m pip install 'recast-atlas[local]' 'yadage<0.21.0'
Collecting yadage<0.21.0
  Using cached yadage-0.20.1-py3-none-any.whl
Collecting recast-atlas[local]
  Using cached recast_atlas-0.3.0-py3-none-any.whl.metadata (4.4 kB)
Collecting click>=7.0 (from recast-atlas[local])
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting jsonschema>=3.0.0 (from recast-atlas[local])
  Using cached jsonschema-4.19.2-py3-none-any.whl.metadata (7.9 kB)
Collecting pyyaml>=5.1 (from recast-atlas[local])
  Using cached PyYAML-6.0.1-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.1 kB)
Collecting yadage-schemas>=0.10.7 (from recast-atlas[local])
  Using cached yadage_schemas-0.11.0-py3-none-any.whl (20 kB)
Collecting pydotplus==2.0.2 (from recast-atlas[local])
  Using cached pydotplus-2.0.2-py3-none-any.whl
Collecting adage>=0.10.1 (from recast-atlas[local])
  Using cached adage-0.11.0-py3-none-any.whl (18 kB)
Collecting packtivity>=0.14.23 (from recast-atlas[local])
  Using cached packtivity-0.16.2-py3-none-any.whl.metadata (5.8 kB)
Collecting pydot (from recast-atlas[local])
  Using cached pydot-1.4.2-py2.py3-none-any.whl (21 kB)
Collecting pygraphviz (from recast-atlas[local])
  Using cached pygraphviz-1.11-cp311-cp311-macosx_11_0_arm64.whl
Collecting pyparsing>=2.0.1 (from pydotplus==2.0.2->recast-atlas[local])
  Using cached pyparsing-3.1.1-py3-none-any.whl.metadata (5.1 kB)
Collecting psutil (from yadage<0.21.0)
  Using cached psutil-5.9.6-cp38-abi3-macosx_11_0_arm64.whl.metadata (21 kB)
Collecting requests>=2.9 (from requests[security]>=2.9->yadage<0.21.0)
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting jsonref (from yadage<0.21.0)
  Using cached jsonref-1.1.0-py3-none-any.whl (9.4 kB)
Collecting jsonpointer>=1.10 (from yadage<0.21.0)
  Using cached jsonpointer-2.4-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting jsonpath-rw (from yadage<0.21.0)
  Using cached jsonpath_rw-1.4.0-py3-none-any.whl
Collecting checksumdir (from yadage<0.21.0)
  Using cached checksumdir-1.2.0-py3-none-any.whl (5.3 kB)
Collecting glob2 (from yadage<0.21.0)
  Using cached glob2-0.7-py2.py3-none-any.whl
Collecting jq (from yadage<0.21.0)
  Using cached jq-1.6.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (6.8 kB)
Collecting networkx>=2.4 (from adage>=0.10.1->recast-atlas[local])
  Using cached networkx-3.2.1-py3-none-any.whl.metadata (5.2 kB)
Collecting attrs>=22.2.0 (from jsonschema>=3.0.0->recast-atlas[local])
  Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting jsonschema-specifications>=2023.03.6 (from jsonschema>=3.0.0->recast-atlas[local])
  Using cached jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
Collecting referencing>=0.28.4 (from jsonschema>=3.0.0->recast-atlas[local])
  Using cached referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
Collecting rpds-py>=0.7.1 (from jsonschema>=3.0.0->recast-atlas[local])
  Using cached rpds_py-0.12.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (3.7 kB)
Collecting mock (from packtivity>=0.14.23->recast-atlas[local])
  Using cached mock-5.1.0-py3-none-any.whl.metadata (3.0 kB)
Collecting charset-normalizer<4,>=2 (from requests>=2.9->requests[security]>=2.9->yadage<0.21.0)
  Using cached charset_normalizer-3.3.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests>=2.9->requests[security]>=2.9->yadage<0.21.0)
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests>=2.9->requests[security]>=2.9->yadage<0.21.0)
  Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests>=2.9->requests[security]>=2.9->yadage<0.21.0)
  Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting jsonschema>=3.0.0 (from recast-atlas[local])
  Using cached jsonschema-4.9.1-py3-none-any.whl (79 kB)
Collecting six>=1.4.0 (from yadage-schemas>=0.10.7->recast-atlas[local])
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 (from jsonschema>=3.0.0->recast-atlas[local])
  Using cached pyrsistent-0.20.0-cp311-cp311-macosx_10_9_universal2.whl.metadata (27 kB)
Collecting ply (from jsonpath-rw->yadage<0.21.0)
  Using cached ply-3.11-py2.py3-none-any.whl (49 kB)
Collecting decorator (from jsonpath-rw->yadage<0.21.0)
  Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)
Using cached packtivity-0.16.2-py3-none-any.whl (36 kB)
Using cached jq-1.6.0-cp311-cp311-macosx_11_0_arm64.whl (412 kB)
Using cached PyYAML-6.0.1-cp311-cp311-macosx_11_0_arm64.whl (167 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached psutil-5.9.6-cp38-abi3-macosx_11_0_arm64.whl (246 kB)
Using cached recast_atlas-0.3.0-py3-none-any.whl (30 kB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached charset_normalizer-3.3.2-cp311-cp311-macosx_11_0_arm64.whl (118 kB)
Using cached networkx-3.2.1-py3-none-any.whl (1.6 MB)
Using cached pyparsing-3.1.1-py3-none-any.whl (103 kB)
Using cached pyrsistent-0.20.0-cp311-cp311-macosx_10_9_universal2.whl (83 kB)
Using cached urllib3-2.0.7-py3-none-any.whl (124 kB)
Using cached mock-5.1.0-py3-none-any.whl (30 kB)
Installing collected packages: ply, glob2, urllib3, six, pyyaml, pyrsistent, pyparsing, pygraphviz, psutil, networkx, mock, jsonref, jsonpointer, jq, idna, decorator, click, checksumdir, charset-normalizer, certifi, attrs, requests, pydotplus, pydot, jsonschema, jsonpath-rw, adage, yadage-schemas, recast-atlas, packtivity, yadage
Successfully installed adage-0.11.0 attrs-23.1.0 certifi-2023.7.22 charset-normalizer-3.3.2 checksumdir-1.2.0 click-8.1.7 decorator-5.1.1 glob2-0.7 idna-3.4 jq-1.6.0 jsonpath-rw-1.4.0 jsonpointer-2.4 jsonref-1.1.0 jsonschema-4.9.1 mock-5.1.0 networkx-3.2.1 packtivity-0.16.2 ply-3.11 psutil-5.9.6 pydot-1.4.2 pydotplus-2.0.2 pygraphviz-1.11 pyparsing-3.1.1 pyrsistent-0.20.0 pyyaml-6.0.1 recast-atlas-0.3.0 requests-2.31.0 six-1.16.0 urllib3-2.0.7 yadage-0.20.1 yadage-schemas-0.11.0
Recast Run 
(recast_118) ➜  issue_118: recast run examples/rome --backend local --tag debug
2023-11-07 09:47:51,219 | packtivity.asyncback |   INFO | configured pool size to 10
2023-11-07 09:47:51,413 |      yadage.creators |   INFO | initializing workflow with initdata: {'did': 404958, 'dxaod_file': 'https://recastwww.web.cern.ch/recastwww/data/reana-recast-demo/mc15_13TeV.123456.cap_recast_demo_signal_one.root', 'xsec_in_pb': 0.00122} discover: True relative: True
2023-11-07 09:47:51,413 |    adage.pollingexec |   INFO | preparing adage coroutine.
2023-11-07 09:47:51,413 |                adage |   INFO | starting state loop.
2023-11-07 09:47:51,430 |     yadage.wflowview |   INFO | added </init:0|defined|unknown>
2023-11-07 09:47:51,517 |     yadage.wflowview |   INFO | added </eventselection:0|defined|unknown>
2023-11-07 09:47:51,579 |     yadage.wflowview |   INFO | added </statanalysis:0|defined|unknown>
2023-11-07 09:47:51,642 |    adage.pollingexec |   INFO | submitting nodes [</init:0|defined|known>]
2023-11-07 09:47:51,662 |       pack.init.step |   INFO | publishing data: <TypedLeafs: {'did': 404958, 'dxaod_file': 'https://recastwww.web.cern.ch/recastwww/data/reana-recast-demo/mc15_13TeV.123456.cap_recast_demo_signal_one.root', 'xsec_in_pb': 0.00122}>
2023-11-07 09:47:51,662 |                adage |   INFO | unsubmittable: 0 | submitted: 0 | successful: 0 | failed: 0 | total: 3 | open rules: 0 | applied rules: 3
2023-11-07 09:47:51,754 |           adage.node |   INFO | node ready </init:0|success|known>
2023-11-07 09:47:51,754 |    adage.pollingexec |   INFO | submitting nodes [</eventselection:0|defined|known>]
2023-11-07 09:47:51,755 | pack.eventselection. |   INFO | starting file logging for topic: step
2023-11-07 09:48:06,228 |           adage.node |   INFO | node ready </eventselection:0|success|known>
2023-11-07 09:48:06,228 |    adage.pollingexec |   INFO | submitting nodes [</statanalysis:0|defined|known>]
2023-11-07 09:48:06,229 | pack.statanalysis.st |   INFO | starting file logging for topic: step
2023-11-07 09:48:14,566 |           adage.node |   INFO | node ready </statanalysis:0|success|known>
2023-11-07 09:48:14,586 | adage.controllerutil |   INFO | no nodes can be run anymore and no rules are applicable
2023-11-07 09:48:14,586 | adage.controllerutil |   INFO | no nodes can be run anymore and no rules are applicable
2023-11-07 09:48:14,586 |                adage |   INFO | unsubmittable: 0 | submitted: 0 | successful: 3 | failed: 0 | total: 3 | open rules: 0 | applied rules: 3
2023-11-07 09:48:21,046 |                adage |   INFO | adage state loop done.
2023-11-07 09:48:21,046 |                adage |   INFO | execution valid. (in terms of execution order)
2023-11-07 09:48:21,046 |                adage |   INFO | workflow completed successfully.
2023-11-07 09:48:21,046 |  yadage.steering_api |   INFO | done. dumping workflow to disk.
2023-11-07 09:48:21,047 |  yadage.steering_api |   INFO | visualizing workflow.
2023-11-07 09:48:21,364 | recastatlas.subcomma |   INFO | RECAST run finished.

RECAST result examples/rome recast-debug:
--------------
- name: CLs 95% based upper limit on poi
  value:
    exp: 0.8924846399964371
    exp_m1: 0.6377501820447065
    exp_m2: 0.4731739008380644
    exp_p1: 1.2720762961732819
    exp_p2: 1.7545752712294322
    obs: 1.3352971254860764
- name: CLs 95% at nominal poi
  value:
    exp: 0.25999040745937085
    exp_m1: 0.10547655600578199
    exp_m2: 0.03889040527686523
    exp_p1: 0.5345040672498215
    exp_p2: 0.8276574946063575
    obs: 0.574709475331039
Logs 
(recast_118) ➜  issue_118: grep "docker run" recast-debug/eventselection/_packtivity/eventselection.step.log 
docker run --rm -i  --cidfile /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection/_packtivity/eventselection.cid      -v /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection:/Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection:rw -v /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/init:/Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/init:rw docker.io/reanahub/reana-demo-atlas-recast-eventselection:1.0 sh -c bash
2023-11-07 09:48:06,168 | pack.eventselection. |  DEBUG | moving on from subprocess execution: docker run --rm -i  --cidfile /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection/_packtivity/eventselection.cid      -v /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection:/Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection:rw -v /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/init:/Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/init:rw docker.io/reanahub/reana-demo-atlas-recast-eventselection:1.0 sh -c bash
2023-11-07 09:48:06,168 | pack.eventselection. |  DEBUG | finally for docker run --rm -i  --cidfile /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection/_packtivity/eventselection.cid      -v /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection:/Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/eventselection:rw -v /Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/init:/Users/dnoll/tmp/recast-atlas/issue_118/recast-debug/init:rw docker.io/reanahub/reana-demo-atlas-recast-eventselection:1.0 sh -c bash
(recast_118) ➜  issue_118: echo $PACKTIVITY_DOCKER_CMD_MOD

(recast_118) ➜  issue_118:

If you are interested we could troubleshoot why you need the PACKTIVITY_DOCKER_CMD_MOD and I don't.
But we do not necessarily need to do this and I am also happy to just leave things as they are.
Also if we troubleshoot this we should probably move the discussion to a new issue.

This issue is correctly "connected to" and "resolved with" yadage/yadage#116.

@matthewfeickert
Copy link
Member

If you are interested we could troubleshoot why you need the PACKTIVITY_DOCKER_CMD_MOD and I don't.
But we do not necessarily need to do this and I am also happy to just leave things as they are.

That might be interesting, but I suspect that this would be related to id groups on our local machines. I'll open up an Issue though for the future as having as unified of an experience is beneficial in the long run. 👍

Though the first order of priority when I get time to do recast development again is fixing yadage/yadage#116 and then getting that tested with the REANA team.

@matthewfeickert matthewfeickert mentioned this issue Nov 7, 2023
Open
This was referenced Jun 14, 2024
Merged
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matthewfeickert matthewfeickert

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@matthewfeickert @Nollde