New logic for STAGE_INPUT valorization #70
Conversation
cobrawap/__main__.py
Outdated
| stage_idx = locate_str_in_list(config_dict['STAGES'], stage) | ||
| if stage_idx is None: | ||
| stage_idx_local = locate_str_in_list(config_dict['STAGES'], stage) | ||
| stage_idx_global = locate_str_in_list([v for k,v in stages.items()], stage) |
There was a problem hiding this comment.
| stage_idx_global = locate_str_in_list([v for k,v in stages.items()], stage) | |
| is_first_stage = locate_str_in_list(list(stages.values()), stage) == 0 |
There was a problem hiding this comment.
Though the definition of the current stage index in the "global" (i.e. the one present in ~/.cobrawap/config file) list of stages is now become no longer useful, and hence left as a comment for potential upcoming changes, still the is_first_stage boolean should refer to the custom-defined list of stages. However, stage_idx_local already does the job, being equal to 0 or not, so introducing a further boolean variable for storing such info looks useless.
cobrawap/__main__.py
Outdated
| @@ -323,35 +323,36 @@ def run_stage(profile=None, stage=None, extra_args=None, **kwargs): | |||
| # lookup stage input file | |||
| pipeline_config_path = config_path / 'configs' / 'config.yaml' | |||
| config_dict = load_config_file(pipeline_config_path) | |||
| stage_idx = locate_str_in_list(config_dict['STAGES'], stage) | |||
| if stage_idx is None: | |||
| stage_idx_local = locate_str_in_list(config_dict['STAGES'], stage) | |||
There was a problem hiding this comment.
keep the variable name as stage_idx
There was a problem hiding this comment.
It's important to differentiate among the stage index in the custom-defined list of stages, and the larger one present in the ~/.cobrawap/config file.
There was a problem hiding this comment.
There is only one stage_idx in use here (the "local" one), so there is no need to differentiate
There was a problem hiding this comment.
Yes, but using "talking" names for variables helps... We can use "stage_idx" as it used to be, or "user_stage_idx", or any other meaningful suggestion...
There was a problem hiding this comment.
let's stick with stage_idx for now.
We may introduce specifications of different stage numeration in the future when the need arises.
cobrawap/__main__.py
Outdated
| config_name=prev_config_name) | ||
| stage_input = output_path / profile / prev_stage / output_name | ||
| if stage_idx_local>0: | ||
| prev_stage = [v for k,v in stages.items()][stage_idx_local-1] |
There was a problem hiding this comment.
Revert to the previous logic here.
This line tries to access the global cobrawap config with the index of the pipeline config!
There was a problem hiding this comment.
This was a leftover bug from previous changes in this PR. Thanks for having pointed it out. Fixed through the last commit.
|
@rgutzen Please recheck the latest changes. |
cobrawap/__main__.py
Outdated
| @@ -323,35 +323,36 @@ def run_stage(profile=None, stage=None, extra_args=None, **kwargs): | |||
| # lookup stage input file | |||
| pipeline_config_path = config_path / 'configs' / 'config.yaml' | |||
| config_dict = load_config_file(pipeline_config_path) | |||
| stage_idx = locate_str_in_list(config_dict['STAGES'], stage) | |||
| if stage_idx is None: | |||
| stage_idx_local = locate_str_in_list(config_dict['STAGES'], stage) | |||
There was a problem hiding this comment.
let's stick with stage_idx for now.
We may introduce specifications of different stage numeration in the future when the need arises.
cobrawap/__main__.py
Outdated
| config_name=prev_config_name) | ||
| stage_input = output_path / profile / prev_stage / output_name | ||
|
|
||
| if stage_idx_local>0: |
There was a problem hiding this comment.
| if stage_idx_local>0: | |
| if stage_idx_local > 0: |
cobrawap/pipeline/utils/Snakefile
Outdated
| logger.warning('No STAGE_INPUT defined for running the stage individually! ' | ||
| 'You can set it via the command line with ' | ||
| '`--config STAGE_INPUT=/path/to/file`.') | ||
| if config['STAGE_NAME']!=get_setting('stages')['1']: | ||
| # logger.warning() | ||
| raise InputError('No STAGE_INPUT defined for running stage \'{}\' individually! '.format(config['STAGE_NAME']) + | ||
| 'You can set it via the command line with ' + | ||
| '`--config STAGE_INPUT=/path/to/file`.') |
There was a problem hiding this comment.
This should be kept as a logger.warning. The stage itself will raise an error when there is a missing input with potentially additional information, that won't be shown if an error is raised before.
Co-authored-by: Robin Gutzen <[email protected]>
|
I also suggest to consider to send an alert to the user any time (as in this case) when the user's settings have been overwritten, something like: logger.warning(Cobrawap is running from first stage - ‘STAGE_INPUT forced to Nan. Your user settings have been overwritten) |
This is a solution to issue #68, where value assigned to key STAGE_INPUT shows circular dependencies on the user-defined list of stages present in the top-level config file.