* fix file naming bug

* fix filename bug with variable as study name

* add tests for the file name special vars changes

* modify changelog

* implement Luc's suggestions

* remove replace line

* first attempt at adding pdf

* fixing build error

* modify changelog to show docs changes

* fix errors Luc found in the build logs

* trying out removal of latex

* reverting latex changes back

* uncommenting the latex_elements settings

* adding epub to see if latex will build

* adding a latex engine variable to conf

* fix naming error with latex_engine

* attempting to add a logo to the pdf build

* testing an override to the searchtools file

* revert back to not using searchtools override

* update changelog

* fix openfoam_singularity issues

* update requirements and descriptions for openfoam examples

* fix bug with output_path and variable substitution

* add tests for cli substitutions

* bump version to 1.10.2

* bump version in CHANGELOG

* fix default worker bug with all steps

* version bump and requirements fix

* Bump certifi from 2022.12.7 to 2023.7.22 in /docs

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](certifi/python-certifi@2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* add all dependabot changes and update CHANGELOG

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add *.conf to the MANIFEST file so pip will grab the redis.conf file

* add note explaining how to fix a hanging merlin server start

* modify CHANGELOG

* add second export option to docs and fix typo

* Version/1.10.3 (#445)

* fix default worker bug with all steps

* version bump and requirements fix

* Bugfix/filename-special-vars (#425)

* fix file naming bug

* fix filename bug with variable as study name

* add tests for the file name special vars changes

* modify changelog

* implement Luc's suggestions

* remove replace line

* Create dependabot-changelog-updater.yml

* testing outputs of modifying changelog

* delete dependabot-changelog-updater

* feature/pdf-docs (#427)

* first attempt at adding pdf

* fixing build error

* modify changelog to show docs changes

* fix errors Luc found in the build logs

* trying out removal of latex

* reverting latex changes back

* uncommenting the latex_elements settings

* adding epub to see if latex will build

* adding a latex engine variable to conf

* fix naming error with latex_engine

* attempting to add a logo to the pdf build

* testing an override to the searchtools file

* revert back to not using searchtools override

* update changelog

* bugfix/openfoam_singularity_issues (#426)

* fix openfoam_singularity issues

* update requirements and descriptions for openfoam examples

* bugfix/output-path-substitution (#430)

* fix bug with output_path and variable substitution

* add tests for cli substitutions

* bugfix/scheduler-permission-error (#436)

* Release/1.10.2 (#437)

* bump version to 1.10.2

* bump version in CHANGELOG

* resolve develop to main merge issues (#439)

* fix default worker bug with all steps

* version bump and requirements fix

* dependabot/certifi-requests-pygments (#441)

* Bump certifi from 2022.12.7 to 2023.7.22 in /docs

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](certifi/python-certifi@2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* add all dependabot changes and update CHANGELOG

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bugfix/server-pip-redis-conf (#443)

* add *.conf to the MANIFEST file so pip will grab the redis.conf file

* add note explaining how to fix a hanging merlin server start

* modify CHANGELOG

* add second export option to docs and fix typo

* bump to version 1.10.3 (#444)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* change hardcoded sphinx requirement

* update CHANGELOG

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix file naming error for iterative workflows

* fixed small bug with new filepath naming

* add VLAUNCHER functionality

* add docs for VLAUNCHER and modify changelog

* re-word docs and fix table format

* add a test for vlauncher

* run fix-style and add a test for vlauncher

* Add the find_vlaunch_var and setup_vlaunch functions.
The numeric value of the shell variables may not be defined until run
time, so replace with variable strings instead of values.
Consolidate the commands into one function.

* Add variable set for (t)csh.

* Run fix-style

* make step settings the defaults and ignore commented lines

* add some additional tests

* remove regex library import

---------

Co-authored-by: Joseph M. Koning <[email protected]>

* add patch for skewed sample hierarchy/additional samples

* update changelog

* catch narrower range of exceptions

* fix typo in batch.py that causes a bug

* change print statements to log statements

* begin work on integration refactor; create fixtures and initial tests

* update CHANGELOG and run fix-style

* add pytest fixtures and README explaining them

* add tests to demonstrate how to use the fixtures

* move/rename some files and modify integration's README

* add password change to redis.pass file

* fix lint issues

* modify redis pwd for test server to be constant for each test

* fix lint issue only caught on github ci

* begin work on integration refactor; create fixtures and initial tests

* update CHANGELOG and run fix-style

* add pytest fixtures and README explaining them

* add tests to demonstrate how to use the fixtures

* move/rename some files and modify integration's README

* add password change to redis.pass file

* fix lint issues

* modify redis pwd for test server to be constant for each test

* fix lint issue only caught on github ci

* add fix for merlin server startup

* update CHANGELOG

* add celery query to see if workers still processing tasks

* fix merlin status when using redis as broker

* fix consumer count bug and run fix-style

* fix linter issues

* update changelog

* update docs for monitor

* remove unused exception I previously added

* first attempt at using pytest fixtures for monitor tests

* (partially) fix launch_workers fixture so it can be used in multiple classes

* fix linter issues and typo on pytest decorator

* update black's python version and fix style issue

* remove print statements from celeryadapter.py

* workers manager is now allowed to be used as a context manager

* add one thing to changelog and remove print statement

* remove a merge conflict statement that was missed

* add base requirements for mkdocs

* set up configuration for API docs

* start work on porting user guide to mkdocs

* add custom styling and contact page

* begin work on porting tutorial to mkdocs

* add new examples page

* move old sphinx docs to their own folder (*delete later*)

* modify some admonitions to be success

* modify hello examples page and port step 3 of tutorial to mkdocs

* fix typo in hello example

* finish porting step 4 of tutorial to mkdocs

* port part 5 of the tutorial to mkdocs

* copy faq and contributing from old docs

* port step 6 of tutorial to mkdocs

* remove unused prereq

* port step 7 of tutorial to mkdocs

* add more detailed instructions on contributing

* move venv page into installation and add spack instructions too

* add configuration docs

* add content to user guide landing page

* port celery page to mkdocs

* rearrange configuration pages to add in merlin server configuration instructions

* port command line page to mkdocs

* finish new landing page

* change size of merlin logo

* port variables page to mkdocs

* fix broken links to configuration page

* port FAQ to mkdocs

* fix incorrect requirement name

* update CHANGELOG

* attempt to get docs to build through readthedocs

* port docker page to mkdocs

* port contributing guide to mkdocs

* add new 'running studies' page

* add path changes to images

* add a page on how to interpret study output

* add page on the spec file

* remove old sphinx docs that are no longer needed

* added README to docs and updated CHANGELOG

* fix copyright and hello_samples tree

* rearrange images/stylesheets and statements that use them

* add suggestions from Luc and Joe

* add tcsh instructions for venv activation

* add Charle's suggestions for the landing page

* change tcsh mentions to csh

* feature/new-status (#442)

* add backend functionality for merlin status

* add frontend functionality for merlin status

* add tests for merlin status

* run fix-style and remove import of deprecated function

* update CHANGELOG

* add more logging statements, make better use of glob

* run fix-style

* clean up test files a bit

* fix test suite after step_name_map mod

* add avg/std dev run time calculations to status

* modify status tests to accommodate new avg/std dev calculations

* fix linter issues

* fix lint issue and add test for avg/std dev calc

* feature/detailed-status (#451)

* Version/1.11.0 (#449)

* fix default worker bug with all steps

* version bump and requirements fix

* Bugfix/filename-special-vars (#425)

* fix file naming bug

* fix filename bug with variable as study name

* add tests for the file name special vars changes

* modify changelog

* implement Luc's suggestions

* remove replace line

* Create dependabot-changelog-updater.yml

* testing outputs of modifying changelog

* delete dependabot-changelog-updater

* feature/pdf-docs (#427)

* first attempt at adding pdf

* fixing build error

* modify changelog to show docs changes

* fix errors Luc found in the build logs

* trying out removal of latex

* reverting latex changes back

* uncommenting the latex_elements settings

* adding epub to see if latex will build

* adding a latex engine variable to conf

* fix naming error with latex_engine

* attempting to add a logo to the pdf build

* testing an override to the searchtools file

* revert back to not using searchtools override

* update changelog

* bugfix/openfoam_singularity_issues (#426)

* fix openfoam_singularity issues

* update requirements and descriptions for openfoam examples

* bugfix/output-path-substitution (#430)

* fix bug with output_path and variable substitution

* add tests for cli substitutions

* bugfix/scheduler-permission-error (#436)

* Release/1.10.2 (#437)

* bump version to 1.10.2

* bump version in CHANGELOG

* resolve develop to main merge issues (#439)

* fix default worker bug with all steps

* version bump and requirements fix

* dependabot/certifi-requests-pygments (#441)

* Bump certifi from 2022.12.7 to 2023.7.22 in /docs

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](certifi/python-certifi@2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* add all dependabot changes and update CHANGELOG

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bugfix/server-pip-redis-conf (#443)

* add *.conf to the MANIFEST file so pip will grab the redis.conf file

* add note explaining how to fix a hanging merlin server start

* modify CHANGELOG

* add second export option to docs and fix typo

* bump to version 1.10.3 (#444)

* bugfix/sphinx-5.3.0-requirement (#446)

* Version/1.10.3 (#445)

* fix default worker bug with all steps

* version bump and requirements fix

* Bugfix/filename-special-vars (#425)

* fix file naming bug

* fix filename bug with variable as study name

* add tests for the file name special vars changes

* modify changelog

* implement Luc's suggestions

* remove replace line

* Create dependabot-changelog-updater.yml

* testing outputs of modifying changelog

* delete dependabot-changelog-updater

* feature/pdf-docs (#427)

* first attempt at adding pdf

* fixing build error

* modify changelog to show docs changes

* fix errors Luc found in the build logs

* trying out removal of latex

* reverting latex changes back

* uncommenting the latex_elements settings

* adding epub to see if latex will build

* adding a latex engine variable to conf

* fix naming error with latex_engine

* attempting to add a logo to the pdf build

* testing an override to the searchtools file

* revert back to not using searchtools override

* update changelog

* bugfix/openfoam_singularity_issues (#426)

* fix openfoam_singularity issues

* update requirements and descriptions for openfoam examples

* bugfix/output-path-substitution (#430)

* fix bug with output_path and variable substitution

* add tests for cli substitutions

* bugfix/scheduler-permission-error (#436)

* Release/1.10.2 (#437)

* bump version to 1.10.2

* bump version in CHANGELOG

* resolve develop to main merge issues (#439)

* fix default worker bug with all steps

* version bump and requirements fix

* dependabot/certifi-requests-pygments (#441)

* Bump certifi from 2022.12.7 to 2023.7.22 in /docs

Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](certifi/python-certifi@2022.12.07...2023.07.22)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* add all dependabot changes and update CHANGELOG

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bugfix/server-pip-redis-conf (#443)

* add *.conf to the MANIFEST file so pip will grab the redis.conf file

* add note explaining how to fix a hanging merlin server start

* modify CHANGELOG

* add second export option to docs and fix typo

* bump to version 1.10.3 (#444)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* change hardcoded sphinx requirement

* update CHANGELOG

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feature/vlauncher (#447)

* fix file naming error for iterative workflows

* fixed small bug with new filepath naming

* add VLAUNCHER functionality

* add docs for VLAUNCHER and modify changelog

* re-word docs and fix table format

* add a test for vlauncher

* run fix-style and add a test for vlauncher

* Add the find_vlaunch_var and setup_vlaunch functions.
The numeric value of the shell variables may not be defined until run
time, so replace with variable strings instead of values.
Consolidate the commands into one function.

* Add variable set for (t)csh.

* Run fix-style

* make step settings the defaults and ignore commented lines

* add some additional tests

* remove regex library import

---------

Co-authored-by: Joseph M. Koning <[email protected]>

* release/1.11.0 (#448)

* bugfix/skewed-sample-hierarchy (#450)

* add patch for skewed sample hierarchy/additional samples

* update changelog

* catch narrower range of exceptions

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joseph M. Koning <[email protected]>

* add functionality for the detailed-status command

* add tests for detailed-status

* fix linter issues

* update changelog

* general cleanup and add log statements

* slightly modify two tests

* default status renderer now uses json status format

* remove inaccurate comment

* bugfix/lsf-gpu-typo (#453)

* fix typo in batch.py that causes a bug

* change print statements to log statements

* release/1.11.1 (#454)

* Add Pytest Fixtures to Test Suite (#456)

* begin work on integration refactor; create fixtures and initial tests

* update CHANGELOG and run fix-style

* add pytest fixtures and README explaining them

* add tests to demonstrate how to use the fixtures

* move/rename some files and modify integration's README

* add password change to redis.pass file

* fix lint issues

* modify redis pwd for test server to be constant for each test

* fix lint issue only caught on github ci

* Bugfix for WEAVE CI (#457)

* begin work on integration refactor; create fixtures and initial tests

* update CHANGELOG and run fix-style

* add pytest fixtures and README explaining them

* add tests to demonstrate how to use the fixtures

* move/rename some files and modify integration's README

* add password change to redis.pass file

* fix lint issues

* modify redis pwd for test server to be constant for each test

* fix lint issue only caught on github ci

* add fix for merlin server startup

* update CHANGELOG

* bugfix/monitor-shutdown (#452)

* add celery query to see if workers still processing tasks

* fix merlin status when using redis as broker

* fix consumer count bug and run fix-style

* fix linter issues

* update changelog

* update docs for monitor

* remove unused exception I previously added

* first attempt at using pytest fixtures for monitor tests

* (partially) fix launch_workers fixture so it can be used in multiple classes

* fix linter issues and typo on pytest decorator

* update black's python version and fix style issue

* remove print statements from celeryadapter.py

* workers manager is now allowed to be used as a context manager

* add one thing to changelog and remove print statement

* Add the missing restart keyword to the specification docs. (#459)

* add Jeremy's suggestion to change vars option to output-path

* remove unnecessary lines from CHANGELOG

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joseph M. Koning <[email protected]>
Co-authored-by: Joe Koning <[email protected]>

* feature/queue info (#461)

* remove a merge conflict statement that was missed

* add queue-info functionality

* add tests for queue-info

* update CHANGELOG

* add try/except for forceful termination of test workers

* change github workflow to use py38 with black instead of py36

* run fix-style with py 3.12 and fix a typo in a test

* add filetype check for dump option

* add banner print statement

* docs/revamped status (#462)

* fix broken image link in README

* add new commands to the command line page

* add monitoring docs layout and complete status cmds page

* fix bug with dumping queue-info to files

* add docs for queue-info

* add documentation for 'query-workers'

* add reference to new query-workers docs and split a paragraph

* fix small bug with --steps option of monitor

* add documentation for monitor command

* update CHANGELOG

* fix dump-csv image for queue-info

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joseph M. Koning <[email protected]>
Co-authored-by: Joe Koning <[email protected]>

* remove a merge conflict statement that was missed

* bump version to 1.12.0

* remove a merge conflict statement that was missed

* add a 'pip freeze' call in github workflow to view reqs versions

* add new retry priority as highest task priority

* update CHANGELOG

* add in MID priority

* change default priority to use priority map MID value

* remove a merge conflict statement that was missed

* add a 'pip freeze' call in github workflow to view reqs versions

* rename the merlin server config page

* add instructions for running a cross-node workflow w/ containerized server

* update CHANGELOG

* remove a merge conflict statement that was missed

* add a 'pip freeze' call in github workflow to view reqs versions

* fix bug with dry run status

* set MANPAGER for detailed-status

* fix bug with 1 sample removing the status file

* add support for multiple workers on one step in status files

* update test suite to accommodate changes to workers in status files

* add catch and potential fix for JSONDecodeError

* fix docstring of a test

* update CHANGELOG.md

* run fix style and add Luc's suggestions

* run fix-style with python 3.12

* added additional check for status file while condensing

* add try/except to catch an error for dumping statuses

* remove a merge conflict statement that was missed

* add a 'pip freeze' call in github workflow to view reqs versions

* bump version to 1.12.1

* remove a merge conflict statement that was missed

* add a 'pip freeze' call in github workflow to view reqs versions

* remove DeepMergeException and add conflict_handler to dict_deep_merge

* add conflict handler to dict_deep_merge

* fix broken tests for detailed-status

* use caplog fixture rather than IO stream

* add ability to define module-specific fixtures

* add tests for read/write status files and conlict handling

* add caplog explanation to docstrings

* update CHANGELOG

* run fix-style

* add pytest-mock as dependency for test suite

* clean up input check in dict_deep_merge

* Add merlin version to banner

* Add python package info to and clean up 'merlin info'

* Add some unit tests

* Force GitHub runner checkout to grab the whole history, fixing CHANGELOG test bug

* Update CHANGELOG to show bugfix to CHANGELOG test

* New github action test to make sure target has been merged into source

* remove a merge conflict statement that was missed

* have status ignore nested workspaces and modify merge rules

* update CHANGELOG

* fixed issue with escape sequences in ascii art

* apply Luc's suggestion

* add setuptools as a requirement since python 3.12 doesn't have it natively

* modify unit tests for status to use pytest rather than unittest

* update CHANGELOG

* add fixtures for status testing and add nested workflow test

* update CHANGELOG

* remove a merge conflict statement that was missed

* add celery results backend patch to stop ChordErrors

* add MERLIN_RAISE_ERROR return code

* add tests to ensure chord error isn't raised

* add RAISE_ERROR to docs

* update CHANGELOG

* fix lint issues

* up the sleep time on the chord error test

* add new steps to the chord err test spec

* add tree statement to the new test for debugging

* upping sleep time to see if that fixes github action for python 3.7

* change sleep time for new test based on python version

* run fix style

* remove specific sleep time for diff python versions

* remove a merge conflict statement that was missed

* bump version to 1.12.2b1

* remove a merge conflict statement that was missed

* fix flux node allocation issue

* allow for vars to be used with nodes settings of workers/batch

* add tests for var usage with nodes

* update CHANGELOG

* run fix-style

* add a version check for flux when getting node count

* update CHANGELOG

* add major version check for flux

* Modifying task id to include directory

* remove a merge conflict statement that was missed

* add pytest coverage library and add sample_index coverage

* run fix style and add module header

* add tests for encryption modules

* add unit tests for util_sampling

* run fix-style and fix typo

* create directory for context managers and fix issue with an encryption test

* add a context manager for spinning up/down the redis server

* fix issue with path in one test

* rework CONFIG functionality for testing

* refactor config fixture so it doesn't depend on redis server to be started

* split CONFIG fixtures into rabbit and redis configs, run fix-style

* add unit tests for broker.py

* add unit tests for the Config object

* update CHANGELOG

* make CONFIG fixtures more flexible for tests

* add tests for results_backend.py

* fix lint issues for most recent changes

* fix filename issue in setup.cfg and move celeryadapter tests to integration suite

* add ssl filepaths to mysql config object

* add unit tests for configfile.py

* add tests for the utils.py file in config/

* create utilities file and constants file

* move create_dir function to utils.py

* add tests for merlin/examples/generator.py

* run fix-style and update changelog

* add a 'pip freeze' call in github workflow to view reqs versions

* re-delete the old config test files

* fix tests/bugs introduced by merging in develop

* add a unit test file for the dumper module

* begin work on server tests and modular fixtures

* start work on tests for RedisConfig

* add tests for RedisConfig object

* add tests for RedisUsers class

* change server fixtures to use redis config files

* add tests for AppYaml class

* final cleanup of server_utils

* fix lint issues

* parametrize setup examples tests

* sort example output

* ensure directory is changed back on no outdir test

* sort the specs in examples output

* fix lint issues

* start writing tests for server config

* add pytest coverage library and add sample_index coverage

* run fix style and add module header

* add tests for encryption modules

* add unit tests for util_sampling

* run fix-style and fix typo

* create directory for context managers and fix issue with an encryption test

* add a context manager for spinning up/down the redis server

* fix issue with path in one test

* rework CONFIG functionality for testing

* refactor config fixture so it doesn't depend on redis server to be started

* split CONFIG fixtures into rabbit and redis configs, run fix-style

* add unit tests for broker.py

* add unit tests for the Config object

* update CHANGELOG

* make CONFIG fixtures more flexible for tests

* add tests for results_backend.py

* fix lint issues for most recent changes

* fix filename issue in setup.cfg and move celeryadapter tests to integration suite

* add ssl filepaths to mysql config object

* add unit tests for configfile.py

* add tests for the utils.py file in config/

* create utilities file and constants file

* move create_dir function to utils.py

* add tests for merlin/examples/generator.py

* run fix-style and update changelog

* fix tests/bugs introduced by merging in develop

* add a unit test file for the dumper module

* begin work on server tests and modular fixtures

* start work on tests for RedisConfig

* add tests for RedisConfig object

* add tests for RedisUsers class

* change server fixtures to use redis config files

* add tests for AppYaml class

* final cleanup of server_utils

* fix lint issues

* parametrize setup examples tests

* sort example output

* ensure directory is changed back on no outdir test

* sort the specs in examples output

* fix lint issues

* start writing tests for server config

* bake in LC_ALL env variable setting for server cmds

* add tests for parse_redis_output

* fix issue with scope of fixture after rebase

* run fix-style

* split up create_server_config and write tests for it

* add tests for config_merlin_server function

* add tests for pull_server_config

* add tests for pull_server_image

* finish writing tests for server_config.py

* add tests for server_commands.py

* run fix-style

* update README for testing directory

* update the temp_output_directory to include python version

* mock the open.write to try to fix github CI

* ensure config dir is created

* update CHANGELOG

* add print of exception to OSError catch in pull_server_image

* change name of config_file in test that's failing

* update CHANGELOG

* add Ryan and Joe's suggestions

* update tests to use newly named functions

* fix linter issue

* update version to 1.12.2

* fix style issue