Refactor workflow generator

[rajeee]

This is in preparation of Introducing measure based upgrades.

Pull Request Description

Refactor the Workflow generator, especially the residential generator, to separate the shema and default values from the core logic.

Checklist

Not all may apply

• Code changes (must work)
• Tests exercising your feature/bug fix (check coverage report on Checks -> BuildStockBatch Tests -> Artifacts)
• Coverage has increased or at least not decreased. Update minimum_coverage in .github/workflows/coverage.yml as necessary.
• All other unit and integration tests passing
• Update validation for project config yaml file changes
• Update existing documentation
• Run a small batch run on Kestrel/Eagle to make sure it all works if you made changes that will affect Kestrel/Eagle
• Add to the changelog_dev.rst file and propose migration text in the pull request

[github-actions]

File	Coverage
All files	87%	✅
base.py	91%	✅
exc.py	57%	✅
hpc.py	79%	✅
local.py	70%	✅
postprocessing.py	84%	✅
utils.py	92%	✅
cloud/docker_base.py	88%	✅
sampler/base.py	78%	✅
sampler/downselect.py	33%	✅
sampler/precomputed.py	93%	✅
sampler/residential_quota.py	61%	✅
test/shared_testing_stuff.py	85%	✅
test/test_docker.py	33%	✅
test/test_local.py	97%	✅
test/test_validation.py	97%	✅
workflow_generator/base.py	89%	✅
workflow_generator/commercial/commercial.py	52%	✅
workflow_generator/residential/residential_hpxml.py	91%	✅

Minimum allowed coverage is 33%

Generated by 🐒 cobertura-action against 1f06049

[nmerket]

I haven't looked at the code really, but my recommendation in general with these sorts of changes is to strive to keep buildstockbatch backwards compatible with older versions of ResStock. Will these changes make it so this will not run ResStock with the prior workflow? If not, try to find a way to make it so it will. That could look like:

1. Creating a new workflow generator for the new workflow, but leaving the old one there alongside it. This is a bit of a maintenance burden and eventually you may want to deprecate the old one.
2. Make the workflow generator able to handle both the old and new workflows by being able to detect the difference between the two and default things in a smart way.

[rajeee]

I haven't looked at the code really, but my recommendation in general with these sorts of changes is to strive to keep buildstockbatch backwards compatible with older versions of ResStock. Will these changes make it so this will not run ResStock with the prior workflow? If not, try to find a way to make it so it will. That could look like:

1. Creating a new workflow generator for the new workflow, but leaving the old one there alongside it. This is a bit of a maintenance burden and eventually you may want to deprecate the old one.
2. Make the workflow generator able to handle both the old and new workflows by being able to detect the difference between the two and default things in a smart way.

Thanks for the suggestions. This one is a plain refactor with no functional change, but will make sure to have backward compatibility in the future PRs.

[joseph-robertson]

Nice.

[joseph-robertson]

You might want to be more descriptive in the changelog entry. Other than that, looks good to me.