Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update GCP docs #59

Merged
merged 7 commits into from
Jan 25, 2024
Merged

Update GCP docs #59

merged 7 commits into from
Jan 25, 2024

Conversation

nweires
Copy link
Collaborator

@nweires nweires commented Jan 18, 2024

  • Add a README for developers, including an architecture diagram.
  • Update installation instructions to include the Terraform setup option.
  • Miscellaneous other cleanups and improvements.

You can download and preview the docs changes at Checks -> Artifacts -> documentation, if you want to see exactly what the html pages will look like.

Copy link

github-actions bot commented Jan 19, 2024

File Coverage
All files 86%
base.py 91%
exc.py 57%
hpc.py 78%
local.py 70%
postprocessing.py 84%
utils.py 91%
cloud/docker_base.py 78%
sampler/base.py 79%
sampler/downselect.py 33%
sampler/precomputed.py 93%
sampler/residential_quota.py 61%
test/shared_testing_stuff.py 85%
test/test_docker.py 33%
test/test_local.py 97%
test/test_validation.py 97%
workflow_generator/base.py 90%
workflow_generator/commercial.py 53%
workflow_generator/residential_hpxml.py 86%

Minimum allowed coverage is 33%

Generated by 🐒 cobertura-action against 400accc

Copy link

@mfathollahzadeh mfathollahzadeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks Natalie!

buildstockbatch/gcp/gcp.py Show resolved Hide resolved
docs/project_defn.rst Outdated Show resolved Hide resolved
* ``batch_array_size``: Number of tasks to divide the simulations into. Max: 10000.
(e.g. ``us-central1``)
* ``batch_array_size``: Number of tasks to divide the simulations into. Tasks with fewer than 100
simulations each are recommended, especially when using spot instances. Max: 10,000.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to avoid loosing too much of simulations or anything in respect to that when preemption happens? maybe adding something like this to justify the recommendation on < 100 sims per task?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added.

docs/project_defn.rst Outdated Show resolved Hide resolved
* ``use_spot``: true or false. This tells the project whether to use
`Spot VMs <https://cloud.google.com/spot-vms>`_ for data simulations, which can reduce
costs by up to 91%. Default: false
* ``use_spot``: Optional. Whether to use `Spot VMs <https://cloud.google.com/spot-vms>`_

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should change the default to true here? thoughts?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I prefer a default of False, because 1) that's the default when creating GCP jobs directly and 2) I think it's best for the default to be the most reliable option, at least as long as using spot instances at scale continues to require manual retries sometimes.

* ``memory_mib``: `Amount of RAM`_ needed in MiB. 2048 MiB per CPU is recommended. Default:
4096.
* ``cpus``: Optional. `Number of CPUs`_ to use. Default: 2.
* ``memory_mib``: Optional. `Amount of RAM`_ needed in MiB. 2048 MiB per CPU is recommended.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably saying "at least 2048 MiB per cpu recommended ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to mention anything about the guardrails here?

docs/run_sims.rst Outdated Show resolved Hide resolved
@nweires nweires merged commit 8b691c8 into gcp Jan 25, 2024
6 checks passed
@nweires nweires deleted the natalie/docs branch January 25, 2024 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants