Skip to content

Commit

Permalink
Clearer documentation on how to set different loss weights
Browse files Browse the repository at this point in the history
  • Loading branch information
frostedoyster committed Jun 6, 2024
1 parent d55b82e commit 64368dd
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 4 deletions.
11 changes: 10 additions & 1 deletion docs/src/architectures/alchemical-model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,15 @@ hyperparameters to tune are (in decreasing order of importance):
This hyperparameter controls the size and depth of the descriptors and the neural
network. In general, increasing this might lead to better accuracy,
especially on larger datasets, at the cost of increased training and evaluation time.
- ``loss_weights``: This controls the weighting of different contributions to the loss
(e.g., energy, forces, virial, etc.). The default values work well for most datasets,
but they might need to be adjusted. For example, to set a weight of 1.0 for the energy
and 0.1 for the forces, you can set the following in the ``options.yaml`` file:
```yaml
loss_weights:
energy: 1.0
forces: 0.1
```


Architecture Hyperparameters
Expand Down Expand Up @@ -110,7 +119,7 @@ bpnn

training
########
The parameters for the training loop are
The hyperparameters for training are

:param batch_size: batch size
:param num_epochs: number of training epochs
Expand Down
12 changes: 9 additions & 3 deletions docs/src/architectures/soap-bpnn.rst
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ bpnn

training
########
The parameters for the training loop are
The parameters for training are

:param batch_size: batch size
:param num_epochs: number of training epochs
Expand All @@ -111,7 +111,7 @@ The parameters for the training loop are
any case, the final summary will be per-structure.
:param loss_weights: Specifies the weights to be used in the loss for each target. The
weights should be a dictionary of floats, one for each target. All missing targets
are assigned a weight of 1.0.
are assigned a weight of 1.0. For example, for a



Expand Down Expand Up @@ -148,7 +148,13 @@ hyperparameters to tune are (in decreasing order of importance):
might need to be adjusted for specific datasets.
- ``loss_weights``: This controls the weighting of different contributions to the loss
(e.g., energy, forces, virial, etc.). The default values work well for most datasets,
but they might need to be adjusted.
but they might need to be adjusted. For example, to set a weight of 1.0 for the energy
and 0.1 for the forces, you can set the following in the ``options.yaml`` file:
```yaml
loss_weights:
energy: 1.0
forces: 0.1
```
- ``layernorm``: Whether to use layer normalization before the neural network. Setting
this hyperparameter to ``false`` will lead to slower convergence of training, but
might lead to better generalization outside of the training set distribution.
Expand Down

0 comments on commit 64368dd

Please sign in to comment.