Tests are too slow

[Luthaf]

Running the full test suite already takes more than 90s, which is long enough to get distracted between tests runs. Ideally we should cut these down to enable faster development!

Here is a list of 20 slowest tests: tox -e tests -- --durations=20

47.99s call     tests/cli/test_train_model.py::test_train_issue_290
20.10s call     tests/cli/test_train_model.py::test_train_explicit_validation_test[True-True-2]
18.95s call     tests/cli/test_train_model.py::test_continue_different_dataset
18.35s call     tests/cli/test_train_model.py::test_empty_test_set
18.15s call     tests/cli/test_train_model.py::test_train[None]
17.69s call     tests/cli/test_train_model.py::test_model_consistency_with_seed[None]
16.96s call     tests/cli/test_train_model.py::test_train_explicit_validation_test[True-True-1]
15.92s call     tests/cli/test_train_model.py::test_train_multiple_datasets
14.87s call     tests/cli/test_train_model.py::test_continue
14.32s call     tests/cli/test_eval_model.py::test_eval_multi_dataset
13.99s call     tests/cli/test_train_model.py::test_train_explicit_validation_test[False-False-2]
13.20s call     tests/cli/test_eval_model.py::test_eval_no_targets
12.74s call     tests/cli/test_train_model.py::test_command_line_override[architecture.training.num_epochs=2]
12.56s call     tests/cli/test_train_model.py::test_train[mymodel.pt]
11.99s call     tests/cli/test_train_model.py::test_model_consistency_with_seed[1234]
11.96s call     tests/cli/test_train_model.py::test_train_from_options_restart_yaml
11.89s call     tests/cli/test_eval_model.py::test_eval_cli
9.54s call     tests/cli/test_eval_model.py::test_eval_export
9.45s call     tests/cli/test_train_model.py::test_train_explicit_validation_test[True-False-2]
9.39s call     tests/cli/test_train_model.py::test_command_line_override[architecture.training.num_epochs=2 architecture.training.batch_size=3]

Ideally, no single test should take longer than 5s. If needed, we could add a new dummy architecture which is only used for tests (maybe just a CompositionModel).

[PicoCentauri]

Yes, I agree. A good dummy model could be composition plus ZBL!