Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Appending to Defaults List via command line doesn't work if config is in subfolder #2935

Open
2 tasks done
siferati opened this issue Aug 6, 2024 · 0 comments
Open
2 tasks done
Labels
bug Something isn't working

Comments

@siferati
Copy link

siferati commented Aug 6, 2024

🐛 Bug

Description

When overriding config groups in the terminal, hydra interprets them as being in the global package.
This has been stated in different issues:

However, this doesn't seem to work when appending to configs loaded via --config-name=subfolder/config_name.
Hydra takes the given override as a relative path and fails to append it (see below for minimal example)

Checklist

  • I checked on the latest version of Hydra
  • I created a minimal repro (See this for tips).

To reproduce

image

# see vanilla config
> python main.py --config-name=server/alpha
db:
  driver: mysql
  user: omry
  password: secret
name: alpha

# override mysql db with postgresql -- works as expected
> python main.py --config-name=server/alpha db=postgresql
db:
  driver: postgresql
  user: postgres_user
  password: drowssap
  timeout: 10
name: alpha

# append a new db configuration -- doesn't work because unlike override, append is treating postgresql as relative path
> python main.py --config-name=server/alpha +db@db_2=postgresql
hydra.errors.MissingConfigException: In 'server/alpha': Could not find 'server/db/postgresql'

Stack trace/error message:

Traceback (most recent call last):
  File "test/main.py", line 11, in <module>
    my_app()
  File "/virtual_env/lib/python3.10/site-packages/hydra/main.py", line 90, in decorated_main
    _run_hydra(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/utils.py", line 222, in run_and_report
    raise ex
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/utils.py", line 219, in run_and_report
    return func()
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 105, in run
    cfg = self.compose_config(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 594, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/config_loader_impl.py", line 142, in load_configuration
    return self._load_configuration_impl(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/config_loader_impl.py", line 253, in _load_configuration_impl
    defaults_list = create_defaults_list(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 746, in create_defaults_list
    defaults, tree = _create_defaults_list(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 716, in _create_defaults_list
    defaults_tree = _create_defaults_tree(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 356, in _create_defaults_tree
    ret = _create_defaults_tree_impl(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 457, in _create_defaults_tree_impl
    return _expand_virtual_root(repo, root, overrides, skip_missing)
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 280, in _expand_virtual_root
    subtree = _create_defaults_tree_impl(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 573, in _create_defaults_tree_impl
    add_child(children, new_root)
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 520, in add_child
    subtree_ = _create_defaults_tree_impl(
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 488, in _create_defaults_tree_impl
    config_not_found_error(repo=repo, tree=root)
  File "/virtual_env/lib/python3.10/site-packages/hydra/_internal/defaults_list.py", line 800, in config_not_found_error
    raise MissingConfigException(
hydra.errors.MissingConfigException: In 'server/alpha': Could not find 'server/db/postgresql'

Expected Behavior

Appending treats the given config as a global package, just like when overriding.
The following command should work instead of resulting in an error:

> python main.py --config-name=server/alpha +db@db_2=postgresql
db:
  driver: mysql
  user: omry
  password: secret
name: alpha
db_2:
  driver: postgresql
  user: postgres_user
  password: drowssap
  timeout: 10

System information

  • Hydra Version : 1.3.0 (tested 1.3.2 too, same problem)
  • Python version : 3.10.12
  • Virtual environment type and version : virtualenv 20.16.5
  • Operating system : Ubuntu 22.04
@siferati siferati added the bug Something isn't working label Aug 6, 2024
@siferati siferati changed the title [Bug] Modifying the Defaults List doesn't work if config is in subfolder [Bug] Appending to Defaults List via command line doesn't work if config is in subfolder Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant