`install --skip-keys="%pkg-a%" pulling in transitive dependency of pkg-a

[130s]

Originally from space-ros/space-ros#189 (reply in thread)

Clarification first

Is rosdep install --skip-keys="pkg-a pkg-b" meant to skip not just the pkgs pkg-a, pkg-b but also the dependency of those 2 pkgs? That's what I expect, but it's not clear from its help.

Citing help on humble

  --skip-keys=SKIP_KEYS
                        Affects the 'check' and 'install' verbs. The specified
                        rosdep keys will be ignored, i.e. not resolved and not
                        installed. The option can be supplied multiple times.
                        A space separated list of rosdep keys can also be
                        passed as a string. A more permanent solution to
                        locally ignore a rosdep key is creating a local rosdep
                        rule with an empty list of packages (include it in
                        /etc/ros/rosdep/sources.list.d/ before the defaults).

If the understanding above is wrong, don't read further, this ticket should be immediately closed.

Issue

rosdep install --skip-keys="pkg-a pkg-b" seems to skip pkg-a, pkg-b pkgs but still installs the dependency of those 2 pkgs.

Verifying the issue statement

In the test done (explained in the folded clause), I expect the transitive dependency (opengl-related pkgs) won't get installed.

Detail of the verification step

Test I did is a bit too elaborated unnecessarilly. Hope it's still valid test to verify the issue...

Spec of a quick test I did:

• Done on osrf/space-ros:humble-2024.07.0 Docker image.

• In this test, I run rosdep with rviz pkg suite that is customized for this test only. And see if OPENGL-related pkgs get installed. Dep tree in this custom suite looks like the following graph (pkgs that are not mentioned here are removed for the sake of the test) so opengl is a transitive dependency, and if it does get installed that's the reported issue:

  rviz2
  |- rviz_ogre_vendor
    |- opengl
    |- :
  |- rviz_rendering
    |- libqt5-opengl
    |- :
  

  More detail of the test

  • I deleted all other pkgs. Also converted rviz2 pkg to only depend on these 2 pkgs i.e.:

    <buildtool_depend>ament_cmake</buildtool_depend>
    <depend>rviz_ogre_vendor</depend>
    <depend>rviz_rendering</depend>
    

• rosdep key of opengl seems to install libgl1-mesa-dev, libglu1-mesa-dev on Ubuntu.

• On this Docker img, the pkgs in question are not installed yet.

  $ apt-cache policy libglu1-mesa-dev
  libglu1-mesa-dev:
    Installed: (none)
    Candidate: 9.0.2-1
  :
  spaceros-user@130s-p16s:~/test$ apt-cache policy libgl1-mesa-dev
  libgl1-mesa-dev:
    Installed: (none)
    Candidate: 24.2.0~kisak3~j
    Version table:
       24.2.0~kisak3~j 500
          500 https://ppa.launchpadcontent.net/kisak/kisak-mesa/ubuntu jammy/main amd64 Packages
  :
  

Prep

$ mkdir ~/test && cd ~/test

$ echo "OPT-1 Manually prepping test pkg suite" && git clone https://github.com/ros2/rviz.git && cd rviz && git checkout humble && echo "Removing except rviz_ogre_vendor  rviz_rendering" && rm -fr docs  LICENSE  README.md   rviz_assimp_vendor  rviz_common  rviz_default_plugins    rviz_rendering_tests  rviz_visual_testing_framework

OR

$ echo "OPT-2 premaid pkg suite" && git clone https://github.com/130s/rviz.git && cd rviz && git checkout humble_test-rosdep

$ cd - && rosdep update
$ rosdep install --from-paths . --ignore-src --skip-keys="rviz_ogre_vendor rviz_rendering"

:

Result

I do see the 2 pkgs (the rosdep key opengl corresponds to) get installed.

$ apt-cache policy libgl1-mesa-dev
libgl1-mesa-dev:
  Installed: 24.2.0~kisak3~j
  Candidate: 24.2.0~kisak3~j
  Version table:

$ apt-cache policy libglu1-mesa-dev
libglu1-mesa-dev:
  Installed: 9.0.2-1
  Candidate: 9.0.2-1
  Version table:
 *** 9.0.2-1 500
        500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages
        100 /var/lib/dpkg/status

[mkhansenbot]

@cottsay can you respond to this? This is affecting multiple users

[cottsay]

If I'm understanding this correctly, you're viewing the collection of packages as a hierarchy, and you're trying to ask rosdep to ignore a branch of the hierarchy and omit dependencies which are only present in the pruned branch.

As I understand it, rosdep does not operate on the packages as a hierarchy like this. It "discovers" packages (typically using a directory walk over --from-paths), and enumerates the dependencies of those discovered packages. The results are de-duplicated and resolved against the rosdep database to system packages. Prior to resolution, the --ignore-src option can filter out any dependencies which also match the name of one of the discovered packages. This may give the illusion that rosdep is establishing relationships between these packages, but I just don't think it operates like that.

If we backtrack a bit here, maybe what you're asking for is not to ignore a branch of the dependency graph but rather to ignore specific packages which were discovered. Essentially a plain filter against the discovered packages. The --skip-keys option affects discovered dependency keys, and just doesn't affect package discovery. It could be specifically extended to do so, but I'm personally not convinced that it's more intuitive behavior overall. Regardless of which behavior you consider to be more intuitive, I don't feel that the current behavior qualifies as a bug.

If the directory walking done by --from-paths is discovering more packages than you want it to, I think you could write a wrapper of sorts to pass explicit package paths to --from-paths which don't contain the packages you're trying to ignore.

[130s]

Thanks for the discussion. Let me give a bit more clarification about the problem, re-using the test I did in OP. In the test I supplied above, have these the source of 2 pkgs in the current dir.

|- rviz_ogre_vendor
  |- opengl
  |- :
|- rviz_rendering
  |- libqt5-opengl
  |- :

(Note the 2nd indented layer and the rest in this diagram above shows the dependency, not the folder structure).

Running the following cmd seems to install the dependency of these 2 packages:

rosdep update && rosdep install --from-paths . --ignore-src --skip-keys="rviz_ogre_vendor rviz_rendering"

This makes me wonder what is --skip-keys option exactly skipping in rosdep sense.

maybe what you're asking for is not to ignore a branch of the dependency graph but rather to ignore specific packages which were discovered.

In the example I just reiterated above, what I expect is neither rviz_{ogre_vendor, rendering} pkgs nor their dependencies to get installed.

[cottsay]

Thanks for clarifying.

This tool enumerates two separate collections: discovered packages, and dependencies of those packages. The --skip-keys argument affects only the enumerated dependencies, and does not skip discovered packages. In your example, those two packages are discovered by the directory traversal done by --from-paths .. Unfortunately rosdep doesn't have robust package selection arguments like colcon does.

[130s]

I see. Hm, I further tested to see if the "dependency of those packages" can be skipped. Using the same test as the one in OP, however, I still see the packages in --skip-keys getting installed.

# apt-cache policy libglu1-mesa-dev
libglu1-mesa-dev:
  Installed: (none)
  Candidate: 9.0.2-1

# apt-cache policy libgl1-mesa-dev
libgl1-mesa-dev:
  Installed: (none)
  Candidate: 23.2.1-1ubuntu3.1~22.04.3

# rosdep install --from-paths . --ignore-src --skip-keys="rviz_ogre_vendor rviz_rendering libgl1-mesa-dev libglu1-mesa-dev"

# apt-cache policy libglu1-mesa-dev
libglu1-mesa-dev:
  Installed: 9.0.2-1
  Candidate: 9.0.2-1

# apt-cache policy libgl1-mesa-dev
libgl1-mesa-dev:
  Installed: 23.2.1-1ubuntu3.1~22.04.3
  Candidate: 23.2.1-1ubuntu3.1~22.04.3

• Pkg-type-A. Pkgs discovered by the directory traversal: rviz_ogre_vendor rviz_rendering
• Pkg-type-B. Dependency of Pkg-type-A: libgl1-mesa-dev libglu1-mesa-dev

If this behavior is not intended then would a patch be considered for review?

[cottsay]

Neither libgl1-mesa-dev nor libglu1-mesa-dev are rosdep keys. They are the names of system packages on Ubuntu. --skip-keys is for skipping rosdep keys.

On Ubuntu, they're brought in by the opengl key:

• https://github.com/ros/rosdistro/blob/e235917b0bb4aec47c3b6f20ab0fb3b5d02ee70c/rosdep/base.yaml#L7585
• https://github.com/ros2/rviz/blob/9bbca6e06b71e05aefbf9b6719aa311049a34074/rviz_ogre_vendor/package.xml#L33

Passing opengl to --skip-keys should ignore them, but it looks like they're dependencies of qtbase5-dev anyway so they'll probably still end up on your system even if rosdep doesn't explicitly install them.