(#14833) docs: add working with dependencies to give a lot more guidance

* docs: fix TOC * docs: first pass at `requirements` * review + fill in generate to build details * tweak my choice of words more * move version ranges docs * point hooks to full explainer * cleanup * cleanup more * Apply suggestions from code review Co-authored-by: SSE4 <[email protected]> Co-authored-by: SSE4 <[email protected]>
conan-io · Dec 20, 2022 · 1be2642 · 1be2642
1 parent 6f24e01
commit 1be2642
Show file tree

Hide file tree

Showing 5 changed files with 180 additions and 53 deletions.
diff --git a/docs/adding_packages/README.md b/docs/adding_packages/README.md
@@ -27,8 +27,6 @@ You can follow the three steps (:one: :two: :three:) described below! :tada:
       * [Components](#components-1)
     * [No Upstream Build Scripts](#no-upstream-build-scripts)
     * [System Packages](#system-packages)
-    * [Verifying Dependency Version](#verifying-dependency-version)
-    * [Verifying Dependency Options](#verifying-dependency-options)
   * [Test the recipe locally](#test-the-recipe-locally)
     * [Hooks](#hooks)
     * [Linters](#linters)<!-- endToc -->
@@ -211,30 +209,6 @@ pacman, brew, choco) and install packages which are missing on Conan Center but
 
 As example there is [xorg](https://github.com/conan-io/conan-center-index/blob/master/recipes/xorg/all/conanfile.py). Also, it will require an exception rule for [conan-center hook](https://github.com/conan-io/hooks#conan-center), a [pull request](https://github.com/conan-io/hooks/pulls) should be open to allow it over the KB-H032.
 
-### Verifying Dependency Version
-
-Some project requirements need to respect a version constraint. This can be enforced in a recipe by accessing the [`dependencies`](https://docs.conan.io/en/latest/reference/conanfile/dependencies.html) attribute.
-An example of this can be found in the [fcl recipe](https://github.com/conan-io/conan-center-index/blob/1b6b496fe9a9be4714f8a0db45274c29b0314fe3/recipes/fcl/all/conanfile.py#L80).
-
-```py
-def validate(self):
-    foobar = self.dependencies["foobar"]
-    if self.info.options.shared and Version(foobar.ref.version) < "1.2":
-        raise ConanInvalidConfiguration(f"{self.ref} requires 'foobar' >=1.2 to be built as shared.")
-```
-
-### Verifying Dependency Options
-
-Certain projects are dependant on the configuration (a.k.a options) of a dependency. This can be enforced in a recipe by accessing the [`options`](https://docs.conan.io/en/latest/reference/conanfile/attributes.html#options) attribute.
-An example of this can be found in the [sdl_image recipe](https://github.com/conan-io/conan-center-index/blob/1b6b496fe9a9be4714f8a0db45274c29b0314fe3/recipes/sdl_image/all/conanfile.py#L93).
-
-```py
-    def validate(self):
-        foobar = self.dependencies["foobar"]
-        if not foobar.options.enable_feature:
-            raise ConanInvalidConfiguration(f"The project {self.ref} requires foobar:enable_feature=True.")
-```
-
 ## Test the recipe locally
 
 ### Hooks

diff --git a/docs/adding_packages/dependencies.md b/docs/adding_packages/dependencies.md
@@ -1,20 +1,181 @@
 # Dependencies
 
-This section outlines all the practices and guidelines for the `requirements()` and `build_requirements()` methods. This includes everything from "vendored" dependencies to
-when and how the versions could be changed.
+This section outlines all the practices and guidelines for the `requirements()` and `build_requirements()` methods. This includes everything
+from handling "vendored" dependencies to what versions should be used.
 
 <!-- toc -->
-## Contents
+## Contents<!-- endToc -->
 
-  * [Rules](#rules)<!-- endToc -->
+## List Dependencies
 
-## Rules
+Since all ConanCenterIndex recipes are to build and/or package projects they are exclusively done in [`conanfile.py`](https://docs.conan.io/en/latest/reference/conanfile.html). This offers a few
+ways to add requirements. The most common way is [requirements](https://docs.conan.io/en/latest/reference/conanfile/methods.html#requirements):
 
-* [Version range](https://docs.conan.io/en/latest/versioning/version_ranges.html) is not allowed.
+```py
+    def requirements(self):
+        self.requires("fmt/9.1.0")
+```
+
+> **Note**: With Conan 2.0, you'll also need to pay attention to new properties like the `transitive_header` attributed which is
+> needed when a project include a dependencies header files in its public headers.
+
+When a project supports a range of version of a dependency, it's generally advised to pick the **most recent available in ConanCenter**.
+This helps ensure there are fewer conflicts with other, up to-date, recipes that share the same requirement.
+
+### Optional Requirements
+
+Many projects support enabling certain features by adding dependencies. In ConanCenterIndex this is done by adding an option, see
+[naming recommendation](conanfile_attributes.md#recommended-names), which should be set to match the upstream project's by default.
+
+```py
+class ExampleConan(ConanFile):
+    options = {
+        "with_zlib": [True, False], # Possible values
+    }
+    default_options = {
+        "with_zlib": True, # Should match upstream's CMakeLists.txt `option(...)`
+    }
+
+    def requirements(self):
+        if self.options.with_zlib:
+            self.requires("zlib/1.2.13")
+```
+
+If a dependency was added (or removed) with a release, then the `if` condition could check [`self.version`](https://docs.conan.io/en/latest/reference/conanfile/attributes.html#version). Another common case is
+`self.settings.os` dependant requirements which need to be added for certain plaforms.
+
+### Build Requirements
+
+In ConanCenter we only assume
+[CMake is available](../faqs.md#why-recipes-that-use-build-tools-like-cmake-that-have-packages-in-conan-center-do-not-use-it-as-a-build-require-by-default).
+If a project requires any other specific tool, those can be added as well. We like to do this with [build_requirements](https://docs.conan.io/en/latest/reference/conanfile/methods.html#build-requirements):
+
+```py
+    def build_requirements(self):
+        self.tool_requires("ninja/1.1.0")
+```
+
+## Accessing Dependencies
+
+It's fairly common to need to pass information from a dependency to the project. This is the job of the [`generate()`](https://docs.conan.io/en/latest/reference/conanfile/methods.html#generate) method. This
+is generally covered by the built-in generators like [`CMakeDeps`](https://docs.conan.io/en/latest/reference/conanfile/tools/cmake/cmakedeps.html)
+However the [`self.dependencies`](https://docs.conan.io/en/latest/reference/conanfile/dependencies.html?highlight=generate) are available.
+
+Alternatively, a project may depend on a specific versions or configuration of a dependency. This use case is again covered by the
+[`self.dependencies`](https://docs.conan.io/en/latest/reference/conanfile/dependencies.html?highlight=validate) within the
+[`validate()`](https://docs.conan.io/en/latest/reference/conanfile/methods.html#validate) method. Additionally it's possible to suggest the option's values while the graph is built through [`configure()`](https://docs.conan.io/en/latest/reference/conanfile/methods.html#configure-config-options)
+this is not guaranteed and not a common practice.
+
+### Handling Requirement's Options
+
+Forcing options of dependencies inside a ConanCenter should be avoided, except if it is mandatory for the library to build.
+Our general belief is the users input should be the most important; it's unexpected for command line arguments to be over ruled
+by specifc recipes.
+
+You need to use the [`validate()`](https://docs.conan.io/en/latest/reference/conanfile/methods.html#validate) method in order to ensure they check after the Conan graph is completely built.
+
+Certain projects are dependent on the configuration (also known as options) of a dependency. This can be enforced in a recipe by
+accessing the [`options`](https://docs.conan.io/en/latest/reference/conanfile/dependencies.html?highlight=options) field of
+the dependency.
+
+```py
+  def configure(self):
+      self.options["foobar"].enable_feature = True # This will still allow users to override this option
+
+  def validate(self):
+      if not self.dependencies["foobar"].options.enable_feature:
+          raise ConanInvalidConfiguration(f"{self.ref} requires foobar/*:enable_feature=True.")
+```
+
+### Verifying Dependency's Version
+
+Some project requirements need to respect a version constraint, this can be done as follows:
+
+```py
+def validate(self):
+    if Version(self.dependencies["foobar"].ref.version) < "1.2":
+        raise ConanInvalidConfiguration(f"{self.ref} requires [foobar>=1.2] to build and work.")
+```
+
+### Passing Requirement's info to `build()`
+
+The [`self.dependencies`](https://docs.conan.io/en/latest/reference/conanfile/dependencies.html) are limited to [`generate()`](https://docs.conan.io/en/latest/reference/conanfile/methods.html#generate) and [`validate()`](https://docs.conan.io/en/latest/reference/conanfile/methods.html#validate). This means configuring a projects build scripts
+is a touch more complicated when working with unsupported build scripts.
+
+In general, with [CMake](https://cmake.org/) project, this can be very simple with the [`CMakeToolchain`](https://docs.conan.io/en/latest/reference/conanfile/tools/cmake/cmaketoolchain.html), such as:
+
+```py
+    def generate(self):
+        tc = CMakeToolchain(self)
+        # deps_cpp_info, deps_env_info and deps_user_info are no longer used
+        if self.dependencies["dependency"].options.foobar:
+            tc.variables["DEPENDENCY_LIBPATH"] = self.dependencies["dependency"].cpp_info.libdirs
+```
+
+This pattern can be recreated for less common build system by, generating a script to call configure or capture the
+required values in a YAML files for example.
+
+> **Note**: This needs to be saved to disk because the [`conan install`](https://docs.conan.io/en/latest/reference/commands/consumer/install.html) and [`conan build`](https://docs.conan.io/en/latest/reference/commands/development/build.html) commands can be separated when
+> developing packages so for this reason the `class` may not persists the information. This is a very common workflow,
+> even used in ConanCenter in other areas such as testing.
+
+```py
+from conan import ConanFile
+from conan.tools.files import save, load
+
+
+class ExampleConan(ConanFile):
+    _optional_build_args = []
+
+    @property
+    def _optional_build_args_filename(self):
+        return os.path.join(self.recipe_folder, self.folders.generators, "build_args.yml")
+
+    def generate(self):
+        # This is required as `self.dependencies` is not available in `build()` or `test()`
+        if self.dependencies["foobar"].options.with_compression:
+            self._optional_build_args.append("--enable-foobar-compression")
+
+        save(self, self._optional_build_args_filename, file)
+
+    def build(self):
+        opts_args = load(self, self._optional_build_args_filename)
+        # Some magic setup
+        self.run(f"./configure.sh {opts_args}")
+```
+
+### Overriding the provided properties from the consumer
+
+> **Note**: This was adding in [Conan 1.55](https://github.com/conan-io/conan/pull/12609) to the generators... we need to
+> write docs for when that's available
+
+## Adherence to Build Service
+
+It's very rare we layout "rules", most often it's guidelines, however in order to ensure graph and the package generated are usable
+for consumer, we do impose some limits on Conan features to provide a smoother first taste to using Conan.
+
+> **Note**: These are very specific to the ConanCenter being the default remote and may not be relevant to your specifc use case.
+
+* [Version ranges](https://docs.conan.io/en/latest/versioning/version_ranges.html) are not allowed.
 * Specify explicit [RREV](https://docs.conan.io/en/latest/versioning/revisions.html) (recipe revision) of dependencies is not allowed.
-* Vendoring in library source code should be removed (best effort) to avoid potential ODR violations. If upstream takes care to rename
-  symbols, it may be acceptable.
 * Only ConanCenter recipes are allowed in `requires`/`requirements()` and `build_requires`/`build_requirements()`.
-* If a requirement is conditional, this condition must not depend on [build context](https://docs.conan.io/en/1.35/devtools/build_requires.html#build-and-host-contexts). Build requirements don't have this constraint.
-* Forcing options of dependencies inside a recipe should be avoided, except if it is mandatory for the library - in which case it must
-  be enforced through the `validate()` methods.
+* [`python_requires`](https://docs.conan.io/en/latest/reference/conanfile/other.html#python-requires) are not allowed.
+
+### Version Ranges
+
+Version ranges are a useful Conan feature, [documentation here](https://docs.conan.io/en/latest/versioning/version_ranges.html). However,
+in the context of ConanCenter they pose a few key challenges when being used generally to consume packages, most notably:
+
+* Non-Deterministic `package-id`: With version ranges the newest compatible package may yield a different `package_id` than the one built
+  and published by ConanCenter resulting in frustrating error "no binaries found". For more context
+  see [this excellent explanation](https://github.com/conan-io/conan-center-index/pull/8831#issuecomment-1024526780).
+
+* Build Reproducibility: If consumers try to download and build the recipe at a later time, it may resolve to a different package version
+  that may generate a different binary (that may or may not be compatible). In order to prevent these types of issues, we have decided to
+  only allow exact requirements versions. This is a complicated issue,
+  [check this thread](https://github.com/conan-io/conan-center-index/pull/9140#discussion_r795461547) for more information.
+
+## Handling "internal" dependencies
+
+Vendoring in library source code should be removed (best effort) to avoid potential ODR violations. If upstream takes care to rename
+symbols, it may be acceptable.
diff --git a/docs/adding_packages/test_packages.md b/docs/adding_packages/test_packages.md
@@ -8,12 +8,12 @@ themselves. It's possible to have ConanCenter run `conan test` on more then one
 <!-- toc -->
 ## Contents
 
-    * [Files and Structure](#files-and-structure)
-    * [CMake targets](#cmake-targets)
-    * [Testing more generators with `test_<something>`](#testing-more-generators-with-test_something)
-    * [Testing CMake variables from FindModules](#testing-cmake-variables-from-findmodules)
-    * [How it works](#how-it-works)
-    * [Minimalist Source Code](#minimalist-source-code)<!-- endToc -->
+  * [Files and Structure](#files-and-structure)
+  * [CMake targets](#cmake-targets)
+  * [Testing more generators with `test_<something>`](#testing-more-generators-with-test_something)
+  * [Testing CMake variables from FindModules](#testing-cmake-variables-from-findmodules)
+  * [How it works](#how-it-works)
+  * [Minimalist Source Code](#minimalist-source-code)<!-- endToc -->
 
 ### Files and Structure
 

diff --git a/docs/error_knowledge_base.md b/docs/error_knowledge_base.md
@@ -63,7 +63,7 @@ Here we use [configure()](https://docs.conan.io/en/latest/reference/conanfile/me
 
 #### **<a name="KB-H008">#KB-H008</a>: "VERSION RANGES"**
 
-It is not allowed to use [version ranges](https://docs.conan.io/en/latest/versioning/version_ranges.html) for the recipes in Conan center, where the dependency graph should be deterministic.
+See [Dependencies Version Ranges](adding_packages/dependencies.md#version-ranges) for details.
 
 #### **<a name="KB-H009">#KB-H009</a>: "RECIPE FOLDER SIZE"**
 

diff --git a/docs/faqs.md b/docs/faqs.md
@@ -354,15 +354,7 @@ Keep reading in the [consuming recipes section](consuming_recipes.md#isolate-you
 
 ## Why are version ranges not allowed?
 
-Version ranges are a useful Conan feature, find the documentation [here](https://docs.conan.io/en/latest/versioning/version_ranges.html). However, in the context of ConanCenter they pose a few key challenges, most notably:
-
-- Non-Deterministic `package-id`
-
-With version ranges the newest compatible package may yield a different package-id than the one built and published by ConanCenter resulting in frustrating error "no binaries found". For more context see [this excellent explanation](https://github.com/conan-io/conan-center-index/pull/8831#issuecomment-1024526780).
-
-- Build Reproducibility
-
-If consumers try to download and build the recipe at a later time, it may resolve to a different package version that may generate a different binary (that may or may not be compatible). In order to prevent these types of issues, we have decided to only allow exact requirements versions. This is a complicated issue, check [this thread](https://github.com/conan-io/conan-center-index/pull/9140#discussion_r795461547) for more.
+See [Dependencies Version Ranges](adding_packages/dependencies.md#version-ranges) for details.
 
 ## How to consume a graph of shared libraries?