Unverified Commit 9bc7474d authored by Philip Taron's avatar Philip Taron Committed by GitHub
Browse files

doc: document `pkgs<theirHost><theirTarget>` better (#506577)

parents 5454924d 9fdde482
Loading
Loading
Loading
Loading
+15 −4
Original line number Diff line number Diff line
@@ -235,13 +235,24 @@ One would think that `localSystem` and `crossSystem` overlap horribly with the t

### Implementation of dependencies {#ssec-cross-dependency-implementation}

The categories of dependencies developed in [](#ssec-cross-dependency-categorization) are specified as lists of derivations given to `mkDerivation`, as documented in [](#ssec-stdenv-dependencies). In short, each list of dependencies for `host → target` is called `deps<host><target>` (where `host`, and `target` values are either `build`, `host`, or `target`), with exceptions for backwards compatibility that `depsBuildHost` is instead called `nativeBuildInputs` and `depsHostTarget` is instead called `buildInputs`. Nixpkgs is now structured so that each `deps<host><target>` is automatically taken from `pkgs<host><target>`. (These `pkgs<host><target>`s are quite new, so there is no special case for `nativeBuildInputs` and `buildInputs`.) For example, `pkgsBuildHost.gcc` should be used at build-time, while `pkgsHostTarget.gcc` should be used at run-time.
The categories of dependencies developed in [](#ssec-cross-dependency-categorization) are specified as lists of derivations given to `mkDerivation`, as documented in [](#ssec-stdenv-dependencies). In short, each list of dependencies for `host → target` is called `deps<theirHost><theirTarget>` (where `theirHost`, and `theirTarget` values are either `build`, `host`, or `target`), with exceptions for backwards compatibility that `depsBuildHost` is instead called `nativeBuildInputs` and `depsHostTarget` is instead called `buildInputs`. Nixpkgs is now structured so that each `deps<theirHost><theirTarget>` is automatically taken from `pkgs<theirHost><theirTarget>`. (These `pkgs<theirHost><theirTarget>`s are quite new, so there is no special case for `nativeBuildInputs` and `buildInputs`.) For example, `pkgsBuildHost.gcc` should be used at build-time, while `pkgsHostTarget.openssl` should be used at run-time.

Now, for most of Nixpkgs's history, there were no `pkgs<host><target>` attributes, and most packages have not been refactored to use it explicitly. Prior to those, there were just `buildPackages`, `pkgs`, and `targetPackages`. Those are now redefined as aliases to `pkgsBuildHost`, `pkgsHostTarget`, and `pkgsTargetTarget`. It is acceptable, even recommended, to use them for libraries to show that the host platform is irrelevant.
Adjacent package sets are defined as `pkgs<theirHost><theirTarget>` attributes, where "their" represents the new attribute set, and "our" represents the "current" package set. Below is a table of adjacent stages and their aliases. See [](#variables-specifying-dependencies) for usage examples.

But before that, there was just `pkgs`, even though both `buildInputs` and `nativeBuildInputs` existed. \[Cross barely worked, and those were implemented with some hacks on `mkDerivation` to override dependencies.\] What this means is the vast majority of packages do not use any explicit package set to populate their dependencies, just using whatever `callPackage` gives them even if they do correctly sort their dependencies into the multiple lists described above. And indeed, asking that users both sort their dependencies, _and_ take them from the right attribute set, is both too onerous and redundant, so the recommended approach (for now) is to continue just categorizing by list and not using an explicit package set.
| Adjacent package set                   | Their host platform | Their target platform |
|----------------------------------------|---------------------|-----------------------|
| `pkgsBuildBuild`                       | Our build platform  | Our build platform    |
| `pkgsBuildHost` or `buildPackages`     | Our build platform  | Our host platform     |
| `pkgsBuildTarget`                      | Our build platform  | Our target platform   |
| `pkgsHostHost`                         | Our host platform   | Our host platform     |
| `pkgsHostTarget` or `pkgs`             | Our host platform   | Our target platform   |
| `pkgsTargetTarget` or `targetPackages` | Our target platform | Our target platform   |

To make this work, we "splice" together the six `pkgsFooBar` package sets and have `callPackage` actually take its arguments from that. This is currently implemented in `pkgs/top-level/splice.nix`. `mkDerivation` then, for each dependency attribute, pulls the right derivation out from the splice. This splicing can be skipped when not cross-compiling as the package sets are the same, but still is a bit slow for cross-compiling. We'd like to do something better, but haven't come up with anything yet.
Now, for most of Nixpkgs's history, there were no `pkgs<theirHost><theirTarget>` attributes, and most packages have not been refactored to use it explicitly. Prior to those, there were just `buildPackages`, `pkgs`, and `targetPackages`. Those are now redefined as aliases to `pkgsBuildHost`, `pkgsHostTarget`, and `pkgsTargetTarget`. It is acceptable, even recommended, to use them to show that only their host platform matters. That is, use `buildPackages` where any of `pkgsBuild*` would do, and `targetPackages` when any of `pkgsTarget*` would do (if we had more than just `pkgsTargetTarget`).

But before that, there was just `pkgs`, even though both `buildInputs` and `nativeBuildInputs` existed. (Cross barely worked, and those were implemented with some hacks on `mkDerivation` to override dependencies.) What this means is the vast majority of packages do not use any explicit package set to populate their dependencies, just using whatever `callPackage` gives them even if they do correctly sort their dependencies into the multiple lists described above. And indeed, asking that users both sort their dependencies, _and_ take them from the right attribute set, is both too onerous and redundant, so the recommended approach (for now) is to continue just categorizing by list and not using an explicit package set.

To make this work, we "splice" together the six `pkgs<theirHost><theirTarget>` package sets and have `callPackage` actually take its arguments from that. This is currently implemented in `pkgs/top-level/splice.nix`. `mkDerivation` then, for each dependency attribute, pulls the right derivation out from the splice. This splicing can be skipped when not cross-compiling as the package sets are the same, but still is a bit slow for cross-compiling. We'd like to do something better, but haven't come up with anything yet.

### Bootstrapping {#ssec-bootstrapping}

+1 −1
Original line number Diff line number Diff line
@@ -431,7 +431,7 @@ Overall, the unifying theme here is that propagation shouldn’t be introducing

##### `depsBuildBuild` {#var-stdenv-depsBuildBuild}

A list of dependencies whose host and target platforms are the new derivation’s build platform. These are programs and libraries used at build time that produce programs and libraries also used at build time. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it in `nativeBuildInputs` instead. The most common use of this `buildPackages.stdenv.cc`, the default C compiler for this role. That example crops up more than one might think in old commonly used C libraries.
A list of dependencies whose host and target platforms are the new derivation’s build platform. These are programs and libraries used at build time that produce programs and libraries also used at build time. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it in `nativeBuildInputs` instead. The most common use of this `buildPackages.stdenv.cc` (the compiler for `buildPackages`, which means that it's from the package set `buildPackages.buildPackages = pkgsBuildBuild`), the default C compiler for this role. That example crops up more than one might think in old commonly used C libraries.

Since these packages are able to be run at build-time, they are always added to the `PATH`, as described above. But since these packages are only guaranteed to be able to run then, they shouldn’t persist as run-time dependencies. This isn’t currently enforced, but could be in the future.