Merge 72469b39 into haskell-updates

The code in this directory is used by the [eval.yml](../../.github/workflows/eval.yml) GitHub Actions workflow to evaluate the majority of Nixpkgs for all PRs, effectively making sure that when the development branches are processed by Hydra, no evaluation failures are encountered.

Furthermore it also allows local evaluation using

Furthermore it also allows local evaluation using:

```

nix-build ci -A eval.full \

  --max-jobs 4 \

  --cores 2 \

  --arg chunkSize 10000 \

  --arg evalSystems '["x86_64-linux" "aarch64-darwin"]'

nix-build ci -A eval.baseline

```

The most important two arguments are:

- `--arg evalSystems`: The set of systems for which `nixpkgs` should be evaluated.

  Defaults to the four official platforms (`x86_64-linux`, `aarch64-linux`, `x86_64-darwin` and `aarch64-darwin`).

  Example: `--arg evalSystems '["x86_64-linux" "aarch64-darwin"]'`

- `--arg quickTest`: Enables testing a single chunk of the current system only for quick iteration.

  Example: `--arg quickTest true`

The following arguments can be used to fine-tune performance:

- `--max-jobs`: The maximum number of derivations to run at the same time.

  Only each [supported system](../supportedSystems.json) gets a separate derivation, so it doesn't make sense to set this higher than that number.

- `--cores`: The number of cores to use for each job.

  Recommended to set this to the amount of cores on your system divided by `--max-jobs`.

- `chunkSize`: The number of attributes that are evaluated simultaneously on a single core.

- `--arg chunkSize`: The number of attributes that are evaluated simultaneously on a single core.

  Lowering this decreases memory usage at the cost of increased evaluation time.

  If this is too high, there won't be enough chunks to process them in parallel, and will also increase evaluation time.

- `evalSystems`: The set of systems for which `nixpkgs` should be evaluated.

  Defaults to the four official platforms (`x86_64-linux`, `aarch64-linux`, `x86_64-darwin` and `aarch64-darwin`).

A good default is to set `chunkSize` to 10000, which leads to about 3.6GB max memory usage per core, so suitable for fully utilising machines with 4 cores and 16GB memory, 8 cores and 32GB memory or 16 cores and 64GB memory.

  The default is 5000.

  Example: `--arg chunkSize 10000`

Note that 16GB memory is the recommended minimum, while with less than 8GB memory evaluation time suffers greatly.

## Local eval with rebuilds / comparison

To compare two commits locally, first run the following on the baseline commit:

```

BASELINE=$(nix-build ci -A eval.baseline --no-out-link)

```

Then, on the commit with your changes:

```

nix-build ci -A eval.full --arg baseline $BASELINE

```

Keep in mind to otherwise pass the same set of arguments for both commands (`evalSystems`, `quickTest`, `chunkSize`).

  compare = callPackage ./compare { };

  baseline =

    {

      # Whether to evaluate on a specific set of systems, by default all are evaluated

      evalSystems ? if quickTest then [ "x86_64-linux" ] else supportedSystems,

      # The number of attributes per chunk, see ./README.md for more info.

      chunkSize ? 5000,

      quickTest ? false,

    }:

    symlinkJoin {

      name = "nixpkgs-eval-baseline";

      paths = map (

        evalSystem:

        singleSystem {

          inherit quickTest evalSystem chunkSize;

        }

      ) evalSystems;

    };

  full =

    {

      # Whether to evaluate on a specific set of systems, by default all are evaluated

      evalSystems ? if quickTest then [ "x86_64-linux" ] else supportedSystems,

      # The number of attributes per chunk, see ./README.md for more info.

      chunkSize,

      chunkSize ? 5000,

      quickTest ? false,

      baseline,

    }:

    let

      diffs = symlinkJoin {

        name = "diffs";

        name = "nixpkgs-eval-diffs";

        paths = map (

          evalSystem:

          let

            eval = singleSystem {

              inherit quickTest evalSystem chunkSize;

            };

          in

          diff {

            inherit evalSystem;

            # Local "full" evaluation doesn't do a real diff.

            beforeDir = eval;

            afterDir = eval;

            beforeDir = baseline;

            afterDir = singleSystem {

              inherit quickTest evalSystem chunkSize;

            };

          }

        ) evalSystems;

      };

    combine

    compare

    # The above three are used by separate VMs in a GitHub workflow,

    # while the below is intended for testing on a single local machine

    # while the below are intended for testing on a single local machine

    baseline

    full

    ;

}

- GCC 9, 10, 11, and 12 have been removed, as they have reached end‐of‐life upstream and are no longer supported.

- `base16-builder` node package has been removed due to lack of upstream maintenance.

- `buildGoModule` now warns if `<pkg>.passthru.overrideModAttrs` is lost during the overriding of its result packages.

- `gentium` package now provides `Gentium-*.ttf` files, and not `GentiumPlus-*.ttf` files like before. The font identifiers `Gentium Plus*` are available in the `gentium-plus` package, and if you want to use the more recently updated package `gentium` [by sil](https://software.sil.org/gentium/), you should update your configuration files to use the `Gentium` font identifier.

- `space-orbit` package has been removed due to lack of upstream maintenance. Debian upstream stopped tracking it in 2011.

- Derivations setting both `separateDebugInfo` and one of `allowedReferences`, `allowedRequisites`, `disallowedReferences` or `disallowedRequisites` must now set `__structuredAttrs` to `true`. The effect of reference whitelisting or blacklisting will be disabled on the `debug` output created by `separateDebugInfo`.

## Installing broken packages {#sec-allow-broken}

There are two ways to try compiling a package which has been marked as broken.

There are several ways to try compiling a package which has been marked as broken.

-   For allowing the build of a broken package once, you can use an environment variable for a single invocation of the nix tools:

    $ export NIXPKGS_ALLOW_BROKEN=1

    ```

-   For permanently allowing broken packages to be built, you may add `allowBroken = true;` to your user's configuration file, like this:

-   For permanently allowing broken packages that match some condition to be built, you may add `allowBrokenPredicate` to your user's configuration file with the desired condition, for example:

    ```nix

    {

      allowBrokenPredicate = pkg: builtins.elem (pkgs.lib.getName pkg) [ "hello" ];

    }

    ```

-   For permanently allowing all broken packages to be built, you may add `allowBroken = true;` to your user's configuration file, like this:

    ```nix

    { allowBroken = true; }

      files = map (def: def.file) res.defsFinal;

      definitionsWithLocations = res.defsFinal;

      inherit (res) isDefined;

      inherit (res.checkedAndMerged) valueMeta;

      # This allows options to be correctly displayed using `${options.path.to.it}`

      __toString = _: showOption loc;

    };

    # Type-check the remaining definitions, and merge them. Or throw if no definitions.

    mergedValue =

      if isDefined then

        if all (def: type.check def.value) defsFinal then

        if type.merge ? v2 then

          # check and merge share the same closure

          # .headError is either not-present, null, or a string describing the error

          if checkedAndMerged.headError or null != null then

            throw "A definition for option `${showOption loc}' is not of type `${type.description}'. TypeError: ${checkedAndMerged.headError.message}"

          else

            checkedAndMerged.value

        else if all (def: type.check def.value) defsFinal then

          type.merge loc defsFinal

        else

          let

        throw

          "The option `${showOption loc}' was accessed but has no value defined. Try setting the option.";

    checkedAndMerged =

      (

        # This function (which is immediately applied) checks that type.merge

        # returns the proper attrset.

        # Once use of the merge.v2 feature has propagated, consider removing this

        # for an estimated one thousandth performance improvement (NixOS by nr.thunks).

        {

          headError,

          value,

          valueMeta,

        }@args:

        args

      )

        (

          if type.merge ? v2 then

            let

              r = type.merge.v2 {

                inherit loc;

                defs = defsFinal;

              };

            in

            r

            // {

              valueMeta = r.valueMeta // {

                _internal = {

                  inherit type;

                };

              };

            }

          else

            {

              headError = null;

              value = mergedValue;

              valueMeta = { };

            }

        );

    isDefined = defsFinal != [ ];

    optionalValue = if isDefined then { value = mergedValue; } else { };

        New option path as list of strings.

      */

      to,

      /**

        Release number of the first release that contains the rename, ignoring backports.

        Set it to the upcoming release, matching the nixpkgs/.version file.

      */

      sinceRelease,

    }:

    doRename {

      inherit from to;