Unverified Commit 0504bc63 authored by Jade Lovelace's avatar Jade Lovelace Committed by GitHub
Browse files

doc/stdenv: rewrite manual build procedure to be closer to an auto-build (#262137) 

* doc/stdenv: rewrite manual build procedure to be closer to an auto-build

This is based on
<https://jade.fyi/blog/building-nix-derivations-manually/> plus some
more original research.

The previous version of this section did not work for your choice of
simple Haskell package, e.g. haskellPackages.hscolour, due to things
like `compileBuildDriverPhase` and other custom phases that it
does not address at all.

It seems more correct to use genericBuild in development to harmonize it
with what is actually done.

I feel a little bit like I am committing a sin by suggesting using the
experimental CLI in the manual (afaict I am the first to do this), but I
have given the old version of the command, and there are justifiable
reasons to do it:
* The noted limitations with env-vars are fixed. The one with the
  non-empty temp directory was one I ran into myself and oh boy was that
  not fun to debug.
* Additionally the outputs are set *before* sourcing `setup.sh`: there
  is an issue with nix-shell where the original version of `$out` winds
  up in `NIX_LDFLAGS` due to _addRpathPrefix, which means that resulting
  executables may not run properly.

It is sad that `nix develop` propagates a wrong value of `SHELL` to
builders, though. It is equally sad that `nix-shell` is essentially
abandoned upstream, with undocumented and not insignificant differences
from `nix develop`.

For the exact script differences:
https://github.com/NixOS/nix/blob/17e6b85d05b3d32df244b1d4e89aa41fd8bdcae8/src/nix-build/nix-build.cc#L516-L551
https://github.com/NixOS/nix/blob/db026103b18fb8b5a719594502edd0f89eb9c268/src/nix/get-env.sh



Co-authored-by: default avatarValentin Gagarin <valentin.gagarin@tweag.io>
parent c70f2bbf

doc/stdenv/stdenv.chapter.md

+43 −6
Original line number Diff line number Diff line
@@ -101,25 +101,62 @@ genericBuild 


### Building a `stdenv` package in `nix-shell` {#sec-building-stdenv-package-in-nix-shell}



To build a `stdenv` package in a [`nix-shell`](https://nixos.org/manual/nix/unstable/command-ref/nix-shell.html), use

To build a `stdenv` package in a [`nix-shell`](https://nixos.org/manual/nix/unstable/command-ref/nix-shell.html), enter a shell, find the [phases](#sec-stdenv-phases) you wish to build, then invoke `genericBuild` manually:



Go to an empty directory, invoke `nix-shell` with the desired package, and from inside the shell, set the output variables to a writable directory:



```bash

cd "$(mktemp -d)"

nix-shell '<nixpkgs>' -A some_package

eval "${unpackPhase:-unpackPhase}"

cd $sourceRoot

eval "${patchPhase:-patchPhase}"

eval "${configurePhase:-configurePhase}"

eval "${buildPhase:-buildPhase}"

export out=$(pwd)/out

```



Next, invoke the desired parts of the build.

First, run the phases that generate a working copy of the sources, which will change directory to the sources for you:



```bash

phases="${prePhases[*]:-} unpackPhase patchPhase" genericBuild

```



Then, run more phases up until the failure is reached.

For example, if the failure is in the build phase, the following phases would be required:



```bash

phases="${preConfigurePhases[*]:-} configurePhase ${preBuildPhases[*]:-} buildPhase" genericBuild

```



Re-run a single phase as many times as necessary to examine the failure like so:



```bash

phases="buildPhase" genericBuild

```



To modify a [phase](#sec-stdenv-phases), first print it with



```bash

echo "$buildPhase"

```



Or, if that is empty, for instance, if it is using a function:



```bash

type buildPhase

```



then change it in a text editor, and paste it back to the terminal.



::: {.note}

This method may have some inconsistencies in environment variables and behaviour compared to a normal build within the [Nix build sandbox](https://nixos.org/manual/nix/unstable/language/derivations#builder-execution).

The following is a non-exhaustive list of such differences:



- `TMP`, `TMPDIR`, and similar variables likely point to non-empty directories that the build might conflict with files in.

- Output store paths are not writable, so the variables for outputs need to be overridden to writable paths.

- Other environment variables may be inconsistent with a `nix-build` either due to `nix-shell`'s initialization script or due to the use of `nix-shell` without the `--pure` option.



If the build fails differently inside the shell than in the sandbox, consider using [`breakpointHook`](#breakpointhook) and invoking `nix-build` instead.

The [`--keep-failed`](https://nixos.org/manual/nix/unstable/command-ref/conf-file#opt--keep-failed) option for `nix-build` may also be useful to examine the build directory of a failed build.

:::



## Tools provided by `stdenv` {#sec-tools-of-stdenv}



The standard environment provides the following packages: