xen: add README and update script

<p align="center">

  <a href="https://xenproject.org/">

    <picture>

      <source

        media="(prefers-color-scheme: light)"

        srcset="https://downloads.xenproject.org/Branding/Logos/Green+Black/xen_project_logo_dualcolor_2000x832.png">

      <source

        media="(prefers-color-scheme: dark)"

        srcset="https://xenproject.org/wp-content/uploads/sites/79/2018/09/logo_xenproject.png">

      <img

        src="https://downloads.xenproject.org/Branding/Logos/Green+Black/xen_project_logo_dualcolor_2000x832.png"

        width="512px"

        alt="Xen Project Logo">

    </picture>

  </a>

</p>

# Xen Hypervisor <a href="https://xenproject.org/"><img src="https://downloads.xenproject.org/Branding/Mascots/Xen-Fu-Panda-2000px.png" width="48px" align="top" alt="Xen Fu Panda"></a>

This directory includes the build recipes for the [Xen Hypervisor](https://xenproject.org/).

Some other notable packages that compose the Xen Ecosystem include:

- `ocamlPackages.xenstore`: Mirage's `oxenstore` implementation.

- `ocamlPackages.vchan`: Mirage's `xen-vchan` implementation.

- `ocamlPackages.xenstore-tool`: XAPI's `oxenstore` utilities.

- `xen-guest-agent`: Guest drivers for UNIX domUs.

- `win-pvdrivers`: Guest drivers for Windows domUs.

## Updating

### Automatically

An automated update script is available in this directory. To produce up-to-date

files for all supported Xen branches, simply run `./update.sh`, and follow the

instructions given to you by the script. Notably, it will request that you verify

the Xen Project code signing PGP key. This README understands that the fingerprint

of that key is [`23E3 222C 145F 4475 FA80 60A7 83FE 14C9 57E8 2BD9`](https://keys.openpgp.org/search?q=pgp%40xen.org),

but you should verify this information by seeking the fingerprint from other trusted

sources, as this document may be compromised. Once the PGP key is verified, it will

use `git verify-tag` to ascertain the validity of the cloned Xen sources.

After the script is done, follow the steps in [**For Both Update Methods**](#for-both-update-methods) below.

#### Downstream Patch Names

The script expects local patch names to follow a certain specification.

Please name any required patches using the template below:

```console

0000-project-description-branch.patch

```

Where:

1. The first four numbers define the patch order.

   **0001** will be applied after **0000**, and so on.

1. `project` means the name of the source the patch should be applied to.

   - If you are applying patches to the main Xen sources, use `xen`.

   - For the pre-fetched QEMU, use `qemu`.

   - For SeaBIOS, use `seabios`.

   - For OVMF, use `ovmf`.

   - For iPXE, use `ipxe`.

1. `description` is a string with uppercase and lowercase letters, numbers and

   dashes. It describes the patch name and what it does to the upstream code.

1. `branch` is the branch for which this patch is supposed to patch.

   It should match the name of the directory it is in.

For example, a patch fixing `xentop`'s output in the 4.15 branch should have

the following name: `0000-xen-xentop-output-4.15.patch`, and it should be added

to the `4.15/` directory.

### Manually

The script is not infallible, and it may break in the future. If that happens,

open a PR fixing the script, and update Xen manually:

1. Check the support matrix to see which branches are security-supported.

1. Create one directory per branch.

1. [Update](https://xenbits.xenproject.org/gitweb/) the `default.nix` files for

   the branches that already exist and copy a new one to any branches that do

   not yet exist in Nixpkgs.

   - Do not forget to set the `branch`, `version`, and `latest` attributes for

     each of the `default.nix` files.

   - The revisions are preferably commit hashes, but tag names are acceptable

     as well.

### For Both Update Methods

1. Make sure all branches build. (Both the `standard` and `slim` versions)

1. Use the NixOS module to test if dom0 boots successfully on all new versions.

1. Clean up your changes and commit them, making sure to follow the

   [Nixpkgs Contribution Guidelines](../../../../CONTRIBUTING.md).

1. Open a PR and await a review from the current maintainers.

## Features

### Pre-fetched Sources

On a typical Xen build, the Xen Makefiles will fetch more required sources with

`git` and `wget`. Due to the Nix Sandbox, build-time fetching will fail, so we

pre-fetch the required sources before building.[^1] To accomplish this, we have

a `prefetchedSources` attribute that contains the required derivations, if they

are requested by the main Xen build.

### EFI

Building `xen.efi` requires an `ld` with PE support.[^2]

We use a `makeFlag` to override the `$LD` environment variable to point to our

patched `efiBinutils`. For more information, see the comment in `./generic.nix`.

> [!TIP]

> If you are certain you will not be running Xen in an x86 EFI environment, disable

the `withEFI` flag with an [override](https://nixos.org/manual/nixpkgs/stable/#chap-overrides)

to save you the need to compile `efiBinutils`.

### Default Overrides

By default, Xen also builds

[QEMU](https://www.qemu.org/),

[SeaBIOS](https://www.seabios.org/SeaBIOS),

[OVMF](https://github.com/tianocore/tianocore.github.io/wiki/OVMF) and

[iPXE](https://ipxe.org/).

- QEMU is used for stubdomains and handling devices.

- SeaBIOS is the default legacy BIOS ROM for HVM domains.

- OVMF is the default UEFI ROM for HVM domains.

- iPXE provides a PXE boot environment for HVMs.

However, those packages are already available on Nixpkgs, and Xen does not

necessarily need to build them into the main hypervisor build. For this reason,

we also have the `withInternal<Component>` flags, which enables and disables

building those built-in components. The two most popular Xen configurations will

be the default build, with all built-in components, and a `slim` build, with none

of those components. To simplify this process, the `./packages.nix` file includes

the `xen-slim` package overrides that have all `withInternal<Component>` flags

disabled. See the `meta.longDescription` attribute for the `xen-slim` packages

for more information.

## Security

We aim to support all **security-supported** versions of Xen at any given time.

See the [Xen Support Matrix](https://xenbits.xen.org/docs/unstable/support-matrix.html)

for a list of versions. As soon as a version is no longer **security-supported**,

it should be removed from Nixpkgs.

> [!CAUTION]

> Pull requests that introduce XSA patches

should have the `1.severity: security` label.

### Maintainers

Xen is a particularly complex piece of software, so we are always looking for new

maintainers. Help out by [making and triaging issues](https://github.com/NixOS/nixpkgs/issues/new/choose),

[sending build fixes and improvements through PRs](https://github.com/NixOS/nixpkgs/compare),

updating the branches, and [patching security flaws](https://xenbits.xenproject.org/xsa/).

We are also looking for testers, particularly those who can test Xen on AArch64

machines. Open issues for any build failures or runtime errors you find!

## Tests

So far, we only have had one simple automated test that checks for

the correct `pkg-config` output files.

Due to Xen's nature as a type-1 hypervisor, it is not a trivial matter to design

new tests, as even basic functionality requires a machine booted in a dom0

kernel. For this reason, most testing done with this package must be done

manually in a NixOS machine with `virtualisation.xen.enable` set to `true`.

Another unfortunate thing is that none of the Xen commands have a `--version`

flag. This means that `testers.testVersion` cannot ascertain the Xen version.

The only way to verify that you have indeed built the correct version is to

boot into the freshly built Xen kernel and run `xl info`.

<p align="center">

  <a href="https://xenproject.org/">

    <img

      src="https://downloads.xenproject.org/Branding/Mascots/Xen%20Big%20Panda%204242x3129.png"

      width="96px"

      alt="Xen Fu Panda">

  </a>

</p>

[^1]: We also produce fake `git`, `wget` and `hostname` binaries that do nothing,

      to prevent the build from failing because Xen cannot fetch the sources that

      were already fetched by Nix.

[^2]: From the [Xen Documentation](https://xenbits.xenproject.org/docs/unstable/misc/efi.html):

      > For x86, building `xen.efi` requires `gcc` 4.5.x or above (4.6.x or newer

      recommended, as 4.5.x was probably never really tested for this purpose)

      and `binutils` 2.22 or newer. Additionally, the `binutils` build must be

      configured to include support for the x86_64-pep emulation (i.e.

      `--enable-targets=x86_64-pep` or an option of equivalent effect should be

      passed to the configure script).

#!/usr/bin/env nix-shell

#!nix-shell -i bash -p gitMinimal curl gnupg nix-prefetch-git nixfmt-rfc-style

# shellcheck disable=SC2206,SC2207 shell=bash

set -e

# Set a temporary $HOME in /tmp for GPG.

HOME=/tmp/xenUpdateScript

# This script expects to be called in an interactive terminal somewhere inside Nixpkgs.

echo "Preparing..."

nixpkgs=$(git rev-parse --show-toplevel)

xenPath="$nixpkgs/pkgs/applications/virtualization/xen"

rm -rf /tmp/xenUpdateScript

mkdir /tmp/xenUpdateScript

# Import and verify PGP key.

curl --silent --output /tmp/xenUpdateScript/xen.asc https://keys.openpgp.org/vks/v1/by-fingerprint/23E3222C145F4475FA8060A783FE14C957E82BD9

gpg --quiet --import /tmp/xenUpdateScript/xen.asc

fingerprint="$(gpg --with-colons --fingerprint "pgp@xen.org" 2>/dev/null | awk -F: '/^pub:.*/ { getline; print $10}')"

echo -e "Please ascertain through multiple external sources that the \e[1;32mXen Project PGP Key Fingerprint\e[0m is indeed \e[1;33m$fingerprint\e[0m. If that is not the case, \e[1;31mexit immediately\e[0m."

read -r -p $'Press \e[1;34menter\e[0m to continue with a pre-filled expected fingerprint, or input an arbitrary PGP fingerprint to match with the key\'s fingerprint: ' userInputFingerprint

userInputFingerprint=${userInputFingerprint:-"23E3222C145F4475FA8060A783FE14C957E82BD9"}

# Clone xen.git.

echo -e "Cloning \e[1;34mxen.git\e[0m..."

git clone --quiet https://xenbits.xen.org/git-http/xen.git /tmp/xenUpdateScript/xen

cd /tmp/xenUpdateScript/xen

# Get list of versions and branches.

versionList="$(git tag --list "RELEASE-*" | sed s/RELEASE-//g | sed s/4.1.6.1//g | sort --numeric-sort)"

latestVersion=$(echo "$versionList" | tr ' ' '\n' | tail --lines=1)

branchList=($(echo "$versionList" | tr ' ' '\n' | sed s/\.[0-9]*$//g | awk '!seen[$0]++'))

# Figure out which versions we're actually going to install.

minSupportedBranch="$(grep "    knownVulnerabilities = lib.lists.optionals (lib.strings.versionOlder version " "$xenPath"/generic.nix | sed s/'    knownVulnerabilities = lib.lists.optionals (lib.strings.versionOlder version "'//g | sed s/'") \['//g)"

supportedBranches=($(for version in "${branchList[@]}"; do if [ "$(printf '%s\n' "$minSupportedBranch" "$version" | sort -V | head -n1)" = "$minSupportedBranch" ]; then echo "$version"; fi; done))

supportedVersions=($(for version in "${supportedBranches[@]}"; do echo "$versionList" | tr ' ' '\n' | grep "$version" | tail --lines=1; done))

# Main loop that installs every supportedVersion.

for version in "${supportedVersions[@]}"; do

    echo -e "\n------------------------------------------------"

    branch=${version/%.[0-9]/}

    if [[ "$version" == "$latestVersion" ]]; then

        latest=true

        echo -e "\nFound \e[1;34mlatest\e[0m release: \e[1;32mXen $version\e[0m in branch \e[1;36m$branch\e[0m."

    else

        latest=false

        echo -e "\nFound \e[1;33msecurity-supported\e[0m release: \e[1;32mXen $version\e[0m in branch \e[1;36m$branch\e[0m."

    fi

    # Verify PGP key automatically. If the fingerprint matches what the user specified, or the default fingerprint, then we consider it trusted.

    cd /tmp/xenUpdateScript/xen

    if [[ "$fingerprint" = "$userInputFingerprint" ]]; then

        echo "$fingerprint:6:" | gpg --quiet --import-ownertrust

        (git verify-tag RELEASE-"$version" 2>/dev/null && echo -e "\n\e[1;32mSuccessfully authenticated Xen $version.\e[0m") || (echo -e "\e[1;31merror:\e[0m Unable to verify tag \e[1;32mRELEASE-$version\e[0m.\n- It is possible that \e[1;33mthis script has broken\e[0m, the Xen Project has \e[1;33mcycled their PGP keys\e[0m, or a \e[1;31msupply chain attack is in progress\e[0m.\n\n\e[1;31mPlease update manually.\e[0m" && exit 1)

    else

        echo -e "\e[1;31merror:\e[0m Unable to verify \e[1;34mpgp@xen.org\e[0m's fingerprint.\n- It is possible that \e[1;33mthis script has broken\e[0m, the Xen Project has \e[1;33mcycled their PGP keys\e[0m, or an \e[1;31mimpersonation attack is in progress\e[0m.\n\n\e[1;31mPlease update manually.\e[0m" && exit 1

    fi

    git switch --quiet --detach RELEASE-"$version"

    # Originally we told people to go check the Makefile themselves.

    echo -e "\nDetermining source versions from Xen Makefiles..."

    qemuVersion="$(grep -ie "QEMU_UPSTREAM_REVISION ?=" /tmp/xenUpdateScript/xen/Config.mk | sed s/"QEMU_UPSTREAM_REVISION ?= "//g)"

    seaBIOSVersion="$(grep -ie "SEABIOS_UPSTREAM_REVISION ?= rel-" /tmp/xenUpdateScript/xen/Config.mk | sed s/"SEABIOS_UPSTREAM_REVISION ?= "//g)"

    ovmfVersion="$(grep -ie "OVMF_UPSTREAM_REVISION ?=" /tmp/xenUpdateScript/xen/Config.mk | sed s/"OVMF_UPSTREAM_REVISION ?= "//g)"

    ipxeVersion="$(grep -ie "IPXE_GIT_TAG :=" /tmp/xenUpdateScript/xen/tools/firmware/etherboot/Makefile | sed s/"IPXE_GIT_TAG := "//g)"

    # Use `nix-prefetch-git` to fetch `rev`s and `hash`es.

    echo "Pre-fetching sources and determining hashes..."

    echo -e -n "  \e[1;32mXen\e[0m..."

    fetchXen=$(nix-prefetch-git --url https://xenbits.xen.org/git-http/xen.git --rev RELEASE-"$version" --quiet)

    finalVersion="$(echo "$fetchXen" | tr ', ' '\n ' | grep -ie rev | sed s/'  "rev": "'//g | sed s/'"'//g)"

    hash="$(echo "$fetchXen" | tr ', ' '\n ' | grep -ie hash | sed s/'  "hash": "'//g | sed s/'"'//g)"

    echo "done!"

    echo -e -n "  \e[1;36mQEMU\e[0m..."

    fetchQEMU=$(nix-prefetch-git --url https://xenbits.xen.org/git-http/qemu-xen.git --rev "$qemuVersion" --quiet --fetch-submodules)

    finalQEMUVersion="$(echo "$fetchQEMU" | tr ', ' '\n ' | grep -ie rev | sed s/'  "rev": "'//g | sed s/'"'//g)"

    qemuHash="$(echo "$fetchQEMU" | tr ', ' '\n ' | grep -ie hash | sed s/'  "hash": "'//g | sed s/'"'//g)"

    echo "done!"

    echo -e -n "  \e[1;36mSeaBIOS\e[0m..."

    fetchSeaBIOS=$(nix-prefetch-git --url https://xenbits.xen.org/git-http/seabios.git --rev "$seaBIOSVersion" --quiet)

    finalSeaBIOSVersion="$(echo "$fetchSeaBIOS" | tr ', ' '\n ' | grep -ie rev | sed s/'  "rev": "'//g | sed s/'"'//g)"

    seaBIOSHash="$(echo "$fetchSeaBIOS" | tr ', ' '\n ' | grep -ie hash | sed s/'  "hash": "'//g | sed s/'"'//g)"

    echo "done!"

    echo -e -n "  \e[1;36mOVMF\e[0m..."

    ovmfHash="$(nix-prefetch-git --url https://xenbits.xen.org/git-http/ovmf.git --rev "$ovmfVersion" --quiet --fetch-submodules | grep -ie hash | sed s/'  "hash": "'//g | sed s/'",'//g)"

    echo "done!"

    echo -e -n "  \e[1;36miPXE\e[0m..."

    ipxeHash="$(nix-prefetch-git --url https://github.com/ipxe/ipxe.git --rev "$ipxeVersion" --quiet | grep -ie hash | sed s/'  "hash": "'//g | sed s/'",'//g)"

    echo "done!"

    cd "$xenPath"

    echo -e "\nFound the following revisions:\n  \e[1;32mXen\e[0m:     \e[1;33m$finalVersion\e[0m (\e[1;33m$hash\e[0m)\n  \e[1;36mQEMU\e[0m:    \e[1;33m$finalQEMUVersion\e[0m (\e[1;33m$qemuHash\e[0m)\n  \e[1;36mSeaBIOS\e[0m: \e[1;33m$finalSeaBIOSVersion\e[0m (\e[1;33m$seaBIOSHash\e[0m)\n  \e[1;36mOVMF\e[0m:    \e[1;33m$ovmfVersion\e[0m (\e[1;33m$ovmfHash\e[0m)\n  \e[1;36miPXE\e[0m:    \e[1;33m$ipxeVersion\e[0m (\e[1;33m$ipxeHash\e[0m)"

    # Set OCaml Version

    read -r -p $'\nEnter the corresponding \e[1;33mOCaml\e[0m version for \e[1;32mXen '"$version"$'\e[0m, or press \e[1;34menter\e[0m for the default value of \e[1;32m4_14\e[0m: ' ocamlVersion

    ocamlVersion=${ocamlVersion:-"4_14"}

    mkdir -p "$branch"/

    rm -f "$branch"/default.nix

    # Prepare any .patch files that are called by Nix through a path value.

    echo -e "\nPlease add any required patches to version \e[1;32m$branch\e[0m in \e[1;34m$branch/\e[0m, and press \e[1;34menter\e[0m when done."

    read -r -p $'Remember to follow the naming specification as defined in \e[1;34m./README.md\e[0m.'

    echo -e "\nDiscovering patches..."

    discoveredXenPatches="$(find "$branch"/ -type f -name "[0-9][0-9][0-9][0-9]-xen-*-$branch.patch" -printf "./%f ")"

    discoveredQEMUPatches="$(find "$branch"/ -type f -name "[0-9][0-9][0-9][0-9]-qemu-*-$branch.patch" -printf "./%f ")"

    discoveredSeaBIOSPatches="$(find "$branch"/ -type f -name "[0-9][0-9][0-9][0-9]-seabios-*-$branch.patch" -printf "./%f ")"

    discoveredOVMFPatches="$(find "$branch"/ -type f -name "[0-9][0-9][0-9][0-9]-ovmf-*-$branch.patch" -printf "./%f ")"

    discoveredIPXEPatches="$(find "$branch"/ -type f -name "[0-9][0-9][0-9][0-9]-ipxe-*-$branch.patch" -printf "./%f ")"

    discoveredXenPatchesEcho=${discoveredXenPatches:-"\e[1;31mNone found!\e[0m"}

    discoveredQEMUPatchesEcho=${discoveredQEMUPatches:-"\e[1;31mNone found!\e[0m"}

    discoveredSeaBIOSPatchesEcho=${discoveredSeaBIOSPatches:-"\e[1;31mNone found!\e[0m"}

    discoveredOVMFPatchesEcho=${discoveredOVMFPatches:-"\e[1;31mNone found!\e[0m"}

    discoveredIPXEPatchesEcho=${discoveredIPXEPatches:-"\e[1;31mNone found!\e[0m"}

    echo -e "Found the following patches:\n  \e[1;32mXen\e[0m:     \e[1;33m$discoveredXenPatchesEcho\e[0m\n  \e[1;36mQEMU\e[0m:    \e[1;33m$discoveredQEMUPatchesEcho\e[0m\n  \e[1;36mSeaBIOS\e[0m: \e[1;33m$discoveredSeaBIOSPatchesEcho\e[0m\n  \e[1;36mOVMF\e[0m:    \e[1;33m$discoveredOVMFPatchesEcho\e[0m\n  \e[1;36miPXE\e[0m:    \e[1;33m$discoveredIPXEPatchesEcho\e[0m"

    # Prepare patches that are called in ./patches.nix.

    defaultPatchListInit=("QUBES_REPRODUCIBLE_BUILDS" "XSA_458")

    read -r -a defaultPatchList -p $'\nWould you like to override the \e[1;34mupstreamPatches\e[0m list for \e[1;32mXen '"$version"$'\e[0m? If no, press \e[1;34menter\e[0m to use the default patch list: [ \e[1;34m'"${defaultPatchListInit[*]}"$' \e[0m]: '

    defaultPatchList=(${defaultPatchList[@]:-${defaultPatchListInit[@]}})

    spaceSeparatedPatchList=${defaultPatchList[*]}

    upstreamPatches="upstreamPatches.${spaceSeparatedPatchList// / upstreamPatches.}"

    # Write and format default.nix file.

    echo -e "\nWriting updated \e[1;34mversionDefinition\e[0m..."

    cat >"$branch"/default.nix <<EOF

{

  lib,

  fetchpatch,

  callPackage,

  ocaml-ng,

  ...

}@genericDefinition:

let

  upstreamPatches = import ../patches.nix {

    inherit lib;

    inherit fetchpatch;

  };

  upstreamPatchList = lib.lists.flatten [

    $upstreamPatches

  ];

in

callPackage (import ../generic.nix {

  branch = "$branch";

  version = "$version";

  latest = $latest;

  pkg = {

    xen = {

      rev = "$finalVersion";

      hash = "$hash";

      patches = [ $discoveredXenPatches ] ++ upstreamPatchList;

    };

    qemu = {

      rev = "$finalQEMUVersion";

      hash = "$qemuHash";

      patches = [ $discoveredQEMUPatches ];

    };

    seaBIOS = {

      rev = "$finalSeaBIOSVersion";

      hash = "$seaBIOSHash";

      patches = [ $discoveredSeaBIOSPatches ];

    };

    ovmf = {

      rev = "$ovmfVersion";

      hash = "$ovmfHash";

      patches = [ $discoveredOVMFPatches ];

    };

    ipxe = {

      rev = "$ipxeVersion";

      hash = "$ipxeHash";

      patches = [ $discoveredIPXEPatches ];

    };

  };

}) ({ ocamlPackages = ocaml-ng.ocamlPackages_$ocamlVersion; } // genericDefinition)

EOF

    echo "Formatting..."

    nixfmt "$branch"/default.nix

    echo -e "\n\e[1;32mSuccessfully produced $branch/default.nix.\e[0m"

done

echo -e -n "\nCleaning up..."

rm -rf /tmp/xenUpdateScript

echo done!