Unverified Commit 76a68e01 authored by Vladimír Čunát's avatar Vladimír Čunát Committed by GitHub
Browse files

revert: gitlab runner docs & vm tests (#441161) (#472751)

parents ff3f7911 02167967
Loading
Loading
Loading
Loading
+0 −6
Original line number Diff line number Diff line
@@ -1185,12 +1185,6 @@
  "module-services-gitlab-maintenance-rake": [
    "index.html#module-services-gitlab-maintenance-rake"
  ],
  "module-services-gitlab-runner": [
    "index.html#module-services-gitlab-runner"
  ],
  "ex-gitlab-runner-podman": [
    "index.html#ex-gitlab-runner-podman"
  ],
  "module-forgejo": [
    "index.html#module-forgejo"
  ],
+1 −1
Original line number Diff line number Diff line
@@ -502,7 +502,7 @@
  ./services/continuous-integration/buildkite-agents.nix
  ./services/continuous-integration/gitea-actions-runner.nix
  ./services/continuous-integration/github-runners.nix
  ./services/continuous-integration/gitlab-runner/runner.nix
  ./services/continuous-integration/gitlab-runner.nix
  ./services/continuous-integration/gocd-agent/default.nix
  ./services/continuous-integration/gocd-server/default.nix
  ./services/continuous-integration/hercules-ci-agent/default.nix
+0 −113
Original line number Diff line number Diff line
@@ -10,7 +10,6 @@ configure a webserver to proxy HTTP requests to the socket.

For instance, the following configuration could be used to use nginx as
frontend proxy:

```nix
{
  services.nginx = {
@@ -41,7 +40,6 @@ The default state dir is `/var/gitlab/state`. This is where
all data like the repositories and uploads will be stored.

A basic configuration with some custom settings could look like this:

```nix
{
  services.gitlab = {
@@ -106,7 +104,6 @@ the [services.gitlab.backup.startAt](#opt-services.gitlab.backup.startAt)
option to configure regular backups.

To run a manual backup, start the `gitlab-backup` service:

```ShellSession
$ systemctl start gitlab-backup.service
```
@@ -119,116 +116,6 @@ will have to run the command as the user that you configured to run
GitLab with.

A list of all available rake tasks can be obtained by running:

```ShellSession
$ sudo -u git -H gitlab-rake -T
```

## Runner {#module-services-gitlab-runner}

GitLab Runner is a CI runner which is an executable which you can host yourself.
A Gitlab pipeline runs operations over a Gitlab Runner. These can include
building an executable, running a test suite, pushing a docker image, etc. The
Gitlab Runner receives jobs from Gitlab which it then dispatches to the
configured executors
([`docker` (`podman`), or `shell` or `kubernetes`](https://docs.gitlab.com/runner/executors)).

The
[services.gitlab-runner.services](https://search.nixos.org/options?query=services.gitlab-runner.services)
documents a number of typical setups to configure multiple runners with
different executors.

The [below example](#ex-gitlab-runner-podman) gives a **more elaborate** example how to
configure a Gitlab Runner with caching and reasonably good security practices.

::: {#ex-gitlab-runner-podman .example}

## Example: Gitlab Runner with `podman` and Nix Store Caching

The [VM tested `podman-runner`](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/continuous-integration/gitlab-runner/runner.nix)
(a NixOS module for reuse) configures an advanced Gitlab runner with the following features:

- The executor is `podman` which gives you better additional safety than
  `docker`. That means every job is run in a `podman` container.

- The following container **images** are built with Nix:

  **Container Images for Gitlab Jobs**:
  - `local/alpine`: An image based on Alpine with a Nix installation
    (attribute `jobImages.alpine`).
  - `local/ubuntu`: An image based on Ubuntu with a Nix installation
    (attribute `jobImages.ubuntu`).
  - `local/nix`: An image based on Nix which only comes with `nix`
    installed (attribute `jobImages.nix`).

  **Images for VM Setup**:
  - `local/nix-daemon-image`: An image with a Nix daemon which is
    used to share the `/nix/store` across jobs (variable `nixDaemonImage`) setup with some essentials derivations `bootstrapPkgs`.
  - `local/podman-daemon-image`: An image with `podman` running as a daemon which is
    used to run `podman` inside the above job containers images
    (variable `podmanDaemonImage`).

- Every job container runs in a `podman` container instance based by default on
  `jobImage.ubuntu`. A pipeline job can override this with `image: local/alpine`.
  - Each job container will have the `/nix/store` mounted from the container
    `nix-daemon-container` (see registration flags
    `--docker-volumes-from "nix-daemon-container:ro"`).

    The `nix-daemon-container` is a single container instance of a
    `nixDaemonImage`. This enables caching of `/nix/store` paths across all jobs
    in **all** runners. This makes **the host VM's `/nix/store` independent of the
    Nix store used in the jobs**, which is good.

    ::: {.note}
    **Security:** If you don't want this you need multiple `nixDaemonImage`
    containers for each registered runner (`gitlab-runner.services.<name>`).
    :::

  - Each job container will have the `/run/podman/podman.sock` socket mounted from the
    `podman-daemon-container`.

    The `podman-daemon-container` is a single container of a `podmanDaemonImage` which runs
    `podman` as a daemon. Job containers can use this daemon to spawn nested containers as well (podman-in-podman).
    **Keep in mind that `bind` mounts are local to the `podman-daemon-container`**
    and can be be worked around with a `podman volume create <vol>` and manual copy-to/copy-from this volume `<vol>`.

    If you only need to build containers you don't need this feature (`podman-daemon-container`), see below point.

    Container configuration files (`auxRootFiles`) are copied to all containers to
    ensure `podman` works consistently inside the job containers.

  - The job containers do **not** mount the `podman` socket from the host (NixOS
    VM) mounted for security reasons.

    ::: {.note}
    Building container images with `buildah` (stripped
    `podman` for building images) inside a job which runs `jobImage.alpine`
    is still possible.
    :::

  - **Cleanup Disk Space**:

    With this setup its really easy to clean the `nix-daemon-container`
    (e.g. if you run out of disk space), then reboot and have the runner in a clean state.
    You can do the following to effectively clean everything and start with fresh volumes safely:

    ```bash
    # Stop the Gitlab runner.
    systemctl stop gitlab-runner.service
    # Stop `systemd`-managed containers, such that they get not recreated
    # when deleting below.
    systemctl stop podman-podman-daemon-container.service \
                  podman-nix-daemon-container.service \
                  podman-nix-container.service \
                  podman-alpine-container.service \
                  podman-ubuntu-container.service || true

    podman container rm -f --all
    podman image rm -f --all
    podman volumes rm -f --all

    reboot
    # Systemd will restart all containers and create volumes etc.
    ```

:::
+1 −2
Original line number Diff line number Diff line
@@ -98,8 +98,7 @@ rec {

        (onFullSupported "nixos.tests.firewall")
        (onFullSupported "nixos.tests.fontconfig-default-fonts")
        (onFullSupported "nixos.tests.gitlab.gitlab")
        (onFullSupported "nixos.tests.gitlab.runner")
        (onFullSupported "nixos.tests.gitlab")
        (onFullSupported "nixos.tests.gnome")
        (onSystems [ "x86_64-linux" ] "nixos.tests.hibernate")
        (onFullSupported "nixos.tests.i3wm")
Loading