Commit 66fb2f53 authored by Mark Ruvald Pedersen's avatar Mark Ruvald Pedersen
Browse files

nixos/proxmox-image: Disable O_DIRECT to fix assert 

Context summary:
'vma create' can't otherwise write to tmpfs such as /dev/shm.
This is important when used from non-nixos machines which may
have /build as tmpfs.

VMA is Proxmox's virtual machine image format that wraps QEMU images,
augmenting these with proxmox-specific configuration file.
proxmox-image.nix uses the VMA tool to create vma image files.
The VMA tool exists as a patchset ontop of QEMU.

VMA writes its output with open() and O_DIRECT flag.
O_DIRECT does not work on Linux tmpfs [1]. Thus:
$ vma create ~/output.vma ...  # works, assuming home isn't tmpfs.
$ vma create /dev/shm/output.vma ...  # fails since /dev/shm is tmpfs
Failure results in assert(*errp == NULL).

O_DIRECT is a cache performance hint.
But it currently blocks our usage of nixos-generate -f proxmox from
Non-NixOS hosts and Docker.

The patch here simply removes O_DIRECT:
vma-writer.c later performs memalign due to O_DIRECT, but this is
safe to do with or without O_DIRECT.
Ideally, this should be fixed in upstream Proxmox: Perhaps by falling
back to open without O_DIRECT.

Another attempt to fix this SIGABRT is [2], which writes the vma file
directory to $out/ folder -- however that may still be tmpfs mounted
which it is in our case.

[1] https://lore.kernel.org/lkml/45A29EC2.8020502@tmr.com/t/
[2] https://github.com/NixOS/nixpkgs/pull/224282
parent 4078f520

nixos/modules/virtualisation/proxmox-image.nix

+16 −0
Original line number Diff line number Diff line
@@ -193,6 +193,7 @@ with lib; 
            sha256 = "sha256-9rN1x5UfcoQCeYsLqrsthkeMpT1Eztvvq74cRr9G+Dk=";

          };

          patches = [

            # Proxmox' VMA tool is published as a particular patch upon QEMU

            (pkgs.fetchpatch {

              url =

                let
@@ -201,6 +202,21 @@ with lib; 
                in "https://git.proxmox.com/?p=pve-qemu.git;a=blob_plain;hb=${rev};f=${path}";

              hash = "sha256-2Dz+ceTwrcyYYxi76RtyY3v15/2pwGcDhFuoZWlgbjc=";

            })



            # Proxmox' VMA tool uses O_DIRECT which fails on tmpfs

            # Filed to upstream issue tracker: https://bugzilla.proxmox.com/show_bug.cgi?id=4710

            (pkgs.writeText "inline.patch" ''

                --- a/vma-writer.c   2023-05-01 15:11:13.361341177 +0200

                +++ b/vma-writer.c   2023-05-01 15:10:51.785293129 +0200

                @@ -306,7 +306,7 @@

                             /* try to use O_NONBLOCK */

                             fcntl(vmaw->fd, F_SETFL, fcntl(vmaw->fd, F_GETFL)|O_NONBLOCK);

                         } else  {

                -            oflags = O_NONBLOCK|O_DIRECT|O_WRONLY|O_EXCL;

                +            oflags = O_NONBLOCK|O_WRONLY|O_EXCL;

                             vmaw->fd = qemu_create(filename, oflags, 0644, errp);

                         }

            '')

          ];



          buildInputs = super.buildInputs ++ [ pkgs.libuuid ];