nixos/test-driver: use vhost-device-vsock for SSH backdoor (#453305) (ea044acd) · Commits · nix / nixpkgs

nixos/doc/manual/development/running-nixos-tests-interactively.section.md

+15 −33

Original line number	Diff line number	Diff line
		@@ -88,52 +88,34 @@ An SSH-based backdoor to log into machines can be enabled with
		}
		```

		::: {.warning}
		Make sure to only enable the backdoor for interactive tests
		(i.e. by using `interactive.sshBackdoor.enable`)! This is the only
		supported configuration.

		Running a test in a sandbox with this will fail because `/dev/vhost-vsock` isn't available
		in the sandbox.
		:::

		This creates a [vsock socket](https://man7.org/linux/man-pages/man7/vsock.7.html)
		for each VM to log in with SSH. This configures root login with an empty password.

		When the VMs get started interactively with the test-driver, it's possible to
		connect to `machine` with
		On the host-side a UNIX domain-socket is used with
		[vhost-device-vsock](https://github.com/rust-vmm/vhost-device/blob/main/vhost-device-vsock/README.md).
		That way, it's not necessary to assign system-wide unique vsock numbers.

		```
		$ ssh vsock/3 -o User=root
		$ ssh vsock-mux//tmp/path/to/host -o User=root
		```

		The socket numbers correspond to the node number of the test VM, but start
		at three instead of one because that's the lowest possible
		vsock number. The exact SSH commands are also printed out when starting
		`nixos-test-driver`.

		On non-NixOS systems you'll probably need to enable
		the SSH config from {manpage}`systemd-ssh-proxy(1)` yourself.

		If starting VM fails with an error like
		The socket paths are printed when starting the test driver:

		```
		qemu-system-x86_64: -device vhost-vsock-pci,guest-cid=3: vhost-vsock: unable to set guest cid: Address already in use
		Note: this requires systemd-ssh-proxy(1) to be enabled (default on NixOS 25.05 and newer).
		machine: ssh -o User=root vsock-mux//tmp/tmpg1rp9nti/machine_host.socket
		```

		it means that the vsock numbers for the VMs are already in use. This can happen
		if another interactive test with SSH backdoor enabled is running on the machine.
		On non-NixOS systems you'll probably need to enable
		the SSH config from {manpage}`systemd-ssh-proxy(1)` yourself.

		In that case, you need to assign another range of vsock numbers. You can pick another
		offset with
		During a test-run, it's possible to print the SSH commands again by running

		```nix
		{
		sshBackdoor = {
		enable = true;
		vsockOffset = 23542;
		};
		}
		```
		In [2]: dump_machine_ssh()
		SSH backdoor enabled, the machines can be accessed like this:
		Note: this requires systemd-ssh-proxy(1) to be enabled (default on NixOS 25.05 and newer).
		machine: ssh -o User=root vsock-mux//tmp/tmpg1rp9nti/machine_host.socket
		```

		## Port forwarding to NixOS test VMs {#sec-nixos-test-port-forwarding}

nixos/doc/manual/development/writing-nixos-tests.section.md

+3 −11

Original line number	Diff line number	Diff line
		@@ -512,19 +512,11 @@ Once you are in the sandbox shell, you can access the VMs (for example, `machine
		with SSH over vsock:

		```
		bash# ssh -F ./ssh_config vsock/3
		bash# ssh -F ./ssh_config -o User=root vsock-mux//tmp/.../machine_host.socket
		```

		For the AF_VSOCK feature to work, `/dev/vhost-vsock` is needed in the sandbox
		which can be done with e.g.

		```
		nix-build -A nixosTests.foo --option sandbox-paths /dev/vhost-vsock
		```

		As described in [](#sec-nixos-test-ssh-access), the numbers for vsock start at
		`3` instead of `1`. So the first VM in the network (sorted alphabetically) can
		be accessed with `vsock/3`.
		The socket paths are printed at the beginning of the test. See
		[](#sec-nixos-test-ssh-access) for more context.

		### SSH access to test containers {#sec-test-container-ssh-access}

nixos/doc/manual/redirects.json

+0 −3

Original line number	Diff line number	Diff line
		@@ -2174,9 +2174,6 @@
		"test-opt-sshBackdoor.enable": [
		"index.html#test-opt-sshBackdoor.enable"
		],
		"test-opt-sshBackdoor.vsockOffset": [
		"index.html#test-opt-sshBackdoor.vsockOffset"
		],
		"test-opt-enableDebugHook": [
		"index.html#test-opt-enableDebugHook"
		],

nixos/lib/test-driver/default.nix

+2 −0

Original line number	Diff line number	Diff line
		@@ -14,6 +14,7 @@
		ty,

		netpbm,
		vhost-device-vsock,
		nixosTests,
		qemu_pkg ? qemu_test,
		qemu_test,
		@@ -56,6 +57,7 @@ buildPythonApplication {
		socat
		util-linux
		vde2
		vhost-device-vsock
		]
		++ lib.optionals enableNspawn [
		systemd

nixos/lib/test-driver/src/test_driver/init.py

+5 −4

Original line number	Diff line number	Diff line
		@@ -148,9 +148,9 @@ def main() -> None:
		type=Path,
		)
		arg_parser.add_argument(
		"--dump-vsocks",
		"--enable-ssh-backdoor",
		help="indicates that the interactive SSH backdoor is active and dumps information about it on start",
		type=int,
		action="store_true",
		)
		log_level_map = {level.name.lower(): level for level in LogLevel}
		arg_parser.add_argument(
		@@ -210,9 +210,10 @@ def main() -> None:
		keep_machine_state=args.keep_machine_state,
		global_timeout=args.global_timeout,
		debug=debugger,
		enable_ssh_backdoor=args.enable_ssh_backdoor,
		) as driver:
		if offset := args.dump_vsocks:
		driver.dump_machine_ssh(offset)
		if args.enable_ssh_backdoor:
		driver.dump_machine_ssh()
		if args.interactive:
		history_dir = os.getcwd()
		history_path = os.path.join(history_dir, ".nixos-test-history")

Admin message