Commit 85a50950 authored by Will Fancher's avatar Will Fancher
Browse files

Revert #481473 "nixos/network-interfaces: remove network-setup" 

Reverts #481473, fixes #509254

Turns out this breaks scripted networking with systemd stage 1. Even
if we assumed that systemd stage 1 wasn't now the default, I still
think that makes this worthy of a revert, in the absence of a fix. But
now that systemd stage 1 is the default, this is also a channel
blocker.

The problem is that wantedBy = [ "foo.device" ]; doesn't work as
hoped. If the device appears in stage 1, then the systemd device unit
becomes active in stage 1 without pulling in any dependency for
configuring it (because that only exists in stage 2). When we
transition to stage 2, systemd maintains its information about unit
state between stages and remembers that the device units were already
active. They do not become reactivated, and consequently their new
stage 2 dependencies do not get pulled in.

The fix is probably to do this with SYSTEMD_WANTS= in udev rules,
rather than with wantedBy. The udev rules will re-fire during stage 2,
though I'm not 100% clear on exactly which events that includes
(e.g. if we'll get new add events or if we need to look for a
different ACTION or anything like that). This is going to require some
experimentation and testing that I don't have time for today.

Given that this is addressing a channel blocker, I think this revert
should be merged if a fix can't be merged instead before the next
unstable-small eval. In either case, staging-nixos should be merged to
master right after.
parent 6f0cbfb3

nixos/doc/manual/configuration/ad-hoc-network-config.section.md

+5 −15
Original line number Diff line number Diff line 
# Ad-Hoc Configuration {#ad-hoc-network-config}



You can use [](#opt-networking.localCommands) to specify shell commands to be

run after the network interfaces have been created, but not necessarily fully

configured.

This is useful for doing network configuration not covered by the existing

NixOS modules. For example, you can create a network namespace and a pair

of virtual ethernet devices like this:

You can use [](#opt-networking.localCommands) to

specify shell commands to be run at the end of `network-setup.service`. This

is useful for doing network configuration not covered by the existing NixOS

modules. For instance, to statically configure an IPv6 address:



```nix

{

  networking.localCommands = ''

    ip netns add mynet

    ip link add name veth-in type veth peer name veth-out

    ip link set dev veth-out netns mynet

    ip -6 addr add 2001:610:685:1::1/64 dev eth0

  '';

}

```
 


::: {.note}

The commands should ideally be idempotent, so it's recommended to perform

cleanups of the state you create (e.g. virtual interfaces), or at least make

sure possible failures are handled.

:::

nixos/doc/manual/configuration/ipv4-config.section.md

+3 −10
Original line number Diff line number Diff line
@@ -26,16 +26,9 @@ servers: 
```



::: {.note}

Addresses and routes for statically configured interfaces and the default

gateway are set up by systemd services named

`network-addresses-<interface>.service`. The name servers configuration,

instead, is performed by `network-local-commands.service` using resolvconf.

:::



::: {.note}

If needed, for example if addresses/routes were added/removed,

you can reset the network configuration by running

`systemctl restart networking-scripted.target`

Statically configured interfaces are set up by the systemd service

`interface-name-cfg.service`. The default gateway and name server

configuration is performed by `network-setup.service`.

:::



The host name is set using [](#opt-networking.hostName):

nixos/doc/manual/release-notes/rl-2605.section.md

+0 −7
Original line number Diff line number Diff line
@@ -261,13 +261,6 @@ See <https://github.com/NixOS/nixpkgs/issues/481673>. 


  Note for NetworkManager users: before these changes NetworkManager used to spawn its own wpa_supplicant daemon, but now it relies on `networking.wireless`. So, if you had `networking.wireless.enable = false` in your configuration, you should remove that line.



- Some implementation details of the NixOS network-interfaces module have been changed:



  - In the "scripted" backend, `network-setup.service` has been removed and the network configuration services are now part of `network.target`, which is now directly pulled into `multi-user.target`.

  - Interface addresses, routes and default gateways are now configured asynchronously as soon as the underlying network devices become available (fixes issue [#154737](https://github.com/NixOS/nixpkgs/issues/154737)).

  - In both "networkd" and "scripted" backends, the configuration of name servers is now part of `network-local-commands.service` (fixes issue [#445496](https://github.com/NixOS/nixpkgs/issues/445496)).

  - The issue that resulted in a completely unconfigured network if both `resolvconf` was disabled and no default gateway configured, has also been fixed.



- `kratos` has been updated from 1.3.1 to [25.4.0](https://github.com/ory/kratos/releases/tag/v25.4.0). Upstream switched to a new versioning scheme (year.major.minor). Notable breaking changes:



  - The `migrate sql` CLI command is now `migrate sql up`

nixos/modules/tasks/network-interfaces-scripted.nix

+136 −175
Original line number Diff line number Diff line
@@ -51,71 +51,6 @@ let 
      (lib.concatStringsSep " ")

    ];



  # Converts an IPv4 address literal to a list of bits

  parseAddr.ipv4 =

    addr:

    let

      pad = b: lib.replicate (8 - builtins.length b) 0 ++ b;

      toBin = n: pad (lib.toBaseDigits 2 (lib.toInt n));

    in

    lib.concatMap toBin (builtins.splitVersion addr);



  # Converts an IPv6 address literal to a list of bits

  parseAddr.ipv6 =

    addr:

    let

      pad = b: lib.replicate (16 - builtins.length b) 0 ++ b;

      fromHex = n: (builtins.fromTOML "n = 0x${n}").n;

      toBin = n: pad (lib.toBaseDigits 2 (fromHex n));

      normal = (lib.network.ipv6.fromString addr).address;

    in

    lib.concatMap toBin (lib.splitString ":" normal);



  # Checks if `addr` is part of the `net` subnet

  inSubnet =

    v: net: addr:

    let

      prefix = lib.take net.prefixLength (parseAddr.${v} net.address);

      match = lib.zipListsWith (a: b: a == b) prefix (parseAddr.${v} addr);

    in

    lib.all lib.id match;



  # Checks if the netmask of all addresses on interface `iface` includes

  # the IP address of `gateway`

  #

  # Note: this is used to check whether networking.defaultGateway relies on

  # the given interface, either explicitly, via the `interface` (optional),

  # or explicitly, by using an address in a subnet of this interface.

  #

  # Configuration of the default gateway is then performed as part of that

  # interface setup in `configureAddrs`, below.

  isGateway =

    v: gateway: iface:

    lib.any lib.id (

      [ (iface.name == gateway.interface) ]

      ++ map (net: inSubnet v net gateway.address) iface.${v}.addresses

    );



  # Checks if `gateway` uses an address from `iface` as default source

  #

  # Note: this is needed to delay the configuration of the gateway and default

  # source until the right interfaces and address have been set up, otherwise

  # the commands will fail.

  hasSource =

    v: gateway: iface:

    builtins.elem gateway.source (map (i: i.address) iface.${v}.addresses);



  # Interfaces corresponding to the default gateways

  gateway4Iface = builtins.filter (isGateway "ipv4" cfg.defaultGateway) interfaces;

  gateway6Iface = builtins.filter (isGateway "ipv6" cfg.defaultGateway6) interfaces;



  # Interfaces corresponding to the default source addresses

  #

  # Note: the use of `head` here is safe because these expressions

  # are evaluated only when `needsSourceIface`, see `configureAddrs` below.

  source4Iface = builtins.head (builtins.filter (hasSource "ipv4" cfg.defaultGateway) interfaces);

  source6Iface = builtins.head (builtins.filter (hasSource "ipv6" cfg.defaultGateway6) interfaces);



  # warn that these attributes are deprecated (2017-2-2)

  # Should be removed in the release after next

  bondDeprecation = rec {
@@ -183,45 +118,83 @@ let 
          else

            optional (!config.boot.isContainer) (subsystemDevice dev);



        # For each interface <foo>, creates a network-addresses-<foo>.service

        # job that performs static address configuration.

        #

        # It has a Wants dependency on <foo>-netdev.service, which creates

        # create the interface, or on a device unit (for hardware interfaces).

        # It also has a BindsTo dependency on the device unit: so, it only gets

        # started after the interface has appeared and it's stopped when the

        # interface disappears.

        #

        # Unless in a container, the job is not made part of network.target, so

        # if an interface is not found (e.g. a USB interface not plugged in) it

        # will not hang the boot sequence.

        #

        # If the interface is the default gateway, the job will also set the

        # default gateway and delay network-online.target.

        configureAddrs =

          i:

          let

            ips = interfaceIps i;

            isDefaultGateway4 = cfg.defaultGateway != null && builtins.elem i gateway4Iface;

            isDefaultGateway6 = cfg.defaultGateway6 != null && builtins.elem i gateway6Iface;

            needsSourceIface4 =

              isDefaultGateway4 && cfg.defaultGateway.source != null && i.name != source4Iface.name;

            needsSourceIface6 =

              isDefaultGateway6 && cfg.defaultGateway6.source != null && i.name != source6Iface.name;



            configureGateway =

              version: gateway:

              optionalString (gateway.address != "") ''

                echo -n "setting ${i.name} as default IPv${version} gateway... "

        hasDefaultGatewaySet =

          (cfg.defaultGateway != null && cfg.defaultGateway.address != "")

          || (cfg.enableIPv6 && cfg.defaultGateway6 != null && cfg.defaultGateway6.address != "");



        needNetworkSetup =

          cfg.resolvconf.enable || cfg.defaultGateway != null || cfg.defaultGateway6 != null;



        networkLocalCommands = lib.mkIf needNetworkSetup {

          after = [ "network-setup.service" ];

          bindsTo = [ "network-setup.service" ];

        };



        networkSetup = lib.mkIf needNetworkSetup {

          description = "Networking Setup";



          after = [ "network-pre.target" ];

          before = [

            "network.target"

            "shutdown.target"

          ];

          wants = [ "network.target" ];

          # exclude bridges from the partOf relationship to fix container networking bug #47210

          partOf = map (i: "network-addresses-${i.name}.service") (

            filter (i: !(hasAttr i.name cfg.bridges)) interfaces

          );

          conflicts = [ "shutdown.target" ];

          wantedBy = [ "multi-user.target" ] ++ optional hasDefaultGatewaySet "network-online.target";



          unitConfig.ConditionCapability = "CAP_NET_ADMIN";



          path = [ pkgs.iproute2 ];



          serviceConfig = {

            Type = "oneshot";

            RemainAfterExit = true;

          };



          unitConfig.DefaultDependencies = false;



          script = ''

            ${optionalString config.networking.resolvconf.enable ''

              # Set the static DNS configuration, if given.

              ${pkgs.openresolv}/sbin/resolvconf -m 1 -a static <<EOF

              ${optionalString (cfg.nameservers != [ ] && cfg.domain != null) ''

                domain ${cfg.domain}

              ''}

              ${optionalString (cfg.search != [ ]) ("search " + concatStringsSep " " cfg.search)}

              ${flip concatMapStrings cfg.nameservers (ns: ''

                nameserver ${ns}

              '')}

              EOF

            ''}



            # Set the default gateway

            ${flip concatMapStrings

              [

                {

                  version = "-4";

                  gateway = cfg.defaultGateway;

                }

                {

                  version = "-6";

                  gateway = cfg.defaultGateway6;

                }

              ]

              (

                { version, gateway }:

                optionalString (gateway != null && gateway.address != "") ''

                  ${optionalString (gateway.interface != null) ''

                  ip -${version} route replace ${gateway.address} proto static ${

                    ip ${version} route replace ${gateway.address} proto static ${

                      formatIpArgs {

                        metric = gateway.metric;

                        dev = gateway.interface;

                      }

                    }

                  ''}

                ip -${version} route replace default proto static ${

                  ip ${version} route replace default proto static ${

                    formatIpArgs {

                      metric = gateway.metric;

                      via = gateway.address;
@@ -230,24 +203,36 @@ let 
                      src = gateway.source;

                    }

                  }

                echo "done"

                ''

              )

            }

          '';

        };



        # For each interface <foo>, create a job ‘network-addresses-<foo>.service"

        # that performs static address configuration.  It has a "wants"

        # dependency on ‘<foo>.service’, which is supposed to create

        # the interface and need not exist (i.e. for hardware

        # interfaces).  It has a binds-to dependency on the actual

        # network device, so it only gets started after the interface

        # has appeared, and it's stopped when the interface

        # disappears.

        configureAddrs =

          i:

          let

            ips = interfaceIps i;

          in

          nameValuePair "network-addresses-${i.name}" {

            description = "Address configuration of ${i.name}";



            wantedBy =

              deviceDependency i.name

              ++ optional config.boot.isContainer "network.target"

              ++ optional (isDefaultGateway4 || isDefaultGateway6) "network-online.target";

            wantedBy = [

              "network-setup.service"

              "network.target"

            ];

            # order before network-setup because the routes that are configured

            # there may need ip addresses configured

            before = [ "network-setup.service" ];

            bindsTo = deviceDependency i.name;

            partOf = [ "networking-scripted.target" ];

            after = [

              "network-pre.target"

            ]

            ++ optional needsSourceIface4 "network-addresses-${source4Iface.name}.service"

            ++ optional needsSourceIface6 "network-addresses-${source6Iface.name}.service"

            ++ deviceDependency i.name;

            after = [ "network-pre.target" ] ++ (deviceDependency i.name);

            serviceConfig.Type = "oneshot";

            serviceConfig.RemainAfterExit = true;

            # Restart rather than stop+start this unit to prevent the
@@ -299,10 +284,6 @@ let 
                  fi

                ''

              )}



              # Set the default gateway

              ${optionalString isDefaultGateway4 (configureGateway "4" cfg.defaultGateway)}

              ${optionalString isDefaultGateway6 (configureGateway "6" cfg.defaultGateway6)}

            '';

            preStop = ''

              state="/run/nixos/network/routes/${i.name}"
@@ -330,13 +311,13 @@ let 
          nameValuePair "${i.name}-netdev" {

            description = "Virtual Network Interface ${i.name}";

            bindsTo = optional (!config.boot.isContainer) "dev-net-tun.device";

            partOf = [ "networking-scripted.target" ];

            after = optional (!config.boot.isContainer) "dev-net-tun.device" ++ [ "network-pre.target" ];

            wantedBy = [

              "network.target"

              "network-setup.service"

              (subsystemDevice i.name)

            ];

            before = [ "network.target" ];

            before = [ "network-setup.service" ];

            path = [ pkgs.iproute2 ];

            serviceConfig = {

              Type = "oneshot";
@@ -362,21 +343,18 @@ let 
              description = "Bridge Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps ++ optional v.rstp "mstpd.service";

              partOf = [

                "network.target"

                "networking-scripted.target"

              ]

              ++ optional v.rstp "mstpd.service";

              partOf = [ "network-setup.service" ] ++ optional v.rstp "mstpd.service";

              after = [

                "network-pre.target"

              ]

              ++ deps

              ++ optional v.rstp "mstpd.service"

              ++ map (i: "network-addresses-${i}.service") v.interfaces;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [ pkgs.iproute2 ];
@@ -470,14 +448,15 @@ let 
              description = "Open vSwitch Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ]

              ++ internalConfigs;

              before = [ "network.target" ] ++ internalConfigs;

              partOf = [

                "network.target"

                "networking-scripted.target"

              ]; # shutdown the bridge when network is shutdown

              # before = [ "network-setup.service" ];

              # should work without internalConfigs dependencies because address/link configuration depends

              # on the device, which is created by ovs-vswitchd with type=internal, but it does not...

              before = [ "network-setup.service" ] ++ internalConfigs;

              partOf = [ "network-setup.service" ]; # shutdown the bridge when network is shutdown

              bindsTo = [ "ovs-vswitchd.service" ]; # requires ovs-vswitchd to be alive at all times

              after = [

                "network-pre.target"
@@ -542,12 +521,12 @@ let 
              description = "Bond Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps;

              partOf = [ "networking-scripted.target" ];

              after = [ "network-pre.target" ] ++ deps ++ map (i: "network-addresses-${i}.service") v.interfaces;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [
@@ -591,12 +570,12 @@ let 
              description = "MACVLAN Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps;

              partOf = [ "networking-scripted.target" ];

              after = [ "network-pre.target" ] ++ deps;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [ pkgs.iproute2 ];
@@ -623,12 +602,12 @@ let 
              description = "IPVLAN Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps;

              partOf = [ "networking-scripted.target" ];

              after = [ "network-pre.target" ] ++ deps;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [ pkgs.iproute2 ];
@@ -668,12 +647,12 @@ let 
              description = "FOU endpoint ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps;

              partOf = [ "networking-scripted.target" ];

              after = [ "network-pre.target" ] ++ deps;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [ pkgs.iproute2 ];
@@ -698,11 +677,12 @@ let 
              description = "IPv6 in IPv4 Tunnel Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps;

              after = [ "network-pre.target" ] ++ deps;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [ pkgs.iproute2 ];
@@ -740,12 +720,12 @@ let 
              description = "IP in IP Tunnel Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps;

              partOf = [ "networking-scripted.target" ];

              after = [ "network-pre.target" ] ++ deps;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [ pkgs.iproute2 ];
@@ -788,12 +768,12 @@ let 
              description = "GRE Tunnel Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps;

              partOf = [ "networking-scripted.target" ];

              after = [ "network-pre.target" ] ++ deps;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [ pkgs.iproute2 ];
@@ -823,15 +803,13 @@ let 
              description = "VLAN Interface ${n}";

              wantedBy = [

                "network.target"

                "network-setup.service"

                (subsystemDevice n)

              ];

              bindsTo = deps;

              partOf = [

                "network.target"

                "networking-scripted.target"

              ];

              partOf = [ "network-setup.service" ];

              after = [ "network-pre.target" ] ++ deps;

              before = [ "network.target" ];

              before = [ "network-setup.service" ];

              serviceConfig.Type = "oneshot";

              serviceConfig.RemainAfterExit = true;

              path = [ pkgs.iproute2 ];
@@ -867,25 +845,8 @@ let 
      // mapAttrs' createGreDevice cfg.greTunnels

      // mapAttrs' createVlanDevice cfg.vlans

      // {

        network-local-commands = {

          after = [ "network-pre.target" ];

          wantedBy = [ "network.target" ];

        };

      };



    # Note: the scripted networking backend consistent of many

    # independent services that are linked to the network.target.

    # Since there is no daemon (e.g systemd-networkd) that is

    # started as part of the system and pulls in network.target.

    # Thus, to start these services we link network.target directly

    # to multi-user.target, this has the same result.

    systemd.targets.network.wantedBy = [ "multi-user.target" ];



    # This target serves no purpose during the boot, but can be

    # used to quickly reset the network configuration by running

    # systemctl restart networking-scripted.target

    systemd.targets.networking-scripted = {

      description = "NixOS scripted networking setup";

        network-setup = networkSetup;

        network-local-commands = networkLocalCommands;

      };



    services.udev.extraRules = ''

nixos/modules/tasks/network-interfaces.nix

+4 −17
Original line number Diff line number Diff line
@@ -747,9 +747,10 @@ in 
      default = "";

      example = "text=anything; echo You can put $text here.";

      description = ''

        Shell commands to be executed after all the network

        interfaces have been created, but not necessarily

        fully configured.

        Shell commands to be executed at the end of the

        `network-setup` systemd service.  Note that if

        you are using DHCP to obtain the network configuration,

        interfaces may not be fully configured yet.

      '';

    };
@@ -1850,20 +1851,6 @@ in 
        '';

      };

    };



    networking.localCommands = lib.mkIf config.networking.resolvconf.enable ''

      # Set the static DNS configuration, if given.

      ${pkgs.openresolv}/sbin/resolvconf -m 1 -a static <<EOF

      ${optionalString (cfg.nameservers != [ ] && cfg.domain != null) ''

        domain ${cfg.domain}

      ''}

      ${optionalString (cfg.search != [ ]) ("search " + concatStringsSep " " cfg.search)}

      ${flip concatMapStrings cfg.nameservers (ns: ''

        nameserver ${ns}

      '')}

      EOF

    '';



    services.mstpd = mkIf needsMstpd { enable = true; };



    virtualisation.vswitch = mkIf (cfg.vswitches != { }) { enable = true; };